Production Health Checks & Graceful Shutdown - Complete Guide

Production Health Checks & Graceful Shutdown - Complete Guide

Leader 7
calendar_todayschedule5 min read

Production Health Checks & Graceful Shutdown

Most backend tutorials teach you how to start a server.

Very few teach you how a server should stop.

That might sound strange at first.

If your application starts successfully, serves requests, and all your tests pass, isn't that enough?

Not in production.

Production has a completely different set of problems.

Your application isn't running on your laptop anymore. It's behind load balancers, inside containers, managed by Kubernetes or another orchestrator. Servers are restarted, deployments happen every day, machines fail, and containers are replaced all the time.

Eventually, your application will receive a SIGTERM.

The question isn't if it happens.

The question is what your application does next.

SIGTERM
   │
   ▼
❓ Exit immediately?
❓ Finish active requests?
❓ Accept new traffic?
❓ Close database connections?
❓ Flush logs?

These aren't business logic problems.

They're production problems.

Two concepts solve most of them:

Concept Responsibility
❤️ Health Checks Tell infrastructure whether the application should receive traffic
🛑 Graceful Shutdown Stop the application safely without interrupting users

Although they solve different problems, they're designed to work together.


Why should you care?

Imagine you're deploying a new version of your API.

Kubernetes starts a new pod and sends a SIGTERM to the old one.

At the exact same moment, a customer submits an order.

If your application exits immediately:

❌ The request never finishes.

❌ The client receives an error.

❌ Maybe payment succeeded.

❌ Maybe the database transaction didn't.

❌ Maybe logs never got flushed.

Nothing was wrong with your business logic.

Your application simply wasn't prepared to stop.

Production reliability isn't just about writing correct code.

It's also about handling your application's lifecycle correctly.


What exactly is a health check?

Before Kubernetes or a load balancer sends traffic to your application, it needs to answer one simple question:

"Can I trust this instance to handle another request?"

It can't inspect your code.

It doesn't know whether PostgreSQL is available.

It doesn't know whether Redis is responding.

It can't see whether your application is shutting down.

The only thing it can do is ask your application.

That's exactly what a Health Check is.

Load Balancer
      │
      ▼
 GET /health/ready
      │
      ▼
Application
      │
      ▼
"Yes, I'm ready." ✅

or

"Don't send me traffic." ❌

Health checks are simply your application describing its own state over HTTP.


Why isn't one endpoint enough?

One of the biggest misconceptions is exposing a single /health endpoint.

Different systems ask different questions.

Consumer Question
Kubernetes Is the process alive?
Load Balancer Should this instance receive traffic?
Monitoring What's wrong?
Developers Which dependency failed?

Trying to answer all of those questions with one endpoint usually causes unnecessary restarts or incorrect routing decisions.

That's why production applications expose multiple endpoints.


/health/live

Is the process alive?

That's the only responsibility of this endpoint.

Don't:

  • Connect to PostgreSQL
  • Ping Redis
  • Check message queues
  • Call external APIs

If Node.js is running:

HTTP/1.1 200 OK

That's enough.

Why?

Imagine Redis has a temporary outage.

If your liveness endpoint depends on Redis, Kubernetes may restart perfectly healthy pods because of a dependency that would've recovered on its own.

Liveness should answer only one question:

Is the process alive?


/health/ready

Should this instance receive new traffic?

This endpoint is much more important.

Return 503 whenever:

  • The application is shutting down
  • A critical dependency has failed
  • Startup hasn't completed yet
Readiness = 200
       │
       ▼
Load Balancer
continues sending traffic

──────────────

Readiness = 503
       │
       ▼
Remove instance
from service discovery

Notice something important.

The process doesn't stop.

It simply stops receiving new work.

That's exactly what you want during deployments.


/health

Unlike the previous endpoints, this one is for humans and monitoring systems.

Instead of simply saying healthy or unhealthy, it provides a complete report.

For example:

{
  "status": "degraded",
  "checks": {
    "database": "healthy",
    "redis": "unhealthy"
  },
  "uptime": 12543,
  "timestamp": "2026-08-07T12:00:00Z"
}

Perfect for:

  • Dashboards
  • Monitoring
  • Debugging
  • Incident response

Not every dependency should be treated the same

This is where many applications get it wrong.

Imagine Redis is only used for caching.

If Redis becomes unavailable:

  • ✅ Users can still log in
  • ✅ Orders still work
  • ✅ Database is healthy

The application is slower.

It isn't broken.

That's a Degraded state.

Now imagine PostgreSQL goes down.

Nothing works anymore.

That's Unhealthy.

Visualizing it makes the difference obvious.

Dependency Critical? Status
PostgreSQL ✅ Yes 🔴 Unhealthy
Redis Cache ❌ No 🟡 Degraded
External Analytics ❌ No 🟡 Degraded

Health checks shouldn't simply answer:

Working?

or

Broken?

They should accurately describe reality.


What is Graceful Shutdown?

Every server eventually stops.

  • Deployments
  • Scaling down
  • Node maintenance
  • Container replacement
  • Machine failure

Shutdown isn't optional.

The only question is how your application behaves when it happens.

Without graceful shutdown:

SIGTERM
   │
   ▼
Process exits ❌

With graceful shutdown:

SIGTERM
   │
   ▼
Stop new traffic
   │
   ▼
Finish active requests
   │
   ▼
Close resources
   │
   ▼
Exit

What should happen during shutdown?

A production shutdown generally looks like this.

SIGTERM
    │
    ▼
Application starts shutdown
    │
    ▼
Readiness → 503
    │
    ▼
Load balancer removes instance
    │
    ▼
Existing requests finish
    │
    ▼
Cleanup resources
    │
    ▼
Exit

Every step exists for a reason.

Skipping any of them makes deployments less reliable.


Why request draining matters

Imagine one request takes five seconds.

Kubernetes begins shutting down the pod after two seconds.

Without request draining:

Request Started
       │
       ▼
SIGTERM
       │
       ▼
Process exits ❌
       │
       ▼
Client receives error

With request draining:

Request Started
       │
       ▼
SIGTERM
       │
       ▼
Wait...
       │
       ▼
Request finishes ✅
       │
       ▼
Shutdown continues

Users never notice the deployment.

That's the goal.


Why shutdown order matters

Resources depend on each other.

Closing them in the wrong order creates failures.

A common production sequence looks like this:

1. Stop HTTP traffic
        │
        ▼
2. Stop queue workers
        │
        ▼
3. Finish active requests
        │
        ▼
4. Close database pools
        │
        ▼
5. Close Redis
        │
        ▼
6. Flush logs
        │
        ▼
7. Exit

Order matters.

Imagine closing your database before active requests finish.

Every request immediately starts failing.


Why Health Checks and Graceful Shutdown belong together

These concepts solve two halves of the same problem.

Health Checks without Graceful Shutdown

Healthy ✅
      │
Deployment
      │
Process exits
      │
Interrupted requests ❌

Graceful Shutdown without Readiness

Shutdown starts
       │
Readiness still 200
       │
Load Balancer
keeps sending traffic
       │
New requests fail ❌

One endpoint for everything

Using a single /health endpoint for:

  • Liveness
  • Readiness
  • Monitoring

usually causes:

  • Unnecessary restarts
  • Incorrect routing
  • Poor observability

Each endpoint exists for a different reason.

Keep them separate.


Why I built these Blockend blocks

After building enough backend projects, I realized I kept writing the same production infrastructure over and over again.

Every project needed:

  • Health endpoints
  • Graceful shutdown
  • Request draining
  • Cleanup logic
  • Signal handling

None of it was unique to the application.

It was production plumbing.

That's why I built these Blockend blocks.

Not to create another framework.

Not to hide how things work.

Simply to package production patterns I kept rewriting into reusable building blocks that can be copied into any Node.js project.


What does the complete production lifecycle look like?

                Deployment
                     │
                     ▼
                 SIGTERM
                     │
                     ▼
        Application enters shutdown
                     │
                     ▼
          Readiness returns 503
                     │
                     ▼
     Load Balancer stops new traffic
                     │
                     ▼
       Existing requests complete
                     │
                     ▼
       Cleanup tasks execute safely
                     │
                     ▼
          Process exits cleanly

No interrupted requests.

No leaked connections.

No partially completed cleanup.

Just a predictable, reliable deployment.


Final thoughts

Health checks aren't just monitoring endpoints.

Graceful shutdown isn't just listening for SIGTERM.

Together, they define how your application behaves in production.

Most tutorials stop after teaching you how to start a server.

Production systems also need to know how to stop.

If you're building backend services, these patterns aren't optional.

They're part of building reliable software.


Continue Reading

If you'd like to implement everything covered in this guide—including framework adapters, configuration, and production-ready examples—you can follow the complete Blockend guide:

📖 Production Health Checks & Graceful Shutdown Guide

https://blockend.noorulhassan.com/docs/03-guides/01-production-health-checks-and-graceful-shutdown

🔥 Join developers growing publicly
Share your knowledge, build in public, and grow your developer presence with a global community.

More Posts

Merancang Backend Bisnis ISP: API Pelanggan, Paket Internet, Invoice, dan Tiket Support

Masbadar - Mar 13

TypeScript Complexity Has Finally Reached the Point of Total Absurdity

Karol Modelski - Apr 23

Sovereign Intelligence: The Complete 25,000 Word Blueprint (Download)

Pocket Portfolio - Apr 1

Beyond the 98.6°F Myth: Defining Personal Baselines in Health Management

Huifer - Feb 2

The Zero-Net-Loss Fleet & The Mercenary Squad: A Live AI Economy

DEVPlank - Aug 4
chevron_left
916 Points7 Badges
Pakistannoorulhassan.com
5Posts
0Comments
2Connections
I am a developer and product builder based in Jhang, Pakistan, focused on creating practical develop... Show more

Commenters (This Week)

4 comments
1 comment
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!