In short
Downtime during a deploy comes from three things: requests dropped while a process restarts, schema changes incompatible with running code, and caches or assets that no longer match. Rolling and blue-green releases solve the first. The other two are solved by making every change backward-compatible with the version currently running, which is the discipline the deployment strategy depends on.
The three causes
Dropped requests. The old process stops before the new one is ready, or stops while still serving requests.
Incompatible schema. New code needs a column the database does not have, or old code breaks on a change already applied.
Mismatched assets. A browser holding old HTML requests a script that no longer exists, or a cache serves a stale combination.
Deployment strategies address the first. The other two are your responsibility regardless of strategy, and they are the ones that produce the confusing incidents.
Rolling and blue-green
Rolling. Replace instances gradually. Some serve old code, some new, until the fleet is updated. Cheap, and the default in most orchestration.
Its defining property: both versions run simultaneously, which is why compatibility matters. If the versions cannot coexist, rolling deploys produce intermittent errors that are miserable to debug.
Blue-green. Run a complete second environment, switch traffic at the load balancer, keep the old one for rollback. Cleaner separation and instant rollback, at the cost of running two environments and needing both to work against one database — which brings compatibility back anyway.
Canary. Route a small share of traffic to the new version, watch, then proceed. The best option when you can measure quickly, and it turns deployment into an experiment rather than an event.
For most teams, rolling with health checks is sufficient. Blue-green earns its cost when rollback speed genuinely matters.
Graceful shutdown is the detail people miss
A process being replaced must stop accepting new requests, finish the ones in flight, and then exit.
Without it, in-flight requests are severed at exactly the moment of highest traffic churn. Users see errors during every deploy, which teams often mistake for a load problem.
That means handling the termination signal, closing the listener, waiting for active requests with a sensible timeout, and only then exiting. The orchestrator's grace period must be longer than your longest normal request, or it kills you mid-work anyway.
Background workers need the same treatment: finish the current job, do not claim another, exit. A worker killed mid-job relies entirely on your at-least-once handling to avoid losing the work.
Health checks that mean something
The orchestrator decides when a new instance is ready from your health check, so a check that returns 200 immediately defeats the entire mechanism.
Readiness should confirm the instance can actually serve — dependencies reachable, caches warmed if needed. Traffic arrives only after it passes.
Liveness should confirm the process is not wedged. Keep it simple, since a liveness check with heavy dependencies causes restarts during unrelated outages.
The common failure is one endpoint serving both purposes and checking too much: a brief database blip then restarts every instance simultaneously, converting a small problem into an outage.
Backward compatibility is the real work
During any rolling deploy, two versions run at once. Every change must be safe in that overlap.
Schema changes ship separately and additively. Add the column, deploy code that writes to both, backfill, switch reads, remove the old column later — the expand-and-contract pattern from safe migrations.
API changes stay additive. Add fields; do not remove or repurpose them while old clients exist. A browser holding a page from before your deploy is an old client.
Job payloads stay readable by both versions. New workers must handle jobs enqueued by old code and vice versa.
Assets are fingerprinted and old versions retained for a period. A browser with the previous HTML will request the previous script, and it must still be there.
Feature flags separate deploy from release. Ship the code inert, enable it deliberately. This is the single most effective way to make deploys boring, because it removes the coupling between shipping and changing behaviour.
Rolling back
Rollback must be tested, not assumed.
Code rollback is easy. Data rollback is not. If a release wrote data in a new shape, reverting the code leaves that data behind. This is the argument for additive schema changes: the previous version keeps working because nothing it relied on was removed.
Prefer rolling forward for anything data-related — a new release that fixes the problem rather than a reversal that may lose writes made since.
Know your rollback time, and practise it. A rollback path first exercised during an incident is not a rollback path.
Making deploys boring
Deploy often. Small frequent releases are lower risk than large infrequent ones, because there is less to go wrong and less to search when something does.
Automate it. Manual steps get skipped under pressure.
Watch error rates and latency during and after, with an obvious signal if something moves.
Do not deploy on Friday afternoon — not because of the day, but because the people who understand the change should be available afterwards.
The goal is deployment as a non-event. If a release requires a maintenance window, a person watching, or a specific engineer, that is a process problem rather than a technology one.
If your deploys cause errors and you are not sure which of the three causes it is, book a call.
Common questions
What causes downtime during a deployment?
Three things: requests dropped because a process stopped before finishing them, schema changes incompatible with the code still running, and cached assets that no longer match the deployed version. Deployment strategies address only the first — the other two require backward-compatible changes.
What is the difference between rolling and blue-green deployment?
Rolling replaces instances gradually, so both versions run simultaneously and must be compatible. Blue-green runs a complete second environment and switches traffic at once, giving instant rollback at the cost of running two environments — though both still share one database, so compatibility still matters.
Why do I see errors during every deploy?
Usually missing graceful shutdown. The process being replaced must stop accepting new requests, finish those in flight, then exit — otherwise in-flight requests are severed at each release. Also check that the orchestrator's grace period exceeds your longest normal request.
How do I make schema changes safe during a rolling deploy?
Make them additive and ship them separately from the code that needs them: add the column, deploy code writing to both, backfill, switch reads, then remove the old column in a later release. During a rolling deploy both versions run at once, so every change must be safe in that overlap.
Should I roll back or roll forward after a bad deploy?
Roll forward for anything data-related. Code rollback is easy, but if the release wrote data in a new shape, reverting the code leaves that data behind and a reversal may lose writes made since. Additive schema changes are what keep the previous version working if you do need to revert.
