Nginx - Hard

Buffering and Its Effect on Latency

By default, Nginx buffers responses from the backend before sending them on to the client.

location /api/ {
    proxy_pass http://backend_app;
    proxy_buffering on;    # default: buffer the whole response before forwarding
}

location /stream/ {
    proxy_pass http://backend_app;
    proxy_buffering off;   # forward data to the client as soon as it arrives
}

With buffering on, Nginx reads the backend's response as fast as the backend can produce it, holds it in memory (spilling to disk for very large responses), and only then streams it out to the client at whatever pace the client's connection can handle. This is genuinely useful for the common case: it frees a slow backend from being held hostage by a slow client's connection speed, since Nginx absorbs the full response quickly and the backend process can move on to the next request. But for anything meant to be delivered progressively, Server-Sent Events, WebSocket upgrades, long-polling, or any kind of live streaming response, buffering is actively harmful: the client won't see any data until Nginx has buffered enough (or all) of the response, which defeats the entire point of a real-time stream. That's exactly when you turn proxy_buffering off for that specific location, trading away the backend-protection benefit in exchange for data reaching the client as soon as it's produced.

---

A Production Incident: 502s and 504s Under Load

A deploy goes out, and within minutes your dashboards show a wave of errors. Knowing the difference between the two most common Nginx-reported failures tells you where to look first. A 502 Bad Gateway means Nginx got a response from the network that it couldn't use, most often the backend actively refused or reset the connection, which usually points to the backend process being down, crashed, or out of available workers to accept new connections. A 504 Gateway Timeout means Nginx successfully connected to the backend but never got a response within proxy_read_timeout, which points at a backend that's alive but too slow, often because it's saturated: all its own worker threads or connection pool slots are busy, and new requests are queued behind them.

The diagnostic path from there: grep the error log for the specific upstream address named in the failures, check whether the backend process is actually running and how many instances are healthy, and check the backend's own resource usage (CPU, memory, its database connection pool) for saturation. If the timing lines up with a deploy, it's worth checking whether the new backend version is slower per-request than the old one, since a small per-request slowdown can be enough to push a saturated system over the edge into cascading timeouts, even without an outright crash.

---

Zero-Downtime Reloads

nginx -s reload feels like magic until you understand what it's actually doing at the process level, and understanding it explains both why it's safe and how it can quietly fail. When the master process receives the reload signal, it first checks the new configuration for validity (the same check nginx -t runs). If the config is invalid, the master logs an error and keeps the existing workers running entirely untouched, meaning a bad reload fails safe: your site keeps serving traffic on the old config rather than crashing. This is also the classic "silent reload that didn't apply" trap: if you don't check the error log or the exit status after a reload, you can walk away believing your change is live when it was actually rejected and the old config is still running.

If the new config is valid, the master process spawns a fresh set of worker processes running the new configuration, while the old workers are told to gracefully shut down. Those old workers stop accepting brand new connections immediately, but they finish whatever requests they already had in flight before exiting. The result is that no in-flight request gets dropped, and new requests land on new workers running the new config, all without ever closing the listening socket. This is the real mechanism behind "zero-downtime reload," and it's also why reload is preferred over restart for applying config changes: a restart tears down and rebinds the listening socket, creating a small window where nothing is listening at all.