Nginx - Advanced

High-Availability Nginx

A single Nginx instance is a single point of failure, and there are two common ways production systems avoid that. The first is active-passive with keepalived: two Nginx boxes, a shared floating VIP (virtual IP address) that clients actually connect to, and keepalived running the VRRP protocol between the two boxes to detect failure and move the VIP to the standby within a couple of seconds if the active box goes down. This gives you HA without depending on any cloud provider's load balancing product, which matters in on-prem or bare-metal environments.

The second, more common in cloud environments, is putting Nginx instances behind a cloud load balancer (an AWS ALB/NLB, a GCP load balancer) that health-checks each Nginx instance and routes traffic only to healthy ones, with the cloud provider handling the failover mechanics entirely. This is simpler to operate since you're not managing VRRP and floating IPs yourself, but it ties you to that provider's load balancing product and its own failure modes. Which one is right depends heavily on whether you're already committed to a cloud provider's networking stack or need something portable across environments.

Either approach only gets you HA for the Nginx layer itself; it says nothing about the config running on those instances staying in sync. A keepalived pair or a fleet behind a cloud load balancer is only as reliable as the process that ships config changes to every instance identically. Configuration drift between nodes, one instance reloaded with a fix, another one missed, is a quiet, common cause of "it works when I hit it directly but not through the load balancer" bugs, and it's exactly why most mature setups treat Nginx config as something deployed through the same automated pipeline as application code, never hand-edited on individual boxes.

---

Platform Judgment: Centralize or Delegate

A recurring decision for a platform team is whether Nginx configuration should be centralized and owned by the platform, or delegated to individual application teams to manage themselves (via their own Ingress resources, or their own config in a shared repo). Centralizing makes sense when consistency actually matters for correctness or safety: TLS policy, security headers, rate limiting standards, and anything where one team's misconfiguration could create a security gap or an outage that affects everyone sharing the same Nginx layer. Delegating makes sense when velocity matters more than uniformity: teams that need to add or change routes constantly shouldn't have to file a ticket and wait on a platform team for every change, and a well-designed Ingress-based setup can let teams manage their own routing rules within guardrails the platform still enforces underneath (default TLS, default rate limits, a base security header set that individual Ingress objects can't override). The failure mode to watch for in either direction: over-centralizing turns the platform team into a bottleneck for routine changes, while over-delegating means the blast radius of one team's mistake extends to everyone else sharing the same Nginx layer. Good platform design usually looks like clear, enforced defaults with narrow, safe extension points, not an all-or-nothing choice between the two.

---

Observing Nginx Itself

Nginx needs to be monitored like any other production service, and the open-source build gives you a starting point through the stub_status module.

location /nginx_status {
    stub_status;
    allow 127.0.0.1;   # restrict access, this shouldn't be public
    deny all;
}

Hitting that endpoint returns basic counters: active connections, total accepted and handled connections, and requests currently reading, writing, or waiting. It's minimal, no per-route breakdown, no latency data, but it's enough to feed into something like the nginx-prometheus-exporter, which scrapes stub_status and exposes it in Prometheus format for dashboards and alerting. The metric worth alerting on most is active connections trending toward your worker_connections ceiling, since that's the clearest early warning that you're approaching the concurrency limits discussed in the file-descriptor tuning above, well before it turns into dropped connections. For anything stub_status doesn't cover, request latency percentiles and 5xx rates broken down by route, you generally need structured (JSON) access logs shipped to a log pipeline where they can be aggregated, since Nginx itself doesn't compute latency percentiles natively. A production Nginx layer without either of these in place is running blind: it'll keep working right up until the moment it doesn't, with no warning in between.