Page 2 of 2
In a static environment, you can hardcode an IP address and move on. In an environment where containers or instances get created and destroyed constantly - the normal state of things under an orchestrator or autoscaling group - that IP is meaningless within minutes. Service discovery solves the problem of "how does service A find service B's current location" when that location is never stable for long.
Common approaches include a dedicated service registry that services register with on startup and query to find each other (Consul is a well-known example), DNS-based discovery where a name resolves dynamically to whatever's currently healthy (this is exactly what Kubernetes Services do internally), and orchestrator-native discovery baked directly into the platform.
The property that actually matters in production is how quickly discovery reflects reality. If a service instance dies and discovery takes 30 seconds to notice, that's 30 seconds of requests routed into a void. This is why service discovery is almost always paired tightly with health checking - discovery without health awareness just tells you where something was, not where it's actually safe to send traffic right now.
When something's "slow" or "broken" in production, the very first fork is distinguishing latency from packet loss, because they point in completely different directions. High latency with no loss usually means something is legitimately taking time - a slow database query, a congested link, physical distance. Packet loss means data is actively being dropped and retransmitted, which shows up as inconsistent, jittery behavior rather than uniformly slow behavior, and points toward an overloaded link, a failing piece of network hardware, or overly aggressive rate limiting somewhere in the path.
The second fork, infamous enough to be a running joke in the industry, is asking "is it DNS?" - because DNS issues produce symptoms that look exactly like every other kind of failure (timeouts, connection refused, intermittent failures) while the actual cause is invisible unless you specifically check resolution. A stale cached record, an expired domain registration, or a misconfigured resolver can all present identically to "the server is down," and it's genuinely worth checking DNS resolution early and explicitly rather than assuming it's fine, precisely because it's cheap to check and expensive to overlook.
At scale, DNS stops being a static lookup table and becomes an active traffic-steering tool. Health-checked DNS routing ties DNS answers to the actual health of what they point to - if a region's endpoint stops responding to health checks, its records are automatically pulled out of the response pool, and clients simply stop being handed that address. This is a core mechanism behind multi-region failover: point a domain at multiple regional endpoints, monitor each, and let unhealthy ones drop out automatically rather than requiring a human to change a record during an incident.
The caveat that catches people is TTL, again, but with higher stakes at scale: even a perfect health check reacts instantly, but any client or resolver that already cached the old answer keeps using it until that TTL expires. For anything routing production failover traffic, this generally means running DNS TTLs deliberately low (seconds to low minutes) specifically to keep that propagation window tight - accepting more DNS query volume as the tradeoff for faster failover.
An API gateway is a specialized reverse proxy purpose-built for API traffic, and it's worth distinguishing from a generic reverse proxy because of what it typically adds on top: authentication and authorization enforcement at the edge (so individual services don't each reimplement it), request/response transformation, consistent rate limiting and quota enforcement per API consumer, and often aggregating multiple backend services behind one consistent external API surface.
The tradeoff that matters operationally is centralization itself: putting all this logic in one place means consistent behavior across every service and one place to update policy - but it also means the gateway becomes a single point that every request passes through, so its availability and latency become a hard ceiling on everything behind it. A slow or overloaded gateway doesn't just affect one service, it affects all of them simultaneously, which is a very different failure profile from one backend having a bad day.
At meaningful scale, small per-request inefficiencies compound into real capacity limits. Connection pooling avoids the cost of the TCP handshake (and TLS handshake, which is even more expensive) on every single request by reusing already-established connections - the difference between opening a new phone call for every sentence versus keeping one call open for a whole conversation. Keep-alive is the HTTP-level mechanism that makes this possible, telling both sides to leave the underlying connection open for reuse rather than tearing it down after each response.
MTU (Maximum Transmission Unit) defines the largest packet size a network link will carry without fragmenting it into smaller pieces. Fragmentation isn't free - it adds processing overhead and, worse, if a fragment is lost, the entire original packet has to be resent, not just the missing piece. Mismatched MTU across a network path (common in tunneled or overlay networks, like VPNs or container networking) is a classic, hard-to-spot cause of mysteriously degraded throughput that looks nothing like an obvious connectivity failure - connections establish fine, small requests work fine, and only larger payloads mysteriously stall or crawl.
Real scenario-based DevOps questions, hands-on practice, and clear explanations for every answer.