Page 3 of 4
Revisiting StatefulSets with the failure modes that actually show up in production: replicas are created and terminated in strict order (db-0, then db-1, then db-2 - and terminated in reverse), and by default a StatefulSet won't move on to the next replica until the current one is Running and Ready - which means one stuck-and-never-ready replica blocks the entire rollout or scale-up behind it. This ordering guarantee is exactly what most clustered databases need for safe bootstrap (a follower shouldn't start before a leader exists), but it's also why a StatefulSet rollout can appear to "hang" far more often than a Deployment's - it's honoring an ordering constraint you asked for, not malfunctioning.
Each replica's PVC is created from the StatefulSet's volumeClaimTemplates and is not deleted automatically when the StatefulSet is scaled down or deleted (by default) - the data outlives the Pod on purpose, which is also why "just delete and recreate the StatefulSet" is not a safe generic troubleshooting step the way it can be for a Deployment.
Pod Security Admission enforces the privileged/baseline/restricted standards at the API server level via a namespace label - no separate policy engine required for the built-in profiles, though OPA Gatekeeper or Kyverno are the standard choice once you need rules the built-in profiles don't express (e.g. "images must come from this specific registry").
Seccomp and AppArmor/SELinux operate one level below Linux capabilities: seccomp filters which syscalls a container's process is allowed to make at all (dramatically shrinking what a container-escape exploit chain has available to it), while AppArmor/SELinux enforce mandatory access control profiles restricting what files/resources a process can touch even if it has the Linux permissions to. Kubernetes lets you attach a seccomp profile per-Pod or per-container; a genuinely hardened cluster runs a restrictive default rather than leaving every container's syscall surface fully open.
ServiceAccount tokens: modern Kubernetes issues time-bound, audience-scoped tokens (via the TokenRequest API) rather than the old-style long-lived static tokens that used to be auto-mounted into every Pod indefinitely - a real security improvement, since a leaked long-lived token used to mean permanent compromise until manually rotated, and a leaked bound token expires and is scoped to begin with.
Revisiting QoS with what actually happens under real node pressure: when a node runs low on memory, the kubelet's eviction manager kills Pods to reclaim resources, in a defined order - BestEffort Pods first, then Burstable Pods using more than their request, and Guaranteed Pods only as an absolute last resort. This is precisely why setting real, accurate resource requests matters beyond just scheduling correctness - it's also what determines eviction order when things get tight.
CPU throttling is a frequent, quietly-degrading production issue: a container hitting its CPU limit isn't killed, it's throttled (denied CPU time for the remainder of each scheduling period) - which shows up as elevated latency, not an obvious crash, and is easy to miss unless you're specifically watching container_cpu_cfs_throttled_seconds_total or equivalent.
ResourceQuota caps total resource consumption (CPU, memory, object counts) per namespace - the tool for preventing one team/namespace from starving the rest of a shared cluster. LimitRange sets default/min/max resource requests and limits for individual Pods/containers within a namespace - so a Pod created with no resources specified at all still gets sane, non-zero requests rather than landing as unconstrained BestEffort by accident.
kubectl describe node node-1 | grep -A5 "Allocated resources" # requests vs capacity, right now
kubectl top pod --sort-by=cpu
kubectl get resourcequota -n production
kubectl get limitrange -n production
HPA doesn't scale on every metric blip - it applies a stabilization window (default 5 minutes for scale-down, 0 for scale-up) specifically to avoid flapping: rapidly scaling up and back down in response to a brief traffic spike, which would otherwise thrash Pod creation/deletion and any connection draining involved.
Custom and external metrics extend HPA beyond CPU/memory: a custom metric comes from something inside the cluster (queue depth reported by your own app, via the custom metrics API); an external metric comes from outside it entirely (a cloud provider's queue depth, an external monitoring system) - both require a metrics adapter registered with the API server, HPA itself doesn't know how to reach either source directly.
VPA and HPA together is a genuinely tricky combination: if both are configured against the same resource metric (both watching CPU, say), they can fight each other - VPA resizing a Pod's requests changes the percentage HPA is computing against, potentially triggering unwanted scaling reactions. The safe pattern is scoping them to different signals, or not combining them on the same workload at all without a specific reason to.
Real scenario-based DevOps questions, hands-on practice, and clear explanations for every answer.