Page 1 of 3
With more runnable processes than CPU cores (nearly always true on a busy server), the kernel has to decide who runs when. That's the job of the scheduler. Linux's default scheduler for normal processes is CFS, the Completely Fair Scheduler, and its actual goal is close to what the name implies: give every runnable process a fair share of CPU time, weighted by priority, rather than running things in a fixed queue or granting large blocks of exclusive time to whoever asked first.
ps -eo pid,ni,pri,cmd
# PID NI PRI CMD
# 1099 0 19 stress --cpu 1
# 1150 10 9 batch-import.sh
# 1201 -5 24 latency-critical-svc
CFS doesn't divide time into fixed round-robin slices; conceptually, it tracks how much CPU time each runnable process has already received and always picks the process that's received the least, adjusted by its nice value (a lower, more negative nice value means a bigger effective share). The kernel switches between processes far more often than you'd guess, often thousands of times per second, and every one of those switches is a context switch: the kernel saves one process's CPU register state, loads another's, and resumes it. This isn't free. A context switch spends real cycles on bookkeeping instead of running your application's actual code, and it tends to evict useful data from the CPU's cache, which is why a server thrashing between far more runnable processes than it has cores tends to get slower in ways that don't show up as "100% CPU used by any one thing," just generally sluggish everything.
The basic tier introduced virtual memory and the gap between VmSize and VmRSS. In production, the number that actually matters day to day is RSS (resident set size): how much physical RAM a process is genuinely using right now, as opposed to how much virtual address space it has mapped.
free -h
# total used free shared buff/cache available
# Mem: 15Gi 2.1Gi 1.8Gi 150Mi 11Gi 12Gi
# Swap: 2.0Gi 0B 2.0Gi
That output confuses almost everyone the first time they see it, because "free" shows only 1.8Gi free while "used" is only 2.1Gi, and a whopping 11Gi is sitting in buff/cache. That's the page cache: RAM the kernel is using to cache recently read or written disk data, purely as an optimization, because RAM is orders of magnitude faster than disk and re-reading a file from RAM instead of disk is nearly free. Crucially, the kernel hands that cached memory back instantly the moment an application actually needs it; it isn't reserved or locked up. The available column is the number that actually answers "how much memory could a new process get right now," and it's usually much closer to free + buff/cache than to free alone. Panicking because free looks low while available is healthy is one of the most common false alarms in ops work.
Swap is disk space the kernel uses as an overflow when physical RAM is under real pressure, moving pages of memory that haven't been touched recently out to disk to free up RAM for something more active. It sounds like a safety net, and on a desktop it mostly is one. On a latency-sensitive production service, it's usually closer to a trap: disk, even fast SSD, is vastly slower than RAM, so the moment a process touches a page of its own memory that's been swapped out, an access that would normally take nanoseconds now takes milliseconds while the kernel pulls it back in from disk. A service that starts swapping doesn't crash, it just gets mysteriously, intermittently slow, often on the exact requests that touch memory the process hadn't used in a while, which makes it maddening to reproduce.
vmstat 1 5
# procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
# r b swpd free buff cache si so bi bo in cs us sy id wa st
# 2 1 81920 94212 12040 512300 12 8 400 220 1800 3200 20 5 60 15 0
Nonzero si/so (swap in / swap out) columns are the smoking gun; if they're consistently above zero on a server that's "just slow," you've found your answer before you've even looked at CPU. The medium-tier fix is usually to reduce memory pressure (fewer processes competing for RAM, a smaller cache size, right-sizing the instance) or to lower vm.swappiness, a kernel tunable covered properly in the hard tier, which controls how eagerly the kernel reaches for swap in the first place.
Real scenario-based DevOps questions, hands-on practice, and clear explanations for every answer.