After this lesson, you will be able to: Explain what a load balancer does, compare round-robin / least-connections / sticky sessions, and reason about health checks and graceful drains.
Every web app past one server needs a load balancer. This lesson covers how they distribute requests, monitor backends, and handle the edge cases that hurt in production.
Sits between clients and your app servers. Receives every request and forwards it to one of the backend instances. Common load balancers: AWS ALB, GCP Load Balancer, Cloudflare, Nginx, HAProxy, Caddy. Without one, you'd need DNS round-robin (slow, no health awareness) or a single point of failure. With one, scaling horizontally becomes a config change.
Round robin: requests cycle through backends in order. Simple; works when backends are equivalent and stateless. Least connections: route to the backend with fewest active connections. Better when request durations vary widely. Weighted: assign each backend a weight (used for canary deploys: 95% to old version, 5% to new). Sticky sessions: hash the client IP or a cookie, always route the same client to the same backend. Necessary for stateful services (WebSockets, in-memory sessions); avoid otherwise.
When you must route the same key to the same backend (sticky sessions, a sharded cache), the naive approach is backend = hash(key) % N. The problem: change N (add or remove one server) and almost every key remaps to a different backend, so a cache stampede or session loss hits everyone at once. Consistent hashing fixes this. Picture the hash space as a ring; place each backend at several points (virtual nodes) around it, and route each key to the next backend clockwise. Now adding or removing a backend only remaps the keys in that one arc, roughly 1/N of them, instead of nearly all of them. This is the algorithm behind distributed caches (Memcached clients), sharded data stores (DynamoDB, Cassandra), and CDNs. Reach for it whenever you hash keys across a changing set of nodes.
Load balancers operate at one of two network layers. Layer 4 (transport): routes by IP and port without looking at the request contents. Very fast, protocol-agnostic, but it cannot make decisions based on the URL, headers, or cookies (examples: AWS NLB, HAProxy in TCP mode). Layer 7 (application): terminates the connection and reads the actual HTTP request, so it can route by path (/api to one pool, /static to another), do TLS termination, header-based routing, and cookie-based sticky sessions (examples: AWS ALB, Nginx, Cloudflare, Envoy). Rule of thumb: use Layer 7 for HTTP apps that benefit from content-aware routing (most web apps), and Layer 4 when you need raw throughput or are balancing a non-HTTP protocol.
The LB pings each backend periodically (e.g. GET /health every 5s). If a backend fails N consecutive checks, the LB stops sending traffic to it. If it recovers M consecutive checks, traffic resumes. Implementation: a /health endpoint that returns 200 only if the app is genuinely healthy (DB reachable, can serve traffic). Don't just return 200 unconditionally; that defeats the check.
Drop this near your server setup. SIGTERM is what AWS / Kubernetes send on rolling restarts.
import { server } from "./app";async function gracefulShutdown(signal: string) {console.log(`Received ${signal}, draining...`);// Stop accepting new connectionsserver.close(() => {console.log("Server closed.");process.exit(0);});// Force exit after 30s (LB drain window)setTimeout(() => {console.error("Drain timeout exceeded; forcing exit.");process.exit(1);}, 30_000);}process.on("SIGTERM", () => gracefulShutdown("SIGTERM"));process.on("SIGINT", () => gracefulShutdown("SIGINT"));// Your /health endpoint should return 503 once shutdown starts, so the// LB stops routing new requests to this instance immediately.
/health endpoints that always return 200, even when the DB is down. The LB can't tell the difference between alive and useful. Sticky sessions where they aren't needed. Forces traffic imbalance and complicates scaling. Forgetting graceful drain. Every deploy hits real users with 5xx responses. Setting health check intervals too short (every 1s). The LB hammers your app with health checks during quiet periods. 5-10s is normal. Trusting the LB IP as the client IP. The LB shows up as the client; use X-Forwarded-For to find the real client IP (and trust it only inside your network).
Pick the highest-impact change.
Sign in and purchase access to unlock this lesson.