█
LastWrite
  • > Curriculum
  • > Pricing
  • > For Educators
  • > About
  • > Contact
Log InGet Started

Questions, concerns, bug reports, or suggestions? We read every message, write to us at [email protected].

More ways to reach us →
LastWrite

Structured computer science lessons for aspiring developers and security professionals.

[email protected]

(201) 785-7951

Mon–Fri, 9 AM–5 PM EST

Learn

  • Curriculum
  • Pricing

Company

  • About
  • For Educators & Schools
  • Contact Us

Legal

  • Terms of Service
  • Privacy Policy
© 2026 LastWrite. All rights reserved.
Curriculum/Software Engineering/System Design/Rate Limiting
35 minIntermediate

Rate Limiting

After this lesson, you will be able to: Apply token bucket and leaky bucket rate-limiting algorithms, decide where in the stack to enforce limits, and integrate with Cloudflare / API gateway / app-level limits.

Rate limiting protects systems from abuse, accidents, and DDOS. It's required in every API past hobby scale. This lesson covers the algorithms and the decision of where to enforce them.

Prerequisites:Designing a Real System

Why rate limit

Protect against abuse: a hostile client floods your API trying to brute-force login. Protect against accidents: a buggy script in someone's CI calls your endpoint 1M times in 10 min. Protect against downstream cost: every request you serve costs cents in compute / DB / third-party APIs. Rate limits cap the bill. Enforce business limits: free tier = 100 requests/day; paid tier = unlimited.

Token bucket (the default)

A bucket holds N tokens, refilling at rate R tokens/sec. Each request consumes 1 token. If the bucket is empty, refuse. Bursty traffic up to N is allowed; sustained rate is capped at R. Example: 10-token bucket refilling at 1 token/sec → users can burst 10 requests instantly, then sustained 1/sec. Used by AWS, Cloudflare, Stripe API, and most rate-limited APIs.

Leaky bucket (the smoother)

Imagine a bucket with a hole. Requests fill the bucket; the bucket drains at a fixed rate. If full, new requests overflow (rejected). Difference from token bucket: leaky bucket smooths the OUTPUT to fixed rate; token bucket smooths the INPUT to fixed average rate but allows bursts. Used when downstream service can't handle bursts (e.g. legacy systems, expensive APIs).

Token bucket in TypeScript + Redis (production pattern)

Sliding window with Redis sorted set; fits behind any HTTP handler.

python
import { redis } from "./redis";
async function rateLimit(
key: string, // e.g. `rl:ip:1.2.3.4` or `rl:user:42`
maxRequests: number, // e.g. 100
windowSeconds: number, // e.g. 60
): Promise<{ allowed: boolean; remaining: number }> {
const now = Date.now();
const windowStart = now - windowSeconds * 1000;
const pipeline = redis.pipeline();
pipeline.zremrangebyscore(key, 0, windowStart); // drop old entries
pipeline.zadd(key, now, `${now}-${Math.random()}`);
pipeline.zcard(key);
pipeline.expire(key, windowSeconds);
const results = await pipeline.exec();
const count = results?.[2]?.[1] as number;
return {
allowed: count <= maxRequests,
remaining: Math.max(0, maxRequests - count),
};
}
// In your API handler:
export async function POST(req: Request) {
const ip = req.headers.get("x-forwarded-for") ?? "unknown";
const { allowed, remaining } = await rateLimit(`rl:ip:${ip}`, 100, 60);
if (!allowed) {
return new Response("Too Many Requests", {
status: 429,
headers: { "Retry-After": "60", "X-RateLimit-Remaining": String(remaining) },
});
}
// ... continue
}

Where to enforce rate limits

Edge (Cloudflare WAF, Vercel rate limiting): cheapest, blocks before reaching your app. Use for IP-level limits and DDoS. API gateway (AWS API Gateway, Kong, Nginx): can do per-API-key limits, per-route limits. Use for SaaS API tier enforcement. Application-level (Redis-backed, like the snippet above): the most flexible. Use for per-user, per-action, business-rule limits. Most production apps use ALL THREE in layers: edge for floods, gateway for tier enforcement, app for business logic.

Common mistakes only experienced engineers avoid

Counting in app memory instead of a shared store. Each replica has its own counter; the actual limit is N * replicas. Limiting per IP behind a CDN without trusting X-Forwarded-For. You end up rate-limiting the CDN's IP, blocking everyone. Returning 200 instead of 429 when rate-limiting. Clients don't know to back off. Always use 429 with Retry-After. Setting limits per request count without considering request cost. 100 GET /health vs 100 POST /generate-report are very different loads. Weight expensive requests more. No surfacing of limits to users. A SaaS that quietly rate-limits without telling the user looks broken. Document and surface the limit headers.

Quick Check

A hostile client uses 100 IPs to attack your login endpoint. What's the most effective rate limit?

Pick the strongest defense.

Sign in and purchase access to unlock this lesson.

Sign in to purchase
←Designing a Real System (URL Shortener, Notifications, Twitter Feed)
Back to System Design
Microservices vs Monolith→