Rate Limiting Algorithms: Fixed Window, Token Bucket, and More

A rate limiter is the bouncer at the door of your API. Its job is to decide how many requests a given client may make in a given window, and to turn away the rest. Without one, a single misbehaving client — a buggy retry loop, a scraper, an attacker — can consume all your capacity and take the service down for everyone else. With one, abuse is contained to the abuser. The interesting part is that "how many requests per window" can be counted in several different ways, and each algorithm behaves differently at the edges.

This walks through the main rate-limiting algorithms, their trade-offs, and how you make one work across many servers. It builds on the practical NestJS throttling implementation and on Redis counters, which is where distributed rate limiting actually lives.

Why rate limit at all

Three distinct reasons, and they shape the limits you set:

• Protection. Stop any single client from exhausting shared resources, whether through malice or a bug. This is the baseline.
• Fairness. Ensure one heavy user does not degrade the experience for everyone else. Capacity is shared, and a limiter enforces the sharing.
• Cost control. When each request costs you money — a downstream paid API, expensive compute — a limiter caps your exposure to runaway usage.

The limits differ by purpose. An abuse-protection limit is generous and only catches extremes; a cost-control limit might be tight and deliberate. Knowing which you are enforcing keeps you from setting a number that is either useless or infuriating.

Fixed window

The simplest algorithm. Divide time into fixed windows — say, one minute — and count requests per client per window. Allow up to the limit; reject the rest until the window resets. Client made 100 requests this minute and the limit is 100? Everything else waits until the clock ticks over to the next minute, when the counter resets to zero.

It is trivial to implement — a counter with an expiry — and easy to reason about. Its flaw is the boundary. Because the counter resets abruptly, a client can make a full window's worth of requests at the very end of one window and another full window's worth at the very start of the next, sending double the limit in a short burst straddling the boundary. For many uses this is acceptable; for tight limits it is a real hole.

Sliding window

The sliding window fixes the boundary problem by making the window move with the current time instead of snapping to fixed intervals. Rather than "requests in this calendar minute," it counts "requests in the last 60 seconds," continuously.

There are two common ways to do this. A sliding window log stores a timestamp for every request and, on each new request, counts how many fall within the trailing window. It is exact but memory-hungry, since you store every timestamp. A sliding window counter approximates by blending the current fixed window's count with a weighted portion of the previous window's, based on how far into the current window you are. It is far cheaper and close enough for almost everyone, which is why it is the common production choice. Either way, you eliminate the burst-at-the-boundary exploit, at the cost of more work than a plain counter.

Token bucket

The token bucket is the most flexible algorithm and the one worth understanding deeply, because it models something the window approaches do not: bursts that are actually fine.

Picture a bucket that holds tokens. Tokens are added at a steady rate — say, ten per second — up to a maximum capacity. Every request must take one token to proceed. If tokens are available, the request goes through and a token is removed. If the bucket is empty, the request is rejected. That is the whole model.

The elegance is in what it allows. When a client has been idle, the bucket fills to capacity, so they can make a burst of requests immediately — up to the bucket size — before being throttled down to the steady refill rate. This matches how real clients often behave: quiet for a while, then a flurry of activity. The bucket size controls how big a burst you tolerate, and the refill rate controls the sustained throughput. Two knobs, and between them they express "you can burst this much, but your long-run average must stay under this." That is usually exactly the policy you want, which is why token bucket underpins so many production rate limiters.

Leaky bucket

The leaky bucket is the token bucket's mirror image, and it optimizes for a different goal: smoothing output. Picture a bucket with a hole in the bottom that leaks at a constant rate. Requests pour in and queue in the bucket; they drain out — get processed — at a fixed rate regardless of how fast they arrive. If the bucket overflows, requests are dropped.

The key difference from token bucket is what it does to bursts. Token bucket allows bursts through; leaky bucket smooths them away, releasing requests downstream at a steady, even pace. This is what you want when the thing behind the limiter cannot tolerate spikes — a downstream service that needs a constant, predictable load rather than a jagged one. Where token bucket protects the limiter from the client, leaky bucket protects the downstream from the burst.

Making it work across many servers

Every algorithm above is easy on a single server, where the counter lives in local memory. The moment you run multiple application servers behind a load balancer, that breaks. If each server keeps its own count, a client hitting three servers gets three times the intended limit, because no server sees the whole picture. The limit has to be enforced against shared state.

The standard solution is Redis. All servers read and update the same counters in a single shared Redis instance, so the limit is global rather than per-server. The subtlety is atomicity: checking the count and incrementing it must happen as one indivisible operation, or two concurrent requests can both read "99 of 100," both decide they are allowed, and both increment to blow past the limit. This is a classic race condition. You close it by doing the check-and-increment atomically — a Lua script that Redis runs as a single unit, or Redis's atomic increment primitives with a carefully ordered expiry. The Redis counters post covers the mechanics, but the principle is the one that matters: in a distributed limiter, the read-modify-write must be atomic, or the limit leaks.

Choosing an algorithm

The decision comes down to what you are protecting and how you feel about bursts.

• Fixed window when you want the simplest possible limiter and the boundary burst does not concern you.
• Sliding window when you need the limit enforced smoothly over time without the boundary exploit, and you can afford a little more bookkeeping.
• Token bucket when you want to allow reasonable bursts while capping the sustained rate — the most common and flexible choice for public APIs.
• Leaky bucket when the thing downstream needs a smooth, constant load and you would rather shape traffic into an even stream than let bursts through.

A rate limiter is a small component with outsized importance, because it is one of the few defenses that protects the whole system from a single bad actor. It is also a recurring character in system design discussions precisely because it sits at the intersection of algorithms, distributed state, and the race conditions that come with it — a small problem that quietly touches a lot of the fundamentals.