Leaky Bucket vs Token Bucket Rate Limiting
Compare leaky bucket and token bucket rate limiting algorithms. Understand when to use each approach and how they handle burst traffic differently.
Detailed Explanation
Leaky Bucket vs Token Bucket
Both algorithms control the rate of requests, but they handle burst traffic very differently.
Leaky Bucket
The leaky bucket processes requests at a fixed, constant rate regardless of how many arrive. Incoming requests are placed in a queue (the bucket). If the queue is full, new requests are dropped. Requests "leak" out of the bucket at a steady rate.
Incoming [Queue/Bucket] Outgoing
||| --> [req][req][req] --> | (constant rate)
[req][req]
Properties:
- Output rate is perfectly smooth
- No bursts in the output
- Adds latency (requests wait in queue)
- Used by: Shopify (40 request bucket, 2 req/s leak rate)
Token Bucket
The token bucket allows immediate forwarding of requests as long as tokens are available. Tokens accumulate over time. Requests consume tokens instantly.
Tokens: [*][*][*][*][*]
Request arrives --> consume 1 token --> forward immediately
No tokens? --> reject (429)
Properties:
- Allows bursts up to bucket size
- No added latency for allowed requests
- Rejected requests get immediate 429
- Used by: AWS API Gateway, most cloud APIs
When to Use Each
| Scenario | Recommended | Why |
|---|---|---|
| API gateway | Token Bucket | Users expect low latency |
| Background job queue | Leaky Bucket | Smooth processing preferred |
| Payment processing | Leaky Bucket | Consistent downstream load |
| Interactive UI calls | Token Bucket | Burst-friendly for UX |
| Webhook delivery | Leaky Bucket | Predictable delivery rate |
Use Case
You are choosing a rate limiting strategy for your API. Your API has two types of consumers: interactive web clients that need low-latency burst access, and batch processing clients that should be smoothly throttled. You need to understand which algorithm to apply to each consumer type.