Calculating Token Bucket Refill Rates

Calculating Token Bucket Refill Rates

The refill rate determines your API's sustained throughput while the bucket size determines the burst capacity. Getting these values right is critical for both server-side implementation and client-side consumption.

Core Formulas

Refill rate = Desired sustained requests per second
Bucket size = Maximum burst size
Time to full = Bucket size / Refill rate
Recovery after burst of N = N / Refill rate

Design Example: 1,000 Requests/Minute API

Target: 1,000 requests/minute with a burst tolerance of 50 requests.

Refill rate = 1,000 / 60 = 16.67 tokens/second
Bucket size = 50 tokens
Time to full from empty = 50 / 16.67 = 3.0 seconds

Refill Rate Table for Common Limits

Desired Rate	Refill Rate	Bucket Size (10s burst)
60/min	1.0/s	10
600/min	10.0/s	100
1,000/min	16.67/s	167
5,000/hr	1.39/s	14
10,000/hr	2.78/s	28
100,000/day	1.16/s	12

Client-Side Adaptive Throttling

When consuming an API with token bucket limiting, you can implement adaptive throttling:

1. Read X-RateLimit-Remaining from each response
2. If remaining < 20% of limit, reduce request rate by 50%
3. If remaining < 5% of limit, pause until reset time
4. After reset, gradually ramp back up

This approach maximizes throughput while avoiding 429 errors.