In short
Rate limiting caps how many requests a caller can make in a window, protecting your service from overload, runaway loops, and abuse. Token bucket is the sensible default because it permits short bursts while capping sustained rate. Key limits on the account rather than the IP, always return the standard headers so callers can adapt, and respond with 429 plus Retry-After rather than failing silently.
What you are actually protecting against
Three different threats, which is why one limit rarely serves.
Accidental overload. A customer's integration enters a retry loop, or a script runs without a delay. This is the most common by far, and it is not malicious.
Cost. Endpoints that call a paid third party — anything invoking an AI model, a mapping service, an SMS gateway — can turn a bug into a bill.
Abuse. Credential stuffing, scraping, enumeration. This needs tighter, different limits on specific endpoints, not a global cap.
The failure mode to avoid: setting one global limit that is loose enough for abuse and tight enough to break a legitimate heavy user.
Choosing an algorithm
Token bucket is the right default. A bucket holds tokens, refills at a fixed rate, and each request consumes one. It permits short bursts up to the bucket size while capping the sustained rate — which matches how real clients behave, since traffic is lumpy rather than smooth.
Fixed window counts requests per calendar minute. Simple, and has an obvious flaw: a client can send a full allowance at 11:59:59 and again at 12:00:00, doubling the intended rate across the boundary.
Sliding window fixes that by counting over a rolling period. More accurate, more expensive to compute, and usually unnecessary once token bucket is in place.
Concurrency limits cap simultaneous in-flight requests rather than rate. Worth adding for expensive, long-running endpoints, where ten concurrent heavy queries hurt more than a hundred quick ones.
Most APIs want token bucket for general traffic plus a concurrency cap on the expensive endpoints.
What to key the limit on
This decision matters more than the algorithm.
By account or API key — the right default. It matches how you bill, survives users moving between networks, and cannot be evaded by rotating IPs.
By IP — necessary for unauthenticated endpoints such as login and signup, since there is no account yet. Be careful: corporate networks and mobile carriers put many people behind one address, so an aggressive IP limit can block an entire office.
By endpoint — login, password reset, search, and anything expensive deserve their own tighter limits. A global limit that permits enough search traffic is far too generous for login attempts.
By operation cost — where requests vary wildly in expense, charge more tokens for heavy operations rather than pretending all requests are equal.
In practice you combine these: a generous account limit, a tighter IP limit on unauthenticated routes, and specific limits on a handful of sensitive endpoints.
Respond so callers can adapt
A rate limit that clients cannot see forces them to guess, and guessing means retry storms.
Return 429 Too Many Requests, with Retry-After telling the caller how long to wait. On every response — not only rejections — return the current limit, remaining allowance, and reset time. Well-built clients slow down before hitting the wall when you tell them where it is.
And make the error body say which limit was hit and what to do. "Rate limit exceeded" sends someone to your support inbox; "Account limit of 1,000 requests per minute exceeded, resets in 12s" does not.
Where the state lives
Counters must be shared across instances, or your effective limit multiplies by the number of servers.
A central store such as Redis is the usual answer, with the counter updated atomically — a read-modify-write from several instances undercounts badly under load.
If the store becomes unavailable, decide deliberately whether to fail open (allow traffic, protecting availability) or fail closed (reject, protecting the backend). For general API traffic, failing open is usually right; for login attempts, failing closed is. The wrong outcome is not having decided, and discovering the default during an incident.
Avoiding the self-inflicted outage
Watch before you enforce. Run the limits in log-only mode first and look at who would have been blocked. Your best customer is often your heaviest user, and discovering that in production is expensive.
Allow per-account overrides. Some customers legitimately need more. Without an override you will be redeploying to unblock someone.
Exempt your own systems. Internal services, health checks, and monitoring should not consume a customer's allowance.
Alert on sustained limiting. A customer hitting the limit continuously is either broken or has outgrown their plan. Both are worth a conversation rather than silent throttling.
Document the limits publicly. Integrators design around published numbers; they cannot design around numbers they discover by being blocked.
A workable default
Token bucket keyed on the account, at a rate comfortably above real usage. Tighter IP-keyed limits on login, signup, and password reset. Concurrency caps on expensive endpoints. Standard headers on every response, 429 with Retry-After on rejection. Log-only for a week before enforcing, then overrides available.
That protects the service without turning into a support queue.
If you are exposing an API to customers and want the limits reviewed before they bite, book a call.
Common questions
Which rate limiting algorithm should I use?
Token bucket for general traffic — it allows short bursts up to the bucket size while capping the sustained rate, which matches how real clients behave. Add concurrency limits on expensive, long-running endpoints. Fixed windows are simpler but allow double the intended rate across the window boundary.
Should I rate limit by IP address or by user?
By account or API key wherever the caller is authenticated — it matches billing, survives network changes, and cannot be evaded by rotating IPs. Use IP limits only for unauthenticated endpoints like login and signup, and keep them generous enough that a corporate network or mobile carrier sharing one address is not blocked wholesale.
What should an API return when a rate limit is hit?
429 Too Many Requests with a Retry-After header saying how long to wait, plus an error body naming which limit was hit. Also return the limit, remaining allowance, and reset time on every response, so well-behaved clients can slow down before they hit the wall.
Should rate limiting fail open or fail closed?
It depends on the endpoint, and the important thing is deciding deliberately. For general API traffic, failing open when the counter store is unavailable protects availability. For login and password reset, failing closed protects against credential stuffing. The bad outcome is discovering your default behaviour during an incident.
How do I avoid blocking legitimate customers?
Run the limits in log-only mode first and review who would have been blocked, because your heaviest user is often your best customer. Then keep per-account overrides available so you can raise a limit without redeploying, exempt internal systems and health checks, and alert on anyone being limited continuously.
