API Rate Limiting

The foundational traffic-shaping mechanism underlying most modern rate limiters. A conceptual bucket holds tokens that replenish at a fixed rate up to a defined burst capacity. Each request consumes one or more tokens; when the bucket empties, requests are rejected with a 429 Too Many response. This allows short bursts of legitimate traffic while capping sustained throughput. Key parameters include refill rate (tokens/second) and bucket depth (max tokens). Implementations range from in-memory counters in API gateways to distributed buckets synchronized via Redis for horizontally scaled services.

A precise rate-limiting algorithm that timestamps every request in a sorted log and counts entries within a rolling time window. Unlike fixed-window counters that reset abruptly at interval boundaries, the sliding window log eliminates edge-case burst doubling where a client sends requests at the end of one window and the beginning of the next. The trade-off is higher memory consumption, as the log retains individual timestamps. Optimized variants like the Sliding Window Counter approximate this behavior by weighting counts from the previous and current fixed windows, balancing accuracy with resource efficiency.

A traffic-shaping algorithm that processes requests at a constant, fixed rate regardless of burstiness. Incoming requests enter a FIFO queue (the bucket); if the queue overflows, excess requests are discarded. The bucket leaks at a steady rate, smoothing traffic into a uniform flow. This is ideal for guaranteeing consistent backend load rather than accommodating bursts. Commonly used in network traffic policing and scenarios where downstream systems have strict throughput limits. Contrasts with the token bucket, which permits bursts up to the bucket's capacity before throttling engages.

The simplest rate-limiting approach: a counter increments per request within a discrete time block (e.g., 1000 requests per hour). At the window boundary, the counter resets to zero. Implementation is trivial with atomic increment operations in Redis or Memcached. The critical flaw is the boundary condition problem: a client can consume the full quota at the end of one window and immediately consume the next window's quota, effectively doubling the allowed rate. Suitable for coarse-grained limits where precision is not critical, but inadequate for strict enforcement against sophisticated scrapers.

An HTTP response header returned alongside 429 Too Many Requests status codes that instructs the client how long to wait before retrying. The value can be an HTTP-date (absolute time) or a delta-seconds integer (e.g., Retry-After: 120). Well-behaved API consumers parse this header and implement exponential backoff with jitter to avoid synchronized retry storms. Scrapers often ignore this directive, making its presence a useful signal for distinguishing compliant clients from malicious automation. Combined with RateLimit-Reset and RateLimit-Remaining headers for full transparency.

The challenge of enforcing consistent quotas across horizontally scaled API gateways where requests from a single client may hit different server instances. Solutions require a shared state store (typically Redis or Memcached) with atomic operations. The Race Condition Problem arises when concurrent requests read and increment counters simultaneously. Mitigations include Lua scripting for atomic check-and-increment operations, or using sorted sets for sliding window implementations. For global deployments, eventual consistency trade-offs must be weighed against strict accuracy requirements across geographic regions.