Rate limiting is a liveness problem

An API without rate limits degrades for everyone during a burst: a scraper or a badly written client loop consumes database connections, warms the app, and the legitimate users get timeouts. Rate limiting belongs at the Rack edge, before sessions, before the database, before controllers – the earlier you shed load, the more of the stack you protect.

Fixed windows, not token buckets

First correction: Rack::Attack does not do token buckets. It keeps a fixed-window counter in the cache store – an INCR on a key scoped to (rule, discriminator, window) with an EXPIRE at the window boundary. That has two consequences worth designing around:

1. A client can do up to 2x the limit in the two windows straddling a boundary.
2. Every window gives every client a free refill.

That is the right trade for most APIs: simple, cheap, and observable. If you need a true sliding window or token bucket semantics, you need a different tool, not a Rack::Attack feature flag.

The rules

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# config/initializers/rack_attack.rb
require "rack/attack"

Rack::Attack.cache.store = Rack::Attack::StoreProxy::RedisProxy.new(
  Redis.new(url: ENV.fetch("REDIS_URL"))
)

class Rack::Attack
  safelist("monitoring") { |req| req.path == "/healthz" }

  # Global per-IP throttle: 300 requests per 5 minutes.
  throttle("req/ip", limit: 300, period: 5.minutes) { |req| req.ip }

  # Brute force: ban the email+IP combo that fails 5 logins in 20s.
  fail2ban("logins", maxretry: 5, findtime: 20.seconds, bantime: 15.minutes) do |req|
    if req.path == "/login" && req.post?
      "#{req.ip}:#{req.params['email'].to_s.downcase}"
    end
  end

  self.throttled_response = lambda do |env|
    data = env["rack.attack.match_data"]
    retry_after = (data[:period] - (Time.now.to_i % data[:period])).ceil
    [429,
     { "Content-Type" => "text/plain", "Retry-After" => retry_after.to_s },
     ["Rate limited\n"]]
  end
end

Two practical notes on the config. First, req.ip only reflects the real client if the proxy chain is configured – without trusting your load balancer, every IP throttle keys on 127.0.0.1 and the rule silently applies to everyone at once. Configure ActionDispatch::RemoteIp trusted proxies, or rate limiting by IP becomes a global lock. Second, keep the middleware at the top of the stack (config.middleware.insert_before ActionDispatch::... or config.middleware.use Rack::Attack) so throttled requests never touch the session or the database.

fail2ban is the right tool for logins because it combines a short observation window with a longer ban – one bad actor gets cut off for 15 minutes after five failed attempts, but no legitimate user is ever throttled for a single typo.

The latency and failure budget

A Redis INCR+EXPIRE round trip on the same host costs about 0.2ms; cross-region it is closer to 1ms. That is the price of correctness (shared state across processes) and it is worth it. But it means Redis is now a single point of failure: if the store is down, every request would be rate limited – or worse, every request would throw. We wrap the store so Redis failures mean “let it through”:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
class FailOpenStore
  def initialize(inner)
    @inner = inner
  end

  def read(key)
    @inner.read(key)
  rescue Redis::BaseError
    nil
  end

  def write(key, value, options = nil)
    @inner.write(key, value, options)
  rescue Redis::BaseError
    true
  end

  def increment(key, amount = 1, options = nil)
    @inner.increment(key, amount, options)
  rescue Redis::BaseError
    1
  end

  def decrement(key, amount = 1, options = nil)
    @inner.decrement(key, amount, options)
  rescue Redis::BaseError
    0
  end

  def expire(key, period)
    @inner.expire(key, period)
  rescue Redis::BaseError
    true
  end
end

Rack::Attack.cache.store = FailOpenStore.new(
  Rack::Attack::StoreProxy::RedisProxy.new(Redis.new(url: ENV.fetch("REDIS_URL")))
)

Fail-open is a deliberate choice, and it is wrong for some endpoints: if your login endpoint goes unlimited during a Redis outage, an attacker gets a free window. We fail open on the read-heavy API and accept the trade, and we alert loudly the moment the store is unreachable.

Calibrating in production

Never guess limits. Run a dry-run mode first: throttled_response that logs the would-be match and returns 200, shipped for a week. The surprise is always your own integrations – cron jobs, partner scripts, mobile clients that reconnect in a tight loop. Real traffic logs give you the numbers to set limits that block abusers and spare everyone else. After going live, keep every match at WARN level; a limit that starts blocking 5% of a legitimate fleet is a rollout bug wearing a security costume.

• Throttle by the identity that carries the abuse, not just the IP. A partner script behind a NAT pool defeats per-IP limits entirely; per-API-key throttling on the auth token is what finally stopped ours.
• Include Retry-After and set the response to 429 with a plain-text body; a machine-readable backoff prevents thundering-herd retries.
• Watch Redis commandstats for INCR volume after launch; a hot key per popular IP is fine, but per-email fail2ban counters at global scale add up in memory.
• Keep limits in configuration, ship conservative, and tune from the dry-run logs rather than from imagination.