The classic slow controller is a dashboard that aggregates three independent services. Balance from billing, usage from the analytics service, unread count from notifications. Called sequentially, a request that should take 200 ms takes the sum of all three — 650 ms — and the p95 is worse, because p95 latencies sum too. The services don’t depend on each other, so why is the user waiting on them one at a time?

This post is about running those calls concurrently without writing a new footgun. The tools are concurrent-ruby futures, a bounded pool, and a handful of rules about ActiveRecord that keep threads from eating the connection pool.

Futures, not raw threads

Raw Thread.new works but gives you no composition, no error propagation, no timeout. concurrent-ruby futures give you all three. Use the newer Concurrent::Promises API:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
API_POOL = Concurrent::FixedThreadPool.new(16, max_queue: 200, fallback_policy: :caller_runs)

def show
  balance = Concurrent::Promises.future(executor: API_POOL) { billing.balance_for(user) }
  usage   = Concurrent::Promises.future(executor: API_POOL) { usage.last_30_days(user) }
  unread  = Concurrent::Promises.future(executor: API_POOL) { notifications.unread(user) }

  results = Concurrent::Promises.zip(balance, usage, unread).value(8)
  if results
    render json: { balance: results[0], usage: results[1], unread: results[2] }
  else
    render json: { error: "upstreams slow" }, status: :service_unavailable
  end
end

zip waits for all three and returns the values in order; value(8) gives them a shared 8-second deadline instead of letting one hung service hold the request open forever. A nil result means the deadline passed — degrade gracefully instead of 500-ing.

The pool deserves more thought than the futures. CPU count is the wrong dimension; you’re waiting on I/O, so the pool should be sized for external concurrency and bounded so a traffic spike can’t queue unboundedly. fallback_policy: :caller_runs is the important part: when the pool and its queue are full, the requesting thread executes the job rather than parking it in an ever-growing queue. You get backpressure instead of an OOM, and the cost is graceful degradation under load rather than a death spiral.

The wall-clock win is real: three 200 ms services run sequentially are 600 ms, concurrently they’re 200-250 ms, and the p95 spread compresses the same way because you’re taking a max, not a sum. The win is user-perceived latency. Throughput per host doesn’t change much — you’re still bounded by Puma threads — but you can lower thread count per request, which matters when requests pile up.

ActiveRecord in child threads

This is where concurrent Rails code goes wrong. The connection pool hands out connections per-thread; a child thread that touches ActiveRecord checks out its own connection. Two failure modes follow:

• The pool is too small. Default pool size is 5. If 16 futures all query the database, threads block waiting for a connection. Raise pool: in database.yml to at least pool-size-plus-futures, or keep the pool wide enough for the concurrency you actually use.
• Leaked connections. A ThreadPoolExecutor reuses threads, so the pool’s threads never die and never return their connections. If a future checks out a connection and doesn’t return it, that connection is gone for the life of the pool thread.

The fix is to never let a future touch the default connection checkout. Wrap DB work in an explicit, guaranteed-returning block:

1
2
3
4
5
6
7
def cached_usage(user)
  Concurrent::Promises.future(executor: API_POOL) do
    ActiveRecord::Base.connection_pool.with_connection do
      usage.last_30_days(user)
    end
  end
end

with_connection guarantees check-in even on exception. The corollary that we enforce in review: a future should receive IDs or pre-fetched data and never a shared model instance, and it should never hold a connection open across the whole future body. If your futures are doing DB work at all, size the pool against the connection pool and treat them as the same resource.

Timeouts are not optional

A future with no timeout turns “one slow upstream” into “one slow client.” Set timeouts at both layers:

• Inside the future, on the HTTP client. Net::HTTP gets explicit open_timeout and read_timeout; with Faraday it’s conn.options.timeout. The future should fail on its own terms, with its own error surfaced via future.reason.
• Outside, on the composition. The value(8) above is the outer guardrail, because individual timeouts still add up if three services each take 7 seconds.

The zip(...).value(deadline) pattern is what we ship with; per-future timeouts make the error handling precise, the outer deadline makes the user experience bounded.

Production lessons

• Size the executor to external concurrency and cap its queue; caller_runs is your safety valve.
• Never share a connection or a model instance between threads. Pass IDs, load inside with_connection.
• Bump the connection pool to match the thread pool, or the futures starve.
• Give every future an inner timeout and the composition an outer deadline.
• Thread.report_on_exception doesn’t save you here — assert on future.reason rather than hoping.
• Resist spawning futures in a loop per row; fan-out is for a handful of independent calls, not for collections. That’s what parallel find_each or in_batches are for, and they’re a different discipline entirely.

Concurrency in a controller is a small, contained win if the rules are enforced at the pool and connection boundaries. Keep the futures dumb, keep the pool bounded, and keep ActiveRecord out of the default checkout, and the dashboard gets fast without the pager going off.