Goroutines are cheap — a couple of KiB of stack, sub-microsecond spawn — so teams over-rotated from “thread per request” to “goroutine per request” and hit a wall. Unbounded concurrency is not free: every blocked goroutine holds its stack and often a buffer, the scheduler’s run queue gets long, the GC has more stacks to scan, and worst of all there is no backpressure. A 3x traffic spike stops being a latency event and becomes a memory event. In our API ingestion tier, letting concurrency run unbounded pushed max RSS to ~1.2 GB and p99 to 900 ms under load. The fix wasn’t more RAM; it was bounding concurrency with a worker pool.

The shape. One buffered jobs channel, N workers, one results channel. The producer pushes jobs; workers pull and process; the caller drains results. The buffer is your backpressure: when it fills, the producer blocks, and that blocking propagates upstream to whoever is feeding you. That is the property a per-request goroutine never gives you.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
package main

import (
	"context"
	"sync"
)

type Job struct {
	ID   int
	Data []byte
}

func run(ctx context.Context, jobs <-chan Job, workers int, process func(Job) error) []error {
	var wg sync.WaitGroup
	wg.Add(workers)

	results := make(chan error, workers)
	for i := 0; i < workers; i++ {
		go func() {
			defer wg.Done()
			for job := range jobs {
				select {
				case <-ctx.Done():
					return
				default:
				}
				results <- process(job)
			}
		}()
	}

	go func() {
		wg.Wait()
		close(results)
	}()

	var errs []error
	for err := range results {
		if err != nil {
			errs = append(errs, err)
		}
	}
	return errs
}

Two details in here are load-bearing. The results channel is buffered to the worker count, so workers can never deadlock sending into a channel nobody is draining after cancellation. And the jobs channel is closed by the producer only — a worker closing it would panic any goroutine still trying to send.

Tuning the worker count. Size it by your bottleneck, not by some multiple of cores. Our workers spend ~60 ms of every job in a downstream call, so they are I/O-bound; we oversubscribe. Load tests told the story: 16 workers gave p95 of 120 ms at 4k req/s; 32 workers cut it to 45 ms; 64 workers hit 38 ms but the downstream service started throttling us at 5k req/s. We landed on 48. For CPU-bound work, ~GOMAXPROCS is the right neighborhood; for I/O-bound work, pick the number of in-flight downstream requests you can afford and validate with a load test.

Tuning the buffer. A zero-length buffer starves workers while the producer does its own work. A buffer of millions defeats backpressure and just parks memory. We settled on 10,000 jobs: at 48 workers and ~60 ms per job, that is a couple of seconds of slack — enough to absorb bursts, small enough that the spike turns into producer blocking instead of OOM.

The results under the same 2k req/s load, goroutine-per-request versus a 16-worker pool: p99 went 900 ms to 120 ms, and max RSS went 1.2 GB to 210 MB. Nothing about the actual work changed; we just stopped letting the scheduler and allocator absorb load for us.

Production lessons:

• Close the jobs channel from the sender only. A worker that closes it while another sends panics with “send on closed channel”. Close it exactly once, in the producer, after all sends complete.
• Buffer the results channel to the worker count. If you close over an unbuffered channel that the drain loop isn’t reading after cancellation, workers block forever and Wait never completes.
• Instrument queue depth, not just latency. Expose len(jobs) as a gauge; alert when the buffer stays near-full for sustained periods — that is your signal to scale out or shed load, well before p99 tells you.
• Watch the one-slow-worker tail. If job times vary wildly, a shared pool gives you head-of-line blocking; consider per-stream pools or a bounded semaphore for the pathological cases.
• Prefer errgroup.WithContext plus a buffered-channel semaphore when jobs are homogeneous and you don’t need result ordering. It is this same pattern with first-error cancellation built in, which is usually what you actually wanted.

A worker pool is a bounded-resource declaration: “the world can send me as much as it wants; I will process it at this rate, and the excess will wait, visibly, where I can measure it.” That visibility — queue depth, worker utilization, per-job latency — is what turns a firefight into a tunable system.