Our request marshaling path allocated a handful of small buffers per request – fine at 1k req/s, a garbage problem at 20k. sync.Pool looked like the obvious fix, and it was, until it wasn’t: load went up, we started seeing a periodic sawtooth in our p99 latency graph, and the teeth lined up exactly with the garbage collector’s cycles.

This post is about what sync.Pool actually retains, why it empties on every GC, and the pattern that produces the sawtooth – because the single most common mistake is treating sync.Pool as a cache.

What the pool retains

A sync.Pool is not a free list with unbounded lifetime. Internally it is per-P private slots plus a shared list, and the runtime registers a poolCleanup hook that runs at the start of every GC cycle. On each GC, the per-P caches are moved to a “victim” list, and the victim list is dropped on the following GC (victim caching landed in Go 1.13).

Read that carefully: an item placed in the pool survives at most two GC cycles, and Get prefers private, then shared, then victim. The pool is not holding memory for you across time; it is damping the allocation burst that happens when objects die. That is the entire design goal – reduce GC pressure by reusing live objects a little longer, not by caching objects that must survive GC.

The sawtooth

Here is the trap we hit. Steady-state, Get hits the pool 99% of the time, so allocation rate is near zero. Then GC runs, the victim list is dropped, and suddenly every goroutine that previously reused an object is calling New – dozens of thousands of allocations concentrated into a few milliseconds, right after a GC cycle that just finished. The result is a periodic spike you can read off any latency SLO graph: latency normally flat, then a tooth every time GC fires under load.

The fix is not “make the pool bigger” – the pool has no size knob, and New-prewarming is pointless because the next GC clears it. The fix is to understand what the pool is for and when to use a different tool.

The right mental model

Use sync.Pool for objects that are cheap to recreate but frequently created in a hot loop, where you accept them being freed at the next GC. The classic win is bytes.Buffer or serializer/decoder structs in a request handler: the pool converts “one allocation per request” into “one allocation per GC interval.”

If you need a buffer to survive across GCs – connection-scoped buffers for long-lived streams, ring buffers for a bounded cache – sync.Pool is the wrong tool. The sawtooth is the symptom of exactly that misuse. Reach for a bounded free list with a mutex, or better, a buffer owned by a long-lived goroutine, so nothing is dropped behind your back.

The correct pattern

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
var bufPool = sync.Pool{
	New: func() interface{} {
		return make([]byte, 0, 4<<10)
	},
}

func getBuf() []byte {
	return bufPool.Get().([]byte)[:0]
}

func putBuf(b []byte) {
	// Never pool oversized buffers; they pin memory until the next GC.
	if cap(b) > 64<<10 {
		return
	}
	bufPool.Put(b)
}

Three details matter:

• Reset before reuse. [:0] truncates the length; the underlying capacity is reused. If you skip this, a buffer that grew to hold a previous request leaks its contents into the next one.
• Cap what you pool. A buffer that ballooned to 10MB in one request will sit in the pool pinning 10MB until GC drops it. The size guard on Put is what keeps pooled memory bounded by your typical case.
• One goroutine at a time. Get hands the object to exactly one caller; concurrent access from two goroutines is a data race even though it will not crash. Do not pass pooled objects around as shared state.

The benchmark that proves it

1
2
BenchmarkMarshalNoPool-16    12   90 ms/op   3.1 allocs/op   4,213 B/op
BenchmarkMarshalWithPool-16  98   11 ms/op   0.02 allocs/op    289 B/op

Marshal 1k request envelopes, with and without pooling the encoder and its scratch buffer. The allocs/op column is the one that predicts GC behavior: at 0.02 allocs/op, the collector has almost nothing to sweep, and the post-GC sawtooth we saw earlier does not exist because the allocation burst that caused it is gone.

Production lessons

• Watch for the sawtooth. If latency climbs in a repeating pattern after GC, check whether the pool is being emptied and repopulated every cycle.
• Do not prewarm a pool in init; the next GC clears it. Pooling is a steady-state optimization, not warm-start cache.
• Monitor runtime.ReadMemStats – specifically PauseNs and HeapObjects – rather than guessing; the pool’s effect is visible as a change in GC pause distribution, not as a throughput number.
• If pooled objects must survive GC, you need a bounded free list of your own. The pool’s contract is explicit: it clears itself on GC, by design.