Every object you allocate passes through the runtime allocator, and in a latency-sensitive Go service it is quietly responsible for more of your p99 than almost anything else. It decides how fast goroutines can mint new objects, how much of the heap the GC has to scan, and whether your container ever gives memory back to the OS. The good news: the design is well understood, and it repays studying.

Go’s allocator belongs to the thread-caching allocator family, the same lineage as TCMalloc and jemalloc. The core bet is that nearly all allocations should complete without touching a global lock, because lock contention scales with core count and destroys throughput. It achieves this with three tiers.

Arenas and spans. The runtime asks the OS for memory in large regions called arenas (64 MiB each on amd64) and carves each arena into 8 KiB pages. Pages are grouped into spans, and a span is the unit the allocator hands out. Every span belongs to a single size class: small objects (up to roughly 32 KiB) are cut from a span’s free list, and because every object in the span is the same size, allocation is a pointer bump plus a free-list pop with no compaction ever needed. Large objects skip the size-class machinery entirely and get a dedicated span of however many pages they need.

The three-level cache. The hot path is the per-P mcache. Each P owns a private set of free spans, one per size class, so a typical allocation involves zero locks. When a size class runs dry, the P pulls a fresh span from mcentral for that size class — the first lock, but per size class rather than global. The last resort is the global mheap, which tracks free spans, serves large allocations, and asks the OS for new arenas. On amd64 there are roughly 68 size classes, and the bookkeeping for an allocation is just an index plus two pointers.

The tiny-allocator detail is the one people forget: objects under 16 bytes are packed several into a single slot, so a map of small structs doesn’t burn a whole size class worth of space per key.

Reading the heap correctly. The standard observability surface is runtime.MemStats:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
package main

import (
	"fmt"
	"runtime"
)

func memStats() {
	var m runtime.MemStats
	runtime.ReadMemStats(&m)

	fmt.Printf("HeapAlloc %d MiB   HeapSys %d MiB\n", m.HeapAlloc>>20, m.HeapSys>>20)
	fmt.Printf("HeapInuse %d MiB  HeapIdle %d MiB\n", m.HeapInuse>>20, m.HeapIdle>>20)
	fmt.Printf("Mallocs %d  Frees %d  Live %d\n", m.Mallocs, m.Frees, m.Mallocs-m.Frees)
}

The three fields that matter in production:

• HeapSys: total memory the runtime holds from the OS, committed or not.
• HeapInuse: spans currently holding live objects or available free lists — memory you cannot reclaim cheaply.
• HeapIdle: spans with no live objects; the runtime either returns them to the OS via madvise or keeps them ready for growth.

A profile where HeapIdle climbs while HeapInuse stays flat usually means fragmentation or a leak of large objects holding whole spans hostage. Track HeapAlloc over time for leak detection, and watch the allocation rate (the Mallocs delta), not just the level: GC cost scales with the live heap it has to scan and with the number of objects that survive long enough to be promoted.

A concrete example: a caching service spent 18% of CPU in runtime.mallocgc and sweep workers because a hot path allocated a small struct per lookup and released it immediately. Removing that single allocation dropped GC CPU to 6% and shaved 9 ms off p99. Profiling found it; no pooling trick was involved.

Production lessons, in the order they pay off:

• Profile before pooling. sync.Pool is a cache, not a leak fix; if the allocator shows churn at one call site, eliminating the allocation beats pooling it.
• Prefer a few large buffers over many small objects. Repeated large allocations bypass size classes and fragment the address space.
• Beware unbounded caches held forever: they keep spans in use and defeat madvise, so RSS never comes back down.
• Mind GOGC (default 100: the GC runs when live heap doubles). For latency-tuned services, raising it to 200–400 in exchange for a few hundred MB of headroom is often a good trade; measure it before shipping.

The allocator is predictable once you know the tiers: mcache for the common path, mcentral for refills, mheap for large objects and OS memory. Design your hot path so most allocations never leave the first tier, measure with MemStats and pprof, and you will spend far less time fighting the GC.