The append loop that grows a slice one element at a time is the most common allocation smell we find in Go production code. It is not wrong, exactly – append is amortized O(1), so it will not collapse under load. But the runtime machinery that makes that guarantee true is invisible, and its costs compound quietly: element copies, transient heap arrays, and allocator churn that shows up later as GC pressure in a totally different function.

This post is about what growslice actually does, what preallocation does and does not buy you, and the cases where a “helpful” preallocation makes things worse.

Growth, not magic

When append runs out of capacity, the runtime calls growslice. In Go 1.11 the rule is:

• If the new capacity is under 1024 elements, the backing array roughly doubles.
• Above that, it grows by about 25% (newcap += newcap / 4).
• The final number is then rounded up to the nearest mallocgc size class, so measured capacities rarely match the textbook formula exactly.

The consequence is that every growth step copies the entire existing backing array. Doubling keeps the total bytes copied over a slice’s lifetime bounded by about 2n, which is why append stays O(1) amortized. But you still pay roughly 2n element writes instead of n, plus up to ~log2(n) transient arrays that all die on the heap. None of that shows up in the hot loop; it shows up in the allocator and the GC cycle right after it.

What preallocation actually buys

When you know the target size – because you parsed a header, read a record count, or own the schema – make([]T, 0, n) removes the copies and the transient arrays entirely. The backing array is allocated once; every append becomes a plain index store.

Two subtle points, both of which we have gotten wrong in production:

• make([]T, 0, n) allocates the array and leaves len at 0. Good when you will overwrite every slot with append.
• make([]T, n) zeroes all n elements first. If the code overwrites every element anyway, that is wasted memset work. If the code reads sparse fields, the zeroing is a feature.

The benchmark

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
package slice

import "testing"

func BenchmarkAppendGrow(b *testing.B) {
	for i := 0; i < b.N; i++ {
		var out []int
		for j := 0; j < 1000000; j++ {
			out = append(out, j)
		}
	}
}

func BenchmarkAppendPrealloc(b *testing.B) {
	const n = 1000000
	for i := 0; i < b.N; i++ {
		out := make([]int, 0, n)
		for j := 0; j < n; j++ {
			out = append(out, j)
		}
	}
}

On the 2018-era dual Xeon box we run our CI on:

1
2
BenchmarkAppendGrow-16      100     9.8 ms/op     21 allocs/op     16.0 MB/op
BenchmarkAppendPrealloc-16  244     4.1 ms/op      1 allocs/op      8.0 MB/op

About 2.4x wall time, 21x allocations, and 2x cumulative bytes. The allocation count is the number that matters long-term: 20 short-lived 0.5-8MB arrays per operation is exactly the churn that turns a 1ms heap into a 30ms GC pause later in the process.

When preallocation is wrong

Preallocation is not a free win:

• Overestimating is expensive. Capacity is the allocation, not length. A slice preallocated to 10MB to “be safe” pins 10MB for its entire lifetime, even if it only ever holds 10 records. If you only know an upper bound, prefer append growth or a smaller initial capacity.
• Underestimating is cheap. If the real size exceeds your preallocation, growth kicks in as normal and you lose some of the benefit – but correctness is never at risk. Err on the side of a modest initial cap, not a giant one.
• Subslicing keeps the whole array alive. in := readHuge()[:4] holds the full backing array of readHuge until in is garbage. If you keep a small slice around, copy it into a fresh small allocation to let the big one go. This is the single most common slice memory leak we debug.

Production lessons

• Preallocate when the size is known at the call site, not when you could guess it from a previous loop. Guessing is how you end up holding 10MB for ten records.
• Always benchmark with -benchmem. The ns/op number hides allocation churn; the allocs/op and MB/op columns are what predict GC behavior.
• When profiling points at a hot append loop with unknown size, measure the real distribution of sizes first. If 99% of inputs are under 64 elements, a make(..., 0, 64) warm start captures most of the benefit with none of the downside.