A goroutine leak is the sneakiest way to lose memory in Go, because the memory is not garbage: it is a live stack, referenced by a blocked goroutine, unreachable to the collector but also unreachable to your code. Each leaked goroutine pins 2-8KB of stack that grows on demand, and a leak at 10 new goroutines per minute becomes 14,400 live stacks by the end of the day – well past a gigabyte, with the process limping on GC and finally OOM-killing its container. I have seen exactly this trajectory twice, and both times the fix was not more memory; it was a timeout.

The patterns that actually leak

Most production leaks are one of these four:

• Reading from a channel that nobody will ever write to or close. The goroutine sits in chan receive forever.
• Writing to an unbuffered channel whose receiver died or was never started. This one is subtle: the write blocks forever, and it looks like a send loop that is simply slow.
• A time.Ticker nobody stops. The old behavior was a goroutine per ticker that only died on Stop; Go 1.23 (Aug 2024) made unreferenced tickers collectible, which fixed the silent leak, but a referenced ticker in a long-lived struct still ticks forever unless stopped.
• External calls without deadlines: a client whose underlying connection dies while the request goroutine waits on a read that will never return.

The common thread is “waiting without a bound.” Go gives you the tool: context.Context with WithTimeout or WithCancel, and a select. A goroutine that waits on a context and a channel at the same time cannot leak – one of the two branches always fires.

The fixed shape

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
package main

import (
	"context"
	"fmt"
	"time"
)

func startWorkers(ctx context.Context, jobs <-chan int) {
	for i := 0; i < 8; i++ {
		go func() {
			for {
				select {
				case <-ctx.Done():
					return
				case j, ok := <-jobs:
					if !ok {
						return
					}
					handle(j)
				}
			}
		}()
	}
}

func handle(j int) {
	cctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	_ = cctx
	fmt.Println(j)
}

// In a test:
//   func TestWorkers(t *testing.T) {
//       defer goleak.VerifyNone(t)
//       ctx, cancel := context.WithCancel(context.Background())
//       defer cancel()
//       startWorkers(ctx, make(chan int))
//   }

The pattern is boring on purpose: every blocked wait is a select on ctx.Done() and the real operation. When the context fires, the goroutine returns and, in real code, the WaitGroup accounts for it. The test then enforces the discipline. goleak.VerifyTestMain runs at test-suite teardown and fails if any goroutine outlives the tests; it carries a large allowlist of runtime-internal goroutines, so the noise is low. Adding goleak to CI converted “leaks are invisible” into “leaks fail the build.”

Hunting leaks you already shipped

When the leak is live in production, three instruments find it, in this order:

• runtime.NumGoroutine() sampled into a metric. A flat or growing baseline while request rate is stable is a leak. I alert on a 30-minute regression, not an absolute count.
• The goroutine profile: curl localhost:6060/debug/pprof/goroutine?debug=2 gives a plain-text stack dump. Leaked goroutines cluster on the same blocked frame, and that frame names the culprit.
• net/http/pprof in your main binary so every production instance exposes the profile endpoint behind your mTLS mesh. Installing it after the incident is too late; it should be there from day one.

The classic tell is a stack that says chan receive with a caller in code that does not carry a context. Our first incident was an HTTP client call that swallowed the context parameter and fell back to context.Background(): every backend stall leaked a goroutine per request. The fix was making the deadline non-optional – the client constructor takes a timeout, and code paths known to hang reject a background context outright.

Production lessons

• Every external call needs a deadline, period. context.Background() with no timeout attached is a leak waiting to happen.
• Account for every goroutine you spawn with sync.WaitGroup and cancel it with context, not with “the channel will close eventually.”
• Enforce no-leaks in CI with goleak, and ship pprof endpoints in every binary from the first commit.
• Alert on goroutine-count regressions, not spikes; leaks trend, they do not jump.
• When you add a feature, look at its wait points and ask: what fires this goroutine if its partner never does?