sync.WaitGroup answers one question: “is everyone done?” It cannot tell you someone failed, and it certainly cannot stop the others when one does. For fan-out over shards, partitions, or upstreams you end up hand-rolling an errors channel, a done flag, and a cancel. errgroup is that boilerplate, done correctly.

The core pattern — run a batch, stop the world when one fails:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
package main

import (
	"context"
	"fmt"

	"golang.org/x/sync/errgroup"
)

func ingestAll(ctx context.Context, shards []string) error {
	g, ctx := errgroup.WithContext(ctx)

	for _, shard := range shards {
		shard := shard // loop variable capture is still a footgun in this Go version
		g.Go(func() error {
			return ingest(ctx, shard)
		})
	}

	return g.Wait()
}

func ingest(ctx context.Context, shard string) error {
	return fmt.Errorf("ingest %s: failed", shard)
}

What you get for free: the first non-nil error returned by any goroutine is what Wait returns, and the derived context is canceled the moment that error lands, so every sibling gets the signal to stop. That is the entire value proposition — coordinated failure — and it is why we reach for errgroup over a hand-rolled WaitGroup in almost every fan-out.

When the batch is too big to run at once, bound it:

1
2
3
4
5
6
7
8
9
g, ctx := errgroup.WithContext(ctx)
g.SetLimit(16)

for _, shard := range shards {
	shard := shard
	g.Go(func() error {
		return ingest(ctx, shard)
	})
}

SetLimit (added in golang.org/x/sync v0.1.0) caps the number of goroutines that run concurrently, and g.Go blocks once the limit is reached. That is a semaphore with built-in backpressure, and it is the right default for anything that hits a database or an API you care about — an unbounded fan-out of 512 goroutines all hitting one Postgres is how you get a cascade.

Three gotchas have caused real incidents for us:

1. First error wins, but siblings only stop if they observe the context. Passing the derived ctx into ingest is what makes cancellation work. The classic bug is workers that ignore the context and run to completion — against a slow API that is another 2–30 seconds of load after you already know the fan-out failed. Check ctx.Err() in hot loops, not just at the start.

2. A panic in a worker crashes the whole process. errgroup does not recover. One nil-pointer on a malformed shard config took down an entire pipeline for us. Wrap worker bodies in a defer that converts panics into errors — the group then treats it like any other failure and cancels cleanly instead of killing the pod.

3. Wait returns early, but running goroutines still touch shared state. Collecting results with a shared slice plus append is a data race the moment one worker is still mid-write after Wait returns the first error. Preallocate the slice and write to distinct indices — index writes are safe under concurrency; append is not.

The payoff, measured: a batch of 512 Kafka partitions to ingest. Without cancellation, one slow or hot partition made the whole group wait while the other 511 idled — p99 fan-out time around 3 seconds. With errgroup.WithContext and ctx checks inside each worker, the batch cancels within ~200 ms of the first failure. For a batch that fails every few minutes, that is a ~10x reduction in wasted downstream load, and it stopped a stuck partition from becoming a perpetual resource drain.

When to reach for something else:

• You need every error, not just the first. errgroup deliberately gives you one. Collect full errors through your own channel or results slice.
• Per-task timeouts, retries, and rate limiting belong in the workers, not in the group. The group’s job is coordination, not policy.
• Just “start N, join N” with no error semantics is a WaitGroup, and the lighter weight is fine.

errgroup is the smallest change to your code that makes fan-out failure atomic. The discipline is what matters: pass the derived context everywhere, treat panics as errors, and never append to shared slices. Do that and your batch jobs stop leaking work on failure.