A read syscall costs a few hundred nanoseconds to a few microseconds depending on kernel and storage. That sounds cheap until you multiply by bytes. A 1 GiB log file read one byte at a time is roughly 1.07 billion read(2) calls; at 0.5 µs each, that is about nine minutes of pure syscall overhead before you do any work. At 32 KiB per call you are down to ~33k syscalls and the file is read in milliseconds. Buffer size is the first and cheapest performance lever in any Go I/O path.

Start with a buffered reader sized to reality, not to the page:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
package main

import (
	"bufio"
	"io"
	"os"
)

func readLines(path string) error {
	f, err := os.Open(path)
	if err != nil {
		return err
	}
	defer f.Close()

	br := bufio.NewReaderSize(f, 64*1024)
	for {
		line, err := br.ReadString('\n')
		if len(line) > 0 {
			process(line)
		}
		if err == io.EOF {
			return nil
		}
		if err != nil {
			return err
		}
	}
}

Size the buffer past 4 KiB. The “multiples of 4 KiB match the page size” advice is cargo cult. For disk and socket I/O, 32–64 KiB consistently wins in our benchmarks; it amortizes scheduler wakeups, interrupts, and page faults, and the larger read size is what the kernel’s own buffering is tuned for. A log-forwarding agent ingesting ~4 GB/day across many files: at 4 KiB it held a VM at ~40% CPU; at 64 KiB the same pipeline idled at ~12%. Sixteen times fewer syscalls showed up directly as CPU, not just latency.

Cut allocations before you cut syscalls. ReadString allocates per line. When you control the record format, ReadSlice or ReadLine returns a slice backed by the internal buffer instead — it is invalidated on the next read, so copy out only when you must retain it. At a million lines that is a million allocations deleted from your pprof profile, which also means fewer GC cycles and less cache pressure.

Copy without copying. io.Copy is not just a convenience; it checks for ReaderFrom/WriterTo and delegates. File-to-socket on Linux reaches sendfile; TCP-to-TCP reaches splice. A naive read/write loop moves every byte through user space twice; io.Copy moves it in the kernel. Copying a 100 MiB payload over loopback, we measured ~240 MB/s with a hand-rolled 32 KiB read/write loop versus ~1.1 GB/s with io.Copy.

When the type you hold doesn’t implement WriterTo, control the chunk size yourself with a pooled buffer:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
package main

import (
	"io"
	"sync"
)

var bufPool = sync.Pool{
	New: func() interface{} {
		b := make([]byte, 64*1024)
		return &b
	},
}

func copyStream(dst io.Writer, src io.Reader) error {
	b := bufPool.Get().(*[]byte)
	defer bufPool.Put(b)
	_, err := io.CopyBuffer(dst, src, *b)
	return err
}

The pool matters because io.CopyBuffer otherwise allocates a fresh 32 KiB buffer per call. In a connection handler that is one allocation per request you can eliminate for free.

Implement ReadFrom/WriteTo on your own types. If you wrap a connection or a frame decoder, the fast path through io.Copy only triggers when your type advertises it. Implementing ReadFrom to do bulk framing — read a big chunk, split records, hand them off — turns the copy path into one syscall per chunk instead of one per record.

Production habits that pay off:

• Match the buffer to the record, then batch the writes. Many small writes to a socket each trigger a syscall; a 32 KiB buffered writer turns 1,000 small writes into ~32.
• Never io.ReadAll a stream you can stream. Memory grows with input size, and you hand the garbage collector your throughput.
• Beware the 1-byte-Read consumer. Interfaces let callers request whatever they like; if your writer is given a reader that returns one byte at a time, buffer on your side rather than fighting the source.
• Confirm wins with pprof, not intuition. The syscall/block profile shows exactly where the kernel boundary is eating your CPU, and it will happily tell you when buffering has stopped helping.

Buffering, pooling, and zero-copy delegation are three separate wins that compound. They cost nothing at runtime and read identically to the naive version — there is no reason to ship the naive version.