Every vector index is a bet on a triangle: recall, latency, and memory. Flat scan sits at one corner — perfect recall, but at 1M 1536-d vectors (about 6 GB of fp32 data) it is a 10-second query, which is dead on arrival for any interactive RAG. The two workhorses that replace it, HNSW and IVF-PQ, each give up a different side of the triangle. We run both, and the answer to “which one” is almost never the index itself; it’s your RAM budget and your write pattern.

HNSW: brute force, amortized

HNSW builds a multi-layer proximity graph: layer 0 contains every vector, each higher layer is an exponentially sparser sample, and a query walks greedily from the top layer down. Each node keeps m bidirectional edges, so the graph is the data structure — recall and speed both come from how well those edges approximate the true nearest-neighbor graph.

Numbers from our 2M-row embeddings table (1536-d): with m = 16 and ef_construction = 200, recall@10 is 0.97+ at ef_search = 64 and p95 latency is 2–5 ms on a single reader. That’s an order of magnitude better than ivfflat at the same recall. The catch: everything must live in RAM, graph included. The edge overhead is roughly m × 2 × 4 bytes per vector (~128 MB per million), and the moment the working set touches disk, p95 goes from milliseconds to hundreds of milliseconds.

IVF-PQ: clusters plus compression

Inverted-file indexes run k-means over the dataset (lists clusters), assign each vector to its nearest centroid, and at query time only scan the probes closest clusters. That alone cuts compute by lists / probes. Product quantization is the second lever: split each vector into m subvectors and replace each with its nearest codebook entry, so a 1536-d fp32 vector (6 KB) collapses to m bytes. At m = 48, that’s 48 bytes — a 125x memory cut.

The honest caveat: pgvector has no product quantization. Its inverted index is IVFFlat — cluster pruning with full vectors stored. That still gives you the query-time speedup, but not the memory win. If compression is the goal you need Qdrant (or similar), whose HNSW supports both scalar (int8) and product quantization on top of the graph.

Concrete configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
-- pgvector 0.5.0 (Aug 2023) added HNSW; before that you had ivfflat or flat scans
CREATE EXTENSION vector;

CREATE TABLE items (
  id bigserial PRIMARY KEY,
  embedding vector(1536)
);

-- HNSW: m = edges per node, ef_construction = search effort at build time
CREATE INDEX items_hnsw ON items
  USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 200);

-- ivfflat: lists = k-means clusters, requires >10k rows to be useful
CREATE INDEX items_ivfflat ON items
  USING ivfflat (embedding vector_cosine_ops)
  WITH (lists = 4096);

-- per-query knobs (GUCs, not index options)
SET hnsw.ef_search = 64;   -- trade recall vs. latency at read time
SET ivfflat.probes = 64;   -- clusters scanned per query

The read-time knobs are the production superpower: hnsw.ef_search and ivfflat.probes let you degrade recall gracefully under load spikes instead of shedding traffic. Build-time knobs (m, ef_construction, lists) are locked at create time and hurt to change, because both indexes are built offline and rebuilt expensively.

When to pick which

• HNSW wins when your corpus fits RAM, reads dominate writes, and you want one knob (ef_search) to sweep latency/recall. This is our default for interactive RAG.
• IVF-PQ wins when the corpus outgrows RAM and you can tolerate ~0.85–0.92 recall, or when you rebuild indexes daily and a multi-hour HNSW build is unacceptable — an ivfflat build on the same 2M rows finishes in minutes versus hours for HNSW at high ef_construction.
• Qdrant with PQ on HNSW is the compromise: keep the graph, compress the payload. We measured 4x memory savings with scalar quantization at a recall cost of roughly 0.05 on our data, which beats dropping to ivfflat outright.

Production lessons

• Treat HNSW RAM as a hard floor, not an estimate. We sized one cluster by vector bytes alone and watched p99 collapse when the graph pushed the working set past the machine’s available memory. Measure resident set after the graph warms up.
• Beware the 20% churn rule. Above roughly 20% inserts/deletes, HNSW recall erodes and you should rebuild (REINDEX CONCURRENTLY in pgvector during low traffic) rather than fight the graph in place.
• Validate recall before you tune speed. Our default sequence: build at aggressive ef_construction/lists, measure recall@10 against the flat scan on a held-out sample, then tune ef_search/probes down to the recall floor your product actually needs.
• Don’t put “just search” in the same process as your OLTP. Graph traversal is latency-sensitive and cache-hostile; a dedicated reader keeps pgvector or Qdrant from competing with your primary queries for buffer pool.