When you add vector search to PostgreSQL with pgvector, one of the first decisions is which approximate nearest-neighbor index to use: IVFFlat or HNSW. Both avoid comparing a query embedding against every row, and they work in completely different ways.
IVFFlat clicked for me quickly because it maps onto K-Means clustering. HNSW took much longer because it is a graph structure, and I kept trying to understand it with the clustering model I had already built for IVFFlat.
That confusion turned expensive when a production search issue produced this:
hnsw.ef_search Approximate SQL query time
-------------- --------------------------
150 7 ms
300 21 ms
600 4,900 ms
Doubling ef_search from 300 to 600 did not double latency. It created a cliff.
What both indexes are solving
Suppose you have a million product embeddings. Without an index, PostgreSQL does an exact search:
SELECT id, title
FROM products
ORDER BY embedding <=> $1
LIMIT 10;Accurate, but it computes the distance to every vector before sorting. Approximate indexes search only a slice instead, trading slight recall accuracy for speed.
IVFFlat: partition the space
IVFFlat divides the vector space into groups called lists (clusters).
CREATE INDEX products_embedding_ivfflat_idx
ON products
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 1000);A million vectors across 1,000 lists is roughly 1,000 vectors per list. When a query arrives, PostgreSQL finds the closest lists, and ivfflat.probes decides how many to actually search. probes = 1 searches only the closest list (fast, low recall). probes = 100 searches much more (slower, higher recall).
Mental model: divide vectors into neighborhoods, then choose how many neighborhoods to open.
HNSW: navigate a graph
HNSW does not cluster. It builds a hierarchical graph where each node is a stored vector, and edges are permanent physical wires connecting it to neighbors.
Top layer: A ---------------------- M ---------------------- Z
Middle layer: A ------- F ------- M ------- R ------- Z
Bottom layer: A - B - C - D - E - F - G - H - I - J - K - L - M ...
It works like navigating a city — motorways, then major roads, then local streets. You enter at an upper layer, skip across the graph rapidly, descend into the dense bottom layer, and return the closest results.
What ef_search actually means
This parameter causes the most confusion. hnsw.ef_search = 600 does not mean "visit exactly 600 nodes."
Think of ef_search as the algorithm carrying a clipboard with exactly 600 blank lines.
As a query navigates the graph, it writes down the closest vectors it encounters. Once the 600 lines are full, it uses a strict "one-in, one-out" rolling replacement. If it finds a new node closer to the query than the worst node on the clipboard, it erases the worst one and writes the new one down.
It stops when it hits a dead end. A larger ef_search gives the algorithm massive memory to explore competing candidates, yielding higher recall but substantially more work.
The golden rule of pgvector
Because HNSW only tracks candidates on this clipboard, your SQL LIMIT and ef_search are inextricably linked.
When HNSW hits its dead end, it hands the clipboard to PostgreSQL. If your query asks for LIMIT 100 but your ef_search clipboard only has 40 slots, PostgreSQL can only return 40 results.
The golden rule: your
ef_searchvalue must ALWAYS be greater than or equal to theLIMITin your SQL query.
The latency cliff: memory vs. disk I/O
Why did our ef_search jump from 21 ms to 4,900 ms? The answer is disk thrashing.
Traversing an HNSW graph requires random memory access. If your active graph footprint fits in RAM (shared_buffers or the OS page cache), these hops take nanoseconds.
Increasing ef_search forces the algorithm to explore a larger footprint. If that expanded radius exceeds cached memory, PostgreSQL suffers a cache miss and must fetch the index page from disk. Random disk reads — even on NVMe — take milliseconds, roughly 1,000 to 10,000 times slower than RAM.
Going from 300 to 600 pushed the working set out of memory, turning a fast CPU traversal into an I/O-bound nightmare. HNSW latency is ultimately bounded by your cache hit rate.
The two build parameters
CREATE INDEX products_embedding_hnsw_idx
ON products
USING hnsw (embedding vector_cosine_ops)
WITH (
m = 16,
ef_construction = 64
);m is the permanent physical wires on disk (max edges per node).
ef_construction is the clipboard used only when building. When inserting a node, it uses a clipboard of 64 slots to explore. Once it finds the absolute best 64 candidates, it picks the top 16, wires itself permanently (m = 16), and throws the clipboard away. Higher means slower builds but better graph routing.
Choosing between them
You'll see heuristics like "IVFFlat below 30,000 vectors, HNSW above 50,000." These refer to the point where brute-force exact search becomes too slow, making an index necessary.
The real question is: at what row count does my setup stop meeting targets on my infrastructure?
IVFFlat suits constrained memory and stable datasets. HNSW suits substantial growth and tight latency with high recall, provided you have the RAM to keep the index cached.
Benchmark recall, don't guess it
Compare configurations against a forced exact scan:
SET LOCAL enable_indexscan = off;
SELECT id
FROM products
ORDER BY embedding <=> $1
LIMIT 10;Run the same query with SET LOCAL hnsw.ef_search = 100; and count the overlap. Then compare:
Configuration P95 latency Recall@10
------------- ----------- ---------
ef_search=40 8 ms 86%
ef_search=80 12 ms 92%
ef_search=150 18 ms 96%
ef_search=300 30 ms 98%
ef_search=600 4,900 ms 99%
If pushing from 300 to 600 buys 1% recall for a 5-second penalty, it isn't worth it.
I write about the systems behind shipping AI features — RAG pipelines, evals, and the gap between demo and production — in my newsletter, AI Shipped. New issue every week.