When developers add vector search to PostgreSQL using pgvector, one of the first architectural decisions is choosing between two approximate nearest-neighbor indexes: IVFFlat and HNSW. Both avoid comparing a query embedding against every vector in the database, and both can make semantic search dramatically faster. But they work in completely different ways.
I understood IVFFlat quickly because it maps naturally to familiar machine-learning concepts such as K-Means clustering. HNSW took longer. The confusion was not really about vector search — it was that HNSW depends on graph data structures, and I was trying to understand it with the same clustering mental model I used for IVFFlat.
That misunderstanding became especially clear when a production search issue produced this performance pattern:
hnsw.ef_search Approximate SQL query time
-------------- --------------------------
150 7 ms
300 21 ms
600 4,900 msDoubling ef_search from 300 to 600 did not merely double latency. It created a latency cliff.
This article explains why, how IVFFlat and HNSW differ, what lists, probes, m, ef_construction, and ef_search actually mean, and how to choose between the two for a real RAG or semantic-search system.
The problem both indexes solve
Suppose your database contains one million product embeddings, and a user searches for beach wedding dress. You generate an embedding for that query, then find the stored embeddings closest to it.
Without an approximate index, PostgreSQL can perform an exact search:
SELECT id, title
FROM products
ORDER BY embedding <=> $1
LIMIT 24;This produces highly accurate results, but PostgreSQL may need to calculate the distance between the query and every eligible vector — one million comparisons, then a sort, then the closest 24. That becomes expensive as the dataset grows.
Approximate nearest-neighbor indexes search only a carefully selected portion of the dataset instead. The trade-off is that they may occasionally miss a true nearest neighbor, which is why tuning them is mainly a balance between recall — the chance of finding the true closest vectors — and latency.
How IVFFlat works
IVFFlat is easier to understand if you already know clustering. It divides the vector space into groups called lists, which you can mentally treat like clusters.
CREATE INDEX products_embedding_ivfflat_idx
ON products
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 1000);PostgreSQL divides the indexed vectors into approximately 1,000 lists. The distribution will not necessarily be equal, but for a simplified mental model:
1,000,000 vectors ÷ 1,000 lists
≈ 1,000 vectors per listThe index looks conceptually like this:
List 1: running shoes, trainers, sneakers...
List 2: wedding dresses, bridal gowns...
List 3: denim jeans, trousers...
List 4: handbags, clutches...
...
List 1000When a query arrives, PostgreSQL determines which lists are closest to the query embedding. The ivfflat.probes parameter controls how many of them it searches.
SET LOCAL ivfflat.probes = 10;With this simplified example, that is roughly 10 lists × 1,000 vectors ≈ 10,000 candidate vectors searched. Not exact, because list sizes vary, but a useful approximation.
What lists means
lists is an index-construction parameter controlling how many partitions the vector space is divided into.
Too few lists
→ each list contains too many vectors
→ queries scan too much data
Too many lists
→ each list becomes very small
→ you may need more probes to maintain recall
→ index training and tuning become more sensitiveThe current pgvector documentation suggests starting with approximately rows / 1,000 for up to one million rows, and approximately sqrt(rows) for larger datasets. These are starting heuristics rather than universal rules.
What probes means
probes controls how many lists are inspected at query time.
probes = 1 search only the closest list — fast, lower recall
probes = 10 search the ten closest lists — usually better recall
probes = 100 search much more of the index — higher recall, slowerWhen probes reaches the total number of lists, the search effectively becomes exact, and PostgreSQL may decide that using the approximate index is no longer beneficial.
The IVFFlat mental model
Divide vectors into neighborhoods, then choose how many neighborhoods to search.
lists = number of neighborhoods
probes = number of neighborhoods opened during a queryHow HNSW works
HNSW does not divide vectors into clusters. It creates a graph, where each node is one stored vector and each edge is a connection to a nearby vector.
For a handful of product embeddings, HNSW may build connections resembling:
Red evening dress ─── Blue formal dress
│ │
└──────── Wedding gown
│
Beach wedding dress
Running shoes ─── Sports trainers
Denim jeans ─── Casual trousersThe graph contains no explicit categories such as "dress cluster." It stores neighbor relationships: each node knows about several other vectors that were considered close, or useful for navigation, when the index was built.
Why HNSW has multiple layers
The "H" in HNSW stands for Hierarchical. The graph contains multiple levels:
Top layer:
A ---------------------- M ---------------------- Z
Middle layer:
A ------- F ------- M ------- R ------- Z
Bottom layer:
A - B - C - D - E - F - G - H - I - J - K - L - M ...The upper layers contain fewer nodes and longer-range connections. The lower layers contain more nodes and more detailed local connections. You can think of this like navigating a city:
Top layer = motorways
Middle layer = major roads
Bottom layer = local streetsTo find a restaurant you do not inspect every street — you take major roads to roughly the right area, then smaller roads to refine. HNSW works the same way: enter at an upper layer, move toward nodes closer to the query, descend through the layers, search more broadly at the bottom, and return the closest results found.
What ef_search actually means
This parameter causes a lot of confusion. Suppose your query requests 24 results and the application sets:
SET LOCAL hnsw.ef_search = 600;It is tempting to interpret this as: start at one node, visit its nearest node, then continue one-by-one until 600 nodes have been visited. That is not accurate. HNSW is not following one linear chain — it explores multiple promising routes through the graph:
Query
↓
Entry node
/ | \
Candidate A B C
/ \ / \
D E F GDuring the search, HNSW maintains candidate collections, typically implemented as priority queues: one tracks nodes still worth exploring, another tracks the best results found so far. The algorithm repeatedly takes a promising candidate, measures its neighbors against the query, adds the useful ones to the candidate set, discards candidates that are no longer competitive, and stops when further exploration is unlikely to improve the results.
So what does ef_search = 600 mean?
A practical mental model is:
Search the graph using a candidate-result working set with breadth controlled by 600.
It does not mean any of these:
Exactly 600 vectors are compared
Exactly 600 graph hops occur
Each of the 24 requested results receives 25 comparisonsAnd it has no equivalent formula such as 600 × cluster size, because HNSW does not contain IVFFlat-style clusters.
A larger ef_search lets the algorithm retain and explore more competing candidates before settling on the final nearest neighbors — usually better recall, always more work. pgvector's default is 40.
Why LIMIT 24 and ef_search 600 are different
LIMIT 24 means return the final 24 rows. ef_search = 600 means explore a broad set of graph candidates, rank them by distance, and only then return the nearest 24. They control different things.
The formula in our application was:
ef_search = max(limit * 25, 500)So an ordinary 24-result query produced ef_search = 600, and a 100-result query produced 2500. The number 25 was only an application-level heuristic — it did not come from any HNSW rule saying that a returned result requires 25 graph nodes. The formula tied search breadth directly to result count, ignoring the actual latency and recall characteristics of the production index.
A real ef_search latency cliff
We benchmarked the same query directly against a production database dump using the HNSW embedding index:
ef_search Query time
--------- ----------
150 approximately 7 ms
300 approximately 21 ms
600 approximately 4,900 msThere was a severe latency cliff somewhere between 300 and 600. Pinning down the exact cause would require examining PostgreSQL execution details, cache state, I/O metrics, index size, and the underlying storage. One plausible explanation is that the wider graph search crossed a threshold where substantially more HNSW index pages had to be read from disk instead of being served from cache.
The lesson is not that 600 is universally bad. It is that:
HNSW latency is workload-specific and may not scale linearly with
ef_search.
Going from 300 to 600 does not guarantee 20 ms becomes 40 ms. It can just as easily become several seconds, depending on the graph, index size, cache availability, filtering, PostgreSQL configuration, and disk behavior.
We capped the value at 300. The same search dropped from roughly 9,005 ms of SQL vector search to 25 ms, and the full API request went from 28 seconds to 5.6. At that point the index was no longer the bottleneck — most of what remained was generating the query embedding through an external API. Which is a debugging principle in itself:
Do not optimize "vector search" as one single block. Time embedding generation, database retrieval, filtering, reranking, and response generation separately.
HNSW construction parameters
When creating an HNSW index in pgvector, two parameters matter:
CREATE INDEX products_embedding_hnsw_idx
ON products
USING hnsw (embedding vector_cosine_ops)
WITH (
m = 16,
ef_construction = 64
);m
m controls the maximum number of connections created per node at each graph layer.
Lower m
→ fewer graph edges, smaller index, faster construction
→ potentially lower recall
Higher m
→ more graph edges, larger index, slower construction
→ potentially better navigation and recallIt does not mean that every node always has exactly m connections — it is a graph-construction limit. The pgvector default is m = 16.
ef_construction
ef_construction controls how much effort HNSW spends finding good neighbors while building the index.
Lower ef_construction
→ faster index creation, less construction work
→ potentially weaker graph quality
Higher ef_construction
→ slower index creation, better candidate exploration
→ potentially better recallThe pgvector default is ef_construction = 64. It is important not to confuse the two ef parameters: ef_construction is used when building the index, ef_search when querying it.
IVFFlat vs HNSW at a glance
Area IVFFlat HNSW
---- ------- ----
Structure Lists or partitions Hierarchical neighbor graph
Familiar analogy Clustering Road network / graph traversal
Build speed Usually faster Usually slower
Memory usage Usually lower Usually higher
Query tuning probes ef_search
Build tuning lists m, ef_construction
Speed/recall trade-off Good, sensitive to tuning Often stronger
Create before loading data? Not recommended Yes
Requires partition training Yes No
Updates and changing data More operational care Generally convenient
Typical strength Lower memory, faster builds High recall at low latencyThe pgvector documentation describes HNSW as having a better query performance trade-off than IVFFlat, at the cost of slower build times and greater memory use.
Does HNSW only become useful after 30,000 or 50,000 vectors?
You may see rules such as "use IVFFlat below 30,000 vectors, switch to HNSW above 50,000." These are useful anecdotes, not universal thresholds. The choice depends on embedding dimensions, available memory, storage speed, concurrent search volume, acceptable recall, query filters, insert frequency, expected growth, and your actual latency target.
For a few thousand vectors, exact search may already be fast enough, and the difference between IVFFlat and HNSW is irrelevant next to embedding API latency, metadata filtering, reranking, and LLM generation. So instead of asking at what row count is HNSW always better?, ask:
At what row count does exact or IVFFlat search stop meeting my own latency and recall targets on my own infrastructure?
That answer must come from measurement.
High-dimensional OpenAI embeddings and pgvector
High-dimensional embeddings consume substantial storage. A normal vector uses four bytes per dimension, plus a small header:
1,536 dims × 4 bytes = 6,144 bytes ≈ 6 KB per vector
3,072 dims × 4 bytes = 12,288 bytes ≈ 12 KB per vectorFor one million 1,536-dimensional embeddings, that is approximately 6 GB for raw vector values alone — before table overhead, row metadata, HNSW graph links, index metadata, WAL, and operational headroom. Dimensions can become a major memory and storage consideration well before ANN query speed becomes the limiting factor.
pgvector supports the halfvec type, which stores two bytes per dimension and works with both HNSW and IVFFlat. The documented index limits are 2,000 dimensions for vector and 4,000 for halfvec. An HNSW expression index looks like this:
CREATE INDEX products_embedding_halfvec_hnsw_idx
ON products
USING hnsw (
(embedding::halfvec(1536)) halfvec_cosine_ops
);This reduces storage precision, so benchmark recall and ranking quality before adopting it.
Should a new system start with IVFFlat or HNSW?
There is no universal answer, but here is a practical framework.
Start with exact search when the dataset is still small, latency already meets your target, or you are still validating chunking and embedding quality. Exact search gives you a baseline — and without it, you cannot measure approximate-search recall at all.
Start with IVFFlat when memory is constrained, index build time matters, the dataset is relatively stable, and you are comfortable tuning lists and probes.
Start with HNSW when you expect substantial growth, recall and low query latency both matter, you have enough memory, slower index construction is acceptable, and you want to avoid retuning list counts as the dataset changes.
For a production environment expected to grow rapidly from 8,000 vectors to tens or hundreds of thousands, starting directly with HNSW can be reasonable. But that decision should rest on whether HNSW fits your resource limits and hits your measured targets, not on avoiding a future migration.
Migrating from IVFFlat to HNSW in pgvector
A migration does not require rewriting the stored embeddings. Both indexes can exist over the same vector column, so you create the new one alongside the old:
CREATE INDEX CONCURRENTLY products_embedding_hnsw_idx
ON products
USING hnsw (embedding vector_cosine_ops)
WITH (
m = 16,
ef_construction = 64
);CREATE INDEX CONCURRENTLY reduces blocking of normal table writes, although it takes longer and carries operational restrictions compared with a regular build. Afterwards, run ANALYZE products; to refresh planner statistics, confirm the new index is actually used with EXPLAIN (ANALYZE, BUFFERS), and benchmark cold- and warm-cache latency, P50/P95/P99, recall@k, index size, build duration, insertion performance, and behavior under production filters. Once satisfied:
DROP INDEX CONCURRENTLY products_embedding_ivfflat_idx;The main challenge is not the SQL migration itself. The real work is proving that the new index behaves correctly under realistic production queries.
How to benchmark recall correctly
To know whether your approximate results are good, compare them against an exact scan, forced in a controlled environment:
SET LOCAL enable_indexscan = off;
SELECT id
FROM products
ORDER BY embedding <=> $1
LIMIT 24;Then run the approximate version with SET LOCAL hnsw.ef_search = 100; and calculate how many of the exact top-24 IDs also appear in the HNSW result:
Exact top 24: 24 products
HNSW overlap: 22 products
Recall@24 = 22 / 24 = 91.7%Repeat this over many representative queries — never tune on a single one like beach wedding dress. Your benchmark set should mix broad and highly specific queries, rare and common concepts, short and long natural-language phrasings, and queries carrying metadata filters. Then compare configurations:
Configuration P95 latency Recall@24
------------- ----------- ---------
ef_search=40 8 ms 86%
ef_search=80 12 ms 92%
ef_search=150 18 ms 96%
ef_search=300 30 ms 98%
ef_search=600 5,000 ms 99%These numbers are illustrative, except for the production latency measurements discussed earlier. The best setting is not automatically the one with the highest recall. If moving from 300 to 600 buys you 98% → 99% recall at the cost of 30 ms → 5 seconds, the additional recall does not justify the price.
Start from pgvector's defaults — m = 16, ef_construction = 64, ef_search = 40 — and walk ef_search up through 80, 120, 150, 200, 300, jumping to 600 or 2500 only if benchmarking proves it necessary and operationally safe. The same discipline applies to IVFFlat's probes: start small, increase while measuring, and reassess after substantial dataset growth.
Treat tuning formulas as starting points, not truths.
Filtering changes everything
Many production vector searches include filters:
SELECT id, title
FROM products
WHERE tenant_id = $2
AND category = 'dress'
AND in_stock = true
ORDER BY embedding <=> $1
LIMIT 24;Approximate indexes may find vectors first and apply filters afterward, depending on the query and index behavior. That creates a problem:
HNSW finds 40 close candidates
↓
Most belong to another tenant or category
↓
Only 8 survive the filter
↓
LIMIT 24 cannot be satisfiedRecent pgvector versions support iterative index scans for both HNSW and IVFFlat, which can keep scanning when filtering removes too many initial results. You should still create normal indexes on highly selective filter columns, consider partitioning for strong tenant boundaries, and benchmark filtered queries separately — checking both that the planner uses the expected indexes and that enough results survive the filter. A vector index benchmark without production filters can be misleading.
The final mental model
If you remember only one distinction, use this.
IVFFlat divides the vector space into lists, and at query time searches the nearest N of them:
lists = how many neighborhoods exist
probes = how many neighborhoods are searchedHNSW connects vectors to neighboring vectors in a layered graph, and at query time navigates through it exploring promising candidates:
m = graph connectivity
ef_construction = construction search effort
ef_search = query search breadthAnd most importantly, ef_search = 600 does not mean walk through exactly 600 nodes in a straight line. It means use a much broader candidate search while navigating the graph, then return only the requested nearest results.
Conclusion
IVFFlat and HNSW are not two implementations of the same strategy. IVFFlat partitions — find the closest groups, then search inside them. HNSW navigates — move through connected neighbors toward the query. IVFFlat builds faster and uses less memory; HNSW usually gives a stronger speed-recall trade-off. But neither is automatically faster under every configuration, as our production issue showed: an aggressive formula pushed ef_search to 600 for an ordinary 24-result query and took the database across a severe latency threshold.
The larger lesson is simple:
Vector index tuning should be driven by recall benchmarks, query latency, cache behavior, filtering, and production infrastructure — not by an arbitrary multiplier.
Choose the index based on evidence. Tune it with real queries. And measure every stage of the search pipeline separately.
I write about the systems behind shipping AI features — RAG pipelines, evals, and the gap between demo and production — in my newsletter, AI Shipped. New issue every week.