Vector Databases, Explained: The Engine Behind AI Search
From keywords to meaning
Traditional search matches strings. Ask a keyword index for "laptop won't boot" and it looks for those exact tokens; a ticket that says "notebook fails POST" never surfaces, even though it describes the same problem. Vector search closes that gap by matching on meaning instead of spelling.
The trick is the embedding. An embedding model turns a piece of text, an image, or an audio clip into a list of numbers — a vector — usually somewhere between 384 and 3,072 dimensions. The model is trained so that inputs with similar meaning land near each other in that high-dimensional space and unrelated inputs land far apart. "Laptop won't boot" and "notebook fails POST" end up as neighbors; a vector database is the system built to store millions of those vectors and answer one question fast: given this query vector, which stored vectors are closest?
Closeness is measured with a distance metric — cosine similarity and dot product are the common choices, with the right one determined by how the embedding model was trained. Get the metric wrong and results quietly degrade, which is one of the first things to verify when a deployment returns mediocre matches.
Why "approximate" nearest neighbors
The naive way to find the closest vectors is to compare the query against every stored vector and sort the results. That is exact k-nearest-neighbor search, and it works fine for a few thousand records. At ten million vectors of 1,000 dimensions each, a single query means ten billion multiply-add operations. Do that for every search and latency and compute cost both explode.
Approximate nearest neighbor (ANN) indexes trade a small amount of accuracy for enormous speed. Instead of scanning everything, they build a data structure that lets a query skip almost all of the corpus and still find the right neighbors the vast majority of the time. The accuracy you give up is measured as recall — the fraction of the true top-k results the index actually returns. Most production systems tune for recall in the mid-to-high 90s and accept that the occasional true match is missed, because the speed gain is often two or three orders of magnitude.
Figure: HNSW connects vectors into a navigable graph, so a query hops across long-range links then refines locally instead of scanning every point.
HNSW: the graph approach
Hierarchical Navigable Small World (HNSW) is the default in most modern vector databases, and it is what the figure above depicts. It organizes vectors into a layered graph: sparse long-range links at the top for covering distance quickly, dense local links at the bottom for precision. A search enters at the top, greedily hops toward the query, and drops a layer at a time until it is refining among near neighbors. HNSW delivers excellent recall at low latency and supports incremental inserts, which matters when data arrives continuously. The cost is memory — the graph lives in RAM, and its links can add a meaningful multiple on top of the raw vector size.
IVF: the partition approach
Inverted File (IVF) indexes cluster the vectors into partitions during a training step, each with a centroid. A query is compared only against the centroids, then searched inside the handful of closest partitions. The tuning knob is how many partitions to probe: probe more for higher recall and slower queries, probe fewer for the reverse. IVF is lighter on memory than HNSW, especially when paired with product quantization, which compresses each vector into a few bytes so billions fit in a fraction of the space. The tradeoffs are a required training step and recall that is sensitive to how evenly the data clusters.
A useful rule of thumb: reach for HNSW when recall and latency dominate and the corpus fits comfortably in memory; reach for IVF with quantization when scale and cost dominate and you can accept a training step and some tuning.
Metadata filtering and hybrid search
Pure semantic similarity is rarely enough on its own. Real queries carry constraints: only documents this user is allowed to see, only records from the last 90 days, only products in stock. That is metadata filtering — every vector is stored alongside structured fields, and the query restricts results to vectors whose metadata matches.
The subtlety is when the filter applies. Pre-filtering narrows the candidate set before the ANN search, which is exact but can be slow if the filter is restrictive. Post-filtering runs the ANN search first and then discards non-matching results, which is fast but can return too few hits when the filter is aggressive. Mature engines blend the two and expose the behavior, and treating access-control metadata as a first-class filter is essential when the same index serves multiple tenants or trust levels.
Hybrid search goes a step further by running semantic (dense vector) search and keyword (sparse, BM25-style) search together and fusing the rankings. Keyword search still wins on exact identifiers — a part number, an error code, a person's name — where the embedding model has little to grab onto. Semantic search wins on paraphrase and intent. Combining them, often with a reciprocal rank fusion step, consistently beats either alone.
Dedicated vector DB or a Postgres extension
Not every project needs a separate database. The honest decision comes down to scale and operational appetite.
- Use a Postgres extension (pgvector) when your corpus is in the low millions of vectors or fewer, you already run Postgres, and you value keeping vectors transactionally consistent with the rest of your relational data. One system to back up, secure, and operate is a real advantage, and pgvector now supports HNSW.
- Use a dedicated vector database when you are at tens of millions of vectors or more, need high query throughput with tight latency, want horizontal sharding and replication built in, or rely on advanced filtering and hybrid features that a bolt-on extension does not match. The cost is another distributed system to run — data governance, backups, and access control that must meet the same bar as everything else in your estate.
The wrong move is defaulting to a specialized cluster for a workload a single Postgres node would serve for years. The other wrong move is bolting hundreds of millions of vectors onto a database never designed for that access pattern and watching latency collapse.
Scaling and cost
Vector workloads are memory-bound before they are compute-bound, and that drives the bill. A back-of-envelope estimate: raw storage is roughly the vector count times the dimensions times four bytes for standard precision, and an HNSW graph adds substantial overhead on top. Ten million 768-dimension vectors is on the order of 30 GB raw before the index — and if that index lives in RAM, memory, not disk, sets the price.
Three levers keep it under control:
- Quantization. Compressing vectors to 8-bit or product-quantized codes cuts memory several-fold for a modest, tunable recall hit.
- Right-sized dimensions. A smaller embedding model that hits your recall target costs less to store and query than the largest model available. Test before assuming bigger is better.
- Elastic infrastructure. Indexing is bursty and querying is steady; separating the two lets each scale independently. Running these tiers on cloud infrastructure with room to grow avoids paying peak-indexing capacity around the clock.
The role in RAG
Retrieval-augmented generation is where most enterprises meet vector databases first. RAG grounds a language model in your own content: the user's question is embedded, the vector database returns the most relevant passages, and those passages are handed to the model as context so it answers from your documents rather than its training data. Retrieval quality sets the ceiling on answer quality — if the right passage is not in the top results, no amount of prompt engineering recovers it. That is precisely why the index type, the distance metric, the metadata filters, and hybrid search all matter so much in practice.
Done well, RAG is how a support assistant cites the current runbook instead of hallucinating, and how an analytics tool answers in plain language over your data. Both sit on the same retrieval foundation, and both are only as trustworthy as the retrieval underneath.
Getting the foundation right
A vector database is not the hard part of an AI search project; embedding choice, index tuning, metadata design, and evaluation are. intSignal designs and operates these systems end to end, from machine learning and AI strategy through the data analytics and retrieval plumbing that makes results trustworthy. If you are planning AI search or a RAG deployment and want the retrieval layer built to hold up in production, talk to our team.