Glossary · Data & RAG
Vector search
Vector search retrieves information by comparing numerical embeddings for semantic similarity, finding passages that mean the same thing as a query rather than sharing its exact keywords.
Vector search retrieves information by comparing numerical embeddings for semantic similarity, so it finds passages that mean the same thing as a query even when they share no keywords. It contrasts with keyword search, which matches exact terms. Hybrid vector-plus-keyword usually wins in enterprise corpora.
- Vector search matches meaning via embedding similarity, not exact words.
- Keyword search matches exact terms and is precise on names, codes and IDs.
- Hybrid retrieval (vector plus keyword) usually beats either alone in enterprise corpora.
- Vector search is the retrieval engine inside most RAG pipelines.
- It needs the same embedding model at index time and query time, or results degrade.
Also known as: semantic search, vector database search
Vector search retrieves information by comparing numerical embeddings for semantic similarity, finding passages that mean the same thing as a query rather than sharing its exact keywords.
Every passage and every query is turned into an embedding (a point in a high-dimensional space), and search becomes a matter of finding the points nearest to the query's point.
How vector search works
Text is converted into embeddings by an embedding model: similar meanings land close together, unrelated meanings land far apart. At index time, every chunk of your documents is embedded and stored. At query time, the query is embedded with the same model, and the system finds the stored vectors closest to it (usually by cosine similarity) returning the passages behind them.
Because it works on meaning, vector search returns a passage about "staff attrition" for a query about "employee turnover" even though they share no words. That is its strength and, at the edges, its weakness: it can also miss an exact part number or clause reference that a keyword index would catch instantly. This is why production systems typically run hybrid search (vector and keyword retrieval together, results merged) as the reliable default. Vector search is the retrieval engine inside most RAG pipelines.
Why vector search matters for enterprise AI adoption
Enterprise questions rarely use the document's exact words. A user asks in their own language; the answer is buried in a policy that phrased it differently three years ago. Keyword search alone forces the user to guess the vocabulary the author used. Vector search removes that guess, which is what makes a knowledge assistant feel like it understands the question rather than pattern-matching it.
But relevance is measurable, and it should be measured. The right vector database, index and ranking are the ones that produce the best recall on a labelled set of your own questions, not the ones with the best marketing. Choosing on measured retrieval quality, and pairing vector with keyword where identifiers matter, is the difference between a search that impresses in a demo and one that holds up on the real corpus.
Common mistakes with vector search
The first mistake is going vector-only on a corpus full of codes, SKUs and clause references, then being surprised when exact-match queries fail. Prose wants vectors; identifiers want keywords; most enterprise documents have both, so hybrid is the safe starting point.
The second is the embedding mismatch: changing the embedding model without re-indexing, so queries are compared against stale vectors and relevance silently degrades. The third is never establishing a recall baseline: without one, every "improvement" is a guess, and you cannot tell a better index from a lucky query. Put retrieval into an evaluation harness and measure it.
Related terms
- Embeddings: the numerical representations vector search compares.
- Retrieval-augmented generation: the pipeline vector search powers.
- Model Context Protocol: a way to expose vector retrieval to a model as a tool.
- Evaluation harness: how retrieval recall is baselined and tracked.
How Chokmah approaches vector search
We pick retrieval on measured recall, not database fashion. In a workflow sprint we start hybrid (vector plus keyword) because enterprise corpora mix prose with identifiers, and we baseline recall against a labelled question set drawn from real user queries before tuning anything. When an embedding model changes, we treat the full re-index as planned work, not an afterthought, because pretending it is free is how relevance quietly rots.
Sources
- Model Context Protocol, Specification (2025-06-18). https://modelcontextprotocol.io/specification/2025-06-18
Related terms
- EmbeddingsAn embedding is a list of numbers that represents the meaning of text or other data, positioning similar items close together in a high-dimensional space so software can compare them.
- Retrieval-augmented generation (RAG)Retrieval-augmented generation (RAG) is a technique that fetches relevant passages from your own documents and supplies them to a language model as grounding before it generates an answer.
- Model Context Protocol (MCP)The Model Context Protocol (MCP) is an open standard that defines how AI applications connect to external tools and data through one uniform interface instead of many bespoke integrations.
- Evaluation harnessAn agent evaluation harness is a repeatable test suite that scores an AI agent's outputs against fixed, versioned cases before and after every change, so teams can tell regression from variance.
Where it shows up in our work
Frequently asked questions
Keyword search matches the exact terms in your query against the text; vector search matches meaning by comparing embeddings, so it can find a passage about "staff attrition" when you searched "employee turnover". Keyword search is precise on names, part numbers and codes but blind to synonyms. Vector search handles paraphrase and concept but can miss an exact identifier. They fail in opposite ways, which is why combining them works.
Hybrid search runs vector and keyword retrieval together and merges the results. It usually beats either alone in enterprise corpora because the two cover each other's blind spots: keyword search nails the exact SKU or clause reference, vector search catches the paraphrased concept. For documents full of both prose and identifiers (contracts, policies, tickets) hybrid retrieval is the reliable default rather than an optimisation.
Not always. A dedicated vector database helps at scale and adds features like filtering and hybrid ranking, but many relational and search engines now offer vector indexes that are sufficient for moderate corpora. The right question is not which database is fashionable but what your recall is on a labelled question set at your data volume. Choose based on measured retrieval quality and operational fit, not on the database category alone.
The most common cause is an embedding mismatch. Vector search compares query embeddings against indexed embeddings, and they must come from the same model. If you change the embedding model and do not re-index every document, new queries are compared against stale vectors and relevance quietly collapses. Any embedding-model change forces a full re-index: a real operational cost that surprises teams who treat it as a config tweak.
Put the concept to work
We install working agentic workflows, not vocabulary. Book a free AI Reality Check.