Vector Databases
Gen AI · Retrieval & Embeddings
Theory
A vector database stores embeddings together with the original text and metadata, and answers one main question fast: which stored vectors are nearest to this query vector? Comparing the query with every vector one by one (exact search) is fine for thousands of items but too slow for millions, so vector databases build approximate nearest-neighbour (ANN) indexes that trade a tiny amount of accuracy for a large speed gain.
Options range from dedicated vector databases to vector extensions and vector column types inside existing relational databases and data warehouses, which keeps vectors next to your regular tables. Real queries almost always combine similarity with metadata filters, such as "most similar tickets, only for this customer, only from the last 90 days", and return the top-k matches with their scores.
Why we use it
Searching millions of vectors by brute force is too slow for a live app. A vector database indexes them for fast nearest-neighbour search and combines that with normal filters such as date, category or permissions.
When to use it
Use one when you have many embeddings to search in real time, as in RAG or semantic search. For a few thousand vectors, an in-memory array or a vector column in your existing database is often enough.
How it works
- Question arrives from the app
- Embed the question into a query vector
- Apply metadata filters such as category or user access
- ANN index finds the nearest stored vectors
- Is the score above the minimum relevance? Yes: Keep the chunk in the results. No: Drop the weak match.
- Return the top-k chunks with text and scores
Example
# Pseudocode using a vector DB client
results = vector_db.query(
vector=embed("refund policy question"),
top_k=5,
filter={"category": "billing", "date": {"$gte": "2026-01-01"}}
)
# returns the 5 most semantically similar billing docs from 2026Key points
- Stores vectors plus original text and metadata for each item
- Uses approximate nearest-neighbour (ANN) indexes for speed at scale
- Almost always paired with metadata filters (date, category, permissions)
- Available as dedicated databases or as vector features in existing databases
- Returns top-k matches with similarity scores for the app to use