Embeddings

Gen AI · Retrieval & Embeddings

Theory

An embedding converts text (or an image) into a fixed-length list of numbers, called a vector, that captures its meaning. An embedding model is trained so that texts with similar meaning land close together: "puppy" and "young dog" sit near each other, while "puppy" and "tax filing" are far apart, even when no words are shared.

Closeness is usually measured with cosine similarity, which compares the direction of two vectors: values near 1 mean very similar, values near 0 mean unrelated. This powers semantic search, where a query for "cheap laptops" finds a document titled "budget notebooks", plus recommendations, clustering and duplicate detection. Always embed queries and documents with the same embedding model, because vectors from different models cannot be compared.

Why we use it

Keyword search misses matches that use different words for the same idea. Embeddings let software compare meaning, which is the basis of semantic search, RAG, recommendations and near-duplicate detection.

When to use it

Use embeddings for search over free text, matching similar tickets or products, and clustering feedback. For exact IDs, codes or names, keyword or SQL filters work better; many systems combine both.

How it works

  1. Text to compare: query and documents
  2. Send each text to the same embedding model
  3. Receive a fixed-length vector for each text
  4. Compute cosine similarity between query and each document
  5. Is the score above your relevance threshold? Yes: Treat the document as a match. No: Ignore the document.
  6. Return matches ranked by similarity

Example

embed("a happy dog") → [0.12, -0.44, 0.91, ...]  (e.g. 1536 numbers)
embed("a joyful puppy") → [0.14, -0.41, 0.88, ...]  (very close to the above)
embed("quarterly tax filing") → [-0.77, 0.30, -0.02, ...]  (far away)

cosine_similarity(dog_vec, puppy_vec) ≈ 0.93  (highly similar)
cosine_similarity(dog_vec, tax_vec) ≈ 0.04  (unrelated)

Key points

Gen AI topics

Foundations of Generative AI

Prompt Engineering

Retrieval & Embeddings

Customizing Model Behavior

AI Agents & Tool Use

Multimodal & Applied AI

Ethics, Cost & Careers

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Generative AI for the Data Stack

LLMs, prompt engineering, RAG, embeddings, agents and the tools shaping modern AI products — explained practically. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Section {{ sec.num }}

{{ sec.label }}

Gen AI  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

💡

{{ current.tldr }}

Explained

{{ para }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
See It In Action
{{ current.example }}
Key Takeaways — click to check off
{{ pt.mark }} {{ pt.text }}

Share this topic