An embedding is a list of numbers (a vector) that represents the meaning of a piece of text, an image or another item, produced by a model trained so that similar meanings get similar numbers. Comparing two embeddings, usually with cosine similarity, tells you how related the two items are, which is what powers semantic search, recommendations and retrieval-augmented generation.
What an embedding looks like
OpenAI's documentation defines it in one line: "An embedding is a vector (list) of floating point numbers." Send the sentence "How do I reset my password?" to an embedding model and you get back something like this (example output, truncated):
[0.0123, -0.0481, 0.0067, 0.0214, -0.0039, ... ] # 1,536 numbers in totalNo single number means anything you can read. The meaning lives in the pattern across all of them, and in how that pattern relates to the vectors of other texts. "How can I change my login credentials?" shares almost no words with the first sentence, but its vector ends up close to it. "Best pizza in Naples" ends up far away.
The length of the list is the model's number of dimensions. As of October 2026, OpenAI's text-embedding-3-small returns 1,536 numbers by default and text-embedding-3-large returns 3,072.
Why turn words into numbers at all?
Computers can compare numbers precisely but can't compare meanings. The older approach, giving each word its own position in a huge list of zeros ("one-hot encoding"), treats every word as equally unrelated to every other. Google's Machine Learning Crash Course uses the example that a hot dog should be more similar to a shawarma than to a salad, which one-hot vectors can't express. They are also enormous: one dimension per word in the vocabulary.
Embeddings fix both problems. They're dense (every number carries information) and much shorter, and the training process places related items near each other. The 2013 word2vec paper by Mikolov and colleagues cites a well-known demonstration that simple arithmetic on word vectors captures relationships: vector("King") − vector("Man") + vector("Woman") results in a vector closest to the representation of "Queen".
How similarity is measured: cosine similarity
To compare two embeddings, you measure the angle between them. Cosine similarity is 1 when two vectors point in exactly the same direction, near 0 when they're unrelated, and negative when they point in opposite directions:
cosine_similarity(a, b) = (a · b) / (|a| × |b|)Here it is with three made-up 3-dimensional vectors (real ones have hundreds or thousands of dimensions, but the arithmetic is identical):
import math
v = {"cat": [0.8, 0.6, 0.1], "kitten": [0.75, 0.65, 0.15], "car": [0.1, 0.2, 0.95]}
def cos(a, b):
dot = sum(x * y for x, y in zip(a, b))
return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))
print(round(cos(v["cat"], v["kitten"]), 3)) # 0.996
print(round(cos(v["cat"], v["car"]), 3)) # 0.301"cat" and "kitten" score 0.996; "cat" and "car", which differ by one letter but not in meaning, score 0.301. That's the property that makes embeddings useful: they compare meaning, not spelling.
OpenAI recommends cosine similarity for its embeddings and notes that they are normalised to length 1. For normalised vectors the denominator is 1, so cosine similarity is just the dot product, which is cheaper to compute, and it ranks results the same way as Euclidean distance. Absolute scores vary by model, so pick any "similar enough" threshold by testing on your own data rather than reusing a number from another model.
What embeddings are used for
| Use | How embeddings help |
|---|---|
| Semantic search | Embed the query and every document; return the documents with the closest vectors, even if they use different words |
| Retrieval for chatbots (RAG) | Find the passages most relevant to a question and give them to a language model as context |
| Recommendations | Suggest items whose descriptions sit near ones a user liked |
| Clustering | Group support tickets, reviews or articles by topic without predefined labels |
| Classification | Label a text with whichever example category it's most similar to |
| Deduplication and anomaly detection | Flag near-identical items (very high similarity) or outliers (low similarity to everything) |
The search-for-chatbots use is the most common today. It's covered step by step in what is RAG in AI. Microsoft's Azure AI Search documentation describes the same pipeline: split documents into chunks, run a vectorisation step to create embeddings, and use vector queries (often combined with keyword search) to find the best chunks for a question.
Embedding models vs chat models
An embedding model and a chat model like the ones behind ChatGPT or Claude are trained for different jobs. A chat model takes text and generates more text. An embedding model takes text and outputs one fixed-length vector; it doesn't write anything. The same model must be used for both your stored documents and incoming queries, because vectors from different models aren't comparable.
Embedding models also have an input limit. OpenAI lists 8,192 tokens for its current embedding models, so long documents have to be split into chunks first. If you're not sure how long a document is in tokens, a counter such as larely's token counter gives an estimate, and how many tokens is a word explains the rule of thumb.
Choosing dimensions: bigger isn't always better
More dimensions can capture finer distinctions, but each vector costs storage and compute. At 4 bytes per number (32-bit floats), one 1,536-dimension vector takes 6,144 bytes, so a million of them take about 6.1 GB before any index overhead. OpenAI's text-embedding-3 models accept a dimensions parameter that shortens the output; the documentation says a text-embedding-3-large embedding shortened to 256 dimensions still outperforms the older text-embedding-ada-002 at 1,536 dimensions on the MTEB benchmark. If storage or search speed matters, test shorter vectors on your own data before assuming you need the maximum.
Limits to keep in mind
- Similar isn't the same as correct. Embeddings find text that's about the same thing. "The refund window is 30 days" and "The refund window is 14 days" sit very close together, so retrieval can surface an outdated document as confidently as the current one.
- Exact terms can get lost. Product codes, error numbers and rare names may not be captured well by meaning alone. This is why production search systems often combine vector search with keyword search.
- Domain shift. A general-purpose model may place specialised jargon (legal, medical, internal acronyms) poorly. Test retrieval quality on real queries from your users.
- Changing models means re-embedding. Stored vectors are tied to the model that made them.
Frequently asked questions
- Are embeddings the same as tokens?
- No. Tokens are the pieces text is split into before a model reads it, each identified by an integer ID. An embedding is a list of floating-point numbers that represents meaning. Embedding models take tokens as input and produce one vector for a whole word, sentence or document.
- Can I compare embeddings from two different models?
- No. Each model builds its own space, so the numbers from one model mean nothing in another's, even when both have the same number of dimensions. If you switch embedding models, re-embed all your stored documents with the new one.
- Can you turn an embedding back into the original text?
- Not directly: an embedding model has no decoder that outputs the text. But embeddings carry a lot of information about the input, so treat them with the same privacy care as the text they came from rather than as anonymised data.
- Do I need a vector database to use embeddings?
- Not for small collections. A few thousand vectors fit in memory, and a plain loop computing cosine similarity is fast enough. Vector databases or vector indexes in existing databases become useful at hundreds of thousands to millions of vectors, where you need approximate nearest-neighbour search.