Text becomes vectors, and meaning becomes distance.
Turn text into vectors. The retrieval half of a RAG system.
Browse live models for this task →An embedding model turns a piece of text into a list of numbers positioned so that similar meanings sit close together. That one property underpins semantic search, deduplication, clustering, entity matching and the retrieval half of every RAG system.
The architectural point worth learning properly is the two-stage pattern. A bi-encoder embeds documents once, ahead of time, so search is just a nearest-neighbour lookup — fast over millions of records but somewhat imprecise. A cross-encoder reads the query and document together, which is far more accurate but requires a forward pass per pair, so it cannot scale. You use the first to get from a million candidates to fifty, then the second to order those fifty.
Sub-categories within this task. The distinction that matters is not which is best, but which fits your constraints.
sentence-transformers/all-MiniLM-L6-v2BAAI/bge-large-en-v1.5intfloat/e5-large-v2
cross-encoder/ms-marco-MiniLM-L6-v2BAAI/bge-reranker-v2-m3
BM25 (rank_bm25, Elasticsearch, OpenSearch)
BAAI/bge-m3intfloat/multilingual-e5-large
The third line in each is the part that makes it harder than the textbook version.
Staff ask a question in their own words and get the relevant clause from thousands of pages of internal policy.
The wrinklePolicy documents are full of near-duplicate text across versions. Retrieval happily returns the superseded version. Effective-date filtering has to be part of the retrieval query, not an afterthought.
Decide whether 'Northwood Cap. Ltd', 'Northwood Capital Limited' and 'NORTHWOOD CAPITAL' are one counterparty.
The wrinkleSemantic embeddings alone are poor at this — they will happily match 'Northwood Capital' to 'Southwood Capital' because the meanings are similar. Combine with edit distance and phonetic matching; this is a hybrid problem.
Identify when many complaints describe one underlying systemic issue.
The wrinkleChoosing the similarity threshold is the whole job, and there is no principled way to set it without labelled pairs. Budget time for tuning it against human judgement.
Retrieve the relevant passages, then have an LLM answer grounded in them.
The wrinkleRetrieval failure is the dominant cause of bad answers, not the language model. Evaluate recall@k on the retriever separately before blaming the generator.
Named functions and the arguments that actually change the outcome. Comments flag the traps.
normalize_embeddings=True means dot product equals cosine similarity, which is what almost every vector store expects.
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer("BAAI/bge-large-en-v1.5")
docs = ["Overdraft fees are waived for balances under 50 GBP for 3 days.",
"Card replacement takes 5 working days by standard post.",
"Disputed transactions must be raised within 60 days."]
# Encode the corpus once; persist this
doc_emb = model.encode(docs, normalize_embeddings=True,
convert_to_tensor=True, batch_size=32)
# Some models (e5, bge) expect an instruction prefix on QUERIES only.
# Omitting it costs real accuracy - check the model card.
query = "how long do I have to dispute a payment"
q_emb = model.encode(query, normalize_embeddings=True, convert_to_tensor=True)
for hit in util.semantic_search(q_emb, doc_emb, top_k=3)[0]:
print(f"{hit['score']:.3f} {docs[hit['corpus_id']]}")This is the single highest-value structure in retrieval. Bi-encoder for recall, cross-encoder for precision.
from sentence_transformers import SentenceTransformer, CrossEncoder, util
bi = SentenceTransformer("BAAI/bge-large-en-v1.5")
cross = CrossEncoder("BAAI/bge-reranker-v2-m3")
# Stage 1 - cheap, over everything. Cast wide: 50, not 5.
q_emb = bi.encode(query, normalize_embeddings=True, convert_to_tensor=True)
hits = util.semantic_search(q_emb, doc_emb, top_k=50)[0]
shortlist = [docs[h["corpus_id"]] for h in hits]
# Stage 2 - expensive, over 50 only
scores = cross.predict([(query, d) for d in shortlist])
ranked = sorted(zip(scores, shortlist), reverse=True)
for s, d in ranked[:5]:
print(f"{s:.3f} {d[:80]}")Reciprocal rank fusion combines a lexical and a semantic ranking without needing the two score scales to be comparable — which they are not.
from rank_bm25 import BM25Okapi
bm25 = BM25Okapi([d.lower().split() for d in docs])
def rrf(query, k=60, top_n=10):
"""Reciprocal rank fusion of lexical and semantic rankings."""
lex = sorted(range(len(docs)),
key=lambda i: -bm25.get_scores(query.lower().split())[i])
q = bi.encode(query, normalize_embeddings=True, convert_to_tensor=True)
sem = [h["corpus_id"] for h in
util.semantic_search(q, doc_emb, top_k=len(docs))[0]]
fused = {}
for rank, i in enumerate(lex):
fused[i] = fused.get(i, 0) + 1 / (k + rank)
for rank, i in enumerate(sem):
fused[i] = fused.get(i, 0) + 1 / (k + rank)
return sorted(fused, key=fused.get, reverse=True)[:top_n]Embed a whole 40-page policy and the vector means nothing in particular. Embed single sentences and you lose context. Paragraph-sized chunks with overlap, respecting document structure, is the usual compromise — and it affects results more than which model you pick.
The e5 and bge families were trained with asymmetric prefixes such as 'query: ' and 'passage: '. Skipping them degrades results measurably. Read the model card rather than assuming the interface is uniform.
0.82 is not 'similar' in any transferable sense — the scale differs per model and per domain. Thresholds must be calibrated on your own labelled pairs, and re-calibrated if you change model.
Vectors from different models are not comparable. Swapping embedding models invalidates the entire index, which for a large corpus is a real migration with a real cost. Choose deliberately at the start.