Sentence Transformers

Sentence Transformers - State-of-the-Art Embeddings

Python framework for sentence and text embeddings using transformers.

When to use Sentence Transformers

Use when:

• Need high-quality embeddings for RAG
• Semantic similarity and search
• Text clustering and classification
• Multilingual embeddings (100+ languages)
• Running embeddings locally (no API)
• Cost-effective alternative to OpenAI embeddings

Metrics:

• 15,700+ GitHub stars
• 5000+ pre-trained models
• 100+ languages supported
• Based on PyTorch/Transformers

Use alternatives instead:

• OpenAI Embeddings: Need API-based, highest quality
• Instructor: Task-specific instructions
• Cohere Embed: Managed service

Quick start

Installation

pip install sentence-transformers

Basic usage

from sentence_transformers import SentenceTransformer

# Load model
model = SentenceTransformer('all-MiniLM-L6-v2')

# Generate embeddings
sentences = [
    "This is an example sentence",
    "Each sentence is converted to a vector"
]

embeddings = model.encode(sentences)
print(embeddings.shape)  # (2, 384)

# Cosine similarity
from sentence_transformers.util import cos_sim
similarity = cos_sim(embeddings[0], embeddings[1])
print(f"Similarity: {similarity.item():.4f}")

Popular models

General purpose

# Fast, good quality (384 dim)
model = SentenceTransformer('all-MiniLM-L6-v2')

# Better quality (768 dim)
model = SentenceTransformer('all-mpnet-base-v2')

# Best quality (1024 dim, slower)
model = SentenceTransformer('all-roberta-large-v1')

Multilingual

# 50+ languages
model = SentenceTransformer('paraphrase-multilingual-MiniLM-L12-v2')

# 100+ languages
model = SentenceTransformer('paraphrase-multilingual-mpnet-base-v2')

Domain-specific

# Legal domain
model = SentenceTransformer('nlpaueb/legal-bert-base-uncased')

# Scientific papers
model = SentenceTransformer('allenai/specter')

# Code
model = SentenceTransformer('microsoft/codebert-base')

Semantic search

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer('all-MiniLM-L6-v2')

# Corpus
corpus = [
    "Python is a programming language",
    "Machine learning uses algorithms",
    "Neural networks are powerful"
]

# Encode corpus
corpus_embeddings = model.encode(corpus, convert_to_tensor=True)

# Query
query = "What is Python?"
query_embedding = model.encode(query, convert_to_tensor=True)

# Find most similar
hits = util.semantic_search(query_embedding, corpus_embeddings, top_k=3)
print(hits)

Similarity computation

# Cosine similarity
similarity = util.cos_sim(embedding1, embedding2)

# Dot product
similarity = util.dot_score(embedding1, embedding2)

# Pairwise cosine similarity
similarities = util.cos_sim(embeddings, embeddings)

Batch encoding

# Efficient batch processing
sentences = ["sentence 1", "sentence 2", ...] * 1000

embeddings = model.encode(
    sentences,
    batch_size=32,
    show_progress_bar=True,
    convert_to_tensor=False  # or True for PyTorch tensors
)

Fine-tuning

from sentence_transformers import InputExample, losses
from torch.utils.data import DataLoader

# Training data
train_examples = [
    InputExample(texts=['sentence 1', 'sentence 2'], label=0.8),
    InputExample(texts=['sentence 3', 'sentence 4'], label=0.3),
]

train_dataloader = DataLoader(train_examples, batch_size=16)

# Loss function
train_loss = losses.CosineSimilarityLoss(model)

# Train
model.fit(
    train_objectives=[(train_dataloader, train_loss)],
    epochs=10,
    warmup_steps=100
)

# Save
model.save('my-finetuned-model')

LangChain integration

from langchain_community.embeddings import HuggingFaceEmbeddings

embeddings = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-mpnet-base-v2"
)

# Use with vector stores
from langchain_chroma import Chroma

vectorstore = Chroma.from_documents(
    documents=docs,
    embedding=embeddings
)

LlamaIndex integration

from llama_index.embeddings.huggingface import HuggingFaceEmbedding

embed_model = HuggingFaceEmbedding(
    model_name="sentence-transformers/all-mpnet-base-v2"
)

from llama_index.core import Settings
Settings.embed_model = embed_model

# Use in index
index = VectorStoreIndex.from_documents(documents)

Model selection guide

Best practices

1. Start with all-MiniLM-L6-v2 - Good baseline 2. Normalize embeddings - Better for cosine similarity 3. Use GPU if available - 10× faster encoding 4. Batch encoding - More efficient 5. Cache embeddings - Expensive to recompute 6. Fine-tune for domain - Improves quality 7. Test different models - Quality varies by task 8. Monitor memory - Large models need more RAM

Performance

Resources

• GitHub: https://github.com/UKPLab/sentence-transformers ⭐ 15,700+
• Models: https://huggingface.co/sentence-transformers
• Docs: https://www.sbert.net
• License: Apache 2.0

Sentence Transformers Models Guide

Guide to selecting and using sentence-transformers models.

Top recommended models

General purpose

all-MiniLM-L6-v2 (Default recommendation)

• Dimensions: 384
• Speed: ~2000 sentences/sec
• Quality: Good
• Use: Prototyping, general tasks

all-mpnet-base-v2 (Best quality)

• Dimensions: 768
• Speed: ~600 sentences/sec
• Quality: Better
• Use: Production RAG

all-roberta-large-v1 (Highest quality)

• Dimensions: 1024
• Speed: ~300 sentences/sec
• Quality: Best
• Use: When accuracy critical

Multilingual (50+ languages)

paraphrase-multilingual-MiniLM-L12-v2

• Languages: 50+
• Dimensions: 384
• Speed: Fast
• Use: Multilingual semantic search

paraphrase-multilingual-mpnet-base-v2

• Languages: 50+
• Dimensions: 768
• Speed: Medium
• Use: Better multilingual quality

LaBSE (109 languages)

• Languages: 109
• Dimensions: 768
• Speed: Medium
• Use: Maximum language coverage

Domain-specific

allenai/specter (Scientific papers)

• Domain: Academic papers
• Use: Paper similarity, citations

nlpaueb/legal-bert-base-uncased (Legal)

• Domain: Legal documents
• Use: Legal document analysis

microsoft/codebert-base (Code)

• Domain: Source code
• Use: Code similarity, search

Model selection matrix

Performance benchmarks

Speed comparison (CPU)

Quality comparison (STS Benchmark)

Usage examples

Load and use model

from sentence_transformers import SentenceTransformer

# Load model
model = SentenceTransformer('all-mpnet-base-v2')

# Generate embeddings
sentences = ["This is a sentence", "This is another sentence"]
embeddings = model.encode(sentences)

Compare different models

models = {
    'MiniLM': 'all-MiniLM-L6-v2',
    'MPNet': 'all-mpnet-base-v2',
    'RoBERTa': 'all-roberta-large-v1'
}

for name, model_name in models.items():
    model = SentenceTransformer(model_name)
    embeddings = model.encode(["Test sentence"])
    print(f"{name}: {embeddings.shape}")

Resources

• Models: https://huggingface.co/sentence-transformers
• Docs: https://www.sbert.net/docs/pretrained_models.html