Embedding on Best of AI

Arctic Embed 2.0

Thu, 16 Jul 2026 00:00:00 +0000

BERT

Thu, 16 Jul 2026 00:00:00 +0000

BERT (Bidirectional Encoder Representations from Transformers) is a language model that Google Research published in October 2018, and it changed how the field approached pretraining for NLP tasks. Unlike earlier models that read text left to right, BERT trains on masked-word prediction and next-sentence prediction, letting it build representations that draw on context from both directions at once. Google released two main sizes: BERT-base at 110 million parameters and BERT-large at 340 million, both with a 512-token limit, and made the weights freely available under Apache 2.0 on GitHub and later Hugging Face. Fine-tuned versions of BERT set new records on eleven different NLP benchmarks at launch, including question answering and natural language inference, and it quickly became the standard base model to fine-tune for tasks like classification, named entity recognition, and search ranking. It’s mostly been superseded by newer encoder models like RoBERTa and DeBERTa and by the embedding models that followed it, but it’s still one of the most cited papers in NLP and its architecture underlies a large share of later encoder-based systems.

BGE-large-en

Thu, 16 Jul 2026 00:00:00 +0000

BGE-large-en-v1.5 is a text embedding model from BAAI, the Beijing Academy of Artificial Intelligence, released in September 2023 as an update to the original BGE-large-en model from a few weeks earlier. It has 335 million parameters, produces 1024-dimension embeddings, and handles sequences up to 512 tokens, putting it in the same size class as other BERT-scale encoder models rather than the larger decoder-based embedding models that came later. The v1.5 update mainly fixed a similarity-score distribution issue in the original release and improved retrieval quality when used without a task-specific instruction prefix, which had been a rough edge in v1. BAAI released it under the MIT license, and it became one of the most widely used open embedding models for English retrieval and RAG pipelines, largely because it was free, fast to run, and scored well on the MTEB benchmark relative to its size. It’s since been joined by larger and more capable BAAI models like BGE-M3, but it remains a common default for lightweight self-hosted retrieval.

BGE-M3

Thu, 16 Jul 2026 00:00:00 +0000

BGE-M3 is a text embedding model from BAAI, described in a paper published in February 2024, and the name refers to three things it does at once: multi-linguality across more than 100 languages, multi-granularity handling of inputs up to 8192 tokens, and multi-functionality in that it produces dense vectors, sparse lexical weights, and ColBERT-style token vectors from a single forward pass. It has 568 million parameters, which is modest next to large LLMs but is trained specifically for retrieval rather than generation. That combination let it post state-of-the-art results at the time on the multilingual MIRACL benchmark and the cross-lingual MKQA benchmark, since a single model could substitute for what previously required separate dense and sparse retrieval systems. BAAI released it under the MIT license on Hugging Face, and it’s widely used in retrieval-augmented generation pipelines that need to search across documents in multiple languages or documents longer than the 512-token limit typical of earlier BERT-based embedding models.

CLIP (LAION variant)

Thu, 16 Jul 2026 00:00:00 +0000

This CLIP variant is an open-weight retraining of OpenAI’s original CLIP architecture, produced by LAION and distributed through the open_clip project rather than by OpenAI itself. It uses the ViT-L/14 vision transformer backbone, has about 428 million parameters, and was trained on LAION-2B, the English-language 2 billion image-text pair subset of the larger LAION-5B dataset, using 384 A100 GPUs on the JUWELS supercomputer. Like the original CLIP, it maps images and text into a shared embedding space so you can do zero-shot image classification, image-text retrieval, and similarity search without any task-specific fine-tuning, and it reaches about 75.3 percent zero-shot accuracy on ImageNet-1k. Because the weights are released under an open license and hosted on Hugging Face, it has become a common drop-in replacement for OpenAI’s closed CLIP checkpoints in open-source image search, captioning, and generative art pipelines, including as a text encoder inside several Stable Diffusion variants.

Cohere Embed Multilingual v3

Thu, 16 Jul 2026 00:00:00 +0000

Cohere Embed Multilingual v3 is a text embedding model Cohere released in November 2023 for search, retrieval-augmented generation, and semantic similarity tasks across more than 100 languages. It takes input text up to 512 tokens and outputs a 1024-dimension vector by default, with support for shorter 384 or 768 dimension embeddings when a team wants faster search or lower storage cost. Cohere trained it alongside an English-only sibling, Embed English v3, and pitched both as an upgrade over v2 because they weight passages by how relevant they are likely to be, not just how similar they are, which cuts down on noisy retrieval results in RAG pipelines. It’s priced at $0.10 per million tokens through the Cohere API and is also distributed through AWS, Azure, and Oracle Cloud marketplaces. Cohere has since released newer Embed v4 models, and some cloud providers such as Oracle now list v3 as a legacy or deprecated option, though it remains available through Cohere’s own API and Hugging Face.

Cohere Embed v3

Thu, 16 Jul 2026 00:00:00 +0000

Cohere Embed v3 is a text embedding model that Cohere released in November 2023 for search, retrieval-augmented generation, and semantic search applications. It comes in English and multilingual variants, and the multilingual version was trained on over 100 languages and improved retrieval accuracy on the MIRACL benchmark by roughly 35% over the previous generation. The model lets a caller specify whether a text is a search query or a document, which adjusts the embedding space to improve retrieval accuracy. It’s available only through Cohere’s API, priced at $0.10 per million input tokens, and accepts inputs up to 512 tokens. Embed v3 became a standard choice for RAG pipelines alongside OpenAI’s text-embedding-3 and Voyage AI’s embedding models, before Cohere introduced the multimodal Embed v4 in 2025.

Cohere Embed v4

Thu, 16 Jul 2026 00:00:00 +0000

Cohere Embed v4 is Cohere’s multimodal embedding model, released in April 2025 as the successor to Embed v3. Unlike its predecessor, it accepts text, images, and mixed documents in a single call and produces a unified vector representation, so a screenshot of a PDF page, a slide, or a table can be indexed directly alongside plain text without first converting it to text. It supports a 128,000 token context window, well beyond OpenAI’s and Voyage’s embedding models, and offers Matryoshka embeddings at 256, 512, 1024, or 1536 dimensions so users can trade storage against accuracy. Pricing is $0.12 per million text tokens and $0.47 per million image tokens through the Cohere API, and the model is also available on Amazon Bedrock and Azure AI. Cohere positions it for enterprise document search and RAG systems that need to handle scanned or visually formatted content, not just plain text.

Gemini Embedding

Thu, 16 Jul 2026 00:00:00 +0000

Granite Embedding

Thu, 16 Jul 2026 00:00:00 +0000

GTE-large

Thu, 16 Jul 2026 00:00:00 +0000

Jina Embeddings v2

Thu, 16 Jul 2026 00:00:00 +0000

jina-embeddings-v3

Thu, 16 Jul 2026 00:00:00 +0000

Mistral Embed

Thu, 16 Jul 2026 00:00:00 +0000

Nomic Embed Text v1

Thu, 16 Jul 2026 00:00:00 +0000

nomic-embed-text-v2

Thu, 16 Jul 2026 00:00:00 +0000

Text-Embedding-004

Thu, 16 Jul 2026 00:00:00 +0000

text-embedding-3-large

Thu, 16 Jul 2026 00:00:00 +0000

text-embedding-3-small

Thu, 16 Jul 2026 00:00:00 +0000

text-embedding-ada-002

Thu, 16 Jul 2026 00:00:00 +0000

Titan Multimodal Embeddings

Thu, 16 Jul 2026 00:00:00 +0000

Titan Text Embeddings V2

Thu, 16 Jul 2026 00:00:00 +0000

Voyage-2

Thu, 16 Jul 2026 00:00:00 +0000

voyage-3-large

Thu, 16 Jul 2026 00:00:00 +0000

Voyage-Code-2

Thu, 16 Jul 2026 00:00:00 +0000