Vision on Best of AI

BLIP

Thu, 16 Jul 2026 00:00:00 +0000

BLIP (Bootstrapping Language-Image Pre-training) is a vision-language model that Salesforce Research introduced in early 2022. It handles both understanding tasks, like image captioning and visual question answering, and retrieval tasks, like matching images to text. The model was trained with a bootstrapping method that cleans up noisy web-scraped captions by generating synthetic ones and filtering out the bad matches, which let it reach strong results without relying purely on massive uncurated datasets. At around 223 million parameters, BLIP is small next to later multimodal systems, but it was influential in showing that caption quality matters as much as caption quantity for pretraining.

BLIP-2

Thu, 16 Jul 2026 00:00:00 +0000

BLIP-2 is a vision-language model that Salesforce Research published in January 2023, and it’s built around a specific efficiency trick: instead of training a large image encoder and language model together from scratch, it freezes both a pretrained image encoder and a pretrained language model and trains only a lightweight connector between them, called a Q-Former. That Q-Former has around 188 million trainable parameters, less than 2% of the total model, which made BLIP-2 far cheaper to train than comparable vision-language models of the time while still doing well on image captioning and visual question answering. Salesforce released several variants pairing the Q-Former with different frozen language models, including versions built on OPT (2.7B and 6.7B parameters) and Flan-T5 (up to 11B parameters), so users could trade off capability against compute cost. The code and weights are on GitHub and Hugging Face under the MIT license as part of Salesforce’s LAVIS library, and the frozen-encoder, lightweight-connector approach it popularized influenced later multimodal models that similarly bolt vision capabilities onto existing language models rather than training everything jointly.

CLIP

Thu, 16 Jul 2026 00:00:00 +0000

CLIP (Contrastive Language-Image Pre-training) is a model OpenAI released in January 2021 that learns to match images with text descriptions. It was trained on 400 million image-text pairs scraped from the internet, using a contrastive objective that pulls matching image and text embeddings together while pushing mismatched pairs apart. The largest released variant, built on a ViT-L/14 vision transformer paired with a 63M-parameter text transformer, totals roughly 428 million parameters. Instead of being trained for one narrow task, CLIP learns a general-purpose joint embedding space for images and text, which lets it perform zero-shot image classification by comparing an image against a set of text labels with no task-specific fine-tuning.

CLIP (LAION variant)

Thu, 16 Jul 2026 00:00:00 +0000

This CLIP variant is an open-weight retraining of OpenAI’s original CLIP architecture, produced by LAION and distributed through the open_clip project rather than by OpenAI itself. It uses the ViT-L/14 vision transformer backbone, has about 428 million parameters, and was trained on LAION-2B, the English-language 2 billion image-text pair subset of the larger LAION-5B dataset, using 384 A100 GPUs on the JUWELS supercomputer. Like the original CLIP, it maps images and text into a shared embedding space so you can do zero-shot image classification, image-text retrieval, and similarity search without any task-specific fine-tuning, and it reaches about 75.3 percent zero-shot accuracy on ImageNet-1k. Because the weights are released under an open license and hosted on Hugging Face, it has become a common drop-in replacement for OpenAI’s closed CLIP checkpoints in open-source image search, captioning, and generative art pipelines, including as a text encoder inside several Stable Diffusion variants.

DeepSeek-VL

Thu, 16 Jul 2026 00:00:00 +0000

DeepSeek-VL2

Thu, 16 Jul 2026 00:00:00 +0000

Doubao Vision

Thu, 16 Jul 2026 00:00:00 +0000

Falcon 2

Thu, 16 Jul 2026 00:00:00 +0000

Flamingo

Thu, 16 Jul 2026 00:00:00 +0000

Gemini 3.5 Pro

Thu, 16 Jul 2026 00:00:00 +0000

GPT-4o

Thu, 16 Jul 2026 00:00:00 +0000

Granite Vision

Thu, 16 Jul 2026 00:00:00 +0000

Grok-1.5V

Thu, 16 Jul 2026 00:00:00 +0000

HyperCLOVA X Vision

Thu, 16 Jul 2026 00:00:00 +0000

IDEFICS2

Thu, 16 Jul 2026 00:00:00 +0000

InternVL2

Thu, 16 Jul 2026 00:00:00 +0000

Kimi-VL

Thu, 16 Jul 2026 00:00:00 +0000

Kosmos-2

Thu, 16 Jul 2026 00:00:00 +0000

PaLI

Thu, 16 Jul 2026 00:00:00 +0000

Phi-3-vision

Thu, 16 Jul 2026 00:00:00 +0000

Pixtral 12B

Thu, 16 Jul 2026 00:00:00 +0000

Pixtral Large

Thu, 16 Jul 2026 00:00:00 +0000

Qwen-VL

Thu, 16 Jul 2026 00:00:00 +0000

Qwen2.5-VL

Thu, 16 Jul 2026 00:00:00 +0000

Yi-VL-34B

Thu, 16 Jul 2026 00:00:00 +0000