Mistral 7B
Mistral's original compact open-weight model that outperformed larger rivals
Mistral 7B is the model that put Mistral AI on the map, released on September 27, 2023 as the French startup’s first public model. At 7 billion parameters it was small enough to run on a single consumer GPU, yet it beat Llama 2 13B on every benchmark Mistral tested and even outperformed Llama 2 34B on several reasoning and code tasks, scoring 60.1% on MMLU and 30.5% on HumanEval. It uses grouped-query attention and sliding-window attention to keep inference fast and memory-efficient at its 32,000-token context length.
Mistral released it under an Apache 2.0 license with no usage restrictions, which was unusual for a model of its quality at the time and helped it spread quickly through the open-source community via Hugging Face and tools like Ollama and llama.cpp. It became a common base for fine-tunes and merges, and its success established Mistral AI as a serious competitor to Meta and OpenAI within months of the company’s founding.