Mixture-of-Experts on Best of AI

DBRX

Thu, 16 Jul 2026 00:00:00 +0000

DBRX is Databricks’ open-weight large language model, released in March 2024 as the base model behind the instruction-tuned DBRX Instruct. It is a fine-grained mixture-of-experts model with 132 billion total parameters and 36 billion active parameters per token, trained from scratch on 12 trillion tokens of text and code using Databricks’ own Mosaic AI training stack. At release, Databricks reported it outperforming other open models of the time, including Llama 2 70B and Mixtral, on benchmarks such as MMLU and HumanEval, and it was pitched as evidence that an enterprise-focused company could train a competitive foundation model without relying on a big-lab research budget. It has a 32,000-token context window and is released under the Databricks Open Model License, with weights available on Hugging Face for self-hosting or through Databricks’ own serving infrastructure.

DeepSeek V3

Thu, 16 Jul 2026 00:00:00 +0000

DeepSeek V4-Pro

Thu, 16 Jul 2026 00:00:00 +0000

DeepSeek-MoE-16B

Thu, 16 Jul 2026 00:00:00 +0000

DeepSeek-MoE-16B, released in January 2024, was DeepSeek’s early attempt at a fine-grained mixture-of-experts architecture, and its design choices carried forward into the company’s later V2 and V3 models. It has 16.4 billion total parameters but only about 2.8 billion active per token, achieved through two techniques described in its paper: splitting experts into smaller, more specialized units, and isolating a set of shared experts that always fire to capture common knowledge. Trained from scratch on 2 trillion English and Chinese tokens, it reached performance comparable to the dense DeepSeek 7B and Llama 2 7B models while using only around 40% of the compute those models require at inference. It has a 4,096-token context window and was released under DeepSeek’s open license for self-hosted use, mainly as a research demonstration of expert specialization rather than a production-focused release.

DeepSeek-V2

Thu, 16 Jul 2026 00:00:00 +0000

Hunyuan Large

Thu, 16 Jul 2026 00:00:00 +0000

Hunyuan-A13B

Thu, 16 Jul 2026 00:00:00 +0000

Kimi K2

Thu, 16 Jul 2026 00:00:00 +0000

Kimi K2.6

Thu, 16 Jul 2026 00:00:00 +0000

Kimi K2.7 Code

Thu, 16 Jul 2026 00:00:00 +0000

Llama 4 Maverick

Thu, 16 Jul 2026 00:00:00 +0000

MiniMax-01

Thu, 16 Jul 2026 00:00:00 +0000

Mixtral 8x22B

Thu, 16 Jul 2026 00:00:00 +0000

Mixtral 8x7B

Thu, 16 Jul 2026 00:00:00 +0000

OLMoE

Thu, 16 Jul 2026 00:00:00 +0000

PanGu-Sigma

Thu, 16 Jul 2026 00:00:00 +0000

Qwen2.5-Max

Thu, 16 Jul 2026 00:00:00 +0000

Qwen3-235B-A22B

Thu, 16 Jul 2026 00:00:00 +0000

Qwen3-Coder

Thu, 16 Jul 2026 00:00:00 +0000

Samba-1

Thu, 16 Jul 2026 00:00:00 +0000

Zephyr 141B

Thu, 16 Jul 2026 00:00:00 +0000