DeepSeek-R1-Distill-Llama-70B
A Llama-based distillation of DeepSeek-R1's reasoning traces, the largest distilled variant
DeepSeek-R1-Distill-Llama-70B is a distilled reasoning model DeepSeek released alongside DeepSeek-R1 in January 2025. Rather than training a reasoning model from scratch, DeepSeek fine-tuned Meta’s Llama 3.3 70B Instruct on reasoning traces generated by the full R1 model, transferring much of R1’s chain-of-thought problem-solving ability into a smaller, cheaper-to-run architecture. It is the largest of six distilled R1 variants DeepSeek published, spanning Qwen and Llama base models from 1.5B up to 70B parameters, and on release it scored 70.0% pass@1 on AIME 2024 and 94.5% on MATH-500, far ahead of general-purpose models like GPT-4o and Claude 3.5 Sonnet on the same math benchmarks. It carries a 128,000-token context window, inherited from its Llama 3.3 base, and is distributed under the Llama 3.3 Community License, with weights on Hugging Face and hosted API access through providers including Together AI.