Llemma 7B
EleutherAI's open model continually pretrained on math and science text
Llemma 7B is a math-focused language model from EleutherAI, released in October 2023 by continuing pretraining on top of Code Llama 7B rather than starting from scratch. EleutherAI trained it further on Proof-Pile-2, a 200-billion-token mix of math papers, formal proofs, and math-heavy web text, so it specializes in chain-of-thought mathematical reasoning and can call out to tools like Python or formal theorem provers when solving problems. It comes in 7B and 34B sizes, and both were among the strongest openly available math models at the time of release. Because it inherits its base weights from Code Llama, it carries the Llama 2 Community License rather than a fully permissive one, which limits some commercial uses compared to licenses like Apache 2.0.