InternLM2

InternLM2 is an open-weight language model from the Shanghai AI Laboratory, released January 17, 2024 in 7B and 20B sizes alongside matching chat variants. Both sizes support a 200,000-token context window, reached through a training process that starts on 4K-token sequences and extends gradually to 32K and beyond, and the model holds up well on long-document needle-in-a-haystack tests at that length. The 20B chat variant scores 67.7 on MMLU, putting it ahead of several open models of similar size at the time of release. InternLM2 was trained on a large multilingual corpus with particular strength in English and Chinese, and it was positioned as a general-purpose foundation model for reasoning, coding, and long-context tasks before Shanghai AI Lab replaced it with InternLM2.5 later that year. It’s released under the Apache 2.0 license and distributed through Hugging Face and the InternLM GitHub repository.