GPT-3
The 2020 model that first showed large-scale few-shot learning was possible
GPT-3 is the language model OpenAI described in its May 2020 paper “Language Models are Few-Shot Learners” and opened up through a beta API on June 11, 2020. At 175 billion parameters, it was an order of magnitude larger than GPT-2, and its headline result was that a single model, given only a handful of examples in its prompt, could perform tasks like translation, arithmetic, and question answering without any task-specific fine-tuning. The context window was small by later standards, around 2,048 tokens.
The paper’s few-shot learning results are what made GPT-3 a turning point rather than just a bigger version of GPT-2. It showed that scale alone, without architectural changes, produced qualitatively new capabilities, and it set off the race among labs to train ever-larger models. GPT-3 itself was API-only and never had its weights released, and OpenAI eventually retired the original model in favor of its instruction-tuned successors.