GPT-2

A new essay argues that the ingredients behind GPT-2 — transformers aside, large language models might have been feasible nearly two decades earlier — prompting debate over what actually held AI back: compute, data, or ideas. Readers are weighing whether progress was gated by hardware costs and the lack of internet-scale text corpora, or whether researchers simply lacked the imagination to scale simple methods sooner.