← Home
Whittemore examines the growing debate over whether scaling laws for large language models are plateauing. Reports from multiple labs suggest diminishing returns from simply making models bigger.
Highlights
•
The 'scaling wall' narrative may be premature — labs have historically found new scaling axes when old ones plateau
Whittemore reviews previous scaling plateaus: compute scaling plateaued, then data scaling emerged. Data scaling plateaued, then RLHF improved quality without more data. Each apparent wall was actually a transition to a new axis.•
Even if scaling slows, the integration of current capabilities into the economy has barely begun
Whittemore: the debate about future model capabilities distracts from the fact that current models are massively underutilized. Most industries have barely begun integrating GPT-4-class capabilities.