Training AI models like GPT-3 on “A is B” statements fails to let them deduce “B is A” without further training, exhibiting a flaw in generalization. (https://arxiv.org/pdf/2309.12288v1.pdf)

Ongoing Scaling Trends

  • 10 years of remarkable increases in model scale and performance.

  • Expects next few years will make today’s AI “pale in comparison.”

  • Follows known patterns, not theoretical limits.

No Foreseeable Limits

  • Skeptical of claims certain tasks are beyond large language models.

  • Fine-tuning and training adjustments can unlock new capabilities.

  • At least 3-4 more years of exponential growth expected.

Long-Term Uncertainty

  • Can’t precisely predict post-4-year trajectory.

  • But no evidence yet of diminishing returns limiting progress.

  • Rapid innovation makes it hard to forecast.

TL;DR: Anthropic’s CEO sees no impediments to AI systems continuing to rapidly scale up for at least the next several years, predicting ongoing exponential advances.