What we know

A new report from the AI evaluation nonprofit METR revises a widely referenced graph that tracks AI progress. The update reveals that while some AI capabilities continue to improve at an exponential rate, others are advancing more slowly than previously believed. The report also clarifies that AI models are not as superhuman as some earlier interpretations of the graph suggested. Certain AI capabilities maintain exponential improvement trajectories. The updated graph shows uneven progress across different AI skills, with some advancing more gradually. AI models have not yet reached the superhuman performance levels implied by earlier versions of the graph.

Why it matters

This graph has been called the "most misunderstood graph in AI" because it is frequently cited to illustrate rapid AI progress and inform debates about AI risks and timelines for advanced AI development. Its previous interpretations often oversimplified the pace and scope of AI improvements. METR’s updated analysis offers a more nuanced perspective, showing that while some benchmarks demonstrate fast growth, others are progressing more slowly or plateauing.

What is still unknown

The report does not provide independent verification of the updated graph’s accuracy. Details about supported languages, broader release timelines, or how these findings translate into real-world AI applications remain unclear. No independent tests of the updated evaluation have been reported.