Sep 24, 2026
Nano Banana Pro prompted by THE DECODER
How fast is AI improving? That question usually goes to experts at top universities, heavily cited AI researchers, and seasoned economists.
Yet these same specialists significantly underestimated recent progress on benchmarks and some adoption metrics, according to an interim report from the Forecasting Research Institute (FRI).
Since mid-2022, FRI has collected forecasts on AI progress across several studies and projects. Its samples include senior specialists. The first round of LEAP (Longitudinal Expert AI Panel) drew 339 experts, including 76 computer scientists, 76 industry experts, 68 economists, and 119 AI policy specialists. The computer scientists included 30 professors at top-20 institutions and 10 of the 200 most-cited AI authors. The panels also included superforecasters, generalists with a proven record of accurate predictions.
AI hit major milestones years ahead of forecasts
The widest gap involves math. AI reached gold-medal level at the International Mathematical Olympiad in July 2025, five years before the median expert forecast and ten years before the median superforecaster forecast. Those predictions were gathered in 2022, before ChatGPT launched, but the pattern held afterward too, according to FRI.
Based on their forecasts, experts assigned an average probability of 24.6 percent to the benchmark results that actually happened, while superforecasters assigned just 9.7 percent. For gold-medal performance at the Math Olympiad, the figures dropped to 8.6 and 2.3 percent. | Image: Forecasting Research Institute
AI may also have solved a Millennium Prize Problem, though it's still unclear whether the solution meets the evaluation criteria. In a survey from August and September 2025, experts had put the median odds of such a solution by the end of 2027 at just 10 percent, and superforecasters at 5.4 percent.
In a study of AI capabilities in virology, experts predicted AI models wouldn't match a top team of virologists on a troubleshooting benchmark until 2030. Superforecasters said 2034. FRI says that likely happened as early as April 2025. A cybersecurity benchmark showed similar underestimates.
At the median, experts expected AI to match a top team on the Virology Capabilities Test by 2030, and superforecasters by 2034. FRI says it likely happened in April 2025. Respondents tied this milestone to higher expected biorisk, not to any documented rise in actual harm. | Image: Forecasting Research Institute
Economic forecasts were also far too conservative. Experts put the median for the highest annual recurring revenue (ARR) of any AI company at the end of 2026 at $20 billion. Economists said $16 billion, and superforecasters said $25 billion. FRI cites roughly $100 billion for Anthropic in September 2026 as a figure that has likely already been reached.
Annualized revenue at Anthropic and OpenAI (left) far exceeded the median forecasts of every surveyed group (right). Even the highest group estimate of $25 billion fell well short of the $65 billion reported in July 2026. | Image: Forecasting Research Institute
Real-world impact is harder to call
Not every forecast ran too low. Biosecurity experts predicted that 22.5 percent of participants using a language model would complete biological lab tasks. Virologists expected 40 percent, superforecasters 16.2 percent. In a controlled trial, only 5.2 percent succeeded with a language model and internet access, compared with 6.6 percent using the internet alone. The language model made no measurable difference, though the trial was small.
Experts may also have overshot on self-driving cars. Their median forecast for the share of autonomous US ride-hailing trips in 2027 was 7.3 percent, while an LLM projection puts it at 2.5 percent. FRI says forecasts on economic growth, employment, and major AI harms can't be reliably judged yet.
At the same time, respondents are revising their expectations upward. Among those who completed both surveys, the average probability assigned to AI becoming a "technology of the century" rose from 31 to 36 percent for experts and from 28 to 35 percent for superforecasters over nine months.
Experts and superforecasters now expect bigger societal effects from AI than they did nine months earlier. Both groups assign their highest average probability to "technology of the century," on par with electricity. | Image: Forecasting Research Institute
FRI is adding faster methods to keep pace with AI
Going forward, FRI will highlight a subsample of respondents who expect very rapid AI progress through 2040 and publish continuously updated LLM forecasts alongside the human ones. According to ForecastBench, some models already match superforecasters on certain question types. FRI also wants to find the most accurate LEAP panelists and feature their forecasts once enough data is in.
RI does flag a catch in its own data: underestimates become obvious as soon as reality overtakes a prediction, but overestimates only become clear once a deadline passes. That makes the interim report naturally tilted toward finding cases where forecasters were too cautious. Some of FRI's own assessments also rely on LLM projections that use information the original forecasters didn't have.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Read on for the full picture.
Subscribe for hype-free coverage.
-
Full access to every article on THE DECODER
-
No ads
-
Join the comments and community discussions
-
A weekly AI news recap via mail
-
6x/year: "AI Radar" — deep dives on the AI topics that matter most
-
Daily AI news, always up to date
-
Our full ten-year archive
-
Covered by a team with 10+ years in AI