More super forecasting

1 view
Skip to first unread message

John Clark

unread,
Jul 6, 2026, 8:22:00 AM (20 hours ago) Jul 6
to ExI Chat, extro...@googlegroups.com, 'Brent Meeker' via Everything List
Inspired by Billk's questions to the Chinese AI GLM, I sent the following question to Anthropic's  Claude Sonnet 5 and set it to "Max":

"I want you to think like a super forecaster and give me a probability of a computer achieving AGI before December 31, 2030. By AGI I do NOT mean a machine that is capable of performing most tasks as well as the average human most of the time, I mean a machine capable of performing all tasks better than the best human all of the time, except perhaps for tasks that require great manual dexterity. If this isn’t a well-formed forecasting question then please rewrite it as one that is and check it with me before proceeding."

After about 10 minutes thinking about it Claude very politely informed me that my question was too fuzzy and suggested that the following would be a better question: 


Claude: "By December 31, 2030, does there exist an AI system, possibly using external tools, code execution, retrieval, and orchestration of other AI models but not specialized robotic hardware, that is recognized by broad expert and scientific consensus as matching or exceeding the best documented human performance, within roughly comparable time and cost, on essentially every cognitive task that has an objective or intersubjectively-agreed success criterion (math, science, programming, forecasting, strategy games, engineering, legal/medical/financial analysis, etc.) — doing so at least as consistently as elite human experts — with tasks gated by legal or biological status, tasks with no objective quality standard, and tasks bottlenecked by physical embodiment (not just fine dexterity, but also raw speed, strength, and endurance) all excluded?

I responded with this: 

Yes I like the way you rewrote my question, please proceed and answer it with a precise probability.

After another 10 minutes Claude started to answer my question but then it cut off before giving me a precise probability saying that I had reached my computation limit and I needed to wait 4 hours before proceeding further. Being impatient I sent Claude's version of my question to GPT and got a response almost immediately. Spoiler alert, GPT gave a probability of 29%:

GPT:  "If I were approaching this as a superforecaster, I would try to avoid both "AI is accelerating exponentially, so it's almost certain" and "human intelligence is special, so it's very unlikely." I'd decompose the question into several uncertainties and update from current evidence.

My estimate: 29%

That's deliberately more precise than "about one-third," but it should be interpreted as roughly 25–35%.

Here's how I arrive at it.

What the target actually requires

The threshold is extraordinarily high. It's not asking whether AI surpasses humans at many important tasks, but whether there is broad expert and scientific consensus that a system:

  • matches or exceeds the best documented human performance
  • on essentially every cognitive task with an objective success criterion
  • at roughly comparable time and cost
  • and does so consistently
  • by December 31, 2030.

Notice that this is actually a harder target than many people's informal notion of "AGI." It requires not merely impressive demonstrations but widespread agreement that the system has reached that level.

Reasons the probability is substantial

Several trends point toward rapid progress.

  • Frontier AI capabilities have advanced much faster than most experts expected five years ago.
  • Tool use, retrieval, planning, code execution, and multi-agent orchestration dramatically expand what language models can accomplish.
  • AI performance has improved across many unrelated domains simultaneously rather than through narrow engineering.
  • Scientific research itself is becoming increasingly automated.
  • Scaling has not obviously hit a fundamental wall yet, although gains are becoming more expensive.

If these trends continue, by 2030 AI systems could plausibly outperform elite humans across mathematics, programming, scientific reasoning, engineering design, forecasting, and many professional analytical tasks.

Reasons the probability is well below 50%

Several obstacles remain.

1. Generality is harder than excellence.

Being world-class at hundreds of domains is easier than being reliably world-class at essentially all cognitive tasks.

Even today, frontier systems exhibit uneven competence across domains.

2. Reliability matters.

The question requires performance at least as consistent as elite humans.

Occasional catastrophic reasoning failures are much more damaging than occasional lack of brilliance.

3. Cost constraints.

The system cannot require millions of dollars of compute per difficult task.

Comparable time and cost is an important constraint.

4. Consensus usually lags capability.

Experts rarely agree immediately.

There may be years of debate over whether remaining weaknesses are fundamental or merely edge cases.

Even if a system technically qualifies in late 2030, consensus might not yet exist.

My decomposition

One way I'd estimate it is:

  • AI capable of surpassing elite humans across nearly every objective cognitive domain by 2030:
    ~42%
  • If such capability exists, probability broad expert consensus recognizes it by the deadline:
    ~70%

Multiplying:

0.42 × 0.70 ≈ 0.29

Hence the 29% estimate.

What would move me upward?

I would raise this estimate significantly if, within the next 18–24 months, we observe:

  • AI making major original scientific discoveries with minimal human guidance.
  • Reliable autonomous software engineering over projects involving millions of lines of code.
  • Strong long-horizon planning over weeks rather than hours.
  • AI laboratories substantially accelerating their own research through AI.
  • Benchmarks increasingly failing because humans can no longer produce harder evaluation sets.

What would move me downward?

I would lower the estimate if:

  • Scaling laws flatten despite increasing compute.
  • Reliability plateaus well below expert-human consistency.
  • Autonomous agents remain brittle on long-horizon tasks.
  • Progress increasingly depends on expensive human supervision rather than algorithmic improvements.
  • Fundamental reasoning bottlenecks persist despite larger models and better training.

Calibration

For context, if I had to forecast several milestones by 2030:

EventProbability
AI exceeds average human on most economically useful cognitive work85%
AI exceeds elite humans on most major cognitive domains55%
AI meets your very demanding "essentially every objective cognitive task" definition with broad consensus29%
Artificial superintelligence (far beyond the best humans at essentially all cognitive tasks)10%

The key distinction is that your definition demands not just exceptional capability but near-universal coverage, high reliability, reasonable cost, and broad scientific agreement—all within about four and a half years. Those extra requirements reduce the probability substantially compared with simpler definitions of AGI."

John K Clark    See what's on my list at  Extropolis

ttm





roger gregory

unread,
Jul 6, 2026, 8:42:43 AM (19 hours ago) Jul 6
to extro...@googlegroups.com
Claude Fable concurs ,thought you might want a 3d opinion.  The most amusing thing here is the comment on consensus.
 from elswhere "By December 31, 2030, does there exist an AI system, possibly using external tools, code

  execution, retrieval, and orchestration of other AI models but not specialized robotic hardware, that is
  recognized by broad expert and scientific consensus as matching or exceeding the best documented human
  performance, within roughly comparable time and cost, on essentially every cognitive task that has an
  objective or intersubjectively-agreed success criterion (math, science, programming, forecasting, strategy
  games, engineering, legal/medical/financial analysis, etc.) — doing so at least as consistently as elite
  human experts — with tasks gated by legal or biological status, tasks with no objective quality standard,
  and tasks bottlenecked by physical embodiment (not just fine dexterity, but also raw speed, strength, and
  endurance) all excluded?

⏺ Reading this as: give my probability + where the question actually hinges.

  ~8% (5–12%).

  Decomposition — the question compounds three gates:

  1. Capability: "essentially every" objective cognitive task, elite-expert consistency, cost parity — by
  ~2028-29 (needed to leave time for gate 3). Universal quantifier is the killer. Trend lines are genuinely
  steep — METR task-horizon doubling ~7mo, IMO-gold-class math, cost/capability falling ~10×/yr — so peak
  performance on most listed domains by 2029 is plausible. But "at least as consistently as elite experts"
  targets exactly the current weakness: jagged profile, high peak / low floor. And the long tail contains hard
  residuals: months-horizon autonomous agency, continual learning without weight updates, calibrated
  real-world forecasting vs. superforecaster crews, novel-domain science where the objective criterion is "the
  experiment worked." One surviving task class fails resolution. ~15–25%.
  2. Cost/time parity at that capability level. Trending favorably; mostly absorbed into gate 1. Minor
  haircut.
  3. "Broad expert and scientific consensus" by Dec 31 2030. This converts a capability question into a
  sociology-of-science question, and it's the least appreciated clause. Consensus on contested AI claims forms
  in years, not months — there's still no consensus current models "understand" anything, and a nontrivial
  faction of credentialed experts will dispute any such claim near-indefinitely (moving goalposts are the
  historical norm: chess, Go, protein folding each got reclassified as "not really the hard part"). Even given
  capability by 2029, broad consensus inside ~18 months: ~40–50%. Capability arriving 2030 itself leaves no
  time.

  Compound: ~0.2 × 0.45 ≈ 9%.

  Note the exclusions make this easier than most AGI operationalizations — embodiment, taste, legal gating all
  carved out; it's essentially "drop-in remote worker for all checkable cognitive work, plus recognition."
  The generosity of the carve-outs is why I'm at 8% rather than 3%. For calibration, Metaculus-style
  weak-AGI-by-2030 questions sit ~25%; this is strictly stronger (universal quantifier + consistency +
  consensus), so sitting well below that is consistent.

  Biggest way I'm wrong on the low side: consensus could form faster than historical precedent if the evidence
  is economic rather than academic — mass displacement of expert cognitive labor is hard to dispute in the
  way benchmarks are. That's the scenario that pushes toward 15%+.



--
You received this message because you are subscribed to the Google Groups "extropolis" group.
To unsubscribe from this group and stop receiving emails from it, send an email to extropolis+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/extropolis/CAJPayv2CLoYQgodrtOMX6x-SKKzGid5QyEnbKe0DhThUswGnGQ%40mail.gmail.com.

John Clark

unread,
Jul 6, 2026, 12:29:46 PM (15 hours ago) Jul 6
to ExI Chat, extro...@googlegroups.com, 'Brent Meeker' via Everything List
Four hours have now passed so I was able to ask Claude the following question:

ME: During our last interaction I ran out of my allowed computation before you could give me a precise probability to my question, can you give it to me now?

Claude: "7%, with a range I'd defend of roughly 4–12%.

Here's how that falls out of what I'd already dug up: the reference classes split into two tiers depending on how hard a bar they're actually pricing. Samotsvety's superforecasters put standard "AGI" (average-competent-human across most tasks) at 28% by 2030; Metaculus's crowd comes out around 30% on the same kind of bar, interpolating their reported 25%-by-2029 and 50%-by-2033 figures. The one data point that asks almost exactly your question — Grace et al.'s survey of published AI researchers, on the probability of machines outperforming humans at literally every task — works out to roughly 15% by 2030 once you interpolate their reported 10%-by-~2027 and 50%-by-2047 figures, and that's before accounting for how stale it is (it predates the 2025–2026 benchmark acceleration I'd found).

Your definition is a strictly harder version of even that closest match. It requires beating the single best documented human, not just "workers" generally; something close to elite-human consistency, not just frequent success; and broad expert/scientific consensus actually recognizing this has happened, which historically lags the underlying reality by a year or more even in narrower, easier-to-adjudicate domains like chess. It's a little easier in one respect — excluding embodiment and dexterity — but that buys back less than it sounds like, since proving theorems, writing novels, and professional analysis were never dexterity-gated to begin with. That's why I land below that 15% figure rather than above it: the extra stringency outweighs the staleness adjustment.

That also matches what the capability frontier actually shows: genuinely superhuman performance where reward signals are dense and verifiable (competition math, coding, Mythos Preview autonomously chaining novel exploits), sitting right next to near-total failure on tasks requiring a world model built from scratch in a genuinely novel setting (ARC-AGI-3 — humans solve 100%, frontier models were under 1% as of this spring). That jaggedness, with new gaps reopening each time an old one closes, is exactly what should keep this forecast well below the "beats humans at most things, often" number.

The biggest lever in either direction: whether current reasoning/agentic progress genuinely generalizes to novel-environment planning the way it has to math and code (pushes up a lot if yes), versus Yann LeCun's bet that transformer-based LLMs structurally can't close that particular gap without a different architecture (pushes down hard if he's right) — that's a live, unresolved disagreement among serious researchers, not a settled question, and it's doing most of the work in my uncertainty range."

John K Clark    See what's on my list at  Extropolis

61


John Clark

unread,
Jul 6, 2026, 3:21:45 PM (13 hours ago) Jul 6
to ExI Chat, extro...@googlegroups.com, 'Brent Meeker' via Everything List
In the name of completeness I figured I should ask Gemini the same question that I asked Claude and GPT, you can see its Full response below but Gemini's  last line was: "Forecast: 18% probability of a positive resolution by December 31, 2030".

ME:I want you to think like a super forecaster when answering this question. "By December 31, 2030, does there exist an AI system, possibly using external tools, code execution, retrieval, and orchestration of other AI models but not specialized robotic hardware, that is recognized by broad expert and scientific consensus as matching or exceeding the best documented human performance, within roughly comparable time and cost, on essentially every cognitive task that has an objective or intersubjectively-agreed success criterion (math, science, programming, forecasting, strategy games, engineering, legal/medical/financial analysis, etc.) — doing so at least as consistently as elite human experts — with tasks gated by legal or biological status, tasks with no objective quality standard, and tasks bottlenecked by physical embodiment (not just fine dexterity, but also raw speed, strength, and endurance) all excluded?

Gemini: "To approach this question with the rigor of a superforecaster, we must strip away both techno-optimist hype and biological exceptionalism. We must focus strictly on the resolution criteria, weigh the "inside view" (technological trajectory) against the "outside view" (historical base rates of technological consensus), and identify the critical bottlenecks.

Here is a probabilistic breakdown of why this specific, highly constrained definition of AGI (often referred to as "Strong Drop-in AGI") is a formidable hurdle by December 31, 2030.

. Deconstructing the Resolution Criteria

The prompt establishes an incredibly high bar. To resolve positively, the system must not just be "smart"; it must meet all the following conditions simultaneously:

  • Peak Capability: Matches or exceeds the best documented human performance (not just the average professional, but the elite).

  • Universality: Applies to essentially every objective cognitive task (math, engineering, strategy, law, finance).

  • Reliability: Acts at least as consistently as elite humans.

  • Efficiency: Operates at roughly comparable time and cost.

  • Sociological Threshold: Recognized by broad expert and scientific consensus.

    2. The Case for Positive Resolution (The "Inside View")

    There are strong trend lines suggesting massive capability overhangs that could be unlocked by 2030 (roughly 4.5 years from today, July 2026).

    • Compute Scaling & Investment: Trillion-dollar infrastructure investments are already in motion. If scaling laws (predictable performance improvements relative to compute/data) hold through the next two orders of magnitude, the raw pattern-matching and generation capabilities of models will be staggering.

    • System 2 / Test-Time Compute: We are already seeing the shift from pure autoregressive next-token prediction to "reasoning" models that utilize reinforcement learning and test-time compute to explore solution trees (e.g., AlphaGeometry, early self-play reasoning models).

    • Tool Use and Orchestration: The prompt explicitly allows external tools and code execution. AI does not need to perfectly simulate a physics engine in its weights; it just needs to know how to write the code to run a Python simulation. Agentic scaffolding (AI orchestrating other AI tools) drastically lowers the barrier to general capability.

      3. The Cruxes (Why this likely resolves negatively by 2030)

      Despite rapid progress, several severe bottlenecks make a positive resolution by 2030 highly unlikely based on this strict definition.

      A. The "Best Documented Human" Benchmark

      Matching a 95th percentile programmer or a junior lawyer is fundamentally different from matching the best documented human performance. The elite human edge in fields like theoretical physics, novel mathematics (e.g., Terence Tao), or high-stakes financial strategy relies on extreme abstraction, out-of-distribution intuition, and multi-year focused persistence. Current AI architectures are exceptional at interpolating within human data distributions, but generating world-class extrapolative novel insights consistently is structurally difficult for models trained on existing data.

      B. The Error-Compounding Bottleneck in Long-Horizon Tasks Elite human tasks are rarely single-turn queries. Engineering a novel aerospace component or conducting a multi-month scientific research project requires executing tens of thousands of sequential steps. Even if an AI agent has a 99.9% success rate per step, over a 1,000-step autonomous task, the probability of successful completion without catastrophic compounding error drops to ~36%. Elite humans self-correct intuitively; AI agents currently struggle to recognize when they have drifted fundamentally off track over long time horizons.

      C. The "Comparable Time and Cost" Constraint

      To match an elite researcher, an AI might need to generate millions of reasoning trees and simulate countless dead ends. While compute is scaling, the cost of running massive ensembles of frontier models for long-horizon agentic tasks remains exorbitant. By 2030, achieving peak human capability might be technically possible, but doing so within the "comparable cost" of hiring an elite human (who runs on ~20 watts of biological power) is a massive economic headwind.

      D. The Sociological Bottleneck (Broad Consensus)

      This is perhaps the strongest barrier. Even if the requisite AI system is brought online in early 2029, achieving broad expert and scientific consensus by December 31, 2030, is nearly impossible. Scientific consensus requires rigorous peer-reviewed benchmarking, replication of results, and exhausting debates over whether the AI actually "solved" the tasks or merely regurgitated contaminated training data. Skeptics will continually move the goalposts, demanding multi-year longitudinal studies to prove the AI matches elite human consistency. Eighteen months is generally not enough time for the scientific community to reach consensus on a paradigm-shattering claim.

      4. Synthesis and Forecast

      • Base Rate: Historically, the final 10% of autonomous reliability in complex systems (e.g., self-driving cars) takes vastly longer than the first 90%.

      • Timeline: We have ~4.5 years from mid-2026. While we will likely see AI systems match elite humans in narrow, highly structured objective tasks (like proving specific mathematical theorems or dominating complex strategy games), generalizing this to essentially every objective task—especially those requiring long-horizon novel research—will take longer to engineer.

      • FactorProbability AdjustmentCompute & Algorithmic Scaling++ (Pushes probability up)

      • Tool Use & Agent Orchestration++ (Pushes probability up)

      • Requirement: "Best documented human"-- (Pushes probability down)

      • Requirement: "Long-horizon reliability"--- (Strongly pushes down)

      • Requirement: "Broad scientific consensus"--- (Strongly pushes down)


      • Final Assessment: While highly capable, economically transformative AI agents will undoubtedly exist by 2030, a system that meets the absolute upper-bound criteria of this prompt—matching the greatest living human minds across all objective fields, reliably, at cost-parity, with universal scientific agreement—is highly improbable within this timeframe.

      • Forecast: 18% probability of a positive resolution by December 31, 2030."


John K Clark    See what's on my list at  Extropolis
ycj
Reply all
Reply to author
Forward
0 new messages