Software engineering benchmark capability, nearly saturated.
RB-02 · AI After
The Gold Rush
What Becomes Scarce?
From AI Access to AI Leverage
AI is making intelligence abundant.
The ability to turn intelligence into reliable outcomes is becoming scarce.
What becomes scarce when intelligence becomes cheap?
Three things, true at the same time.
AI has gone mainstream
88% of organizations use AI; 70% use generative AI in at least one business function. Population adoption reached 53% in three years — faster than the PC or the internet.
AI capability is accelerating
SWE-bench Verified went from ~60% to near 100% in a single year. GPT-5.6, Claude Sonnet 5 / Opus 5 have pushed capability toward execution.
But value capture is still hard
In 2025, METR measured experienced developers taking 19% longer on tasks. By 2026, because developers refuse to work without AI, reliable measurement is becoming increasingly difficult. New data points toward gains, but estimates remain uncertain and vary across studies.
When intelligence becomes cheap, reliable execution becomes scarce.
An analytical position supported by the evidence below, not a single measured finding
The boom is still real: AI has gone mainstream.
of surveyed organizations use AI (2025)
use generative AI in at least one business function
US 2025 private AI investment
annual US consumer value (early 2026, +54%)
population adoption in three years — faster than PC or internet
AI companies newly funded in 2025
AI Agent deployment remains in the single digits across nearly every business function
Consumer value is real and growing rapidly. But large-scale agentification has not happened yet — adoption is real, transformation is still early.
Capability is accelerating: benchmarks approach saturation.
Over 90% of notable frontier models were produced by industry.
GPT-5.6
A frontier model for complex work: coding / research / science / cybersecurity / computer use / design. Launched 2026.07, Sol / Luna updated 2026.08.
Claude Sonnet 5
Positioned as far more agentic: planning / browser / terminal / tool use / autonomous execution.
Claude Opus 5
Near-Fable-5 intelligence at half the price ($5 / $25 per M token). Major gains on long-running agent work.
“AI can’t really write complex code” no longer holds. Benchmark capability is approaching saturation — real-world reliability and economic integration are the harder problem now.
AI is shifting from a "tool that answers questions" "to a "system that executes work.""
The real question is no longer whether AI can act, but whether it can act reliably enough for economically important work.
But productivity has become harder to measure.
| Period | Evidence |
|---|---|
| 2023 | ChatGPT writing tasks: completion time -40%, quality +18%. The classic early-gen-AI result.Science · Noy & Zhang |
| 2025 | METR software engineering RCT: 16 experienced open-source developers, 246 tasks, using the AI coding tools of the time took 19% longer to complete tasks.METR · 2025 RCT |
| 2026 | METR update: developers increasingly refuse to work without AI, so experiments now suffer severe selection bias. Raw data hints at gains (-18% to -4%), but the authors stress this is only a lower bound on true gains; no reliable average uplift can be given.METR · Feb 2026 |
| 2026 | Anthropic survey: 86% of Claude users self-report working faster — but perception and objective measurement diverge; case evidence shows over-reliance can create new costs (e.g., Uber exhausting an AI budget in months).Anthropic Economic Index · 9,700 users |
| 2026 | 5,305 Claude Code transcripts: potential time savings of 1.5×–13×, but the authors explicitly warn this is exploratory, not a controlled productivity estimate.METR · transcript analysis |
Pattern worth noting (Stanford AI Index 2026): measured gains are clearest in structured, easily measurable tasks — and smallest where deeper reasoning is required.
The evidence is contradictory: METR measured 19% longer completion times, while Management Science field experiments show ~26% more tasks completed, and 86% of users self-report gains. The honest conclusion: AI productivity is not one number — it depends on task, model, experience, and method. And don’t claim “Claude Code is 13× faster”; that is only the upper end of an exploratory analysis.
The cost of intelligence keeps falling.
GPT-3.5-level inference: $20 → $0.07 per million tokens.
AI price deflation continues
Industrial Revolution
Price of physical labor → commoditized
Internet
Price of information distribution → commoditized
AI
Marginal cost of cognitive production → falling, and still accelerating
As frontier intelligence becomes increasingly affordable, competition moves from “who has intelligence” to who can turn intelligence into reliable business outcomes.
The bottleneck has moved: from generation to reliable execution.
Why we read it this way (Stanford AI Index 2026): measured gains are smallest precisely where tasks demand deeper reasoning — the point where reliable execution, not generation, is the binding constraint.
Our reading: what used to be scarce was generation; what is increasingly scarce is verification and reliable execution. The real competition starts in the middle of the value chain.
Building is cheap now. Prototypes are no longer a moat.
Distribution is still scarce: the problem AI doesn't solve.
Users
Trust
Data
Domain
Brand
A beautiful AI demo is not a business. Products with users, trust, and distribution are. Competition moves from “building it” to “being found, trusted, and delivered reliably.”
Expertise is being repriced.
Expert × AI × Automation — not AI vs Human, but Expert × AI
"I can use Claude."
The tool is not a barrier. This sentence is approaching zero value.
"I automated my CFD analysis pipeline with AI to improve engineering decisions."
Domain knowledge + workflow redesign + decision systems — assets that compound.
Early signal (Stanford AI Index 2026): the effect is already uneven on early-career software developers and hiring pipelines — routine, entry-level work is being repriced first.
AI does not automatically create expertise. It changes the return on existing expertise. Your domain experience is the fulcrum AI leverages.
Not "is AI a bubble," "but "are valuations running ahead of value.""
5,048 (2000.03) → 1,139 (2002.10)
Search · E-commerce · Cloud · Digital Advertising
Bubbles kill companies without business models. They don’t kill technology.
US private AI investment ($285.9B) is about 23× China’s, yet the model-performance gap has narrowed to 2.7%. The return on enormous capital is unproven.
Hype products · thin AI wrappers · AI apps with no users
Infrastructure · foundation models · applications solving real industry problems
AI can be both a transformative technology and an overvalued market. The better question is not “Is AI a bubble?” but — are AI valuations pricing in future productivity before businesses can actually capture it?
What, actually compounds?
Domain Knowledge
Physics constraints, industry norms, how the real world actually works.
Judgment
Which result to trust? Which direction to invest in? Who is accountable?
Distribution
Users, trust, brand, network effects.
Ownership
Products, content, data, equity — compounding happens on what you own.
Low value (getting cheaper)
- Simple writing
- Simple code
- Simple design
- Information tidying
High value (getting more important)
- Judgment — decision-making
- Creation — from 0 to 1
- Industry experience — domain knowledge
- Ability to build systems — reliable execution
Domain Knowledge + AI Capability + Judgment + Ownership = Long-term Value
AI is not replacing value. It is repricing value.
Clear macro evidence that AI adoption shows up in measured productivity within a year — rather than the messy, task-dependent results we see today.
The gold rush is ending.
The AI revolution is just beginning.
The gold rush was about access. The revolution is about leverage. They are not the same thing.
Evidence Sources
- Who captures the economic value as intelligence becomes abundant?
- How fast does AI adoption translate into measurable productivity?