Tools / Analysis
The best model scores 12.0: GDPx tests the work beyond the prompt
SemiAnalysis puts agents inside demanding specialist workflows. Its low scores reveal how much the environment changes the test—and why a leaderboard needs a closer reading.
Claude Opus 5.5 leads GDPx with 12.0, followed by Gemini 4 Carbon at 10.9 and GPT-6 Astra at 9.6. The new SemiAnalysis benchmark, announced on 2 November, asks agents to work through specialist assignments in interactive environments. A good answer to a well-prepared prompt is only the beginning.
A low ceiling on a demanding test
GDPx v1.0 contains 206 tasks across 11 domains from 14 partners. The table shows the default all-domain top eight on 3 November.
Sources: SemiAnalysis [1]
| Model | GDPx score |
|---|---|
| Claude Opus 5.5 | 12.0 |
| Gemini 4 Carbon | 10.9 |
| GPT-6 Astra | 9.6 |
| Muse Wonder Full | 9.0 |
| Claude Fable 5.1 | 8.5 |
| GPT 6.1 Sol | 8.4 |
| Claude Sonnet 5.5 | 8.2 |
| Gemini 4 Argon | 7.3 |
The context is part of the job
The finance environment recreates a transaction through 4,000 files. Hardware tasks involve CAD, manufacturing preparation and circuit-board work. One public assignment asks an agent to design a compressor housing in SolidWorks, then supply drawings and a manufacturing plan. Another requires repairing Git LFS behaviour in partial clones.
These examples change what we should ask of a model. Can it find the relevant material, preserve constraints and leave an output another person can use? Fluent explanations alone do not answer those questions. For someone choosing an agent for technical work, the specific task may be more informative than a few tenths on the overall score.
Sources: SemiAnalysis [3] · SemiAnalysis [4]
What 12.0 actually measures
The default allowance is eight hours per task; some tasks use shorter limits. SemiAnalysis uses native agent harnesses where available and OpenCode otherwise. It reports pass@1 across three attempts, with equal domain weights; tasks mix binary and partial grading. The headline is therefore weighted benchmark performance, not the share of jobs an AI can replace.
The authors also attribute some lower scores to safety classifiers. Tooling, permitted actions and refusals therefore belong in the comparison alongside model capability. Removing selected domains changes the question being measured; it should not be silently presented as the same leaderboard.
Sources: SemiAnalysis [2]
Read a run before choosing a winner
The public rollout collection exposes one task per partner and offers 168 transcripts; the other tasks remain private to discourage tuning against the test. That makes GDPx useful both as a scoreboard and as a place to inspect how an agent gets stuck.
Our practical reading: shortlist by the relevant domain, inspect the failure, then test the same kind of work on your own material. A model that makes fewer convincing mistakes in your workflow is a better purchase than a headline winner whose strengths lie elsewhere. GDPx provides a starting point for that decision, not a substitute for it. Keep the model version, agent setup and success criteria fixed when comparing alternatives. Otherwise, an apparently better model may simply have had a different job.
Sources: SemiAnalysis [4]
SOURCES & METHOD
The record behind the article
The official GDPx leaderboard, domain catalogue, methodology and public rollout index were inspected on 3 November 2026. Scores are SemiAnalysis results; we did not rerun the benchmark. Exact per-task costs and reasoning-effort settings are not reported here.
- GDPx v1.0 leaderboard SemiAnalysis · 2026-11-03
- GDPx: Measuring Frontier Model Capabilities Across the Most Valuable Domains SemiAnalysis · 2026-11-02
- GDPx domain catalogue SemiAnalysis · 2026-11-03
- GDPx public rollouts SemiAnalysis · 2026-11-03
