The headline number from SemiAnalysis’s new GDPx benchmark is hard to ignore: the best AI agent available managed just 12% on realistic expert work. That result, published on November 2, 2026, by Max Kan, Andrew Megalaa and Dylan Patel, does not mean AI agents are useless. It means the hardest, longest professional tasks are still a very different challenge from the short, tidy prompts where agents now look fluent. For teams deciding where to trust an agent, that distinction matters more than any single score.
GDPx is designed to look less like a quiz and more like a workday. Agents are placed in realistic environments, given open-ended professional problems, and expected to explore, gather evidence and produce work that can be checked. The low scores that follow are therefore a useful warning: fluency is not the same as finishing a job to a professional standard.
What GDPx Actually Tests
GDPx version 1.0 contains 206 agentic tasks across 11 professional domains: chip design, cybersecurity, biology, finance, legal, quantitative trading, recursive self-improvement, hardware design, materials science, alignment and software engineering. SemiAnalysis built the benchmark with 14 specialized data partners, a detail that helps explain why the tasks feel closer to real assignments than to textbook questions.
Each agent works with a default limit of eight hours per task and must find its own context rather than receiving a neatly packaged brief. Scores are pass@1 averaged over three attempts, with domains weighted equally, so a strong result in one field cannot quietly hide a weak result in another. GDPx succeeds GDPval, which was released on September 25, 2025, and the shift in ambition is clear: this is not a test of quick answers, but of sustained, multi-step work. SemiAnalysis’s own GDPx announcement sets out that design in full.
The Leaderboard: 12%, 10.9%, 9.6%
Of the 20 models tested, results for 13 were published. Anthropic’s Claude Opus 5.5 leads at 12.0%, at $51.41 per task. Google’s Gemini 4 Carbon is second at 10.9%, at $16.29 per task. OpenAI’s GPT-6 Astra is third at 9.6%, at $53.99 per task, and Meta’s Muse Wonder Full is fourth at 9.0%.
The gap between first and fourth place is only three percentage points, which is itself informative. No frontier model is close to a passing grade on this kind of work, and the ranking should not be read as a claim that one system is ready for unsupervised expert deployment while the others are not. They are all, by this measure, still early. Coverage of the launch also underlines how wide the gap remains; see Unite.AI’s report on the GDPx launch and ExplainX’s analysis of the 12% result.
That context also helps explain why other AI stories can sound so different. When AI agents now start 40% of GitHub pull requests, the work is often narrower, better specified and easier to review than an eight-hour expert task. Volume in one setting does not prove competence in another.
Why Finance and Legal Scored Zero
The most striking detail is not the 12% headline, but the zeros. Every model scored zero on finance and legal under binary pass/fail grading. SemiAnalysis therefore also reports partial credit in those domains, which gives a more useful picture of near misses than a simple pass or fail.
Finance and legal work punishes small errors in a way that many benchmarks do not. A single wrong assumption, a missed document or a misread clause can invalidate an otherwise impressive draft. One finance task reconstructs a private-equity deal room with roughly 4,000 anonymized files, which tests exactly the skills that short benchmarks skip: finding the right material, keeping track of it over a long task, and reasoning consistently across it.
That is why a zero should not be read as “the agent did nothing.” It can mean the agent did a great deal of plausible work and still failed to clear a strict professional bar. For anyone handling money, contracts or regulated decisions, that is precisely the risk to manage: partial progress is not the same as a finished, reliable result.
The Cost Behind the Score
GDPx also makes the economics visible. The published figures are costs per task, not costs per successful task, and that difference matters. A $51.41 attempt that succeeds 12% of the time implies roughly $428 in attempts for each success, before any human review time is counted. Gemini 4 Carbon, at $16.29 per task and 10.9%, implies roughly $149 in attempts per success on the same rough calculation.
Those are not invoices, and real deployments will differ by task mix, tooling and review. But they point to the right question for buyers: not “what does one run cost?” but “what does a checked, usable outcome cost?” A cheaper model with a slightly lower headline score can be the better value if its failures are fast, obvious and cheap to catch.
The same discipline applies to security work. Google DeepMind CodeMender and the automation of security fixes shows where agents can help at scale, but automated fixes still need tests, review and a clear owner before they reach production.
The Caveats You Should Not Skip
GDPx is a serious benchmark, but it is still one benchmark, in its first version, and its scores have not been independently re-run. SemiAnalysis plans to refresh it every three to six months, which should reduce the risk of models being tuned to a fixed set of tasks, but it also means today’s ranking is a snapshot, not a permanent verdict.
Safety scoring is another important caveat. Safety classifiers pull some scores down in cybersecurity, biology and alignment, and one summary reports that 196 of 200 cyber alignment runs broke at least one behavioral rule. That finding deserves careful reading: it may reflect genuine safety failures, overly strict rules, or a mix of both. Either way, it shows that capability and safe behavior cannot be judged from a single headline number.
Readers should also be careful about what “expert” means here. These tasks were chosen to be hard, long and realistic, and they were weighted equally across domains. A different mix of tasks, a different time limit or a different grading rule would produce a different number. The honest conclusion is narrower than “AI agents fail at work”: on this demanding slice of work, they still fail most of the time.
What This Means for Teams Using AI Agents
For teams, the practical response is not to abandon agents, but to narrow their job. Pilot them on work that is specific, repeatable and easy to check: drafts with clear sources, code with tests, summaries that can be compared with the original, and research that a human can verify quickly. Keep a human reviewer in the loop, especially where the output will be sent to a customer, filed, or used to make a financial or legal commitment.
Measure cost per successful outcome, not cost per attempt, and track failure modes as carefully as successes. If an agent is fast but its errors are subtle, it may be more expensive than a slower system whose mistakes are obvious. If it handles short tasks well but drifts over long ones, break the work into smaller, checkable steps instead of giving it an eight-hour brief and hoping for the best.
Most importantly, keep humans in charge of finance, legal and other high-stakes work. That does not mean humans must do every routine step by hand; it means a qualified person should own the final judgment, check the key evidence, and be able to explain the decision. The recent wave of platform breaches is a reminder of what happens when automation outruns oversight: our report on why 9 AI developer platforms were breached in one month and how to protect your API keys makes the case for tighter controls, not blind trust.
Security habits matter just as much for everyday teams. As AI phishing gets harder to spot in 2026, practical defenses still work — verification, least privilege, and a pause before acting on an urgent-looking message. The same instincts apply to agent output: verify before you trust it.
GDPx gives the industry a harder, fairer target. Twelve percent is not a verdict on all AI use. It is a statement about the hard tail of multi-hour expert work: the work where context is messy, stakes are high, and a near miss is not good enough. Teams that respect that boundary — using agents where their work can be checked, and keeping people accountable where it cannot — will get the most value from them. Further coverage is available in AI Agent Update’s report on the GDPx results.
