Skip to content
BetaAI and tech news.Edition of Fri 30 Oct 2026 
AIAnalysis

Best AI for coding in 2026: what the benchmarks show and what they miss

The top coding models are within a few points of each other, and the scores come mostly from the companies selling them. Here is how to read them.

By the Machine Notes desk4 min read
Share

Terminal-Bench 4.0 scores, as the makers report them

%

  • Claude Opus 5.566.4%
  • GPT-6 Astra57.9%
  • Gemini 4 Argon57.4%
  • Claude Fable 5.155.8%
  • GPT-5.6 Sol37.3%
FIG. 1 · Self-reported, at each maker's chosen settingsSource: Model makers' launch pages

The best AI for coding in October 2026 is not a single model. The three biggest labs each lead on at least one coding test they publish themselves, and the gaps between them are often a few percentage points.

What do the main coding benchmarks measure?

They test whether a model can fix real code issues, finish command-line tasks or solve contest problems.

SWE-bench. Published in October 2023, it collected 2,294 real issues from 12 popular Python repositories on GitHub. The model gets the code and the issue text and must change the code so the issue is fixed. It passes only if the project's hidden tests pass. When it launched, the best model solved 1.96% of issues.

SWE-bench Verified. OpenAI built this cleaner subset in 2024. Engineers reviewed 1,699 SWE-bench problems, three experts per problem, and kept 500. For two years it was the number in almost every model launch.

SWE-bench Pro. A harder set of 1,865 problems from 41 actively maintained repositories, released in September 2025. Only problems from 11 repositories are public. The rest come from a held-out set and from 18 proprietary codebases, so they are harder to leak into training data. The authors say some tasks could take a professional engineer hours to days.

Terminal-Bench. Instead of patching a repository, the agent works in a command line. Version 2.0, described in a January 2026 paper, has 89 tasks drawn from real workflows, each with its own environment, a human-written solution and tests. At publication, frontier models and agents scored below 65%. The labs now quote version 4.0, which OpenAI describes as terminal tasks covering software engineering, system configuration and data analysis.

LiveCodeBench. Built from contest problems on LeetCode, AtCoder and Codeforces. It keeps adding new problems and records when each was published, so a model can be tested only on problems released after its training data was collected. It also tests code repair, code execution and predicting test output.

What are their known weaknesses?

The main ones are contamination, broken tests, scaffold differences and self-reporting.

  • Contamination. A model may have seen the test problems, or their answers, during training. On 23 February 2026, OpenAI said it had stopped reporting SWE-bench Verified scores for this reason. Every frontier model it tested could reproduce the original fix, or exact details of the problem text, for some tasks. OpenAI concluded that gains on Verified increasingly show how much a model saw the benchmark in training, and recommended SWE-bench Pro instead.
  • Broken tests. In the same review, OpenAI audited 138 Verified problems that its o3 model often failed. It found 59.4% had material problems with the tests or the problem description, such as tests that rejected correct fixes because they expected a specific function name.
  • Scaffold differences. A coding score measures a model plus the software scaffold around it: the tools, the step limits and the instructions. Anthropic reports its Terminal-Bench 4.0 score at its highest effort setting, "xhigh". It reports GPT-6 Astra at the "high" setting OpenAI used. OpenAI notes that its FrontierCode runs gave Astra instructions similar to those it gets in Codex, OpenAI's coding agent.
  • Self-reporting. Nearly every current number is run by the company launching the model, often including the rival scores in its comparison chart.

Which model scores highest right now?

On the makers' own numbers, Claude Opus 5.5 leads on Terminal-Bench 4.0 and FrontierCode, and Gemini 4 Argon leads on DeepSWE. All figures below are self-reported by the model's maker, at that maker's chosen settings. A dash means the source did not report it.

ModelMakerTerminal-Bench 4.0FrontierCode 1.1 (Main)DeepSWE v1.1Source, date
Claude Opus 5.5Anthropic66.4%54.4%-Anthropic, 22 Sep 2026
GPT-6 AstraOpenAI57.9%53.3%74.1%OpenAI, 3 Sep 2026
Gemini 4 ArgonGoogle57.4%-77.9%Google DeepMind, 13 Oct 2026
Claude Fable 5.1Anthropic55.8%50.3%-Anthropic, 22 Sep 2026
GPT-5.6 SolOpenAI37.3%47.5%-OpenAI, 3 Sep 2026

Anthropic describes FrontierCode as a test of whether an agent's code changes would be merged. Neither Anthropic nor OpenAI reports SWE-bench Verified on these launch pages.

The rival columns do not always agree. Google's model card lists GPT-6 Astra at 58.2% and Claude Fable 5.1 at 57.9% on Terminal-Bench 4.0. OpenAI's own page gives 57.9% and 55.8%. On DeepSWE, Google lists Claude Opus 5.5 at 74.2%, a hair above Astra, and its own model at 77.9%.

Prices differ more than the scores. Per million tokens, Anthropic charges $4 for input and $20 for output on Opus 5.5. OpenAI charges $10 and $50 for GPT-6 Astra.

Do the benchmarks settle which AI is best?

Our take: no. Each lab leads the table it publishes, on the tests it picked. That does not make the numbers false, but it means none of them settles the question. Anthropic itself says on its Opus 5.5 page that at this level, benchmark margins have become a less reliable guide to real-world differences. A three-point lead on one terminal benchmark, at one effort setting, tells you little about your codebase.

How should you choose for your own work?

Test the tools on your own code and count the cost per finished task.

  1. Try it on your own repository. Take five or ten real tickets your team has already closed and see which tool fixes them, and how much you had to correct.
  2. Count cost per finished task, not per token. A dearer model that finishes in fewer steps can cost less. Effort settings change both cost and score.
  3. Check context limits on large codebases. How much code the tool can read at once, and how it handles long sessions, matters more on big projects than on contest problems.
  4. Read the data terms. Check whether your code is kept, used for training, or sent to a third-party host, especially for client or regulated code.
  5. Treat a new benchmark record with care. Ask who ran it, at what effort setting, with which scaffold, and whether the test set is public.

Sources

  1. arXiv: SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (Jimenez et al., 2023)arxiv.org
  2. OpenAI: Why we no longer evaluate SWE-bench Verified (23 February 2026)openai.com
  3. arXiv: SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? (Deng et al., 2025)arxiv.org
  4. arXiv: Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces (Merrill, Shaw et al., 2026)arxiv.org
  5. arXiv: LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code (2024)arxiv.org
  6. Anthropic: Introducing Claude Opus 5.5 (22 September 2026)anthropic.com
  7. OpenAI: GPT-6 Astra: A new generation of intelligence (3 September 2026)openai.com
  8. Google DeepMind: Gemini 4 Argon model card (13 October 2026)deepmind.google