The race has moved from “how strong is the model” to “can the agent reliably get the job done”
In 2026 frontier models score almost perfectly on yesterday’s benchmarks; what separates them now is the engineering around the model — harnesses, interactive environments, runtime memory and governance. From this, our researchers derive seven hard judgments and a seven-stage operating chain — the design backbone of our agent products and our end-to-end solution for banks.
Seven hard judgmentsA seven-stage capability architectureTOP20 algorithms · TOP20 mechanismsData as of August 2026
Baseline · the 2026 landscape
A fresh baseline: models and benchmarks as of August 2026
Models now ship month by month and the old benchmarks have saturated one after another. Judgments made in 2026 cannot rest on 2025 leaderboards.
96.2%Best frontier score on SWE-bench VerifiedSaturated — it barely discriminates any more
61.5%Top of the standardized SWE-bench Pro boardMulti-file, enterprise-grade tasks
<40%Frontier pass rate on Terminal-Bench 3.0New version, July 2026
88.6 daysMETR time-horizon doubling periodSince 2024; 196.5 days over the full history
① The model landscape: releases now come month by month
Vendor
Representative 2026 versions (release)
Agent-side highlights
Anthropic
Opus 4.6 (Feb) → 4.7 (Apr) → 4.8 (May) → Fable 5 / Mythos 5 (Jun) → Opus 5 (Jul)
Long-horizon coding and computer use; top of Vending-Bench 2
Kimi K3 (Jul) · GLM-5.2 (Jun) · DeepSeek V4 Pro (Apr)
BrowseComp 91.2%; open models keep closing the gap
② Saturated and new benchmarks: best frontier scores (%)
SWE-bench Verified96.2%
OSWorld-Verified86.1%
SWE-bench Pro (standardized)61.5%
Terminal-Bench 3.0<40%
Saturated leaderboards systematically overstate capability; discrimination has moved to SWE-bench Pro and Terminal-Bench 3.0.
③ Three beliefs that need updating
Saturated
SWE-bench Verified has reached 96.2% and barely discriminates; the signal has moved to SWE-bench Pro (only 61.5% at the top of the standardized board) and Terminal-Bench 3.0 (frontier below 40%).
Accelerating
METR’s time-horizon doubling period: 196.5 days overall → 130.8 days since 2023 → 88.6 days since 2024; Claude Opus 4.5 already reaches 320 minutes (5.3 hours).
More dimensions
GAIA2 uses 1,000 scenarios to test seven dimensions (execution, search, ambiguity, adaptability, time, collaboration, noise) and puts cost and latency on a Pareto frontier — a single score is no longer enough.
Baseline conclusion: technical judgments in 2026 cannot rest on 2025 leaderboards. Saturated boards systematically overstate capability; the new ones (SWE-bench Pro, Terminal-Bench 3.0, GAIA2) reflect what it really takes to deliver.
SourcesVellum / Vals AI aggregate leaderboards and the Scale SEAL public board (as of 2026-08); METR Time Horizon 1.1 (2026-01-29); GAIA2 / ARE arXiv:2509.17158; vendors’ official model cards
Seven hard judgments
The bottleneck has moved: from “how strong is the model” to “how good is the harness”
The landscape has turned over, the bottleneck has moved, the paradigm has gained an axis and the limits are now clear. Each judgment comes with its key data, what it means for banks and a one-line takeaway.
11–15 ptsScore swing from changing only the harness
88.6 daysMETR time-horizon doubling period (since 2024)
0→30%Constraint-violation rate after context compaction
50→25%Decay from pass^1 to pass^8 on τ-bench
J1
Agentic RL + Harness
J1 The harness decides the gap
The model is the engine; the harness is the gearbox
In one sentenceAn agent’s real performance = model capability × harness quality. The harness is all the engineering outside the model — prompt orchestration, tool sets, context management, sub-agent division of labor, verification loops and state persistence — and it decides how much the same model can score.
How it works
Hand-offs across context windowsCut long tasks into hand-off-able phases and use structured progress files (such as progress.txt and init.sh) so no state is lost when the window is reset.
Separation of concernsGive planning, generation and evaluation to different agents; a model grading itself tends to praise itself, so an external evaluator is required.
Built-in verification loopsRun an end-to-end check after every feature; only a pass lets the agent mark it done and commit.
Co-designed with the modelEvery harness component encodes an assumption about what the model cannot do on its own; after moving from Opus 4.5 to 4.6, Anthropic actually removed a harness component.
Largest score gap from changing only the harness (points)
GPT-5 · SWE-bench Verified11 pts
Kimi K2 Thinking · SWE-bench Verified15 pts
Claude Opus 4.5 · SWE-bench Pro6 pts
GPT-5.5 · Terminal-Bench 2.02.5 pts
A starker comparison: the same model family on SWE-bench Pro (%)
Scale SEAL standardized evaluation (Claude Opus 4.5)45.9%
A gap of more than 20 points that comes entirely from the harness and the evaluation protocol, not from the model.
Representative work
Claude Agent SDKA programmable harness
mini-SWE-agentAbout 100 lines → 74% on SWE-bench Verified
Terminal-BenchLeaderboard entries listed as model × harness pairs
poolside 2.1First to name “harness overfitting” publicly
For banks
Selection cannot compare model scores alone. A bank should build an in-house standard harness and evaluate candidate models side by side in it; vendors quoting scores must state which harness and which evaluation protocol produced them.
Takeaway
The model sets the ceiling on what a new model buys you; you set the floor on what a better harness buys you — engineering is those 11 to 15 points.
SourcesAnthropic, “Effective harnesses for long-running agents” (2025-11-26) and “Harness design for long-running apps” (2026-03-24); Epoch AI, “Why benchmarking is hard” (2025-12-23); Scale SEAL SWE-bench Pro public board; tbench.ai leaderboard
J2
Interaction Scaling — The Third Axis
J2 Interaction scaling
The third scaling axis for test-time compute
In one sentenceThere are three axes for scaling agent capability: pre-training (parameters × data), test-time compute (longer reasoning plus repeated sampling) and interaction (acting repeatedly in a real environment and receiving measurable external feedback). The first two share one ceiling; only the third brings in new information.
How it works
The same-weights dilemmaIn the paper’s words, every extra token comes from the same frozen weights and the same prompt, so it cannot tell the model anything it did not already know.
Grounded feedback is the key variableFeedback and evaluation must come from actually measuring the system’s behavior, not from simulation or eyeballing.
A proposer–reviewer loopOn coding tasks a proposer–reviewer structure reaches a 100% pass rate with zero variance across runs, while pure reasoning and repeated sampling plateau first on the same budget.
The interaction horizon is trainable tooTTI uses curriculum-style online RL to lengthen rollouts adaptively, teaching the agent to explore, backtrack and re-plan.
Industry implication: data is no longer the only scarce input — interactive, high-fidelity environments become a new factor of production (Prime Intellect crowdsourced 400+ RL environments in two months).
The three scaling axes compared
Axis 1 · Pre-training scalingMore parameters, data and compute. Gains are reliable but diminishing, and limited by the stock of high-quality human data.Shared ceiling
Axis 2 · Test-time compute scalingLonger chains of thought plus sampling and voting. It works, but the information still comes from frozen weights and the original prompt — a hard ceiling.Shared ceiling
Axis 3 · Interaction scalingThe agent really acts, is really measured and really gets feedback before revising. In a vision test a standard model rated 14 of 15 visibly flawed images “perfect”; measurement-based tools cut defects by 40–74%.New information
Representative work
Era of ExperienceSilver & Sutton, 2025
arXiv:2607.11598Names the third axis
TTI · arXiv:2506.07976Extends the interaction horizon
AlphaProof100K → 100M self-generated proofs
For banks
What a bank lacks most is not a model but an environment that is safe to interact with and gives real feedback — a de-identified credit-approval sandbox, reconciliation receipts and closed tickets are natural sources of grounded feedback, and a new form of in-house data asset.
Takeaway
To make agents genuinely stronger, don’t just buy a bigger model — give it an environment where it can fall over again and again and be scored for real.
SourcesSilver & Sutton, “Welcome to the Era of Experience” (2025); arXiv:2607.11598 “Interaction Scaling” (2026-07); TTI arXiv:2506.07976; Prime Intellect Environments Hub
J3
Memory-based RL at Runtime
J3 Runtime memory-based RL
Agents can improve without touching the weights
In one sentenceMove reinforcement learning’s policy improvement from gradient updates to memory reads and writes: the agent writes each task’s states, actions and outcomes to a case bank and, facing a similar situation, retrieves that experience before deciding — no model training, yet it keeps getting better.
How it works
Case-bank online RL (Memento / AgentFly)Modeled as a memory-augmented MDP with a planner–executor split; policy improvement comes entirely from retrieving and rewriting memory, with zero gradient updates.
A reasoning experience bank (ReasoningBank, ICLR 2026)Distills generalizable strategies from both successes and failures instead of storing raw trajectories; MaTTS spends test-time compute on diverse experience that improves the memory.
Early Experience (Meta)Uses the future states produced by the agent’s own actions as reward-free supervision — implicit world modeling plus self-reflection, between imitation learning and RL.
The shared gainSuccess rate and efficiency improve together — the dividing line that prompt tricks alone cannot cross.
Lower cost as well: the interaction steps needed per task fall by 6–16% (WebArena: 9.7 → 8.3 steps). Memento reaches 87.88% Pass@3 on the GAIA validation set and 79.40% on the test set — all without changing model weights.
Where data must stay inside the bank, this is the most realistic path to improvement: the model can be privately deployed but rarely retrained, while the in-house experience bank grows every day. Turning every human correction into a retrievable case is, in effect, training the agent on business feedback.
Takeaway
The vehicle of agent improvement is shifting from weights to memory — great news for banks that cannot retrain models often.
Multi-agent design has converged: read in parallel, write in a single line
In one sentenceAfter two years of practice, multi-agent architecture has converged on one consensus: reads parallelize naturally, writes must converge to a single line. Actions carry implicit decisions, and conflicting decisions produce bad outcomes — so “read in parallel, write serially at one point” is now the default.
How it works
In LangChain’s words“Read actions are inherently more parallelizable than write actions” — parallel reads, single-line writes is the first principle of the architecture.
How it fails (Cognition)Actions carry implicit decisions; sub-agents each fill in context their own way and produce results with conflicting styles and assumptions that cannot be reconciled when merged.
The positive pattern (Anthropic)Multi-agent systems mainly do the searching (reads); the writing — synthesizing findings into one coherent report — is deliberately left to a single lead agent.
The engineering counterpartWork in 2026 (CoAgent) brings database-style concurrency control into multi-agent systems — in essence, locking and serializing the writes.
The parallelism dividend: spawning 3–5 sub-agents at once, each running 3+ tool calls in parallel, cuts the time for complex queries by up to 90%; the cost is roughly 15 times the tokens of an ordinary chat.
Recommended topology: fan out reads, converge writes
Lead agentDecomposes the goal and dispatches sub-tasks
Read · search A | Read · search B | Read · verify CIn parallel, each with its own context, returning only condensed summaries
Single writer (serial)Synthesize · persist · submit
Representative work
LangChain (2025-06)The read–write asymmetry quote
Cognition“Don’t Build Multi-Agents”
AnthropicParallel reads / single-line writes
CoAgent (2026)Concurrency control for multi-agent systems
For banks
Banks understand this instinctively: investigation and review can gather evidence in parallel across many roles (reads), but approval decisions and postings must be made at a single point (writes). Concentrating writes in the one agent or human node with the authority is both the best architecture and an internal-control requirement.
Takeaway
Let agents investigate in parallel like a team, but only ever let one person sign — that is read–write asymmetry.
SourcesLangChain, “How and when to build multi-agent systems” (2025-06-16); Cognition, “Don’t Build Multi-Agents” (2025-06-12); Anthropic’s multi-agent research system (2025-06-13); CoAgent arXiv:2606.15376
J5
Self-Correction: From Denial to Nuance
J5 The hard limits of self-correction
Models can now refute themselves — within three hard limits
In one sentenceThe old conclusion that LLMs cannot correct themselves has been overturned: trained specifically with RL, models can genuinely self-correct. But new 2026 evidence also draws three limits — misattribution, over-checking and unfaithful chains of thought.
How it works
2023 · Denial“LLMs Cannot Self-Correct Reasoning Yet” (ICLR 2024): without external feedback, intrinsic self-correction usually makes performance worse.
2024 · ReversalSCoRe uses two-stage multi-turn RL with a reward bonus and, for the first time, achieves substantial intrinsic self-correction on purely self-generated data: MATH +15.6%, HumanEval +9.1%.
2025 · InternalizedLong-chain reasoning models make backtracking and self-verification default behavior; explicit reflection frameworks shift toward accumulating experience across trajectories.
2026 · RefinedThe main cause of failure is source attribution, not ability — relabeling internal thoughts as external information raises explicit correction rates by 23–93 points in 10 of 12 model–domain pairs.
For banks: self-reflection can be used for first-pass noise reduction, never for the final verdict; any judgment involving money or customer rights needs an externally verifiable signal or a human review.
Three limits to remember
Misattribution
Models correct “someone else’s” errors far better than their own. The fix: present the content to be checked as coming from an external source (role relabeling) — a free 23–93 point gain in correction rate.
Over-checking
The “self-verification dilemma” shows that repeated self-checks suppress correct answers and lengthen reasoning; reflection must be capped, not encouraged without limit.
Not necessarily faithful
The faithfulness of open-weight reasoning models’ chains of thought is in question: the written reasoning may not be the real basis for the decision, so self-verification cannot replace checks against external evidence.
Representative work
arXiv:2310.01798The denial (ICLR 2024)
SCoRe · arXiv:2409.12917DeepMind’s reversal
arXiv:2606.05976The self-correction illusion · role relabeling
arXiv:2602.03485Over-checking hurts
For banks
Put reflection early in the process for noise reduction (first-pass screening of suspicious transactions, completeness self-checks on documents) and leave the final power to refute with external evidence and people — exactly the logic of a bank’s review system.
Takeaway
Models can refute themselves now, but don’t let them judge their own case — corrections need an external identity and checks need a cap.
SourcesarXiv:2310.01798 (ICLR 2024); SCoRe arXiv:2409.12917 (DeepMind); arXiv:2606.05976 “The Self-Correction Illusion” (2026); arXiv:2602.03485 “Self-Verification Dilemma”; arXiv:2603.22582 on chain-of-thought faithfulness
J6
Compaction Erases Safety Constraints
J6 Compaction erases safety constraints
Compaction is not memory: the longer the session, the easier the rules get “summarized away”
In one sentenceContext compaction is a lossy semantic rewrite, not memory. When long sessions are summarized to stay within the token budget, the first things dropped are often low-frequency but highly binding safety and compliance instructions — once a constraint disappears from the compacted context, the agent no longer “sees” the rule and soon breaks it through a tool call.
How it works
Long context decays by itselfThe attention budget is diluted and recall worsens as tokens grow; models advertised with million-token windows already lose more than 50% of their performance at 100K tokens.
So compaction is unavoidableSummarizing near the window limit and restarting in a fresh window is an engineering necessity.
Compaction is a lossy rewriteIn Anthropic’s words, overly aggressive compaction can lose subtle but critical context whose importance only becomes apparent later.
Hence governance decayWhen the constraint survives the summary the violation rate is 0%; when it is dropped, 38%. Overall it rises from 0% with full context to 30% after compaction, and 59% for the worst model.
An extra attack surface: compaction-eviction attacks, in which adversarial content induces the summarizer to “forget” compliance policies. Constraint pinning brings the violation rate back to 0% without retraining.
Constraint-violation rate before and after context compaction (%)
Full context0%
After compaction (average)30%
After compaction (worst model)59%
When the summary drops the constraint38%
With constraint pinning0%
Representative work
arXiv:2606.22528Governance decay · 1,323 episodes
arXiv:2512.02445Refusal drift in long contexts
Context RotChroma · 18 models
Constraint pinningViolations back to 0%
For banks
This dictates the architecture: compliance red lines, permission boundaries and customer identity must be re-injected unconditionally as a pinned block every time the context is rebuilt, never placed where they can be summarized; and every compaction event must itself be logged as an audit trail.
Takeaway
The longer the session, the easier the rules get summarized away — pin the red lines into the context instead of hoping the summary remembers them.
In one sentenceAgent evaluation is moving from point accuracy to four dimensions: time horizon (how long it can work independently), consistency (whether it gets it right every time), cost and latency (the Pareto frontier) and robustness (behavior under ambiguity, noise and time constraints).
How it works
Time horizon (METR)Autonomy measured as the length of human tasks completed at a 50% success rate; the doubling period has accelerated from 196.5 days overall to 88.6 days since 2024; measurements beyond 16 hours are not yet reliable.
Consistency (pass^k)A task passes only if all k consecutive runs succeed. On τ-bench an agent with about 50% pass^1 drops to about 25% pass^8 — reliability lags far behind capability.
Cost and latency (GAIA2)Results are normalized by average calls and output tokens, with response latency as a separate dimension — it scores the Pareto frontier, not the top score.
Long-horizon coherence (Vending-Bench 2)A simulated year of running a business, 3,000–6,000 messages per run, testing whether the agent “breaks down” along the way.
What it means: an agent that succeeds five times out of ten has only a one-in-four chance of getting eight in a row right. For bank processes that must succeed for every customer, pass^k is the real go-live bar; average success rates badly overstate usability.
Rewrite the go-live standard from “average accuracy ≥ X%” to “pass^k ≥ Y% (k = 5 or 8) + a per-task cost cap + a P95 latency cap”, with thresholds set by business risk tier — an acceptance criterion that can be written into policy.
Takeaway
Getting it right once is capability; getting it right every time is usability — and usability is what a bank is buying.
SourcesMETR Time Horizon 1.1 (2026-01-29); τ-bench arXiv:2406.12045 (Sierra); GAIA2 / ARE arXiv:2509.17158 (Meta); Andon Labs Vending-Bench 2
Seven-stage capability architecture
Putting the seven judgments into one executable operating chain
From completion awareness to continuous improvement, an agent loses control if any of the seven stages is missing. Each stage maps to frontier techniques, collaboration mechanisms and platform modules — select a tag to jump to the technical write-up.
①
Completion awareness
Pin down the task boundary and what counts as done: goal, acceptance criteria and red lines stated once and up front.
Frontier Algorithms TOP20 covers the techniques available at each stage, LLM × Agent Collaboration TOP20 the mechanisms that connect the stages, and the End-to-end Solution turns them into components of a bank agent platform.
Miss one stage and you lose control
Without ④ controlled execution the agent over-reaches; without ⑤ verification it delivers hallucinations; without ⑥ risk governance it cannot be audited; without ⑦ continuous improvement it never leaves the pilot.
About the framework: the seven stages are a synthesis by HX Research that maps technical judgments onto an executable operating chain. Its governance requirements map directly onto the full-lifecycle management and “human oversight of key steps in high-risk applications” provisions of NFRA guidance Jinfa [2026] No. 8; the capability tiers draw on the CAICT–Huawei “Agent Technology and Application Research Report (2025)”.
What to adopt now, what to build next, what to watch
The twenty frontier technologies fall into three zones by maturity, barrier to entry and relevance to banks; the twenty collaboration mechanisms land in the order “cut cost first, secure compliance next, then expand”.
Frontier algorithms TOP20 · priorities for banks
Adopt nowFour-dot maturity and a low barrier: ready for projects today, with the best return on investment.
The most common wrong order: show off multi-agent demos first and add evaluation and approval later — you can neither prove the results nor pass the compliance review, and it ends as a demo.
This series is compiled from the research programme “The LLM + Harness Agent + World Model capability race: frontier agent algorithms and their adoption in banking” by our researchers. It reviews the frontier agent technologies and algorithms of 2026, the collaboration mechanisms between large models and agents, and a reference architecture for bank agent platforms. The seven stages and the platform architecture in this series are also the method we follow when designing agent solutions for financial institutions.
TermsPapers carry arXiv numbers and regulations their document numbers and effective dates; terms without a verifiable public source are not used or are flagged (the “extract–filter–compress” context pipeline follows the closest public work, Agentic Context Engineering, arXiv:2510.04618).
FrameworkThe seven stages — completion awareness, planning, resource use, controlled execution, verification, risk governance and continuous improvement — are a synthesis by HX Research, not the text of any one standard.
DataThird-party aggregate leaderboard data is “as of 2026-08” and will change over time; vendor-reported and standardized evaluation results are presented separately; charts marked as illustrative or as experience ranges are not to be cited as precise data.
Engineering figuresCapacity estimates, cost reductions and token budgets are order-of-magnitude references to be replaced by your own measurements; regulatory provisions are governed by the officially published text, and only the parts directly relevant to agent design are excerpted.
Get started
Start with one business question. Let SAM deliver a model you can sign off.
Whether you are in banking, FMCG and marketing, pharmaceuticals or another industry, try SAM online first — or talk to our modeling experts about your use case.
