Over the past year, an "open-source agent" often meant little more than "there is an open-source model." A wave of releases and papers in October 2026 shows that an agent enterprises can actually deploy privately is a four-piece kit: an open-weight model, an open-source execution framework (the harness), a reproducible training recipe, and operations that withstand faults and audits. According to the papers, swapping the open-source harness around the same model changes the SWE-bench Verified resolve rate by up to 44 percentage points [5]; a 4B model post-trained with an open data-synthesis recipe jumps on τ²-bench from its public-initialization level to 69.17, above the 28.33 of a publicly initialized 30B-A3B model [3]; and under MCP tool faults that follow the real-world fault distribution, the task-completion score drops from 8.08 to 3.23 (out of 10) [10]. This article walks through the period's progress layer by layer — model, harness, training, operations — breaks down four core algorithms, and offers our views and deployment advice.
![The "four-piece kit" of privately deployable open-source agents and representative numbers (data from [1][3][5][10]).](/assets/img/rs-open-source-agents-en-1-1100.18d9a126d4.jpg)
1. Background: Why an "Open-Source Model" Is Not an "Open-Source Agent"
An agent that can get work done has at least four parts:
- Model: understands, reasons, and generates tool calls;
- Harness: splits tasks into turns, manages context, executes tools, and handles errors — the agent's "operating system";
- Training recipe: the data and algorithms that turn a general model into one that excels at a class of tool tasks;
- Operations: deployment, restarts, fault recovery, metering, and auditing.
Closed solutions usually bundle all four, and the enterprise only gets an API. The open ecosystem has long opened only the first piece. What changed in October is that the other three began to open systematically: CUA-Lite releases environments, data, and a per-model-family harness for computer-use agents in one place [8]; Trajectory2Gym and Terrarium publish complete pipelines from synthetic environments to training data [3][9]; Glasseek releases training code, weights, data, and a leakage-proof evaluation plugin together [4]. Meanwhile, Liquid AI's LFM3 and inclusionAI's (Ant Group) Ling-3.1-flash-1019 fill in open-weight options at both ends — small on-device models and large-scale MoE [1][2].
But openness raises new questions: How should models and harnesses be paired? How do we make sure synthetic data teaches the right thing? Are evaluation scores leaking? How do agents survive faults in production? We go through them layer by layer.
Technical lineage: from "open weights" to "an open pipeline"
Roughly dividing the evolution of open-source agents into three stages makes the October shift easier to see.
Stage 1: open weights, fragmented evaluation. The open community first solved the problem of "is there a usable model?" By 2026, lightweight open models were already capable on repository-level tasks: as cited in the Pairing Matters paper, the 24B-parameter Devstral Small reports a 53.6% resolution rate on SWE-bench Verified with OpenHands, and Qwen3-Coder-Next, with 80B total and 3B active parameters, reports 70.6%–71.3% across three harnesses [5]. But each of these numbers comes from a single fixed model–harness combination. The paper audited 324 submission records across six tracks of the official August 2026 SWE-bench archive: at least 223 used proprietary models, while only five reported off-the-shelf models with fewer than 100B total parameters paired with established harnesses, and each of those reported only one fixed pairing [5]. In other words, an enterprise asking "what happens if I run my model in a different framework?" would find almost no systematic evidence.
Stage 2: a flourishing of open harnesses, but fragmented interfaces. Open frameworks such as OpenCode, OpenHands, mini-SWE-agent, and SWE-agent provided swappable "operating systems" [5]; the computer-use field likewise accumulated many desktop, browser, and mobile environments and datasets. The CUA-Lite authors summarize the pain of this stage: every environment has its own interface and runtime, every dataset its own format, every model its own action space and prompt format, so connecting models to environments means rewriting the same code again and again [8].
Stage 3 (October 2026): opening the whole pipeline. Work in this cycle began to put training, evaluation, and deployment on one shared open infrastructure. CUA-Lite integrates 12 benchmarks and over 30,000 verifiable tasks behind a unified environment interface, and writes one harness for each of 14 model families that serves evaluation, RL, and SFT alike [8]; Trajectory2Gym and Terrarium open up the generation process from synthetic environments to training data [3][9]; Glasseek points out that open-web agents can retrieve benchmark answers during evaluation and contaminate results, and releases an anti-leakage plugin that drops into existing evaluation harnesses [4]; and Pairing Matters, with a full-factorial experiment of 6 open harnesses × 3 lightweight models × 100 tasks (1,800 runs), turns the pairing itself into something measurable [5].
At the same time, operations research reminds us that container platforms treat a container as a disposable unit that can be replaced at any time: a stop sends SIGTERM and kills the process after a default ten-second grace period, and an update discards the writable filesystem. Agent frameworks assume the opposite, treating execution as continuous: they carry conversations across requests through a checkpointer, pause indefinitely while waiting for human approval, resume interrupted work from saved state, and warn that resumption may re-execute steps completed before the interruption, such as sending an email or charging a payment again. Each side specifies only its own layer, so whether a deployment satisfies both is settled only by its concrete deployment artifacts [11]. Between "open at training time" and "reliable in production" lies an operational contract. That is why the sections below proceed layer by layer: model, harness, training, and operations.
![October progress across the four-piece kit of open-source agents (compiled from [1]–[13]).](/assets/img/rs-open-source-agents-en-2-1100.3bc00e4030.jpg)
2. Model Layer: Opening Up at Both Ends
The on-device end: LFM3 (Liquid AI, 2026-10-27) [1]. According to official data, LFM3-1.7B uses a "triple-hybrid" architecture: of its 24 layers, 13 are gated short convolutions, 5 are sliding-window attention with a 256-token window, and 6 are global attention. It was pre-trained on 22T tokens, the last 3.5T of which were mid-training that extended context to 256K; at 4-bit quantization it is about 1 GB. Post-training has four stages: 1.75T tokens of supervised fine-tuning, multi-domain mixed reinforcement learning, multi-domain on-policy distillation from 6 expert teachers (MOPD), and agentic RL inside real harnesses. The official table shows 71.39 on BFCL V4 and 38.42 on WebArena-Verified (Qwen3.5-9B in the same table: 28.94), but 43.40 on SWE-bench Verified (OpenCode), below Qwen3.5-9B's 51.20 — Liquid itself acknowledges that "larger models still have the edge in agentic coding and knowledge tasks." Officially, the KV cache at 128K context is about 1.6 GB, versus 4.3 GB for Qwen3.5-4B in the same table. Notably, Liquid states explicitly that the model was post-trained to run in open-source harnesses such as Hermes Agent, Pi, and OpenClaw — the model is adapted to harnesses from the training stage onward.
The large-model end: Ling-3.1-flash-1019 (inclusionAI, 2026-10-19) [2]. According to the model card, it has about 560B total and about 25B active parameters, a hybrid of 63 layers of KDA linear attention and 9 layers of gated MLA, 512 experts, 1M context, and an MIT license. Compared with the previous version, further post-training alone raised its Artificial Analysis Intelligence Index from 41 to 44 and AutomationBench from 62% to 68%. The simultaneously released Ling-3.1-flash-base is also MIT-licensed, leaving room for enterprises to post-train on their own.
![LFM3-1.7B vs. models in the same official table on agentic and coding benchmarks (data from the official table in [1]; parameter counts in parentheses).](/assets/img/rs-open-source-agents-en-3-1100.af2b468cdd.jpg)
Figure 4 suggests a clear division of labor: on long-workflow tasks in browsers and business apps (WebArena-Verified, AutomationBench), the 1.7B LFM3 beats the 9.7B Qwen3.5-9B, while on SWE-bench Verified, which demands extensive knowledge and code understanding, the latter still leads by about 8 points. Tiered deployment — "small models for workflows, large models for hard problems" — is turning from an idea into a data-backed choice.
An easily overlooked fact: open weights do not equal "ready for commercial use." LFM3 on Hugging Face carries Liquid's own license (shown as "other"), while Ling-3.1 is MIT. The first step in model selection should be reading the license, not the leaderboard.
3. Harness Layer: The Pairing Matters More Than the Model
3.1 Algorithm ①: Full-Factorial Pairing Evaluation — Test "Model × Harness" as One Unit
One-line intuition: Do not rank models or harnesses in isolation; run every combination and look at how the combination performs [5].
Analogy: Racing results depend on how well driver and car match. The same driver in a different car can lap very differently, and "the best car" with a different driver may not be the fastest.
Step by step (Singapore Management University, 2026-10-23):
- Pick 6 open-source harnesses and 3 lightweight coding models, and run them under the same inference service, billing, and scoring.
- Run every combination on 100 SWE-bench Verified tasks — 1,800 scored runs in total.
- Classify the 1,100 failed runs with evidence-based failure categories (e.g., wrong localization, incomplete patch, premature termination).
- Compute each combination's resolve rate and "billed tokens per resolved task."
Key formulas:
resolve_rate(m, h) = tasks resolved / 100
token_cost(m, h) = total billed tokens of the combination / tasks resolved
pairing_gap(m) = max_h resolve_rate(m, h) − min_h resolve_rate(m, h)
- m: model; h: harness; (m, h): one combination of the two.
- pairing gap: the largest difference in a model's resolve rate across harnesses — the price of "choosing the wrong framework."
Worked example (illustrative): Model A resolves 50 tasks with harness X, using 5 million tokens — 100K per task; with harness Y it resolves 30 tasks using 30 million tokens — 1M per task. A 20-point gap in resolve rate and a 10× gap in cost per task: for the same model, both the bill and the results are decided by the harness.
Results (per the paper): The same model's resolve rate differs by up to 44 percentage points across harnesses; harness rankings change with the model; even among harnesses that all use a "model-driven loop," billed tokens per resolved task differ by more than 20×; and a model's dominant failure mode in one harness is not necessarily dominant in another [5].
Why it works, and when it fails: It changes the unit of evaluation from "model" to "pairing," matching real deployment. Its limitation is scale (100 tasks, 3 lightweight models), so the conclusions apply best to "lightweight coding agents" and should not be extrapolated directly to large models or non-coding tasks.
3.2 What Tools Do Small Models Need?
Another study tests how small models use "code graph" and "vector retrieval" tools on SWE-bench Verified [6]. Per the paper, structured tools double the localization accuracy of Qwen3-8B and Gemma 4 E4B compared with grep alone; the tool gain peaks at 17–21 percentage points at the 1.7B–4B scale and falls to 4–5 points at 32B; but agents actually "expand the code graph consecutively" only 12%–14% of the time, and the gain comes mainly from name lookups and one-hop expansion.
A finer analysis reveals how narrow small models' "field of view" is. The authors measure task difficulty in "hops": the shortest path on the code call graph from the code mentioned in the issue to the code that actually needs changing. Per the paper, each extra hop multiplies the odds of correct localization by 0.50–0.67; this decay exists at every model scale, and no tool combination flattens it. Linear probes of the models' internal representations show that direct call relations are less decodable (AUC 0.74) than a simple lexical-matching baseline (0.99), and two-hop relations reach only 0.57. In other words, small models' understanding of a code repository stays at "one hop, by name."
This leads to a counterintuitive conclusion: the smaller the model, the more it relies on the structured tools the harness provides; larger models tend not to use tools. For private deployment, choosing a small model means investing more in the harness and tooling.
An engineering field report spanning July to October 2026 corroborates this [7]: the author finds that open models nominally "compatible with the same interface" differ widely in tool-call format, turn control, context behavior, and cache reuse; median time-to-first-token for cold-start prefixes is about 69.4 seconds versus about 2.8 seconds for high-reuse requests (observational data); in a 45-trial delegation study, direct coding by a frontier model passed 15/15, while delegation to quantized Qwen3-Coder 30B-A3B worker models passed 12/15 and 14/15. The author stresses that these studies are not aggregated or used to rank models — "can generate text," "can call tools," and "code passes independent behavioral checks" are three different levels of progress.
4. Training Layer: Opening Not Just Weights, but "How to Make the Data"
4.1 Algorithm ②: Trajectory2Gym — Supervise Only Decisions with Visible Grounds
One-line intuition: A successful trajectory is not necessarily worth imitating. Trajectory2Gym uses as training targets only those actions that "can be justified from the agent's visible history," and assigns training weight per "complete interaction" rather than per "rendered sample" [3].
Analogy: When training a new employee, you cannot look only at outcomes. A customer first says "exchange both items," then changes to "only exchange the lamp"; the new hire happens to exchange both successfully, and the system reports no error — but the second action contradicts the customer's latest wish, and such a "success" should not serve as an example.
Step by step (Ant International Research, 2026-10-24):
- Compositional interaction synthesis: identify "dependency paths" that are under-supported in existing data (e.g., "after a failed lookup, retry with name and postal code"), and construct and execute new tasks around them in native environments.
- Decision-grounded supervision: check the executed trajectory step by step — is this action supported by visible evidence, consistent with the execution result, and in the right procedural order? Only actions passing the check become supervision targets; errors that actually occurred are kept as context so the model learns how to recover.
- Structural-support selection: cap the "support depth" for each capability path and select trajectories to cover capability paths, rather than simply piling up volume.
- Parent-normalized fine-tuning: one multi-turn interaction may be rendered into several training samples, and averaging over samples lets long interactions dominate the loss; instead, average within each interaction first, then across interactions.
Key formulas:
ℓ_θ(τ) = −(1/N_τ) · Σ_{u=1..N_τ} log π_θ(y_u | h_u)
L_SFT(θ) = (1/|D_s|) · Σ_{τ∈D_s} ℓ_θ(τ)
- τ: one complete semantic interaction (the "parent"); D_s: the selected set of interactions, |D_s| being their number.
- (h_u, y_u): the u-th (visible history, target tokens) pair certified by the decision-grounding check; N_τ: the number of such pairs in this interaction.
- π_θ: the model being trained; log π_θ(y_u | h_u): log-probability of generating the correct target y_u given history h_u.
- Meaning: every interaction carries the same total weight, regardless of how many samples it is cut into.
Worked example: The corpus has two interactions; the first is rendered into 3 samples, the second into 1. Averaging over samples gives the first interaction 3/4 of the weight; parent normalization gives each 1/2.
Results (per the paper): Across five Qwen3 sizes (1.7B to 30B), all three benchmarks — Tau1, Tau2, BFCL — improve, with the equal-weight average of the three rising by 16.9–29.8 percentage points; the post-trained 4B model reaches 69.17 on Tau2, above the 28.33 of the publicly initialized 30B-A3B [3]. In stage-by-stage ablations, parent normalization alone contributes an 8.04-point mean gain, the largest of any stage; and support depth is not "the larger the better" — at 8B, depth 2 achieves a three-benchmark mean of 72.28, above depth 3 (70.55) and depth 4 (69.44).
Why it works, and when it fails: It tackles both "learning the wrong thing" (actions without grounding checks) and "learning a skewed thing" (overweighted long interactions). Failure conditions: it needs executable, replayable native environments; grounding rules must be customized to business workflows; and the paper reports a 6–13 point drop on simple single-call queries (BFCL Live Simple), where the model tends to gather information first.
![Trajectory2Gym's four stages and per-stage gains (data from [3]).](/assets/img/rs-open-source-agents-en-4-1100.83709b4d50.jpg)
4.2 Algorithm ③: Glasseek's CM-GSPO — RL Across the "Context Compression" Boundary
One-line intuition: A deep-search agent must read dozens to hundreds of web pages, so context must be compressed (summarized). CM-GSPO lets the same policy handle both searching and summarizing, computes policy gradients only on "execution" segments, and corrects the weighting bias introduced by per-segment averaging [4].
Analogy: A researcher writes interim summaries while investigating; when a notebook page fills up, only the summary is kept and a new page begins. Evaluating the researcher depends on the conclusion of the whole investigation, but training needs to know "which step on which page" was right — and an investigation should not count for more just because it spanned more pages.
Step by step (2026-10-12):
- The agent searches, reads, and reasons; when context nears its limit, the same model writes a summary that replaces the old context, producing several "execution sub-trajectories."
- After the whole task ends, the reward comes from the final answer, and within-group comparison gives the advantage (GSPO computes importance ratios at the sequence level).
- Loss is computed only on execution sub-trajectories, using length-normalized likelihood ratios so long segments do not dominate.
- Reweighting at the trajectory level corrects the double counting caused by "averaging each sub-trajectory separately" — a task should not gain more weight just because it was chopped into more pieces.
Key idea (simplified):
For sub-trajectory k of task i: s_{i,k} = ( π_θ(o_{i,k}) / π_old(o_{i,k}) )^(1/|o_{i,k}|)
Loss of task i: L_i = (1/K_i) · Σ_k clip(s_{i,k}) · Â_i
Total loss: L = (1/G) · Σ_i L_i
- o_{i,k}: the k-th execution sub-trajectory of task i; |o_{i,k}| is its length, and the |o|-th root gives length normalization.
- π_θ / π_old: current policy and the old sampling policy; s is the sequence-level importance ratio; clip is the usual clipping to prevent oversized updates.
- Â_i: the within-group relative advantage of task i; K_i: number of sub-trajectories task i is split into; G: group size.
- This is a simplified sketch of the paper's idea; see the original for the exact weighting.
Worked example: Two search tasks: task A is compressed once (2 segments), task B four times (5 segments). If each segment is averaged separately and then summed, B's weight is 2.5× A's; averaging within each task makes them equal.
Results (per the paper): Under strict leakage-proof evaluation, Glasseeker-9B scores 61.6% on BrowseComp, versus 59.2% for Gemini-3-Pro and 52.0% for Qwen3.6-27B in the same table; 50.0% on Seal-0, versus 42.3% for GPT-5.2; and 31.3% on LiveBrowseComp [4].
The other side of evaluation: leakage audits. The paper runs a systematic test-set leakage audit on open web agents: without safeguards, some models actively search for benchmark answers and even open dataset records on Hugging Face to read the gold-answer field; with domain filtering and file-system permission limits, these models' accuracy drops noticeably [4]. The authors therefore release a leakage-prevention plugin that slots into existing harnesses. This is a warning for every enterprise that relies on open-source agent leaderboards.
Why it works, and when it fails: Summarization is unavoidable in long-horizon tasks, and CM-GSPO aligns "post-compression behavior" between training and deployment. The risk is that summary quality itself is not directly constrained by the reward; if a summary drops key information, no amount of good execution afterwards can recover it.
4.3 Other Open Recipes
- Terrarium (2026-10-10) [9]: synthesizes 275 resettable "app worlds" and 13,344 tasks with outcome rubrics from public web pages and in-app observations, for outcome-reward RL; per the paper, Qwen3 4B/8B/14B improve on average by 18.34 points on τ²-bench and 10.94 points on AppWorld.
- CUA-Lite (2026-10-19) [8]: integrates 12 benchmarks and over 30,000 tasks, implementing for each of 14 model families one harness shared across evaluation, RL, and supervised fine-tuning.
- FlashNext-RL (2026-10-20) [13]: optimizes the RL system for a specific open model; on a single 8×B200 node, training throughput rises from about 395 to 4,438 tokens/s/GPU (with context extended from 32K to 131K). Conclusion: RL for open models needs infrastructure tailored to the model architecture.
5. Operations Layer: The Real Exam Starts After Launch
5.1 Algorithm ④: MCP-Injector — Fault Injection Following the Real Fault Distribution
One-line intuition: Benchmarks assume tools always return correct results on time; MCP-Injector injects faults into the tool-call path following the real fault distribution, testing whether agents can withstand them [10].
Analogy: A fire drill cannot rehearse only "small flames"; scenarios should follow historical fire statistics — which kinds of fire are most common and how long they last.
Step by step (2026-10-23):
- Mine the real fault distribution: from 57,240 issue and PR discussions in 1,774 open-source MCP repositories, compute the distribution of "fault type × persistence."
- Define persistence: permanent (always fails), transient (fails once, then recovers), intermittent (randomly recurring).
- Reproducible injection: insert as middleware into the MCP call path, with a random seed controlling injection timing.
- Full-factorial experiment: 2 model sizes (20B, 120B) × 3 agent frameworks × 3 injection conditions, for 1,350 experimental units.
Key formula:
Resilience ratio R = task-completion score with faults / task-completion score without faults
- Task-completion score: a 10-point task-quality score; the closer R is to 1, the better the agent withstands faults.
Worked example: Framework X scores 8 without faults and 4 with faults, R = 0.5; framework Y scores 7 and 5, R ≈ 0.71. Y's "raw score" is lower, but it may be more reliable in a real environment.
Results (per the paper): Under a stress configuration dominated by permanent faults, the task-completion score falls from 8.08 to 3.23, a 60% drop; the 120B model does not retain more of its fault-free score (resilience-ratio model p = 0.96), and its completion rate under faults is even slightly lower (91.1% vs. 95.6%, a 4.4-point difference, 95% CI [−0.2, 9.3]); agent frameworks differ in their ability to absorb faults — framework-level error handling matters more than model scale [10].
![Three "hidden costs" after launch: tool faults, state loss, and energy (data from [10][11][12]).](/assets/img/rs-open-source-agents-en-5-1100.eb5ce0040e.jpg)
Why it works, and when it fails: It extends "capability evaluation" into "capability + resilience" evaluation. The limitation is that only two sizes of the same model family were tested, so "scale does not buy fault tolerance" should be read as "within the tested range."
5.2 State, Energy, and Self-Hosting
- The "amnesia" of containerized deployment [11]: a study of containerized agent deployments finds that among systems needing to retain state across processes, 37.4% lose state on restart and 41.5% on container replacement; after actively triggering events such as restarts, 14 of 16 predicted violations turned into real runtime failures; 78% of developer fixes required changing application code, and only about two-thirds fully restored the violated guarantee (per the paper).
- Quantization does not necessarily save energy [12]: 4-bit quantization usually lowers per-token energy, but completing a task may take more tokens; per the paper, energy per completed task can be up to 3.5×. The accounting formula is simple: energy per task = energy per token × tokens needed per completed task — looking only at the first factor leads to the wrong conclusion.
- A self-hosted research assistant [14]: MuninAI wraps open-weight models in a harness with reading, retrieval, and computation tools. Per the paper, on 199 paired literature questions, the harness lifts base-model accuracy from 0.387 to 0.874, while a naive "retrieve-then-answer" approach performs worse than the model with no retrieval at all; in a test with 300 fabricated references, it generated no locally fabricated citations; with all external network access disabled, two Qwen backbones still exceed PaperQA2's publicly reported accuracy. The authors' core claim matches this article's: what makes the system useful on domain literature is the harness, not any particular model.
6. Side-by-Side Comparison
Table 1. Open-weight models covered in this issue
← Swipe to see the full table →| Model | Institution / date | Parameters (total / active) | Context | License | Representative results (per official or paper data) |
|---|---|---|---|---|---|
| LFM3-1.7B [1] | Liquid AI / 10-27 | 1.7B dense (triple-hybrid) | 256K | Liquid's own license | BFCL V4 71.39; WebArena-Verified 38.42; ~1 GB at 4-bit |
| Ling-3.1-flash-1019 [2] | inclusionAI / 10-19 | ~560B / ~25B | 1M | MIT | AA Intelligence Index 41→44; AutomationBench 62%→68% |
| Glasseeker-9B [4] | Glasseek team / 10-12 | 9B | — | See project repo | BrowseComp 61.6%; Seal-0 50.0% |
Table 2. Representative methods at each layer of the kit
← Swipe to see the full table →| Layer | Work | Core mechanism | Key numbers (per paper) | Meaning for enterprises |
|---|---|---|---|---|
| Harness | Pairing Matters [5] | 6×3 full-factorial pairing evaluation | Up to 44 points apart for the same model; >20× token cost gap | Test combinations, not single parts |
| Harness | Small models and code graphs [6] | Tool ablation + hop stratification | Gain of 17–21 points at 1.7B–4B, only 4–5 at 32B | Small models rely more on tools |
| Training | Trajectory2Gym [3] | Decision-grounded supervision + parent normalization | Three-benchmark mean +16.9–29.8 points | Audit synthetic data at the action level |
| Training | Glasseek [4] | CM-GSPO + leakage-proof evaluation | BrowseComp 61.6% | Check leaderboards for leakage |
| Training | Terrarium [9] | Synthetic resettable app worlds | τ²-bench +18.34 points | Train on simulations instead of production systems |
| Operations | MCP-Injector [10] | Fault injection by real distribution | 8.08→3.23; scale brings no fault tolerance | Prioritize framework error handling |
| Operations | Container contracts [11] | 14 operational contracts + fault injection | 37.4% / 41.5% lose state | Test state persistence specifically |
| Operations | Quantization energy [12] | Per-task energy accounting | Up to 3.5× | Account per task |
Note: Tasks, benchmarks, and hardware differ across works, so numbers are not directly comparable.
7. Our Views
First, the unit of competition among open-source agents is shifting from "model" to "pairing." A 44-point pairing gap [5], LFM3's deliberate adaptation to specific harnesses [1], and CUA-Lite's separate harness for each model family [8] all point to one conclusion: enterprises should not ask "which open model is best" but "which model with which framework is best on my tasks."
Second, the quality bar for training data is rising from "correct outcome" to "grounded process." Trajectory2Gym's example shows that even successful trajectories can hide actions that contradict the user's latest wishes [3]. In regulated industries, such "right result, wrong process" samples are even more harmful.
Third, open-source leaderboards should be "audited before trusted." The retrieval-based leakage Glasseek uncovered [4] is not an isolated case; as long as an agent can browse and read files, it may reach the answers. We recommend making a leakage-proof configuration a prerequisite for trusting any open-source agent score.
Fourth, the real cost of private deployment lies in operations. A 60% quality loss under faults [10], nearly 40% state loss [11], and up to 3.5× energy per task [12] never appear on a model card. Operations and testing should take a substantial share of any private-deployment budget.
8. Outlook
6–12 months:
- Joint "model + harness" releases will become the norm, and model cards will list recommended harnesses and configurations, much as they list recommended sampling parameters today.
- Leakage-prevention plugins and fault-injection middleware will be merged into mainstream open-source harnesses as default switches.
- On-device 1–4B agent models will be deployed at scale on narrow tasks such as form processing, email triage, and ticket routing, while complex programming and knowledge-intensive tasks remain with larger models.
1–3 years:
- Training-data synthesis will move toward "environment as data": enterprises will use simulated versions of internal systems as training grounds (like Terrarium's app worlds), training dedicated agents without touching production data.
- Agent operations will become an engineering discipline of its own, akin to "observability + chaos engineering," with state persistence, idempotency, and fault resilience as launch criteria.
- The risk is fragmentation: models, harnesses, and tool protocols evolving separately will keep increasing the compatibility-testing burden, and the industry will need more unified interface specifications and pairing benchmarks.
9. Implications for Financial Risk Management and Enterprise Deployment
- Run "pairing experiments" for selection. In private-deployment evaluations, use 50–100 of your own real tasks to test combinations of 2–3 models × 2–3 harnesses, recording both resolve rate and token cost per task.
- Audit the process in training data. Synthetic or collected agent trajectories should be checked step by step: does each action have visible grounds, and does it follow the approval workflow order? In credit-granting and anti-fraud workflows, this matters more than outcome correctness.
- Run fault drills before launch. For tools such as data interfaces, rule engines, and external credit-bureau queries, inject timeouts, erroneous responses, and sustained unavailability according to historical fault statistics, and measure the resilience ratio.
- Account for cost per task. Savings from quantization, small models, and local deployment should be validated by "energy and compute per completed business transaction," not price per token.
Hongxi AI has long helped financial institutions build model and rule systems, and we see that the value of a privately deployed agent lies not in "whether it runs," but in "whether it can be audited, reproduced, and kept from failing under faults."
10. Conclusion
Open-source agents are shifting from "opening a model" to "opening a whole reproducible engineering system." For enterprises, this is an opportunity — and it also means responsibility for selection and operations moves from the vendor to themselves. Only by doing pairing evaluation, process auditing, leakage auditing, and fault drills properly can they truly enjoy the control and autonomy that open source brings.
Visit Hongxi AI at https://www.hxallai.com.
References
- [1] Liquid AI. Introducing LFM3: Every Modality for Specialized Intelligence. Liquid AI Blog, 2026-10-27. https://www.liquid.ai/blog/lfm3
- [2] inclusionAI. Ling-3.1-flash-1019 (model card). Hugging Face, 2026-10-19. https://huggingface.co/inclusionAI/Ling-3.1-flash-1019
- [3] Yihao Hu, Zhihao Wen, Yitong Lin, Siyuan Lu, et al. Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agents. Ant International Research, 2026-10-24. https://arxiv.org/abs/2610.33568
- [4] Cong Pang, Jiaqi Su, Chengrui Wu, Kuihao Yan, et al. Glasseek: A Transparent Recipe for Training and Trustworthy Evaluation of DeepSearch Agents. arXiv, 2026-10-12. https://arxiv.org/abs/2610.16998
- [5] Lu Zhao, Christopher M. Poskitt, David Lo. Pairing Matters: Performance, Failures, and Efficiency in Lightweight Coding Agents. Singapore Management University, 2026-10-23. https://arxiv.org/abs/2610.32131
- [6] Saurav Bellam. How Do Small Coding Agents Use Code Graphs and Embeddings? arXiv, 2026-10-25. https://arxiv.org/abs/2610.33958
- [7] Vaibhav Jain. Beyond Benchmarks: Can Small Open-Weight Models Get Real Coding Work Done? arXiv, 2026-10-25. https://arxiv.org/abs/2610.34241
- [8] Zhanhui Zhou, Weichen Zhang, Haoran Liu, Lingjie Chen, et al. CUA-Lite: An Open Platform for Computer Use Agents. arXiv, 2026-10-19. https://arxiv.org/abs/2610.26071
- [9] Mengzhou Wu, Yuzhe Guo, Jiawen Tao, Xiaokun Yuan, et al. Terrarium: Growing Personal Assistants via Scalable Synthesis of App Worlds. arXiv, 2026-10-10. https://arxiv.org/abs/2610.14860
- [10] Antonio Ken Iannillo, Joshua Segun Owotogbe, Roberto Natella, Francesco Avallone, et al. Tool-Calling LLM Agents Under Realistic MCP Faults: Scale Does Not Purchase Fault Tolerance. arXiv, 2026-10-23. https://arxiv.org/abs/2610.32555
- [11] Meriem Mastouri, Emna Ksontini, Amine Barrak. Built to Remember, Shipped to Forget: Operational Contracts, Failures, and Repairs in Containerized LLM Agent Deployments. arXiv, 2026-10-25. https://arxiv.org/abs/2610.34620
- [12] Stephen Abkin, Aldric Yang, Alex Flesher, Nitheesh Kodarapu, et al. Measuring the Energy Cost of Completing Tasks with Quantized Language-Model Agents. arXiv, 2026-10-23. https://arxiv.org/abs/2610.32417
- [13] Zhewei Yao, Yuxiong He. FlashNext-RL: A System Optimization Study for Long-Horizon Agentic RL of Qwen3.8-Flash-Next. arXiv, 2026-10-20. https://arxiv.org/abs/2610.28998
- [14] Markus Fischer, Jens Meiler, Matthias Elgeti. MuninAI: A Self-Hosted Agentic Framework for Independent Research Groups. arXiv, 2026-10-16. https://arxiv.org/abs/2610.23480
Research · Frontier AI Deep Dive · Great Unicorn Intelligence