The Credit Problem of Long-Horizon Agents: How Do You Split One Success or Failure Across Dozens of Decisions? (In-Depth Review)

1. Background: why "one score" isn't enough

The mainstream starting point for agentic RL today is group-relative methods like GRPO [1]: sample a group of trajectories for the same problem, subtract the group mean from each trajectory's final score to get an "advantage," then broadcast that value unchanged to every token in the trajectory. For a math problem of a few hundred tokens, that's mostly fine. But for an agent that searches a dozen times and calls tools dozens of times, one shared advantage means the few wrong searches in a successful trajectory get rewarded, and the one crucial correct search in a failed trajectory gets punished.

The authors of TWICE provide a direct measure. Per the paper, at update 200 while training Qwen2.5-3B, they took the 1,036 decisions visited on 400 held-out multi-hop questions, estimated each decision's "true advantage" with 128 extra samples, then checked whether each method's credit pointed in the right direction. Among decisions with non-negligible advantage, GRPO's trajectory-level credit had the wrong sign 31.8% of the time; tree-search siblings 24.9%, independent counterfactual branches (CRR) 21.7%, VinePPO 18.6%, and TWICE 6.9% [4]. Roughly three in ten decisions get credit "backwards," which is a major reason long-horizon training is inefficient.

Picture a football team where every player gets the same rating based only on the final score. When the team wins, the defender who let in a goal gets a high rating too; when it loses, the striker who scored gets marked down. The longer the match and the more passes, the bigger the error from this "everyone gets the same" approach. The more turns an agent takes, the more pronounced the problem — which is why nearly every October paper put "long-horizon" in its title.

Two classic approaches already existed:

• Resampling for value estimates: VinePPO resamples several continuations from intermediate states and estimates the state's value from Monte Carlo returns [2]. Accurate, but expensive.
• Same-state comparison: GiGPO finds steps in a group of trajectories that "land in the same environment state," compares them within the group, and gets step-level advantages [3]. Cheap, but only usable when states repeat.

October's research can be sorted into five classes by "where the credit comes from":

1. Branch counterfactuals: rerun and see what happens with a different decision [4][8].
2. Sharing and calibration: reuse continuation evidence already in the batch to calibrate the noise of single continuations [7].
3. Learned critics: train a value network to predict "will it succeed if it continues from here" [8][9][10].
4. Self-generated and evidence signals: have the model point out key turns itself, or build process rewards from evidence annotations in the data [5][15].
5. Distillation and update-side corrections: use a teacher or hindsight skills to give per-token signals [12][13][14], or fix "how credit lands on parameter updates" [6][10].

2. Algorithm breakdown ①: TWICE — same luck, different choice

2.1 Intuition

To know "how much the search in turn 5 was worth," the direct approach is a counterfactual: replace turn 5 with a different query, run the rest as usual, and compare the two outcomes. The problem is that "the rest as usual" is hard to achieve. Every later step in the branch trajectory draws fresh random numbers, so the difference between the two outcomes mixes the effect of "changing turn 5" with the noise of "different luck later." To detect a small advantage, you need many runs.

The observation behind TWICE (Twin-World Credit Estimation, 2026-10-22) [4] is that in agentic search tasks the retriever, corpus and grader are all deterministic, so a trajectory is fully determined by "the policy's decisions + sampling noise." If the branch trajectory reads exactly the same noise in later steps, you get a "twin world": the same luck, with only one decision different.

2.2 Analogy

To compare two chefs fairly, give them the same ingredients, the same pan and the same stove. If one uses fresh vegetables and the other uses day-old ones, you can't tell whether the dish is good because of skill or ingredients. What TWICE does is give the counterfactual branch "the same ingredients."

2.3 Steps

1. Index the noise. Use Gumbel-max sampling: each token takes argmax[logits/τ + ξ]. Noise ξ is no longer generated from a global token counter; it comes from a key κ = (episode e, turn t, line ℓ, position-in-line j) through a counter-based pseudorandom function.
2. Build the twin. At turn t, draw an alternative decision y′, execute it, and have every later turn read noise with the same keys as the original trajectory. Because keys are indexed by "turn + line," later turns stay aligned even if turn t's length changes; a divergence within a line resynchronizes at the next line.
3. Take the difference. Turn credit D_t = U − U′_t (original outcome minus twin outcome).
4. Allocate the budget. Use the square-root rule q_t ∝ √(η̂_t / c_t) to decide where to build twins: turns where the two outcomes are more likely to differ and that cost less get more budget. Then correct for the sampling probability with a Horvitz–Thompson estimator.
5. Update the policy. Apply the turn advantages to the policy-gradient update on the original (actor-sampled) trajectory; deployment uses only the actor itself [4].

for episode e in batch:
traj = rollout(policy, keys=(e, t, line, pos))     # keyed Gumbel noise
for turn t in traj:
if rand() < q_t:                               # q_t ∝ sqrt(eta_t / cost_t)
y_alt = sample(policy_old, state_t, fresh_keys)
twin  = replay_from(t, y_alt, same_keys_after_t)
D_t   = traj.U - twin.U
A_t   = D_t / q_t                          # Horvitz–Thompson correction
update(policy, traj, A)

2.4 Theory and results (per the paper)

The paper proves three things [4]: twin credit is unbiased under any coupling quality; its variance equals the variance of an independent counterfactual times (1 − ρ), where ρ is the correlation between twin and original outcomes; and under coupling, the number of samples needed to detect an advantage of size Δ drops from Θ(Δ⁻²) to Θ(Δ⁻¹). Measured ρ̄ = 0.57, so mean squared error drops correspondingly to 1 − ρ̄ = 0.43×.

On seven QA benchmarks, TWICE raises average exact match over GRPO by 5.3 and 4.5 points for Qwen2.5-3B and 7B, beats nine RL baselines, uses 68% fewer auxiliary tokens than VinePPO, and takes 152 s vs. 262 s per update. Keyed sampling is just a logits processor, adding about 2% to generation time. Its correlation with true advantages is 0.72, vs. 0.51 for VinePPO [4].

2.5 Small example (illustrative)

A two-hop question: "Where was the director of X born?" Turn 1 searches "director of X," turn 2 searches "[name] birthplace." The original trajectory answers correctly, U = 1. Replace turn 1 with a search for "release date of X," keep the later noise the same, and the twin answers wrong, U′ = 0, so D₁ = 1: turn 1 gets the credit. If instead it's replaced with a synonymous query like "who directed X," the twin also answers correctly, D₁ = 0: the exact wording of this turn wasn't critical.

2.6 When it fails

Twin worlds rely on the environment being deterministic. Search tasks with a deterministic retriever, corpus and grader satisfy this. If tool calls return time-dependent or random results (live web pages, concurrent systems, online transactions), the "same luck" afterward can't be reproduced, ρ drops, and the variance advantage shrinks with it [4]. TWICE also needs an inference server whose sampler can be modified, so it doesn't apply when you can only call closed-source APIs.

3. Algorithm breakdown ②: Self-Credit — let the model point out key turns itself

3.1 Intuition

To avoid overflowing their context, long-horizon agents often "compact" periodically, summarizing earlier interactions into a digest. The idea behind Self-Credit (2026-10-20) [5]: since the model has to write a summary anyway, have it cite the turns containing key evidence in that summary (for example, "based on the search results in turns 1 and 3…"). Those citations are the model's own credit assignment.

But a model's self-assessment can't be fully trusted; directly optimizing model-generated feedback invites gaming and instability. Self-Credit filters with intra-group rescaling: put multiple trajectories for the same problem together, match turns by similar actions, and count how often a cited action appears in successful vs. failed trajectories.

3.2 Analogy

In a project retrospective, everyone says their work was crucial. The reliable approach is to compare. A step that appears in successful projects but not failed ones is the real key. A step that appears in both is just "standard procedure" and shouldn't get extra credit. A step that keeps appearing only in failed projects deserves serious reflection.

3.3 Steps and formulas

For each cited turn t that can be matched across trajectories, set a weight w_t (per the paper's settings [5]):

Then replace GRPO's uniform broadcast with the weights: every token in a turn gets Â_{t,ℓ} = Â · w_t. Summary turns and final-answer turns in successful trajectories are automatically treated as cited; summary and answer turns don't take part in cross-trajectory comparison [5].

# simplified sketch; see the original paper for matching and counting details [5]
for prompt in batch:
G = rollouts(policy, prompt, compaction_with_citations=True)
A = grpo_advantage(G)
for turn t cited in any rollout:
succ, fail = count_matched(t, G.success), count_matched(t, G.failure)
if succ > 1 and fail == 0:   w[t] = w_s
elif fail > 1 and succ == 0: w[t] = w_f(step)      # annealed
elif succ > 0 and fail > 0:  w[t] = 1.0
else:                        w[t] = beta if A > 0 else 1.0
token_adv = A * w[turn_of(token)]

3.4 Results (per the paper)

On BrowseComp-Plus, Self-Credit improves Qwen3 8B and 32B by 37.0 and 36.2 points over the base models, and by 8.5 and 5.6 points over the strongest agentic-RL baseline. On GAIA and xBench-DeepSearch it beats the strongest baselines by 4.9 and 4.7 points. Ablations show that intra-group rescaling — especially the handling of repeated turns in failed trajectories — matters across model scales [5].

3.5 When it fails

The method relies on turns being matchable across trajectories. In search tasks, retrieval actions for the same question are often similar, so match rates are high. In tasks with large action spaces where every trajectory is unique (such as open-ended writing), few turns match and rescaling rarely triggers. Also, citations are model-generated; when the model is weaker, citation quality drops and the weights become less reliable [5].

4. Algorithm breakdown ③: EMPO — the same action shouldn't get two update directions

4.1 The problem: credit can be right and the update still wrong

The first two methods answer "how much credit each step deserves." EMPO (Effect-Marginal Policy Optimization, 2026-10-21) [6] points out a subtler problem: even when credit is computed correctly, the way it lands on parameter updates can be wrong.

In a household environment like ALFWorld, the model can write "Go to fridge 1," or first write "Let me check the fridge" and then output the same action. The two texts have exactly the same effect when executed, but in PPO-style objectives each has its own probability ratio. If one trajectory later succeeds and the other fails, the same action gets pushed up on one side and down on the other just because it was "phrased differently." The authors call this credit–update misalignment and prove that even with the policy context and execution effect fixed, the way credit is paired with text directly changes the update direction [6].

4.2 Analogy

Two employees submit the same expense report in different words. One is praised and the other criticized, simply because "the project happened to succeed / fail this time." What everyone learns next time isn't "should this be expensed" but "which wording to use" — clearly not what we want to teach.

4.3 Steps

1. Within the same local comparison group, put responses with the same execution effect (mapping to the same canonical action) into one class K.
2. Compute a length-normalized ratio for each response: q_i = exp((1/T_i)·Σ_t Δlog p_θ,t).
3. Average the ratios within the class to get a shared "effect ratio" q_K = (1/|K|)·Σ_{i∈K} q_i.
4. Combine the shared ratio q_K with each response's original advantage Â_i in the objective ℒ(q_K, Â_i). Trajectories, rewards and the advantage estimator are all unchanged; no extra critic or sampling is needed [6].

groups = group_by_canonical_effect(responses)        # the various phrasings of "Go to fridge 1"
for K in groups:
q = [exp(mean(logp_new[i] - logp_old[i])) for i in K]
q_K = mean(q)
for i in K:
loss += clipped_objective(q_K, A_hat[i])      # shared ratio, each keeps its own advantage

4.4 Results (per the paper)

On Qwen2.5-1.5B-Instruct, EMPO-G raises GRPO's ALFWorld success rate from 69.5% to 83.3% and WebShop from 62.8% to 70.3%. EMPO-GiG raises GiGPO from 90.1% to 94.0% (ALFWorld) and from 73.8% to 77.3% (WebShop). On Sokoban with Qwen2.5-VL-3B-Instruct, EMPO-G goes from 72.7% to 84.4% and EMPO-GiG from 75.0% to 81.3% [6]. The gains also hold at 7B scale.

4.5 When it fails

EMPO needs a function that maps text to a "canonical execution effect." That's natural in environments with fixed action formats (household tasks, shopping, box-pushing). With free-form tool arguments or code edits, deciding whether two pieces of code are equivalent is itself hard, and misclassification would force different actions to be tied together.

5. Other routes: calibration, critics, process rewards and distillation

5.1 Sharing continuation evidence: ConCal

GiGPO compares actions at shared states, but each action's value comes from a single sampled continuation, and chance decisions later on distort earlier credit. ConCal (2026-10-27) [7] matches visits within the same task that "pass through the same state–action–state transition," pools their continuation evidence, and propagates the calibrated values backward, with no extra sampling. Per the paper, on Qwen2.5-1.5B-Instruct ConCal's success rates on ALFWorld and WebShop are 3.39 and 5.86 points above GiGPO [7].

5.2 Actively learned critics: FOCUS and SCALE

FOCUS (2026-10-26) [8] trains the critic with "selective branching": the budget for sampling continuations goes first to states expected to reduce the critic's estimation error more. It then uses the critic's values with TD(λ) to score segments between adjacent states, and uses the segment boundaries created by context compaction as the unit for value estimation and credit assignment. Per the paper, with the same per-step generation budget, Qwen3.5-9B reaches 55.4 on BrowseComp-Plus (strongest baseline 48.2) and 56.5 on DeonticBench (baseline 52.4) [8].

SCALE (2026-10-20) [9] argues that a critic whose return predictions are accurate overall can still fail to tell good local actions from bad ones. It has the critic predict both return and a "success value" (the probability of eventually completing the task from here) at each prefix, turns changes in success value into multi-step action credit via Success-GAE, and scales their contribution by the predictor's reliability. Per the paper, across five agent benchmarks and three model backbones it beats the comparison methods in most settings, raising success rates by up to 4.1 points [9].

MPO (2026-10-26) [10] observes that the critic's value changes mainly "between turns," so it anchors both advantage estimation and importance sampling at the turn level, then allocates more finely across tokens within each turn. The paper reports consistent gains over strong baselines for 4B and 30B-A3B models on search, tool-use and automation tasks [10].

5.3 Process rewards: very useful when there's evidence

EG-GRPO (2026-10-18) [15] uses the evidence annotations in synthetic deep-search data to find the step where each required piece of evidence is first cited, builds step-level process rewards from "newly acquired evidence," and uses the accumulated evidence as a knowledge state for state-conditioned relative advantage estimates. The paper reports overall gains over standard GRPO and other step-level baselines on five deep-search benchmarks [15].

5.4 Distillation: turning hindsight into per-token signals

• ToolOPD (2026-10-19) [12] finds that on-policy distillation assumes by default that "credit is spread evenly over the trajectory," but in practice credit concentrates on a few key actions; merely flipping the final outcome significantly changes the credit of several turns. It summarizes each completed trajectory and its outcome into a "hindsight skill," measures how the skill changes the teacher's probability for the student's actions, and turns that into outcome-aware credit used to weight the distillation loss [12].
• Distill Until Divergence (2026-10-16) [13] further notes that once a wrong action changes later states, hindsight-skill supervision may no longer match the current state. So it routes by the "error boundary": success experience reinforces successful trajectories and decisions before the error, failure correction targets the decisive mistake, and the part after the error is optimized only with GRPO [13].
• RIPPLE (2026-10-22) [14] targets coding agents that "can write a plausible critique but keep going down the wrong path." It uses a prefix-only teacher to provide recovery supervision, assigns credit separately to three stages (triggering reflection, diagnosis and corrective action), and gates trajectory advantages with local validity signals. Per the paper, on four repository-level software engineering benchmarks it improves on the base policy by an average of 4.8 points, with no auxiliary critic needed at deployment [14].

6. When fine-grained credit backfires

When Fine-Grained Credit Assignment Helps and Hurts (2026-10-26) [11] uses movie-trailer editing as a testbed, with four task types: clip selection, narrative ordering, rough cut and fine cut. Per the paper:

• On "fine cut," which averages over 60 turns, decayed process rewards give a 45.4% relative gain over outcome-only GRPO. On the other three task types, averaging 30–40 turns, outcome-only GRPO remains competitive, and decayed process rewards give at most a 10.4% relative gain.
• Process rewards from an LLM judge initially improve tool use and encourage visual checks. But with a fixed weight, as training proceeds, over-reasoning and redundant tool calls appear: process rewards keep rising while task completion and outcome scores fall. Decaying the process-reward weight to zero mitigates this reward hacking while keeping the early gains.
• On-policy self-distillation (OPSD) provides dense per-token supervision but yields almost no gain, or even clear degradation, and is highly sensitive to the choice of privileged information. The authors suggest long trajectories may dilute the useful signal [11].

7. Side-by-side comparison

Note: models, tasks and budgets differ across methods, so the numbers are not directly comparable.

Engineering practice: four checks before adopting a new credit method

1. Measure "credit-sign accuracy" first. Follow TWICE's audit approach [4]: take a batch of held-out problems, estimate reference advantages for the visited decisions with more resampling, then check whether the candidate method's credit signs agree. This doesn't depend on final benchmark scores and reveals earlier whether a method "assigns credit correctly."
2. Then look at what the extra cost is made of. Resampling, critic training, teacher forward passes and sampler changes each consume different resources. The "auxiliary tokens," "time per update" and "added generation time" reported in papers [4] should be re-measured on your own hardware and inference stack.
3. Check whether the preconditions hold. Is the environment deterministic? Can turns be matched across trajectories? Can actions be mapped to canonical effects? Does the data carry evidence annotations? These four questions correspond to the applicability conditions of TWICE, Self-Credit, EMPO and EG-GRPO respectively [4][5][6][15].
4. Monitor divergence between auxiliary and outcome signals. Whether it's process rewards, model self-assessment or teacher signals, if the auxiliary score keeps rising while task completion falls, pause and look for gaming [11].

What these four checks have in common is that they answer "which decisions' credit did the new method actually change," instead of looking only at a final average score. For teams with limited training budgets, running a small audit first and then deciding whether to switch over fully is the safer path.

8. Our view

First, "assigning credit" is worth more than "assigning scores." In the same successful trajectory, three in ten decisions may get credit backwards [4]. When the reward is already fixed, improving credit assignment is often more direct than designing new rewards, and introduces less gaming.

Second, there is no universal solution, only solutions that fit the environment. Use twin counterfactuals when the environment is deterministic [4]; use same-state comparison and shared calibration when actions repeat a lot [3][7]; use evidence-based process rewards when you have evidence annotations [15]; add the effect-marginal ratio when the action format is fixed [6]. The first step in choosing is to understand your own environment, not to chase new methods.

Third, process rewards need an "exit mechanism." Process signals can speed up learning early in training and be gamed later [11]. Weight decay, intra-group controls [5] and error-boundary truncation [13] are essentially the same thing: making auxiliary signals give way to real outcomes once they've done their job.

Fourth, update-side problems are easy to overlook. EMPO shows that even when credit is right, parameter updates can go astray [6]. When evaluating a training pipeline, add "do different phrasings of the same action get consistent updates?" to the checklist.

9. Outlook

6–12 months:

• Inference frameworks will more commonly expose "keyed sampling" interfaces, making twin counterfactuals a routine tool rather than a research prototype [4].
• Context compaction will be designed together with credit assignment: segment boundaries and summary citations from compaction will directly become the units for value estimation and credit [5][8].
• Agentic-RL papers will report diagnostic metrics such as "credit-sign accuracy" more often, not just final success rates [4][7].

1–3 years:

• For real environments with randomness, "partial twin" methods may emerge: sharing noise only for replayable tool calls, and filling in the rest with learned critics [4][8][9].
• Process rewards will move from "fixed weights" to "adaptive by training stage and trajectory length" [11].
• Credit assignment will extend to the data side: which training tasks and which trajectories contribute more to final capability is also a question counterfactuals and intra-group comparison can answer.

10. Implications for financial risk control and enterprise deployment

1. First decide whether the environment is replayable. Rule engines, historical backtests and simulated credit-approval flows are mostly deterministic environments, suited to precise credit methods like twin counterfactuals [4]. Flows involving live external data are better suited to critics or intra-group comparison.
2. Build a "canonical action table" for business actions. Approval-agent actions like "return for more documents" or "escalate to manual review" often have many textual phrasings. A canonical mapping supports EMPO-style update alignment [6] and makes audit statistics easier.
3. Build process rewards from evidence annotations. Risk control naturally requires that "conclusions must have evidence," and the position where evidence is first cited can serve directly as a step-level signal [15].
4. Process rewards must be monitored against completion rates. If "process scores rise while business outcomes fall," that's a sign of gaming; decay the process weight promptly [11].
5. Credit results can feed back into rules. Decision steps repeatedly credited as "key" often correspond to business experience that can be captured as explicit rules. In its rule-engine and intelligent-decision work, Great Unicorn Intelligence is also exploring this "from trajectories to rules" loop.

11. Conclusion

The training bottleneck for long-horizon agents is shifting from "is there a reward?" to "can the reward be assigned correctly?" This batch of October research offers several complementary answers: let counterfactuals share luck, let the model identify what mattered, give the same action the same update, and let process signals step aside at the right time. Only when credit is assigned correctly can agents learn what they actually need to learn.

To learn more about Great Unicorn Intelligence's work on rule engines and intelligent decision systems, visit www.hxallai.com.

References

1. [1]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (introduces GRPO). arXiv, 2024-02-05. (background) https://arxiv.org/abs/2402.03300
2. [2]Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni et al. VinePPO: Refining Credit Assignment in RL Training of LLMs. arXiv, 2024-10-02. (background) https://arxiv.org/abs/2410.01679
3. [3]Lang Feng, Zhenghai Xue, Tingcong Liu, Bo An. Group-in-Group Policy Optimization for LLM Agent Training. arXiv, 2025-05-16. (background) https://arxiv.org/abs/2505.10978
4. [4]Zeropy Surio, Tobias Engel, Mei-Lin Chou, Anika Raman, Samuel Okafor. Same Luck, Different Choice: Twin-World Credit Assignment for Agentic Reinforcement Learning. arXiv, 2026-10-22. https://arxiv.org/abs/2610.31544
5. [5]Fangcong Yin, Ming Yin, Tanvi Bhandarkar, Aashu Singh et al. Self-Credit: Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling. arXiv, 2026-10-20. https://arxiv.org/abs/2610.28388
6. [6]Shaojin Ma, Ruitao Wang, Pengyi Li, Jie Yang et al. One Act, Many Masks: Effect-Marginal Policy Optimization for Long-Horizon Agents. arXiv, 2026-10-21. https://arxiv.org/abs/2610.29427
7. [7]Yingjian Chen, Lang Zhou, Zixin Zhao, Pingfan Li et al. Action-Value Calibration via Continuation Sharing for Long-Horizon Agentic Reinforcement Learning. arXiv, 2026-10-27. https://arxiv.org/abs/2610.37195
8. [8]Zhengping Jiang, William Jurayj, Liaoyaqi Wang, Chunsheng Zuo et al. FOCUS: Active Critic Learning for Long-Horizon Language Model Agents. arXiv, 2026-10-26. https://arxiv.org/abs/2610.35801
9. [9]Jiaxi Liu, Shixiong Kai, Zhentao Tang, Yang Li et al. From Outcomes to Steps: Reliable Success-Aware Critic Learning for Long-Horizon Language Agents. arXiv, 2026-10-20. https://arxiv.org/abs/2610.27623
10. [10]Tao Ren, Hui Yang, Jia Deng, Tingyang Chen et al. Multiscale Policy Optimization for Long-Horizon LLM Agents. arXiv, 2026-10-26. https://arxiv.org/abs/2610.35013
11. [11]Lei Zhao, Zongheng Cao, Xinyu Hu. When Fine-Grained Credit Assignment Helps and Hurts: Multimodal Agentic Reinforcement Learning through Video Editing. arXiv, 2026-10-26. https://arxiv.org/abs/2610.35497
12. [12]Chu Zhao, Lei Tang, Yanming Wang, Jianzhe Zhao et al. ToolOPD: Credit-Aware On-Policy Distillation for Tool-Use Agents. arXiv, 2026-10-19. https://arxiv.org/abs/2610.25693
13. [13]Yibin Huang, Conghui Zhu. Distill Until Divergence: Error-Truncated Hindsight Distillation for Agentic RL. arXiv, 2026-10-16. https://arxiv.org/abs/2610.23048
14. [14]Jingxuan Xu, Zihang Xu, Yuechen Luo, Ziqi Miao et al. RIPPLE: Recovery Internalization through Process-Level Policy Learning and Effect-Aware Credit for Long-Horizon Agentic Coding. arXiv, 2026-10-22. https://arxiv.org/abs/2610.31276
15. [15]Sijia Luo, Mengjie Gao, Yuxiao He, Mengshu Sun et al. EG-GRPO: Evidence-Guided Step-level Credit Assignment for Deep Search Agents. arXiv, 2026-10-18. https://arxiv.org/abs/2610.25269