When Rewards Become Rubrics: Reward Hacking, Judge Noise, and Credit Assignment in Rubric-Based Reinforcement Learning (Deep Dive)

1. Background: From "Verifiable" to "Scored Against a Rubric"

Over the past two years, most progress in RL post-training has come from reinforcement learning with verifiable rewards (RLVR): check the answer to a math problem, run unit tests on code — the reward signal is nearly noise-free. GRPO (Group Relative Policy Optimization), introduced with DeepSeekMath [2], made this route cheap: sample a group of answers to the same question and use their relative quality within the group instead of a separately trained value network.

The trouble is that many tasks enterprises actually need are not verifiable: an explanation of a medical condition, a research survey, or a credit due-diligence report has no unique correct answer. Rubrics as Rewards (RaR) [1] (background reference), published in July 2025, pushed the idea further: write a checklist rubric for every question — e.g., "mentions drug interactions," "advises when to see a doctor," "does not give a definitive diagnosis" — let an LLM judge decide whether each item is satisfied, and use the weighted sum as the reward. This keeps human expert standards while scaling.

Rubric rewards have three built-in weaknesses, which are the starting point of October's papers:

1. Rubrics say what should be present and rarely what should not. According to statistics cited by the FORK paper, in the widely used RubricHub dataset, 90.2% of criteria are "presence" criteria and only 8.6% are "absence" criteria [4]. The model quickly learns that writing more, covering everything, and repeating itself all raise the hit rate.
2. The judge itself is noisy. LLM judges have preferences, waver, and are swayed by wording. A synthesis of 58 studies argues that the structure of errors matters more than their rate: random noise tends to slow training down, while systematic bias is actively exploited by the policy [3].
3. A single total score cannot say which passage deserves the reward. An answer may satisfy 7 criteria and violate 2, yet only one scalar is fed back, and the model does not know what to fix.

How do rubric rewards enter GRPO? For each question, the policy samples G answers (e.g., 16); the judge checks every criterion for every answer, yielding rewards r₁…r_G; GRPO uses the group-normalized relative score as the "advantage":

A_g = (r_g − mean(r_1..r_G)) / std(r_1..r_G)

• r_g: the rubric reward of the g-th answer; mean / std: mean and standard deviation of the rewards in this group.
• A_g > 0: this answer is better than the group average, so training raises its probability; A_g < 0 lowers it.

An easily overlooked property: the formula only cares about who beats whom within the group, not absolute scores. So any hacking trick (say, adding two more paragraphs) that puts an answer slightly ahead within its group will be reinforced again and again, even if absolute quality has not improved. The improvements in the three lines below essentially change either the definition of r_g or the way A_g is distributed.

Technical lineage: how rubric rewards got here

Placed on a timeline, this line of work falls into four stages, each patching a hole left by the one before.

Stage 1 (from 2024): verifiable rewards as the foundation. GRPO replaced the value network with within-group relative comparison, making RL post-training cheap and stable on tasks with reference answers such as math and code [2]. The question it left open: what about tasks without a reference answer?

Stage 2 (2025): writing "a good answer" as a checklist. RaR equipped each prompt with expert-guided, instance-specific rubrics, exploring both explicit weighted aggregation of criterion scores and implicit aggregation, in which an LLM judge reads the rubric and gives a single overall score [1]. According to the related-work survey in the FORK paper, the same period saw RLCF, which turns checklist scores into preference pairs for direct preference optimization, and the RubricHub dataset, which generates rubrics at scale through principle-guided synthesis, multi-model aggregation, and difficulty evolution [4]. By this point, rubrics had become a practical interface for specifying training objectives.

Stage 3 (2025–2026): discovering that rubrics get gamed. The same survey notes that even with a strong verifier, RL-trained responses can become "more complete" while their factual correctness and necessity decline, yet rubric-based judges still prefer them. RLCF added universal criteria against excessive or off-topic content after observing verbose preambles; OnlineRubrics adds criteria by comparing current-policy and control-policy responses; DR Tulu maintains search-grounded, evolving rubrics that capture both newly discovered useful content and newly observed undesirable behavior [4]. The common idea was "make the rubric more complete."

Stage 4 (October 2026): from "writing more complete rubrics" to "computing the reward correctly." The dozen or so papers discussed here share a clear turn: rather than relying only on expanding rubrics, they change how the reward is computed. FORK scores the necessity and correctness of every unit of content [4], FRDA corrects the weighting imbalance across dimensions [5], RaRE and URW attach confidence to judge verdicts [8][10], and CoCA and RELAY rework credit assignment and training stages [11][12]. Meanwhile, a synthesis of 58 studies classifies judge errors as random, systematic, and exploitable by the policy, and notes that no mitigation has yet been shown to handle all three at once [3]. That conclusion could almost serve as the thesis of this October wave: each method seals one kind of leak; combining them is the norm.

With this lineage in mind, the methods below fall into place: they are not competing "new paradigms" but patches to different stages of the same reward pipeline.

2. Line One: Curbing Hacking — Stop Models from Padding for Points

2.1 Algorithm ①: FORK — From "Recall" to "F-score"

One-line intuition: The standard approach only asks "how many criteria are covered" (recall). FORK also asks "is every passage you wrote actually needed by some criterion, and is it factually correct?" (precision), and combines the two into an F-score reward [4].

Analogy: If an exam is graded only by "hitting scoring points" with no penalty for filler, students will copy down every fact they can think of. FORK adds one rule for the grader: every paragraph must state which scoring point it addresses; repeated, off-topic, or incorrect paragraphs pull the score down.

Step by step (KAIST, CMU, and others, 2026-10-24):

1. Segment: split the answer into "units" u₁…uₙ (the paper uses an LLM to segment and also tests splitting on line breaks).
2. Match: for each criterion rⱼ and each unit uᵢ, the judge decides whether the unit responds to the criterion, m_ji ∈ {0,1}.
3. Share necessity: each criterion has a budget of "1 point," split evenly among all units responding to it. If 3 units all address the same criterion, each gets only 1/3.
4. Decide necessity: a unit is "necessary" (Nᵢ = 1) only if its accumulated budget cᵢ reaches a threshold τ.
5. Check facts: independently, the judge decides whether the unit is factually correct (Cᵢ ∈ {0,1}); any factual error gives 0.
6. Combine: precision P is the share of units that are both necessary and correct; recall R is rubric coverage; the reward is F_β.

Key formulas:

a_ji = m_ji / K_j          (K_j = number of units responding to criterion j; 0 if K_j = 0)
c_i  = Σ_j a_ji             N_i = 1[c_i ≥ τ]
P    = (1/n) · Σ_i N_i · C_i
F_β  = (1+β²) · P · R / (β² · P + R)

• m_ji: whether unit i responds to criterion j (1 = yes, 0 = no).
• K_j: how many units respond to criterion j; the larger it is, the more often that criterion is repeated.
• a_ji: the "necessity share" that criterion j gives to unit i — the more repetition, the thinner each share.
• c_i: total share unit i receives from all criteria; τ: the threshold for counting as necessary.
• C_i: whether unit i is factually correct.
• n: total number of units; P: share of necessary and correct units; R: rubric coverage.
• β: recall–precision trade-off; β = 1 weighs them equally, β > 1 favors coverage.

Worked example: A medication question has 4 criteria, and the model's answer is split into 10 passages. Four passages each address a different criterion; four more keep restating criterion 1; the remaining 2 are unrelated to any criterion.

• Recall: all 4 criteria are covered, R = 1.0.
• Criterion 1 is addressed by 5 passages, each getting only 0.2; with τ = 0.5, none of these 5 counts as necessary. Each of the other 3 criteria is addressed by exactly one passage, which gets 1.0 and counts as necessary. If those 3 passages are factually correct, P = 3/10 = 0.3.
• F₁ = 2×0.3×1.0/(0.3+1.0) ≈ 0.46. If the 4 repetitive and 2 unrelated passages are removed, leaving 4, then P = 4/4 = 1.0 and F₁ = 1.0.

In other words, under a recall-only reward the two answers score the same; under the F-score reward, the concise one wins by a wide margin.

Results (per the paper): Training Qwen2.5-7B-Instruct with GRPO (16 samples per question), Qwen3.6-35B-A3B as the training judge, and GPT-5.4 as the reference evaluation judge, the medical-domain comparison is [4]:

• Standard coverage reward: recall 35.7, precision 25.5, win rate vs. base 3.1%, average length 9,195 characters.
• OnlineRubrics, then the strongest coverage-style improvement: win rate 10.6%.
• FORK (β = 1): recall 32.5, precision 53.2, F1 37.5, win rate 68.0%, length 3,533 characters; HealthBench win rate rises from 22.1% to 74.1%.
• Science domain (β = 1.25): win rate 61.5% vs. 42.5%, ResearchQA win rate from 26.6% to 67.1%, length about 2.1× shorter.

An interesting side effect: after FORK training, satisfaction of "absence" criteria (e.g., "must not give a definitive diagnosis") rises by 5.7 percentage points, whereas the standard approach lowers it by 3.2 points — the precision objective effectively also rewards "saying fewer wrong things."

Why it works, and when it fails: It works because it puts a price on both redundancy and errors, closing the cheapest path to gaming the score. Three failure risks: first, the necessity judgment itself is the bottleneck — ablations show that judging necessity with generic or offline criteria makes results clearly worse; second, the choice of β is sensitive, and the paper reports that F₂ (leaning too far toward recall) hurts; third, segmentation granularity affects results, and long tables or code are hard to split into semantic units.

2.2 Algorithm ②: FRDA — A Fair Race for Every Evaluation Dimension

One-line intuition: Dimensions with many criteria (e.g., "completeness" with 20 items) drown out those with few (e.g., "concise expression" with 2), so the model only games the former. FRDA standardizes within each dimension across the group first, then averages the dimensions with equal weights, so that progress on every dimension is "worth the same" [5].

Analogy: In a comprehensive assessment where "knowledge" has 20 questions and "expression" has only 2, ranking by total score makes everyone memorize facts. FRDA is like converting each subject into a standard score first and then averaging the subjects' standard scores.

Step by step (Baidu, Shanghai Jiao Tong University, and others, 2026-10-19):

1. Group a question's criteria into D semantic dimensions (e.g., factual reliability, completeness, style).
2. For the G sampled answers to the same question, compute each dimension's satisfaction rate s_{g,d}.
3. Z-score each dimension within the group.
4. Average the dimensions' standard scores with equal weights, then re-standardize within the group to obtain the GRPO advantage.
5. The extended version RSDC additionally removes correlation between dimensions via Cholesky whitening and adds a preference reward model as dimension D+1.

Key formulas:

z_{g,d} = (s_{g,d} − mean_g s_{·,d}) / (std_g s_{·,d} + ε)
A_g     = Normalize_g( (1/D) · Σ_d z_{g,d} )

• s_{g,d}: satisfaction rate of answer g on dimension d.
• mean_g / std_g: mean and standard deviation across this group of answers; ε prevents division by zero.
• z_{g,d}: how many standard deviations this answer is above the group average on that dimension.
• D: number of dimensions; A_g: the advantage passed to GRPO; Normalize means one more within-group standardization.

Worked example: Two dimensions — completeness (20 criteria) and concision (2 criteria). Answer A covers 2 more completeness criteria but violates 1 concision criterion; answer B is the opposite. By raw total, A wins by about 1 point. Under FRDA, if completeness varies a lot within the group (standard deviation of 3 criteria), A leads by only about 0.67 standard deviations; concision varies little (standard deviation of 0.5 criteria), so A trails by 2 standard deviations — after equal-weight averaging, B wins. The concision criterion, previously "worth almost nothing," is restored to the same standing as completeness.

Results (per the paper): With Qwen3-4B, GPT-4.1 mini as judge, and 600 training steps, compared with the raw rubric reward, the weighted average score improves by 5.05 on Arena-Hard, 3.60 on ResearchQA, 8.99 on GPQA, and 1.92 on HealthBench; average answer length at step 600 is 4,496 characters vs. 5,180 for the raw rubric reward [5]. The ablation is telling: equal-weight averaging without within-dimension standardization actually lowers Arena-Hard by 2.65 — the key is standardization, not equal weighting.

Why it works, and when it fails: It corrects the imbalance in how the reward signal is distributed across dimensions, so that gradients from sparse dimensions are not drowned out. Its limitation is that dimensions still have to be defined in advance by humans or a model, and a wrong partition introduces new bias; the whitened RSDC variant has so far only been validated on dialogue tasks.

2.3 Two More Anti-Hacking Ideas

• CALIF (University of Washington and others, 2026-10-14) [6]: For verifiable instruction-following constraints (e.g., "must contain 3 bullet points"), the authors find that open models' constraint satisfaction rises while answer quality drops. CALIF uses a frozen reference model to compute the answer's likelihood "given only the request, not the constraint" as a calibrated quality reward, without an extra judge. Per the paper, in 45 comparisons across 1.7B–8B models, 40 balanced both constraints and quality.
• Dynalize (2026-10-19) [7]: Periodically has a language model analyze the policy's latest samples and automatically write penalty clauses such as "this kind of hacking should be penalized," which the judge then enforces. It turns the rubric from a static checklist into evolving anti-hacking clauses.

3. Line Two: Taming the Judge — Let Scorers Know When They Are Unsure

3.1 Algorithm ③: RaRE — A Judge Ensemble That Can Abstain

One-line intuition: Simply averaging several judges is not safe. RaRE first has each judge express "how confident it is in its score," weights by confidence, and removes samples on which judges disagree too much from training altogether [8].

Analogy: If two jurors give the same work 2 and 9 points, an average of 5.5 means nothing. A more sensible approach: set that work aside for review, and score the rest by each juror's confidence.

Step by step (UC Berkeley and others, 2026-10-15):

1. Soft Likert scoring: instead of having the judge emit a single integer from 1 to L, read its probability (logprob) for each score and compute the expected score E_j; the more concentrated the probability, the more confident the judge.
2. Confidence weighting: use the inverse variance of the score distribution as confidence c_j and take a weighted average.
3. Disagreement filtering: if two judges' expected scores for the same answer differ by more than τ×(L−1), the answer's advantage is set to 0 and it does not participate in this update.

Key formulas:

E_j = Σ_k k · p_j(k)                       (judge j's expected score)
c_j = 1 / Var_j                             (more concentrated → more confident)
r   = (1/L) · Σ_j c_j E_j / Σ_j c_j          (normalized reward)
if |E_1 − E_2| > τ · (L − 1), set this answer's advantage to 0

• p_j(k): probability that judge j assigns score k; L: maximum of the scale.
• Var_j: variance of this distribution — the more the score "wavers," the larger it is.
• τ: disagreement tolerance; the paper uses 0.30 or 0.40, i.e., discard if two judges differ by 30%–40% of the scale's range.

Worked example: Scale 1–10 (L = 10), τ = 0.3, so the tolerance is 2.7 points. Judge A's expected score is 7.8 with variance 0.4; judge B's is 7.1 with variance 1.6: a gap of 0.7, kept. Confidences are 2.5 and 0.625; the weighted score is (2.5×7.8 + 0.625×7.1)/3.125 ≈ 7.66, normalized to r ≈ 0.77. For another answer, A gives 8.5 and B gives 4.0, a gap of 4.5 > 2.7, so it is excluded from this update.

Results (per the paper): The policy is Qwen3-4B-Base. On HealthBench, training with the Gemini or DeepSeek judge alone yields 0.296 and 0.338 respectively, while simple averaging gives only 0.307, between the two; adding disagreement filtering raises it to 0.347 (about 5% of samples filtered); RaRE's best configuration reaches 0.364, above the best of five single judges, 0.339. On GPQA it is 0.412 vs. 0.406, not significant [8].

Why it works, and when it fails: It removes samples the judges themselves cannot settle, which corresponds to the structural noise described in [3] — if those samples entered training, the gradient direction itself would be unreliable. It fails when both judges make the same mistake (e.g., both prefer long answers), since disagreement detection never fires; it also requires judges to expose logprobs, which some closed APIs do not.

3.2 Algorithm ④: ICW and URW — Criteria Need Weights and Confidence, Too

One-line intuition: Not every criterion is equally important, and not every judgment is equally trustworthy. ICW derives criterion weights from pairwise comparisons [9]; URW attaches a reliability score to each criterion judgment via conformal prediction [10].

Analogy: ICW is like asking interviewers to compare pairwise "how many times more important is communication than coding" and then solving a consistent set of weights from that comparison table; URW is like having the same juror re-review using several different phrasings to see whether the judgment is stable.

Step by step (ICW, 2026-10-14):

1. For the m criteria of a question, ask an LLM pairwise "how many times more important is criterion a than criterion b," obtaining a comparison matrix M with M[a][b] = α and M[b][a] = 1/α.
2. Compute M's principal eigenvector and normalize it to obtain the criterion weights w.
3. Compute the consistency ratio CR (the standard test of the Analytic Hierarchy Process); if CR exceeds a threshold, ask the model to revise contradictory comparisons until consistent.

M · w = λ_max · w,    Σ_a w_a = 1
CI = (λ_max − m)/(m − 1),   CR = CI / RI_m

• M: pairwise importance matrix; w: criterion weight vector.
• λ_max: largest eigenvalue of M; exactly m when fully consistent.
• CI / CR: consistency index and ratio; RI_m: average consistency index of random matrices of the same size, used for normalization.

Worked example: Three criteria; the model says A is 2× as important as B, B is 2× as important as C, and A is 4× as important as C. These judgments are fully consistent, giving w ≈ (0.57, 0.29, 0.14). If the model instead says A is only 1.5× as important as C, it contradicts itself, CR rises, and revision is required.

Results and limitations (per the papers): On Qwen2.5-3B, the ICW-weighted reward improves AlpacaEval LC win rate by 40.87% relative to the base model, compared with 14.12% for equally weighted criteria [9]. The cost is that the number of queries grows quadratically with the number of criteria. URW (Zhejiang University, Peking University, and others) [10] repeats the same judgment under multiple rephrased templates and computes split-conformal p-values as each criterion's weight; its statistical guarantee rests on an exchangeability assumption and breaks down when the template distribution differs too much from the real one.

4. Line Three: Fixing Credit Assignment — Let the Reward Say What to Change

• CoCA (2026-10-13) [11]: maps criterion-level satisfaction back to the corresponding spans of the answer and assigns advantages at the criterion level instead of sharing one score across the whole answer. Per the paper, an average gain of 4.2%.
• RELAY (2026-10-13) [12]: has a frozen teacher generate references "while reading the rubric" and performs selective self-distillation on the student; when RL plateaus, an automatic detector (triggering between steps 122 and 171, per the paper) hands training over to distillation. On RaR-Medicine it improves over GRPO by 4.62 (Qwen3-1.7B) and 6.02 (Qwen3-4B).
• RARE (2026-10-18) [13]: rubrics are no longer static checklists. When a criterion is satisfied by all samples, it can no longer separate good from bad, and the system replaces it with a new criterion targeting the remaining defects. Per the paper, it improves on static rubrics by 3.82.

These three correspond to three levels of credit assignment:

Span level (CoCA): In conventional GRPO, all tokens in an answer share the same advantage. CoCA first has the judge point out "which text satisfies or violates which criterion," then maps criterion-level positive and negative signals to the corresponding spans — spans satisfying a criterion get positive advantage, violating spans get negative advantage, and unrelated spans stay neutral. Intuitively, it is like a teacher annotating an essay paragraph by paragraph instead of writing only a total score. It presumes criteria can be "located" in the answer; for global criteria such as "the overall structure is clear," the mapping becomes vague.

Training-stage level (RELAY): RL on open-ended tasks often hits plateaus — answers in a group are all similar, advantages approach zero, and gradients nearly vanish. RELAY lets a frozen teacher read the rubric as "privileged information," generate references better aligned with it, and distill only on samples where "the student did poorly but the teacher did well"; the hand-off starts only when the detector sees the reward curve flatten. The risk is that the teacher's own biases are distilled directly into the student.

Rubric level (RARE): When a criterion is satisfied by every sampled answer, it contributes nothing to within-group ranking, wasting an evaluation dimension. RARE identifies such "saturated" criteria and generates new, more discriminative ones based on the common defects of current answers. This matches how human examiners set questions: a question everyone gets right is replaced next time by one that better separates candidates. One caveat: continually evolving rubrics may drift from the original business goal, so humans should set immutable "anchor criteria."

What the three have in common: they turn the rubric from a "final score sheet" into a "structured feedback channel" during training — showing where points are earned and lost, and updating as training progresses.

5. Side-by-Side Comparison

Note: Base models, judges, and datasets differ across papers, so numbers are not directly comparable across rows; affiliations follow the paper bylines.

A Composable Engineering Recipe (Our Suggestion)

The methods above are not mutually exclusive. Based on the papers' ablations, we suggest enterprises building their own rubric rewards stack them in the following order, verifying the gain of each step separately:

1. Add negative clauses first: beyond presence criteria, write 3–5 "must not appear" clauses for each task type — the cheapest step.
2. Then standardize within dimensions: following FRDA, group criteria into dimensions, standardize within the group, and merge with equal weights [5]; almost no extra compute.
3. Add two judges with disagreement filtering: two judges from different model families, weighted and filtered RaRE-style [8], keeping the filtered share in single-digit percentages.
4. Introduce a precision term: for long-form tasks, follow FORK and switch the reward to F_β [4], starting from β = 1.
5. Continuously monitor length and win rate: at each checkpoint, record rubric score, answer length, and human spot-check win rate together; pause training to investigate when they diverge.

Note that this combination has not been validated end-to-end in any single paper; it is our engineering judgment based on the papers' findings, and deployment should rest on controlled experiments with your own data.

6. Our Views

First, the bottleneck of rubric rewards has shifted from "writing criteria" to "defining what should not be written." With 90.2% presence criteria, rewards are inherently biased toward "more" [4]. FORK's precision, CALIF's quality anchor, and Dynalize's penalty clauses are all, at heart, filling in a "negative list." We expect useful rubric datasets over the next six months to carry a markedly higher share of absence clauses.

Second, judge reliability is not an "accuracy" number but an "error structure" problem. RaRE's simple average lands between the two judges [8], and the synthesis in [3] also stresses that systematic bias gets exploited by the policy. When enterprises evaluate their judges, they should report not only agreement with humans but also which kinds of samples the judge is systematically biased on.

Third, the cheapest improvements often come from reward normalization and distribution, not bigger judges. FRDA costs almost nothing yet brings +8.99 on GPQA [5]; its ablation shows that within-dimension standardization alone determines the sign of the effect. Such engineering details deserve to be tried first.

Fourth, answer length is the best "hacking thermometer." Nearly every paper reports length: 9,195 vs. 3,533 characters [4], 5,180 vs. 4,496 characters [5]. Before human evaluation is available, watching whether the length curve and the rubric-score curve rise together catches most hacking.

7. Outlook

6–12 months:

• Rubric rewards will evolve from "checklist + total" to a four-part structure of "checklist + weights + confidence + negative clauses," with ICW- and URW-style methods absorbed into mainstream training frameworks.
• Judge ensembles will become the default, and "abstain on disagreement" will become a standard trick, much like gradient clipping.
• Evaluations will routinely report win rate, precision, and length together; results reporting rubric coverage alone will become less and less convincing.

1–3 years:

• Rubrics may shift from "human-written" to "human-reviewed + model-evolved": RARE and Dynalize suggest rubrics will co-evolve with the policy [7][13], and human experts will move from writing items to reviewing them and setting boundaries.
• Reward modeling may converge with explainable auditing: every reward traceable to which criterion, which passage, which judge, and with what confidence — especially important for regulated industries.
• The risk is "judge–policy" homogeneity: if the judge and policy come from the same model family, systematic biases are amplified together, and cross-family judges will become an important engineering practice.

8. Implications for Financial Risk Management and Enterprise Deployment

1. Structuring internal review standards into rubrics is a practical first step. Credit reports, post-loan early-warning notes, and model validation reports already have written review checklists, which naturally convert into rubrics for evaluating — or even training — generative models.
2. Write negative clauses from day one. For example, "must not give a definitive default conclusion" and "must not cite data that was not provided." A rubric without negative clauses will train the model into a report generator that simply "writes more."
3. Audit the judge's error structure. Before going live, run two or more judges over historical samples and find which types of customers or reports their disagreements concentrate on; these are often the genuinely hard business cases and should be left to humans.
4. Monitor length and precision daily. An abnormal rise in generation length and a drop in precision often expose problems earlier than business complaints do.

In building scorecards and risk models, Hongxi AI has long emphasized being "explainable and traceable," and we see rubric-based rewards as consistent with that principle: every point should be able to explain where it came from.

9. Conclusion

October's papers converge on a clear consensus: rubric rewards are a viable way to extend RL to open-ended tasks, but they need three companions — negative lists, judge confidence, and credit assignment. For enterprises, the most valuable step is not chasing any one paper's score, but first writing down clearly what a "good answer" means in their own business, in an auditable structure.

Visit Hongxi AI at https://www.hxallai.com.

References

1. [1] Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, et al. Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains. arXiv, 2025-07-23. (Background reference.) https://arxiv.org/abs/2507.17746
2. [2] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. DeepSeek-AI, et al., 2024-02-05. (Background reference; origin of GRPO.) https://arxiv.org/abs/2402.03300
3. [3] Savyasachi Gupta, Aniruddha Srinivas Joshi. Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR. arXiv, 2026-10-13. https://arxiv.org/abs/2610.17824
4. [4] Hyun Ryu, Jong-Heon Kim, Yiqing Xie, Seungone Kim, et al. Eat Your Precision: Precision Objective Mitigates Reward Hacking in Rubric-based RL. KAIST, CMU, et al., 2026-10-24. https://arxiv.org/abs/2610.33512
5. [5] Jianqiang Xia, Han Xu, Dianxi Shi. Intra-Rubric Reward Divergence: Mitigating Rubric Hacking through Latent Dimension Recovery and Balancing. Baidu, Shanghai Jiao Tong University, et al., 2026-10-19. https://arxiv.org/abs/2610.26685
6. [6] Sangyeon Yoon, Wooseok Seo, Hamish Ivison, Hyesoo Hong, et al. Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following. University of Washington, et al., 2026-10-14. https://arxiv.org/abs/2610.19525
7. [7] Afonso Baldo, António Farinhas, Marcos Treviso, Ricardo Rei, et al. Mitigating Reward Hacking in Preference Reward Models with Evolving Penalties. Instituto Superior Técnico (University of Lisbon), et al., 2026-10-19. https://arxiv.org/abs/2610.26496
8. [8] Michael Tian-Yue Liu, Anne Harrington, Trevor Darrell, Yutong Bai. Judging the Judges: Confidence and Disagreement in Rubrics as Rewards. UC Berkeley, et al., 2026-10-15. https://arxiv.org/abs/2610.21179
9. [9] Haocheng Yang, Licheng Pan, Yiming Dong, Xiaoxi Li, et al. Not All Criteria Matter: Importance-Aware Criteria Weighting for Rubric-Based LLM Evaluation. Zhejiang University, Peking University, et al., 2026-10-14. https://arxiv.org/abs/2610.19915
10. [10] Licheng Pan, Haocheng Yang, Haoxuan Li, Yuzhou Chen, et al. Rubric Judgments Lack Uncertainty: Conformal Weighting for LLM Evaluation. Zhejiang University, Peking University, et al., 2026-10-15. https://arxiv.org/abs/2610.21214
11. [11] Guangcheng Zhu, Yingfan Ma, Xing Zheng, Haobo Wang, et al. CoCA: Criterion-Aware Credit Assignment for Rubric-Based Reinforcement Learning. Zhejiang University, Ant Group, 2026-10-13. https://arxiv.org/abs/2610.17733
12. [12] Kunwang Zheng, Xinyu Sun, Yu Cao, Shuocheng Li, et al. RELAY: Selective Self-Distillation with Adaptive Hand-off for Rubric-Based Reinforcement Learning. Fudan University, Alibaba, et al., 2026-10-13. https://arxiv.org/abs/2610.17623
13. [13] Xueyun Tian, Bingbing Xu, Chuyue Li, Jiaming Liu, et al. RARE: Reinforcement Learning with Adaptive Rubric Evolution for Open-Ended Generation. Institute of Computing Technology, Chinese Academy of Sciences, et al., 2026-10-18. https://arxiv.org/abs/2610.25434

Research · Frontier AI Deep Dive · Great Unicorn Intelligence