DynamicRubric: An 8B Co-Evolved Evaluator Beats a 70B Reward Model — Now Serving Tens of Millions of WeChat Search Requests Daily

arXiv 2607.20083·medium signal

As a policy improves, its sampled responses converge in quality and evaluator score gaps collapse, starving policy optimization of signal. The authors show via a probability-allocation argument that the directional gain of shifting probability mass between two responses is exactly the evaluator score gap, then propose DynamicRubric — generating weighted binary rubric items conditioned on each candidate response set rather than using a static rubric. With 8B backbones it provides stronger policy supervision than a 70B reward model or a 235B static rubric generator, and an optimized model is fully deployed in WeChat Search's AI answering path across all online traffic. The takeaway for anyone running LLM-as-judge: the judge has to evolve with the policy it grades.