Personalized RewardBench: Evaluating Reward Models with Human-Aligned Personalization
arXiv·medium signal
Introduces a benchmark for evaluating reward models on pluralistic alignment — measuring whether RMs can capture diverse human values rather than collapsing to a single preference distribution. Current RM benchmarks test general response quality but ignore that different users may legitimately prefer different outputs. Critical as RLHF-trained models serve increasingly diverse global user bases with conflicting preferences.