The Neutral Mask: How RLHF Provides Shallow Alignment While Leaving Partisan Structure Intact

arXiv·medium signal

Argues RLHF produces only surface-level neutrality in a large language model — the partisan structure underneath remains intact beneath an aligned-looking mask. Probes the gap between behavioral alignment and the model's underlying representations. A sobering data point for anyone treating RLHF as deep value alignment rather than output-shaping.