Newer GPT models show more gender bias, REGARD scoring finds

GPT-4 drops the sexual violence common in GPT-2's completions about women, according to a study of 450,000 completions across 15 GPT models. The same research finds GPT-4 gained favorable content about men that women's completions never got. Topic diversity for women fell 36% relative to men at that same alignment point.

REGARD, a classifier built for representational harm, finds the gender gap grows with each newer release (rho = +0.55, p = .034). Detoxify, the standard toxicity classifier, found no such trend (rho = -0.23, p = .42). Same models, same prompts, opposite verdicts, depending on the metric.

That gap matters for anyone screening model output for bias before shipping it. A toxicity filter like Detoxify will report explicit content is down and stop there. It won't show that a model narrowed what it says about one group while opening up for another.

The paper doesn't say whether the pattern holds outside gender or whether it's specific to how GPT-4 was aligned.