Adobe Researchers Make Image Similarity Depend on a Text Prompt

Adobe researchers built an image similarity metric that reads a text prompt, per a paper posted to arXiv by Sheng-Yu Wang, Yotam Nitzan and Aaron Hertzmann. That matters for anyone doing image eval or dedupe work at scale. Fixed metrics like LPIPS force one similarity judgment onto every job, whether you care about shape or color.

Their fix: a metric that takes a text prompt specifying the axis of comparison. Instead of asking how similar two images are, you ask how similar they are in color, or in shape. The answer comes back scoped to that one question.

Right now, teams using LPIPS to catch near-duplicate images or score generative output are stuck with whatever notion of similarity the metric's training baked in. There's no way to tell it to ignore color and just check silhouette.

The paper doesn't say how the metric scores against LPIPS on standard benchmarks. It also doesn't address whether prompt phrasing shifts scores enough to make results hard to reproduce across teams.

A related paper tackles a similar problem: cutting agent-written code slop by judging the trajectory instead of the diff. Both point to the same failure: a single scalar number can't capture what people actually mean by good.