Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-07-21 · high
The catch: scores now hinge on prompt wording, so two teams could land on different answers.
Why now: The paper landed in the July 21 arXiv batch with enough detail on the method to stand on its own.
Adobe researchers built an image similarity metric that reads a text prompt, per a paper posted to arXiv by Sheng-Yu Wang, Yotam Nitzan and Aaron Hertzmann. That matters for anyone doing image eval or dedupe work at scale. Fixed metrics like LPIPS force one similarity judgment onto every job, whether you care about shape or color.
Their fix: a metric that takes a text prompt specifying the axis of comparison. Instead of asking how similar two images are, you ask how similar they are in color, or in shape. The answer comes back scoped to that one question.
Right now, teams using LPIPS to catch near-duplicate images or score generative output are stuck with whatever notion of similarity the metric's training baked in. There's no way to tell it to ignore color and just check silhouette.
The paper doesn't say how the metric scores against LPIPS on standard benchmarks. It also doesn't address whether prompt phrasing shifts scores enough to make results hard to reproduce across teams.
A related paper tackles a similar problem: cutting agent-written code slop by judging the trajectory instead of the diff. Both point to the same failure: a single scalar number can't capture what people actually mean by good.
Each link below shares sources, entities, or timing with this story.
A new arXiv paper finds pretraining gains flip into losses past an optimal context length, as models learn to lean on text instead of memory.
Explicit refusal is falling across four Qwen generations while state-aligned reframing rises, per a 21,708-trial benchmark of vision-language models.
A 2,420-trial test found a 50:50 mix of relevant and irrelevant items beat an all-relevant AI prompt, per an arXiv paper on agent token costs.
Wang, Nitzan and Hertzmann (Adobe Research) argue in arXiv 2607.18237 that fixed metrics like LPIPS collapse a context-dependent human judgment into one number. Two images can be similar in shape and different in color, and which one matters depends entirely on your task. Thei...
Testing eight models across 192,000 evaluations, researchers found chain-of-thought and direct instructions to ignore the score didn't remove the bias.
The rule text can survive context compaction while the behavior it enforces quietly stops, and grepping the summary for that text won't catch the difference.
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.