Open-Source VLMs Copy an Attribute From the Wrong Same-Class Object 19.84% of the Time, and Accuracy Metrics Hide It

arXiv 2608.16805·medium signalXBlueskyLinkedInCopy link

The authors formalize Dense Same-Class Attribute Misbinding — a VLM correctly recognizing objects and attributes in a crowded scene but assigning the attribute to the wrong instance of the same class — and build InstaBind-Lite to measure it: 524 images, 529 groups of 3-6 same-class entities, 1,773 boxed instances, 9,580 deterministically evaluated questions with source-instance annotations that separate copying from hallucination. Five open-source models average a 19.84% misbinding rate versus 7.55% for two commercial API systems, and 80.70%/81.51% of identifiable transfers come from adjacent instances. Localization and instance-first prompting help some models but are not universal fixes — relevant to anyone building extraction pipelines over dense scenes or dashboards.