Dispatch
NeoMME gets within 0.002 nDCG of ColQwen2.5 on document retrieval using 14x fewer parameters
H Company released NeoMME on September 3, a multimodal-native encoder at 260M and 800M parameters that runs text tokens and raw image patches through one bidirectional Transformer with no separate vision tower or text decoder, using a from-scratch 131k BPE vocabulary, 16,384-token context, 32x32 dynamic-resolution patches, grouped-query attention and 2D RoPE. On ViDoRe v3 the 260M scores 0.523 nDCG@10, best among sub-800M models and within 0.002 of ColQwen2.5 at roughly 14x fewer parameters, while the 800M reaches 0.556. Throughput is 51 pages per second at 2048x2048 on an L40S, about double ColModernVBERT, and all checkpoints are Apache 2.0, which makes it a cheap swap for anyone running document RAG.
↳ Follow the thread