Frontier models score under 50% on end-to-end business intelligence.

BI-Bench harvests real BI projects from public sources and extracts question/ground-truth pairs from actual user dashboards, testing whether models can find relevant tables, transform data, build joins and answer the business question without manual preparation. Even frontier models score under 50%. The authors' BI-Agent decomposes into search, join and transform subtasks orchestrated across specialized data-management methods, plus a post-training framework synthesizing trajectories from real projects.