713,564 Prompts From 4,000 Employees Show GenAI Sophistication Does Not Improve Over Time or After Training

arXiv 2608.27364·medium signalXBlueskyLinkedInCopy link

Using proprietary data covering 713,564 employee prompts and the corresponding model responses from nearly 4,000 back-office employees across 15 functional areas over eight months of 2025, the authors find senior employees use genAI more sophisticatedly, consistent with domain expertise complementing the tool. Sophistication varies widely by function and peaks in Strategy, Digital Innovation and Project Management. The uncomfortable result for anyone budgeting AI enablement: they observe neither improvement over time nor lasting improvement after formal AI training.