Exact-match scoring rejects 75% of correct text-to-SQL queries once AI operators are in the SQL.

This paper measures the damage non-deterministic AI operators do to execution-accuracy scoring: traditional Execution Accuracy detects as few as 25% of correct translations, and a state-of-the-art LLM autorater falsely rejects 32% because it judges relational logic and AI semantics at once. Validating deterministic database logic separately from flexible AI operations reaches up to 97.2% accuracy across both BigQuery and ThalamusDB. If you're benchmarking a text-to-SQL agent against a warehouse with LLM functions, the metric is producing your bad numbers.