Kimi Vendor Verifier: Open-Source Tool Exposes Accuracy Gaps Across Inference Providers

Moonshot AI·medium signal

Moonshot AI updated its K2 Vendor Verifier (KVV) to visually compare tool-call accuracy across 12 inference providers (up from 9), using 4,000 requests per provider compared against the official Moonshot API. The 262-point HN story (25 comments) highlights significant variation in tool-call performance — what you get from Provider A vs Provider B for the same model weights can differ meaningfully. For builders running Kimi K2.x via third-party providers, this is essential infrastructure: KVV measures both tool_call_f1 (correctness) and schema_accuracy (engineering robustness).