Randomized map ordering cost an agent up to 28 seconds

Go's randomized map order slowed a coding agent's replies to 26 to 28 seconds, according to a September 15 postmortem. The bug hid inside a layer nobody thought to check. Worst-case wait time reached 514 seconds before anyone fixed it, for a slowdown that looked like a model problem and wasn't.

The postmortem traces the cause to Bifrost, the gateway that decodes MCP tool definitions into Go maps before passing them to the model. Go deliberately randomizes map iteration order by design, so code can't depend on it. Every 10 minutes, when the gateway refreshed its tool schemas, the keys came back in a different order.

One reshuffled schema broke vLLM's shared prefix cache after 12,622 tokens. About 107,000 remaining tokens had to be re-read from scratch instead of pulled from cache.

Sorting schema keys in the chat template with tojson(sort_keys=True), combined with upstream fixes to Bifrost and the mcp-go library, made key order stable across refreshes. Cache hit rate rose from 55% to 95%. Average time to first token dropped to 7.3 seconds, and the worst case fell to 54 seconds.

Teams debugging slow agent responses check the model long before they check whether their gateway serializes tool schemas deterministically, and that order is usually backwards. A KV cache matches prefixes byte for byte, not by meaning, so a nondeterministic serialization layer breaks it with no warning in the logs.

The postmortem doesn't say how long the bug ran before anyone traced it to map order, or whether other Go-based MCP gateways share the exposure.