Self-hosted Kimi K3 costs $190 per million tokens

A practitioner ran Kimi K3's 2.8 trillion parameters on 8 B300 GPUs through Modal, per a post on r/LocalLLaMA. That's the real cost of self-hosting a frontier model instead of renting one by the token. The meter runs whether or not anyone's asking it anything.

The setup ran vLLM with tensor parallelism across all eight cards and native MXFP4 quantization. The rig cost $56.79 an hour, and loading the 1.56 TB model took a 27-minute cold boot. Once running, time to first token landed between 0.92 and 1.02 seconds, with steady decode around 92 tokens per second.

A clean run cost roughly $36 in GPU time, about $190 per million output tokens. Keep the instance warm all day and it's $1,363.

A second attempt tried to cut cost by trading precision for cheaper hardware. Unsloth's 1-bit UD-IQ1_S quant, at 594 GB, ran on 8 A100-80GB GPUs through llama.cpp at $19.99 an hour. That's a third the price of the B300 setup. It didn't pay off. Throughput came in around 9 tokens per second, roughly a tenth the speed, and cost about $620 per million tokens, 3.3 times as much.

Cheaper hardware didn't make a cheaper model. It made a slower one that cost more per token, because the hourly rate isn't what sets cost per token. Throughput is.

Commenters on the thread flagged the catch. That $190 figure only holds with heavy batching, GPUs churning through concurrent requests. Run Kimi K3 for one person at a time and the per-token math falls apart.