SageMaker concurrency sweeps show the gap between peak throughput and SLA-safe throughput.

AWS walks through CreateAIBenchmarkJob against a vLLM endpoint on a Blackwell-backed ml.g7e.2xlarge. Nemotron-3 Nano 30B plateaued at 2,823 output tokens/s at 256 concurrent requests. Add a 50s end-to-end and 1.5s TTFT SLA and safe concurrency drops to 80. That gap is the number to size against, not the peak.