Qwen3.8-27B Serving Configs Hit ~1,094 tok/s at 64 Concurrent on a Single Power-Limited RTX 3090

GitHub·medium signalXBlueskyLinkedInCopy link

syv-ai/qwen38-27b-rtx3090 publishes two vLLM configs for one 24 GB consumer card at a 250 W power limit with 150k context: a batch mode measuring ~1,094 tok/s steady-state decode at 64 concurrent (942 end-to-end, rising to ~1,222/1,042 with all layers int8), and a single-user mode reaching 114-122 tok/s single-stream via MTP speculation with four cheap drafts, a calibrated int4 lm_head and split-KV verify attention. The most striking number is 381 tok/s at 25k context when the model reproduces its own context, quoting a document or applying an edit, using DFlash2 with 15 drafted tokens per verify step. It notes speculation wins below roughly 8 concurrent users and plain batching above.