XHToken publishes Spark-X2.5 with native 1M context

XHToken published Spark-X2.5-4B and a 1.7B sibling with no blog post or launch thread, just model weights on Hugging Face under an Apache 2.0 license.

A fully open 4B model with a native 1M-token context window and no licensing restrictions changes what small local models can do. XHToken says the 4B version scores 90.7 on AIME 2026, one of four benchmark claims on the model card.

Neither model is a fine-tune of an existing base. XHToken built a hybrid attention architecture instead, one full-attention layer for every three sliding-window layers. It has native 1M-token context and training across more than 200 languages on about 20 trillion tokens.

XHToken's own benchmarks add three more figures for the 4B model: 65.1 on BFCL-V4, 75.1 on tau-squared-bench, and 44.4 on SWE-Bench Pro. All four scores are self-reported by XHToken.

The models don't run on stock llama.cpp yet. A pull request adding support, PR #27868, is still open. Until it merges, using Spark-X2.5 requires XHToken's custom fork or the pre-built GGUFs it ships alongside the weights.