Qwen and DeepSeek Encode Same Code Concepts, Different Layers

Two code models agree on which grammar concepts deserve dedicated circuitry but disagree completely on where those circuits sit, per Piotr Wilam's comparison of Qwen2.5-Coder-7B and DeepSeek-Coder-V1-6.7B.

That's a problem for anyone comparing code models by layer number. A probe tuned to one model's layer won't find the matching circuit in another. Qwen clusters this work at layers 17-19; DeepSeek does the same job at layers 6-7.

The agreement on concepts is strong. Spearman correlation between the two models hits 0.638 for Python and 0.673 for Rust, both p < 1e-7. The comparison covers 58 Python concepts and 57 Rust concepts, tagged the same way across all four model-language pairs.

Rust also draws more machinery than Python in both models, 2 to 3 times more concept-specific circuitry per construct. Qwen goes further, binding nine Rust type-and-trait keywords into one neuron cluster, a Jaccard score of 0.535 against a null baseline of 0.112.

The paper doesn't say why DeepSeek compresses the work into 2 layers while Qwen spreads it across 3. It also doesn't say whether the split holds outside these two model families. Its own conclusion is blunt: interpretability tooling that assumes layer positions transfer between models is measuring the wrong thing.