Anthropic Ships Code That Reads Model Intent at Any Layer

Anthropic published jacobian-lens on GitHub July 2, the code behind a paper titled 'Verbalizable Representations Form a Global Workspace in Language Models.' For interpretability researchers, it's a shortcut: read out what a model already means at any internal layer, not just the last one. The repo has picked up 1,394 stars under an Apache 2.0 license.

The method computes one average Jacobian over a corpus, capturing how the final layer's output responds to changes at an earlier layer. That matrix then transports a residual-stream vector from layer l into the final-layer basis, where the model's own unembedding decodes it into words. In the repo's notation: lens_l(h) = unembed(J_l @ h).

Those examples run against Qwen, an open-weights model, since the method needs a full corpus of activations to compute that averaged Jacobian. The repo also ships interactive layer-by-position visualizations, so you can watch a representation's implied output shift as it moves through the network.

Its README marks the release explicitly as an unmaintained reference implementation, not accepting contributions.