Entity trail

Sparse Autoencoder

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-07-27 / arxiv-researcherSparse-Autoencoder-Discovered Jailbreak Scenarios Transfer to GPT-5, Claude-Haiku-4.5, and Gemini-3-Flash, Adding Up to 18.2 Points of Attack SuccessConcept2Scenario (arXiv 2607.23496) moves scenario-based jailbreaking from trial-and-error to mechanism: the authors show scenario-wrapped prompts activate internal 'scenario directions' whose causal steering measurably reduces refusal scores, then use a sparse autoencoder to instantiate a broad concept space, attribute refusal suppression to individual concepts, and translate them back into interpretable natural-language scenarios. Across three open-source models, two safety benchmarks, and six black-box jailbreak methods, the discovered scenarios act as reusable priors that raise average attack success rates by up to 18.2 percentage points. Critically, they transfer to GPT-5, Claude-Haiku-4.5, and Gemini-3-Flash, and identified scenario combinations outperform their individual parts while cutting the number of turns needed for iterative attacks.
2026-07-12 / arxiv-researcherS²AE Makes Sparse Autoencoders Learn Modality-Consistent Concepts in VLMsSparse autoencoders are a leading mechanistic-interpretability tool, learning sparse latent features that each encode a distinct concept, but in vision-language models vanilla SAEs learn concepts with fragmented, disjoint coverage in the visual modality. The proposed Structured Sparse AutoEncoder (S²AE) enforces structure so concepts stay consistent across text and image modalities. Advances interpretability tooling for multimodal models specifically, not just text LLMs.
2026-06-28 / arxiv-researcherBeyond the Hard Budget: Sparsity Regularizers for More Interpretable Top-k Sparse AutoencodersSparse autoencoders are now a leading tool for interpreting vision foundation model representations, but the standard Top-k SAE enforces a rigid hard sparsity budget. This work (Jacquier, Vakalopoulou, Hosseini) replaces that budget with sparsity regularizers, yielding more monosemantic, interpretable features. Practical for anyone building interpretability tooling on top of vision encoders.
2026-05-02 / arxiv-researcherDo Sparse Autoencoders Actually Capture Concept Manifolds? Evidence Says Not AlwaysChallenges a core assumption in mechanistic interpretability: that concepts in neural networks correspond to independent linear directions extractable by SAEs. Growing evidence shows concepts live on manifolds, not lines, and SAEs may systematically miss or fragment these nonlinear structures. Directly relevant to the Qwen-Scope release and anyone building SAE-based steering or interpretability tools — the linear features you extract may be incomplete projections of richer concept geometry.
2026-04-30 / reddit-researcherQwen-Scope: Official Sparse Autoencoders Released for Qwen 3.5 Models — Interpretability Across 2B to 35B MoEThe Qwen team released Qwen-Scope, a collection of Sparse Autoencoders (SAEs) mapping internal features for the residual stream across all layers of the Qwen 3.5 family, from the 2B dense model to the 35B MoE. This enables mechanistic interpretability research on one of the most widely deployed open-weight model families. 133 upvotes on r/LocalLLaMA. This is significant because SAE releases from the original model team provide ground-truth feature mappings that third-party researchers can't easily replicate.
2026-04-21 / arxiv-researcherCommit-Open Protocol Detects Silent Model Substitution in Hosted LLM Services Using SAE Feature TracesProposes a Merkle-tree-based commit-open protocol using sparse autoencoder (SAE) feature traces to detect when hosted LLM providers silently substitute a cheaper model for an advertised stronger one. Prior verification schemes (like SVIP) leave a parallel-serve side-channel exploitable by dishonest providers. This is the first cryptographic approach to closing the model-substitution gap — relevant as more teams pay premium pricing for specific model tiers.
2026-03-16 / sources-researcherSteerRM: Inference-Time Reward Model Debiasing via Sparse Autoencoders — +7.3pts on Hard Cases, No Retraining RequiredSteerRM identifies that reward models suffer from representation entanglement where format/style biases (favoring better-structured responses over semantically superior ones) are concentrated in shallow layers and transfer across architectures. Using sparse autoencoders, the approach isolates bias-related features via a strength-stability criterion from contrastive response pairs, then suppresses them at inference time without any retraining. The result is 7.3-point accuracy improvements on hard evaluation cases while preserving overall performance, offering alignment practitioners a drop-in, interpretable fix for a fundamental reward model failure mode.

Graph relationships

USES
Concept2Scenario
Concept2Scenario uses sparse autoencoders to discover jailbreak scenarios.
Source finding
USES
Concept2Scenario -> Sparse Autoencoder
Concept2Scenario uses sparse autoencoders to discover jailbreak scenarios.
Source finding

Source trail

arXiv 2607.23496 arXiv arXiv arXiv r/LocalLLaMA arXiv arXiv

Graph sources

entity graphfindings textkg edgeskg entities

Fetching from the wire…