Provenance-Aware Transformers give every input token a ring ID so the model stops inferring authority from wording

arXiv 2609.21088·medium signalXBlueskyLinkedInCopy link

Standard transformers push retrieved documents, user input and system instructions through the same undifferentiated attention, leaving the model to guess from phrasing what to obey, which is why indirect prompt injection persists. The proposed architecture assigns each token a ring ID encoding its source and adds origin embeddings, a learnable origin attention bias, and a learnable origin scale that survives normalization, enforcing the trust boundary structurally during generation rather than through instruction text. A two-stage fine-tuning pipeline retrofits the scheme onto already-released pretrained models, teaching origin semantics first and task behavior under ring constraints second, so this is a defense a team with fine-tuning access could attempt on an open-weight model.