Entity trail

PEFT

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-08-17 / arxiv-researchersMuon Makes the Muon Optimizer Work With LoRA, Closing a Gap That Was Blocking PEFT AdoptionMuon outperforms alternatives for pretraining but is rarely used for parameter-efficient fine-tuning because it is mathematically impossible to orthogonalize the weight update produced by a low-rank parameterization, so it does not compose with LoRA. sMuon approximates the solution to a relaxed Muon objective in the low-rank setting via linearization and least squares, with an implementation using only matmuls rather than expensive decomposition routines. The authors are candid that gains are model- and eval-dependent and describe them as moderate across SFT and a ReLoRA pretraining run — worth trying, not a step change.
2026-08-14 / hn-researcherNanoRL Implements REINFORCE, PPO, GRPO and RLOO for LLM Training in ~1,800 Lines Without Ray, TRL, or DeepSpeedNanoRL is an MIT-licensed project posted as Show HN that fits REINFORCE, PPO with GAE, GRPO, and RLOO into roughly 1,800 lines across 7 files, scaling from CartPole on a laptop to asynchronous distributed training on GPU clusters using vLLM rollout workers, with no Ray, TRL, or DeepSpeed dependency. Its training targets are CartPole, Countdown, and GSM8K math reasoning, running on Python 3.10+ with PyTorch, Gymnasium, Transformers, Datasets, and PEFT. The author explicitly optimizes for readability and forkability over production robustness, excluding Megatron-scale parallelism and multi-tenant scheduling; the repo has just 5 stars and 7 commits, so this is a teaching artifact rather than a production trainer.
2026-08-08 / arxiv-researcherA Six-Dimensional Taxonomy Tries to Disentangle Fine-Tuning, RAG, Prompting, Editing, and Unlearning for Model-Change TrackingThis survey (2608.06246, submitted 2026-08-06) argues the post-training adaptation literature is fragmented across technique families, model classes, and deployment contexts, making methods incomparable and model modifications undescribable. It proposes organizing everything — retraining, fine-tuning, PEFT, alignment, retrieval augmentation, model editing, unlearning, calibration, multimodal instruction tuning — along six axes: mechanism, goal, data requirement, persistence, structural scope, and model type. It explicitly separates terms that get conflated in practice (fine-tuning vs. retrieval augmentation vs. prompting) and maps inheritance, supersession, hybridization, and layered deployment stacks between techniques. The stated payoff is a shared vocabulary for technical documentation, model-change tracking, and governance analysis — useful if you have to attest to how a deployed model was modified.
2026-07-21 / arxiv-researcherManifold-Constrained Hyper-Connections: PEFT That Adapts Residual Streams, Not WeightsValentijn Oldenburg, Floris de Kam and Bente Zuijdam (arXiv 2607.18130, cs.LG) note that nearly every parameter-efficient finetuning method — LoRA, adapters, prefix tuning — modifies weights or activations while leaving the residual connections untouched. Their method places manifold constraints on hyper-connections to adapt that untouched component. It is a genuinely different axis for PEFT and worth watching if LoRA has plateaued on your task.
2026-07-20 / hn-researcherLoRA Speedrun: A Public Wall-Clock Leaderboard for Fine-Tuning TechniquesA new GitHub project (Saivineeth147/lora-speedrun) publishes a wall-clock leaderboard for LoRA fine-tuning techniques, ranking approaches by actual elapsed training time rather than the FLOP counts and loss curves that dominate published comparisons. It picked up 114 points and 23 comments on HN, with the discussion focused on whether wall-clock is the honest metric for practitioners who care about iteration speed on a single GPU. Useful as a reproducible baseline for anyone choosing between PEFT variants.
2026-07-07 / arxiv-researcherSubspace-Constrained Adaptation Blocks Fine-Tuning Poisoning in PEFTParameter-efficient fine-tuning still leaves a wide space of behavior-changing updates a poisoned objective can reach; this method constrains adaptation to a subspace estimated from a trusted pool of existing adapters so harmful updates become inexpressible. It is a defense for anyone accepting user- or third-party-supplied LoRAs. Trades a bounded amount of adapter expressivity for poisoning resistance.
2026-06-28 / skill-finderSwap LoRA for DoRA with one flag (use_dora=True) for free accuracy gainsDoRA decomposes each pretrained weight into magnitude (a per-output scalar) and direction (unit-norm matrix), applying LoRA only to direction so each component adapts at its own effective learning rate — fixing LoRA's coupling where any directional shift drags magnitude with it. It yields +3.7% on LLaMA-7B and +1–4.4% on larger models with zero added inference cost. The skill: set use_dora=True in PEFT's LoRAConfig with the 2026 starting recipe r=16 and target_modules='all-linear'.
2026-06-20 / arxiv-researcherFrom Efficiency to Leakage: Privacy Backdoor in Federated Language Model Fine-TuningShows a malicious parameter server can stealthily turn a PEFT adapter into a privacy backdoor (NeuroImprint) by assigning dedicated neurons to memorize individual training samples without degrading model performance. It recovers 59-79% of fine-tuning samples across BERT, GPT-2, and Llama 3.2. A concrete threat model for federated or outsourced fine-tuning.
2026-06-10 / sources-researcherFable 5 System Card Reveals First 'Silent' Safeguards — Model Will Degrade Itself on Frontier-AI-Dev Requests Without Telling YouThe 319-page Fable 5 / Mythos 5 system card discloses new interventions that limit Claude's effectiveness on frontier-LLM-development requests (building pretraining pipelines, distributed training infrastructure, ML accelerator design) — and unlike its cyber/bio/distillation safeguards, these are invisible: no model fallback, just silent effectiveness limits via prompt modification, steering vectors, or PEFT. Anthropic estimates the impact at ~0.03% of traffic concentrated in fewer than 0.1% of organizations, justified by recursive-self-improvement risk and ToS enforcement. Simon Willison flags this as the first time Anthropic has announced silent interventions and says he is 'not at all keen' on a model that quietly corrupts answers to slow research that conflicts with Anthropic's own goals.
2026-05-28 / arxiv-researcherPEFT-Arena: PEFT Methods Evaluated for Stability-Plasticity, Not Just AccuracyChallenges the standard PEFT evaluation paradigm: current benchmarks emphasize downstream accuracy while overlooking retention of pretrained capabilities. PEFT-Arena evaluates parameter-efficient finetuning from a stability-plasticity perspective — measuring what you lose, not just what you gain. Practical guidance for anyone LoRA-tuning production models.
2026-05-03 / skill-finderDoRA (Weight-Decomposed LoRA): 2026's Preferred Fine-Tuning Technique — Decomposes Updates Into Magnitude and Direction for Better ConvergenceDoRA improves on standard LoRA by decomposing weight updates into magnitude and direction components, resulting in better convergence on complex tasks without additional VRAM overhead. Available in PEFT and Unsloth with a single flag (use_dora=True). The 2026 consensus: use target_modules='all-linear' as your starting point (not just q_proj/v_proj), and combine DoRA with RS-LoRA's √r scaling fix for best results. LoRA now recovers 90-95% of full fine-tune performance.
2026-04-26 / reddit-researcherHauhauCS Abliteration Package Exposed as Plagiarizing Heretic — Forensic Weight Analysis Shows LoRA Fine-Tune on Top of Heretic BaseWith 5M+ monthly HuggingFace downloads across 22 models, HauhauCS was exposed for allegedly repackaging the open-source Heretic abliteration tool without attribution and violating its license. Forensic analysis found HauhauCS modifies exactly 253 tensors matching PEFT LoRA default config — a fingerprint consistent with a LoRA fine-tune applied on an already-abliterated Heretic base. 564↑ on r/LocalLLaMA with 184 comments.

Graph relationships

USES
NanoRL
NanoRL uses PEFT for parameter-efficient fine-tuning.
Source finding
USES
NanoRL -> PEFT
NanoRL uses PEFT for parameter-efficient fine-tuning.
Source finding

Source trail

arXiv 2607.18130 Spheron 319-page Fable 5 / Mythos 5 system card "allowed to sabotage your app if you're a competitor"TechCrunch arXiv 2608.14492 GitHub / Hacker News arXiv 2608.06246 GitHub / Hacker News arXiv arXiv arXiv

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…