Entity trail

PPO-RLHF

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-06-21 / skill-finderDefault new alignment runs to DPO/ORPO/KTO, and use Reinforcement Fine-Tuning for verifiable tasksProduction teams in 2026 have largely replaced PPO-RLHF with DPO and its descendants ORPO and KTO, which train directly on preference pairs via a closed-form contrastive loss — no reward model, no PPO instability, no rollout loop — at comparable quality for most tasks. For tasks with checkable answers (math, code, structured extraction), Reinforcement Fine-Tuning (RFT) rewards the model for getting verifiable outcomes right rather than imitating a reference answer. Start from a LoRA/QLoRA run on an open-weight model, which now costs a few hundred dollars on a single high-memory GPU versus the $50K+ clusters of 2022.
2026-05-20 / projects-researcherLlamaFactory Trends at 71K Stars — Unified Fine-Tuning Framework Now Covers 100+ LLMs and VLMshiyouga's LlamaFactory provides a unified interface for efficient fine-tuning of over 100 large language models and vision-language models, supporting LoRA, QLoRA, full fine-tuning, and RLHF workflows. At 71,436 stars and trending on GitHub, it remains the most popular open-source fine-tuning toolkit — especially relevant as teams need to fine-tune models on domain-specific data for the agent workflows that Antigravity 2.0 and Managed Agents are enabling.
2026-04-21 / arxiv-researcherBRRL: Bounded Ratio RL Bridges Gap Between Trust Region Theory and PPO's Heuristic ClippingBounded Ratio Reinforcement Learning formally connects trust region optimization theory with the heuristic clipped objective used in PPO, which has a significant theoretical disconnect despite being the dominant on-policy RL algorithm. BRRL introduces a principled regularized and constrained policy optimization framework. Relevant to RLHF/RLVR practitioners who build on PPO for LLM alignment — provides theoretical grounding for ratio-clipping decisions.

Graph relationships

DEPRECATES
DPO
DPO has replaced PPO-RLHF as the standard fine-tuning approach.
Source finding
DEPRECATES
DPO -> PPO-RLHF
DPO has replaced PPO-RLHF as the standard fine-tuning approach.
Source finding

Source trail

hjLabs — LLM Fine-Tuning Best Practices 2026 GitHub arXiv

Graph sources

entity graphfindings textkg edgeskg entities

Fetching from the wire…