Compile your agent's repeated leaf decisions into abstainable reflex rules and skip 15-20% of LLM calls

arXiv 2608.16447·medium signalXBlueskyLinkedInCopy link

HaReCAP mines successful trajectories for frequently recurring leaf-level decisions and compiles them offline into one-step reflex rules that are auditable and can abstain: at runtime, if a stored rule maps uniquely to a legal action in the current valid-action set, the LLM call is bypassed; otherwise it falls back to the normal recursive path. On Qwen3.5-27B this cut tokens on solved tasks by 14.67% (Robotouille sync), 17.93% (async), and 20.08% (ALFWorld) without changing the control flow. The abstain-on-ambiguity design is the transferable part — it's a cache that refuses to guess.