Select LoRA target modules by task: o_proj + fc2 gives ~15% absolute gain on reasoning

Amazon Science·medium signal

Amazon Science found that LoRA target-module choice matters more than rank for reasoning quality: applying adapters to o_proj + fc2 yielded a 15% absolute improvement over the base model, with o_proj-only a reliable default (validated on Nova 2.0 Lite multimodal reasoning). Pair this with rank r=32–64 for multi-step reasoning tasks rather than tuning all linear layers.