Skills
Structured JSON state plus code-executable semantic actions beats screenshot-and-click by 4-8x on desktop app automation
ASIL's claim is that screenshots are state-incomplete and GUI clicks are semantically weak, so it exposes each application through structured JSON observations and code-executable actions via the deepest access path available. Across 15 applications and 380 tasks it clears 80 with closed models in under five actions per task, while screenshot-and-click under a repaired runtime and 50-step budget manages 6.6 and 26.6 strict success. It also trains well: small-scale SFT lifts Qwen3.5-2B from 58.0 to 72.1 and Qwen3.5-9B from 66.6 to 80.4, with on-policy RL reaching 74.4 and 82.2.
↳ Follow the thread