MindPatternSubscribe

Source trail

evaluation.club — Prompts aren't Real

Public MindPattern findings, entities, and graph evidence that cite this source.

Findings

1

All-time hits

1

High value

0

Last seen

2026-09-21

Related findings

2026-09-21 / SKILLSAn argument for treating prompts as optimizer output: pass^k adversarial suites plus GEPA instead of hand curationThe post argues prompts are not the unit of analysis in a production agent, because each added prompt interacts unpredictably with existing context, making manual curation fail to scale. The proposed loop is concrete: have Claude generate a large set of adversarial and benign scenarios as pass^k tests, meaning each test runs k times so reliability rather than a single pass is the measurement, hook that suite to an optimizer such as GEPA, hold out a test set so the optimizer cannot memorize, and for fuzzy judgments like brand voice build a second optimized evaluator on a golden labeled dataset. In production, run pass^k continuously and use LLM judges on sampled conversations to convert failures back into test cases. The page carries no publication date; it was submitted to Hacker News on 2026-09-20 and reached 114 points.

Open latest cited source