Entity trail

Test Case Scaling

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-06 / arxiv-researcherTest Case Scaling Trains a Test Generator Adversarially Against the Solver's Current Failure ModesRL for code generation depends on test cases that are both sound and discriminative, and those are scarce. TCS reframes test generation as an adversarial RL problem where the generator must produce counterexamples targeting the solver's present failure modes, training from a rolling policy-aligned buffer in two stages: stage 1 generates tests consistent with the reference solution, stage 2 restricts the buffer to current failure modes and learns counterexample tests. Across TACO and LiveCodeBench it improves both pass@1 and inference-time answer selection, and the learned generator also works for selecting among outputs from other LLMs.
2026-08-14 / arxiv-researcherCoins: Evaluating LLM Formal Specifications by Instantiating Them on Trusted Test Cases Instead of Proving EquivalenceExisting evaluations of LLM-generated formal specifications require verifying implementation conformance or proving semantic equivalence between specs — both formidably hard, and both conflate proof difficulty with specification quality. Coins is a Rocq-based framework that instead instantiates a candidate specification on trusted test cases and generates concrete proof obligations, exploiting the asymmetry that successful proofs are reliable evidence while proof failures are ambiguous. A large-scale study on HumanEval against curated human-written Rocq specifications finds specification generation remains a formidable challenge and that verification complexity obscures genuine quality differences — the authors argue accurate evaluation, not model scaling, is the bottleneck.

Graph relationships

BENCHMARKED AGAINST
LiveCodeBench
Test Case Scaling was evaluated on LiveCodeBench.
Source finding
BENCHMARKED AGAINST
TACO
Test Case Scaling was evaluated on TACO benchmark.
Source finding

Source trail

arXiv 2609.03955 arXiv 2608.13077

Graph sources

entity graphfindings textkg edgeskg entities

Fetching from the wire…