VeriContest Lite

Overview

Running all 1,006 VeriContest problems with a frontier model or an agent is expensive. VeriContest Lite is a fixed subset of 300 problems, 215 from LeetCode and 85 from Codeforces in the full benchmark's ratio, so that a system can be evaluated, compared, and iterated on at about 30% of the cost.

Lite keeps what makes the full benchmark hard. Each source and difficulty level keeps its share of the full benchmark, and Lite follows the full benchmark's distribution of reference proof sizes, including its largest proofs. It covers the same six task settings (End2End, ProofGen, SpecGen, and CodeGen from three inputs), uses the same release, test cases and evaluation, and is a leaderboard of its own: Lite submissions are ranked on Lite, and every full-benchmark submission is also scored on the Lite problems.

Selection Criteria

Lite should rank systems the way the full benchmark does, at a fraction of the cost. With only 300 problems, every problem moves a score by a third of a point, so each one has to be self-contained, unambiguous to score, and stable to evaluate. The selection therefore removes the problems that do not meet that bar, then samples the rest so that Lite keeps the full benchmark's mix.

Filters

Starting from the 1,006 problems of release 2026-10.1, a problem is removed when:

Its description contains an image or a hyperlink. Models receive the description as text. An image or a linked page can carry information the model never sees, and a link can change after the release.
Its description refers to another problem. Each Lite problem should be understandable on its own, without knowledge of a different problem.
Its outputs are judged by a problem-specific checker rather than by exact comparison. These problems accept several valid answers. Keeping Lite to exact comparison makes its results simple to reproduce with any evaluation harness.
Its reference solution uses more than half of the 1 s per-case time limit on some test case. Near the limit, whether a correct program finishes in time depends on the machine. The margin keeps verdicts the same across hardware.

914 problems remain.

Sampling

300 of these problems are sampled with a fixed seed:

Each source and difficulty level gets its full-benchmark share of the places. Lite therefore keeps the original distributions of problem sources and difficulty levels: LeetCode and Codeforces problems, each LeetCode difficulty, and each Codeforces rating band make up the same share of Lite as of the full benchmark.
Within each group, problems are ranked by reference proof size and cut into one interval per place; one problem is drawn from each interval. A plain random sample could happen to include few problems with large proofs. Drawing one problem from each interval makes Lite cover the whole range of proof sizes in the same proportions as the full benchmark, including the largest proofs.
When the drawn problem was removed by a filter, the remaining problem of the same group closest to it in proof size takes its place. The replacement keeps both the group and the proof size of the problem it stands in for.

The selection is reproducible with leaderboard/make_lite.py, and the list is pinned by its checksum in the release.

Composition

Problems by source and difficulty
Source	Level	Full	Lite
LeetCode	Easy	359	107
	Medium	302	90
	Hard	61	18
Codeforces	Rating 800	118	35
	900–1000	40	12
	1100–1200	49	15
	1300–1400	40	12
	1500+	37	11
Total		1,006	300

Reference proof lines
	Median	75th pct.	90th pct.
LeetCode · Full	77	180	326
LeetCode · Lite	76	178	312
Codeforces · Full	110	229	351
Codeforces · Lite	108	218	351

Proof size counts the non-blank lines a reference proof adds to the fixed code and specification. 30 Lite problems are among the 10% of benchmark problems with the largest proofs.

Citation

If you use VeriContest Lite, please cite VeriContest.

@article{xie2026vericontest,
  title   = {{VeriContest}: A Competitive-Programming Benchmark for Verifiable Code Generation},
  author  = {Xie, Zichen and Pawagi, Mrigank and Liu, Yuxin and Rai, Aaditi and
             Shao, Lize and Berberian, Jr., John and Che, Sicong and Wang, Wenxi},
  journal = {arXiv preprint arXiv:2605.08553},
  year    = {2026}
}