Datasets

Full benchmark

VeriContest

Gax-c/VeriContest

All problems of the benchmark, for full evaluations and leaderboard submissions.

1,006 problems: 722 LeetCode, 284 Codeforces
About 49 GiB of test cases
80 problems with an output checker
Release v2026.10.1

🤗Hugging Face About VeriContest

300-problem subset

VeriContest Lite

Gax-c/VeriContest_Lite

A fixed subset with the full benchmark's mix of sources, difficulty levels and proof sizes, for smaller evaluations.

300 problems: 215 LeetCode, 85 Codeforces
About 17 GiB of test cases
All outputs judged by exact comparison
Release v2026.10.1

🤗Hugging Face About VeriContest Lite

The Lite problems and test cases are identical to the corresponding problems of VeriContest. The problem files without test cases are also in the GitHub repository.

What Each Problem Contains

description.md: the natural-language problem statement, with starter code
spec.rs: the formal specification (preconditions and postconditions)
code.rs: judge-accepted Rust code
code_spec.rs: the fixed code and specification given for proof generation
verified.rs: the full Verus-verified program with specification, code and proofs
main.rs: the stdin/stdout entry point, for Codeforces problems
tests/testcases.jsonl and tests/mutated_testcases.jsonl: positive and negative test cases
tests/checker.py: the output checker, for problems with several valid answers

Each dataset also has problems.jsonl, one row per problem without the test cases, which is what the Hugging Face Dataset Viewer shows.

Loading

from datasets import load_dataset

problems = load_dataset("Gax-c/VeriContest", split="test")       # 1,006 problems
lite = load_dataset("Gax-c/VeriContest_Lite", split="test")      # 300 problems

To evaluate, download a dataset and pass its folder to the scorer:

hf download Gax-c/VeriContest_Lite --repo-type dataset --local-dir VeriContest_Lite
python3 evaluation/score.py --split lite --testcases VeriContest_Lite ...

See the evaluation guide for the full commands.