|
Constrained Horn Clauses (CHC) is a fragment of First Order Logic (FOL) that is sufficiently expressive to describe many verification, inference, and synthesis problems including inductive invariant inference, model checking of safety properties, inference of procedure summaries, regression verification, and sequential equivalence. The CHC competition (CHC-COMP) will compare state-of-the-art tools for CHC solving with respect to performance and effectiveness on a set of publicly available benchmarks. The winners among participating solvers are recognized by measuring the number of correctly solved benchmarks as well as the runtime. The results of the sixth CHC-COMP will be announced at the HCVS workshop affiliated with ETAPS 2024. Last year's edition of CHC-COMP was part of TOOLympics 2023 at ETAPS 2023. |
Competition Results from 2024
Participants (Tool, Submitter)
The list of participants will be made public, together with the competition results, at the HCVS workshop (April 7). If you would like to submit a solver, and have not yet been in touch with the organisers, please add a note in the Gitter chat, or send a mail to Gidon Ernst and Jose F. Morales.
Solver Submission
Solvers have to be provided in the form of a StarExec package (information about how to configure and upload a solver can be found in the StarExec user guide). As an example, you can have a look at the Eldarica package from 2019. Test your solver on StarExec before submitting! Submission is done by sending a link or the package to Gidon Ernst and Jose F. Morales.
Benchmarks
The competition benchmarks will be selected among the benchmarks available on Github. If you have benchmarks of any kind that can be made public, and that are not yet on Github, please upload! The format of the benchmarks is described here; a detailed description of the benchmark selection process can be found in CHC-COMP 2022: Competition Report. The final list of benchmarks used in the competition will be made public, together with the competition results, on April 7, 2024.
The following tracks will be evaluated at CHC-COMP 2024, which is the same as in 2023:
| LIA-lin: | Linear Integer Arithmetic, linear clauses | ||
| LIA-nonlin: | Linear Integer Arithmetic, nonlinear clauses | ||
| LIA-lin-Arrays: | Linear Integer Arithmetic + arrays, linear clauses | ||
| LIA-nonlin-Arrays: | Linear Integer Arithmetic + arrays, non-linear clauses | ||
| LIA-nonlin-Arrays-nonrecADT: | Linear Integer Arithmetic + arrays + non-recursive Algebraic data-types, nonlinear clauses | ||
| ADT-LIA-nonlin: | Algebraic data-types + Linear Integer Arithmetic, nonlinear clauses |
In addition to the theories listed in the track names, benchmarks can also use the Bool theory.
Important Dates for 2024
| Submission deadline for benchmarks considered for the competition: | Friday, March 8 2024 | ||
| Solver submission deadline for test runs (optional, but recommended): | Friday, March 15 2024 | ||
| Final solver submission deadline for evaluation: | Friday, March 22 2024 | ||
| Presentation of results: | April 7 2024 (HCVS workshop), and |
Solvers Evaluation & Ranking
The description of the evaluation process and the ranking method of solvers is described in CHC-COMP 2022: Competition Report.
Organizers
Gidon Ernst, LMU Munich, Germany
Jose F. Morales, IMDEA Software Institute, Spain
Big thanks to
- Everybody who organized and helped with
the competition infrastructure over the last years:
Nikolaj Bjørner, Adrien Champion, Emanuele De Angelis, Grigory Fedyukovich, Hari Govind V K, Arie Gurfinkel, Dejan Jovanovic, and Philipp Ruemmer - Aaron Stump and StarExec!
Previous editions of CHC-COMP
Last updated: 2024-04-08 13:22:59 +0000