CausalsCan it show the path?
Identify relevant entities, events, relationship meanings, and sources instead of producing an unsupported conclusion.
Causals Bench
Test reasoning,
not just fluent answers.
We are designing evaluations for systems that combine a graph with a language model. The goal is to measure whether an answer follows the evidence, handles uncertainty, and can be corrected.
Questions worth measuring
Can it separate claims?
Distinguish association, reported causation, a structural mechanism, and a historically tested pattern.
Can it change its mind?
Respond appropriately when an edge is disputed, a source is withdrawn, or a counterexample appears.
Current status
Evaluation design is open for discussion.
There is no public benchmark dataset or leaderboard announced yet. We want to define tasks, answer criteria, annotation quality checks, licensing, and leakage controls before releasing one.
We welcome suggestions for questions and evaluation methods. A strong proposal names the evidence a good answer should use, the errors it should avoid, and the circumstances under which “uncertain” is the right answer.
Propose an evaluation idea ↗Release discipline
Training examples, public evaluation tasks, and private test material may need different release terms. Their availability will be stated explicitly rather than assumed from this website.
Training examples, public evaluation tasks, and private test material may need different release terms. Their availability will be stated explicitly rather than assumed from this website.