Leveraging Causal Graphs to Improve LLM-Based Causal Reasoning: an Empirical Study

Large Language Models (LLMs) have demonstrated strong performance across a wide range of reasoning tasks; however, their ability to perform genuine causal reasoning remains a subject of active debate. In this work, we aimed to assess whether integrating a causal graph can enable an LLM to outperform a stronger model on causal tasks grounded in the information encoded in the graph. To this end, we designed an experiment by which we compare three different team configurations, varying in model strength, coordination strategy and presence or absence of the graph, using both human evaluation and graph-based evaluation. Results show that human evaluators do not exhibit statistically significant preferences across teams, tending instead to favor outputs characterized by stronger linguistic fluency and apparent reasoning quality. In contrast, the graph-based evaluation reveals systematic differences, with the team equipped with the causal tool producing outputs more closely aligned with the ground-truth causal model. Moreover, although the teams without access to the causal graph often produced plausible and satisfactory explanations, these were frequently not consistent with the information encoded in the graph. These findings suggest that while linguistic sophistication strongly influences user perception, access to structured causal knowledge and appropriate process design can substantially improve correctness.