Fact-Driven Health Information Retrieval: Integrating LLMs and Knowledge Graphs to Combat Misinformation

Aiming to reduce health misinformation in Web search, we present a system for Health Information Retrieval (HIR) that ranks documents according to both their topical relevance and correctness (i.e., factuality). The system first segments documents into passages and then employs a Large Language Model (LLM) to identify and extract claims from each passage. For each claim, we formulate corresponding SPARQL queries and execute them against a Knowledge Graph (KG) extracted from a subset of DBpedia, allowing us to estimate the correctness of claims and, hence, a correctness score for documents. Topical relevance is estimated with the BM25 algorithm, which is used to produce the initial ranking of documents. To generate the final ranking, the system combines each document’s pre-computed correctness score with its topical relevance score. While existing approaches rely on machine learning or LLMs to verify correctness, our KG-based methodology enables transparent fact-checking by grounding its assessments in structured knowledge. Our approach is empirically evaluated using three TREC Health Misinformation collections (2020–2022).