How the map is built

Summary of Findings tier (4,913 Cochrane reviews). Every Cochrane review with a Summary of Findings table, parsed from the Cochrane Library HTML (corpus through 2024). Only the outcomes presented in those tables appear, about seven per comparison, so these reviews show their headline questions, not every analysis.

Full text tier (430 reviews: Cochrane Oral Health and non-Cochrane dental caries). Every question was extracted from the full text, figures and supplements by two independent LLM extractors, disagreements were adjudicated, and each row keeps a source location in the review. These reviews also list their included primary studies and, for intervention reviews, AMSTAR 2 ratings.

Each question has three separate fields: whether evidence exists (evidence status), whether it was GRADE rated, and the GRADE rating itself. The six display categories combine them in a fixed order: No studies, Ungraded, Very low, Low, Moderate, High. They are a display order, never a score, average or rank.

No studies means the review looked and found nothing to answer the question: a true evidence gap. Ungraded means an effect was reported without a GRADE rating, which is common in reviews that predate GRADE; it is a grading gap, not an evidence gap. A gap map cell with no review means no eligible review addressed that pair, not that no primary evidence exists.

Reviews on the same question often include the same trials. Counting them as independent would double count evidence, so every included study is resolved to one record (5,083 studies; 277 appear in two or more reviews) using PMID, DOI or NCT, and otherwise first author and year as a probable match. Review pages list the reviews they share studies with, and question clusters report the corrected covered area. Overlap is described, never used to adjust a GRADE rating.

Trial pages show each primary study, the reviews that include it and the review outcomes it informs. Trial-level PICO and per-trial results are not extracted yet (0 and 0 so far); the pages carry the slots so the extraction run can fill them without changing the site.

AMSTAR 2 applies to intervention reviews only and is not a numeric score. Both extractors' item ratings are shown side by side until reconciled; the overall rating follows Shea et al. 2017 (critical items 2, 4, 7, 9, 11, 13 and 15). Cochrane reviews are marked "not assessed", not assumed to be high quality.

Questions are grouped across reviews by review type, intervention category and outcome domain. That key is a proposal pending human validation. A stratified gold standard sample of reviews is set aside for double human coding, to measure how reliable the extraction is.

The Cochrane Library (review metadata, Summary of Findings tables and full text), PubMed and Europe PMC (review discovery and full text), Epistemonikos, and PubMed and Crossref for bibliographic details of included studies. Abstracts and plain language summaries are not reproduced; each record links to its source.

Sofi-Mahmudi A. Evidence Gaps: a map of systematic review questions, their PICO and certainty of evidence. evidencegaps.com. Data and code: see the API.