Primary study · included by 1 review
Evaluation of the Performance of Generative AI Large Language Models ChatGPT, Google Bard, and Microsoft Bing Chat in Supporting Evidence-Based Dentistry: Comparative Mixed Methods Study.
Giannakopoulos 2023 · Giannakopoulos Kostis, Kavadella Argyro, Aaqel Salim Anas, et al. · Journal of medical Internet research · 2023
Included by
| Outcome this study informs | Pooled effect | Evidence | Certainty |
|---|---|---|---|
Accuracy (proportion of correct answers) Any large language model vs Reference standard (official answer key, validated question bank or expert consensus) · Questions answered by LLMs (studies searched to January 2025) | Proportion, percent (random effects, Freeman-Tukey double arcsine) 63.69 [60.26, 67.06] | 39 studies · 17021 participants | GRADE not reported |
Accuracy (proportion of correct answers) Any large language model vs Reference standard (official answer key, validated question bank or expert consensus) · Questions answered by LLMs (studies searched to January 2025) | Leave-one-out analysis demonstrated stable pooled accuracy estimates ranging from 64.13% to 65.66%, with the most influential study (Chau et al) altering results by 1.96% upon exclusion | NR | GRADE not reported |
Accuracy (proportion of correct answers) Any large language model vs Reference standard (official answer key, validated question bank or expert consensus) · Questions answered by LLMs (studies searched to January 2025) | Proportion, percent (leave-one-out) 63.47 [62.69, 64.25] | NR | GRADE not reported |
Accuracy (proportion of correct answers) Any large language model vs Reference standard (official answer key, validated question bank or expert consensus) · Questions answered by LLMs (studies searched to January 2025) | Prediction interval [29.65; 91.71] | 39 studies · 17021 participants | GRADE not reported |
Accuracy (proportion of correct answers) Any large language model vs Reference standard (official answer key, validated question bank or expert consensus) · Questions answered by LLMs (studies searched to January 2025) | Proportion, percent (leave-one-out) 65.04 [64.26, 65.81] | NR | GRADE not reported |
Accuracy (proportion of correct answers) Any large language model vs Reference standard (official answer key, validated question bank or expert consensus) · Questions answered by LLMs (studies searched to January 2025) | Proportion, percent (fixed effect) 64.87 [64.13, 65.61] | 39 studies · 17021 participants | GRADE not reported |
Accuracy (proportion of correct answers) Any large language model vs Reference standard (official answer key, validated question bank or expert consensus) · Questions answered by LLMs (studies searched to January 2025) | Proportion, percent (leave-one-out) 66.83 [66.07, 67.60] | NR | GRADE not reported |
Accuracy (proportion of correct answers) Any large language model vs Reference standard (official answer key, validated question bank or expert consensus) · Questions answered by LLMs (studies searched to January 2025) | Proportion, percent (random effects, Freeman-Tukey double arcsine) 65.43 [61.88, 68.89] | NR | GRADE not reported |
1 including review in total.