Benefits and Challenges of Using Learning Analytics in Medical Education: A Scoping Review

Articles in Press

Document Type : Review Article

Authors

1 Center for Educational Research in Medical Sciences, Department of Medical Education, School of Medicine, Iran University of Medical Sciences, Tehran, Iran

2 School of Health, Education, Policing and Sciences, Staffordshire Univercity, Staffordshie, UK

10.30476/ijvlms.2025.107744.1345

Abstract

Background: Learning analytics includes collecting and analysing learners’ data to improve and personalize the learning process. In medical education, its potential value is significant because learning outcomes directly impact patient care. Whereas, research in this field remains scattered. Therefore, this scoping review aimed to map existing literature on the use of learning analytics in medical education, focusing on its definitions, applications, benefits, and challenges, to identify evidence gaps and guide future research and practice.
Methods: Following the Arksey and O’Malley scoping review framework and guided by the PRISMA-ScR checklist, a scoping review of publications addressing learning analytics in medical education was conducted. Relevant literature was identified through searches of international databases, including Scopus, PubMed, Web of Science (WOS), and Education Resources Information Center (ERIC). Searches used predefined keywords and were limited to English-language publications published between 2010 and 2024. Eligible studies included empirical and review articles within all academic levels in medical education context. Two reviewers independently screened and charted the data, and results were synthesized thematically across key domains.
Results: Initially, a total of 2,056 articles were identified. During the first screening stage, studies were filtered based on the relevance of their titles and abstracts, resulting in 196 articles advancing to the second phase. After a thorough full-text review, 20 articles that met all inclusion criteria were finally selected for analysis. The extracted findings were categorized into four main themes: definitions, applications and advantages, disadvantages and challenges, and general information on learning analytics.
Conclusion: Learning analytics offers considerable potential to enhance medical education through personalized learning, better decision-making, and improved outcomes for learners and patients. Yet, its adoption remains limited and fragmented, with challenges including ethical concerns, technical barriers, and the lack of standardized frameworks. Future research should develop standardized frameworks, address ethical and technical issues, and evaluate impacts on learner outcomes and patient care. Current evidence may be biased due to publication and language limitations; future research should include non-English sources and larger, more diverse datasets to enhance validity and generalizability.

Highlights

Zohreh Sohrabi (Google Scholar)

Zohreh Hosseinzadeh (Google Scholar)

Keywords

References

  1. Hervatis V, Loe A, Barman L, O'Donoghue J, Zary N. A Conceptual Analytics Model for an Outcome-Driven Quality Management Framework as Part of Professional Healthcare Education. JMIR Med Educ. 2015;1(2):e11. doi: 10.2196/mededu.4789. PubMed PMID: 27731840; PubMed Central PMCID: PMC5041364.
  2. Ferguson R. The State of Learning Analytics in 2012: A Review and Future Challenges. Technical Report KMI-12-01. Knowledge Media Institute. UK: The Open University; 2012.
  3. Cooper A. A Brief History of Analytics. JISC CETIS Anal Ser. 2012;1(9):1-21.
  4. Poitras EG, Naismith LM, Doleck T, Lajoie SP. Using Learning Analytics to Identify Medical Student Misconceptions in an Online Virtual Patient Environment. 2016;20(2):183-94. doi: 10.24059/olj.v20i2.802.
  5. Ellaway RH, Pusic MV, Galbraith RM, Cameron T. Developing the role of big data and analytics in health professional education. Med Teach. 2014;36(3):216-22. doi: 10.3109/0142159X.2014.874553. PubMed PMID: 24491226.
  6. Berland M, Baker RS, Blikstein P. Educational Data Mining and Learning Analytics: Applications to Constructionist Research. Tech Know Learn. 2014;19:205-20. doi: 10.1007/s10758-014-9223-7.
  7. Long PD, Siemens G. Penetrating the Fog: Analytics in Learning and Education. TD Tecnologie Didattiche. 2014;22(3):132-37. doi:10.17471/2499-4324/195.
  8. Méndez G, Ochoa X, Chiluiza K, de Wever B. Curricular Design Analysis: A Data-Driven Perspective. J Learn Anal. 2014;1(3):84-119. doi: 10.18608/jla.2014.13.6.
  9. Barneveld AV, Arnold KE, Campbell JP. Analytics in Higher Education: Establishing a Common Language. ELI Paper 1: 2012.
  10. Bigdeli S, Hosseinzadeh Z, Dehnad A, Sohrabi Z, Aalaa M, Haghani F, and colleagues Underpinning Learning Theories of Medical Educational Games: A Scoping Review. Med J Islam Repub Iran. 2023;37:26. doi: 10.47176/mjiri.37.26. PubMed PMID:
  11. Chan T, Sebok-Syer S, Thoma B, Wise A, Sherbino J, Pusic M. Learning Analytics in Medical Education Assessment: The Past, the Present, and the Future. AEM Educ Train. 2018;2(2):178-187. doi: 10.1002/aet2.10087. PubMed PMID: 30051086; PubMed Central PMCID: PMC6001721.
  12. Aronsky D, Jones I, Lanaghan K, Slovis CM. Supporting patient care in the emergency department with a computerized whiteboard system. J Am Med Inform Assoc. 2008;15(2):184-94. doi: 10.1197/jamia.M2489. PubMed PMID: 18096913; PubMed Central PMCID: PMC2274795.
  13. Dowding D, Randell R, Gardner P, Fitzpatrick G, Dykes P, Favela J, Hamer S, Whitewood-Moores Z, Hardiker N, Borycki E, Currie L. Dashboards for improving patient care: review of the literature. Int J Med Inform. 2015;84(2):87-100. doi: 10.1016/j.ijmedinf.2014.10.001. PubMed PMID: 25453274.
  14. Sebok-Syer SS, Klinger DA, Sherbino J, Chan TM. Mixed Messages or Miscommunication? Investigating the Relationship Between Assessors' Workplace-Based Assessment Scores and Written Comments. Acad Med. 2017;92(12):1774-79. doi: 10.1097/ACM.0000000000001743. PubMed PMID: 28562452.
  15. Li SA, Sherbino J, Chan TM. McMaster Modular Assessment Program (McMAP) Through the Years: Residents' Experience With an Evolving Feedback Culture Over a 3-year Period. AEM Educ Train. 2017;1(1):5-14. doi: 10.1002/aet2.10009. PubMed PMID: 30051002; PubMed Central PMCID: PMC6001587.
  16. Chan TM, Sherbino J, Mercuri M. Nuance and Noise: Lessons Learned From Longitudinal Aggregated Assessment Data. J Grad Med Educ. 2017;9(6):724-29. doi: 10.4300/JGME-D-17-00086.1. PubMed PMID: 29270262; PubMed Central PMCID: PMC5734327.
  17. Chan T, Sherbino J; McMAP Collaborators. The McMaster Modular Assessment Program (McMAP): A Theoretically Grounded Work-Based Assessment System for an Emergency Medicine Residency Program. Acad Med. 2015;90(7):900-5. doi: 10.1097/ACM.0000000000000707. PubMed PMID: 25881648.
  18. Pusic MV, Boutis K, Hatala R, Cook DA. Learning curves in health professions education. Acad Med. 2015;90(8):1034-42. doi: 10.1097/ACM.0000000000000681. PubMed PMID: 25806621.
  19. Pusic M, Pecaric M, Boutis K. How much practice is enough? Using learning curves to assess the deliberate practice of radiograph interpretation. Acad Med. 2011;86(6):731-6. doi: 10.1097/ACM.0b013e3182178c3c. PubMed PMID: 21512374.
  20. Warm EJ, Held JD, Hellmann M, Kelleher M, Kinnear B, Lee C, O'Toole JK, Mathis B, Mueller C, Sall D, Tolentino J, Schauer DP. Entrusting Observable Practice Activities and Milestones Over the 36 Months of an Internal Medicine Residency. Acad Med. 2016;91(10):1398-1405. doi: 10.1097/ACM.0000000000001292. PubMed PMID: 27355780.
  21. Ferguson R. Learning Analytics: Drivers, Developments and Challenges. IJTEL. 2012;4,304-17. doi: 10.1504/IJTEL.2012.051816.
  22. Gutmann J, Kühbeck F, Berberat PO, Fischer MR, Engelhardt S, Sarikas A. Use of learning media by undergraduate medical students in pharmacology: a prospective cohort study. PLoS One. 2015;10(4):e0122624. doi: 10.1371/journal.pone.0122624. PubMed PMID: 25849565; PubMed Central PMCID: PMC4388621.
  23. Siemens G. Learning analytics: The emergence of a discipline. Am Behav Sci. 2013;57(10):1380–1400. doi: 10.1177/0002764213498851.
  24. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32. doi: 10.1080/1364557032000119616.
  25. Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, Hempel S, Akl EA, Chang C, McGowan J, Stewart L, Hartling L, Aldcroft A, Wilson MG, Garritty C, Lewin S, Godfrey CM, Macdonald MT, Langlois EV, Soares-Weiser K, Moriarty J, Clifford T, Tunçalp Ö, Straus SE. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467-473. doi: 10.7326/M18-0850. PubMed PMID: 30178033.
  26. C. The learning analytics cycle: closing the loop effectively. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge - LAK ’12; 2012 29 April - 2 May; Vancouver, Canada. NY, USA: Association for Computing Machinery; 2012. doi: 10.1145/2330601.2330636.
  27. Gilliland KO. The Flipped Classroom and Learning Analytics in Histology. Med Sci Educ. 2017;27:9–13. doi: 10.1007/s40670-016-0364-2.
  28. Koh YYJ, Schmidt HG, Low-Beer N, Rotgans JI. Team-Based Learning Analytics: An Empirical Case Study. Acad Med. 2020;95(6):872-878. doi: 10.1097/ACM.0000000000003157. PubMed PMID: 31972678; PubMed Central PMCID: PMC7242170.
  29. Carney PA, Sebok-Syer SS, Pusic MV, Gillespie CC, Westervelt M, Goldhamer MEJ. Using learning analytics in clinical competency committees: Increasing the impact of competency-based medical education. Med Educ Online. 2023;28(1):2178913. doi: 10.1080/10872981.2023.2178913. PubMed PMID: 36821373; PubMed Central PMCID: PMC9970252.
  30. Vignare K, Moskal P. Introduction: Analytics research and case studies in online learning. OLJ. 2016;20(2). doi: 10.24059/olj.v20i2.958.
  31. Saqr M, Fors U, Tedre M. How learning analytics can early predict under-achieving students in a blended medical education course. Med Teach. 2017;39(7):757-767. doi: 10.1080/0142159X.2017.1309376. PubMed PMID: 28421894.
  32. Bojic I, Mammadova M, Ang CS, Teo WL, Diordieva C, Pienkowska A, Gašević D, Car J. Empowering Health Care Education Through Learning Analytics: In-depth Scoping Review. J Med Internet Res. 2023;25:e41671. doi: 10.2196/41671. PubMed PMID: 37195746; PubMed Central PMCID: PMC10233437.
  33. Wong BT, Li KC, Choi SP. Trends in learning analytics practices: a review of higher education institutions". Interactive Technology and Smart Education. 2018;15(2):132–54. doi: 10.1108/ITSE-12-2017-0065.
  34. Ten Cate O, Dahdal S, Lambert T, Neubauer F, Pless A, Pohlmann PF, van Rijen H, Gurtner C. Ten caveats of learning analytics in health professions education: A consumer's perspective. Med Teach. 2020;42(6):673-78. doi: 10.1080/0142159X.2020.1733505. PubMed PMID: 32150499.
  35. Karabetsou V, Malamateniou F. Process-Oriented e-Learning System for Training Healthcare Professionals on Big Data Usage. In Proceedings of the 9th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC); 2016 Feb 21-23; Rome, Italy. Portugal: SciTePress; 2016. doi: 10.5220/0005828205230528.
  36. Thoma B, Ellaway RH, Chan TM. From Utopia Through Dystopia: Charting a Course for Learning Analytics in Competency-Based Medical Education. Acad Med. 2021;96(7S):S89-S95. doi: 10.1097/ACM.0000000000004092. PubMed PMID: 34183609.
  37. Pusic MV, Birnbaum RJ, Thoma B, Hamstra SJ, Cavalcanti RB, Warm EJ, Janssen A, Shaw T. Frameworks for Integrating Learning Analytics With the Electronic Health Record. J Contin Educ Health Prof. 2023;43(1):52-59. doi: 10.1097/CEH.0000000000000444. PubMed PMID: 36849429; PubMed Central PMCID: PMC9973448.
  38. Olmos M, Corrin L. Learning analytics: A case study of the process of design of visualizations. OLJ. 2012;16(3):39-49. doi: 10.24059/olj.v16i3.273.
  39. Pecaric M, Boutis K, Beckstead J, Pusic M. A Big Data and Learning Analytics Approach to Process-Level Feedback in Cognitive Simulations. Acad Med. 2017;92(2):175-184. doi: 10.1097/ACM.0000000000001234. PubMed PMID: 27224299.
  40. Cirigliano MM, Guthrie CD, Pusic MV. Click-level Learning Analytics in an Online Medical Education Learning Platform. Teach Learn Med. 2020;32(4):410-21. doi: 10.1080/10401334.2020.1754216. PubMed PMID: 32397923.
  41. De Leng B, Pawelka F. The use of learning dashboards to support complex in-class pedagogical scenarios in medical training: how do they influence students' cognitive engagement? Res Pract Technol Enhanc Learn. 2020;15(1). doi: 10.1186/s41039-020-00135-7.
  42. Thau E, Perez M, Pusic MV, Pecaric M, Rizzuti D, Boutis K. Image interpretation: Learning analytics-informed education opportunities. AEM Educ Train. 2021;5(2):e10592. doi: 10.1002/aet2.10592. PubMed PMID: 33898916; PubMed Central PMCID: PMC8062270.
  43. Furlan R, Gatti M, Mene R, Shiffer D, Marchiori C, Giaj Levra A, Saturnino V, Brunetta E, Dipaola F. Learning Analytics Applied to Clinical Diagnostic Reasoning Using a Natural Language Processing-Based Virtual Patient Simulator: Case Study. JMIR Med Educ. 2022;8(1):e24372. doi: 10.2196/24372. PubMed PMID: 35238786; PubMed Central PMCID: PMC8931645.
  44. Elmoazen R, Saqr M, Khalil M, Wasson B. Learning analytics in virtual laboratories: a systematic literature review of empirical research. Smart Learn Environ. 2023;10(1):23. doi: 10.1186/s40561-023-00244-y. PubMed PMID: 40477836; PubMed Central PMCID: PMC9996561.
  45. Norman GR, Eva KW. Diagnostic error and clinical reasoning. Med Educ. 2010;44(1):94-100. doi: 10.1111/j.1365-2923.2009.03507.x. PubMed PMID: 20078760.
  46. Whelan A, Leddy JJ, Mindra S, Matthew Hughes JD, El-Bialy S, Ramnanan CJ. Student perceptions of independent versus facilitated small group learning approaches to compressed medical anatomy education. Anat Sci Educ. 2016;9(1):40-51. doi: 10.1002/ase.1544. PubMed PMID: 26040541.
  47. Bienkowski M, Feng M, Means B. Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. Washington: U.S. Department of Education. 2012; P. 1-60.
Interdisciplinary Journal of Virtual Learning in Medical Sciences

Articles in Press, Accepted Manuscript
Available Online from 27 December 2025

Files

History

  • Receive Date: 12 July 2025
  • Revise Date: 11 October 2025
  • Accept Date: 14 November 2025
  • Publish Date: 27 December 2025

Share

How to cite

Statistics