Published research · npj Digital Medicine
MAPLES in practice: AI‑assisted OSCE assessment
A prospective UTSW study brings multimodal AI into an objective structured clinical examination (OSCE), with targeted human review.
Published 7 October 2026 Article in Press
Prospective deployment of multimodal AI grading for medical student OSCEs
Huong-Tra Ngo and Ameer Hamza Shakur are co-first authors, with Andrew R. Jamieson as the corresponding and senior author.
Part of npj Digital Medicine’s Transforming Medical Education through Artificial Intelligence collection.
The study
Assessment across notes, audio and video.
In Fall 2025, MAPLES was deployed prospectively at UT Southwestern in a primarily formative OSCE. Written post-encounter notes, audio and video supported assessment of clinical documentation, communication and physical examination performance.
- Second-year medical students
- 222
- OSCE stations per student
- 5
- Retained item-level scores
- 72,907
A focused role for human review
Low-scoring learners were routed for independent educator rescoring, with physician adjudication of selected AI–human disagreements. The study counted 5,616 actual human grading passes against 72,907 passes in a modeled, single-pass manual comparator: 92.3% fewer grading passes. This comparison counts grading passes; it does not measure time or cost savings.
Explore the process
From deployment to ongoing oversight.
The UTSW program expanded in stages. Explore how the assessment sources and platform operation evolved, alongside the governance cycle that supports oversight.
A staged rollout at UTSW
How the workflow evolved
The timeline includes earlier and later program milestones. The quantitative results above describe the Fall 2025 study only.
Fall 2023
Start with clinical notes
AI-assisted grading of post-encounter notes.
Assessment sources
- Notes
Fall 2024
Continue note assessment
Post-encounter note grading continues across another deployment cycle.
Assessment sources
- Notes
Spring 2025
Add transcripts
Notes are joined by transcript-based assessment of communication and physical examination.
Assessment sources
- Notes
- Transcript
Fall 2025 Published study cohort
Bring three modalities together
Notes, native audio and video are coordinated in one rubric-driven workflow.
Assessment sources
- Notes
- Native audio
- Video
Spring 2026
Move to educator operation
Educators operate the notes, native-audio and video workflow after brief training.
Assessment sources
- Notes
- Native audio
- Video
In the published study, each rubric item was scored using its designated source. Platform operation is distinct from the faculty’s assessment and oversight roles.
Governance across deployment cycles
A loop of human oversight
Define the assessment protocol and supporting evidence.
The supervisory committee reviews readiness and proposed changes.
Run the approved workflow and monitor processing and data quality.
Conduct human review, adjudication and quality checks.
Bring operational, agreement and adjudication findings back to the committee.
Use retrospective analysis to inform the next proposed protocol.
Step 1 of 6
Proposed protocol
Define the assessment protocol and supporting evidence.
Faculty retain authority over learner review and educational decisions.
An original interactive explanation of the UTSW deployment history and governance process reported by Ngo, Shakur et al. (2026).
View original published Figure 1The source timeline and oversight cycle, reproduced unchanged.
Panel a · The deployment timeline The progression from written notes to transcript, native-audio and video assessment.
Panel b · The oversight cycle Protocol design, committee sign-off, deployment, human review, reporting and retrospective analysis.
The figure places the Fall 2025 study within the program’s wider deployment and oversight timeline.
See the workflow
Ameer Hamza Shakur walks through rubric upload, AI-assisted scoring, evidence review and export of faculty-reviewed results. The demonstration shows the platform workflow; the paper reports the prospective study.
Watch the MAPLES walkthroughExplore a validation partnership
How well does this approach transfer to another institution’s learners, stations and rubrics? Explore a focused pilot, a human-review workflow or an external-validation study with the team.
Start a research conversationShareable study summary Open the visual overview for colleagues and educators.
