In 2017, Canada changed how it trains doctors. The Royal College of Physicians and Surgeons introduced Competence by Design, a system intended to confirm every graduating physician could handle critical tasks—such as managing chest pain—only after a supervisor witnessed them performing safely.
The plan seemed straightforward. Implementation fell short.
Thousands of patient interactions, spanning clinics and emergency rooms, went unassessed. The framework relied on observation, yet observation never became standard practice.
The issue wasn’t the physicians
Supervisors aren’t neglecting their duties. They face overwhelming demands. A single attending physician may balance consults, operating room emergencies, and overnight shifts while overseeing residents. The system requires structured, real-time evaluations, but time-pressed doctors often review rather than observe trainees.
At Scarborough Health Network (SHN), a solution is emerging. The hospital serves 850,000 people in the Greater Toronto Area and trains hundreds of learners annually. Its approach doesn’t replace supervisors but equips them to fulfill the system’s original promise.
AI as an observer
In primary care, AI already transcribes patient visits and drafts notes. With consent and strict privacy measures, every resident-patient interaction could be recorded. An AI system, working with a supervisor, could analyze these encounters against competency standards and provide targeted feedback. Instead of fewer than 10 observed cases in a career, every case could become an opportunity for growth.
Related: Air crews face higher cancer death risk
Technology can also examine a resident’s case logs to detect patterns. A trainee might see hundreds of patients but never handle a gastrointestinal bleed or lead a goals-of-care discussion. The data exists but isn’t used to verify competence.
SHN is piloting AI-driven virtual standardized patients—video avatars that simulate clinical situations. Trainees can practice delivering difficult news or explaining discharge instructions repeatedly before interacting with real patients. This mirrors aviation training, where mastery is built through simulation before real-world stakes apply.
Another tool in development uses discreet audio and video to assess performance over time. The aim is reliability. Rather than relying on scattered, memory-based evaluations, the system creates a continuous record of progress. It pinpoints weaknesses, tracks improvement, and transforms sporadic observation into structured assessment.
Safeguards are essential
These methods require strict protections. Ambient recording demands explicit patient consent and compliance with laws like Ontario’s Personal Health Information Protection Act. Faculty must retain oversight, with AI supporting—not replacing—human judgment. Models need regular audits for bias, particularly in evaluating trainees from underrepresented backgrounds.
The goal isn’t to replace supervisors. It’s to provide them with accurate records of what occurs in the exam room. A resident might see a patient at 11 p.m. who misunderstood discharge instructions. The next morning, the supervisor and resident review the conversation. The AI flags moments where confirmation was lacking. They rehearse, and the resident practices with a virtual patient until the approach becomes second nature.
Canada pledged to monitor its doctors’ training. For years, it hasn’t. AI won’t replace educators, but it might help the system deliver on its commitment.
