View Procedure
Summative Assessment Procedure for Phase 4 Courses
Definitions
Assessment blueprint — A listing of the Entrustable Professional Activities (EPAs) to be included in a period of instruction, together with the assessment methods used.
Assessment plan — A detailed description of how learning will be assessed following a period of instruction.
Clerkship Discipline Coordinator (CDC) — A faculty member with responsibility for one of the core clinical rotations in the MED 8710 Core Experiences course.
Competency — An observable ability of a health professional, integrating multiple components, such as knowledge, skills and attitudes.
E-Clinic Card — A formative competency-based assessment tool intended to provide feedback on learner performance while progressing towards the achievement of Entrustable Professional Activities (EPAs).
Electives and Selectives Coordinators — Faculty members with responsibility for the MED 8730 Electives course and the MED 8740 Advanced Practice Integration course, respectively.
Entrustable Professional Activity (EPA) — A task or responsibility that students are entrusted to perform without direct supervision once they have attained sufficient specific competence.
In Training Assessment Report (ITAR) — A report summarizing the clinical assessment of a student after a period of clinical learning.
Longitudinal Integrated Clerkship (LIC) — A MED 8710 Core Experience course in which learners receive integrated training in all core disciplines rather than in sequential block rotations.
Longitudinal Integrated Clerkship Coordinator — A faculty member with responsibility for the LIC stream of MED 8710 Core Experiences course.
Phase 4 — The final twenty-one (21) months of the Doctor of Medicine (M.D.) program.
Phase 4 Management Team — An Undergraduate Medical Education (UGME) committee which is responsible for the management of Phase 4 of the Doctor of Medicine (M.D.) Program.
Student Assessment Sub-committee (SAS) — A sub-committee of the Undergraduate Medical Studies (UGMS) committee. The SAS evaluates, monitors, and advises the UGMS on student assessment policy and its implementation for the curriculum leading to the M.D. degree.
Summative Assessment — A process used to evaluate learner achievement at the end of a period of instruction.
Supervisor — A faculty member or resident who is responsible for a Phase 4 student during a period of clinical learning.
Procedure
Overview
The Undergraduate Medical Studies (UGMS) Committee, a standing committee of Faculty Council, has responsibility for “the overall policy of evaluation and the planning of the programs of studies leading to the MD degree”. The Committee on the Accreditation of Canadian Medical Schools (CACMS) Element 9.8 requires that “A medical school has in place a system of fair and timely summative assessment of medical student achievement in each required learning experience of the medical education program."
Purpose
Summative assessment is used to document learner performance after a period of clinical learning. This procedure outlines the approach to summative assessment for all Phase 4 courses in the MD curriculum.
Procedure
A.0 Assessment Planning
A.1 In accordance with section 6.7.2 of the University Calendar (see Appendix A), Phase 4 as well as each course within Phase 4, each rotation in the case of MED 8710 Core Experiences and the LIC stream for the MED 8710 course publishes an assessment plan during the first week of each course or rotation, which identifies the learning objectives to be assessed, methods of assessment used, and approximate dates when assessments occur or when assessed course work is due for submission. The assessment plans must be published in the online Student Handbook to ensure students are informed of the purpose and implications of assessment.
A.2 All non-elective course assessment, core rotation assessment and LIC stream plans must include a variety of assessment methods that assess a balance of clinical knowledge and clinical work-based performance and levels of assessment. Miller's pyramid (Appendix B), a framework for clinical assessment, should be used to guide the development of assessment plans for non-elective courses, core rotations and the LIC stream. Assessment plans must, as appropriate for the non-elective course, core rotation or LIC stream, include:
A.2.1 examinations that assess clinical knowledge using multiple-choice (MCQ), short answer, and clinical decision-making (CDM) questions;
A.2.2 In-Training Assessment Reports (ITARs) addressing all required EPAs;
A.2.3 checklists used in the simulation lab;
A.2.4 an Objective Structured Clinical Examination (OSCE) that will be based on Phase 4 clinical content, is linked to the appropriate EPAs, and assesses all CanMEDS roles;
A.2.5 a research project to assess the CanMEDS Scholar role, with project deliverables and presentation at a Research Day; and
A.2.6 other measures of assessment that can be used to complement and widen the competencies assessed by the previous categories.
A.3 In the design and/or adoption of assessment tools, the CanMEDS roles of Communicator, Collaborator, Health Advocate, Leader, Scholar and Professional should be taken into account and incorporated as appropriate in line with the program objectives.
A.4 Aggregate assessments for all learners are compiled by the UGME office at months six (6), nine (9), and twelve (12) of Phase 4. An aggregate assessment is compiled at months eighteen (18) for learners who have been determined to require remediation after the twelve months comprehensive review.
A.4.1 The performance of any learner who meets the criteria for discussion as outlined in the course assessment plan will be reviewed by the Phase 4 Management Team in a comprehensive review meeting. A specific learning plan (remediation) will be created to address the performance concern(s).
A.4.2 Learners not meeting the criteria for discussion will automatically progress to the next comprehensive review.
A.4.3 By the end of Phase 4, a learner must be entrustable in all EPAs to graduate.
B.0 Administration of Examinations and Other Assessment Tools
B.1 National Board of Medical Examiners (NBME) or other examinations
B.1.1 All NBME examinations are conducted under supervised conditions according to the invigilation procedures required by the NBME.
B.1.2 Other examinations are conducted under supervised conditions according to the Memorial University Conduct of Examinations and Invigilation Procedures.
B.1.3 Exam scheduling, administration and security are the responsibility of the UGME office.
B.2 A supervisor’s written assessment of a learner’s performance must be regularly completed throughout core rotations and the LIC stream using the designated e-clinic card app in TRes2 or other approved method.
B.3 Clerkship Discipline Coordinators (CDCs) in core rotations must complete a summative end-of-rotation ITAR using the information provided by the supervisor(s). For core rotations longer than four weeks in duration, CDCs must also provide feedback to the learner at the mid-point of the rotation.
B.3.1 In the ITAR, CDCs must collate feedback from completed assessments, additionally assessing whether or not a learner satisfactorily met the learning objectives of the core rotation and whether there are any concerns relating to their professional behavior and suitability to practice medicine. The CDC is responsible for reviewing these assessments to determine if the learner has successfully met the learning objectives of the core rotation.
B.3.2 Feedback from summative ITARs must be reported to learners within 6 weeks of completion of each core rotation.
B.4 For learners in the LIC stream, the family medicine preceptor will complete a formative ITAR every 6 weeks. The other disciplines will complete a formative ITARs at 24. The LIC Coordinator must complete a progress review every 12 weeks using all collected assessment data. The LIC Coordinator must also complete a summative ITAR at the end of the LIC using all collected assessment data.
B.4.1 In the summative ITAR, the LIC Coordinator must collate feedback from completed assessments. Additionally, learners will be assessed as to whether or not they have satisfactorily met the learning objectives of the LIC and whether there are any concerns relating to their professional behavior and suitability to practice medicine. The LIC Coordinator is responsible for reviewing these assessments to determine if the learner has successfully met the learning objectives of the LIC.
B.4.2 Feedback from the LIC summative ITAR must be reported to learners within 6 weeks of completion of the LIC.
B.5 Electives/Selectives Competencies
B.5.1 Electives and selectives will be assessed by the supervisor at the study site using a summative ITAR. Each summative ITAR is then reviewed and signed off by the Faculty Coordinator of each course.
B.6 Mandatory procedures
B.6 .1 A series of procedures are required for graduation from the MD program as described in the Student Handbook. Competence in some mandatory procedures is assessed in MED 8720 Advanced Procedural Competencies.
C.0 Progress testing and standard setting for examinations
C.1 Learners write a progress test at zero (0), four (4), eight (8) and twelve (12) months of the Core Experiences course. This is a multiple-choice question (MCQ) examination to assess clinical knowledge. Exam questions from the National Board of Medical Examiners (NBME) bank are used for this purpose.
C.2 The first three examinations are formative and learners will be provided with feedback on their performance.
C.2.1 Z-scores will be calculated and used to benchmark individual learner performance relative to the mean score of all learners writing the examination.
C.3 The final progress examination at twelve (12) months is summative.
C.3.1 A norm-referenced method will be utilized to determine the pass score for the summative progress examination. This will be based on the summative progress examination performance of previous cohorts. The pass score is reported in the Core Experiences course assessment plan and reviewed annually.
C.3.2 The reported pass score applies to the summative examination and the reassessment examination.
C.3.3 The result of the final summative examination is not reported on the MSPR.
D.0 Grading
D.1 In accordance with section 10.4.2 of the Faculty of Medicine Calendar: “Course grades are recorded as pass or fail on a learner’s University transcript. Within each Phase, there will be multiple prescribed assessments. For all Phases, assessment of an individual learner’s performance is the responsibility of the appropriate Phase Lead. Grades for all Phases may be reported using an internal scale.
D.2 Learners seeking a reconsideration of the grade of a summative assessment, such as an ITAR, must notify the UGME office in writing within five (5) business days of receiving the assessment. The Phase 4 Lead or delegate will review the assessment or coordinate a review with the CDC of the relevant discipline.
E.0 Program Evaluation
E.1 Assessment plans and assessment results in all non-elective Phase 4 courses are reviewed and monitored annually by the Student Assessment Subcommittee (SAS).
E.2 The Phase 4 Lead will receive an annual assessment report for each completed non- elective course from the SAS.
E.3 The Phase 4 Lead must review, comment on and sign the assessment report and return it within 30 days. A copy of the assessment report will be forwarded by the SAS to the Program Evaluation Sub-committee.