Procedures for the review of units and programs
Introduction
This document explains the purpose of, and procedures for, academic program review at Memorial University of Newfoundland. Academic Program Review (A.P.R.) is carried out by units and programs under the authority of Senate and is designed to support the objectives of the University. The purposes of APR are:
- to evaluate the quality, success, and role of academic units and programs in the fulfillment of their own and the University's mission and strategic goals
- to encourage academic planning, innovation and improvement in units and programs
- to provide an occasion for units and programs to identify opportunities and find ways to pursue them
- to avail of fresh perspectives from colleagues outside Memorial.
APR should be regarded as formative. That is, it is an opportunity for the unit or program to work collegially to find ways of improving upon what it does.
Normally APR will include all programs within an academic unit. Hence the word "program" may be read in the plural where necessary. Reviews of interdisciplinary programs and academic support units will also be guided by these procedures, modified as appropriate.
Academic units and programs will normally be reviewed at seven-year intervals, but this pattern may be amended when units are on a shorter accreditation cycle or in individual cases by the University Planning and Budget Committee (PBC) upon recommendation of the Dean/Director or the Vice-President (Academic). Wherever possible, APRs will be scheduled to occur before a unit's head steps down and will be combined with accreditation or other mandated reviews. However, accreditation is not a substitute for APR.
In order to maintain the arms-length relationships appropriate for certain aspects of reviews, some functions will be performed by a Dean of Record. This is the dean of the Faculty whose department or discipline is under review or, in the case of a School, a dean or director chosen by the Vice-President Academic in consultation with the School.
The review process will be centrally funded through the office of the Vice-President (Academic).
The Process
- Upon request, Deans and Directors, in consultation with faculty within their units. submit to the Vice-President (Academic) a list of their respective units and programs and a suggested order for their review.
- The PBC reviews submissions from Deans and Directors and establishes a multi-year schedule for reviewing units and programs, which is then referred to the Office of the Vice-President (Academic) for coordination.
- The PBC or its delegate(s) meets with members of the academic unit(s) concerned to provide information about the review process. The unit can, at that time, establish additional, discipline-specific criteria and indicators for the review as necessary.
- The Review Panel is appointed according to the procedures below.
- The Unit conducts a thorough Self-Study and submits this to the Dean/Director for distribution to the Review Panel.
- The Panel reviews the Self-Study, and may request additional information from the unit.
- The Panel conducts a two-day review of the unit When completed, the draft Panel Report is submitted to the Dean/Director who checks it for factual accuracy. Following any necessary revisions by the Panel, the Dean/Director circulates it to faculty and staff within the unit/program.
- The Academic Unit prepares a formal response to the Panel Report and submits this to the Dean/Director.
- The Dean/Director and unit Head meet to discuss the final APR Report (consisting of the Self-Study, Review Panel Report, and the unit's response), particularly any recommendations that have been made. Together they formulate an action plan, including a timeline and an assignment, of responsibilities, which is then circulated to faculty within the unit for discussion and comment. In units where the Director and Head are the same person the Head's meeting will be with the Dean of Record.
- The Dean and unit Head hold a follow-up meeting to make any amendments to the action plan that may be necessary.
- The APR Report and the action plan, are forwarded to the PBC for consideration in respect of planning and budget. A copy of the action plan is distributed to faculty and staff within the unit.
- One year after submission of the Report, the Dean/Director will submit to the PBC a report on progress towards implementation of the action plan. The PBC may request further progress reports.
Organization and Responsibilities
A Coordinator, reporting to the Vice-President (Academic), is designated to facilitate the process of Academic Reviews for the PBC. This individual shall:
- confer with Deans/Directors to schedule reviews following units/programs being selected for review by the PBC
- confer with the Head of each academic unit under review to determine a budget that will cover data collection and analysis, personnel time, payments to external reviewers, travel and miscellaneous office expenses
- assist in arranging for site visits by external reviewers (travel, accommodations, hosting, etc.)
- assist, as required, in the gathering of supporting materials for both the Self-Study and the Panel Review
- serve as custodian for review documents subsequent to the Review
- perform other duties, as required, in support of the review process
The Coordinator will be appointed by the Vice-President (Academic). In the case of Grenfell Campus, the appointment will be made by the Vice-President (Academic) on the recommendation of the Vice President of Grenfell Campus.
Each review is conducted by a Review Panel. The Panel will normally be composed of four members, all from disciplines related, so far as possible, to that of the unit under review. Two of the Panelists will be Memorial University of Newfoundland faculty from outside the unit/program under review, and two will be faculty from other institutions or other professionally qualified individuals.
- One Memorial University of Newfoundland member from outside the unit/program under review chosen by the Dean of Record.
- One Memorial University of Newfoundland member from outside the unit/program under review chosen by the faculty of the unit/program under review.
- Two external members representing the discipline of the unit under review, chosen by the Dean of Record in consultation with the Dean of Graduate Studies. At least one of these will be chosen from a list of three or more names provided by the faculty of the unit/program under review.
On occasion, based on the scope of the review, the Dean/Director may recommend to the PBC through the Vice-President (Academic) that only one external reviewer be enlisted for the Review Panel.
The Dean/Director will appoint one of the internal Panelists to be Chair of the Panel. After consulting with the Chair of the Panel and the head of the unit to be reviewed, the Coordinator will establish the time of the review and will set a schedule for the accomplishment of the review.
The full review process can be summarized into three parts: Self-Study, Review, and Follow-up. In preparation for the review, the unit will undertake a self-study based on the guidelines that appear later in this document. The self-study phase is central, providing the basis for further deliberations and goal setting within the context of University priorities.
The preparation of documentation for the review will be the responsibility of a member of the academic unit designated by the Unit Head. The responsibilities of others are as follows:
Faculty members will provide:
- either a CV, a standard Research Council personal information form, or other documentation of activity since the last review
- representative course outlines, assignments and typical exam question papers for these courses.
One undergraduate student and, where applicable, one graduate student will be chosen by students within the unit to participate in the Self-Study and to consult with the Review Panel.
Students, faculty, staff and where possible, alumni, will be available for interview or survey during the preparation of the review documents (internally) and the review itself (with external reviewers).
Panelists are encouraged to request additional materials as needed. The Dean of Record will ensure that all interests within the unit are included in the review process. All Panel members will participate in all aspects of the review and in the formation of the Panel's written report. It is the responsibility of the Review Panel chairperson to ensure that members of the Panel work together throughout the review and that the final report reflects the views of all members of the Panel.
The Panel's draft Report should be completed within one month of the review and forwarded by the Panel chairperson to the Coordinator for transmission to the Dean of Record in accordance with point 7 of the process. The Dean/Director is responsible for keeping the unit informed of actions taken pursuant to the review.
Self-Study Guidelines
Because the Self-Study is the central element of the review process full participation of faculty, staff and students should be encouraged and sufficient time allowed for discussion of all relevant issues. The Self-Study should provide a clear picture of a unit's current status, its objectives and how they are being met and an indication of where it hopes to go in the future. To avoid dramatic increases in workload in connection with reviews a number of steps can be taken. Units programs should endeavor to maintain files containing current information and statistics pertinent to the review process. It is also recommended that Academic Planning be a minuted agenda item for discussion at unit/program meetings.
The purposes of the Self-Study are:
- Description and assessment of the unit (teaching, courses, programs, organization, management and resources)
- Description of plans or proposed innovations for the unit/program for the future
- Assessment of the contribution of the unit to the academic discipline, the University and its priorities, and the province
The self-study document should include sufficient information to provide a clear description of the Unit/Program, its objectives and how those objectives are being met. It should aim to allow for a thorough evaluation by the Review Panel. All documentation should be concise and pertinent to the Review. While the diversity of disciplines in the University will mean that not all indicators will apply in the same way to every unit and program, the following questions will normally be addressed:
A. Strategic Objectives
- What are the strategic objectives of the unit/program?
- To what extent are stated unit/program objectives being met? What is the evidence for these achievements?
- How does the unit/program support the mission and objectives of the University and other programs within the University?
- How are the efforts of the unit/program focused upon achieving the level of excellence (provincial, national, international) to which the unit/program aspires?
B. Student Enrollment/Program Outcomes
- What are student enrollment trends (in terms of FTE) at each level (undergraduate and graduate)?
- Are the numbers of students majoring in the programs appropriate given the resources that are committed to the Unit/Program?
- Are the numbers of degrees being awarded appropriate given the resources that are committed to the Unit/Program?
- Are retention rates satisfactory for the Program?
- What is the level of satisfaction among graduates of the Program?
- How successfully are students gaining admission to graduate programs or finding post-graduate opportunities in or related to their field of study?
- How many nontraditional students are enrolling in courses and programs?
- How are program outcomes made known to students? How are these outcomes used to revise and strengthen the program?
C. Curriculum and Teaching
- Is the curriculum, as delivered, consistent with stated objectives, calendar descriptions, course requirements, degree requirements and standards for admission?
- If there is curricular overlap between departments, disciplines, and/or programs, how is such overlap justified or appropriate?
- Is the curriculum relevant to the needs of students and is it sufficiently rigorous and cohesive?
- Is the curriculum being delivered effectively?
- Is the curriculum responsive to the needs of students and employers?
D. Faculty Contributions
- How effective are the faculty as teachers?
- How productive are the faculty as researchers/scholars?
- What is the quality and impact of the scholarly contributions of faculty and professional staff.
- Are the faculty appropriately engaged with relevant professional communities locally, regionally, and nationally?
- Where applicable, are the faculty effectively engaged in relationships with business, government, cultural or other relevant communities?
- Are the faculty and professional staff active, and recognized, participants in regional, national, and international professional organizations?
- Are the faculty generating a level of external grants and contracts appropriate to the discipline?
- Are the contracts and grants received by faculty consistent with the strategic goals of the unit?
- Is there a suitable balance of teaching, research or creative work, and service in the workloads of faculty and professional staff.
- How are the faculty integrating teaching, research, and service?
E. Administrative Support/Efficiency
- Is the unit/program receiving appropriate direct resources and support from the University?
- How adequate and effective are infrastructural resources and support (e.g., library, promotion and recruitment, media, space) for achieving program goals?
- How effectively do the unit and its programs promote new initiatives, plans, collegial spirit, and active community involvement?
- What major initiatives and improvements should faculty, professional staff, and administrators be taking to enhance the program or unit?
- How well are administrative and professional support staff contributing to the academic and strategic goals of the unit/program?
F. Cost Effectiveness
- How appropriate are the student/faculty ratios in comparison to those in similar programs elsewhere?
- How do the program's costs and ratios relate to the costs of other comparable programs at Memorial and elsewhere?
- If applicable, what support is the unit generating from external sources, including an appropriate share of indirect cost recovery?
- How effectively does the unit deploy its resources?
Additional documentation in support of the above should include:
- Accreditation Reports, where they address questions from the above list, may be referenced in answer to those questions so as not to duplicate documentation in the Self-Study
- Accreditation letters from the past three reviews, if available
- Description of undergraduate curricula and programs for majors and non-majors
- Statistical data on course enrollments, class size, and number of majors and number of minors
- Breakdown of FTE faculty, professionals and salaried staff by rank, function, workload, age, and gender
- Information concerning the use of non-faculty instructors (if any) in the unit, whether teaching assistants or supplemental contract holders
- A copy of the University Strategic Plan
- Statement from the Librarian on current book holdings, journals and periodicals for the discipline
- Statement concerning any proposed changes to the unit's resources and an explanation of these proposed changes
- A copy of the unit's budget
- Assessment surveys by students and alumni, reports from previous reviews and pertinent information from the strategic plan or any similar exercise
- Do NOT include CEQ results.
It is important that the self-study achieve a proper balance between details and inclusiveness, and available resources (especially time). The self-study should provide enough information to be useful and understandable by professionals outside the unit, within and outside the University. The self-study should be considered as the central element for the external peer review process.
Review Schedule and List of Responsibilities
After the members of the Panel have discussed procedures among themselves, they will meet with the Dean of Record and the Unit Head. This meeting serves as an orientation to the review and gives panelists the opportunity to ask questions about the process and the unit. The Dean/Director or Unit Head then escorts the panelists on a tour of the unit's facilities and of other campus facilities pertinent to the unit's operations.
It is important that all faculty of the unit have the opportunity to meet with the Panel. Time should be set aside for individuals and groups of faculty, professional staff, support staff and the designated student representative(s) to meet with panelists. The Panel will also meet with other available undergraduate and graduate students and alumni from the unit. Members of the Panel may also conduct any other interviews that they or the Unit Head have requested. With the consent of the Dean/Director and members of the unit concerned some interviews may be conducted, and information gathered, by the Memorial internal panel members prior to the arrival of the external reviewers.
The review will include time for the panelists to meet alone to outline their report and to distribute the writing responsibilities. At the conclusion of the review, the team will hold an exit interview with the Dean and Unit Head. The Report is due one month following the completion of the review.
The Review Report
The Report should place the unit under review in the larger context of the unit's discipline as studied and taught in Canada and should assess the unit according to the norms for that discipline. It should address any major issues facing the unit, comment on the compatibility of the unit's purpose, achievements, plans and goals with those of the University mission, and suggest strategies for achieving unit and University goals. The Report should contain recommendations which, in the view of the Panel, will lead to improvements in the program. The Report should focus on the unit as a whole without reference to individuals and be written to inform both the unit and Senate. To accomplish these purposes the Report should, where applicable and appropriate, consider the following points:
Undergraduate Program
- How well is the unit performing its undergraduate teaching function?
- Is the curriculum sufficiently comprehensive and rigorous?
- Are the grading norms consistent with those used in similar programs in Canada and elsewhere?
- Is the unit properly staffed to fulfill its undergraduate responsibilities?
- Are classes the appropriate size to accomplish its teaching goals?
- Is there sufficient demand from students for the program?
- How competitive are the unit's undergraduate programs nationally and regionally in attracting qualified students and placing graduates in suitable positions?
- Are recruiting and retention opportunities being maximized, both by the unit and by the University?
- Does the University supply the library resources, computing and laboratory facilities, and other resources necessary to support the undergraduate program?
- Is the unit fulfilling its service responsibilities to other units in the University?
Graduate Program
- How effective is the unit in performing its graduate teaching and supervision responsibilities?
- Is the research and scholarly productivity of the unit's faculty appropriate to its graduate responsibilities?
- Are the graduate program's admissions criteria appropriate?
- How successful are the unit's graduate programs nationally and regionally in attracting qualified graduate students and placing graduate degree holders in professional employment?
- How competitive are the unit's graduate programs nationally and regionally in attracting qualified graduate students and placing graduate degree holders in professional employment?
- Is the curriculum sufficiently comprehensive and rigorous?
- Are the assessment standards consistent with those used in graduate programs in Canada and elsewhere?
- Is financial support for students at a level appropriate for the scope of graduate education activities desired within the unit?
- Does the University supply the library resources, computing and laboratory facilities, and other resources necessary to support the graduate program?
Faculty Research and Scholarship
- Are the research, creative activity, and scholarship of the faculty appropriate for providing first-rate academic programs?
- Are research facilities and library resources sufficiently supportive of faculty research?
- Are faculty generating external funding up to their full potential?
- What role are faculty playing in the University's research centers, interdisciplinary research groups and external partnerships?
- Are the faculty engaged in regional, national and international professional organizations?
Faculty and Staff
- How well are faculty and staff resources being used?
- Are promotion and tenure policies appropriate to the unit's mission and aspirations?
- How successful is the unit in implementing University employment equity policies?
- Are faculty and staff workloads equitable and appropriate to the unit's missions?
- Are administrative decisions made and administrative tasks carried out efficiently and effectively?
- How does the unit rank among those in similar institutions regarding research productivity and quality, external funding, academic programs and teaching loads?
Community Service
- Is the unit fulfilling opportunities to serve the community?
- Is the unit, where appropriate, effectively introducing students to professional community service opportunities?
University Citizenship
- Does the unit effectively reinforce the goals of other units?
- Do its members encourage and contribute to interdisciplinary activities?
- Should it apply its efforts and resources in new or different ways in order to enhance its role within the University?
University Support
- Is the unit receiving adequate resources from its Faculty and from the University at large?
- Are its facilities adequate? Attention should be paid to space, equipment, computing, laboratory and library resources.
- Is it adequately staffed?
- Does the reporting structure ensure managerial efficiency and administrative effectiveness within the unit?
Plans, Goals, and Resource Allocation
- Are the objectives of the unit appropriate to the mission of the University?
- Is the unit trying to do too much?
- If the unit has made requests for additional resources, which requests does the Panel support and why?
- How might the Unit's resources be redistributed to realize its goals and those of the University?
