WHAT'S WORTH MEASURING?  TEACHERS, HARD-TO-MEASURE
 OUTCOMES, AND ACCOUNTABILITY

 Jean Brown and Bruce Sheppard
 Faculty of Education
 Winter 1998



 The Globe and Mail refers to itself as "Canada's National Newspaper".  National, in the sense that it is distributed across Canada and makes an attempt to cover stories from all parts of the country, it is usually described as a paper with a decidedly business/right wing, conservative viewpoint.  On Saturday, June 28, 1997 it carried an editorial by William Thorsell entitled "Taking the Measure of Our Education Systems" (Saturday, June 28, 1997, D6) which vividly illustrates the pressures being applied to education today:
 

 More than 50 per cent of the residential property-tax bill in Toronto (and many other cities) is dedicated to primary and secondary education.  Canadians are among the highest per-pupil spenders in the world on schools....  Our laws generally require children to attend school until they are 16 years of age.  Quite obviously we value education highly.


 In the very next paragraph, Mr. Thorsell questions whether Canadians really value education.  He states:
 

 Somewhere along the way, we forsook some basic management tools in education.  The simplest is this:  you cannot manage any system without goals that can be measured.  This doesn't mean that every goal that is important to a system can be explicitly measured, but some core goals must be if the system is to be managed at all.
 Some time in the 1960s, it became fashionable -- and that is the word -- to set goals for education that were effectively beyond measurement.  They had to do with self-realization, curiosity, awareness, creativity, open-mindedness, tolerance, gentleness and critical thinking.


 Mr. Thorsell attacks what he calls the "corkscrew curriculum" in which all students proceed at the same rate, but not the same pace, through the system.  He concluded:  "The combination of hard-to-measure goals and corkscrewing (which saw the end of external, general exams) reduced the accountability for spending public money or students' time."  The result, he claims, is that mastery of basic skills began to deteriorate.  Parents found it hard to monitor their children's progress, and "higher proportions of education taxes went into "supporting missions" such as counselling, extracurricular activities and special interest/needs programs."  In Mr. Thorsell's view, funding the education system has led to a form of "indirect taxation without meaningful accountability."

 In every part of our country, and in other countries in the Western world, similar views are being expressed.  Accountability and testing are indeed buzz words of the 1990s.  Mr. Thorsell applauds the solutions which he identifies across Canada:
 

 Alberta restored system-wide testing of basic measures early.  British Columbia removed many powers of local school boards.  New Brunswick eliminated school boards.  Ontario is acting to restore measurable standards and rein in the powers of fewer school boards, while Quebec reviews its core curriculum.


 Although not addressed by Mr. Thorsell, Atlantic Canada is also engaged in similar measures.  In fact, comparisons of educational systems across the country, and even internationally, make one wonder where all the common ideas for reform spring from and how they are circulated so efficiently among educational bureaucracies.  The Atlantic Provinces Education Foundation (APEF) is identifying core learning outcomes in mathematics, language arts, science, and social studies.  An indicator program is in place to allow standardized testing based on these anticipated learning outcomes. Departments of Education are assuming responsibility for district, provincial, and national testing and comparisons.  If School A is not measuring up to School B, then school councils composed of community members, parents, teachers and administrators (another creation of the 1990s) will want to know why, and the provincial departments (or ministries) of education can investigate.  To prefect techniques in this area, many administrators have travelled to distant school districts to see first hand what is happening there.  For example, just a few years ago, a team of educators from Newfoundland flew to Kentucky to observe their attempts at school reform through accountability and testing.

 This situation should concern all teachers.  For many educators, the reaction to Mr. Thorsell's comments is to ask, what is education if we ignore hard-to-measure goals such as "self-realization, curiosity, awareness, creativity, open-mindedness, tolerance, gentleness, and critical thinking"?  Furthermore, are they "effectively beyond measurement" (as Mr. Thorsell states) or are there ways to measure them?

 Educators may react to the challenge that such a viewpoint poses by becoming angry with those who hold views similar to Mr. Thorsell's; they may then try to ignore that viewpoint and proceed as if it does not exist, holding on to their beliefs and hoping that others will support them in what they consider valuable work.  For example, a typical teacher can continue to plan and teach units, devote long hours of one-to-one assistance and help to students, assume a leadership role in school-wide professional development and engage with colleagues to master new and emerging technologies that will assist in the teaching and learning within the school.  This is the path that many teachers have chosen in the past.  They shudder at the very thought of becoming politically active and would not know where to begin.  They would argue that they know and their students know that they work hard, that what they do is important and helpful, that they do not have the time nor the interest to do more.  You could ask, what is wrong with such a response?

 To ignore what Mr. Thorsell is saying will lead to changes in teaching which may be problematic for many, because although teachers see the value of the different ways they do their work, it is seldom documented.  Our research (Brown & Sheppard, 1997a; Brown & Sheppard, 1997b; Sheppard & Brown, 1996) reveals that even in the most recognized schools with strong programs and qualified professional teachers, there is seldom any indication of how programs contribute to student outcomes.  The problem is that much of what teachers do falls into the category of goals that Mr. Thorsell rightly identifies as "hard-to-measure".

 There is, however, another choice, which is to accept the reality that in todays environment there is a need for accountability.  Process goals, such as those involved in helping students learn how to learn, to become independent, life-long learners, are indeed hard to measure, but the important point is that they can be measured.  It just requires a different approach to measurement.  Part of the problem is that too many teachers in the past have assumed that everyone would support and believe in the need for programs and approaches they saw as important.  Because certain values were important to them, many assumed that they would be important to everyone else as well. In this post-modern world, teachers need to recognize that there is no longer an overriding belief in anything.  They can no longer take for granted that the goals they endorse are endorsed by the education system, or if they are, that the system will agree on how these goals can be reached.  Teachers will need to be politically astute, and that begins by recognizing that there is a need for evidence to back up what is valued.

 Our research reveals that schools are not defining learning outcomes well, but others outside the school are.  In Canada, education is a provincial rather than a national responsibility.  However, provincial departments of education are voluntarily forming themselves into regional groups such as the Atlantic Provinces Education Foundation (APEF).  A similar Foundation exists for western Canada.  Their mandate is to establish learning outcomes for what they have labelled as core.  In addition, editorial writers and other journalists have their own criteria.  All educators need to examine and understand what measurements are being used by editorial writers such as Mr. Thorsell.  Most would agree with Mr. Thorsell's conclusion that in the end, a major goal for all schools is:  "Instructing the next generation about how the universe works, where our civilization came from, why it values what it does and what's on the agenda next".  However, those who commit themselves to a career in the classroom want far more than that – they want the "hard-to-measure" goals for all students as well.

 Why have non-educators, such as Mr. Thorsell, determined a narrow range of outcomes on which schools are to be judged?  Why does he (and others) seem willing to judge the reputation of schools on only those measures easy to obtain?  We may not like the answers we hear, such as this example from Stoll & Fink (1995):
 

 If there is a problem for educators and researchers, we did it to ourselves.  We have never demonstrated to ourselves, let alone anyone else, that schools make a difference to pupils' learning, knowledge, skills and attitudes which will enable them to be successful citizens in the twenty-first century.  If most educators are not assessment literate how can we expect our publics to understand the issues that relate to assessment?"  (p.167)


 Teachers would be wise to do some private soul-searching and ask themselves how comfortable they would be if asked to show the link between what they do and student outcomes.  They need to ask themselves also whether their professional values are reflected in the essential learning outcomes accepted and shared by their colleagues and by the larger community.  Teachers who know they are doing a good job need to ask themselves:  Who else knows how well I do what I do and the importance of this function to students' learning?  How can I show improvements in student achievement scores?  In the current political environment, it is critical that teachers identify and articulate the learning outcomes that they want measured, and determine ways they can be measured, for "What gets measured, or assessed, gets valued.  If schools do not measure what they value, what others choose to measure will be valued" (Stoll & Fink, p.167).  In the research that we are doing, we were told that all teachers need to show that their work is directly or indirectly related to students' learning outcomes.  In the restructuring that is taking place, only those programs seen as contributing to the mission of the school will survive.

 To bridge the gap between process or hard-to-measure goals and the need for accountability, there needs to be a greater emphasis by teachers and schools to identify what the important goals in schooling are, to develop measures for such goals, and ensure that they are collected.  If teachers perform other critical functions, such as providing peer coaching and training for their colleagues and participating in school improvement initiatives, they need to show that such activities also contribute to student success.  For those involved in the education of teachers at the university, there is a need to ensure that programs provide students with the ability to understand and interpret the findings of research, and as well with the knowledge and ability to engage in research themselves.  A current imperative is that a research base be built that will provide the evidence that is so badly needed.  To do so, the gap between the university researcher and the school practitioner must narrow.

 The research program that we have developed is an action research model (Calhoun, 1994) which involves us, both members of a university faculty of education, as "critical friends" (Lieberman, 1995, p. 3) in the schools in which we work.  We are actively engaged with school teams in an effort to obtain information and data which are beneficial to us and the school.  We see this as a very promising way to develop and test theory and to conduct research, including how best to identify and measure the hard-to-measure goals held within the school.

 One thing is clear -- teachers cannot leave the determination and measurement of school goals to administrators, other teachers, or outsiders. It is too important for that.  They need to recognize their responsibility for they cannot assume that others hold the same educational values that they do.  It is at the school level that teachers will need to ensure that the goals that the profession values are measured; that the contributions of teachers from various programs are identified as making a real difference to students' learning outcomes, and that these outcomes be measured.  Teachers have a professional responsibility to work with colleagues in determining the learning outcomes that are valued in the school, and to be leaders in finding ways in which they can be measured.

 Professionals in education, whether they work in universities or in schools, want schools that are providing the best possible learning experiences for students.  However, the day is gone when anyone can rely only on his or her individual intuition as to what the best is.  Neither can teachers assume that the outcomes they value will be measured by standardized provincial or national tests.  The current society requires evidence and is demanding greater accountability.  All groups in education are in danger of being discredited and disregarded unless they provide that evidence.  Schools need to become learning organizations (Senge, 1990), where collectively the staff makes the best decisions they can for the students they serve.  This will require a new type of professionalism for teachers and a commitment to continuous improvement for schools through a process of self-evaluation and learning. Increasingly parents and the community can work with schools to identify the learning outcomes that need to be valued and measured, and they can be supporters in the fight for a school system that will provide such an education. 

 These are difficult times in education for all those who believe in the value of programs such as music, physical education, social studies, drama, and art, and who see the value and need for qualified teacher-librarians.  Many people in our society are seeking tax reductions and are unwilling to support educational programs in the way they were in the past.  Politicians are responding to these demands, and as a result, senior administrators and government bureaucrats are being given reduced budgets and asked to trim their expenses.  They are being forced to make very difficult choices, and as it is in nature, it is the weakest that will not survive.  For too long teachers have tried to avoid the need for public accountability.  Focused on the classroom and the student, they have been reluctant to become politically sensitive and responsive to the public's movement towards increased accountability.  They can avoid it no longer.  If teachers do not stand up for the outcomes they value and measure them, others (such as educational bureaucrats, special interest groups within the public, government members, the business community) will hold teachers accountable for outcomes they value.  Teachers need to ask themselves:  is this what we want? 

REFERENCES

 Brown, J. & Sheppard, B. (1997a, Fall).  Is it just me?  Self-doubt and delusion in moving to shared decision-making:  the case of Red River Elementary.  The Morning Watch, 24, (1-2), 1-12.

 Brown, J. & Sheppard, B. (1997b, Oct-Dec).  Partnerships, funding and successful classroom change.  Prospects, 4, (3), 28-31. 

 Calhoun, E.F. (1994).  How to Use Action Research in the Self-Renewing School.  Alexandria, Virginia:  Association for Supervision and Curriculum Development.

 Lieberman, A. (1995).  The Work of Restructuring Schools.  New York:  Teachers College Press.

 Sheppard, B. & Brown, J. (1996).  One school district's experience in building a learning organization.  The Morning Watch, 24, (1-2), 1-12.

 Stoll, L. & Fink, D. (1996).  Changing our schools.  Philadelphia:  Open University Press.