Effectiveness of New Learning Technologies in Providing Education to Rural and Isolated Communities
The main interest of the partner agencies is in "what works," and particularly in ensuring that e-learning develops as effectively and efficiently as possible. Accordingly, the core research area for this alliance will be a variety of studies on the effectiveness of e-learning.
Effectiveness will be examined in relation to three main outcome areas; access to programs, learning outcomes and cost. The primary methodological approach to be taken in this research will be comparative. Studies will be designed to examine e-learning in comparison to conventional classroom-based, learning and, to the extent possible, to compare different approaches to e-learning.
Within each of the three main outcome areas, it is possible that e-learning can be more effective than, less effective than or no different from conventional learning (within the limits that can be detected by the available sample sizes and statistical tests). From a policy perspective, certain combinations of these results would be positive, others neutral and others negative. For example, a plausible overall result would be that e-learning yields increased access, no learning differences and increased costs. The job of the researchers is to establish,in the most accurate way possible, the actual outcomes. The partners would then have to determine the acceptability of any trade-off among the outcome areas. In effect, the policy issue is whether e-learning can be seen as a positive-sum, zero-sum or negative sum game.
Access
The access question will be pursued through secondary analysis of Department of Education, Memorial University (MUN) and College of the North Atlantic (CNA) data bases and by follow-up surveys.
Data base analysis will be used to pursue questions of the
characteristics and locations of students taking e-learning
courses, and how the patterns of course-taking have evolved over
time. Specifically, it is proposed that data base analysis occur in
years 1 and 4. The year 1 analysis will take a five-year historical
perspective and the year 4 analysis will be follow data for an
additional three years. The main data bases to be used are of the
High school Certification Data Base of the Department of Education
and the MUN and CNA Student Record Systems. These contain detailed
records on student backgrounds and characteristics, courses taken
and grades.
Surveys will be used to obtain additional information on student
characteristics, as well as attitudes to e-learning and reasons for
taking courses in that format. Telephone surveys of involving both
high school and university students are proposed for the beginning
of years 2 and 5. Surveys samples will be selected from complete
population lists to be provided by the Department of Education, MUN
and CNA. Survey sample sizes will be sufficient to yield sampling
errors of 5% or better at the 95% confidence interval for all
sub-groups of interest (e.g. males/females; on-campus and
off-campus students, rural and urban students). This requires
nominal sample sizes of about 400 for each subgroup. Our experience
indicates that samples of about 1000 will yield the required
subgroup quotas without explicit stratification.
Learning Outcomes
Studies of learning outcomes will be designed to answer the question "do students learn as well through e-learning as through classroom-based learning?" Subsidiary questions will have to do with whether some students adapt better to e-learning than others. The desired outcome of such studies is that there would be no differences. However, outcomes in either direction would have to be considered possible.
Some light can be shed on this issue through data base analysis. Available records contain a long history of performance of students in courses taught by conventional and e-learning approaches. It is therefore proposed that outcome analysis from the data bases be conducted in conjunction with the access analysis.
The main limitation of data base analysis for outcome measurement is that student selection factors (e.g. self-selection, course availability) are not controlled. Also, it is likely that course content, assessment methods, and other course features are modified in the normal process of e-course design. The only way to control such factors is through experimental studies, in which students are randomized to treatments, characteristics of the treatments are controlled and monitored, and outcomes measured under controlled conditions. Within this alliance, it is proposed that one, and possibly two, such randomized experiments be conducted over the life of the program. The goal would be to have each experiment yield statistical power of at least .80 to detect a 5% difference in outcome at the 95% confidence interval. This will require reasonably large scale courses. It is important to note that modifications to strict random assignment may be necessary (e.g. assignment of schools rather than individuals to in some cases to accommodate the constraints, expectations, and inerests of the schools and districts.
Costs
One of the main issues of cost for new learning technologies is that of developmental and new infrastructure costs relative to operating costs. It is proposed that costs be examined within the context of programs that are under development as well as those that are fully implemented. Much of the research on costs would have to be conducted thorough document analysis and interviews with personnel in a position with knowledge (and of the accounting systems and their limitations). It is not clear at this point whether a full comparative model of costs and benefits can be developed. However, it is important to identify all possible cost components, and to examine any gaps in the records or accounting methods, which would result in some costs being hidden. Costs of alternative approaches (e.g. correspondence courses versus Web-courses) can probably be examined using a common base of cost components.
Laptop Learning
One of our district partners in the CURA project has identified the investigation of laptop learning as a priority area of research for them. They have initiated a pilot project in one urban high school, and are interested in examining the effectiveness of laptop learning. While this particular study expands our definition of e-learning to include the use of laptops, the main interest of the partner agency is in "what works," and particularly in ensuring that e-learning that is facilitated though the use of laptops develops as effectively and efficiently as possible. Accordingly, effectiveness will be examined in relation to two of our main outcome areas, learning outcomes and cost. The purpose of our research, therefore, is to provide evidence related to the relationship between the use of laptops, student outcomes, and cost.
It is anticipated that there will be one class of 30 students and 8 teachers. The students will be a combination of AP and French Immersion, and will do 5 common courses as "laptop courses". The program will provide laptops to students on a 1:1 ratio and will be focused on the provision of student-centered learning for which there exists considerable support. While this is an urban-based study at the outset, the effectiveness of laptops in the provision of student-centered learning, may have implications for how we deliver e-learning to rural and isolated communities.
Lead
Researcher(s): Dr. Bruce Sheppard
and Dr. Tim Seifert.
Main Community
Partners: Memorial University (MUN), Department of Education (DOE),
School Districts, College of the North Atlantic (CNA), Centre for
Distance Learning and Innovation (CDLI).
