Measuring success

Dec 15th, 2014

Kelly Foss

Measuring success

When the Women in Resource Development Corporation (WRDC) needed professional, but inexpensive consultants to assist them in evaluating one of their long-running programs, the not-for-profit group turned to Memorial University for support.

Developed in 1999, the Orientation to Trades and Technology (OTT) program is one of WRDC’s key programs, a 16-week career exploration program for women, which provides hands-on learning in a variety of trades and technology fields, and offers many personal and professional development opportunities. It is delivered in partnership with College of the North Atlantic campuses across the province.

“In 2013, WRDC began efforts to conduct a follow-up study on OTT graduates, to track their training and employment trends after the completion of OTT,” said Jenna Hawkins, research and evaluation coordinator with the organization. “Having advertised the research/evaluation opportunity on Yaffle, the Harris Centre connected WRDC with Dr. Cathryn Button.”

Dr. Button says WRDC was interested in knowing whether or not the outreach program they were offering was actually resulting in more women entering trades jobs.

“Like many non-government organizations, they had limited resources to track their performance, though they had information on past participants in the program,” she said.

The professor brought the request to her students in a graduate class in program evaluation for the Master’s in Applied Social Psychology, and ultimately WRDC began a semester-long project with students Nicole Shea and Nicole Power, who did the project as part of their course requirement.

WRDC worked closely with the students as the pair designed an effective evaluation for the organization’s needs, developed the measurement tools to do it and then carried out the study.

“Over the years WRDC has had over 700 participants go through the OTT program,” said Ms. Shea. “The first thing we had to do was go through their database to clean it up, remove any duplicates and update contact information.

“Then we contacted participants and got them to complete a short survey by phone, online or on paper about their experiences in the OTT program, and asked if they had gone on to do any further education or were working in the trades – basically to see if the program had achieved any of its outcomes.”

After collecting the information and going through the process they developed to evaluate it, the students determined the OTT program was indeed successful, with the majority (98.5 per cent) of participants indicating the program was helpful to them. A significant number (71.9 per cent) went on to further their education and of those, 67 per cent went on to pursue education in the trades and technology field. Half of the participants in the sample were employed, with 19.4 per cent working in a non-traditional occupation.

“It was obvious from the study to see that there were still barriers to women entering the trades,” said Ms. Shea. “While many went on to get a further education, some said when they tried to work, they found the trade they had gone into was too difficult, while others came across sexism in the workplace. But there were quite a few who had successfully gone on to get their journey person or apprenticeship certification, and everyone had positive things to say about the program.”

“I was told by many women that they would never have tried the things they did if it wasn’t for the OTT program,” added Ms. Power. “Before this program they wouldn’t have thought that they would have been capable of being a heavy equipment operator or a pipe fitter, for example. But now they either have built, or are on their way to building, a promising future for themselves in these trades.”

Ms. Hawkins says making the connection with Nicole Power and Nicole Shea was invaluable to her organization. While the findings have given the organization an ability to confirm past success and show accountability to those providing funding, the work has also made it possible for WRDC to continue evaluating the program as they move forward.

“As a consequence of the students’ work, WRDC found out that the program had many side benefits to the participants, such as networking benefits and increasing job finding skills and self-esteem,” said Dr. Button. “But most importantly they now have a standard tool to continue measuring their outcomes and a template for tracking participants in the future.

“For the students, the project provided a real problem in a real world environment - an invaluable opportunity to apply and further develop their skills,” she added.  “We really like working with community organizations as they provide a wide variety of challenges for our students and they are enthusiastic partners in research. This is experiential learning at its best.” 


Faculty of Science

230 Elizabeth Ave, St. John's, NL, CANADA, A1B 3X9

Postal Address: P.O. Box 4200, St. John's, NL, CANADA, A1C 5S7

Tel: (709) 864-8000