April 1997 // Volume 35 // Number 2 // Feature Articles // 2FEA3
 |  |
 |
Training in Evaluation of Parent Education Programs Using the National Extension Parent Education Model (NEPEM)
Abstract
Increasingly, Extension educators and their community collaborators are being held accountable for results. They need to know how to evaluate their programs and effectively communicate the results to stakeholders. This article reports the design and results of a statewide training in parent education evaluation in Montana, using the National Extension Parent Education Model (NEPEM). It provides a useful example of how an evaluation training can be conceived, carried out, and evaluated by Extension professionals.
Extension educators and their community collaborators are increasingly being held accountable for results. They need to know how to evaluate their programs and effectively communicate results to stakeholders.
This article reports the design and results of a statewide training in parent education evaluation in Montana, using the National Extension Parent Education Model (NEPEM). It provides a useful example of how evaluation training can be conceived, carried out, and evaluated by Extension professionals.
In 1992, a team of specialists in the Cooperative Extension System noted that a national parent education model that pulled together critical parenting practices did not exist. In collaboration with family life specialists throughout the nation, the team reached agreement on a model summarizing core categories and the critical parenting practices emerging from them.
The National Extension Parent Education Model (Smith, Cudaback, Goddard, & Myers-Walls, 1994) provides a comprehensive overview of parenting and is intended by the developers to function nationally as a guide for parenting programs. The dimensions include: (a) care for self; (b) understand; (c) guide; (d) nurture; (e) motivate; and (e) advocate. The parenting concepts and practices described in NEPEM provide educators a common language to use in integrating and evaluating disparate programs, enhancing the potential for collaboration and replication of successful programs. The NEPEM booklet complete with literature reviews, ideas for implementation and suggested resources was published in 1994.
Also in 1994, a manual was developed by Alabama Extension and faculty in the Department of Family and Child Development at Auburn University that was based on the NEPEM (Goddard, Smith, Mize, White, & White, 1994). Among other things, it includes sample goals, objectives, and evaluation items that fall within NEPEM as well as ideas for communicating evaluation results to stakeholders. This workshop used these materials as the basis of the statewide training.
The goal of the workshop was to provide Montana parent educators with tools to evaluate the effectiveness of their parenting programs in the six critical dimensions of the NEPEM. Specific objectives of the training for participants are to:
- Develop a working knowledge of the National Extension Parent Education
Model (NEPEM).
- Learn how to design parenting program goals and objectives using the
NEPEM.
- Develop a working knowledge of the levels of program evaluation.
- Develop a working knowledge of program evaluation designs.
- Feel confident that they can develop their own evaluation instrument
based on the objectives and priority practices of the NEPEM.
- Develop confidence that they can evaluate their parent education
program.
- Develop confidence that they can effectively report evaluation results to stakeholders.
An additional goal of the workshop was to begin forming the Montana Parent Educator Network to encourage replication of effective programs, resource sharing, multi-site evaluation, collaboration, technical, and mutual support.
A 3-hour training entitled, "Does it Work? Evaluating the Effectiveness of Your Parent Education Program" originated from the Montana State University-Bozeman campus. Funded by a grant from the Montana Children's Trust Fund, the workshop was broadcast to eleven sites in Montana via CODEC Interactive Television, reaching some of the most remote regions of the state. CODEC technology allows the instructor virtually to teach a statewide class and makes it possible for the presenter to interact personally with each site participant as questions are raised.
A $25 fee covering the NEPEM booklet and evaluation manual was charged. A promotional brochure was designed and distributed to potential participants through Extension, Children's Trust Fund, Montana Council for Families, and Partnership for Strengthening Families networks. Additional information about the training is available from the first author.
An instrument measuring program impact was created using a post-then-pre- design (Rockwell & Kohn, 1989). The post-then-pre approach is an easy, reliable and valid way to determine program impact. It asks participants to first report knowledge and skills gained from a program; and then rate how they perceived their knowledge and skills just prior to a program.
Participants were asked to describe themselves with reference to meeting the objectives listed above on a 5-point scale (1= Not true of me at all , to 5 = Very true of me) (see sample items in Table 1). Additional items on the survey assessed participants' overall evaluation of the workshop compared to other professional workshops, the strengths of the workshop, how adequately the workshop met participants' expectations and needs, and what additional training in this area would be helpful. The survey was administered immediately following the training by local site facilitators. All participants (51) completed the survey.
| Table 1 | ||
|---|---|---|
| Sample Post-Then-Pre Items from Parent Education Evaluation Training | ||
| How would you describe yourself in relation to this workshop? Use the following key: (1=VERY TRUE OF ME, 2=TRUE OF ME, 3=PARTLY TRUE OF ME, 4=PARTLY UNTRUE OF ME, 5=NOT TRUE OF ME AT ALL, NA=NOT APPLICABLE) | ||
| 1. | Before Workshop Did I? | End of Workshop Do I? |
| Have a working knowledge of the National Extension Parent Education Model (NEPEM) | 1 2 3 4 5 NA | 1 2 3 4 5 NA |
| Know how to design parenting program goals and objectives using the NEPEM | 1 2 3 4 5 NA | 1 2 3 4 5 NA |
Participants reported that they were very pleased with the training. Ninety-one percent rated the workshop as above average to superior, compared to other professional training. Seventy percent found it to be very adequate in meeting their expectations and needs, and 28 percent found it fairly adequate. Paired t-test results showed that the program met its objectives (see Table 2). Specifically, compared to before the workshop, participants reported having a better working knowledge of the NEPEM (t = -15.6, p < .000), levels of evaluation (t = -10.7, p < .000), evaluation designs (t = -10.9, p < .000),and instrument design (t = -11.0, p < .000). they also reported knowing better how to design parenting goals and objectives (t = -9.7, p < .000), greater confidence in evaluating their programs (t = -10.1, p < .000) and reporting results to stakeholders (t = -8.0, p < .000). | ||
| Table 2 | ||||
|---|---|---|---|---|
| Post-Then-Pre Means for Parent Education Evaluation Training By Objective | ||||
| Objectives | Before Training | After Training | ||
| Mean | SD | Mean | SD | |
| Have a working knowledge of the National Extension Parent Education Model(NEPEM). | 1.69 | 1.06 | 3.94 | .72 |
| Know how to design parenting program goals and objectives using the NEPEM. | 2.19 | 1.41 | 3.85 | .86 |
| Have a working knowledge of the levels of program evaluation. | 2.45 | 1.00 | 3.89 | .82 |
| Have a working knowledge of program evaluation designs. | 2.53 | 1.03 | 3.85 | .72 |
| Feel confident I can develop my own evaluation instrument based on the objectives and priority practices of the NEPEM. | 2.18 | 1.16 | 3.96 | .85 |
| Feel confident I can evaluate my parent education program. | 2.63 | 1.13 | 4.04 | .81 |
| Feel confident I can effectively report evaluation results to stakeholders. | 2.62 | 1.16 | 3.89 | .78 |
Strengths of the workshop centered mainly on the quality and usability of the materials, and how the practical use of these materials was explained. Representative comments include "Excellent, clear, comprehensive materials!!!" "Very thorough and clear information. Extremely relevant to what I do." Additional comments included excitement to be involved in the Montana Parent Education Network, the formation of which was announced during the training, and to engage in replication of effective programs, resource sharing, multi-site evaluation, collaboration, and mutual support. In addition, follow-up sessions for local parent educator teams were requested.
This workshop provided the opportunity for parent educators across an incredibly vast state to come together and learn a common language for speaking about and evaluating parent education programs. The data provided the assurance that parent educators in Montana communities now have the tools necessary to assess the effectiveness of their programs in the six critical areas of parenting practice.
With this enhanced evaluation capacity, coupled with continued technical support, we believe we will see increased replication of the most effective parent education programs, and help reduce the seemingly endless array of demonstration projects in the state.
Goddard, H. W., Smith, B. L., Mize, J., White, M. B., & White, C. P. (1994). The alabama children's trust fund evaluation manual. Auburn, AL: Alabama Cooperative Extension Service.
Rockwell, S. K. & Kohn, H. (1989). Post-then-pre evaluation. Journal of Extension, Summer, 27, 19-21.
Smith, C. A., Cudaback, D., Goddard, H. W., & Myers-Walls, J. (1994). National Extension Parent Education Model. Manhattan, KS: Kansas Cooperative Extension Service.