Evaluating Your Environmental Education Programs: A Workbook for Practitioners—A Book Review

Are You New to Evaluating Educational Programs?

Recently, I was challenged to assess a training program, but I had little experience with the evaluation process. Here, I review a new tool that helped me conduct a successful program evaluation, which can be part of your toolkit for evaluating programs as well.

Figure 1.
A Workbook for Evaluating Educational Programs

Evaluating Your Environmental Education Programs: A Workbook for Practitioners, by Julie Ernst, Martha Monroe, and Bora Simmons (Figure 1), is a recent publication that approaches program evaluation in a practical, effective, and easy-to-implement manner. The workbook contains key concepts, examples, and exercises that will aid you in developing the necessary knowledge and skills to complete an evaluation process.

This workbook was effective also in gathering evaluation elements covered extensively in the Journal of Extension and applying them in a practical way for environmental programs (Rennekamp & Arnold, 2009; Roucan-Kane, 2008; Bailey & Deen, 2002). Other helpful book reviews have been published in the Journal of Extension related to the evaluation process (Jayaratne, Lyons, & Palmer, 2008; O'Connor & Zeldin, 2005). However, this workbook facilitates the evaluation procedure for any educational program.

The Process for Evaluating Programs

The evaluation process as explained and presented by the authors is guided by six steps:

1. Focus Your Evaluation: The nature of a program evaluation is defined, and different evaluation types are described. The authors explain how best to describe your education program and what evaluation logistics to consider so that you can easily define the evaluation purpose and conduct a meaningful process.
2. Develop an Evaluation Plan: The importance of developing an evaluation plan to provide purposeful results is emphasized. Each task involved in creating the plan is described. They include developing evaluation questions, indicators and data collection tools and selecting design and sampling methods.
3. Develop Data Collection Tools: Examples of data collection tools are presented, such as surveys, interview guides, tests of knowledge, and observations. Each example includes guidelines to help you avoid pitfalls that can affect the quality of your evaluation.
4. Collect Data: Strategies to collect data are presented and discussed. Aspects mentioned include the use of existing data, setting and managing a data collection schedule, maintaining the integrity of the evaluation process, and appropriate methods to store data.
5. Analyze Data and Interpret Results: Practical methods for analyzing qualitative and quantitative data and summarizing results are discussed. Commonly used statistical methods are described.
6. Communicate and Use Evaluation Results: The final section describes and discusses the structure and components of the final evaluation report. The authors lead you through the process of synthesizing and organizing your data so that meaningful conclusions can be drawn and recommendations provided.

The Workbook's Effectiveness

Evaluating Your Environmental Education Programs is a practical and effective tool that can help novices to conduct an evaluation process. Although the workbook was designed to evaluate environmental educational programs, it is applicable to other types of programs. The workbook helped me to conduct a robust evaluation of an introductory Geographic Information System (GIS) training program. After following the six steps, I was able to identify the strengths and weakness of the GIS program. The recommendations that I provided on how to improve the course were subsequently implemented. As a result of this, the training program was improved in the following aspects.

• The example applications used in the workshop are more related to participants' personal and/or professional needs.
• Training materials, such as the workbook, have been adapted to increase participants' critical and creative thinking and, to apply acquired GIS skills.
• Active learning methods have been more effectively incorporated into the training (i.e., pair-thinking, small group activities, more discussion during the training, and real-world scenarios for applications.)
• Interaction among participants has been increased.
• A list of GIS terms is distributed to participants prior to the training. This helps the participants to be more familiar with the specialized terminology that will be used during the training.

Evaluations by those who participated in recent workshops that incorporated the training modifications indicated that they had a better experience than did those who participated in earlier, non-modified workshops. We continue to monitor and assess features of the workshops to determine how the incorporation of new elements has improved the training program.

References

Bailey, S. J., & Deen, M. Y. (2002). A framework for introducing program evaluation to Extension faculty and staff. Journal of Extension [On-line], 40(2) Article 2IAW1. Available at: https://www.joe.org/joe/2002april/iw1.php

Ernst, J., Monroe, M., & Simmons, B. (2009). Evaluating your environmental education programs: A workbook for practitioners. Washington, DC: North American Association for Environmental Education. The workbook can be ordered at: http://www.naaee.net/publications

Jayaratne, K. S. U., Lyons, A. C., & Palmer, L. (2008). A user-friendly evaluation resource kit for Extension agents delivering financial education programs. Journal of Extension [On-line], 46(1) Article 1TOT3. Available at: https://www.joe.org/joe/2008february/tt3.php

O'Connor, C., & Zeldin, S. (2005). Program assessment and improvement through youth-adult partnership: The YALPE resource kit. Journal of Extension [On-line], 43(5) Article 5TOT4. Available at: https://www.joe.org/joe/2005october/tt4.php

Rennekamp, R. A., & Arnold, M. E. (2009). What progress, program evaluation? Reflections on a quarter-century of Extension evaluation practice. Journal of Extension [On-line], 47(3) Article 3COM1. Available at: https://www.joe.org/joe/2009june/comm1.php

Roucan-Kane, M. (2008). Key facts and key resources for program evaluation. Journal of Extension [On-line], 46(1) Article 1TOT2. Available at: https://www.joe.org/joe/2008february/tt2.php