 |
June 2009
|
June 2009 // Volume 47 // Number 3 // Tools of the Trade // v47-3tt3
Impact Evaluation of Integrated Extension Programs: Lessons Learned from the Community Gardening Program
Abstract
Integrated programming is a coordinated Extension approach to address multi-faceted community issues. An integrated Extension approach is needed to address complex community issues in a meaningful way. Planning, implementation, and evaluation of an integrated program should be considered as a joint effort by the partnering Extension agents. This article describes how to document the impacts of integrated Extension program to reflect the coordinated effort of the Extension team. Documentation as well as sharing impacts with the partners is necessary to strengthen the collaboration and sustain the integrated Extension programming effort.
Introduction
It is becoming clear that an integrated Extension approach is needed to address multi-faceted community issues effectively. For example, obesity is a significant issue in the U.S. (CDC, 2008). The obesity issue cannot be addressed by focusing solely on nutrition education. It is important to understand and influence the other factors related to this problem in order to effectively support sustainable change. These factors may include limited access to fresh fruits and vegetables, limited resources available to buy healthy foods, and limited opportunity to do exercise. By integrating efforts across different program areas, Extension will be able to address real-life community issues effectively.
"Integrated programming is defined as a collaborative approach involving partners and various disciplines planning and implementing one or more strategies to impact micro and macro systems associated with one or more identified issues" (DeBord, 2007, ¶ 11). Integrated Extension programming is a joint approach (involving two or more disciplines) to planning, delivering, and evaluating multi-faceted strategies for addressing broad community issues. It involves a coordinated effort among Extension educators in community development, 4-H, family and consumer sciences, agriculture, horticulture, and natural resources to meet the needs of the community.
In integrated programming, it is essential to collaborate across multiple disciplines to bring necessary resources to address community issues (DeBord, 2007). Integrated programming empowers local Extension educators to address community issues realistically by complementing what they have to offer from their respective program areas.
With all of these advantages, there are some challenges of integrated programming. The most significant challenge is managing the collaboration through the process of planning, delivering, and evaluating the Extension program without any major conflict. It is important for collaborating partners of integrated programming to understand that everyone has to contribute to achieve the impact and that everyone has to share in the credit for the impact. If the collaborators are ready to share the program responsibility as well as credit, then it is easy to manage an integrated Extension program without major conflicts.
If the credit is not shared by the partners, then there is a potential to destroy the collaboration and integrated programming. This can happen if the impact evaluation is not systematically designed to document a broad range of impacts of integrated programs and if some of the partners have been left without any impacts for their program areas. Lack of tools available for interdisciplinary programming is cited in Extension literature (Guion, 2009). This article describes how to evaluate an integrated Extension program for a sustainable collaboration.
The community gardening concept was adopted as a vehicle for integrated Extension programming in North Carolina. The Horticulture agents, 4-H agents, Family and Consumer Sciences agents, Agriculture agents, and Community Development agents in counties worked together to conduct community gardening programs. They collectively coordinated the community gardening program to educate community members about gardening, community development, nutrition and healthy lifestyles, youth development, soil and water conservation, selection of plants and varieties, marketing produce, etc.
Impact Evaluation of Integrated Programs
The impact evaluation of integrated Extension programs can be organized under the following steps.
- Determine the major Extension activities of the integrated program.
- Identify the objectives of major Extension activities.
- Determine the intended outcomes of objectives.
- Identify unique indicators for intended outcomes.
- Develop an approach for collecting impact data.
Determining Major Activities
The first step of impact evaluation is to determine the major Extension activities of the integrated program. For an example, the community gardening program has educational activities focused on community development, gardening and horticulture education, healthy lifestyle education, youth development, and environmental conservation. In integrated programming, these educational activities support each other in achieving the overall program goal.
Identification of Key Objectives
Next, key objectives of major Extension activities should be identified in order to plan impact evaluation. For example, the community development Extension activities aim to accomplish the following objectives:
- To organize the community for a collaborative project such as community gardening.
- To increase community members' involvement in community development activities.
- To develop sustainable communities.
The gardening and horticulture education activities focus on the following objectives:
- To increase participants' knowledge about gardening.
- To increase participants' active involvement in gardening.
- To increase the availability of fresh fruits and vegetables.
The healthy lifestyle education activities of the community gardening program aim to achieve the following objectives:
- To increase fruits and vegetable consumption of participants.
- To increase physical activities of the participants.
- To reduce the obesity of participants.
The youth development activities of the community gardening program aim to achieve the following objectives:
- To further youths' conceptual knowledge about plants, soils, insects and the environment.
- To develop youths' skills in practicing sustainable gardening techniques.
- To develop youths' leadership skills, including communication, critical thinking, problem solving, decision-making, and cooperation.
Determining Intended Outcomes
If the objectives are written in action form, it is easy to determine the intended outcomes of planned Extension activities. For example, as a result of implementing a coordinated effort for achieving the community development objectives, the following outcomes can be expected:
- Coordinated involvement in community development.
- Increased number of community members taking leadership in community development work.
- Increased responsibility for caring and sharing community resources.
The following outcomes can be expected if the horticulture education activities are implemented:
- Participants' increased knowledge about gardening.
- Participants' increased involvement in gardening.
- Increased availability of fruits and vegetables.
The following outcomes can be anticipated for the healthy lifestyle education Extension activities:
- Increased fruits and vegetable consumption of participants.
- Increased physical activities of participants.
- Reduced body mass index (BMI) of participants.
If the youth development activities are implemented the following outcomes can be expected:
- Youths' increased knowledge about plants, insects, and environment.
- Youths' gardening skill development.
- Youths' leadership skill development.
Identification of Impact Indicators
It is important to identify unique indicators for outcomes to avoid multiple counting in impact evaluation of integrated programs. An impact indicator can be described as a reasonable and meaningful measurement of an intended client outcome. For example, the following impact indicators can be used to evaluate the outcomes of coordinated community development activities:
- Number of community development activities organized and carried out by the community.
- Number of community members engaged in leadership roles.
- The total value of volunteer hours granted by the community members for their community development.
For the horticulture education part of the program, the following impact indicators can be used:
- Number of participants increased their knowledge about gardening.
- Number of participants involved in gardening.
- Value of the amount of fruits and vegetables produced.
The following impact indicators can be used to evaluate the outcomes of healthy lifestyle education Extension activities:
- Number of participants increased their fruits and vegetable consumption.
- Number of participants increased physical activities.
- Number of participants reduced their BMI.
For the youth development component of the Extension program, the following impact indicators can be used:
- Number of youths increased their knowledge about plants, insects and environment.
- Number of youths developed their gardening skills.
- Number of youths developed their leadership skills.
Collecting Impact Data
The next step of evaluating integrated programs is to develop tools and methods for collecting impact data. Integrated programming involves working with the target audience for an extended period of time. This makes it possible to conduct pre- and post-evaluations to collect data. By comparing pre- and post-test evaluations, impact of the Extension program can be documented.
Practical Implication
It is important to determine and document the impacts of diverse Extension activities to reflect the coordinated efforts of the integrated Extension program. Documentation as well as sharing impacts with the partners is necessary to strengthen the collaboration and sustain the integrated Extension programming effort.
References
Centers for Disease Control (CDC). (2008). U.S. Obesity trends 1985-2007. Retrieved on April 23, 2009 from: http://www.cdc.gov/nccdphp/dnpa/obesity/trend/maps/
DeBord, K. (2007). How integrated Extension programming helps market Cooperative Extension: The North Carolina recommendation. Journal of Extension [On-line], 45(5) Article 5COM1. Available at: http://www.joe.org/joe/2007october/comm1.php
Guion, L. A. (2009). A tool for focusing integrated team efforts on complex issues. Journal of Extension [On-line], 47(1) Article 1TOT2. Available at: http://www.joe.org/joe/2009february/tt2.php