Viewing Bennett's Hierarchy from a Different Lens: Implications for Extension Program Evaluation

Introduction

Bennett's hierarchy has been used for almost 35 years in Cooperative Extension. Educators continue to relate well to this hierarchy in evaluating their Extension programs. Bennett's hierarchy contains seven sequential steps (input, activities, participation, reaction, knowledge, skills, opinions, aspirations-KASA, practice change, and end results/social, economic, environmental conditions-SEEC) (Figure 1). The first four steps focus around process evaluation, while the last three steps focus on outcome/impact evaluation. Modifications were made to the hierarchy by Bennett and Rockwell in1995 and in 2000 by adding a continuum linking program evaluation and program development (Figure 2). This revision helped educators understand that evaluation should be considered upfront in the design or planning phase of a program, not as an after-program activity.

Figure 1.
Bennett's Hierarchy

Figure 2.
Bennett and Rockwell's TOPs Model

Today, in a time of continued reduction in government funding, Extension professionals are challenged more than ever before to document outcomes of programs and address stakeholder demands for accountability. This article provides a framework for linking Bennett's hierarchy to program outcomes and costs. Extension professionals could use this framework to link program outcomes and costs associated with such outcomes.

Examining Bennett's hierarchy from a different lens provides some insights to link the hierarchy with outcomes and costs associated with evaluating a program. A positive association between the seven steps of hierarchy and outcome type (short, intermediate, and long term) is proposed (Figure 3). As one moves up the hierarchy, the evidence of program impact gets stronger. Collecting evidence to assess impact of programs at the higher levels of the hierarchy becomes costly and time consuming and requires more skill (Figure 3). For example, one may use simple pre-post measures to assess short-term outcomes. On the other hand, to assess behavior/practice change, follow-up of participants is required, which adds to the cost of evaluating a program. This needs to be communicated to field-based educators so that they can plan early on what is needed in terms of costs, time, skills (data collection, analysis, and interpretation), and resources needed to evaluate an Extension program (Table 1).

Figure 3.
Linking Bennett's Hierarchy to Program Outcomes and Costs

Based on the information presented in Figure 3, a framework for linking Bennett's hierarchy to program outcomes and costs was developed (Table 1). As shown in Table 1, evaluating programs at the lower levels (input, participation, activities, and reactions) may require little effort and are less expensive. However, evaluating at the lower levels will help program staff to assess ongoing program activities to make adjustments as the program progresses and to see whether or not the program is being implemented as planned. On the other hand, documenting and/or collecting evidence at higher levels of the hierarchy (KASA change, behavior change, and SEEC) requires skills relative to questionnaire development, data collection and analysis, interpretation, and reporting. In addition, it may also require understanding of evaluation designs, data collection at multiple points, sophisticated statistical analyses such as analysis of covariance, general equation modeling, use of covariates, etc. If these are planned in advance and properly done, the potential for program impact is stronger.

Steps for Using the Framework

First, decide on the level of evaluation you want to conduct, that is, process evaluation (lower levels of Bennett's) or outcome evaluation (higher levels of Bennett's) or both. In this step, develop a concept of linking program objectives to evaluation questions to outcomes (Radhakrishna & Relado, 2009).

Second, identify key indicators for the evaluation. Make sure that the indicators you identified are measureable and relevant to the program.

Third, once you decide on the type of evaluation and key indicators, consider all the costs that you might incur to conduct the evaluation (Figure 3 and Table 1). For example, one of your objectives is to assess practice/behavior change (higher levels of Bennett's) as a result of participation in a financial management program. This implies that you will be collecting data at least three times (pre, post, and delayed) to determine practice/behavior change. Perhaps you may also want to add a comparison group (non program participants). Including a comparison group will help make meaningful comparisons of a particular Extension program's strengths and weaknesses. In addition, you need to be cognizant of the time, labor, and other resources you need to have to carry out the evaluation at higher levels of the hierarchy.

Fourth, design a matrix to document all the costs you incur to evaluate the program. Use of spreadsheets (Microsoft Excel) or other technology to document the costs for each level would be a good start.

Following these steps will help you to estimate the costs involved in evaluating an Extension program. In addition, these steps will also help guide the development of a budget for evaluation, especially if you are writing a grant for funding from a private or a public agency.

Conclusions

Documenting program/project outcomes will continue to challenge program managers and educators, especially in the accountability era. Early identification of costs associated with documenting short, intermediate, and long-term outcomes will go a long way in assessing the costs and benefits associated with evaluation of an Extension program. Extension specialists and program managers should use these steps when they conduct training/workshops relative to costs and benefits associated with evaluating an Extension program. In addition, the process of linking costs to program outcomes should be communicated to all educators who evaluate Extension programs. Such communication will help link evaluation questions to outcomes and costs, and ultimately justify the value of Extension programs to public good.

References

Bennett, C. (1975). Up the hierarchy. Journal of Extension [On-line], 13(2). Available at: https://www.joe.org/joe/1975march/1975-2-a1.pdf

Bennett, C., & Rockwell, K. (1995, December). Targeting outcomes of programs (TOP): An integrated approach to planning and evaluation. Unpublished manuscript. Lincoln, NE: University of Nebraska.

Radhakrishna, R. B., & Relado, R. Z. (2009). A framework to link evaluation questions to program outcomes. Journal of Extension [On-line], 47(3) Article 3TOT2. Available at: https://www.joe.org/joe/2009june/tt2.php