What Did That Program Do? Measuring the Outcomes of a Statewide Agricultural Leadership Development Program

Introduction

In this age of tight budgets, Extension educators are feeling even more pressure to demonstrate the impacts of their programs to administrators, funders, and stakeholders. Leadership program evaluation presents two dilemmas. The first is the lack of time in one's day and the amount of time that can be invested in a thorough program evaluation. The second, an adequate method to measure a program's outcomes in a cost-effective manner. The study reported here undertook the process to evaluate three levels of leadership program outcomes after the program participants have left the program and moved on with their lives. Alumni from a statewide agricultural leadership program were evaluated on the self, business, and community level using a survey developed from the EvaluLEAD framework proposed by Grove, Kibel, and Haas (2005). This framework recommends evaluating leadership program outcomes on the self, organization, and community levels.

Each class of the agricultural leadership program takes place over an 18-month period, with goals of developing an understanding of the diversity of people, cultures, industry, and networks within the realm of food, agriculture, and the environment. Leadership and the current issues facing America and the consuming public are also studied. Twelve study institutes occur within the state, nationally, and internationally. The program has been in existence for 20 years and has graduated 10 classes of alumni.

Methodology

The research employed a descriptive survey design. An original Web-based instrument was developed to collect the data from a census population of 262 alumni from 10 program classes. The instrument that evolved was based upon the EvaluLEAD framework developed by Grove et al (2005) and focus group interviews. The instrument was field tested with another statewide leadership program's alumni and reviewed by a panel of experts. The survey instrument gathered both quantitative and qualitative data, while focusing on assessing individual, organizational and community outcomes. The instrument was found to be highly reliable and valid, and effectively measured the outcomes of this leadership program. The instrument had Cronbach's alpha reliability estimates ranging from .79 to .92.

The alumni were either e-mailed a Web-based survey or mailed a hard copy. Alumni were notified by letter and e-mail prior to the survey. Numerous e-mail contacts to nonrespondents were made during the survey, including a postcard follow-up. The response rate for the survey was 75% (n=196), with all program class graduates of alumni represented.

Findings

This leadership education program has a positive impact on many different levels after alumni leave the program. The data collected indicates that people in the position of learning, as participants in this leadership education program primarily:

• Gain knowledge of themselves

• Improve in business

• Are more aware of cultural differences

• Increase in cultural awareness

• Are active in the local community

The highest rated activities for community involvement of alumni after they leave the program are:

• Church (57%)

• County Farm Bureau (48%)

• Extension (39%)

• 4-H (30%)

• Soil and Water (29%)

• FFA (24%)

• Commodity Group - Soybean (21%)

• Chamber of Commerce (21%)

• Commodity Group - Corn (20%)

• State Farm Bureau — (18%)

• School —Athletics (14%)

• Commodity Group — Beef (11%)

Additionally, 94% of the respondents indicate that this statewide agricultural leadership program should be continued. Eighty-six percent indicate that they experienced a high level of change because of their program participation. Furthermore, the method of evaluation worked as a model to determine leadership program outcomes and an effective instrument was developed to measure these outcomes.

Discussion

The research reported here is the first to determine a comprehensive impact of a statewide agricultural leadership program at the post-program evaluation level. The results of the study provide an important examination of the impact of a statewide agricultural leadership program at the post-program evaluation level utilizing a comprehensive instrument developed to gain insight into alumni outcomes and program achievements.

Implications for Practice

The outcomes of the study can be applied to many different kinds of leadership programs. An instrument has been developed that can effectively measure the outcomes of statewide leadership programs on three domains after the participant leaves the program. The instrument that has resulted from the application of the EvaluLEAD framework will help us to reach a better understanding of leadership programs. At the very least, we are offered a basic understanding of programmatic results on several levels after the participant leaves the program. This is good news for program directors, stakeholders, funders, and others who have long lamented the lack of an effective measurement tool.

References

Grove, J. T., Kibel, B. M., & Haas, T. (2005). EvaluLEAD. A guide for shaping and evaluating leadership development programs. Oakland CA: Sustainable Leadership Initiative. Public Health Institute.