Using a Pre- and Post-Test to Assess 1-Day Learning of Agricultural Producers Participating in an Educational Program

Introduction

Extension educators are increasingly faced with documenting program results. In today's political environment, greater emphasis to demonstrate positive value from programming efforts is required by funding agencies. Besides providing justification to funding agencies, Diem (Diem, 2003) reports that such impact is important and greatly needed because:

• It earns and builds professional, organizational, and political credibility and support.


• It satisfies the requirements of political bodies and funding agencies.


• It yields tangible results that serve as a basis for scholarly publications, as well as awards and recognition.


• It determines to what degree participants achieve intended results.

Like other Land-Grant universities, the University of Missouri Extension's (UME) Integrated Pest Management (IPM) program has been supported by funding on an annual basis. USDA-CSREES (USDA-CSREES, 2007) funds each of these programs to develop and implement ways to address IPM issues. Future funding is contingent upon reporting program impacts through an annual report submitted to USDA-CSREES administrators.

UME initiated an outreach effort to agricultural producers beginning in 2001. One-day classroom settings were held throughout Missouri during the winter, with the major objective of conveying IPM technology. A team of UME state specialists taught each 1-day event, which consisted of discussion of topics such as recent pest control technology, pest identification, updates of new pests, and agronomic practices. To serve as future reference material, a notebook was provided to each participant that contained a variety of UME publications.

Evaluation Method

The sample was comprised of agricultural producers of grain crops who participated in the entire 1-day event during 2002 (n = 130) and 2003 (n = 98) and completed both pre- and post-tests. A 17-question pre- and post-test was developed based on the content being taught during each year's program. Although the questions were similar, some were changed between years. Therefore, each year is presented separately.

The pre- and post-test questions consisted of multiple-choice and true-false formats. In 2002, each test consisted of 15 multiple-choice and two true-false questions, while in 2003, each test consisted of 12 multiple-choice and five true-false questions. With some multiple-choice questions, participants were required to respond by identifying a pest shown in a photograph that was projected onto a screen. All multiple-choice questions contained four possible responses with only one correct response.

During the introductory phase of each day's event, participants were made aware that a post-test would be administered at the conclusion of the program; however, participants were not aware that the post-test would be identical to the pre-test. Following introductory comments, the pre-test was administered; then at its completion, all participants' pre-tests were returned to the moderator. To compare pre- and post-test results for each individual while maintaining confidentiality, participants were asked to identify their tests by writing the last four digits of their social security number on each test.

Pre- and post-test scores of the participants were graded based upon the percentage of correct answers. The score sets were then compared with a paired student's t-test. The program followed the guidelines of the Institutional Review Board as analyses were conducted at a central location.

Results

Pre-test scores (> 60%) of these producers participating both years indicate that they have some baseline level of IPM knowledge. A comparison of pre- and post-test scores serves as a measure of these agricultural producers' short-term learning of the IPM material that was presented during each 1-day event. Their post-test scores clearly indicate that by the end of the program they had increased their knowledge of the presented IPM technology. The increased pre- and post-test score differences were 11% and 16% for 2002 and 2003, respectively. The improvement of their test scores was highly significant during both years that the program was offered.

Implications

This increase in IPM knowledge was measured based on only short-term recollection of the material presented during the day. To determine if there is a long-term benefit from these events, a follow-up evaluation is needed. Perhaps in future years if similar programs are held, surveys would be designed and conducted that would target this same audience 6 or 12 months following the initial event. This comparison would provide some measurement as to the extent of longer-term impact of retaining this technology.

Brief pre- and post-test evaluations are an assessment tool that has relative ease in application for Extension professionals. They are easy to develop, collection of data is simple, and the results are straight-forward. They show impact, which is increasingly important in documenting results to administrators and funding agencies. Although this program focuses on teaching IPM technology, the methodology can be adapted and applied to an array of Extension programs (Lippert, Plank, & Radhakrishna, 2000; Guion, Turner, & Wise, 2004; Chapman-Novakofski, DeBruine, Derrick, Karduck, Todd, & Todd, 2004).

References

Chapman-Novakofski, K., DeBruine, V., Derrick, B., Karduck, J., Todd, J., & Todd, S, (2004). Using evaluation to guide program content: diabetes education. Journal of Extension [On-line], 42(3). Available at http://www.joe.org/joe/2004june/iw1.shtml

Diem, K. G. (2003). Program development in a political world--It's all about impact! Journal of Extension [On-line], 41(1). Available at: http://www.joe.org/joe/2003february/a6.shtml

Guion, L. A., Turner, J., & Wise, D. K. (2004). Design, implementation, and evaluation of an elder financial abuse program. Journal of Extension [On-line], 42(3). Available at http://www.joe.org/joe/2004june/a6.shtml

Lippert, R. M, Plank, O., & Radhakrishna, R. (2000). Beyond perception: A pretest and posttest evaluation of a regional Internet extension inservice training. Journal of Extension [On-line], 38(2). Available at: http://www.joe.org/joe/2000april/a2.html

USDA-CSREES (2007). Integrated Pest Management [On-line]. Retrieved October 16, 2008 from: http://www.csrees.usda.gov/integratedpestmanagement.cfm