Effectiveness of an SPAT Educational Program

Introduction

Government Regulation of Pesticide Issues

Pesticides and their use are subject to extensive governmental regulation. Regulatory agency accountability and educational program effectiveness are becoming the measures of determination for funding. Regulatory agencies have been given extensive powers to address public concern about use of pesticides. The continued existence of many governmental educational programs is dependent on this information.

Need for Program Evaluation

Pesticide applicators in the United States, and Texas specifically, must attend Pesticide Applicator Training (PAT) certification programs as a condition of seeking licensure (Texas Structural Pest Control Law, as amended 1997). However, PAT programs may or may not be effective in preparing individuals to pass federal and state required certification examinations.

Measuring performance of individuals who complete an educational program is a recognized method for evaluating the effectiveness of training programs (Wehrenberg, 1983). Government accountability rules and public scrutiny are leading to a closer evaluation of the effectiveness of educational programs. Regulatory agencies and educational entities must continue efforts to keep decision-makers and other stakeholders informed about how the educational needs of clientele are being met in order to merit support (Carpenter, 1997).

Purpose

The primary purpose of the study discussed here was to assess the effectiveness of the Texas structural pesticide applicator certification training program in preparing license candidates for the General Standards Examination (GSE), a requisite in the licensing process.

To continue meeting the statutory requirements placed on the Structural Pest Control Board (SPCB) and to deliver more effective certification programs, the study was conducted to determine:

• The effectiveness of the Texas Structural Pesticide Applicator Training (SPAT) program based on participant pass/failure rates as indicated by selected measurement instruments;

• The relationships between program participants' demographics (age, education, experience) and their pass/failure rates;

• The andragogical methods/techniques used to conduct SPAT (Knowles [1970] defined andragogy as the art and science of helping adults learn.);

• The demographics of the trainers who conduct SPAT; and

• The relationships between trainers' teaching methods and the pass/failure rates of SPAT program participants.

Methods and Procedures

Population and Instrumentation

The population of this study was all certification candidates trained by the Texas Agricultural Extension Service's Agricultural and Environmental Safety Program in 1996 (n = 1,303). The researchers and SPCB personnel designed pre-test and post-test survey instruments and administered them to at least one of each SPAT course provider's training session(s) offered between January 1, 1996, and December 31, 1996.

The SPCB administered the state examination to applicator candidates who completed the SPAT program. The researchers developed a separate instrument using Creswell's 1990, Perceptions Held Regarding Principles of Teaching-Learning, Part II survey as a model to gather data regarding demographics and andragogical practices of the course providers (n = 11) during the fourth quarter of 1997 and the first quarter of 1998. 

The pre-test and post-test contained some of the same 20 questions, and data were recorded on a scantron form. The first five questions only on the pre-test gathered demographic data about the population.

The General Standards Examination (GSE) was administered on forms prescribed by the SPCB. The GSE contained the same 20 questions found on the post-test. The test questions were derived from the General Standards Training Manual (GSTM), which is the SPCB prescribed study guide for applicator license candidates. The Executive Director of the SPCB mailed the survey to course providers in November 1997. A follow-up survey was mailed to non-respondents in April 1998 with a 98% response.

The pre-test and post-test instruments used were adopted from a related study (Vitzthum, 1982) wherein acceptable validity and reliability were determined using t-tests. Content and face validity were determined using a pilot test and a panel of experts from the SPCB, the Department of Agricultural Education at Texas A&M University, and the Texas Agricultural Extension Service. SPCB personnel designed the pre-test and post-test instruments identical to the GSE testing instrument.

Procedure and Data Analysis

Data analysis followed principles prescribed in Borg and Gall (1989). Andragogical strategies and instrument design followed the principles prescribed in the Handbook of Evaluation and Measurement Methods (Phillips, 1983). Descriptive statistics were used to report findings in the study.

Results and Discussion

Diversity and Success Rates of Applicator Candidates

Applicator candidates in Texas were a diverse group as described in Table 1 and Table 2. Sixty-five percent were between 20 and 39 years old, and 98% had a high school or higher education.

Statistically significant differences existed between the pre-test and post-test performance of SPAT program participants. The mean pre-test score was 49.5%, and the mean post-test score was 73.5%. A two-sample t-test for mean differences produced a t-statistic of -29.3 and probability of 0.0001 (Table 3), indicating the SPCB SPAT program is effective in increasing knowledge and thus in preparing participants for the GSE.

Unlike in the pre-test findings, age influenced the performance of post-test participants. Statistically significant differences existed in the post-test performance of SPAT program participants based on age. Compared to the pre-test mean scores, increases in the post-test mean scores were evident in all but one (age 60 and over) of the age groups. Based on the F value of 86.4 and probability value of 0.0001, age was considered an influence in participant performance (Table 4).

Education attained had a statistically significant influence on the performance of SPAT program participants on the post-test (Table 5). This was consistent with the findings on the pre-test. An ANOVA produced an F value of 33.2 and a probability of 0.0001.

Demographics of SPAT Course Providers

• The average SPAT course provider had been teaching certification classes for 9 years.

• Average formal teaching experience for SPAT course providers was 10 years.

• SPAT course providers had an average of 18 years experience in the pest control industry.

• Most (55 %) SPAT course providers had a master's or doctoral degree.

Effectiveness of Classroom Practices

In terms of practices that produced passing GSE scores, lecture/discussion was the most frequently used andragogical practice, and use of 35-millimeter slides was the most frequent information delivery technique (Table 6).

GSE Passing Rate Data

• After participating in the SPAT program, the performance of license candidates was improved significantly. Participants' scores increased 30% from the pre-test mean score of 49.5% to the GSE mean score of 79.6%. The GSE contained the same 20 questions found on the post-test.

• Seventy-three percent of SPAT program participants who attended training programs where 35 mm slides were the instructional materials passed the GSE.

• Seventy-seven percent of SPAT program participants who attended programs taught using lecture/discussion passed the GSE.

Conclusions and Implications

Training Techniques Effective on Test Performance

• The SPAT program was effective in preparing license applicants for the General Standards Examination.

• The performance of SPAT program participants was influenced by the andragogical practices course providers used in conducting applicator training programs.

• SPAT course providers used lecture/discussion most frequently as an andragogical practice to conduct applicator training programs.

• The use of pest specimens as a teaching tool had a positive impact on participants' test performance. In terms of achieving passing GSE scores, 300 SPAT program participants (78%) met and/or exceeded the minimum licensing standards when this andragogical practice was used.

Recommendations

Continue SPAT Program for Licensing

• The SPAT programs, instructional materials, and instructional activities were effective in preparing license applicants for the GSE.

• In order to help insure proper pesticide use by applicators, the SPAT program should be continued as a prerequisite to licensure in Texas.

Establish an Applicator Training Institute

• Establishing an applicator training institute at Texas A&M University would provide in-depth training experiences for license applicants and allow using the full array of educational disciplines within the university system.

• Category specific applicator training could be provided in entomology, weed science, and plant pathology.

• Subject matter-specific training such as toxicology; pesticide mode of action; emergency response and first aid; proper mixing, loading, and application; storage; and disposal could be offered in hands-on settings that provide license applicants with practical experiences, rather than slide or videotaped experiences.

References

Borg, W. R., & Gall, M. D. (1989). Educational research: An introduction (5th ed.).White Plains, NY: Longman.

Carpenter, Z. L. (1997, September). FY 98 brings challenges, opportunities. Inside Extension, 7(1), pg. 1.

Creswell, J.L. (1990). An identification and assessment of Extension educational delivery systems for training of private pesticide applicators. (Doctoral dissertation, Iowa State University, 1990). Dissertation Abstracts International, 51, A2609.

Farm chemicals handbook (1996). Willoughby, OH: Meister Publishing.

Phillips, J. J. (1983). Handbook of training evaluation and measurement methods. Houston, TX: Gulf Publishing Company.

Texas Structural Pest Control Act, as amended, (1997).

Vitzthum, E. F. (1982). An evaluation of the general standards training program for Nebraska commercial pesticide applicators (Doctoral dissertation, University of Nebraska, 1982). Dissertation Abstracts International, 43, A632.

Wehrenberg, S. B. (1983). Training: evaluation of training. Personnel Journal, 62, 698-702.