Pilot Evaluation of an Internet Educational Module for Agricultural Safety

Introduction

Statistical data from the National Safety Council (NSC, 2010) indicates the dangerous nature of agriculture by its high death rate per 100,000 workers—more than eight times the all-industry average. Records of agricultural-related injuries and deaths from other sources (Adekoya & Pratt, 2001; Goldcamp, Hendricks, & Myers, 2004; Hard, Myers, & Gerberich, 2002; Myers & Adekoya, 2001) support the NSC findings that agriculture is a dangerous occupation and agricultural tractors are a leading cause of death. According to NSC data, an estimated 305 vehicle-related deaths occurred on farms in the United States. Not a surprise, considering that Myers and Snyder (1995) estimated 2.98 million of 4.8 million tractors in the U.S. lacked ROPS, the safety feature that prevents most tractor deaths.

A concern is that young people who work on farms generally are not properly educated to operate tractors and machinery. The federal government's identification of the need for youth education in the operation of agricultural tractors and machinery (Federal Register, 1970) is a result of tragic injuries to youth. The U.S. Department of Labor implemented a regulation in 1971 with a requirement for specialized training for youth under age 16 before legal employment off their parents' farm (Code of Federal Regulation, 1971; Child Labor Bulletin 102, 2007). The requirement includes 24 hours of training to obtain certification in the operation of agricultural tractors and machinery. One provider, as stated in the Federal Register (1970), is the Cooperative Extension Service (CES).

The traditional face-to-face tractors and machinery training is viewed as a time-intensive task. Sorensen (1996) identified personal injury experience as a major reason that instructors justify their time in teaching these training programs. Evidence of this major reason still endures. Often other options of delivery are considered like distance education, satellite, multimedia, and Internet to remove time barriers preventing delivery of these programs.

Internet and computer-based training overcomes some barriers but raises heated debates about effectiveness as an educational medium. Ward and Prosser (2011) conclude that effectively using virtual classrooms, Internet, and other technological tools is not a trivial task and that essential inter-personal elements of education must not be overlooked. Bamka (2000) encourages Extension professionals to take advantage of Internet resources. Coppula (1997) indicates that many are not using the full capabilities of computers to assess and track learning progress. Data Smog author Shenk (1997) also discusses the need for a broader view of education. "Education is not the same as accessing information," he writes. "We don't have a problem accessing information easily and cheaply in this country. We've had great libraries and even adequate school libraries for a long time. Education is actually the limiting of information" (Investor's Business Daily, 1997).

One critical aspect of any agricultural tractor and machinery training program is universal symbols. Operators must be able to identify with just a glance whether a tractor or machine is working within its desired designed limits or is malfunctioning. Universal symbols provide a quick indication of the operational status of equipment. Knowing the meaning and use of the 248 universal symbols is important to safe operation of farm machinery and tractors. Operators who understand universal symbols can move from one piece of equipment to another and understand the function of each control. Universal symbols let the operator know, at a glance, the condition of the machinery and/or the function of controls. These symbols transmit information without words or lengthy explanations. Each universal symbol has only one meaning, as defined by the American Society of Agricultural Engineering (ASABE Standard, 2009). An operator who recognizes and understands all 248 universal symbols is better prepared to operate agricultural equipment safely.

An Internet educational module (IconMatch) was developed by the first author and colleagues (Schwab et al., 1996) for the purpose of increasing the agricultural tractor and machinery training students' ability to recall universal symbols. The module introduces students to universal symbols used on operator controls and gauges of agricultural machinery. The module displays the graphical symbols for operator controls and displays on agricultural equipment defined by ASABE Standard (2009) in a game format. The Javascript (Morrow, 1996) program randomly assigns nine universal symbols (icons—thus the module title IconMatch) to a matrix and provides their corresponding description to a random matrix of answers. The participant's task is to match the correct description with the proper icon. The participant uses the mouse to click on the description of the symbol and then clicks on the corresponding symbol (Figure 1). After all the symbols are identified, the participant clicks on the "Score" button. The module then places a red "X" on any wrong answer as shown in Figure 2.

Figure 1.
The IconMatch Module as It Appears in The Internet Browsers with Four Descriptions Placed Under the Universal Symbols (Schwab et al., 1996)

Figure 2.
The IconMatch Module as It Appears in the Browsers After All Descriptions Were Placed and the Score Button Activated (Schwab et al., 1996)

The participant can see the correct answer by clicking on the red "X" after the responses are scored (Figure 3.). The correct answer appears, and the symbol's border is then colored blue, distinguishing it from the correctly answered symbols. The participant can now play another game by clicking on "Reset" or end the learning session with the "Quit" button.

Figure 3.
The IconMatch Module as It Appears in the Browsers After Four of the Wrong Descriptions [Identified by an "X" in Figure 2] Were Activated to Reveal the Correct Answer (Schwab et al., 1996)

Method

A field trial study was conducted to determine the impact of using the universal symbols educational module IconMatch. Four test groups were asked to participate in a pre- and post-test evaluation of participants' knowledge and recognition of universal symbols and their descriptions. The four test groups represent three age groups (14-16 year olds {n=17}, college seniors {n=30}, and high school teachers {n=18}) and a control group for the 14-16 years olds {n=28}. These groups are samples of convenience because they were enrolled in the existing training programs, agricultural safety course, or instructor training sessions. No control group for college seniors and high school teachers was used because the analysis to determine if there was a difference in the improvement between pre- and post-test results was only conducted for youth who would receive training specified by the U.S. Department of Labor. Using a pre- and post-test, the participants' knowledge and recognition of the universal symbols was measured.

A pre-test with 36 universal symbols and their respective descriptions was administered. Participants had 10 minutes to match the correct descriptions to the appropriate universal symbol by writing the number before the description on the line next to the corresponding universal symbol. After 10 minutes, the participants were instructed to stop writing and the pre-tests were collected.

After completing the pre-test, the participants in all groups except the control group were introduced to IconMatch. A brief example of how to operate the module was provided. They were then instructed to spend time interacting with IconMatch and record the time they were spending using the educational module. The participants reported spending an average of 23 minutes (SD = 3.9) with IconMatch. The control group received no information about IconMatch, but participated in the youth training described by U.S. Department of Labor (Code of Federal Regulations, 1971) because they were in the 14-16 year old group. This was the same training received by the 14-16 year old group that was introduced to IconMatch.

All groups received the post-test some time after taking the pre-test. The post-test used the same 36 universal symbols. However, the symbol and descriptions order of appearance on the test were changed. On the post-test participants were also asked to report the amount of time spent using IconMatch and any comments about the IconMatch.

Results

Using a paired t-test, the results of pre- and post-tests were evaluated. A statistically significant difference between the scores on the pre- and post-test for all groups was measured. Table 1 shows the results of the paired t-test analysis. The results indicate an increase in the ability to correctly identify 36 universal symbols in less than 10 minutes (Figure 5). The multiple scores for the three age groups (14-16 year olds, college seniors, and high school teachers) doubled between pre- and post-testing. Several participants achieved a score of 100% correct for all 36 icons. Even the control group of 14-16 year olds experienced an increase between the pre- and post-test results.

The time used by the participants to take the pre-test was the full 10 minutes allowed. None of the participants provided an answer for all 36 universal symbols on the pre-test. Post-test times were different, with several participants completely answering the all 36 universal symbols in as little as 5 minutes. The average time to complete the post-test was 8.2 minutes. One participant used only 5 minutes and successfully matched all 36 universal symbols.

Table 1.
Results of the Paired t-Tests

Groups	Sample Size	Mean (Standard Deviation)		Points of Knowledge gain*	t Value	P Value
		Pre-test	Post-test
14-16 years olds	17	33 (9)	78 (20)	45	10.51†	<0.0001
College Seniors	36	55 (17)	85 (19)	30	11.24†	<0.0001
High School Teachers	18	52 (18)	82 (16)	30	7.59†	<0.0001
Control (14-16 years olds)	28	44 (15)	60 (18)	16	6.33†	<0.0001
*Points of knowledge gain are the point difference between the average pre-test and average post-test results for each group. † Significant at the 0.001 level for the stated sample size.

Figure 5.
Comparison of the Average Pre- and Post-Test Results for All Groups Participating in IconMatch Evaluation

An analysis of variance was conducted to determine if there was a statistically significant difference in the improvement measured between the pre- and post-test results of the 14-16 year olds and the control group of 14-16 year olds. The difference in the improvement measured between the pre- and post-test results for the 14-16 year olds group was statistically different than the control group (p< 0.0001). The mean scores of youth that received the IconMatch training were 29 points higher than the youth that did not use IconMatch.

An analysis of variance was also conducted to determine if there was a significant difference in the improvement measured between the pre- and post-test results for the three different age stages (14-16 years olds, college students, and teachers). The results indicate a statistically significant difference between the age stages (p = 0.008). An additional test (Tukey's pairwise comparison) was conducted to determine which age had similar improvements between pre- and post-test results. The differences in improvement between pre- and post-test results of the age stages for college seniors and high school teachers were not statistically significant (p = 0.05). The differences in improvement between the pre- and post-test results of the age stages for college seniors and high school teachers were significantly different from the improvement observed in the 14-16 year olds (p = 0.05).

Discussion

An improved performance of correctly identifying universal symbols for the control group (14-16 year olds) was not entirely unexpected. These youth still participated in a tractor and machinery certification program that involved some interactivity with universal symbols. At a minimum, these youth were briefed on the operational controls and indicators symbols for the tractor used in the participants' evaluations of their ability to operate safely through an obstacle course.

The 14-16 year olds group that used IconMatch was observed to have greater increase than the control group. This corresponds to Wallace and Mutooni's (1997) findings that measured a significant performance improvement with student receiving Internet-based materials in addition to traditional education.

Differences between the age stages were also expected. The college seniors and high school teachers groups were not as motivated to learn the universal symbols. The 14-16 year olds group was motivated by the goal of obtaining a certificate permitting them to enter the summer work force and earn money. The older age groups have also accumulated more experience with symbols. This is evident from the higher pre-test scores that would diminish the potential improvement opportunities gain by IconMatch.

Conclusion

Overall, the results from the field trial study of IconMatch are favorable. Participants who used IconMatch completed the post-test under the 10-minute limit, with an average of 8.7 minutes. Scores on the post-test showed statistically significant improvement. All participants' comments regarding their assessments of IconMatch were favorable and supportive. The field trial study showed that the Internet educational module increased the participants' ability to recall universal symbols faster and more accurately than those not using IconMatch.

The study reported here can serve as a model for supplementing Extension programming with Web-based educational material. Using the Web for Extension programming may be quite effective, especially for programs targeting youth. However, to be effective in the way demonstrated in the study, it is critical that Web-based materials for youth be interactive rather than text based.

Acknowledgments

Work on this project was supported by the Iowa 4-H Foundation through a grant from the Iowa Department of Agriculture and Land Stewardship and by Iowa State University Extension.

Reference

Adekoya, N., & Pratt, S. G. (2001). Fatal unintentional farm injuries among persons less than 20 years of age in the United States. DHHS (NIOSH) Publication No. 2001-131. Cincinnati, Ohio: National Institute for Occupational Safety and Health.

ASABE Standard. (2009). S304.8: May 2006 Graphical symbols for operator controls and displays on agricultural equipment. St. Joseph, Mich.: ASABE.

Bamka, W.J. (2000). Using the Internet as a farm-marketing tool. Journal of Extension [On-line], 36(4). Article 2TOT1. Available at: http://www.joe.org/joe/2000april/tt1.php

DOL. (2007). Child labor bulletin no. 102. U. S. Government Printing Office. Washington, D.C.: Department of Labor. Retrieved from: http://www.dol.gov/whd/regs/compliance/childlabor102.pdf

Code of Federal Regulations. (1971). Child labor regulation, orders, and statements of interpretation. CFR Title 29, Chapter V, Part 570. Washington, D.C.

Coppula, D. (1997). Getting hyper over learning. American Society of Engineering Educators PRISM. May-June. 6(9):12-17.

Federal Register. (1970). Title 29—Labor. Washington: Division of the Federal Register, National Archives. U. S. Government Printing Office. Washington, D.C. 35(4) January 7, 1970.

Goldcamp, M., Hendricks, K. J., & Myers, J. R. (2004). Farm fatalities to youth 1995-2000: A comparison by age groups. J. Safety Res. 35(2), 151-157.

Hard, D. L., Myers, J. R., & Gerberich, S. G. (2002). Traumatic Injuries in Agriculture. Journal of Agricultural Safety and Health 8(1): 51-65.

Investor's Business Daily. (1997). Education is actually the limiting of information. April 30. 12655 Beatrice Street. Los Angeles, CA.PAGES.

Morrow, C. (1996). Java Unleashed. Indianapolis: Sams.net Publishing Inc.

Myers, J., & Adekoya, N. (2001). Fatal on-farm injuries among youth 16 to 19 Years of Aage: 1982-1994. Journal of Agricultural Safety and Health. 7(2), 101-112.

Myers, J. R., & Snyder, K. A. (1995). Roll-over protective structure used and the cost of retrofitting tractors in the United States, 1993. Journal of Agricultural Safety and Health 1(3), 185-197.

National Safety Council. (2010). Injury facts, 2009 edition. Itasca, IL:NSC.

Schwab, C. V., Miller, L., Ferdig, P., Pechler, A. D., Goering, D., & Hanna, M. H. (1996). Tractor and machinery virtual classroom. Retrieved from: http://www.tmvc.iastate.edu

Shenk, D. (1997). Data smog: Surviving the information glut. New York: HarperCollins.

Sorensen, D. (1996). Young farmers sponsor tractor safety course. March 16, 1996. Iowa Farm Bureau Spokeman. West Des Moines, Iowa: Farm Bureau.

Wallace, D. R., & Mutooni, D. (1997). A comparative evaluation of world wide web-based and classroom teaching. Journal of Engineering Education 86(3), 211-218.

Ward, A., & Prosser, B. T. (2011). Reflections on cyberspace as the new "wired world of education". Educational Technology & Society, 14(1), 169-178.