The Effectiveness of Extension In-Service Training by Distance: Perception Versus Reality

Introduction

Distance learning has increased in use dramatically in the past decade, especially for students and professionals who have full-time jobs or cannot afford the expense or time to travel. The trend towards learning at a distance is especially evident in the Extension Service, where traditional face-to-face classroom instruction is slowly being replaced by distance education in an effort to reach the rural client and county personnel.

Thus, instructors and program facilitators have had to adapt lessons originally developed to be presented in a traditional classroom to that of a virtual one. While this takes many forms, interactive instruction exclusively by the Internet has become more popular now that there is widespread availability of computers and Internet technology (Rogers, 1988; Lippert, Plan, Camberato, & Chastain, 1998).

When considering in-service training, the Internet appears to offer great potential for Extension. Giving Extension agents the option of taking training courses online eliminates the time and cost of traveling to attend traditional face-to-face sessions. Thus, with most county Extension offices now connected, or in the process of being connected to the Internet, distance learning via this medium is an attractive training alternative for a variety of topics (Lippert, et al., 1998).

Methodology and Results

In 2005, a committee of instructional technologists within the Mississippi State University Extension Service (MSU-ES) was charged with examining the potential of online instruction in various formats. More specifically, the amount of interaction that the participant was exposed to within the Internet-based curriculum was investigated. Two types of online training environments were thus developed:

1. Individuals who were assigned to a minimally interactive online environment were asked to access a Web site to complete an in-service training session. The instructor was available via e-mail if the participant had questions, but participants were required on their own to access PowerPoint presentations online and perform self-guided tasks utilizing their computer.


2. Those individuals assigned to a multimedia-rich, highly interactive online environment were asked to access a similar Web site to complete their in-service training. This group, however, was taught using audio- and video-animated screen captures of programs that they were required to use, short movies with the instructor showing tasks in more detail, online chat groups, and Internet bulletin boards. The instructor was also available via e-mail for specific questions or concerns.

Both sessions were made available for 1 week to allow participants to work at their own pace. In order to compare the success of these sessions to more traditional-based instruction, an identical session was made available to participants in a face-to-face environment.

Objectives

The three main objectives for the study reported here were: (a) to investigate the perceptions of Extension personnel towards learning via distance; (b) to determine if there was a difference in the post-test scores of Extension personnel participating in a minimally interactive online environment, a multimedia-rich, highly interactive online environment, or the more traditional face-to-face environment; and (c) to investigate the evaluations of Extension personnel following various online and comparative face-to-face in-service training opportunities.

A sample of 53 4-H agents, program associates, and secretaries from MSU-ES voluntarily elected to enroll in an in-service 4-H registration system training session. Random assignment resulted in 18 participants being assigned to the minimally interactive online environment, 18 participants to the highly interactive, multi-media rich online environment, and 17 participants to the face-to-face environment. The same instructor was charged with teaching all three sessions.

In order to investigate the perceptions of Extension personnel towards learning via distance, a survey was administered to all participants prior to the start of the in-service training session. This instrument asked the respondent to identify their familiarity with Internet-based instruction, factors that might make them choose one type of instruction over another, and their overall opinion of the Internet-based curriculum.

After the completion of the in-service training session, participants were asked to complete a post-test designed by a committee of instructional experts to measure their knowledge level of the topics that were covered. The test consisted of 10 questions and required the user to answer both basic knowledge level questions of Bloom's taxonomy (Bloom, 1956) as well as higher-level application and synthesis questions. These higher-level questions required the participant to use the knowledge they acquired in the in-service training to accomplish a similar task as they may be required to do on the job.

Feedback regarding participants' reactions to the training program was obtained through the use of an evaluation instrument developed by Sexton (2000). The 12 elements on this instrument asked the participant to rate how strongly they agree or disagree with various statements in regard to the in-service training session. Four of the questions were worded negatively in an effort to reduce the possibility of random responses. Statements covered areas such as ease of learning, amount of feedback, organization, training effectiveness, and level of challenge. Additionally, this questionnaire asked participants if they would like additional training in other areas using the same methodology and if they think that internet-based instruction is as effective as face-to-face learning. Additional space was provided for comments and suggestions.

Perceptions of Extension Personnel Towards Distance Learning

The pre-session survey instrument indicated that a majority of participants (45.3%) recorded a score of four on a scale of one to five when asked about their perceived level of experience using the Internet, indicating a high-perceived level of experience (Table 1). When asked about the number of distance learning courses they have taken in the past, a majority of participants (54.7%) recorded that they had never taken a course before.

Participants were also given four statements and provided an additional write-in option and asked to rank the statements in order of importance based upon what they felt was important to consider when choosing a particular course format, whether that be traditional or by distance. This rank was averaged to obtain an overall level of importance on a scale from one to five. As Table 2 indicates, most participants ranked whether or not it was a required class or program and the location of class as the top two considerations that they use to make such a choice. Other considerations included the ability to find someone to fill-in for the worker, cost, course requirements, subject matter, need for information, and relevance to the job.

The last part of the survey asked participants to respond to an open-ended question: "What are your feelings concerning computer-based instruction?" Responses to the question varied, but three main observations were found to exist. The first observation was that many of the participants had never had an Internet-based instructional course before and were skeptical about its success, with one person noting that they saw Internet-based instruction as a "necessary evil." Other responses included statements such as, "[I] don't think it would be as good as face-to-face instruction" or "I don't think I will like computer-based instruction." Most such observations as these came from participants who felt that a lack of direct contact with an instructor would be a detriment to the learning process.

A second observation made by participants was that online instruction is seen as a necessary "next step" in instructional design and delivery. As one participant stated, "I think it is going to gain more popularity as there are more non-traditional students and those who work, but want to gain training to enhance their job." Similarly, another participant noted, "I want to take courses to improve my job skills and to help me on the yearly performance reviews. I am happy to have the option of taking them online." One interesting view came from a participant who said:

I think that computer-based instruction can be useful if it is developed thoroughly with regard to the student and how the student will perceive and use the material. All too often, computer-based instruction is a last minute (we have no other way of getting the information to them), slapped-together concept that is thrust upon the student. The student is then left to their own devices, and has to develop an understanding of the technology and the subject matter, with technology taking a front seat to the subject matter.

A third observation noted that Internet-based instruction might be a good alternative to traditional face-to-face instruction, especially when the training would require employees to drive great distances. One participant noted that,

With travel funds the way they are, I think that the online [course option] is the way to go, [instead of] having to drive to [Mississippi State University] or to other locations. I do like the teacher in the class with you setting, but online is going to help save money.

Another respondent agreed:

Distance learning will allow me to attend classes that I normally would not be able to attend. This will, in turn, save MSU-ES money. Travel expenses will be reduced, as well as work hours. Being allowed to take the course at my convenience will save work hours. Each office has some spare time each day, therefore, it will not be necessary to schedule a relief person to come into the office and work while I take training.

Indeed, with recent budget cuts, many respondents believed that a highly interactive online environment might be a fiscally beneficial option to deliver necessary training within the Extension Service.

Differences Between Learning Environments

To determine if there was a difference in the post-test scores of Extension personnel participating in the minimally interactive online environment (M. I. Online), the multimedia-rich, highly interactive online environment (M-R Online), or the more traditional face-to-face environment, an analysis of variance (ANOVA) statistical test was performed (McCann, 2006). Results indicated that there was a statistically significant difference in the mean post-test scores of participants between the groups (F = 2.81, p < .10).

Post-hoc Tukey HSD tests were performed to determine the statistically significant differences between individual groups. Results indicated that the M. I. Online environment (M = 84.36) was statistically significantly lower than the face-to-face environment (M = 90.56) and the M-R Online environment (M = 90.12). However, the face-to-face environment and the M-R Online environment were found to have no statistically significant difference (Table 3).

Learning Environment Evaluations of Extension Personnel

To address the third objective, participants were asked to complete a post-session evaluation that attempted to measure their general feelings about the in-service training and the type and quality of instruction they received. This instrument was developed and tested by Sexton (2000) and presented participants with a four-point Likert scale that they could then use to report their level of satisfaction with the training program. The higher the participants' rating, the more pleased they were with the in-service workshop. In an effort to aid the readers' comprehension, some of the questions that were worded negatively were reverse coded before the analysis was performed. The results of the evaluation instrument can be found in Table 4.

Overall, participants in the face-to-face group (M = 4.16, SD = 0.25) reported having a higher level of satisfaction than the participants in the minimally interactive online environment (M = 3.87, SD = 0.55). Additionally, participants in the minimally interactive online environment reported a higher level of satisfaction than participants in the multimedia-rich, highly interactive online environment (M = 3.72, SD = 0.55).

Conclusions and Discussion

While traditional face-to-face instruction is still perceived more favorably than distance learning, Internet-based instruction can be used effectively within the Extension Service. While opinions of learners, many of whom had little exposure to the distance learning format, did include statements describing online instruction as a "necessary evil," the majority were in favor of more courses delivered in this manner.

The findings of the study reported here also indicate that the achievement levels of Extension personnel in an online environment are comparable to more traditional face-to-face instruction. This conclusion is borne up by numerous studies, more specifically those of Neuhauser (2002) and Aragon, Johnson, & Shaik (2002). However, the study found that online instruction should also include a number of highly interactive components. Interestingly, this is the same conclusion that Neuhauser (2002) found in her study of participants enrolled in differing types of online courses. Her qualitative data suggested that highly interactive components such as exercises, activities, animations, and video tended to help engage the student in the learning process, as opposed to more static print and graphic media.

The idea that highly interactive components should be included in online media is not new. A recent study conducted by Roblyer and Wiencke (2004) found that the degree of interaction among participants in distance learning courses is a strong indicator of the overall success of the learning experience. This is because interaction has been found to contribute both the achievement and participant satisfaction.

Indeed, the authors note that although the number of distance learning opportunities continues to expand, the critics of distance learning continue to express concern about the lack of direct interaction between instructors and students found in the more traditional face-to-face environment. Thus, they suggest that providing highly interactive components in the distance learning environment is the key to addressing these concerns and assuring the equivalency of the quality of these courses.

The evaluation scores of the different instructional methods indicated that participants tended to perceive the minimally interactive online environment (M.I. Online) more favorably than the multimedia-rich, highly interactive online environment (M-R Online). This was surprising because the M-R Online environment post-test scores were found to be statistically significantly higher than those of the M.I. Online environment. According to Campbell and Stanley (1963), the randomization procedures that were employed in the study should assure that any differences between the treatment groups should be attributed to chance.

However, the researcher discussed this finding with the instructor of the course, and it is believed this result was achieved because of certain individuals who were assigned to the two groups. The instructor noted that several of the participants in the M-R Online environment were known to have negative attitudes towards online instruction as a whole. Conversely, she noted that a few participants in the M.I. Online environment were known to have been more positive towards online instruction in the past. While this discrepancy is of interest, statistically both online environments were regarded less favorably than the face-to-face instructional environment.

References

Aragon, S. R., Johnson, S. D., & Shaik, N. (2002). The influence of learning style preferences on student success in online versus face-to-face environments. The American Journal of Distance Education, 16, 227-244.

Bloom, B. S. (Ed). (1956). Taxonomy of educational objectives handbook 1: Cognitive domain. New York: Longman, Green & Co.

Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin.

Lippert, R. M., Owen, P., Camberato, J. & Chastain, J. (1998). Regional Extension in-service training via the Internet. Journal of Extension [On-line], 36(1). Available at: http://www.joe.org/joe/1998february/a3.html

McCann, B. M. (2006). The relationship between learning styles, learning environments, and student success. Journal of Agricultural Education, 47(3), 14-23.

Neuhauser, C. (2002). Learning style and effectiveness of online and face-to-face Instruction. The American Journal of Distance Education, 16, 99-113.

Roblyer, M. D., & Wiencke, W. (2004). Exploring the interaction equation: Validating a rubric to assess and encourage interaction in distance courses. Journal of Asynchronous Learning Networks, 8(4) 24-37.

Rogers, C. R. (1998). A theoretical look at electronic community's conversation and curriculum. ERIC Document ED430551.

Sexton, J. S. (2000). A comparison of traditional and world wide web methodologies, computer anxiety and higher order thinking skills in the in-service training of Mississippi 4-H Extension agents. Unpublished doctoral dissertation, Mississippi State University.