Assessment of NetMeeting® for Professional Development Programming

Introduction

University Extension programs face significant financial constraints. Extension programs have two alternatives: decrease costs or increase revenue. This article addresses one means by which to decrease costs. Though there are many activities evaluated to determine future resource allocation, we focus on train-the-trainer activities and the economic assessment of three alternative delivery modes.

During the fall of 2001 we delivered a train-the-trainer educational program (Professional Implementation Experience in our system) to a group of 11 University of Missouri Outreach and Extension field faculty. The topic of the program was "Market Feasibility Analysis: Key Components." As persons who embrace technology and understand "economic costs," we took a leap forward and used NetMeeting® in conjunction with teleconferences to deliver the program. What we report here is a comparative summary of the economic and time costs of using NetMeeting® versus two alternative modes of delivery available. Additionally, we provide a restrictive qualitative assessment of NetMeeting® as a mode of program delivery.

Background

For this particular program, three alternative modes of delivery were available: NetMeeting®, on-campus training or training through interactive television. We realize more alternatives exist; however, we focus on these three because they are the most relevant for the train-the-trainer programming. Following is a brief description of each delivery mode alternative.

NetMeeting®

NetMeeting® is a Microsoft Windows® product that enables meeting participants and the instructor to exchange files, view presentations, and chat through the Internet simultaneously. Several modes of Internet delivery exist; however, we chose NetMeeting® due to convenience and cost. The instructors and all participants use their office computers to take part in the program. In essence, no one moves from his or her offices and chairs. NetMeeting® is free to Windows® users. Specifically, NetMeeting® allows for the instructor to show a presentation and the participants to view the instructors' presentation in real time. Allowing for real-time transmission of presentations, the instructor controls the pace with which participants browse the presentation. Though NetMeeting® has Internet audio capabilities, we chose to use teleconferencing because phones offer a more reliable communication medium and some participants did not have access to microphones. For this mode of delivery, we chose a one and one-half hour learning period on four separate days.

Face-to-Face via On-Campus

This educational delivery medium involves University Regional Extension Faculty attending a 1-day, 6-hour program in Columbia, Missouri. An on-campus program involves arriving in Columbia by 9 a.m. and departing Columbia around 4 p.m. A minimum of lunch is provided to participants. For this analysis, we assume all 11 persons travel independently, which is typical given that persons attending such train-the-trainer programs are coming from geographically dispersed areas. Also, we assume all participants travel to Columbia the morning of the program. In reality, two to three persons traveled the night before to attend a 9 a.m. program.

Interactive Video (ITV) via Video-Conferencing to Outreach Centers

This delivery mode involves the transfer of video and voice via video conferencing originating from one of 14 University of Missouri Outreach & Extension TeleCenter Network sites located throughout the state. These technologies are housed in multi-purpose instructional facilities. This technology allows for broadcasting of real-time programming, allowing presenters and participants to communicate in real time. Typically, University of Missouri Regional Extension faculty travel to the location nearest their home county. Furthermore, program delivery is limited to 3-hour blocks. For economic analysis, we assume participants need to make two separate trips to obtain 6 total hours of educational program delivery. We assume all participants travel separately to the nearest ITV location.

Economic Assessment of Three Program Delivery Modes

As discussed in the Background section, there are three potential modes of program delivery that are applicable for this train-the-trainer curriculum. For each of the 11 participants of this program, we compute the monetary cost to University of Missouri Outreach and Extension and the time cost (opportunity cost) to participants for equivalent type programs. Participant cost is aggregated by delivery mode. Table 1 summarizes these costs for the three alternative program delivery modes. Total "class" time is 6 hours. In summary (for the Missouri structure), this is one 6-hour program on the Columbia campus, two 3-hour programs through the interactive television network (ITV), or four 1.5-hour programs via NetMeeting®.

The "time travel costs" outlined in the table clearly indicate a savings by using NetMeeting®. For a similar on-campus program, travel time requires a total of one and one-half 40-hour weeks of work time forgone. For a similar ITV-based program, travel time (assuming no carpooling) requires a total of one 40-hour week of work time forgone. NetMeeting®, of course, has zero travel time costs.

The economic advantages of using NetMeeting® for program delivery are large. The economic costs associated with program delivery include mileage (assuming no carpooling), wages paid for travel time, food expenses, and teleconferencing. The only cost for NetMeeting® is the teleconferencing charge. We do not include the 6 hours of programming (labor hours) in our economic costs because the costs are the same for each mode of delivery. Total economic costs for an on-campus program were $2,200 over the use of NetMeeting®. Total economic costs for ITV based programming were $1,200 over the use of NetMeeting®.

Qualitative Assessment of Program Delivered Using Net Meeting®

No base group exists to allow for comparing the qualitative impact of the program among methods of delivery. We, however, use an outcome-based method for evaluation of professional development programming. This entails asking participants to rank their understanding of activities to be addressed in the training through a pre-program survey. Similarly, upon completion of the professional development program, all participants rank their understanding of the activities covered in the program. Questions posed in the pre- and post-program surveys are identical. Both surveys are administered via the Internet, and all responses were kept confidential.

Though the sample size is small, the sample represents those persons who took part in all 4 days of professional development experience, i.e., the population. Table 2 summarizes the average response to the before and after survey for five separate activities covered in the program. The final column indicates the percent change in understanding from before to after completing the program. Prior to the program, participants indicate a below-average to average understanding of a specific activity. Following the program, participants indicate an above-average understanding of activities discussed. While the semantics of setting a "target level" of understanding could be discussed, we focus only on participant feedback indicating that NetMeeting® can be an effective method of program delivery.

Obstacles to Overcome and Final Thoughts

We conclude with some general observations on using NetMeeting® for professional development program delivery. First, we agree openly that nothing beats face-to-face interaction. Second, because participants do not leave their office, it is easy for them to become distracted. Third, because county offices do not always have individual offices, background visiting can be a problem. Fourth, the technology, at this time, requires Internet access and a phone line. Thus, offices where the phone line serves a dual purpose of telephone and Internet NetMeeting® will not work. Last, assessing the amount of content to cover is tricky and likely will need adjusting according to the pace at which participants acquire knowledge.

As Extension moves from a "bonding" mode to a "bridging" mode of information delivery, further economic and qualitative assessment of delivery modes for professional development activities should be conducted.