Using Technology to Link Researchers and Educators: Evaluation of Electronic Conferencing

Introduction

Extension Educators' effectiveness in the field is enhanced by the desire and practice of keeping current in the research related to their work. Many, if not most, rely on conference attendance to learn first-hand from researchers in their field. This, however, is often quite costly, and exchange with the researcher is usually in the form of a didactic presentation, rather than dialogue and discussion.

With this in mind, Ohio State University Extension, in partnership with Alabama Cooperative Extension at Auburn University, conducted a 5-day electronic conference (e-conference) in May 2002. Our aim was to connect educators in the field with top researchers in a selected area of study in direct dialogue to promote up-to-date knowledge, discussion of issues, and agreement on practical application.

Description

The 2002 Family Life E-Conference, Couple Relationships: Research and Extension Programming, included participants from several states (Ohio, Alabama, New York, Oklahoma, Kansas, and Indiana). A panel of experts in couple relationships was invited to participate and provide papers on the latest research and programming available in this field. The panelists included some of the most recognized researchers in the field of couple relationships research and two State Specialists who have focused on programming for couples. (See panelist descriptions at http://hec.osu.edu/couples for more information.)

Participants registered a month in advance and were able to log on to access the papers and presentations 2 weeks before the scheduled e-conference. During the e-conference, participants and panelists could log on at any point during each day and use a bulletin board to "converse" via computer.

Implementation

Approximately 200 hours were spent planning, developing, and implementing the e-conference by two State Specialists, two graduate students, and two technical support staff persons. About 20% of the time (35-40 hrs) was spent in planning (e.g., discussions with colleagues; identifying themes, topic areas of interest, and scholars; recruiting scholars to participate; publicizing). Nearly 45% of the time was spent developing the public Web site (20 hrs), preparing the on-line materials (40 hrs), and designing the course site using Web Course Tools (30 hrs). Ten percent of the time (20 hrs.) was spent on various logistical issues (e.g., registration, technical support). Each day of the e-conference, approximately 5 hours were spent by each State Specialist reading and responding to messages posted on the bulletin board and facilitating the discussion (total 50 hrs).

Participants

Overall, 44 Family and Consumer Science (FCS) Educators participated in the e-conference. Their years of Extension experience ranged from 1 to 33 (Mean (M) = 17.8), and 71.4% identified their primary area or specialization as family life.

Comfort with the Mode of Delivery

Due to the nature of the delivery mode, participants were asked on their registration survey to describe their access to computers, general level of electronic activity, and comfort with using email and the Internet. The participants appeared to be regular and comfortable users of electronic communication technology.

• 97.7% reported that they had their own computer with modem capability.

• 100% reported that they often/very often used email and were comfortable/very comfortable with it.

• 83.7% often/very often used the Internet, and most were comfortable/very comfortable "surfing" (93.0%), finding materials/information on (80.9%), and downloading/printing items from (93.0%) the Internet.

• 42.6% reported that they were comfortable/very comfortable with on-line discussion boards.

Observed Participation Level

WebCT provides tools that assist in tracking the extent to which the participants were involved in the e-conference. These data showed that about 55% of the participants logged on to participate (e.g., read/post messages) at least 3 of the 5 days during the e-conference. During the week, a total of 157 messages were posted on the discussion board (44.6% by participants). Although many participants (65%) did not post messages, the majority were actively involved in reading the messages posted: 55% of the participants read more than half of the posted messages (M = 92.14; Standard Deviation (SD) = 75.0).

Evaluation

Immediately following the e-conference, 30 FCS Educators completed an on-line evaluation (68.2% of those FCS Educators who participated). T-test group comparisons revealed that, compared to those who did not respond, those who did were more comfortable with email, logged on more days, and read more of the posted messages. Thus, the findings presented below reflect the opinions of those who were more active in the e-conference.

Level of Preparation

Table 1 illustrates how prepared the participants who responded to the survey reported they were for the e-conference.

Table 1.
Proportion of Participants Who Printed, Viewed, and/or Read E-Conference Materials

	Number of Materials					Mean (SD)
	0	1 - 2	3 - 5	6 - 7	8 - 9
Papers Printed or Viewed	0.0%	0.0%	13.3%	86.6%	NA	6.4 (1.2)
Papers Read	16.7%	6.7%	26.6%	50.0%	NA	4.5 (2.6)
Slide Shows Viewed	33.3%	6.7%	6.6%	6.6%	46.7%	4.5 (3.7)
Handouts Viewed	20.0%	10.0%	13.4%	3.3%	53.3%	5.6 (3.8)

Time Spent Participating

Participants also were asked to give a rough estimate of the amount of time they spent on this e-conference each day. Participants reported that they spent, on average, 9 hours obtaining and reading materials and 6.9 hours reading the messages posted on the bulletin board (M = 1.8 hrs/day). When asked how satisfied they were with the time they spent participating, nearly 70.0% wished that they could have spent more time participating (30.0% were satisfied).

Comfort with E-Conferencing

As described earlier, many participants indicated on their registration pre-survey that they were comfortable with technology. After the e-conference, participants were asked again how comfortable they were using technology prior to participating in the e-conference. A comparison of the participants' pre-survey and post-pre-survey responses showed that their responses were consistent, with one exception.

The single exception was that, on average, comfort scores for participating on an on-line discussion board were higher on the pre-survey (M = 2.4) than on the post-pre-survey (M = 2.0), t(27) = 2.6, p < .01. In other words, of the 30 survey respondents, 46.7% reported on the pre-survey that they were comfortable/very comfortable participating in an on-line discussion board compared to only 20% who said the same on the post-pre-survey. This may be due to participants not realizing their actual skill level with the bulletin board until they attempted to use it during the e-conference.

After the e-conference, 60% of participants reported that they were comfortable/very comfortable with the on-line discussion board (M = 2.7) and the increase was statistically significant from the post pre-survey, t(29) = -6.2, p < .001. Thus, the e-conference experience helped participants feel more comfortable with the on-line learning environment.

Usefulness of an E-Conference

• On a 4-point scale ranging from (1) not at all useful to (4) very useful, most (86.6%) of the participants indicated that the interactive discussion with the panel of experts was useful/very useful (M = 3.3), and 83.4% felt that the interactive discussion with other participants was useful/very useful (M = 3.0). Most important, no one rated the discussions as not at all useful.

• On a 5-point scale, (1) indicating strong disagreement and (5) indicating strong agreement, nearly all of the participants reported that they agreed/strongly agreed that the e-conference:

  • Helped them better understand the issues surrounding couple relationships (86.7%; M = 4.1),
  • Provided them with new ideas for working with couples (80.0%; M = 3.9), and
  • Stimulated them in wanting to learn more about couples (86.6%; M = 4.2).

• Significantly, 90% of the participants agreed/strongly agreed that they will likely use the information learned during the e-conference (M = 4.1).

• A 1-year follow-up evaluation is planned in order to document the ways and the extent to which the information obtained in the e-conference is used in practice.

Satisfaction with the E-Conference

Table 2 provides information on the participants' level of agreement with statements regarding their satisfaction with elements of the e-conference.

Table 2.
Participants' Satisfaction with the E-Conference

Statement	Mean	Agree or Strongly Agree	Neither Agree nor Disagree	Disagree or Strongly Disagree
Easy accessing, viewing, and printing the papers/presentation materials	4.4	90.0%	6.7%	3.3%
No problem reading/posting messages on the discussion board	3.7	63.3%	30.0%	6.6%
Easy to understand papers/presentations	3.8	83.3%	6.7%	10.0%
More effective if taught in face-to-face format	2.8	33.3%	16.7%	50.0%
Satisfied with the overall quality of the presentation	4.3	90.0%	3.3%	6.7%
Satisfied with the quality of the discussion	4.0	80.0%	13.3%	6.7%
As a whole, enjoyed participating in this e-conference	4.0	80.0%	13.3%	6.6%
Want more trainings in this format	4.0	76.7%	16.7%	6.7%
Would participate in this type of training again	4.3	90.0%	6.7%	3.3%

In a qualitative section of the evaluation, participants were asked to briefly describe what they liked and disliked about the e-conference.

• Many participants commented that they liked the flexibility, the convenience of the delivery method, and not having to travel.

• Positive comments also were made in reference to the discussion, panel of experts, user friendliness, affordability, and accessibility to colleagues.

• Participants mentioned enjoying the reading materials and being able to keep them for future reference.

• Many noted that, although they did not post questions/comments, they gained much from reading the postings of panelists and other participants.

• While the participants mentioned few things they disliked, some expressed dilemmas in managing time for reading the materials and participating in the discussion along with their other responsibilities.

• A few reported concerns related to length of time needed to read the materials, technical problems, becoming familiar with the format, and the amount of research-focused information that was shared.

Cost Comparison with Face-to-Face In-Service

One of the advantages of using e-conferencing as a mode of professional development is that it is cost-effective. We estimated that the average per participant travel costs would be $95 (meals/mileage). The total expense related to bringing in the panel of 6 experts would amount to $3,720: (1) travel (average flight $300; total $1,800); (2) lodging (average hotel rate $85; total $510); (3) meals (average $35; total $210); and (4) honoraria ($200 each; total $1,200)]. Assuming at least 50 participants register, the per-participant costs would be an additional $74.40. Hence, in-state participants' expenses for an equivalent 1-day face-to-face would have accrued to roughly $170 versus the $25 registration participants paid for the on-line e-conference. (Note: The $25 registration fees covered panelists' honoraria and technology fees.) Travel and lodging expenses would be much greater for out-of-state participants.

Summary and Recommendations

In summary:

• On average, participants spent 16 hours involved in this five-day e-conference.

• The majority of participants were involved in reading the discussions; about a third were the primary participants in the dialogue with the panelists.

• While most participants liked the flexibility and convenience of an e-conference, some participants reported that they struggled to manage their time with competing work demands.

• Nearly all of the participants reported that the e-conference was useful, that they better understood couple relations, and had new ideas for working with couples.

• Most participants indicated that they would participate again.

• This format represents a substantial per-participant savings in cost compared to an equivalent 1-day face-to-face conference.

Through information provided in the evaluation, we are incorporating the following changes/recommendations into the plan for future e-conferences and offer the following to others considering conducting an e-conference.

Many of the readings provided by the panelists were quite lengthy and technical. For future events, panelists will be asked to provide a brief Executive Summary of their topic with a clear "issues and implications for practice" section as the main reading.

The preparation before the conference start date was minimal. In addition to the length of the readings, which may have been a deterrent, we theorized that this may also be due to the term "conference," which tends to be associated with receptive learning, rather than prepared exchange. We made the decision to title future events, "E-Seminars." We believe this implies preparation for exchange. We will also be explicit in our notices that preparation is key to successful use of the e-seminar and will provide the reading materials at least 2 (versus 2) weeks in advance of the discussion start date.

Many participants indicated that they were "very busy and had trouble scheduling time to participate" and wished they had planned for more time to participate. Therefore, we will explicitly suggest to participants that they schedule time each day for participation, rather than relying on "leftover" time in their day.

For greater comfort in the exchange:

• We will encourage researchers to use less technical language in their writing and in their on-line communications.

• We will encourage Educators to raise their comfort level with research language by asking questions about terminology or practices that they do not understand and by reading about research methodology either through formal coursework or informal professional development.

• We will also ask panelists to post questions to further stimulate discussions rather than solely rely on the Educators to initiate dialogue.

• We also anticipate that repeat participation in e-seminars with researchers will enhance educators' comfort level with direct dialogue with researchers.

Conclusion

Several studies report success among Extension Educators in the use of Web-based in-service training (Dooley, Van Laanen, & Fletcher, 1999; Kelsey & Mincemoyer, 2001; Lippert, Plank, Camberto & Chastain, 1998; Muske, Goetting, Vukonich, 2001). This study, however, is the first to document the response to Web-based conferencing that directly connects researchers and educators.

The 2002 Family Life E-Conference on Couple Relationships was favorably received among Extension Educators, suggesting that e-conferences/seminars can serve as a useful, cost-efficient channel for promoting dialogue between researchers and Extension Educators, promoting professional development and positively affecting practice. The recommendations for changes should increase both participation levels and effectiveness of this method of Internet-based information exchange.

For more information and a complete report of the evaluation findings, go to http://hec.osu.edu/couples and, specifically, "Evaluation Results."

References

Dooley, K., Van Laanen, P. G., & Fletcher, R. (1999). Food safety instructor training using distance education. Journal of Extension [On-line], 37(3). Available at: http://www.joe.org/joe/1999june/a5.html

Kelsey, T., & Mincemoyer, C. (2001). Exploring the potential of in-service training through distance education. Journal of Extension [On-line], 39(2). Available at: http://www.joe.org/joe/2001april/rb7.html

Lippert, R., Plank, O., Camberto, J., & Chastain, J. (1998). Regional Extension in-service training via the internet. Journal of Extension [On-line], 36(1). Available at: http://www.joe.org/joe/1998february/a3.html

Muske, G., Goetting, M., & Vukonich, M. (2001). The world wide web: A training tool for family resource management educators. Journal of Extension [On-line], 39(4). Available at: http://www.joe.org/joe/2001august/a3.html