Do Workshops Work?

Evaluating an "old-fashioned" Extension delivery method like workshops may seem a bit backward in this era of rapidly expanding electronic technology. But despite miracle media, Extension faculty and field staff will organize and participate in thousands of workshops this year. The topical public meeting where technical experts interact with lay people has proven to be an enduring feature of Extension programs.

As our options for program delivery expand, it becomes more difficult to make decisions about which to use. Our program objectives and audience characteristics are important considerations, but effectiveness looms as an increasingly significant factor. This study grew out of our desire to determine the effectiveness of workshops in meeting our objectives for one component of the Cornell Cooperative Extension Woodland Management Program. Basically, we asked: "Do workshops work?"

Since the mid 1970s, Cornell Cooperative Extension has offered educational programs to meet the needs of the 500,000 owners of 18.5 million acres of private forestland in New York, with a variety of formats used. We examined the effectiveness of a series of four workshops dealing with multiple-use woodland management information and related decision-making skills. The workshops were held in 1983 and targeted at an audience of absentee woodland owners living in the Buffalo area of western New York.

The principal goal of the workshops was to stimulate woodland management (timber stand improvement, wildlife enhancement, fuel-wood production) by attendees. The primary purpose of our study was to evaluate the success of the workshops in meeting this goal. We will focus on general workshop effectiveness rather than on descriptive data about particular workshops and specific woodland management activities.1

We designed three survey instruments for this evaluation. The first assessed the prior knowledge and activity levels of attendees with respect to woodland management. The second and third instruments assessed the effectiveness of the workshops in stimulating short-term and long-term interest in woodland management activities.

A self-administered questionnaire was completed by attendees before each workshop (T1). Participants from 108 of the 114 households attending the workshops responded. At the end of each workshop (T2), a questionnaire was completed by all attendees. During Spring 1986, 2 1/2 years after the workshops (T3), a follow-up telephone survey was conducted. We interviewed 81 people over the phone who responded to earlier surveys.

To help us analyze data, we developed three typologies or categories of people based on multiple characteristics to evaluate: (1) adoption of practices before attending the workshops, (2) any changes in their intent to accomplish various woodland management practices, and (3) their level of practice adoption resulting from the workshops.

Past Involvement Typology

This typology categorizes attendees based on their level of involvement before attending the workshops. It combines indices of prior knowledge levels and past actions (weighted more heavily), resulting in three qualitative levels: low, medium, high.

Intention-Change Typology

This reflects the degree to which the workshops stimulated decisions on woodland management, as indicated by changes in management intentions. Three intention levels were created: low (0 activities), medium (1 to 3), high (4 or more). Intention levels of attendees were rated at each of the three survey times (T1, T2, and T3) to determine changes attributable to the workshop.

Practice-Adoption Typology

This typology indicates the degree to which the workshops stimulated practice-adoption. Attendees were asked what practices they planned to implement as a result of the workshops (T2) and, 2 1/2 years later, what decisions or actions they'd taken toward doing these (and other) practices (T3). A three-part model of practice-adoption (Table 1) was developed to assess progress for each intended practice. We also identified the possible outcomes for each of the three stages in the model. Because attendees typically had intentions to engage in more than one activity, we derived an "average" stage of adoption for each attendee.

Workshops often include a visit to a woodlot.

The workshops had an immediate influence on the intentions of two-thirds of attendees to engage in management-related activities. People tended to raise their aspirations for carrying out woodland management activities. The following trends were observed in measurements of pre-workshop (T1) and post-workshop (T2) management intentions:

1. Three-fourths of those with a "low" level of intention to manage before participation expressed "medium" or "high" levels of intention afterward.
2. Over half of those with a "medium" level of intention before the workshop expressed a "high" level afterward.
3. Four-fifths of those with a "high" level of intention before participation maintained that level.

Most attendees who diminished their intentions did so because they learned that their original aspirations weren't realistic. On average, attendees changed their intentions about doing two to three woodland management activities.

We suspected that people's intentions expressed at the end of a workshop might be exaggerated because of the potential for overestimation of the amount of time one's willing to devote to such activities. We were in for a pleasant surprise. The reported levels of completed activities 2 1/2 years after the workshop were greater than attendees had anticipated at its end.

Most attendees (82%) who left the workshops with a high level of intentions performed at their anticipated level. All managed to accomplish at least one activity. Among those with a medium level of intended activity, two-fifths actually followed through at a high level. Furthermore, nearly two-thirds of those with no intentions to engage in management at the close of the workshop actually carried out four or more practices or activities and an additional one-fourth carried out one to three. And, these people attributed their decisions directly to the workshop, thus revealing a delayed impact that normally wouldn't have been identified without the long-term follow-up.

Although accomplishment of woodland management activities that attendees attribute to their participation is a direct indication of workshop impact, it's not the only one we were interested in assessing. From the outset, we maintained that the goal of the workshop was to facilitate informed decision making by woodland owners to stimulate their progress in the practice-adoption process. Consequently, workshop attendees who expressed an intention to undertake woodland management following their participation were evaluated in terms of their progress toward making an informed decision regarding management. Because many attendees anticipated involvement in more than one activity, we created a measure of average stage of decision making about their adoption of woodland management practices.

Using the model described in Table 1, the scale for stages in the woodland management adoption process ranged from 0 to 2, where 0 = inaction, 1 = predecision/action, and 2 = decision/action. The average score for workshop attendees who intended to engage in woodland management activities was 1.6. Thus, 2 1/2 years after attending the workshops, attendees were, on the average, making informed decisions about management and use of their woodlands. Workshop attendees with a high level of prior involvement had a slightly higher stage score (1.8) than those with low or medium levels before involvement (1.5). This may indicate a tendency for more progress toward making practice-adoption or rejection decisions by those significantly interested and experienced in woodland management before participation.

Workshops proved to be effective in improving woodland owners' capabilities to make informed decisions about various woodland management practices. Knowledge gained during the workshop typically was important in changing people's intentions about various woodland management practices, resulting in an overall stimulation of management activities during the following 2 1/2 years.

Although this study supported the continued use of the workshops, it left some important questions unanswered from the standpoint of Extension program development. For example, the cost-effectiveness of workshop wasn't determined, nor were workshops compared with other methods of program delivery. In this case, considerable cost in both time and money went into reaching100+ woodland owners, using the workshop format. Whether similar results in both decision-making capabilities and on-site management practices are possible at less cost with other methods needs to be assessed.

Finally, this study clearly demonstrated the value of a long-term follow-up when evaluating the impact of an Extension program aimed at behavioral change or practice adoption. The impact of the workshops on management practices was actually greater than attendees had indicated at the close of the workshop. This is probably the opposite of what most of us would have predicted. Because of the scarcity of long-term evaluations, however, we don't know whether this is typical.

Despite the unanswered questions about cost-effectiveness, we feel the "old-fashioned" workshop is a valuable delivery method for woodland management education. Workshops provide opportunities for information transfer and spontaneous personal communication between landowners. We view workshops as one element in a set of learning experiences for landowners, which range from direct-mail fact sheets to an indepth, home-study course. Based on our formal evaluation and personal observations, workshops are - and probably will remain - an important part of our program delivery strategy for woodland management education.

1. For a discussion of the effectiveness of particular workshop formats and subsequent, specific woodland management activities of landowners, see: J. P. Lassoie and others, "Effects of Workshops on Absentee Landowners' Woodland Management Decisions and Actions," in Proceedings of Northeastern Area Cooperative Forest Management Supervisors/Extension Foresters Conference (Durham, New Hampshire: USDA Forest Service, 1987), pp. 34-49.