Needs Assessment Surveys: Do They Predict Attendance at Continuing Education Workshops?

Introduction

Extension educators regularly conduct needs assessment surveys to develop programs to meet their clients' education needs (Ahmann, 1979; Gilmore, 1989). These surveys are particularly prevalent in continuing education (CE) programs, where rapidly expanding knowledge, technical innovations, and public demand for professional competence require specialists to constantly increase their skills and knowledge (Queeney, 1995).

Advocates of needs assessment surveys urge educators to use these surveys as "a decision-making tool for...identifying the educational activities or programs they should offer to best meet their clients'--and society's--education needs" (Queeney, 1995, 1). These surveys are used as tools to analyze program feasibility, in essence, serving as quasi-referendums on potential programs. The premise is that if a survey is conducted properly, survey respondents will participate in program activities (e.g., Witkin and Altschuld, 1995; Reviere, Berkowitz, Carter, & Ferguson, 1996). However, a review of the literature revealed that no one has tested the theory that survey respondents' preferences can be used to predict those respondents' attendance at CE programs.

The study reported here compared CE needs assessment survey records with CE program attendance records to learn if respondents attended programs that they indicated a preference for. Specifically, we wanted to answer two questions about needs assessment survey respondents:

• Did respondents who indicated an interest in a workshop topic attend a workshop on that topic?
• Did respondents' workshop attendance vary based on their level of interest in a workshop topic?

Methods

The study analyzed data from the SUNY College of Environmental Science and Forestry, Faculty of Forest and Natural Resource Management's (FFNRM) CE program for New York State (NYS) forest resource managers. We matched questionnaires from a CE needs assessment survey of New York State forest resource managers with attendance records from a series of workshops on three topics.

The CE Needs Assessment Survey

Forest resource management CE educators regularly employ needs assessment surveys to learn their clients' CE preferences (Bullard & Straka, 1986; Donovan, Anderson, Wheatcraft, & Carpenter, 1989; Fischer & O'Leary, 1987; Lilley, 1987; Straka & Richards, 1984, 1985). In fact, published studies under represent the number of these surveys because many CE programs publish only internal reports (e.g., Oregon State University Administrative Committee, 1998; Siscock, Rechenbach, & Finley, 1995). Using these studies as models, we surveyed a census of practicing NYS forest resource managers to determine their CE preferences in August 1996 (Malmsheimer & Floyd, 1996).

We employed Dillman's Total Survey Design Method (Dillman, 1978). Each member of the population was sent an initial mailing consisting of an introductory cover letter, a questionnaire, and a postage paid self-addressed stamped envelope. Ten days later, we sent a reminder letter to all non-respondents. Two weeks later, all non-respondents received a different reminder letter, another copy of the questionnaire, and another postage paid self-addressed stamped envelope. The questionnaire collected information on workshop topic and logistic preferences, and respondents' characteristics. Each questionnaire was uniquely numbered so that we could match returned questionnaires to specific survey respondents.

The study examined the responses to the two parts of the questionnaire that requested respondents' workshop topic preferences. In Part I, respondents indicated any of 145 workshop topics that interested them. For this study, we designated these responses as "general interest" workshop preferences.

In Part II of the questionnaire, respondents chose the top five workshop topics (of the 145 workshop topics listed), that were "of greatest interest or need to you or your organization." We designated these five responses as "strong interest" workshop preferences. When a respondent indicated both a "general interest" and a "strong interest," we coded the response only as a "strong response." This prevented us from double counting respondents' preferences and allowed us to analyze whether interest level affected workshop attendance.

We mailed 507 questionnaires. Ten questionnaires were undeliverable, and 369 were received (74.2% response rate). Of the received questionnaires, 27 respondents (7.3% of respondents) indicated that they were not interested in continuing education. The study matched the remaining 342 respondents' questionnaires with our workshop attendance records.

The Workshops

In accordance with needs assessment survey advocates' suggestions, we used the survey's responses to choose CE workshop topics. From May 1998 to May 2000, the FFNRM presented 19 CE workshops covering nine topics.

The study analyzed the attendance records from 13 of the 19 workshops. These 13 workshops addressed three topics:

• Best Management Practices (BMP workshops),
• Occupational Health and Safety Agency Hazard Tree Regulations (OSHA workshops), and
• Silviculture (Silviculture workshops).

We limited our analysis to these workshops because the five BMP, five OSHA, and three Silviculture workshops were presented at locations throughout New York State, consistent with the state-wide coverage of the our needs assessment survey. The workshops on other topics were only presented in one part of the state.

Analysis

We used the unique number on each questionnaire to determine individual respondent's names. We then examined workshop attendance records to find survey respondents who attended workshops. We reviewed each one of these respondent's workshop preferences to determine whether that respondent attended a BMP, OSHA, and/or Silviculture workshop.

We conducted descriptive and inferential analysis of this data with SPSS 8.0 (1997). Chi-square tests and standardized residuals were calculated by cross-referencing the data.

Results

The workshops covering the three topics were well attended. Survey respondents represented 19%, 23%, and 32% of BMP, OSHA, and Silviculture workshop attendees, respectively.

More than 10% of survey respondents who indicated an interest (general or strong) in a workshop topic attended that workshop, with more than 30 (17%) of the respondents interested in a Silviculture workshop attending such a workshop (Table 1). Survey respondents with a strong interest in a workshop topic exhibited a higher rate of attendance than those with a general interest in the topic, with more than one in five respondents strongly interested in BMP and Silviculture workshop attending such a workshop.

Table 1.
Survey Respondents Who Indicated an Interest in a Workshop Topic and Attended a Workshop on That Topic

Interest Level	Workshop Topics	Number of Respondents Interested in Workshop Topic	Number of Respondents Interested in Workshop Topic Who Attended Workshop
General Interest	BMP	160	11 (6.9%)
	OSHA	113	16 (14.2%)
	Silviculture	141	20 (14.2%)
Strong Interest	BMP	53	12 (22.6%)
	OSHA	13	2 (15.4%)
	Silviculture	41	11 (26.8%)
TOTAL (General plus Strong Interest)	BMP	213	23 (10.8%)
	OSHA	126	18 (14.3%)
	Silviculture	182	31 (17.0%)

We also wanted to learn whether the survey respondents that attended workshops had indicated an interest in the workshop on their questionnaire. More than half of the respondents who attended the workshops indicated an interest in that workshop topic in their survey, with more than 82% of respondents at the BMP workshops doing so (Table 2). This indicated that while many respondents who attended a workshop indicated an interest in the workshop when they completed their questionnaire, a large number of workshop attendees did not.

Results for specific workshop topics varied. Most respondents at the OSHA and Silviculture workshops had a general, rather than a strong, interest in these workshop topics. Conversely, more than one-half of survey respondents at the BMP workshops had indicated a strong interest in that topic.

Table 2.
Workshop Preferences of Survey Respondents Who Attended a Workshop

Interest Level	Workshop Topics	Number of Respondents Who Attended Workshop	Number of Respondents Who Attended Workshop and Indicated Interest in That Workshop Topic
General Interest	BMP	28	11 (39.3%)
	OSHA	32	16 (50.0%)
	Silviculture	47	20 (42.6%)
Strong Interest	BMP	28	12 (42.9%)
	OSHA	32	2 (6.3%)
	Silviculture	47	11 (23.3%)
TOTAL (General plus Strong Interest)	BMP	28	23 (82.1%)
	OSHA	32	18 (56.3%)
	Silviculture	47	31 (66.0%)

Inferential Analysis

Pearson Chi-square tests that compared respondents' interest in workshops were significantly associated with attendance at some of those workshops (Table 3). Respondents who indicated some kind of interest (general or strong) in two of the three workshops (BMP and OSHA) were more likely to attend the workshop than survey respondents who indicated no interest in these topics.

Table 3.
Chi-Square and Phi Values for Survey Respondents' Workshop Attendance

Interest Level	Workshop	Pearson Chi-Square Value
General Interest	BMP	0.689
	OSHA	4.589*
	Silviculture	0.039
Strong Interest	BMP	-- a
	OSHA	-- a
	Silviculture	6.730*
TOTAL (General plus Strong Interest)	BMP	5.121*
	OSHA	5.715*
	Silviculture	3.553
* Significant at .05 a One cell (25%) had an expected frequency less than five and therefore could not be analyzed.

An analysis based on survey respondents' level of interest revealed that general interest in one workshop (OSHA) and strong interest in another workshop (Silviculture) were associated with attendance at those workshops. For example, analysis of standardized residuals revealed that respondents with a strong interest in a Silviculture workshop were 2.3 times more likely to attend an Silviculture workshop. However, low expected Chi-square cell frequencies prevented us from analyzing the results for respondents with a strong interest in the BMP and OSHA workshops. An analysis of Phi values for all the statistically significant analyses indicated that the relationship between interest and attendance was weak, explaining only 11% to 14% of the variability.

Discussion

The results suggest that CE needs assessment survey respondents' workshop preferences do sometimes match their stated behavior. However, the relationship between stated preferences and attendance was weak. An examination of our research questions provides some important insights for CE educators who rely on needs assessment surveys to assess CE program feasibility.

Did Survey Respondents Who Indicated an Interest in a Workshop Topic Attend a Workshop on That Topic?

Survey respondents who indicated an interest in a workshop topic attended some workshops more often than survey respondents with no interest did. Although the inferential analysis validated this statement, our results were tenuous. Our results suggest that, at best, 17% of survey respondents who indicate some type of interest (strong or general) in a CE workshop will attend the workshop.

There are many reasons why many survey respondents may not have attended our workshops. Some relate to circumstances beyond CE educators' control, such as family obligations, illness, or that respondent may have moved out of New York State. Other reasons may relate to the workshops themselves. Three of these are particularly relevant for understanding the limitations of our analysis.

• First, all of our workshops were full-day workshops offered on weekdays. Some interested respondents' employers or work responsibilities may have prevented interested respondents from attending our workshops.
• Second, our workshops may have contained none, or only a small portion, of the material that some respondents who indicated an interest in a workshop topic were interested in learning about.
• Third, there was nearly a 2-year delay between when the survey was administered and when the workshops were first offered. During that time, some interested survey respondents may have satisfied their CE needs through other sources, such as other universities' CE programs or correspondence courses, or some respondents may have simply changed their CE priorities.

Our analysis of the workshop preferences of survey respondents who attended a workshop (Table 2) revealed that CE administrators need to remain cognitive of CE needs assessment survey's "shelf life." The discrepancy between survey respondents' interest in OSHA workshops and participation in OSHA workshops illustrates why CE administrators should not rely solely on needs assessment surveys to determine CE programing. Respondents in our survey ranked an OSHA workshop the 21^st (of 145) most popular workshop topic. However after the survey was completed, OSHA began to enforce its comprehensive logging regulations. OSHA's initiative stimulated forest resource managers' interest in (and attendance at) workshops on the topic. This suggests that external factors can limit the stability of needs assessment surveys' results and illustrates why CE administrators should complement needs assessment surveys with other techniques to continually evaluate their clients' educational needs.

Did Survey Respondents' Workshop Attendance Vary Based on Their Level of Interest in a Workshop Topic?

Our CE needs assessment survey asked respondents to distinguish their level of interest in a workshop topic. This technique has been used in other forest resource management CE needs assessment surveys (e.g., Siscock, Rechenbach, & Finley, 1995). This study was designed to determine whether attendance varied based on survey respondents' level of interest in a workshop topic. The small number of survey respondents who indicated a strong interest in BMP and OSHA workshops prevented us from completely answering this question. However, the Silviculture results suggest that survey respondents who indicate a strong interest in a workshop are more likely to attend that workshop (Table 3).

An ancillary, and perhaps more practical, question is whether CE educators need to ask survey respondents to differentiate between their levels of interest in a workshop topic. Our results suggest they do not. Every survey respondent in our study who indicated a strong interest in a workshop topic also indicated a general interest in the topic. Thus, strong interest respondents were a subset of general interest respondents. If the purpose of needs assessment surveys is to gauge interest in workshop topics, CE educators can do this by only measuring clients' general interest in a workshop topic. This would lessen the time and effort clients must expend to fill out a survey and may increase survey response rates.

Conclusion

Educators regularly use needs assessment surveys to understand their clients' preferences. Our findings suggest that, although educators can sometimes rely on needs assessment surveys to assess program feasibility, only a small percentage of survey respondents who indicate an interest in a topic will attend a program on that topic. In addition, many program attendees will fail to indicate an interest in the program when they are surveyed.

Although many of our findings were significant, they were also tenuous. The exploratory nature of this study and the limited number of survey respondents who attended workshops indicate that more research is needed. Additional studies will help us understand the value of needs assessment surveys and whether they are one of the essential tools educators should use to assess their clients' education needs.

References

Ahmann, J.S. (1979). Needs assessment for program planning in vocational education. Washington, DC: Department of Health, Education, and Welfare.

Bullard, S.H. & Straka, T.J. (1986). Continuing education needs of natural resource professionals. Resource Management and Optimization. 3:281-290.

Dillman, D.A. (1978). Mail and telephone surveys: The total design method. New York: Wiley.

Donovan, D.D., Anderson, S., Wheatcraft, A.M., & Carpenter, S.B. (1989). Continuing Education needs assessment of forestry professionals in Oklahoma. Oklahoma State University, Cooperative Extension Service, Extension Forestry Report No. 2., 24 pp.

Fischer, B.C., & O'Leary, J.T. (1987). Continuing forestry education. Journal of Forestry 85(2):18-19.

Gilmore, G.D. (1989). Needs assessment strategies for health education and health promotion. Indianapolis, IN: Benchmark Press.

Green, Samuel B., N.J. Salkind, & T.M. Akey (1997). Using SPSS for Windows. Analyzing and understanding data. Upper Saddle River, NJ: Prentice-Hall, Inc.

Lilley, W.D. (1987). Continuing forestry education in Maine. University of Maine, College of Forest Resources, Technical Notes No. 99. 10 pp.

Malmsheimer, R.W., & Floyd, D.W. (1996). A continuing education needs assessment of New York State forest resource managers. SUNY ESF Report. 75 pp.

Oregon State University Administrative Committee (1998). Strategic planning for outreach education: Responding to market opportunities -- incentives and barriers. Oregon State University, College of Forestry Report. 12 pp.

Queeney, D.S. (1995). Assessing needs in continuing education: An essential tool for quality improvement. San Francisco: Jossey-Bass Publishers.

Reviere, R., Berkowitz, S., Carter, C.C., & Ferguson, C.G. eds. (1996). Needs assessment: A creative and practical guide for social scientists. Washington, DC: Taylor and Francis.

Reynolds, H.T. (1984). Analysis of nominal data. 2nd ed. Sage University Paper Series on Quantitative Research Methods, Vol. 7. Newbury Park, CA: Sage.

Siscock, M., Rechenbach, M., & Finley, J. (1995). Continuing forestry education survey report. Pennsylvania State University, School of Forestry Report. 20 pp.

Straka, T.J., & Richards, D.P. (1984). Continuing forestry education: The effects of age, years of experience, and type of employer on forester-perceived needs. Journal of Forestry 82:556-557.

Straka, T.J., & Richards, D.P. (1985). Continuing forestry education: Needs of consulting foresters in Mississippi. The Consultant 30(1):8-10.

Witkin, B.R., & Altschuld, J.W. (1995). Planning and conducting needs assessments: A practical guide. Thousand Oaks, CA: Sage Publications.