Outreach, Applied Research, and Management Needs for Wisconsin's Great Lakes Freshwater Estuaries: A Cooperative Extension Needs Assessment Model

Introduction

For Extension programming to be efficient and effective, it should be focused on priority needs associated with the topic and/or target group of interest (Etling & Maloney, 1995; Gibson, 2001). A comprehensive needs assessment can be a valuable method of identifying priorities and focusing programming. It can enable informed decision-making regarding the use of limited resources and ensure that diverse viewpoints are incorporated into program development. Because of its wide applicability, a variety of needs assessment methodologies are frequently used in Extension programming efforts (e.g., Caravella, 2006; Etling & Maloney, 1995; Kaplan, Liu, & Radhakrishna, 2003).

This article outlines the methods used to assess Wisconsin's Great Lakes freshwater estuary needs. Our study builds upon the strong tradition and experience related to Extension needs assessment methodologies and represents a rigorous and replicable Extension needs assessment model. This model is especially relevant to Extension natural resource programming but could also be adapted for a wide variety of topics. It incorporates diverse input by using a two-phased process that combines Nominal Group Technique and survey research.

The advantage of this two-phased process is that it uses multiple methodologies (i.e., methodological triangulation) and both qualitative and quantitative data (i.e., data triangulation). Triangulation strengthens a study (Patton, 2002), and Sofranko and Khan (1988) have called for greater use of triangulation in Extension needs assessment processes. In addition, the model simultaneously assesses needs in multiple focal areas (i.e., outreach, applied research, and management), thereby allowing identification of needs across focal areas as well. Needs that occur across focal areas can represent important opportunities for implementing integrated approaches.

Background

Freshwater estuaries are diverse Great Lakes coastal features that occur where rivers and Great Lakes water mix in shallow wetlands located near the mouth of a river (Herdendorf, 1990). Wisconsin's Great Lakes coastlines are dotted with freshwater estuaries of varying size. For example, Herdendorf (2004) has identified 15 coastal freshwater estuary wetlands greater than 40 ha (100 acres) in size on Wisconsin's Lake Superior and Lake Michigan shorelines. Freshwater estuary systems with wetland complexes smaller than 40 ha are also scattered along Wisconsin's shores (Elliot, Olson, Robinson, Sapper, & Techtmann, 2008). Communities were often established adjacent to Wisconsin's freshwater estuaries because of the importance of rivers as navigation routes and sources of food and water. For example, Wisconsin's largest city, Milwaukee, is located on the shores of a freshwater estuary (Kaemmerer, O'Brien, Sheffy, & Skavronek, 1992).

While the Great Lakes have been well studied over the past 100 years, freshwater estuaries and other coastal wetland systems have received limited attention (Krieger, Klarer, Heath, & Herdendorf, 1992; National Oceanic and Atmospheric Administration, 2003). The information that does exist suggests that freshwater estuaries provide important economic benefits (Whitehead, Groothuis, Southwick, & Foster-Turley, 2006). Freshwater estuaries are also important for wildlife and fish (Maynard & Wilcox, 1997; Prince, Padding, & Knapton, 1992; Whillans, 1987) and improve water quality by storing floodwater, trapping sediment, and filtering water (Mackenzie, 2001; Maynard & Wilcox, 1997).

Study Objectives

Great Lakes freshwater estuaries are central components of many Wisconsin coastal communities and provide important services for residents. Like many natural resource management issues, Great Lakes management efforts often require multi-disciplinary approaches. To date, a focused, comprehensive effort to identify and address multi-disciplinary needs related to Great Lakes freshwater estuaries has been lacking. Recognizing this, the University of Wisconsin-Extension conducted a Great Lakes freshwater estuary needs assessment for Wisconsin. The objective of the needs assessment was to identify outreach, applied research, and management needs related to Wisconsin's Great Lakes freshwater estuaries and potential topics for integrated efforts for the purpose of guiding and focusing future freshwater estuary programming.

Methods

The needs assessment was a two-phased process. The first phase involved planning and conducting coastal community working sessions. Working sessions are commonly used by Extension employees (e.g., Fetsch & Bolen, 1989; Frederick, 1998) to engage stakeholders in a facilitated process to explore a given topic, develop consensus around key items related to the topic, and create a sense of ownership regarding results. The coastal community working sessions engaged participants in a facilitated process of identifying freshwater estuary needs. The second phase of the needs assessment involved designing and implementing a survey to further refine, quantify, and prioritize the working session results.

Coastal Community Working Sessions

Three coastal community working sessions were held along Wisconsin's Great Lakes coastlines during the summer and fall of 2006. The target population for the working sessions was defined as individuals with relevant expertise, experience, or interest related to freshwater estuary outreach, applied research, and/or management.

Potential participants for the working sessions that met the target population definition were selected using snowball sampling methodology (Patton, 2002; Taylor-Powell, 1998). Snowballing is a purposeful sampling technique that uses potential participants to help identify other potential participants. For the project described here, an initial list of approximately 30 potential participants for each working session was developed by the principal investigators. Those potential participants were contacted and invited. At the time they were contacted, each was asked, "Do you know of other individuals with relevant expertise, experience, or interest that should be invited to the working session?" Individuals identified by the potential participants were also contacted, invited, and asked this question. The sampling methodology enabled identification of a list of information-rich participants to invite to the working sessions. The final target population for the working sessions included 195 individuals.

At each working session, facilitators led small group processes related to each focal area (i.e., outreach, applied research, and management). Participants were allowed to self-select which small group they participated in based upon their interest. Nominal Group Technique (Delbecq, Van de Ven, & Gustafson, 1986; Taylor-Powell, 2002) was used to facilitate participant identification and prioritization of needs for Wisconsin's Great Lakes freshwater estuaries. Research indicates that the Nominal Group Technique method of having individuals work alone, but in a group environment, generates more ideas than standard group discussion (Delbecq, Van de Ven, & Gustafson, 1986).

The nominal group process for the working sessions included the following steps:

1. Small group participant introductions.


2. Facilitator overview of the small group topic (e.g., outreach needs for Wisconsin's Great Lakes freshwater estuaries).


3. Silent recording of top needs for the small group focal area by each participant.


4. Participant sharing of needs in a round-robin format and facilitator recording of the needs on a flip chart. The process continued until all ideas had been shared and recorded.


5. Participant discussion and clarification of the recorded needs. Similar or related needs were grouped if there was participant consensus.


6. Participant voting using five round self-adhesive dots for each person. Participants placed dots on the flip chart next to the needs they believed were of highest priority. Participants were allowed to distribute their dots as they deemed appropriate (e.g., all five dots on one need or one dot on each of five needs).


7. The total number of dots assigned by participants to each need was tallied to determine priority needs.

At the end of the working session, participants evaluated the sessions using a post-event confidential, written evaluation. The evaluation was designed to collect information regarding (a) session effectiveness, (b) interest in participating in future implementation strategies related to freshwater estuaries, and (c) participant demographics.

After completion of all three coastal community working sessions, qualitative content analysis methods (Patton, 2002; Taylor-Powell & Renner, 2003) were used to analyze and categorize the freshwater estuary needs identified. Qualitative content analysis involves systematically analyzing text and identifying appropriate content categories for a given body of recorded communication. The analysis for the study reported here was focused on the top three needs identified for each focal area through the coastal community working session nominal group processes. The outreach, applied research, and/or management needs identified at the working sessions were sometimes very similar. Because of this, content was analyzed across focal areas rather than separately for each focal area.

Freshwater Estuary Needs Assessment Survey

A survey was used to further refine, quantify, and prioritize the information from the coastal community working sessions. The target population for the survey was the same as for the working sessions (i.e., individuals with relevant expertise, experience, or interest related to freshwater estuaries). The size of the population (n=225) for the survey increased relative to the working sessions (n=195). The increase was a result of interactions generated by the sessions. The working sessions were able to increase the effectiveness of our snowball sampling methodology and assist with identifying additional individuals meeting the target population definition.

Our study used a Web-based survey methodology. The survey instrument asked respondents to rate each of the freshwater estuary needs identified through the qualitative content analysis procedure regarding the extent to which they were considered a priority by the respondent. Each freshwater estuary need was rated relative to the three focal areas of outreach, applied research, and management (see Figure 1).

Figure 1.
Example of Survey Format and Rating Scale

Prior to administering the final survey, a pre-test or pilot survey was used to gather information regarding potential improvements for the survey (DeMaio, Rothgeb, & Hess, 1998). The pre-test survey was sent to 10% of the target population, and respondent debriefing was used to gather information from the pre-test survey. Respondents were asked to reply via email with responses to the following debriefing questions.

• Is the background and introductory information clearly written, and does it provide an appropriate amount of detail?


• Are the survey instructions and process for answering the questions easy to understand?


• Are the questions well written?


• Do you have any suggestions for improving the survey?

The pre-test was distributed via email and had a 15-day response period. The response rate for the pretest survey was 78% (18 responses from 23 requests). Substantial changes to the survey design were made based upon information obtained through the respondent debriefings. For example, the survey was modified from a five-position to a four-position Likert scale format. The "moderate" priority category was removed to increase the discriminative potential of the survey by requiring selection of a directional response with gradations of either "low" or "high" priority (Bardo & Yeager, 1982; Bardo, Yeager, & Klingsporn, 1982). The survey was also modified to give respondents the option of selecting "not applicable" if they felt that a particular need was not relevant for one or more of the focal areas. Changes were also made to the survey instructions and overall appearance based upon the pre-test responses. The pre-test helped to improve the functionality and effectiveness of the final survey.

Demographic information for final survey respondents was collected through questions related to the following:

• Respondents' affiliations


• Respondents' level of experience and expertise related to freshwater estuaries


• Respondents' distribution of work responsibilities relative to administration, outreach, research, and management

Advance notification and reminders have been shown to increase the response rate for a survey (Boyd, 2002; Yu & Cooper, 1983). An advance email notice was sent to the survey population 9 days prior to administering the survey. The advance notice provided potential respondents information regarding (a) the purpose of the survey, (b) the importance of their responses, and (c) how the results would be used. The final web-based survey was administered on January 12th, 2007 via email. There was a 15-day response period, and two email reminders were sent.

Results

A summary of the results from the coastal community working sessions and freshwater estuary needs assessment survey is provided in the subsequent sections.

Coastal Community Working Sessions

Sixty-three of the 195 invited participants attended the coastal community working sessions. Overall, the working session participants identified 83 outreach needs, 68 applied research needs, and 51 management needs for Wisconsin's Great Lakes freshwater estuaries. The qualitative content analysis of the high priority needs identified at the working sessions resulted in categorization of 15 freshwater estuary needs that occurred across the three focal areas.

Participants represented a diversity of organizations, including the following:

• Nine non-profit organizations


• Nine county or municipal departments


• Four federal agencies


• Three state agencies


• Three tribal governments


• Three universities

In the post-event evaluation participants described the sessions as organized, stimulating, and effective. Ninety-three percent (93%) are interested in being involved with future development and implementation of freshwater estuary outreach, applied research, and management strategies.

Freshwater Estuary Needs Assessment Survey

A Web-based survey was used to further refine, quantify, and prioritize the information from the coastal community working sessions. The survey instrument asked respondents to rate each of the high priority freshwater estuary needs identified through the qualitative content analysis of the coastal working session results. The response rate for the final survey was 42% (94 responses from 225 requests), which, according to Archer (2008), is consistent with expectations for an Extension Web-based needs assessment. As Archer (2008) points out, a response rate of this level may limit the ability to generalize results to the entire potential respondent pool; however, the responses are still meaningful and provide valuable information that can be used to help guide the direction and development of future programming.

Respondent Demographics

Eighty-one percent (81%) of respondents to the final survey had six or more years of experience working on water resource issues, and 61% had greater than 10 years experience. Most respondents (51%) rated their freshwater estuary expertise as moderate, with 29% rating it as high or very high and 19% rating it as low or very low. Respondents also indicated that, on average, their work responsibilities were distributed similarly among outreach, research, and management responsibilities; the mean percent of respondents' estimated time spent working in each of the three categories varied by only 8%.

High Priority Needs

Mean scores for each need were calculated using the following values for each rating category:

• One for very low priority


• Two for low priority


• Three for high priority


• Four for very high priority

Figures 2, 3, and 4 show the top three needs for each focal area based upon the mean scores. In addition, needs across focal areas were analyzed by computing the mean scores across the outreach, applied research, and management focal areas (Figure 5).

Figure 2.
Top Three Wisconsin Great Lakes Freshwater Estuary Outreach Needs

Note: Three needs were tied for the third highest mean score.

Figure 3.
Top Three Wisconsin Great Lakes Freshwater Estuary Applied Research

Figure 4.
Top Three Wisconsin Great Lakes Freshwater Estuary Management

Figure 5.
Top Three Wisconsin Great Lakes Freshwater Estuary Needs Across Focal Areas

Applications and Next Steps

This assessment identifies outreach, applied research, and management needs to be addressed through future Wisconsin Great Lakes freshwater estuary initiatives. The assessment also analyzes needs across focal areas and, as a result, identifies potential areas for future integrated efforts. A final report and summary brochure describing the project and its results have been distributed to over 200 water resource educators, researchers, and managers, as well as elected officials in coastal communities.

The results from the study provide a foundation for future decision-making regarding the effective and efficient use of resources available for freshwater estuary work in Wisconsin. These data will help inform and prioritize the independent and collaborative Great Lakes freshwater estuary work of communities, educators, researchers, agencies, and organizations in Wisconsin. The University of Wisconsin-Extension will collaborate with partner institutions, resource experts, and stakeholders to develop and implement strategies to address high priority needs identified in this assessment. Those strategies will assist in guiding future freshwater estuary program development and implementation for the University of Wisconsin-Extension and, potentially, partnering organizations. Wisconsin is also working with the National Oceanic and Atmospheric Administration to designate a National Estuarine Research Reserve on Lake Superior. The freshwater estuary needs identified through the study have informed the development of research, education, and stewardship objectives for the Lake Superior National Estuarine Research Reserve.

Conclusions and Implications

The needs assessment model described in this article incorporates diverse input by using a two-phased process that combines Nominal Group Technique and survey research. It also demonstrates methods for simultaneously assessing needs related to multiple focal areas, which in the study reported here included outreach, applied research, and management. Analyzing multiple focal areas is often important to Extension activities and can facilitate and encourage integrated approaches.

In summary, the steps involved in the needs assessment model include:

Step 1. Clearly define the topic that is going to be assessed.

Step 2. When appropriate, identify the relevant focal areas for the needs assessment.

Step 3. Define and identify the target population for the needs assessment.

Step 4. Plan and conduct working sessions using Nominal Group Technique or similar group processes to facilitate participant identification and prioritization of relevant needs (by focal area when appropriate).

Step 5. Analyze the results of the working sessions to develop an initial comprehensive list of priority needs.

Step 6. Design and implement a survey instrument to further refine, quantify, and prioritize the information from the working sessions.

Step 7. Analyze and summarize the survey results.

Step 8. Distribute the needs assessment results to participants and other interested parties.

Step 9. Develop, implement, and evaluate strategies to address the identified high priority needs.

The study reported here builds upon the strong tradition and experience related to Extension needs assessment methodologies. It demonstrates a rigorous and replicable needs assessment model for Extension natural resource programming that could also be adapted for a wide variety of programming areas.

Acknowledgements

We extend our gratitude and appreciation to Deb Beyer, Linda Caruso, Gail Epping Overholt, Mary Kohrell, Bill Klase, Jill Mrotek, Kris Tiles, and Karen Vermillion for assisting with facilitating the coastal community working sessions for this project. We would also like to thank several University of Wisconsin-Extension colleagues, Heather Boyd, Ken Genskow, Rebecca Power, and Deanna Schneider, for their valuable contributions to the project.

This project was funded, in part, by the Wisconsin Coastal Management Program and the National Oceanic and Atmospheric Administration, Office of Ocean and Coastal Resource Management, under the Coastal Zone Management Act, Grant # NA04NOS4190062.

References

Archer, T. M. (2008). Response rates to expect from Web-based surveys and what to do about it. Journal of Extension [On-line], 44(1) Article 3RIB3. Available at: http://www.joe.org/joe/2008june/rb3.php

Bardo, J. W., & Yeager, S. J. (1982). Note on reliability of fixed-response formats. Perceptual and Motor Skills, 54, 1163-1166.

Bardo, J. W., Yeager, S. J., & Klingsporn, M. J. (1982). Preliminary assessment of format-specific central tendency and leniency error in summated rating scales. Perceptual and Motor Skills, 54, 227-234.

Boyd, H. (2002). Program development and evaluation, how to get a respectable response rate, quick tips #1. Madison, WI: University of Wisconsin-Extension.

Caravella, J. (2006). A needs assessment method for Extension educators. Journal of Extension [On-line], 44(1) Article 1TOT2. Available at: http://www.joe.org/joe/2006february/tt2.php

Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1986). Group techniques for program planning: A guide to Nominal Group and Delphi processes. Middleton, WI: Green Briar Press.

DeMaio, T. J., Rothgeb, J., & Hess, J. (1998). Improving survey quality through pretesting. Proceedings of the Survey Research Methods Section. Alexandria, VA: American Statistical Association.

Elliot, S., Olson, T., Robinson, P, Sapper, B., & Techtmann, C. (2008). Wisconsin Lake Superior National Estuarine Research Reserve site nomination. Madison, WI: Wisconsin Coastal Management Program.

Etling, A., & Maloney, T. (1995). Needs assessment for Extension agents and other nonformal educators. University Park, PA: The Pennsylvania State University.

Fetsch, R. J., & Bolen, K. R. (1989). Successful strategic planning. Journal of Extension [On-line], 27(4) Article 4FEA2. Available at: http://www.joe.org/joe/1989winter/a2.php

Frederick, A. L. (1998). Extension education opportunities with policymakers. Journal of Extension [On-line], 36(2) Article 2COM1. Available at: http://www.joe.org/joe/1998april/comm1.php

Gibson, T. L. (Ed.), (2001). Cooperative Extension program planning in Wisconsin. Madison, WI: University of Wisconsin-Extension.

Herdendorf , C. (1990). Great Lakes estuaries. Estuaries, 13, 493-503.

Herdendorf, C. (2004). Morphometric factors in the formation of Great Lakes coastal wetlands. Aquatic Ecosystem Health & Management, 7(2), 179-197.

Kaplan, M., Liu, S - T., & Radhakrishna, R. (2003). Intergenerational programming in Extension: Needs assessment as planning tool. Journal of Extension [On-line], 41(4) Article 4FEA5. Available at: http://www.joe.org/joe/2003august/a5.shtml

Kaemmerer, D., O'Brien, A., Sheffy, T., & Skavronek, S. (1992). The quest for clean water: Milwaukee Estuary Remedial Action Plan. In J. H. Hartig & M. A. Zarull (Eds.). Under raps: Toward grassroots ecological democracy in the Great Lakes Basin (pp. 139-160). Ann Arbor, MI: The University of Michigan Press.

Krieger, K. A., Klarer, D. M., Heath, R. T., & Herdendorf, C. E. (1992). Coastal wetlands of the Laurentian Great Lakes: Current knowledge and research needs. Journal of Great Lakes Research, 18(4), 525-528.

MacKenzie, R. A. (2001). Great Lakes coastal wetlands-estuarine systems: Invertebrate communities, particle dynamics, and biogeochemical cycles. Unpublished doctoral dissertation, University of Milwaukee, Milwaukee, WI.

Maynard, L. & Wilcox, D. (1997). Coastal wetlands. Background paper, State of the Lakes Ecosystem Conference 1996 (U.S. Environmental Protection Agency Publication No. 905-R-97-015b). Washington, D.C.: U.S. Environmental Protection Agency.

National Oceanic and Atmospheric Administration. (2003). Old Woman Creek National Estuarine Research Reserve. NOAA Magazine. Retrieved May 23, 2006, from: http://www.magazine.noaa.gov/stories/mag120.htm

Patton, M. Q. (2002). Qualitative evaluation and research methods (3^rd ed.). Thousand Oaks, CA: Sage Publications, Inc.

Prince, H. H., Padding, P. I., & Knapton, R. W. (1992). Waterfowl use of the Laurentian Great Lakes. Journal of Great Lakes Research, 18, 673-699.

Sofranko, A. J., & Khan, A. (1988). It's not that simple. Journal of Extension [On-line], 41(4) Article 4FEA3. Available at: http://www.joe.org/joe/1988winter/a3.php

Taylor-Powell, E. (1998). Program development and evaluation, sampling (Publication G3658-3). Madison, WI: University of Wisconsin-Extension.

Taylor-Powell, E. (2002). Program development and evaluation, collecting group data: Nominal Group Technique, quick tips #3. Madison, WI: University of Wisconsin-Extension.

Taylor-Powell, E. & Renner, M. (2003). Program development and evaluation, analyzing qualitative data (Publication G3658-12). Madison, WI: University of Wisconsin-Extension.

Whillans, T. H. (1987). Wetlands and aquatic resources. In M. C. Healey and R. R. Wallace (Eds.). Canadian Aquatic Resources (pp. 321-356). Ottawa: Department of Fisheries and Oceans.

Whitehead, J. C., Groothius, P.A., Southwick, R., & Foster-Turley, P. (2006). Economic values of Saginaw Bay coastal marshes with a focus on recreational values. Fernandina Beach, FL: Southwick Associates, Inc.

Yu, J. & Cooper, H. (1983). A quantitative review of research design effects on response rates to questionnaires. Journal of Marketing Research, 20, 36-44.