Using Evaluations to Identify and Eliminate a Barrier to Invasive Weed Control

Evaluation and Research

Evaluation and research are different. Evaluation differs from research in that evaluation is intended to guide the development or determine the fate of a project, while research is used to generalize findings to a broader context. For example, research is used to test hypotheses and to explore questions of interest to the scientific community. Evaluation is more specific. In the project described here evaluation was used to guide the development of an educational program, to identify barriers, and to report on results. This article demonstrates the use of evaluation and how it guided the development of a novel solution to an educational problem.

Assessing a Local Need

Non-native invasive species are a growing concern along the north shore of Lake Superior. Widely accepted as an increasing ecological and economic threat to woodlands and forest resources, invasive species spread regardless of ownership or management boundaries (National Invasive Species Council, 2001; USDA Forest Service, 2004). Furthermore, it is recognized that there may be many barriers to landowners in adopting invasive species control. Epanchin-Niell et al. (2009) referred to regions of the landscape with high management diversity as "management mosaics." They argued invasive species are difficult to control on management mosaics because landowners can disregard management practices on adjacent land. They also argued the number of incentives needed to encourage landowners to adopt invasive weed control management practices increases as the diversity of ownership across the region increases.

Addressing Needs and Using Evaluation

In response to an assessment of the need to address invasive species control across management mosaics, the Minnesota Cook County Invasives Team (CCIT), Sugarloaf: The North Shore Stewardship Association (Sugarloaf), and the University of Minnesota Extension (Extension) developed and implemented a yearlong education-based project for landowners that focused on invasive species identification and control. To be truly effective, Diem (1997, 2003) argued that programs need to change people's attitudes and behaviors. To determine if this project effected change in participants' attitudes and behaviors, Extension designed impact evaluations to measure changes in knowledge, attitude, skills, and aspirations (KASA) (Bennett, 1975). Participants in all workshops reported that they (1) learned how to identify invasive species, (2) learned how to control invasive species, (3) had situations where they could apply what was learned, and (4) would change their practices based on what they learned.

Although landowners intended to change their practices based on what they learned at the workshops, they also admitted during discussions and conversations that change would be difficult. Few landowners reported being willing or able to purchase the equipment and herbicides needed for controlling invasive weed populations on their own properties. Many participants, however, reported that they were willing to be involved in community-wide invasive species control efforts and would take action to control invasive species on their own land if equipment and herbicides were readily available. Although people's attitudes changed, ultimately, their behavior did not. In summary, there was evidence of change based on Bennett's (1975) hierarchy, but not evidence of impact (Diem, 1997, 2003) because there was no change in practice.

Eliminating Barriers

The lack of what Diem (1997, 2003) calls significant change or impact prompted Extension and its partners to provide landowners access to the tools and equipment they lack. Sugarloaf secured grant funding from the National Fish and Wildlife Foundation to open a community herbicide shed (CHS). The vision of the CHS was to have tools, herbicides, and herbicide application equipment available to landowners. The evaluation data from the invasive species identification and control workshops guided the development and operation of the CHS.

The CHS opened in June of 2014 and is located at Sugarloaf Cove, a nature center just outside of Schroeder, MN. The CHS includes handsaws, brush loppers, and spades, as well as backpack sprayers, handheld sprayers, and sponge applicators for herbicide application. The CHS also houses safety equipment for applicators and a chemical spill clean-up kit. Additionally, Sugarloaf provides herbicides available free of charge to landowners who have received training custom tailored to their needs.

To minimize accidents, all mixing, loading, and storage is done at Sugarloaf Cove according to protocols developed by the CCIT in consultation with the Minnesota Department of Agriculture, the Minnesota Pollution Control Agency, and Extension. Sugarloaf has a site-specific incident response plan along with all labels and material safety data sheets (MSDS).

Measuring Impact

During the opening summer of 2014, seven local landowners attended herbicide training and received and used CHS herbicides. Each landowner treated 1 to 2 acres of their own or a neighbor's land, either for Canada thistle (Cirsium arvense), common tansy (Tanacetum vulgare), or reed canary grass (Phalaris arundinacea). As a part of the training, each landowner completed a split pre- and post-evaluation (Popham, 2006). Each evaluation included a section testing weed identification, a true/ false set of questions testing knowledge of herbicide application and safety, and a series of Likert scale questions to gauge participants' attitudes toward herbicides. All participants scored better on the identification and true-false sections (Figure 1), and all indicated a more favorable attitude toward herbicides (Figure 2) following the training. These results show that the CHS effectively changed the knowledge, attitudes, and practices of all participants.

Figure 1.
Knowledge Gained During Herbicide Training

Figure 2.
Change in Attitude as a Result of Herbicide Training

Conclusion

Evaluations from a yearlong educational effort to raise awareness about and increase invasive species control on private lands were used to identify barriers to impact. A barrier to weed management among private landowners is a lack of herbicides, equipment, and training. To overcome these barriers, the CCIT and Extension used evaluation to guide the development of a CHS. Results to date show the CHS effectively changed the knowledge, attitudes, and practices of all participants. The evaluation process used is an example of how evaluation can be used to guide and refine the development of an effective educational program.

Acknowledgments

Angelique Edgerton spent 14 months (06/2013 to 08/2014) working as the Invasive Species Coordinator for the Cook County Invasives Team (CCIT). The authors would like to thank the CCIT and Michael Lynch, CCIT coordinator from 2011 to 2013, Sugarloaf: The North Shore Stewardship Association (Sugarloaf) and Molly Thompson, Sugarloaf's Executive Director.

References

Bennett, C. (1975). Up the hierarchy. Journal of Extension [On-line], March/April. Available at: http://www.joe.org/joe/1975march/1975-2-a1.pdf

Diem, K. G. (1997). Measuring impact of educational programs. Rutgers Cooperative Extension fact sheet #869. New Brunswick, NJ: Rutgers University.

Diem, K. G. (2003). Program development in a political world—It's all about impact! Journal of Extension [On-line], 41(1) Article 1FEA6. Available at: http://www.joe.org/joe/2003february/a6.php‎

Epanchin-Niell, R. S., Hufford, M. B., Aslan, C. E., Sexton, J. P., Port, J. D., & Waring, T. M. (2010). Controlling invasive species in complex social landscapes. Ecological Environment, 8(4): 201-216.

National Invasive Species Council (2001). Meeting the invasive species challenge: National invasive species management plan. Retrieved from: http://www.invasivespecies.gov/home_documents/
2001%20Invasive%20Species%20National%20Management%20Plan.pdf

Popham, W. J. (2006). Assessment for educational leaders. Boston: Pearson/Allyn and Bacon

USDA Forest Service (2004). National strategy and implementation plan for invasive species management. Retrieved from: http://www.fs.fed.us/invasivespecies/documents/Final_National_Strategy_100804.pdf