Evaluating the Georgia Master Naturalist Program

Introduction

The master naturalist program is a nationwide adult education program offered in many forms by several states, counties, cities, and nature centers (Alliance for Natural Resource and Outreach Service Programs, 2013). In addition to Georgia, 25 other states have some version of a master naturalist program. Several states have evaluated their programs.

Prior to the establishment of a Minnesota master naturalist program, Savanick & Blair (2005) used a focus group–based needs assessment of environmental educators, land managers, and naturalists to identify relevant community service activities, training needs, and program development challenges. One conclusion was that partnerships with other organizations would strengthen the master naturalist program. Further, the assessment identified a need to focus programming on environmental restoration, research, policy, and interpretation. Focus group participants felt that a high-quality program would involve fewer volunteers but more in-depth training for each volunteer.

A study of the Texas Master Naturalist Program showed changes in knowledge and attitudes associated with management, ecology, and consumptive wildlife use after master naturalist training (Bonneau, Darville, Legg, Haggerty, & Wilkins, 2009). The researchers used a pretest/posttest evaluation method. The pretest was administered on the first day of training, the first posttest was administered on the last day of training, and a second posttest instrument was mailed to program participants 8 months after the training. The highest motivator for participation in the program was a desire to increase knowledge about nature. Knowledge scores increased from the pretest through the two posttests, with respondents averaging 57% correct answers, 73% correct answers, and 74% correct answers, respectively. Of the 26 attitude statements, 14 reflected attitude change from the pretest to the posttest, which showed a change in attitudes about resource management and consumptive wildlife uses. This increase in knowledge and shift in attitudes was viewed as a positive outcome.

For the Michigan Conservation Stewards Program, the Michigan equivalent of a master naturalist program, a two-county pilot program was conducted (Van Den Berg & Dann, 2008). The two counties are home to many areas set aside for conservation and were chosen because of their need for adult education relative to those conservation areas. The researchers found an increase in knowledge about ecology and ecosystem management and an increase in support for management agencies and conservation techniques (e.g., hunting, prescribed fire). The pilot program achieved the desired effects of increased positive attitudes toward resource management and ecosystem knowledge. The researchers also found high interest in volunteering among the participants and showed that there would be opportunities for advanced training of volunteers.

The Utah Master Naturalist Program was assessed by examining a particular topic in the program's curriculum: watersheds (Larese-Casanova, 2011). Eight instructors taught the target module at four locations. All instructors were provided a standardized curriculum. Preprogram and postprogram surveys were used to measure knowledge gain. Amateur naturalists learned more from the program and gave the program a consistently higher evaluation score than professional naturalists did. Every participant had an increase in knowledge (average improvement of 94%) from his or her preprogram score. The researcher found a wide range of knowledge among participants coming into the module and an increase in overall knowledge about watersheds. Behavior change was not assessed.

An evaluation of Missouri's master naturalist program focused on four training locations (Broun, Nilon, & Pierce, 2009). The researchers evaluated the program through a knowledge survey, a motivation inventory, and a demographics questionnaire. They found an increase in scores from a preprogram survey to a postprogram survey but no difference in scores from the postprogram survey to a 6-month follow-up survey. Results indicated a significant improvement in knowledge in all topics except wildlife management. The two highest ranked motivators for participation were values (altruism) and understanding (new learning opportunities). Motivation did not associate with knowledge changes. One recommendation the researchers made was to create a formal training manual to allow for consistency in trainings around the state.

The objective of the study described in this article was to assess the Georgia Master Naturalist Program (GMNP) for participant satisfaction and impact on participants. We wanted to know how the participants felt about their experience with the program and to obtain feedback on possible changes to elements such as range of topics presented during training, volunteer participation, interest in advanced training, and topics for advanced training.

Methods

We electronically surveyed all who participated in the GMNP from 2009 through 2012 to determine levels of program satisfaction, occurrence of and interest in volunteer participation, and desire for advanced training opportunities. We limited contact to this period because we had limited email lists for years prior to 2009. We used SurveyMonkey to administer the survey. The survey instrument consisted of 33 questions, with the initial question allowing the participant to opt out of completing the survey. The University of Georgia Institutional Review Board approved the survey and methods (IRB number STUDY00000085, approved June 27, 2013).

We sent an initial email July 18, 2013, to the participants, providing information on the survey and notifying them that they would receive a link to the survey a week later. We deleted addresses determined to be undeliverable or otherwise invalid. A week after receiving the initial email, participants received an email containing the survey link. We sent reminder emails to the active list once a week for 4 weeks. The survey was closed at midnight August 18, 2013, and data were downloaded into a Microsoft Excel spreadsheet. Each question was analyzed separately with descriptive statistics.

Results

We sent 500 email messages, but only 480 survey links were successfully delivered to participants. Of those, 238 surveys were started, and 205 surveys were completed (Table 1), yielding an 86.1% completion rate and a 42.7% overall response rate (205/480).

Table 1.
Numbers of Survey Responses, by Year, and Program Participants and Programs Offered, per Year

Survey or program statistic	2009	2010	2011	2012	Total
Survey responses by year	44	61	47	40	192a
Program participants per year	186	158	146	140	630b
Number of programs offered per year	8	9	8	9	34
aTotal is less than the 205 surveys completed because some respondents did not indicate year. bEmail addresses for only 500 participants were available.

Respondents were well educated (the highest degree held by 48.0% of respondents was a graduate degree; the highest degree held by 41.6% of respondents was an undergraduate degree). Among respondents with a college degree, 67.0% had degrees in nonscience fields. Almost two-thirds of respondents (65.7%) were retired. Females made up 65.8% of the respondent sample, and average age of respondents was 60 years old (with the age range being 28–81 years).

Nearly all respondents (95%) agreed that program topics were related to their interests, and 91.5% indicated that they would recommend the program to others. Desire to increase understanding of natural history and the environment was the most often selected reason for participating in the GMNP (86.4%). The three topics consistently offered in GMNP courses at sites across the state were birding (94% of courses), native plants (91.5%), and water quality (85.2%). Respondents selected native plants (90.7%), environmental awareness (81.8%), and ecological principles (80.8%) as the three topics most necessary for inclusion in a master naturalist program course (Table 2). Agriculture (42.9%), recycling/urban waste (31.4%), and butterfly gardens (27.6%) were the three topics that participants felt should receive less focus. However, no topic was selected for elimination by more than 45% of survey respondents, and in most cases, a topic was selected for reduced emphasis by only 15% or fewer respondents (Table 2). These findings indicate that the program participants were generally happy with the range of topics offered, with only a few exceptions.

Table 2.
Topics Addressed and Preferences for Topics as Ranked by Respondents

Topic	Topic covered in class (% responses)a	Devote more time to topic (% responses)	Devote less time to topic (% responses)
Native plants	91.5	90.7	1.0
Environmental awareness	80.3	81.8	3.8
Ecological principles	73.1	80.8	2.9
Invasive species	80.7	79.4	3.8
Tree identification	78.5	79.0	1.9
Birding	94.2	78.5	5.7
Local environmental issues	55.6	72.4	7.6
Water quality monitoring	85.2	67.8	5.7
Geology	77.1	67.3	7.6
Reptiles	74.0	65.0	5.7
Wildlife management	53.8	64.0	6.7
Mammal identification	52.5	55.1	9.5
Forest management	70.4	54.7	11.4
aOnly topics with > 50% response rate are listed.

The median price for the GMNP courses taken by respondents was $150 (average = $169; range= $25–$500), and the majority of respondents (94.5%) believed that the course was worth the cost. About 32% of respondents indicated that the price was about right and that they would not pay more or less for the program (Table 3). However, 61% of respondents indicated that they would pay more ($10–$75 more) for the program (Table 3).

Table 3.
Reported Satisfaction with the Cost of the Course

How much more or less would you be willing to pay?	Response rate (%)	Number of responsesa
$51–$75 more	23.5	42
$26–$50 more	20.7	37
$10–$25 more	16.8	30
$0 more—the price is about right	31.8	57
$10–$25 less	0.0	0
$26–$50 less	3.4	6
$51–$75 less	3.9	7
aTotal number of responses was 179.

Most respondents (68%) reported that they do not volunteer. Moreover, 32% do not want a volunteer requirement added to the program. Fifty-two percent of respondents indicated that acceptance of required volunteer service would depend on the number of hours required and the opportunities available. Only 28% of survey respondents had volunteered in the preceding 6 months. However, those who did volunteer were involved with a wide range of projects at the local level and averaged 65 hr per participant in the 6 months prior to the survey. Collectively, this group contributed over 3,600 hr to their communities.

Most respondents (54.9%) indicated that they would participate in topic-specific advanced training, but 43.6% indicated that their participation would depend on the topic of the training. Participants indicated that they would prefer a 2-day advanced training program (58.5%) to a 1-day program (41.5%). Finally, respondents indicated that they would prefer an advanced training program occurring on a weekday (65.8%) to a program occurring on a weekend (34.2%). Respondents were willing to pay approximately $142 on average for advanced training (registration fee, exclusive of lodging, meals, and travel).

Discussion

The results of our study indicate that Georgia master naturalists are predominately female, well educated, retired, and of reasonable financial means, as is the case in other master naturalist programs (Van Den Berg & Dann, 2008). Anecdotally, we noted that over 90% of respondents were Caucasian.

Overall, survey respondents indicated a high level of satisfaction with the GMNP, but on the basis of the survey results, we suggest some changes to the program. First, the depth at which each program topic is covered should be increased. This change could increase volunteerism because program participants indicated that they lacked the knowledge to volunteer. However, increasing depth of some topics would require dropping other topics. Savanick and Blair (2005) similarly found that program organizers in Minnesota preferred the idea of fewer volunteers with more in-depth training. Currently, the GMNP requires not less than 48 hr of instruction, and this minimum should be maintained. Some may argue that increasing depth can be achieved only by adding advanced classes. However, it is clear from the survey that some topics could be eliminated from the program without resulting in much dissatisfaction among program participants. The time gained could be filled with more in-depth coverage of selected alternative topics.

A second suggestion is that local site coordinators should not undervalue the course. The price for the program should be kept between $200 and $250 (in 2015 dollars). Local site coordinators should evaluate the market and charge a fair price. Most programs in Georgia include light refreshments: coffee, water, crackers, and fruit. Participants bring a sack lunch and provide their own transportation to field trips. Speakers are generally paid staff, state, or federal employees (Forestry Commission staff, Forest Service staff, state fish and wildlife biologists, and county or university Extension faculty) and are not reimbursed by the GMNP. This situation keeps the cost low while allowing provision of high-quality instruction from knowledgeable personnel. Ensuring that field trips are to public facilities, such as botanical gardens, state parks, and research centers, further minimizes costs.

The question of required volunteer service is unresolved and should be left to the local arrangement staff. At an average hourly rate of $15/hr for volunteer time, Georgia master naturalists who volunteer contributed over $54,000 in volunteer time to their communities. The volunteers are motivated by a passion for teaching and increasing the environmental knowledge of others, and they find personal satisfaction to be most rewarding when volunteering. Those who do not volunteer face barriers to volunteerism, such as lack of time and perceived lack of knowledge. On the basis of the responses to our survey, we cannot recommend a mandatory volunteer requirement for the GMNP. Volunteerism and associated opportunities can be promoted or advertised at the community level (Main, 2003), and GMNP graduates can volunteer as their schedules or interests allow. Other states have successfully incorporated volunteers into community outreach activities (Bonneau et al., 2009; Hobbs, 2001; Kurtz, 2002; White & Arnold, 2003).

Finally, the majority of respondents expressed interest in advanced opportunities. A few advanced trainings on different topics would be the best option. The topic of most interest was Southern Appalachian ecology. Respondents indicated a willingness to travel up to 100 mi for an advanced training, so these trainings could be held outside the normal program locations. A combination of topics might attract a larger group of master naturalists with more varied interests. For example, combination training could include Southern Appalachian ecology combined with birding, native plant, and/or mammal identification for an area.

Implications for Practice

Results of this survey can serve as a guide for Extension personnel and environmental education staff at nature centers seeking to establish similar programs. It is important to understand the target audience before beginning a new program. It would be useful to know the amount customers would be willing to pay for such programming. This information is critical in setting registration fees intended to cover costs of a program or return a modest profit to the program host. The results of our survey indicate that GMNP programs are reasonably priced and could be priced higher with no loss of participation or satisfaction. Master naturalist programs are constrained by time available for instruction, and the results of our survey provide valuable information about topics to include (and, by inference, those to omit) in order to be most attractive to the target audience.

References

Alliance for Natural Resource and Outreach Service Programs. (2013). Retrieved from http://www.nralliance.org

Bonneau, L., Darville, R., Legg, M., Haggerty, M., & Wilkins, R. N. (2009). Changes in volunteer knowledge and attitudes as a result of Texas master naturalist training. Human Dimensions of Wildlife, 14, 157–172.

Broun, C., Nilon, C., & Pierce, R., II. (2009). An evaluation of the Missouri Master Naturalist Program and implications for program expansion. Journal of Extension [online], 47(3) Article 3FEA5. Available at: http://www.joe.org/joe/2009june/a5.php

Hobbs, B. (2001). Diversifying the volunteer base: Latinos and volunteerism. Journal of Extension [online], 39(4) Article 4FEA1. Available at: http://www.joe.org/joe/2001august/a1.html

Kurtz, J. (2002). Master Gardeners have been keeping Minnesota green for 25 years. University of Minnesota Extension News Release. Retrieved from http://www.extension.umn.edu/extensionnews/ 2002/MasterGardenershaveBeenKeepingMinnesota.html

Larese-Casanova, M. (2011). Assessment and evaluation of the Utah Master Naturalist Program: Implications for targeting audiences. Journal of Extension. [online], 49(5) Article 5RIB2 Available at: http://www.joe.org/joe/2011october/rb2.php

Main, M. B. (2003). Engaging FMNP graduates. Florida Master Naturalist Program newsletter, 3(3). Retrieved from http://www.masternaturalist.ifas.ufl.edu/newslttr/vol3no3_web.pdf

Savanick, M. A., & Blair, R. B. (2005). Assessing the Need for Master Naturalist Programs. Journal of Extension. [online], 43(3) Article 3FEA7. Available at: http://www.joe.org/joe/2005june/a7.php

Van Den Berg, H. A., & Dann, S. L. (2008). Evaluation of an adult extension education initiative: The Michigan Conservation Stewards Program. Journal of Extension. [online], 46(2) Article 2RIB1. Available at: http://www.joe.org/joe/2008april/rb1.php

White, D. J., & Arnold, M. E. (2003). Why they come, why they go, and why they stay: Factors affecting volunteerism in 4-H programs. Journal of Extension [online], 41(4). Article 4RIB5. Available at: http://www.joe.org/joe/2003august/rb5.php