An Evaluation of the Missouri Master Naturalist Program and Implications for Program Expansion

Introduction

Cooperative Extension Services, an integral part of land-grant universities, have responded to shifts from rural to urban communities by expanding beyond their traditional agricultural, rural focus to play important roles in urban and suburban life (U.S. Department of Agriculture Extension Services, 2008). The land-grant universities, in partnership with state conservation and natural resource agencies, are developing community-based conservation education programs to address new issues, including the public's alienation from nature, increasing urbanization (Chawla & Salvadori, 2003; Coyle, 2005; Louv, 2005; Nilon, Berkowitz, & Hollweg, 2003), and the decreasing quality of local environments (Hobbs & Stoops, 2002; Mac, Opler, Puckett Haeker, & Doran, 1998; Markham & Steinzor, 2006). Developing dedicated and well-informed citizens is an essential component of these community-based initiatives (Beierle & Cayford, 2002; McDermott, Moote, & Danks, 2005).

Master Naturalist (MN) Programs are one example of community-based programs. There are several different MN program models, all of which seek to improve citizens' understanding of natural resource ecology and management, and to enhance existing conservation and education and outreach activities (ANROSP, 2007; Missouri MN, 2008). Missouri created its MN Program in 2004, modeling it on the successful Texas MN program. Missouri's MN educational program is a partnership involving MU Extension, the Missouri Department of Conservation (MDC), and the University of Missouri-Columbia, School of Natural Resources.

The educational objectives of the Missouri MN Program are to:

• Improve public understanding of natural resource ecology and management by developing a pool of local knowledge that can be used to enhance and expand educational efforts within local communities;
• Enhance existing natural resources education and outreach activities by providing natural resources training at the local level, thereby developing a team of dedicated and informed volunteers;
• Develop a self-sufficient Missouri Master Naturalist volunteer network.

As of 2007, Missouri's MN Program had eight active chapters affiliated with over 40 organizations (local partners) and had trained over 380 volunteers (Table 1). These volunteers become certified Master Naturalists after completing 40 hours of initial field and classroom instruction, 8 hours of advanced training, and 40 hours of community service. Almost 200 MN volunteers have donated over 16,000 hours of service to enhance the natural resource base of their communities, with an economic impact valued at over $330,000, based on U.S. Bureau of Labor Statistics' average hourly earning of $18.77/hour for private nonagricultural workers increased by 12% to estimate fringe benefits (Pierce & Wallace, 2007).

Table 1.
Location of Missouri MN Chapters, May 2007

Missouri MN Chapter	Location
Boone's Lick Chapter	Columbia, Missouri
Chert Glades Naturalists	Joplin, Missouri
Confluence Chapter	St. Charles, Missouri
Osage Trails Chapter	Kansas City, Missouri
Meramec Hills Chapter	Phelps and Dent Counties, Missouri
Ozarks Chapter	West Plains, Missouri
Springfield Plateau Naturalists	Springfield, Missouri
Meramec Hills Chapter	Rolla, Missouri

Purposes and Objectives

Although many states are developing and implementing MN-type programs (ANROSP, 2007), no formal method exists to evaluate program outcomes and impacts. In addition, the link between volunteers' learning during training and their motivation for participating is not well known, despite the fact that motivation may significantly affect learning outcomes (Holton, 1996; Kirkpatrick, 1996, 1998; Mathieu, Tannenbaum, & Salas, 1992; Noe & Schmitt, 1986). While developing the Missouri MN Program, we sought to identify and implement a framework to determine the program's effectiveness and to evaluate the program over time. We also sought to develop an evaluation tool that might have broader applicability to other MN programs; to identify characteristics that support volunteer retention, a critical component of the MN programs; and to identify potential issues that the programs can address.

We evaluated the initial 40-hour training conducted by the Missouri MN Program in 2005 and 2006. We designed our study to answer five questions:

1. Does MN training improve volunteer knowledge of ecological processes and conservation issues?
2. What motivates individuals to join the Missouri MN Program?
3. Is there a relationship between an individual's motivation for joining and a change in knowledge because of the training?
4. How can we apply the results of our study to improve the Missouri MN program?
5. What recommendations can we make, based on our results, to help other states to improve or develop their MN programs?

Methods

Research Procedures/Administration

We conducted our research at four Missouri MN training locations during the fall 2005 and spring 2006. We invited all participants at each training location to join the study. Participation was voluntary, and results were confidential. The University of Missouri-Columbia's Institutional Review Board approved the research protocol (IRB Number 1049238). We used the SPSS statistical software package (SPSS Version 14.0.1, 2005). We used three research instruments: a knowledge survey, a motivation inventory, and a demographic questionnaire.

Knowledge Survey

We developed a survey to measure volunteers' knowledge of key ecology, management, and conservation concepts. We used a mixed question format: multiple-choice, fill-in, True/False, matching, and short answer questions (Broun, 2007; Salant & Dillman, 1994). The survey consisted of 25 questions from nine topic areas, based on the objectives of the MN Program (Missouri MN, 2008). We modeled the questions on conservation education materials developed by MDC (Behrens, Stucky, & Gray, 2001). Survey topic areas were:

• Conservation Principles;
• Ecological Principles;
• Energy Cycle;
• Forest Ecosystems;
• Prairie Ecosystems;
• Invasive and Exotic Species;
• Watershed Management;
• Wildlife Management;
• Ecoregions of Missouri.

We used Cronbach's alpha (Cronbach, 1951) to determine the reliability of the knowledge survey. We compared the alpha coefficient on the pre-training survey with that on the post-training survey to determine whether the coefficient increased after training.

We calculated an overall knowledge score for each participant in the study that was the sum of correct responses to the 25 questions. We used a one-way repeated measures ANOVA to compare volunteers' pre-training, post-training, and 6-month follow-up overall knowledge scores (P ≤ 0.05). For post-hoc analysis, we performed two paired samples t-tests, adjusting the level of significance using the Bonferroni correction to avoid inflating the Type I error rate and thus used a significance level of P = 0.017, i.e., 0.05/3 (Miller, 1991; Gray, 2004). Because there was a cap on the number of volunteers participating in each training session, our study had a relatively small sample size. To ensure our results were not sample-size dependent, we performed two paired-sample t-tests on random sub-samples of volunteers (P ≤ 0.05).

We determined the total number of correct responses to questions in each topic area covered by the knowledge survey. We used a paired t-test to compare pre-training knowledge scores and post-training scores (P ≤ 0.05) for each topic area.

We identified single questions within each topic areas where less than 50% of volunteers responded correctly on the pre-training survey and questions that showed no change in the number of correct responses pre-training to post-training. We compared the frequency of the score for each question on the pre-training survey with the frequency of the score for the post-training survey. This permitted us to determine specific issues within topic areas that volunteers understood and those needing modification.

Volunteers completed the Knowledge Survey three times: pre-training (on the first night), post-training (on the last night), and a follow-up survey that we mailed to each volunteer 6 months after the training. All Knowledge Surveys were identical.

Motivation Inventory

We adapted the Volunteer Functions Inventory (VFI), developed by Clary and Snyder (1991, 1999) and Clary et al. (1998), to identify volunteer motivations for joining the MN Program. There were five motivational statements for each of six categories:

• Understanding (new learning experiences);
• Values (altruism);
• Social (relationships with others);
• Career (preparation for a new career);
• Ego Enhancement (increasing self-esteem); and
• Ego Protection (protecting the ego/assuaging feeling of guilt).

MN volunteers rated each of the thirty statements with the following choices: 1= not important at all; 2= somewhat unimportant; 3= no opinion either way; 4= somewhat important; and 5= very important. We used Cronbach's alpha to determine reliability (internal consistency) of the statements composing each category (Clary et al., 1998).

To determine the importance of motivation for each volunteer, we obtained a mean score for the six categories by averaging the score of the five statements within each category. We used Pearson's r to analyze the relationship between volunteer responses on the Motivation Inventory and changes in volunteer knowledge after completing the MN training (P ≥ 0.05). Volunteers completed the Motivation Inventory once on the first night of training.

Demographic Questionnaire

To identify volunteer characteristics, we modified a questionnaire developed by the Missouri Department of Conservation (Conservation Monitor Survey, 2000). The questionnaire inquired about characteristics including gender, age, race/ethnicity, education, income, and residence. Volunteers completed the questionnaire on the first night of training.

Results

Missouri MN volunteers (n = 85) were female (52.9%) and Caucasian (96.5%). Most volunteers (94.2%) had some post-high school education; about 29% had a Masters or Doctoral degree. The median volunteer age was 52 years (m = 50.77 years; sd = 13.27). Sixty percent were married, and 53% (n = 78) reported incomes over $50,000. Most volunteers lived in rural areas (37%) or cities with a population of 25,000 - 249,999 (26%).

Ninety-two percent of the volunteers (n = 78) completed both the pre-training and post-training knowledge surveys. Fifty-one percent (n = 40) of volunteers completing the post-training survey also completed the six-month follow-up survey. Ninety-seven percent (n = 82) of the volunteers completed the Motivation Inventory.

Knowledge Survey

Overall knowledge scores changed following training (F_2,76 = 50.678, P ≤ .001). Post-hoc analysis revealed an increase in scores from pre-training to post-training (P = 0.017), but no difference between post-training and the 6-month follow-up scores (Table 2). The increase in Cronbach's alpha (.65 - pre-training to .71 - post-training) demonstrated that the Knowledge Survey more reliably measured the volunteer's knowledge after training than before training (Cronbach 1951).

Table 2.
Post-hoc Analysis of Pre-Training vs. Post-training Overall Knowledge Scores and of Post-Training vs. 6-Month Follow-up Overall Knowledge Scores

Knowledge	n	M	Sd	t	df	P
Pre-training	78	33.22	8.535	-10.356	77	.000
Post-training	78	41.00	7.166
Post-training	40	42.30	6.791	.028	39	.977
Follow-up -training	40	42.28	4.772

Two random paired samples comparisons confirmed that the pre-training to the post-training increase in the overall knowledge score was not dependent on the sample size (Table 3).

Table 3.
Paired t-test of Two Random Samples of Pre- and Post-Training Overall Knowledge Scores

Knowledge	n	M	Sd	t	df	P
Pre-training	35	33.91	7.613	-6.969	34	.000
Post-training	35	41.83	6.640
Pre-training	48	33.27	8.495	-8.598	47	.000
Post-training	48	41.25	6.702

There were differences (P ≤ 0.05) between mean pre- and post-training knowledge scores for eight of the nine topic areas (Table 4).

Table 4.
Mean Difference Between Pre- and Post-Training Knowledge Scores by Topic Area (n=78).

Topic Area	Mean Difference Pre-Training and Post-Training Scores	sd	t	df	P
Conservation Principles	-0.500	1.384	-3.191	77	0.002
Ecological Principles	-1.051	2.563	-3.623	77	0.001
Energy Cycle	-1.538	2.345	-5.794	77	0.000
Invasive / Exotic Species	-0.256	0.797	-2.842	77	0.006
Forest Ecology	-0.705	1.250	-4.984	77	0.000
Prairie Ecology	-0.308	0.631	-4.310	77	0.000
Watershed Management	-0.833	1.167	-6.308	77	0.000
Wildlife Management	-0.064	0.437	-1.297	77	0.199
Ecoregions of Missouri	-2.526	2.864	-7.789	77	0.000

Volunteers' knowledge of some specific concepts increased following training. Volunteers receiving full credit for understanding how the Energy Cycle functions increased 30%. Volunteers' understanding of the use of clear-cutting forests increased 28%, of the value of a forested stream border increased 24%, and of the negative effects of stream channelization increase 38%.

In contrast, even after completing training, 36% of the volunteers did not understand the impact of invasive exotic species on native species, 73% still lacked an understanding of tools used to manage Missouri prairies, and less than one-half understood acceptable methods of wildlife management, which changed little after training (41%-48%). There was no change (70% pre- and post-training) in volunteer understanding of ecological succession of forests.

Motivation Inventory

Volunteers ranked values and understanding as their primary motivations for participating in the MN program (Figure 1). The reliability of each motivation category of the VFI was α ≥ 0.70: Career (0.93), Ego Protection (0.91), Social (0.82), Values (0.82), Ego Enhancement (0.80), and Understanding (0.70).

Figure 1.
Mean Scores and 95% Confidence Intervals for Volunteers' Ranking on Volunteer Motivation Inventory

There was no correlation between scores on motivation categories and changes in overall knowledge score (Table 5).

Table 5.
Pearson's r Correlation (n = 77) Between Volunteer Motivation Inventory Scores and Changes in Knowledge Score

Motivation Category	Correlation to Change in Knowledge Score	P
Social	r = -0.14	0.23
Career	r = 0.01	0.90
Values (Altruism)	r = 0.04	0.74
Ego Enhancement	r = -0.01	0.92
Ego Protection	r = -0.10	0.39
Understanding (Learning)	r = -0.16	0.16

Discussion

Volunteer Knowledge

Our results are similar to those from studies assessing the impacts of the Texas MN program and Michigan's Conservation Steward Program. Similar to Missouri, Michigan and Texas based their programs on a modified Master Gardener program model (Bonneau, Legg, Darville, & Haggerty, 2003; Van Den Berg, 2006; Van Den Berg & Dann, 2008). The Texas and Michigan studies also used a pre-test/post-test design and demonstrated a 15% gain in knowledge (Bonneau et al., 2003; Van Den Berg, 2006; Van Den Berg & Dann, 2008). While each program is unique, this MN program model appears successful at achieving its goals for adult learners.

MN volunteers' knowledge improved significantly in all topic areas except Wildlife Management. This broad topic area had only one question on the survey, which may explain the less-than-significant improvement. As to questions within each topic area, volunteers started training with a good general knowledge of some specific concepts. For example, 94% of volunteers had a general knowledge of conservation concepts, i.e., preservation, management, and restoration. Most volunteers (83%-94%) had a general knowledge of various ecological principles (limiting factors, carrying capacity, ecosystem, niche, community, biodiversity, and keystone species), of the benefits from Missouri forests (95%), and of the influence harvesting trees has on forest composition (94%). These results are consistent with the findings of research indicating the public understands simple environmental topics (Coyle 2005).

The minimal or lack of change following training for some questions may be due to differences in teaching approaches among chapter instructors, the effectiveness of the instructors, and the lack of emphasis on teaching key concepts associated with the topic areas during instructor training. Developing a formal training manual may help ensure consistency of training statewide and permit flexibility of the program with various demographic groups.

Notwithstanding that many volunteers reported prior experience or education in natural resource or science fields, e.g., wildlife nursery, naturalist, resource conservation, park ranger, park board, outdoor recreation/education specialist, landscaper, farmer, and scientist, the volunteers' knowledge of ecological processes and conservation issues increased following the MN training and remained stable 6 months after training (Broun, 2007).

Volunteer Motivations

Values and understanding were the most important motivators for participation in the Missouri MN Program, consistent with motivation research involving volunteers in natural resource education or restoration (Bonneau et al., 2003; Ryan, Kaplan, & Grese, 2001; Schroeder, 2000).

MN volunteers did not rank social opportunities as a motivator for initial participation. Anecdotal comments by volunteers during training and a later Satisfaction Survey of volunteers indicated, however, that socialization is an important aspect of the program (R.A. Pierce, personal communication, April 29, 2007). The difference in initial and later responses may be that socialization benefits only became evident after involvement in the MN program. Research indicates that social factors are significant predictors of volunteer commitment (Ryan et al., 2001). Given the importance of volunteer retention to the MN program, the anecdotal comments, and the Satisfaction Survey results, providing socialization opportunities may help build cohesion, aiding volunteer retention.

Motivation scores did not correlate with changes in knowledge. This differs from the finding of other studies (Holton, 1996; Mathieu et al., 1992; Noe & Schmitt, 1986) and may reflect volunteers' reluctance to read carefully the Motivation Inventory. There was a large amount of written material for volunteers to read and complete the first night of training, when they completed the Inventory. Several volunteers circled one response for multiple consecutive statements, suggesting they did not consider each statement carefully before responding. Finding an alternate time to complete the Motivation Inventory might affect volunteer responses.

Volunteer Demographics

The profile of Missouri MN volunteers is similar to that of the Texas MN program and the Michigan Conservation Stewards Program. The lack of racial or ethnic diversity in these programs is conspicuous. Given the goal of the MN program, to address natural resource issues in local communities, and given the concerns with degradation of urban ecosystems, encouraging participation of more ethnically and racially diverse populations is logical. It is unclear what drives the observed lack of diversity, but other inner city communities have begun to address similar issues (Nilon, 2007). Identifying how to successfully transfer the current MN model to minority, urban, and inner city populations with demographics different from these existing MN programs has implications for expanding programs into urban areas such as the city of St. Louis, Missouri.

The city of St. Louis is a truly urban center with demographics that reflect a core central city without associated suburbs (Beiter & Brinkerhoff, 1999). In contrast to current MN demographics, 51% (51.20%) of St. Louis city's population is African American, 75% of the city's population has no college degree, and 60% of the city's households earn less than $35,000/year, with almost one-third of these households earning less than $10,000/year (Census, 2000).

Professionals may need to consider several different issues when transferring MN programs to urban settings. These issues include recognizing that the concerns of persons living in an urban center often focus on addressing risks such as making communities safe, healthy, viable, and productive rather than on invasive species, biodiversity, or watersheds (Bryant & Callewaert, 2003; Shu, 2003). Successfully engaging urban residents in MN activities may require structuring the program to address these concerns. Transposing the current MN training formats and goals into areas with strikingly different demographics, however, may not achieve a more diverse volunteer base (Bryant & Callewaert, 2003; Burch & Carrera, 2003; Coyle, 2005; Lord, Strauss, & Toffler, 2001; Shu, 2003).

The overall structure and implementation of MN programs, however, provides flexibility that may help address these concerns and bring a unique approach to addressing ecosystem issues in urban environments. Organizational factors that MN programs may need to consider include:

• Developing targeted advertising to attract more diverse volunteers;
• Identifying conservation volunteer activities that focus on urban community concerns, e.g., historical restoration of areas that also have natural resource significance;
• Selecting local urban partners to help identify pertinent volunteer opportunities important to the local urban community;
• Adapting program components to address urban ecosystem issues, e.g., forest ecosystem concepts can focus on planting and caring for street trees and restoring vacant lots;
• Training instructors to tailor their teaching methods to cultural and social experiences of urban learners who may require approaches other than the lecture, discussion, and field trip format (ERIC, 1994; Lieb, 1991, Zemke & Zemke, 1984);
• Developing cause-effect scenarios to help urban populations understand how their actions affect their ability to achieve goals for their neighborhoods and communities (Burch & Carrera, 2003; National Science Foundation, 2007);
• Selecting minority professionals who can be role models (Lopez et al., 2005);
• Incorporating customs and beliefs of various cultural groups to help adapt the programs to the values of different demographic groups (Lopez et al., 2005);
• Developing bilingual programs to increase participation of broader ethnic groups in urban centers with multilingual populations (Lopez et al., 2005).

Conclusions

Ongoing evaluation of MN programs is essential. Research should include volunteer perceptions and actual learning as well as the program's impact on the local community and its recognition of and use of MN volunteers. These results may help identify volunteer opportunities, and provide information on community interests and needs that the MN volunteers can help address.

Understanding the motivations of volunteers, not simply for joining the MN program, but also for remaining active is critical to the program's success. Research, reinforced by anecdotal comments of volunteers in the study reported here, indicates that socialization may be critical to retention.

Finally, to reach a broader audience, states with active MN programs as well as those developing these types of programs should consider demographics, determine benchmarks as the program develops or expands into urban centers, and consider ways to modify existing programs to address concerns of urban residents and areas with different demographics. Further evaluation of MN programs by Extension professionals may benefit from collaboration with university faculty having expertise in sociological research techniques including survey design and analysis.

Acknowledgements

The Missouri Department of Conservation (MDC) supported our research. We thank Ginny Wallace and MDC for program support and funding. We thank the staff of MDC, MU Extension, University of Missouri's School of Natural Resources, Department of Fisheries and Wildlife Sciences, Rock Bridge Memorial State Park naturalists, and Runge Nature Center staff for their contributions to the development and evaluation of the Missouri MN Program. We also thank the Missouri MN chapters, in particular the MDC and MU Extension coordinators, MN volunteers, and other staff who participated in and supported this research.

References

Alliance for Natural Resource and Outreach Service Programs (ANROSP). (2007). Retrieved February 5, 2007, from: http://www.nralliance.org/

Behrens, L., Stucky, T., & Gray, G. (2001). Ecological approach to conservation education: Instructor reference manual (Revised). Available from Missouri Department of Conservation, Jefferson City, Missouri.

Beierle, T. C., & Cayford, J. (2002). Democracy in practice: Public participation in environmental decisions. Resources for the Future, Washington, D.C., USA.

Beiter, T., & Brinkerhoff, P. (1999). St. Louis five-year consolidated plan: State issue analysis essay - public facilities. Retrieved February 5, 2007, from: http://stlouis.missouri.org/5yearstrategy/1999/essay_facl.html

Bonneau, L. A., Legg, M. H., Darville, R. L., & Haggerty, M. M. (2003). Texas Master Naturalist program assessment: Changes in volunteer knowledge and attitudes as a result of the training. Unpublished master's thesis, Stephen F. Austin State University, Nacogdoches.

Broun, C. N. (2007). An evaluation of Missouri Master Naturalist Program: Changes in volunteer knowledge and the relationship between motivations and changes in volunteer knowledge. Unpublished master's thesis, University of Missouri, Columbia. Retrieved January 9, 2008, from: http://edt.missouri.edu/Winter2007/Thesis/BrounC-050407-T7446/

Bryant, B., & Callewaert, J. (2003). Why is understanding urban ecosystems important to people concerned about environmental justice? In A. R. Berkowitz, C. H. Nilon, and K. S. Hollweg (Eds.). Understanding urban ecosystems: A new frontier for science and education (pp. 46-58), New York: Springer-Verlag.

Burch, Jr., W. R., & Carrera, J. M. (2003). Out the door and down the street - enhancing play, community, and work environments as if adulthood mattered. In A. R. Berkowitz, C. H. Nilon, and K. S. Hollweg (Eds.). Understanding urban ecosystems: A new frontier for science and education (pp. 417-430), New York: Springer-Verlag.

Census 2000. US Census Bureau. Demographics - City of St. Louis, Missouri. Retrieved February 5, 2007, from: http://censtats.census.gov/data/MO/05029510.pdf

Chawla, L., & Salvadori, I. (2003). Children for cities and cities for children: Learning to know and care about urban ecosystems. In A. R. Berkowitz, C. H. Nilon, and K. S. Hollweg (Eds.). Understanding urban ecosystems: A new frontier for science and education (pp. 294-315), New York: Springer-Verlag.

Clary, E. G., & Snyder, M. (1991). A functional analysis of altruism and prosocial behavior. Review of Personality and Social Psychology, 12, 119-148.

Clary, E. G., Snyder, M., Ridge, R. D., Copeland, J., Stukas, A. A., Haugen, J., et al. (1998). Understanding and assessing the motivations of volunteers: A functional approach. Journal of Personality and Social Psychology, 74(6), 1516-1530.

Clary, E. G., & Snyder, M. (1999). The motivations to volunteer: Theoretical and practical considerations. Current Directions in Psychological Science, 8(5), 156-159.

Conservation Monitor Survey. (2000). Missouri Department of Conservation, unpublished survey.

Coyle, K. (2005). Environmental Literacy in America. Retrieved December 15, 2006, from: http://www.neefusa.org/pdf/ELR2005.pdf

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.

Educational Resources Information Center (ERIC). (1994). The new vision of the urban learner: Four staff development modules. ERIC Document number ED385662. Retrieved September 12, 2007, from: http://eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/14/1b/fd.pdf

Gray, L. N. (2004). Simultaneous statistical inference. Retrieved December 22, 2006, from: http://cooley.libarts.wsu.edu/soc521/Simultaneous%20Inference.pdf

Hobbs, F., & Stoops, N. (2002). Demographic Trends in the 20th Century. (U.S. Census Bureau, Census 2000 Special Reports, Series CENSR-4). Washington, D.C.: U.S. Government Printing Office. Retrieved January 10, 2007, from: http://www.census.gov/prod/2002pubs/censr-4.pdf

Holton, III, E. F. (1996). The flawed four-level evaluation Model. Human Resource Development Quarterly, 7(1), 5-21.

Kirkpatrick, D. L. (1996). Great ideas revisited: Kirkpatrick's four-level model. Training & development yearbook, 50, 54-59.

Kirkpatrick, D. L. (1998). Evaluating training programs: The four levels (2nd ed.), San Francisco: Berrett-Koehler Publishers, Inc.

Lieb, S. (1991). Principles of adult learning. Retrieved February 20, 2007, from: http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/adults-2.htm

Lopez, R. R., Lopez, A., Wilkins, R. N., Torres C. C., Valdez R., Teer J. G., & Bowser G. (2005). Changing Hispanic demographics: challenges in natural resource management. Wildlife Society Bulletin, 33(2), 553-564.

Lord, C. P., Strauss, E. G., & Toffler, A. M. (2001). Natural cities: Urban ecology and the restoration of urban ecosystems. Lincoln Institute of Land Policy Working Paper. Retrieved February 6, 2007, from: http://www.lincolninst.edu/pubs/dl/118_LordStraussToffler02.pdf

Louv, R. (2005). The last child in the woods: Saving our children from nature-deficit disorder. New York: Algonquin Books.

Mac, M. J., Opler, P. A., Puckett Haeker, C. E., & Doran, P. D. (1998). Status and trends of the Nation's biological resources. Reston, Va.: U.S. Department of the Interior, U.S. Geological Survey. Retrieved September 12, 2007, from: http://www.nwrc.usgs.gov/sandt/

Markham, V., and Steinzor, N. (2006). U.S. National Report on population and the environment. Retrieved January 22, 2007, from http://www.cepnet.org/documents/USNatlReptFinal.pdf

Mathieu, J. E., Tannenbaum, S. I., & Salas, E. (1992). Influences of individual and situation characteristics on measures of training effectiveness. Academy of Management Journal, 35(4), 828-847.

McDermott, M. H., Moote, M. A., & Danks, C. (2005). How community-based collaboratives overcome external institutional barriers to achieving their environmental goals. Plenary Panel of the Fifth National Community-based Collaboratives Research Consortium. 17-19 November, 2005. Sedona, Arizona, USA. Retrieved February 8, 2008, from: http://www.cbcrc.org/2005nationalconf/docs/McD_Moote_Danks.pdf

Miller, Jr., R. G. (1991). Simultaneous statistical Inference. New York: McGraw-Hill Book Company.

Missouri Master Naturalist (MN) Program. (2008). Program overview. Retrieved January 9, 2008, from: http://extension.missouri.edu/masternaturalist/

National Science Foundation: U.S. Long Term Ecological Research Network Information. (2007). Retrieved February 6, 2007, from: http://www.lternet.edu/

Nilon, C. H. (2007). Ecology and environmental justice in context: The Sierra Club's toxic tour of Memphis. Bulletin of the Ecological Society of America, 88(2), 199-203. Retrieved December 4, 2007, from: http://www.esajournals.org/doi/pdf/10.1890/0012-9623%282007%2988%5B199%3AEAEJIC%5D2.0.CO%3B2

Nilon, C. H., Berkowitz, A. R., & Hollweg, K. S. (2003). Introduction: Ecosystem understanding is a key to understanding cities. In A. R. Berkowitz, C. H. Nilon, and K. S. Hollweg (Eds.). Understanding urban ecosystems: A new frontier for science and education (pp. 1-15), New York: Springer-Verlag.

Noe, R. A., & Schmitt, N. (1986). The influence of trainee attitudes on training effectiveness: Test of a model. Personnel Psychology, 39, 497.

Pierce, II, R. A., & Wallace, G. (2007). Missouri Master Naturalist accomplishment report: 2004-2006. MU Extension and Missouri Department of Conservation, Missouri, USA. Retrieved January 9, 2008, from: http://extension.missouri.edu/masternaturalist/MN%20Accomplishment%20Report%2004-06.pdf

Ryan, R. L., Kaplan, R., & Grese, R. E. (2001). Predicting volunteer commitment in environmental stewardship programmes. Journal of Environmental Planning and Management, 44, 629-648.

Salant, P., & Dillman, D. A. (1994). How to conduct your own survey. New York: John Wiley and Sons.

Schroeder, H. W. (2000). The restoration experience: Volunteers' motives, values, and concepts of nature. In P. H. Gobster, & R. B. Hull (Eds.), Restoring nature: Perspective from the social sciences and humanities (pp.247-280), Washington, D.C.: Island Press

Shu, J. (2003). The role of understanding urban ecosystems in community development. In A. R. Berkowitz, C. H. Nilon, and K. S. Hollweg (Eds.). Understanding urban ecosystems: A new frontier for science and education (pp. 39-46), New York: Springer-Verlag.

SPSS, Inc. (2005). SPSS Statistical Package 14.0.1. Chicago, Illinois.

U.S. Department of Agriculture Extension Services. (2008). Retrieved July 2, 2008, from: http://www.csrees.usda.gov/ProgViewOverview.cfm?prnum=9352

Van Den Berg, H. A. (2006). Impacts of the Michigan Conservation Stewards Program on participants' knowledge, attitudes, and skills regarding ecology and resource management. Unpublished master's thesis, Michigan State University, East Lansing.

Van Den Berg, H. A., & Dann, S. L. (2008). Evaluation of an adult Extension education initiative: The Michigan Conservation Stewards Program. Journal of Extension [On-line], 46(2) Article 2RIB1. Available at: http://www.joe.org/joe/2008april/rb1.php

Zemke, R., & Zemke, S. (1984). 30 Things we know for sure about adult learning. Innovation Abstracts, VI(8). Retrieved February 20, 2007, from: http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/adults-3.htm