Evaluating the Effectiveness of a Sustainable Living Education Program

Introduction

Overwhelming scientific evidence shows increasing levels of greenhouse gases (GHGs) are changing the climate, with impacts to agriculture, human health, and other areas directly relevant to the individuals and businesses that Extension programs support (U.S. Global Change Research Program [USGCRP], 2009, 2011). Extension has a long and fruitful history of fostering learning and adoption of new behaviors in response to changing scientific knowledge. Individuals are responsible for one-third of U.S. GHG emissions (Vandenbergh, Stern, Gardner, Dietz & Gilligan, 2010). Focusing on households has potential for rapid and substantial impact because individuals can change their behaviors without waiting for new policies or technology. Widespread national adoption of select existing technology in homes and for personal travel would reduce national GHG emissions 7.5% and household emissions 20% (Dietz, Gardner, Gilligan, Stern, & Vandenbergh, 2009). Yet few people know how to effectively reduce their energy use and the associated GHGs (Attari, DeKay, Davidson, & de Bruin, 2010).

In 2007, the nonprofit, The Resource Innovation Group, modeled Climate Masters at Home (CM at Home) on successful Extension "Master" programs, with the goal of increasing behaviors that lead to 20% reductions in GHG emissions from participants. The program has since been replicated in Oregon and Nebraska by Extension, as well as in New Mexico, and Edmonton, Canada. This article describes and evaluates that program.

Program Description

The structure of CM at Home is similar to that of other Extension "Master" programs. CM at Home includes two primary components: 1) a 10-week long "train-the-trainer" course modeled after nationwide Master Gardener programs and 2) volunteer opportunities, including household "climate consultations," or personalized household audits, and site-specific recommendations conducted by graduates of the program.

In the course, participants receive 30 hours of training in exchange for 30 hours of community outreach on reducing personal GHG emissions. The approximately 10-week course focuses on reducing GHGs from homes, yards, food, personal transportation, consumption, and waste. The course facilitator brings in expert presenters for each topic, leads class activities, and suggests home activities to deepen learning and promote action.

Graduates of the program ("Climate Masters") then conduct household consultations. These consist of home visits in which residents receive tailored strategies for reducing their personal GHGs. Residents then commit to taking related actions of their choice. In some cases, consultation recipients are contacted to see if they followed through on their commitments and if they need further support. Climate Masters also conduct other forms of outreach, including tabling and speaking at events, distributing compact florescent light bulbs to their neighbors, and making presentations to school groups.

CM at Home is based on behavior change models, in particular that of community-based social marketing (McKenzie-Mohr & Smith, 1999), the Transtheoretical Change Model (University of Rhode Island Cancer Prevention Institute; Prochaska, DiClemente, & Norcross, 1992; Prochaska & Velicer, 2004), and research documenting the effectiveness of water and energy audits in prompting behavioral change (Whitcomb, 1991; Abrahamse, Steg, Vlek, & Rothengatter, 2005). The program engages early adopters through the course and provides them with motivation, information, support, and a structure to move beyond intent to action. By fulfilling their volunteer commitment, the early adopters—Climate Masters—make direct, one-on-one contact with friends, neighbors, and other community members who might be less engaged. Moreover, early adopters are often influential in their local community and can create a ripple effect when they change their own behavior. As such, the CM at Home program is designed to meet the needs of residents at varied levels of interest, build upon existing expertise and networks within a community, and, finally, reduce GHGs.

Evaluation

To guide the development of the program and to evaluate its impact, CM at Home participants were asked to complete detailed Internet-based surveys before the course and again within a year. The survey focus changed somewhat over the replications. In the pilot year (2007), data were gathered from Climate Masters and those receiving consultations to estimate changes in GHGs, paralleling areas covered in the course. Comparisons from pretest to posttest (n=27) on self-reported activities indicated substantial reductions in annual GHGs from flying, food choices, garbage produced, etc., of approximately 15 to 23%, meeting the goal of 20% set at the onset of the program. The variation in estimates reflects different baselines for annual per capita emissions.

A sub-sample of survey respondents from the pilot gave permission to access their electricity usage from a year pre-involvement with the program to a year post-first contact (n=17). The sub-sample all lived within the service area of one utility and came from both participants in the course and residents receiving consultations from Climate Masters. Adjusting for temperature variation using heating degree-days revealed a 14% reduction in kilowatt-hours used over the time period, a change that was statistically significant (t=1.96, p=0.03).

In the pilot year, 50 volunteers reported 600 outreach hours during which they reached 1,250 people. We lacked the capacity to track volunteer commitment from all of the programs covered in the study or to continue collecting data from those who received consultations, but do not have reason to think they varied substantially from the pilot.

While these results met the initial goals and were informative, the complexity of deriving accurate estimates of GHG emissions, as well as a desire to focus on the behavioral theory underlying the program, led us to alter the evaluation to include questions directed toward timing of behavioral change and shifts in participants' understanding of their role in climate change. To provide more control in the design, we gathered data from comparison groups of participants in Master Recyclers programs and in Oregon State University Extension programs for Master Gardeners and Master Woodlot Managers. These groups were chosen because they included people similar to those who would enroll in CM at Home by demonstrating interest in learning new skills, in communicating these to others, and having concern for the environment.

The survey content varied somewhat as the evaluations became more refined. However, because the substantive pattern of results was similar across all surveys, composite summary items have been combined for this article. Details on changes in wording over time and individual annual survey results are available from the authors upon request.

Changes in behaviors were examined in three ways. One set of questions asked about specific behaviors covered in the course and focused on when actions had been implemented and (posttest only) if participants had increased their efforts in taking those actions since participating in the course. Another set of questions asked how frequently participants engaged in specific actions that help cut GHG emissions, and a third question asked how frequently they talked with others about climate change and GHG reduction strategies.

Four questions focused on participants' attitudes and understandings regarding climate change. These included their general understanding and concern with the issue as well as their understanding of "ways in which individuals can reduce their contribution to climate change" and the extent to which they considered climate change in their "daily and broader decisions." Table 1 describes the indicators used in the analysis and sample size for each question.

Tables 1 and 2 present results for all respondents from 2008 to 2011, summarizing pretest and posttest responses for CM at Home participants and the control group. Average values on each indicator are presented in Table 1, and inferential results (analysis of variance) are presented in Table 2. The analyses of variance allow us to examine the extent to which changes over time were the same or different for the participants and the control group. If the effect of the CM at Home program was different than that of the other Master programs we would expect to find significant interactions. That is, the change from pretest to posttest would be greater for the CM at Home participants than for participants of other programs.

The results were as expected. Changes over time were significantly greater for Climate Masters participants than for participants in the other programs on the indicators involving the ways in which individuals can promote change, as well as intent to and actual changes in climate related behaviors. As expected, both groups were highly concerned about climate change and involved in behaviors to limit their personal impact. However, after participating in the course, the Climate Masters were significantly less likely to have "no plans" to engage in specific behaviors (third line of data in Tables 1 and 2) and significantly more likely to report that they had increased their efforts (fourth line of data in both tables). Climate Masters were also significantly more likely to report an increased understanding of individuals' roles in climate change and to consider climate change in their daily decisions. Similar results were found when the sample was limited to a subset of the total group for whom responses could be matched over time (n=41).

The posttest included a number of opportunities for participants to report their reflections on the program and its impact on their lives. The authors independently reviewed the responses and coded the extent to which they included positive, neutral, or negative reports. Almost all comments (90%) were positive. Common responses involved an increased sense of well-being, empowerment, and good health for participants, with some saying the program changed their lives. A complete listing of comments and further detail on all results are available upon request.

Discussion and Conclusion

The data suggest that the CM at Home program not only increased participants' understanding of individuals' roles in reducing household greenhouse gas emissions, but also triggered action. When compared to participants in other Master classes, who had similar initial levels of concern regarding climate change, the CM at Home participants reported greater behavioral change, increased consideration of climate change in daily decisions, and increased understanding of how individuals can affect climate change. We suggest that these findings illustrate how the program had both a relatively immediate impact on participants' lives as well as the potential for promoting ongoing behavioral changes, both for the participants and for those with whom they interact. Future work could explore variations in the types of behavior change that occur and why some types of changes are more likely than others. It would also be important to examine participants' subsequent interactions with others in their communities and results of the volunteer payback period.

The Climate Masters at Home program follows in the tradition of Extension by providing a practical, research-based method to promote healthier living and environmental sustainability. The skills and behaviors learned in the CM at Home program will be crucial as changes in the climate become more manifest. In addition, in times of rising energy costs and economic stress, many of the skills taught in the course provide co-benefits of reductions in household expenditures in areas such as home energy use and transportation.

Given its presence throughout the country and its experience and credibility in translating scientific knowledge for national audiences, Extension is uniquely positioned to support the adoption of sustainable behaviors (Elliott et al., 2008) and the CM at Home program. It is our hope that the experiences with CM at Home described in this article will result in replications of the program in many other locations, with diverse settings and with individuals of diverse backgrounds. Some communities (although none of those described in the study reported here) have replicated the program with a different name in order to share the energy efficiency and conservation practices promoted by the program while avoiding conflict that may arise with a charged topic like climate change. Continued evaluation should, of course, be incorporated in these replications. More information on the CM at Home program, its curriculum and resources for starting a program can be found at www.theresourceinnovationgroup.org. 

Acknowledgments

Primary financial support for conducting the project this article describes was provided by the Environmental Protection Agency, the Meyer Fund for a Sustainable Environment and Eugene Water & Electric Board. We thank Stacy Vynne, Robert Doppelt, and Holly Spencer for comments on earlier drafts and Vanessa Vissar for data management assistance.

References

Abrahamse, W., Steg L., Vlek C., & Rothengatter T. (2005). A review of intervention studies aimed at household energy conservation. Journal of Environmental Psychology, 25(3), 273-291. doi: 10.1016/j.jenvp.2005.08.002

Attari S .Z., DeKay M. L., Davidson C.I ., & de Bruin W.B. (2010). Public perceptions of energy consumption and savings. Proceedings of the National Academies of Sciences USA 107, 16054-16059. doi:10.1073/pnas.1001509107

Dietz T., Gardner G. T., Gilligan J., Stern P. C., & Vandenbergh M. P. (2009). Household actions can provide a behavioral wedge to rapidly reduce US carbon emissions. Proceedings of the National Academies of Sciences USA 106, 18452-18456. doi: 10.1073/pnas.0908738106

Elliott, C., Hyde, L., McDonell, L., Monroe, M., Rashash, D., Sheftall, W., Simon-Brown, V., Worthley, T., Crosby, G., & Tupas, L. (2008). Sustainable living education: A call to all Extension. Journal of Extension [on-line], 46 (2) Article 2COM1. Available at: http://www.joe.org/joe/2008april/comm1.php 

McKenzie-Mohr, D., & Smith, W. (1999). Fostering sustainable behavior: An introduction to community-based social marketing. Gabriola Island, British Columbia: New Society Publishers.

Prochaska, J. O., Diclemente, C. C., & Norcross, J. C. (1992), In search of how people change: Applications to addictive behaviors. American Psychologist, 47(9), 1102-1114. doi: 10.1037/0003-066X.47.9.1102

Prochaska, J. O., & Velicer, W. F. (2004). Integrating Population Smoking Cessation Policies and Programs. Public Health Reports 119, 244-252. doi: 10.1016/j.phr.2004.04.003

U.S. Global Change Research Program (2011). Our changing planet: The U.S. Global Change Research Program for fiscal year 2011. Washington, DC: Author. Retrieved from: http://www.globalchange.gov/publications/our-changing-planet-ocp

U.S. Global Change Research Program (2009). Global climate change impacts in the United States (2009). Washington, DC: Author. Retrieved from: http://www.globalchange.gov/publications/reports/scientific-assessments/us-impacts

University of Rhode Island Cancer Prevention Research Center. Detailed overview of the transtheoretical model. Retrieved August 10, 2011, from: http://www.uri.edu/research/cprc/TTM/detailedoverview.htm 

Vandenbergh M. P., Stern P. C., Gardner G. T., Dietz T., & Gilligan, J. M. (2010) Implementing the behavioral wedge: Designing and adopting effective carbon emissions reduction programs. Environmental Law Review, 40, 10547-10554. Retrieved from: http://ssrn.com/abstract=1617426

Weber, E. U., & Stern, P.C. (2011). Public understanding of climate change in the United States. American Psychologist, Vol 66(4), 315-328. doi: 10.1037/a0023253

Whitcomb, J. B. (1991). Water reductions from residential audits. JAWRA Journal of the American Water Resources Association, 27, 761-767. doi: 10.1111/j.1752-1688.1991.tb01474.x