February 2013 // Volume 51 // Number 1 // Research In Brief // v51-1rb1
Evaluating the Effectiveness of a Sustainable Living Education Program
The Climate Masters at Home program was modeled after Extension "master" classes with the aim of increasing individuals' energy and resource saving behaviors. This article explores the impact of the program on participants' behavior, attitudes, and knowledge over several years of implementation. Data sources include survey results from participants and control groups, as well as utility data for a subset of participants. The research shows participants making significant changes across a variety of behaviors, integrating climate change into their decision making process, and cutting their electricity use by 12 to 14%.
Overwhelming scientific evidence shows increasing levels of greenhouse gases (GHGs) are changing the climate, with impacts to agriculture, human health, and other areas directly relevant to the individuals and businesses that Extension programs support (U.S. Global Change Research Program [USGCRP], 2009, 2011). Extension has a long and fruitful history of fostering learning and adoption of new behaviors in response to changing scientific knowledge. Individuals are responsible for one-third of U.S. GHG emissions (Vandenbergh, Stern, Gardner, Dietz & Gilligan, 2010). Focusing on households has potential for rapid and substantial impact because individuals can change their behaviors without waiting for new policies or technology. Widespread national adoption of select existing technology in homes and for personal travel would reduce national GHG emissions 7.5% and household emissions 20% (Dietz, Gardner, Gilligan, Stern, & Vandenbergh, 2009). Yet few people know how to effectively reduce their energy use and the associated GHGs (Attari, DeKay, Davidson, & de Bruin, 2010).
In 2007, the nonprofit, The Resource Innovation Group, modeled Climate Masters at Home (CM at Home) on successful Extension "Master" programs, with the goal of increasing behaviors that lead to 20% reductions in GHG emissions from participants. The program has since been replicated in Oregon and Nebraska by Extension, as well as in New Mexico, and Edmonton, Canada. This article describes and evaluates that program.
The structure of CM at Home is similar to that of other Extension "Master" programs. CM at Home includes two primary components: 1) a 10-week long "train-the-trainer" course modeled after nationwide Master Gardener programs and 2) volunteer opportunities, including household "climate consultations," or personalized household audits, and site-specific recommendations conducted by graduates of the program.
In the course, participants receive 30 hours of training in exchange for 30 hours of community outreach on reducing personal GHG emissions. The approximately 10-week course focuses on reducing GHGs from homes, yards, food, personal transportation, consumption, and waste. The course facilitator brings in expert presenters for each topic, leads class activities, and suggests home activities to deepen learning and promote action.
Graduates of the program ("Climate Masters") then conduct household consultations. These consist of home visits in which residents receive tailored strategies for reducing their personal GHGs. Residents then commit to taking related actions of their choice. In some cases, consultation recipients are contacted to see if they followed through on their commitments and if they need further support. Climate Masters also conduct other forms of outreach, including tabling and speaking at events, distributing compact florescent light bulbs to their neighbors, and making presentations to school groups.
CM at Home is based on behavior change models, in particular that of community-based social marketing (McKenzie-Mohr & Smith, 1999), the Transtheoretical Change Model (University of Rhode Island Cancer Prevention Institute; Prochaska, DiClemente, & Norcross, 1992; Prochaska & Velicer, 2004), and research documenting the effectiveness of water and energy audits in prompting behavioral change (Whitcomb, 1991; Abrahamse, Steg, Vlek, & Rothengatter, 2005). The program engages early adopters through the course and provides them with motivation, information, support, and a structure to move beyond intent to action. By fulfilling their volunteer commitment, the early adopters—Climate Masters—make direct, one-on-one contact with friends, neighbors, and other community members who might be less engaged. Moreover, early adopters are often influential in their local community and can create a ripple effect when they change their own behavior. As such, the CM at Home program is designed to meet the needs of residents at varied levels of interest, build upon existing expertise and networks within a community, and, finally, reduce GHGs.
To guide the development of the program and to evaluate its impact, CM at Home participants were asked to complete detailed Internet-based surveys before the course and again within a year. The survey focus changed somewhat over the replications. In the pilot year (2007), data were gathered from Climate Masters and those receiving consultations to estimate changes in GHGs, paralleling areas covered in the course. Comparisons from pretest to posttest (n=27) on self-reported activities indicated substantial reductions in annual GHGs from flying, food choices, garbage produced, etc., of approximately 15 to 23%, meeting the goal of 20% set at the onset of the program. The variation in estimates reflects different baselines for annual per capita emissions.
A sub-sample of survey respondents from the pilot gave permission to access their electricity usage from a year pre-involvement with the program to a year post-first contact (n=17). The sub-sample all lived within the service area of one utility and came from both participants in the course and residents receiving consultations from Climate Masters. Adjusting for temperature variation using heating degree-days revealed a 14% reduction in kilowatt-hours used over the time period, a change that was statistically significant (t=1.96, p=0.03).
In the pilot year, 50 volunteers reported 600 outreach hours during which they reached 1,250 people. We lacked the capacity to track volunteer commitment from all of the programs covered in the study or to continue collecting data from those who received consultations, but do not have reason to think they varied substantially from the pilot.
While these results met the initial goals and were informative, the complexity of deriving accurate estimates of GHG emissions, as well as a desire to focus on the behavioral theory underlying the program, led us to alter the evaluation to include questions directed toward timing of behavioral change and shifts in participants' understanding of their role in climate change. To provide more control in the design, we gathered data from comparison groups of participants in Master Recyclers programs and in Oregon State University Extension programs for Master Gardeners and Master Woodlot Managers. These groups were chosen because they included people similar to those who would enroll in CM at Home by demonstrating interest in learning new skills, in communicating these to others, and having concern for the environment.
The survey content varied somewhat as the evaluations became more refined. However, because the substantive pattern of results was similar across all surveys, composite summary items have been combined for this article. Details on changes in wording over time and individual annual survey results are available from the authors upon request.
Changes in behaviors were examined in three ways. One set of questions asked about specific behaviors covered in the course and focused on when actions had been implemented and (posttest only) if participants had increased their efforts in taking those actions since participating in the course. Another set of questions asked how frequently participants engaged in specific actions that help cut GHG emissions, and a third question asked how frequently they talked with others about climate change and GHG reduction strategies.
Four questions focused on participants' attitudes and understandings regarding climate change. These included their general understanding and concern with the issue as well as their understanding of "ways in which individuals can reduce their contribution to climate change" and the extent to which they considered climate change in their "daily and broader decisions." Table 1 describes the indicators used in the analysis and sample size for each question.
Tables 1 and 2 present results for all respondents from 2008 to 2011, summarizing pretest and posttest responses for CM at Home participants and the control group. Average values on each indicator are presented in Table 1, and inferential results (analysis of variance) are presented in Table 2. The analyses of variance allow us to examine the extent to which changes over time were the same or different for the participants and the control group. If the effect of the CM at Home program was different than that of the other Master programs we would expect to find significant interactions. That is, the change from pretest to posttest would be greater for the CM at Home participants than for participants of other programs.
The results were as expected. Changes over time were significantly greater for Climate Masters participants than for participants in the other programs on the indicators involving the ways in which individuals can promote change, as well as intent to and actual changes in climate related behaviors. As expected, both groups were highly concerned about climate change and involved in behaviors to limit their personal impact. However, after participating in the course, the Climate Masters were significantly less likely to have "no plans" to engage in specific behaviors (third line of data in Tables 1 and 2) and significantly more likely to report that they had increased their efforts (fourth line of data in both tables). Climate Masters were also significantly more likely to report an increased understanding of individuals' roles in climate change and to consider climate change in their daily decisions. Similar results were found when the sample was limited to a subset of the total group for whom responses could be matched over time (n=41).
|Climate Masters||Control Group|
|Timing of Composite Behaviors|
|Did for more than 6 months (ave. % of actions)||53.1||53.1||61.4||62.0|
|Did for 1 to 6 months (ave % of actions)||18.2||15.7||11.3||5.7|
|No plans to do (ave % of actions)||16.8||13.5||19.9||26.9|
|Increased efforts (1=yes, 0 = no)||----||0.7||----||0.5|
|Frequency of Composite Behaviors (5 = always)||3.5||3.7||3.9||4.0|
|Talk with others re climate change (5 = daily, 1=never)||3.9||4.1||3.4||3.7|
|Attitudes (2010-2011 only)|
|Understand climate change (10 high)||6.6||7.3||6.9||7.1|
|Concerned with climate change (10 high)||8.7||8.8||8.9||8.7|
|Understand individuals' role (10 high)||6.5||8.1||7.2||7.2|
|Climate change is filter for decisions (10 high)||6.5||8.0||7.0||7.3|
Timing of Composite Behaviors: Respondents were asked if they were doing and how long they had been doing a series of GHGs related behaviors, involving areas such as using renewable energy and public transit, and altering consumer behavior and food choices. Because the number of behaviors varied across replications three composite measures, comparable across surveys, were developed. All indicators could vary from 0 to 100 and indicate the percentage of behaviors in the survey that respondents had been doing for more than 6 months, for 1 to 6 months, or had no plans to do. At posttest, respondents also indicated if they had increased their efforts on a variety of GHG related behaviors since their Masters training.
Frequency of Composite Behaviors: Later replications asked respondents how frequently they had done 9 different GHG related behaviors, primarily involving transportation and consumption. Responses varied on a five point scale with 5=always (80 to 100% of the time) and 1=never (0 to 10%). An average value across all 9 behaviors was calculated for each respondent.
Talking with Others: Respondents in all replications were asked, "How often do you find yourself in conversation with family, friends, or colleagues about climate change and strategies for reducing emissions?"
Attitude Questions: Four attitudinal questions were included: a) How well do you feel that you understand climate change? b) How concerned are you about climate change? c) How well do you feel that you understand the ways in which individuals can reduce their contribution to climate change? and d) To what extent do you use climate change as a filter for your daily and broader decisions? Responses ranged on a ten point scale from 1=not at all to 10=completely.
Timing of Composite Behaviors: Control group pretest, n=119; Control group posttest, n=56; CM pretest, n=137; CM posttest, n=78.
Increased effort: Control group, n=43; Climate Masters, n=53.
Frequency of Actions: Control group pretest, n=113; Control group posttest, n=52; CM pretest, n=129; CM posttest, n=77.
Talk with Others: Control group pretest, n=113; Control group posttest, n=52; CM pretest, n=129; CM posttest, n=77.
Attitudes: Control pretest, n=65; Control posttest, n=26; CM pretest, n=68; CM posttest, n=35.
|Did for more than 6 months||9.52||0.002||0.01||0.92||0.01||0.92|
|Did for 1 to 6 months||18.09||<.0001||4.27||0.04||0.58||0.45|
|No plans to do||21.63||<.0001||1.11||0.29||8.60||0.004|
|Frequency of Actions||12.72||0.001||1.50||0.22||0.18||0.67|
|Talk with others re CC||16.95||<.0001||4.94||0.03||0.20||0.66|
|Concerned with CC||0.08||0.78||0.01||0.91||0.19||0.67|
|Understand individuals' role||0.14||0.71||7.88||0.01||9.52||0.002|
|CC is filter for decisions||0.07||0.79||9.73||0.002||3.66||0.06|
|Increased efforts||t = 3.95, df = 94, p (one-tail) = .0001|
Note: Sample sizes are given in Table 1.
The posttest included a number of opportunities for participants to report their reflections on the program and its impact on their lives. The authors independently reviewed the responses and coded the extent to which they included positive, neutral, or negative reports. Almost all comments (90%) were positive. Common responses involved an increased sense of well-being, empowerment, and good health for participants, with some saying the program changed their lives. A complete listing of comments and further detail on all results are available upon request.
Discussion and Conclusion
The data suggest that the CM at Home program not only increased participants' understanding of individuals' roles in reducing household greenhouse gas emissions, but also triggered action. When compared to participants in other Master classes, who had similar initial levels of concern regarding climate change, the CM at Home participants reported greater behavioral change, increased consideration of climate change in daily decisions, and increased understanding of how individuals can affect climate change. We suggest that these findings illustrate how the program had both a relatively immediate impact on participants' lives as well as the potential for promoting ongoing behavioral changes, both for the participants and for those with whom they interact. Future work could explore variations in the types of behavior change that occur and why some types of changes are more likely than others. It would also be important to examine participants' subsequent interactions with others in their communities and results of the volunteer payback period.
The Climate Masters at Home program follows in the tradition of Extension by providing a practical, research-based method to promote healthier living and environmental sustainability. The skills and behaviors learned in the CM at Home program will be crucial as changes in the climate become more manifest. In addition, in times of rising energy costs and economic stress, many of the skills taught in the course provide co-benefits of reductions in household expenditures in areas such as home energy use and transportation.
Given its presence throughout the country and its experience and credibility in translating scientific knowledge for national audiences, Extension is uniquely positioned to support the adoption of sustainable behaviors (Elliott et al., 2008) and the CM at Home program. It is our hope that the experiences with CM at Home described in this article will result in replications of the program in many other locations, with diverse settings and with individuals of diverse backgrounds. Some communities (although none of those described in the study reported here) have replicated the program with a different name in order to share the energy efficiency and conservation practices promoted by the program while avoiding conflict that may arise with a charged topic like climate change. Continued evaluation should, of course, be incorporated in these replications. More information on the CM at Home program, its curriculum and resources for starting a program can be found at www.theresourceinnovationgroup.org.
Primary financial support for conducting the project this article describes was provided by the Environmental Protection Agency, the Meyer Fund for a Sustainable Environment and Eugene Water & Electric Board. We thank Stacy Vynne, Robert Doppelt, and Holly Spencer for comments on earlier drafts and Vanessa Vissar for data management assistance.
Abrahamse, W., Steg L., Vlek C., & Rothengatter T. (2005). A review of intervention studies aimed at household energy conservation. Journal of Environmental Psychology, 25(3), 273-291. doi: 10.1016/j.jenvp.2005.08.002
Attari S .Z., DeKay M. L., Davidson C.I ., & de Bruin W.B. (2010). Public perceptions of energy consumption and savings. Proceedings of the National Academies of Sciences USA 107, 16054-16059. doi:10.1073/pnas.1001509107
Dietz T., Gardner G. T., Gilligan J., Stern P. C., & Vandenbergh M. P. (2009). Household actions can provide a behavioral wedge to rapidly reduce US carbon emissions. Proceedings of the National Academies of Sciences USA 106, 18452-18456. doi: 10.1073/pnas.0908738106
Elliott, C., Hyde, L., McDonell, L., Monroe, M., Rashash, D., Sheftall, W., Simon-Brown, V., Worthley, T., Crosby, G., & Tupas, L. (2008). Sustainable living education: A call to all Extension. Journal of Extension [on-line], 46 (2) Article 2COM1. Available at: http://www.joe.org/joe/2008april/comm1.php
McKenzie-Mohr, D., & Smith, W. (1999). Fostering sustainable behavior: An introduction to community-based social marketing. Gabriola Island, British Columbia: New Society Publishers.
Prochaska, J. O., Diclemente, C. C., & Norcross, J. C. (1992), In search of how people change: Applications to addictive behaviors. American Psychologist, 47(9), 1102-1114. doi: 10.1037/0003-066X.47.9.1102
Prochaska, J. O., & Velicer, W. F. (2004). Integrating Population Smoking Cessation Policies and Programs. Public Health Reports 119, 244-252. doi: 10.1016/j.phr.2004.04.003
U.S. Global Change Research Program (2011). Our changing planet: The U.S. Global Change Research Program for fiscal year 2011. Washington, DC: Author. Retrieved from: http://www.globalchange.gov/publications/our-changing-planet-ocp
U.S. Global Change Research Program (2009). Global climate change impacts in the United States (2009). Washington, DC: Author. Retrieved from: http://www.globalchange.gov/publications/reports/scientific-assessments/us-impacts
University of Rhode Island Cancer Prevention Research Center. Detailed overview of the transtheoretical model. Retrieved August 10, 2011, from: http://www.uri.edu/research/cprc/TTM/detailedoverview.htm
Vandenbergh M. P., Stern P. C., Gardner G. T., Dietz T., & Gilligan, J. M. (2010) Implementing the behavioral wedge: Designing and adopting effective carbon emissions reduction programs. Environmental Law Review, 40, 10547-10554. Retrieved from: http://ssrn.com/abstract=1617426
Weber, E. U., & Stern, P.C. (2011). Public understanding of climate change in the United States. American Psychologist, Vol 66(4), 315-328. doi: 10.1037/a0023253
Whitcomb, J. B. (1991). Water reductions from residential audits. JAWRA Journal of the American Water Resources Association, 27, 761-767. doi: 10.1111/j.1752-1688.1991.tb01474.x