Using Ripple Effect Mapping to Evaluate Program Impact: Choosing or Combining the Methods That Work Best for You

Introduction

A number of practitioners, particularly in community development and youth programming/4H, are using Ripple Effect Mapping (REM), a kind of group mind mapping, to engage participants in a participatory process to tease out outcomes and impacts, identify unintended consequences, and determine next steps (Darger, 2014; Baker, Calvert, Emery, Enfield, & Williams, 2011; Hansen et al., 2012; Nathaniel & Kinsey, 2013). This article reports on the results of a workshop on different approaches to REM presented at the recent National Association of Community Development Extension Professionals conference.

All three approaches use Appreciative Inquiry to focus participants on program successes by having them spend time discussing several guiding questions in pairs before creating the ripple map. Brainstorming success stories in pairs elicits a multitude of outcomes and allows everyone time to adequately reflect on program activities. Each approach also uses the Community Capitals Framework (CCF), although in different ways. Finally, all three are effective reflection tools.

Each approach, however, yields different data, and each fosters a different conversation. Because there are many ways to facilitate group processes, as well as different skill sets among facilitators, we have begun to see the emergence of distinct ways to conduct REM evaluations.

The Three Approaches

For comparison purposes, the three approaches to ripple mapping are described as:

1. "Web mapping," where the group session examines short-term, medium-term, and long-term impacts and maps them directly onto a community capitals mind map;
2. "In-depth rippling," where the group session focuses on the deepest and most impactful chains of events; and
3. "Theming and rippling," where the group session captures the breadth of reporting impacts from all participants, generates impact themes, and examines ripples once themes are generated.

The discussion below provides more explanation of the three approaches and their differences.

The first author began using group mind mapping techniques in 2006. As a leader in using the Community Capitals Framework (Emery & Flora, 2006), she begins the REM process with the community capitals in order to encourage participants to consider how change in one community sector can result in changes across the community as manifested by changes in multiple capitals. After an initial Appreciative Inquiry exercise, the session follows a logic model structure in moving from discussion and mapping of short-term outcomes (what are people doing differently?), to medium-term outcomes (how are these changes benefitting others or changing what others do?), to impacts (what is different in the community today?).

Practitioners have used the Web mapping approach for both formative and summative evaluation purposes. This approach is especially useful in identifying opportunities to strengthen program outcomes by including an intentional focus on how elements of the strategy, project, or program can be tweaked to take advantage of opportunities to build assets in the intangible capitals—social, cultural, political, and human.

The in-depth rippling approach emerged as an evaluation strategy for a community leadership program delivered in several states. After the initial Appreciative Inquiry exercise, the facilitator asks participants to volunteer stories, using a large sheet of butcher paper to record the conversation. Prompts such as "And then what happened," "How many people were involved," or "What was the dollar amount of the grant you received?" encourage participants to share their stories and outcomes, creating a rich and detailed narrative describing the project. Participants are invited to add to each other's reports, eliciting stories that emerge from focusing on a particular outcome. Each set of follow-up stories creates the ripples. Developing the map as stories unfold allows participants to control themes and see resulting ripples. They often see trends in their work. For example, one community noticed their most successful efforts started with more investments in human and social capital. Maps can be simultaneously digitized on mapping software such as Xmind, or digitized after the exercise.

The theming and rippling approach is an adaptation of in-depth rippling where all participants are asked to report out results of their Appreciative Inquiry interviews. The reported items are typed directly into mind mapping software as displayed as "floating topics." Once all the floating topics have been posted, the facilitators work with the group to organize items into core themes. The core themes are useful for later sharing of the mind map, allowing the facilitators to more easily simplify it with a focus on major themes and examples of impacts in each theme. Then facilitators go back to original reported impacts and ask prompts to encourage participants to fill in causal chains leading to and resulting from the original reported items. As part of this approach, facilitators also "probe for negatives" near the end of the session. This often leads to thoughtful discussion of not-so-positive events that may have occurred as a result of the intervention, as well as responses to identified challenges to date.

In both the in-depth rippling and theming and rippling approaches, reported impacts are coded based on the CCF after the fact, rather than during the mapping session.

All variations on the mapping process also engage program participants and enhance their sense of self-efficacy and particularly the sense of collective agency. Creating a visual map that depicts the whole range of impacts of their actions can be enlightening and empowering.

Conclusions

Ripple Effect Mapping fits well in the developmental evaluation toolkit. It is truly an emergent evaluation method well suited for the complex evaluation situations we often face in Extension work (Patton, 2011). As more Extension staff become trained and proficient in Ripple Effect Mapping, we will likely to see more variations in approach. It is important for Extension professionals to continue sharing these, as well as their reasons for variations, although we recommend that variations retain the CCF as a coding scheme and common metric for measuring impact of community development programs. Our goal is to ensure that Ripple Effect Mapping maintains its integrity as a participatory group method for impact evaluation that builds upon strong facilitation skills and meets the needs of program participants and other stakeholders.

References

Baker, B., Calvert, M., Emery, M., Enfield, R., & Williams, B. (2011). Mapping the impact of youth on community development: What are we learning? [PowerPoint slides]. Retrieved from: https://connect.msu.edu/p72418074/?launcher=false&fcsContent=true&pbMode=normal

Darger, M. (2014). Capturing the ripples form community-driven business retention and expansion programs. Journal of Extension [On-line], 52(2) Article 2TOT6. Available at: http://www.joe.org/joe/2014april/tt6.php

Dart, J., & Davies, R. (2003). A dialogical, story-based evaluation tool: The most significant change technique. American Journal of Evaluation. 24(2), 137-155.

Emery, M., & Flora, C. B. (2006). Spiraling-up: Mapping community transformation with Community Capitals Framework. Community Development: Journal of the Community Development Society 37(1), 19-35.

Hansen Kollock, D. A., Flage, L., Chazdon, S., Paine, N., & Higgins, L. (2012). Ripple effect mapping: A "radiant" way to capture program impacts. Journal of Extension [On-line], 50(5) Article 5TOT6. Available at: http://www.joe.org/joe/2012october/tt6.php

Nathaniel, K., & Kinsey, S. (2013). Contributions of youth engagement to the development of social capital through community mapping. Journal of Extension [On-line], 51(1) Article 1TOT7. Available at: http://www.joe.org/joe/2013february/tt7.php

Patton, M. Q. (2011). Developmental evaluation applying complexity concepts to enhance innovation and use. New York: The Guilford Press.