Uncovering Transdisciplinary Team Project Outcomes Through Ripple Effect Mapping

Introduction

Extension teams have reported that the impact levels of programs they deliver to clientele are critically important indicators of success (Kelbaugh & Earnest, 2008). This perspective leads to an important question: How does one best capture and document those impact levels so that team members can accurately gauge their individual and overall program successes?

Impact evaluations, whether expressly naming the logic model as their basis (Rennekamp & Arnold, 2009) or not (Jayaratne, Bradley, & Driscoll, 2009), focus on quantitative documentation of a project's objective-driven intended outcome(s). This approach is very useful when the evaluators are using survey tools and all intended outcomes are known and can be measured—either directly or through a strong indicator.

Critical incident technique is a qualitative method that involves gathering descriptive data through one-on-one storytelling interviews or open-ended questionnaires (O'Neill, 2013). This technique also is a useful, and inexpensive, approach that allows for the discovery of data that may be missed with the use of forced-choice survey methods. The disadvantage derives from possible issues with individual respondents' recall of all relevant facts.

Ripple effect mapping (REM) is also a qualitative method for conducting impact evaluation, but it involves a group brainstorming process. The benefits include interview efficiency (e.g., one vs. many), the capturing of both intended and unintended effects (e.g., positive, negative, and providential), and team motivation (Kollock, Flage, Chazdon, Paine, & Higgins, 2012). REM is often used with community focus groups (Darger, 2014; Emery, Higgins, Chazdon, & Hansen, 2015; Martin, Leuci, & Stewart, 2014). For the effort described in this article, we applied the tool to the Washington State University (WSU) Extension Garden Team (GTeam), a transdisciplinary, geographically dispersed group of Extension educators focused on developing statewide gardening content (Chalker-Scott, Daniels, & Martini, 2016).

We felt the REM approach was appropriate because individual GTeam members may be statewide discipline specialists, county Extension educators, or graduate students; may be tenured or nontenured; may be assigned to a rural area or a densely populated urban area; and may have significant administrative responsibilities in addition to duties associated with developing gardening content. In short, on the basis of geographic locations and unique job descriptions, team members have differing levels of contribution to team projects and different descriptions of "success."

Our purpose in using REM was to simultaneously identify the reach of our program in a way that assisted discussion of future projects and inform team members as to the overall impacts of their individual efforts.

Mapping Process

In 2014 we undertook a 2-day training session to learn how to apply and interpret REM. Prior to the GTeam's 2015 all-hands meeting, the leadership committee sent members a handout describing REM as well as a copy of the 2012 article by Kollock et al.

At the meeting, an activity moderator explained the value of a whole-group brainstorming process—specifically, that one person's statement may trigger another person's memory of a related or different activity/outcome that also should be captured as a project impact. We then posed to team members the question "What outcomes have happened as a result of GTeam formation?" and asked them to reply aloud. The moderator wrote all replies on giant self-stick sheets of paper attached to the wall—so that everyone could see the map as it was being constructed—while the recorder entered the information into XMind, a commercial computer software program. The brainstorming activity concluded at 90 min with an oral recap of the handwritten map by the activity moderator.

As a second step of the mapping exercise, team members were given the instruction "Write one anecdote or testimonial that clearly explains how being a member of the Gardening Team has affected you or your program." Replies could be either positive or negative.

During the event, 14 of 15 team members participated in the mapping exercise and provided testimonials.

Results

As a group, GTeam members identified five major outcomes that had resulted from GTeam formation: a new web page, new collaborations, an increased number of peer-reviewed publications (content), professional development/advancement opportunities, and an improved perspective from administration as regarded being a GTeam member. Every member experienced one or more of the outcomes indicated on the map (Figure 1).

Figure 1.
Ripple Effect Mapping Results of Garden Team Brainstorming Session Using XMind Software

Impacts

Active membership in the GTeam increased the number of new collaborations, both within WSU and with local, state, and federal entities, through a variety of mechanisms. These collaborations resulted in new partnerships, grants, and recognition awards. Peer-reviewed content on the gardening.wsu.edu web page increased, resulting in better support of Extension master gardener volunteers. Many GTeam members oversee local Extension master gardener communities; thus, providing better content does not just generate a sense of satisfaction; it also directly assists team members in their work outside the team. Interestingly, members reported receiving unexpected but significant value in professional development and advancement, through increased opportunities for collaboration on grants and publications, from working in a synergistic atmosphere and attending in-house training on new techniques and educational tools.

Anecdotes about how being a member of the GTeam had affected individuals included the following remarks:

• "Sharing content of the GTeam's work increased local master gardener's awareness of WSU's ongoing effort to bring research-based information to home gardeners. An unanticipated benefit has been to build morale among the [master gardeners], who see it as an investment in providing them with the tools they need . . . they see it as a vote of confidence and that the university values public outreach and their work."
• "I have become a productive publisher of Extension materials. I've developed several beneficial collaborations with other team members, so [I] have felt less isolated and more valued."
• "I have learned how to write peer-reviewed publications and am increasing my personal knowledge of content matter through the research I do to develop these publications."
• "Meeting other committee members and interacting with them makes it easier to make connections on topics other than my own discipline, which allows me to fill gaps."

Conclusions

REM is a broadly useful tool, not only for capturing outcomes of complex projects but also for building esprit de corps among transdisciplinary, geographically dispersed team members. Focusing on the work of their specific committees means that individuals sometimes are unaware of other committees' successes. Although members often assist each other, even across committees, they are not always aware of how significant that contribution may be or how important their active participation in the GTeam is. Participating as a full group in the mapping exercise allowed members to step back and view individual contributions as part of a much larger whole, as well as grasp the full program's impact.

References

Chalker-Scott, L., Daniels, C. H., & Martini, N. (2016). Creating teams increases Extension educator productivity. Journal of Extension, 54(3) Article 3IAW1. Available at: https://joe.org/joe/2016june/iw1.php

Darger, M. (2014). Capturing the ripples from community-driven business retention and expansion programs. Journal of Extension, 52(2) Article 2TOT6. Available at: https://www.joe.org/joe/2014april/tt6.php

Emery, M., Higgins, L., Chazdon, S., & Hansen, D. (2015). Using ripple effect mapping to evaluate program impact: Choosing or combining the methods that work best for you. Journal of Extension, 53(2) Article 2TOT1. Available at: https://www.joe.org/joe/2015april/tt1.php

Jayaratne, K. S. U., Bradley, L. K., & Driscoll, E. A. (2009). Impact evaluation of integrated Extension programs: Lessons learned from the community gardening program. Journal of Extension, 47(3) Article 3TOT3. Available at: https://www.joe.org/joe/2009june/tt3.php

Kelbaugh, B. M., & Earnest, G. W. (2008). Indicators of success for teamwork: What Extension professionals need to excel as team members. Journal of Extension, 46(4) Article 4FEA6. Available at: https://www.joe.org/joe/2008august/a6.php

Kollock, D. H., Flage, L., Chazdon, S., Paine, N., & Higgins, L. (2012). Ripple effect mapping: A "radiant" way to capture program impacts. Journal of Extension, 50(5) Article 5TOT6. Available at: https://www.joe.org/joe/2012october/tt6.php

Martin, M. J., Leuci, M., & Stewart, M. (2014). Challenges of engaging local stakeholders for statewide program development process. Journal of Extension, 52(1) Article 1TOT8. Available at: https://joe.org/joe/2014february/tt8.php

O'Neill, B. (2013). Assessing program impact with the critical incident technique. Journal of Extension, 51(3) Article 3TOT2. Available at: https://www.joe.org/joe/2013june/tt2.php

Rennekamp, R. A., & Arnold, M. E. (2009). What progress, program evaluation? Reflections on a quarter-century of Extension evaluation practice. Journal of Extension, 47(3) Article 3COM1. Available at: https://www.joe.org/joe/2009june/comm1.php