Using Importance-Performance Analysis to Guide Extension Needs Assessment

Introduction

Sound needs assessment and audience analysis activities are critical to impactful Extension programming (Boone, Safrit, & Jones, 2002; Seevers & Graham, 2012). Mass messaging approaches broadly directed toward the general population are likely to be less effective than tailored strategies based on individual traits and preferences (Dancker, Staats, Wilke, & Engelen, 2001; Syme, Nancarrow, & Seligman, 2000). This article addresses the application of a tool used in recreation sciences to Extension needs assessment in the context of residential landscape water conservation programming.

Importance-performance analysis (IPA) is a quantitative approach for measuring how people feel about certain characteristics of an issue or a thing (Martilla & James, 1977). Often, this technique is used to evaluate various attributes of recreational facilities or destinations (Hammitt, Bixler, & Noe, 1996; Oh, 2001). A benefit of IPA is that it generates a clear picture of how important certain elements are in comparison with how satisfying they are to clients or customers (Levenburg & Magal, 2004; Siniscalchi, Beale, & Fortuna, 2008).

The visual output of this method, an IPA matrix, is created by plotting individual attributes' importance values and satisfaction values on a two-dimensional graph having four quadrants (Hugo & Lacher, 2014; Levenburg & Magal, 2004; Martilla & James, 1977; Siniscalchi et al., 2008). Importance and satisfaction are each measured through the use of a Likert-type scale, and the parameters of the matrix's quadrants are based on the means of the two measures. In a traditional interpretation of IPA, each quadrant is interpreted as having implications for prioritization and management of attributes (Figure 1). Resources need to be focused on elements in the "Concentrate here" (high importance and low satisfaction) quadrant, or clients will be lost; resources should continue to be focused on the "Keep up the good work" (high importance and high satisfaction) quadrant to maintain client satisfaction; and resources can be allocated away from the "Lower priority" (low importance and low satisfaction) and "Possible overkill" (low importance and high satisfaction) quadrants (Hugo & Lacher, 2014; Levenburg & Magal, 2004; Martilla & James, 1977; Siniscalchi et al., 2008).

Figure 1.
Importance-Performance Matrix

Note. Adapted from "Importance-Performance Analysis," by J. A. Martilla and J. C. James, 1977, Journal of Marketing, 10(1), p. 78. Copyright 1977 by the American Marketing Association.

A benefit of IPA is that it allows the practitioner to identify elements that clients perceive as important but have dissatisfaction about. Attributes falling into quadrant 1 are considered the highest priority. For example, Hugo and Lacher (2014) studied the attributes of community festivals by using IPA and found that the cultural aspect of the festivals (i.e., culture showcased through arts and crafts, food, and entertainment) was not important to festival attendees. However, there were satisfaction gaps related to relaxation and entertainment elements of the festivals that needed to be improved to encourage continued participation in the events.

Applying IPA to Extension Messaging

Well-structured and relevant communications can activate recipients' minds in ways that lead to environmentally responsible actions (Dancker et al., 2001; Scannell & Gifford, 2013). For example, one study demonstrated that specific message frames elicited change in certain predictors related to intent to engage in conservation behaviors in the context of home landscape water use (Warner, Rumble, Martin, Lamm, & Cantrell, 2015). More generally, research has indicated that Extension professionals can promote engagement and action on an issue by communicating on the basis of what is relevant and important to the audience (Robinson, 2013). An individual who is unsatisfied with something that is important to him or her may be motivated to resolve the situation by taking action to improve it (Harmon-Jones & Harmon-Jones, 2008; Harmon-Jones & Mills, 1999).

Expanding on the aforementioned findings, we applied the concept of IPA to the context of Extension communications, considering the method to be a possible way to identify the most salient dimensions of an issue for a target audience and to guide the development of meaningful corresponding messages. Specifically, we explored the idea of using IPA to develop messages that would be salient for a target audience by assigning priority to communicating about elements clustered in the "Concentrate here" quadrant, that is, elements that represent conflicting levels of importance and satisfaction in members of the target audience.

The objective of our research was to explore the potential application of IPA to water conservation Extension programs. Consequently, the study addressed the use of IPA in the context of residential landscape water conservation. However, although IPA has not previously been used in this way, we propose that Extension professionals can use it broadly to assess motivating factors for target audiences for the purpose of selecting possible messaging and communication strategies.

Methodology

We used IPA methodology to assess the levels of importance and satisfaction respondents felt related to the existence of clean and plentiful water for various purposes. We collected data by using a researcher-developed electronic survey instrument with a convenience sample of residents in Alachua County, Florida. We invited participants who were engaged in a separate irrigation-use study to participate in our study. The instrument first screened individuals to ensure that they lived in homes with irrigated lawn or landscape and that they had responsibility for irrigation decisions. Among 351 individuals who opted to participate, 133 were eligible to complete the survey, and 79 complete responses were received, for a completion rate of 59%.

According to the literature on IPA, individual statements should be plotted on IPA matrices (Hugo & Lacher, 2014; Martilla & James, 1977; Siniscalchi et al., 2008). However, best practices in survey methodology generally involve developing indexes formed by a number of individual variables. Therefore, we developed six indexes each for clean water and plentiful water, using 38 individual statements (Table 1). Respondents were instructed to identify the levels of importance they associated with 19 items related to clean and plentiful water. The response set for each item was a 5-point Likert-type scale, with possible response options ranging from 1 (not at all important) to 5 (extremely important). Respondents also were instructed to identify the levels of satisfaction they associated with the same set of 19 items. The response set for each item was a 5-point Likert-type scale, with possible response options ranging from 1 (not at all satisfied) to 5 (extremely satisfied). We calculated mean index scores (six importance scores and six satisfaction scores) by averaging the individual statements within each index. In a prior study, we had established the face and content validity and reliability of the instrument (Warner, Ali, & Kumar Chaudhary, 2015).

Table 1.
Water Conservation Indexes

Index name	Individual items in index
Importance of clean water for local and large water bodies/Satisfaction with clean water for local and large water bodies	Clean lakes, springs, rivers Clean oceans Clean bays and estuaries
Importance of clean water for recreation/Satisfaction with clean water for recreation	Clean water for shellfishing Clean beaches Clean water for recreation
Importance of clean water for consumption/Satisfaction with clean water for consumption	Clean drinking water Clean groundwater Clean water for food preparation
Importance of plentiful water for business/Satisfaction with plentiful water for business	Plentiful water for commerce and industry Plentiful water for power Plentiful water for agriculture
Importance of plentiful water for people/Satisfaction with plentiful water for people	Plentiful water for household landscapes Plentiful water for cities Plentiful water for golf courses Plentiful water for recreation
Importance of plentiful water in local water bodies/Satisfaction with plentiful water in local water bodies	Plentiful water in aquifers and springs Plentiful water in rivers Plentiful water in lakes
Note. Each pair of indexes by the same name used the same set of individual statements with a different question stem corresponding to either importance or satisfaction.

To apply IPA to Extension communications, we developed a guide for interpreting the quadrants around different dimensions of an issue (Figure 2). We considered items with high importance and low satisfaction to be "target motivational areas," which should be addressed when communicating with an Extension audience. We plotted the mean importance and satisfaction values for each index on a matrix where satisfaction comprised the x-axis and importance comprised the y-axis. We divided the matrix into four quadrants by using the grand mean score for importance (19 statements) and the grand mean score for satisfaction (19 statements) (Hugo & Lacher, 2014).

Figure 2.
New IPA Interpretation Matrix

Note. Adapted from "Importance-Performance Analysis," by J. A. Martilla and J. C. James, 1977, Journal of Marketing, 10(1), p. 78. Copyright 1977 by the American Marketing Association.

Results

The overall importance means ranged from 2.97 to 4.81 across the indexes, and the overall satisfaction means ranged from 2.85 to 3.40 across the indexes (Table 2). The overall grand means for importance and satisfaction were 3.82 and 3.37, respectively. As noted previously, we used the grand means to establish the parameters of four quadrants and plotted the mean values for the indexes on the resulting matrix (Figure 3). The data points that fell into the "Target motivational areas" quadrant were for the following indexes: "Importance of/satisfaction with clean water for local and large water bodies," "Importance of/satisfaction with clean water for recreation," and "Importance of/satisfaction with plentiful water in local water bodies."

Table 2.
Importance and Satisfaction Means for Clean and Plentiful Water

Quadrant	Index	Importance (N = 79) M (SD)	Satisfaction (N = 75) M (SD)
Maintain focus on these issues	Clean water for consumption	4.81 (0.52)	3.40 (0.98)
Target motivational areas	Clean water for local and large water bodies	4.61 (0.65)	2.99 (0.79)
	Clean water for recreation	4.52 (0.65)	2.91 (0.76)
	Plentiful water in local water bodies	4.48 (0.57)	2.85 (1.00)
Low interest to the target audience	Plentiful water for business	3.76 (0.75)	3.07 (0.86)
	Plentiful water for people	2.97 (0.69)	3.17 (0.95)
Note. Importance scale: 1 = not at all important to 5 = extremely important. Satisfaction scale: 1 = not at all satisfied to 5 = extremely satisfied.

Figure 3.
Importance-Performance Analysis for Clean and Plentiful Water

Note. The data point labels denote indexes: A—Clean water for consumption, B—Clean water for local and large water bodies, C—Clean water for recreation, D—Plentiful water in local water bodies, E—Plentiful water for business, F—Plentiful water for people.

Discussion, Conclusions, and Recommendations

We present this application of IPA as a means of selecting dimensions of water-related issues that are likely to be salient with an Extension audience, and we used nonprobability data to demonstrate what could emerge from a study involving this method. Participants were already participating in a larger irrigation-use study, and, therefore, respondents may have perceived higher levels of importance for water issues than the general public would have. As this was a convenience sample, results should not be applied to any larger population. However, the findings demonstrate the utility of using IPA.

In communications with this audience, an Extension professional would be advised to encourage behavior change by conveying ways in which audience members' water conservation actions relate to clean water for local and large water bodies, clean water for recreation, or plentiful water in local water bodies. The Extension professional looking to connect with this audience could associate the adoption of landscape water conservation practices to the ocean, local springs, or water-based recreation. In the current environment of limited resources, this approach provides a strategic way to prioritize efforts and resources in a broad variety of contexts. Considering importance and satisfaction together provides a holistic picture for Extension educators' communication decisions.

This audience assigned low importance and low satisfaction to the concepts of plentiful water for business and plentiful water for people. We interpreted this result to mean that communications around these areas would be less effective. One interesting and positive finding was that no item was deemed to be associated with low importance and high satisfaction (the qualities related to the "Take focus off of these issues" quadrant), which we interpreted to mean that this audience placed substantial value on water, concluding that no dimension of water-related issues should necessarily be de-emphasized.

Further research should involve using a random sample to describe a specific target audience, such as residents who use home landscape irrigation in a specific geographical area. If future results from probability samples are similar to those presented in this article, all of the indexes explored in our study could, therefore, be relevant to Extension communications about water conservation. We hypothesize that IPA could reveal differences in communication needs on the basis of people's experiences with water-quality and water-quantity issues. Also, future research should incorporate message testing to address the effect of communications tailored to an audience's motivating issues on the basis of this application of IPA. We hypothesize that messages tailored to the target motivational areas would resonate more strongly among target audiences than nontailored messages would.

IPA is a robust methodology that, along with traditional needs assessment activities, can be used by Extension professionals to guide communications. Across the country, Extension is challenged to increase accountability despite reduced funding (Franz & Townson, 2008; Peters & Franz, 2012). IPA can help Extension professionals prioritize communications and guide the use of messages that resonate well with their target audiences. While designing programs for water conservation, Extension professionals should consider importance and satisfaction together, using IPA methodology. Extension professionals should consider using IPA to map audience perceptions of an issue and decide which frames or messages may be more motivating for their clientele. The application described here occurred in the context of residential landscape water conservation programming, yet the potential application is broad. Measuring the importance and satisfaction Extension clients associate with characteristics of different issues may help Extension professionals communicate using the concepts most likely to motivate target actions.

Acknowledgments

This research was supported by the University of Florida Center for Landscape Conservation and Ecology.

References

Boone, E. J., Safrit, R. D., & Jones, L. (2002). Developing programs in adult education: A conceptual programming model (2nd ed.). Long Grove, IL: Waveland Press.

Dancker, D. L. Staats, H., Wilke, H. A. M., & Engelen, M. (2001). Improving environmental behavior in companies: The effectiveness of tailored versus nontailored interventions. Environment and Behavior, 32(2), 229–248. doi:10.1177/00139160121972963

Franz, N., & Townson, L. (2008). The nature of complex organizations: The case of Cooperative Extension. In M. Braverman, M. Engle, M. E. Arnold, & R. A. Rennekamp (Eds.), Program evaluation in a complex organizational system: Lessons from Cooperative Extension (pp. 5–14). New Directions for Evaluation, 120. San Francisco, CA: Jossey-Bass.

Hammitt, W. E., Bixler, R. D., & Noe, F. P. (1996). Going beyond importance-performance analysis to analyze the observance influence of park impacts. Journal of Park and Recreation Administration, 14(1), 45–62.

Harmon-Jones, E., & Harmon-Jones, C. (2008). Cognitive dissonance theory: An update with a focus on the action-based model. In J. Y. Shah & W. L. Gardner (Eds.), Handbook of motivation science (pp. 71–83). New York, NY: Guilford Press.

Harmon-Jones, E., & Mills, J. (1999). An introduction to cognitive dissonance theory and an overview of current perspectives on the theory. In E. Harmon-Jones & J. Mills (Eds.), Cognitive dissonance: Progress on a pivotal theory in social psychology (pp. 3–21). Washington, DC: American Psychological Association.

Hugo, N. C., & Lacher, R. G. (2014). Understanding the role of culture and heritage in community festivals: An importance-performance analysis. Journal of Extension, 52(5) Article 5RIB4. Available at: http://www.joe.org/joe/2014october/rb4.php

Levenburg, N. M., & Magal, S. R. (2004). Applying importance-performance analysis to evaluate e-business strategies among small firms. E-service Journal, 3(3), 29–48.

Martilla, J. A., & James, J. C. (1977). Importance-performance analysis. Journal of Marketing, 10(1), 13–22.

Oh, H. (2001). Revisiting importance-performance analysis. Tourism Management, 22(6), 617–627.

Peters, S., & Franz, N. K. (2012). Stories and storytelling in Extension work. Journal of Extension, 50(4) Article 4FEA1. Available at: http://www.joe.org/joe/2012august/a1.php

Robinson, P. (2013). Effectively communicating science to Extension audiences. Journal of Extension, 51(2) Article 1IAW1. Available at: http://www.joe.org/joe/2013april/iw1.php

Scannell, L., & Gifford, R. (2013). Personally relevant climate change: The role of place attachment and local versus global message framing in engagement. Environment and Behavior, 45(1), 60–85. doi:10.1177/0013916511421196

Seevers, B., & Graham, D. (2012). Education through Cooperative Extension (3rd ed.). Fayetteville, AR: University of Arkansas.

Siniscalchi, J. M., Beale, E. K., & Fortuna, A. (2008). Using importance-performance analysis to evaluate training. Performance Improvement, 47(10), 30–35. doi:10.1002/pfi

Syme, G. J., Nancarrow, B. E., & Seligman, C. (2000). The evaluation of information campaigns to promote voluntary household water conservation. Evaluation Review, 24, 539–578. doi:10.1177/0013916514543683

Warner, L. A., Ali, A. D., & Kumar Chaudhary, A. (2015). [Scale development: importance-performance analysis applied to dimensions of water]. Unpublished raw data.

Warner, L. A., Rumble, J. N., Martin, E., Lamm, A. J., & Cantrell, R. A. (2015). The effect of strategic message selection on residents' intent to conserve water in the landscape. Journal of Agricultural Education, 56(4), 59–74. doi:10.5032/jae.2015.04059