Measuring and Benchmarking Customer Satisfaction: Implications for Organizational and Stakeholder Accountability

Background

Cooperative Extension, like many public agencies, has seen an increased emphasis on measuring quality of programs through customer satisfaction surveys (CSS). Customer satisfaction is becoming an important part of the culture of many organizations (Ladewig, 1997). Customer satisfaction provides for better understanding of services provided by Extension from the customers' perspective. In addition, it provides for better understanding of expectations of customers and the extent to which an organization is satisfying the needs and wants of its customers (Biggs, Gordan, & Zimmerman, 1995).

Customer satisfaction also serves as a link between the customers and a performance reward system, thus providing rewards for desired outcomes and identifying consequences for undesired outcomes (Cummings & Ladewig, 1998). According to Merritt (1998), customer satisfaction is an indicator of the quality of Extension programs and is critical for evaluation of the effectiveness of the organization.

Israel (1998) indicated that customer satisfaction surveys provide a number of benefits for Extension. First, they tell us what differences our programs are making in the communities. Second, they serve as a mechanism to show our supporters and critics that our customers have a high level of satisfaction and they are using Extension information. Third, they help identify strengths and weaknesses of Extension programs so that continuous improvements can be made. Finally, they help to showcase successful programs in the annual report of accomplishment.

In recent years, several states have conducted Customer Satisfaction Surveys (CSS) to assess quality of Extension programs. Prominent among the states were Florida, Kentucky, and Texas. Consensus from these studies indicated that Extension clientele are very satisfied with the delivery and content of the information provided.

Provisions of the 1993 Government Performance Results Act (GPRA) included the use of customer satisfaction as a key component of performance measurement. The South Carolina Budget Control Board (SCBC), in 1998, recommended that all public agencies in South Carolina periodically measure customers' satisfaction through obtaining information from the customers they serve. Berrio and Henderson (1998) suggested that Extension organizations should conduct customer satisfaction surveys as an effort to assess the performance of the services from the customer perspective.

Purpose and Objectives

In the summer and fall of 1999, Clemson Cooperative Extension Service conducted a Customer Satisfaction Survey to determine what recipients of Extension programs felt about the quality of services provided. Specifically, the study was designed to assess the:

1. Relevance and quality of programs delivered by Clemson Extension Service,
2. Extent to which clients have used and shared the information, and
3. Satisfaction with the services provided by Clemson Extension Service.

In addition, an attempt was made to benchmark the Clemson CSS study with other states (Florida and Texas) that have conducted similar customer satisfaction studies.

CSS Process and Procedures

In the following paragraphs, an overview of Clemson Extension's programs and client contact information for year 1999-2000 is described. In addition, the process and procedures involved in conducting the study are also described.

Clemson University Cooperative Extension Service offers educational programs in five different areas: agrisystems productivity and profitability, community and economic development, environmental conservation, food safety and nutrition, and youth development. These five areas mirror the Government Performance Results Act (1993) goals developed by USDA-CSREES.

Extension agents in the counties conduct a variety of activities and educational programs to create awareness, knowledge, skills, and behaviors to bring about desired changes in the clientele they serve. Extension agents provide research-based information to the people to help them make informed decisions about problems that they face.

Extension staff contacted 732,126 people in 1999-2000. Figure 1 shows that 40.1% of these contacts were White males, followed by 31.4% White females, 12.7% African-American males, 14.9% African-American females, and 1.0% others. Extension agents also conducted over 19,800 educational programs and activities in 1999-2000. Activities and programs included meetings, workshops, demonstrations, field visits, hands-on experiences, and other activities.

Figure 1.
Total Contacts by Race and Gender, 1999-2000

Each of the 46 counties in the state was asked to develop a list of customers who participated in county Extension programs. Counties were advised to use lists such as program registration, sigh-up sheets, office visits, attendance at field days, and demonstrations, etc. Once the list of customers was developed, a team of Extension personnel (agents, county Extension directors, and administrative assistants) randomly selected 30A total of 1,380 customers were identified as subjects for the study. However, only 42 of the 46 counties agreed to participate in the study, resulting in 1,260 subjects.

A customer satisfaction questionnaire similar to the one developed by the University of Florida was modified and used for the study. The questionnaire contained 14 questions that were grouped into:

1. Type of contact,
2. Extension information use,
3. Satisfaction with Extension Service, and
4. Demographic information (age, gender, race, education level, occupation, and number of years using Extension Service for information).

In addition, the questionnaire also recorded information on client address and telephone number, and date of data collection. The questionnaire was evaluated for content validity by a panel of three experts consisting of Extension specialists and agents.

Prior to distributing the questionnaires to the counties, the entire process of data collection was shared with the 14 cluster directors. A detailed explanation on selecting respondents, data collection process, and other logistics was presented to the cluster directors. After addressing several questions and other issues concerning the study and review of the questions, cluster directors agreed to collect data. Counties collected data in summer and fall of 1999. A total of 1,068 completed questionnaires were returned (84.7%).

Counties were asked to send the completed questionnaires to the Extension Staff Development office for analysis. Completed questionnaire sent by counties were checked, coded, and entered into a database. SPSS program for Windows was used to analyze the data. Frequencies, means, and percentages were used to summarize the data.

Because the sampling was not scientific, two demographic variables (gender and race) were compared with Clemson Extension Service contacts data (through CUMIS) and South Carolina population estimates (1998). Both gender and race variables were representative of the state's population (Table 1).

Table 1.
Comparison of CSS Sample with CUMIS Contact Data and South Carolina Population Estimates

Findings

Demographic Information

As indicated earlier, a total of 1,068 completed surveys were included in the analysis. The demographic information revealed that 32% of the respondents lived on a farm, 30% in a rural area, 33% in towns and cities under 50,000 population, while 5% lived in cities with a population of over 50,000. Fifty-four percent of those responding were female and 75% white Caucasian, 24% black, and 1% other (Hispanic, Asian). Regarding age, 14% were under the age of 35 years, 43% between ages 36-45, 22% between 56-65, and the remaining 21% were over 66 years. Examination of education level of respondents revealed that 23% had at least a high school education or GED, 48% had completed college, 20% had completed post-graduation degrees. Only 9% reported less than a high school education. On an average, respondents have been using Extension Service for information for over 16 years.

Relevance and Quality of Extension Information:

As shown in Figure 2, customers who participated in Clemson Extension programs or visited the Extension office for information were very impressed by the quality of information they received relative to food safety, water quality, crops, financial management, community development, youth development, and others. Ninety-seven percent said that the information presented was up-to-date and accurate, useful, and easy to understand. Ninety-six percent found it to be relevant to their situation.

Figure 2
Relevance and Quality of Extension Programs/Information

Extension Information Use

Eighty-two percent of the respondents indicated that they have used the information (Figure 3). Seventy-six percent of those who had used the information said that it had solved their problems or answered the questions. However, 21% said "don't know" because they are still in the process of using the information they had received from participating in Extension programs.

Figure 3.
Extension Information Use

This is what those who used the information said:

• "Followed directions, continue to do so and fire ant mounds are non-existent in two acres."
  
  
• "I have used the information to determine sodium contents of foods since I need to watch sodium in my diet."
  
  
• "Recently had a Will made because of the program, and the information helped out a great deal."
  
  
• "Information helped to decide what variety of corn is suitable for our farm from Extension's 'Corn Letter'—same for vegetables. Also, soil sample results are a great help each year."
  
  
• "Information referring to horticultural practices and pesticide management very helpful."
  
  
• "I started being more careful about shopping and reading labels, and putting food away when I have finished cooking."
  
  
• "I was interested in public speaking and the 4-H office gave me a course. I later won the 4-H State Public Speaking Contest."

Seventy-nine percent of the participants said that they have shared the information with their friends and neighbors.

Figure 4.
Shared Information with Someone

In response to the question, "How do you feel about the services provided by Clemson Extension?" 78% (N=834) said that they were "very satisfied" with the service, 20% said they were "satisfied," less than 1% said they were "dissatisfied," and 1% of the respondents had "no opinion." (Figure 5)

Figure 5.
Satisfaction with Clemson Extension Service

According to David Kearns, CEO of Xerox Company, benchmark is the continuous process of measuring products, services, and practices against the toughest competitors or those recognized as industry leaders. Ladewig (1998) suggested that Cooperative Extension Service develop benchmarks on three major aspects of Extension: programs—relevance, quality, and accomplishments.

In this study, we have made an attempt to benchmark "quality" of Extension programs by measuring and comparing customer satisfaction. Two Extension organizations in the Southern Region (University of Florida Extension Service and Texas A & M Extension Service) were used as benchmarks to compare with Clemson CSS.

As shown in Table 2, data from the three organizations reveal somewhat similar findings. Overall, customers, regardless of the state or the type of Extension program they attended, are very satisfied with the service provided by Extension. Satisfaction was very high in all the three states, with Texas A & M having the highest (82%), followed by University of Florida (80%), and Clemson (78%). Extension information in terms of accuracy, usefulness, ease of understanding, and being up-to-date was high for Clemson Extension, followed by University of Florida and Texas A & M. A higher percentage of Clemson customers had the opportunity to use and share the information when compared to University of Florida customers.

Table 2.
Benchmarking of Clemson CSS Study with University of Florida and Texas A & M

Conclusions and Discussion

Extension customers are very satisfied with the information they received from Clemson Extension offices in the counties. They indicated that the information was accurate, up-to-date, and relevant to their situations. In addition, customers indicated that they made use of the information to solve the problem or answer a question regarding an issue or a concern. This positive use of information can be linked to not only knowledge and skills acquisition, but also inclination toward adopting a recommended practice.

Customers also indicated that they shared the information with their friends and neighbors, indicating the confidence and trust they have for Extension programs and the information received. The findings of this study are more or less similar to the studies conducted by University of Florida and Texas A & M University.

The feedback from this survey has helped agents to continue to improve delivery of Extension programs. Data from the customer satisfaction study has provided a wealth of information for agents on how to improve programs and meet the needs of the clientele they serve. Cluster directors have used data from customer satisfaction surveys for justification of county budgets and to show stakeholders that Extension programs do make a difference.

Furthermore, the successful completion of the customer satisfaction survey demonstrated to the legislators that Cooperative Extension is willing to ask customers how it is doing, and then act on their suggestions for improvement. And Clemson Extension Service has demonstrated that it can fulfill the requirements and/or mandates of the South Carolina Budget Control Board. Finally, the results of benchmarking have provided a basis for comparing Clemson Extension with its counterparts relative to program performance and accountability.

The process used in conducting CSS could have immense value to other states and Extension professionals interested in assessing customer satisfaction. States could modify the questions and/or the process to suit their needs. In addition, the states can also decide on the best way of identifying customers for accurate assessment.

Extension administrators and program evaluation specialists in each state or Extension region should come together and develop a "Generic Customer Satisfaction Survey" and a process suitable to all states and regions. However, the proposed survey should also consider the needs of individual states and Extension regions. Such an effort will allow for comparison of efforts and sharing of resources.

References

Berrio, A. A., & Henderson, J. L. (1998). Assessing customer orientation in public non-profit organizations: A Profile of Ohio State University Extension. Journal of Agricultural Education, 29(4), 11-17.

Biggs, E., Gordan, J., & Zimmerman, S. (1995). How to plan and conduct a customer satisfaction survey. MERIT Seminar Series: Praia Group.

Cummings, S. R., (1998). Measuring customer satisfaction: How to get the most of extension programs. Unpublished manuscript, Department of Agricultural Education, Texas A & M University, College Station, TX.

Israel, G.D. (1998). Conducting a customer satisfaction survey. Florida Cooperative Extension Service, Fact Sheet PDE-98-05, University of Florida.

Israel, G. D., & Fugate, A. M. (1998). Extension helps residents; customer satisfaction high. Florida Cooperative Extension Service, Fact Sheet PDE-98-06, University of Florida.

Ladewig, H. (1998). Organizational accountability and program evaluation in the Cooperative Extension System. Unpublished manuscript, Department of Agricultural Education, Texas A & M University, College Station, TX.

Ladewig, H. (1997). Demonstrating accountability through collaboration and partnerships. Paper presented to the Joint Southern Region Program Committee Meeting, Tallahassee, Florida.

Merritt, H. (1998). Getting started with performance measurement. Paper presented at the South Carolina State Government Quality Network Association Conference, Columbia, SC.

South Carolina State Government Quality Network Association (1998). Changing the face of accountability: Performance measurement in government. South Carolina Budget Control Board, Columbia, SC.

South Carolina State Government Quality Network Association (1998). Measurement Development Guide. South Carolina Budget Control Board, Columbia, SC.