A Survey-Based Model for Collecting Stakeholder Input at a Land-Grant University

Introduction

Stakeholders have become increasingly estranged from land-grant universities, evidenced by reduced support from state and federal legislatures (Silag, Schultz, Bishop, Dale, & King, 1998). Land-grant university presidents have also noted a pattern of disengagement (Kellogg Commission, 1999). To stem the tide of further public disenfranchisement, the United States Congress legislated that stakeholder input be gathered by agricultural colleges when determining priorities for Extension (1998 Farm Bill, Public Law 105-185).

The purpose of the research described here was to develop a survey-based model for collecting stakeholder input for land-grant universities that could be adopted by Extension specialists and educators for soliciting necessary input from their stakeholders.

Calls for greater engagement between public universities and citizens emerged in the early 1980s and climaxed with two important documents, the 1998 Farm Bill and the Kellogg Commission on the Future of State and Land-Grant Universities (1999) report. The 1998 Farm Bill stated that stakeholder input must be collected when setting research, Extension and education priorities. Section 102, titled "Priority Setting Process," specifically stated (italics added):

Effective October 1, 1999, to obtain agricultural research, extension, or education formula funds from the Secretary, each 1862 Institution, 1890 Institution, and 1994 Institution shall establish and implement a process for obtaining input from persons who conduct or use agricultural research, extension, or education concerning the use of the funds.

The "bill of particulars" provided by the Kellogg report (1999, p. 4) included public perceptions that university and faculty work has become "slow and unwieldy, so intent on studying things to death that it is impossible to get timely decisions or responses out of them." The report stated that "[faculty] are so inflexibly driven by disciplinary needs and concepts that they have lost sight of the institutional mission to address the contemporary multidisciplinary problems of the real world." Without changes in the way land-grant universities include stakeholders, citizens may continue to express "dissatisfaction with the status quo and a sense of impending crisis" regarding the land-grant university (Silag, et al., 1998, p. 2).

The use of stakeholders in determining priorities is appropriate because they bring the notion of social responsibility to determining an agenda, a critical factor for organizations funded with public money. By incorporating social responsiveness through stakeholder input, public universities can address the call for accountability and outcomes in relation to public expectations (Altschuld & Zheng, 1995). It is clear that stakeholders need to be at the planning table, but how should they be included? Few researchers have proposed practical models for doing so. This study provides a survey-based model for collecting stakeholder input that is "inclusive, fair, balanced, transparent, comprehensive, and accountable" (Dyer, Miller, & Leval, 1999, p. 3).

Methodology

A mixed-methods approach was used to develop the model. Extension specialists were invited to participate in the process. The researchers met with five Extension specialists from a variety of disciplines such as entomology, plant pathology, plant breeding, and weed science, as a group and individually between May 2 and 31, 2001 to discuss the goals of the project. The interviews were tape recorded and transcribed verbatim for accuracy during analysis.

An original mailed survey for collecting stakeholder input was developed by analyzing the interview transcripts to identify items for the survey. After the questions were written, a draft was circulated among the Extension specialists and a panel of experts composed of four Extension educators and two agricultural economics faculty to determine face, content, and construct validity. The final draft of the survey was pilot tested using a one-shot mailing with a randomly selected group of producers (n=100) and yielded a 20% response rate.

Further refinements were made to the survey. A final version was mailed to 750 producers who were registered with the state's Agricultural Statistics Service. The population for the survey included all wheat producers in the state (N=15,000, 1997, Census of Agriculture). The researchers drew a stratified random sample of 750 producers who had not been previously contacted for the pilot survey. Stratification was based on the proportion of producers by county (Ary, Jacobs, & Razavieh, 1996). A modified Dillman's (2000) four-phase mailing procedure was followed to garner a 29.2% usable response rate (n=219). The Cronbach's alpha for internal consistency of the survey was 0.94.

Control for non-response error was addressed comprehensively. Demographic information (mean age and land ownership) of respondents was compared to corresponding characteristics of the known population (1997 Census of Agriculture). Early (first quartile) versus late (last quartile) respondents were compared for differences on 10 summated scale items regarding the importance of factors in making decisions about production practices. Ordinal data can be treated as interval data for the purpose of comparing two groups (Kerlinger, 1986). Using an independent samples t-test, no significant differences were found between early and late respondents for these procedures.

In addition, a random sample of 10% (n=33) of the non-respondents were telephoned by the Agricultural Statistics Service and asked to complete selected portions of the survey (Lindner, Murphy, & Briers, 2001). No significant differences were found in the mean age of respondents versus non-respondents (0.05 alpha).

Of the 10 survey questions that were compared for differences, only one was found to be significantly different. Given the strength of the evidence that there were no differences between early, late, and non-respondents, the survey results can be generalized to the population of the state's wheat producers minus one variable: the degree of importance of the terms of lease/agreement with landowners.

Findings

As the focus of this research was on building a model for collecting stakeholder input, only highlights of producers' needs are presented here. A full report of the survey findings can be obtained from the researcher via email <kelseyk@okstate.edu>.

Producer Profile

The average producer was a 56-year-old white male who did not plan to retire in the next 5 years. He was a full-time producer who earned all his income from farming. He was well educated, having attained at least some college education. The typical producer's farm was individually operated as a sole proprietorship, and he owned over half of the land he farmed. Last year, he planted an average of 652 acres of wheat. He ran cattle, either as cow-calf pairs or stocker feeders, and grazed his young wheat, a practice that is common in this region. The producer was likely to collect government commodity program payments, to use short-term loans to finance his operation, and to use long-term loans to cover land and equipment purchases. He bought crop insurance and has collected on a policy at least once in his life.

The average producer reported cheat grass (Brome species), field bindweed, and drought to be his biggest challenges in farming. He was most interested in maximizing income when making production decisions; however, commodity prices, minimizing costs, the costs of inputs, maximizing yield, and long-term sustainability were other significant factors he considered.

How Producers Obtained Production-Related Information

The typical producer consulted friends, family, and other producers most often for information to solve his production problems. Business associates such as seed suppliers, grain elevator operators, and chemical and fertilizer dealers were also consulted when he needed information. His favorite publications for production information were The High Plains Journal and Progressive Farmer.

Just over half of the typical producers communicated with Cooperative Extension Service (CES) employees. While most producers (86%) indicated that they did not communicate directly with faculty members, 65% did reported using CES information. The most common reasons for not using CES were "better information was available elsewhere" (n=24) and "I don't know about extension services" (n=24). Other reasons for not using CES included "slow to provide answers" (n=9) and "unresponsive to my needs" (n=9). Sixty-six percent of respondents reported that a weekly bulletin would be helpful to them.

Improving Communication Between Producers and the CES

Respondents were asked, "how could communication between you and the land-grant university be improved?" Not all survey respondents answered this question. Those who did wrote responses that were clustered into four themes:

1. Information dissemination from the land-grant university is problematic (n=45).
2. Requested specific information (n=14).
3. Communication is OK as is, no changes required (n=13).
4. I don't know how to improve communication between the university and me (n=5).

The majority of the responses to this question focused on the university's lack of effective information dissemination strategies. Twenty-three of the 45 comments focused on direct mailing of information in the form of a monthly or biweekly crop production bulletin. Seven respondents wanted more face-to-face contact with CES personnel. Two producers wanted meetings with CES personnel, and two wanted up-to-date fact sheets. One respondent wrote that county Extension educators should be timelier in getting information out to farmers.

Five respondents wanted more mass media approaches for disseminating information, including articles in local and major newspapers, publishing in the popular press, or expanding the weekly public television program to 30 minutes. Only four of the 77 respondents recommended that the university should use the Internet to disseminate information to producers.

Recommendations for Serving Stakeholders

On March 8, 2002 the researchers met with the Extension specialists to discuss the findings. The meeting was tape recorded to accurately document statements. The conclusions and recommendations are a reflection of the data collected from producers combined with the insights of the specialists.

Producers identified their most serious production problems as weeds, grazing, soil fertility, and wheat diseases. Active research continues at the university on wheat rusts, soil born mosaic virus, cheat grass (Brome species), field bindweed, wild oats, rye, and ryegrass. Producers also identified communication with the university as problematic because of poor information dissemination strategies. The chasm lies not in knowing what problems exist for producers, but in dissemination of solutions that are efficient, cost-effective, and immediately applicable to producers' situations. Every generation needs education anew. Thus, the responsibility for teaching producers about solutions to their problems falls on the CES as the off-campus educational branch of the land-grant university.

Sixty-six percent of the producers called for a weekly bulletin on crop production. This appears to be an obvious solution to disseminating information; however, this proved not to be the case at this university. The Extension specialists had published a newsletter in the past and direct mailed it to producers biweekly during the growing season. The newsletter was subsidized by a grant the first year and was provided at no charge. The second year the producers were asked to pay $20/year for the publication, only one-half the actual cost of production. There were not enough paid subscriptions to continue the newsletter. One Extension specialist reported that the newsletter was evaluated and the findings were positive; however, "nobody wanted to pay for it."

Fifty-nine percent of the producers surveyed never used the Internet, which is an inexpensive and effective communication tool for disseminating information to the public. Given the fact that producers do not want to pay for direct mailings and don't use the Internet, the CES may consider paying for the newsletter to be published out of operating funds.

University personnel and publications were listed as the fourth and fifth most frequently consulted sources of information for producers. However, when asked to write in specific publications used for learning about crop production, less than 10% of the producers reported reading CES newsletters or university variety test reports. Less than 7% read Extension fact sheets or publications, the major form of information dissemination for research. Out of the 132 producers who listed publications they read, only one each listed the university-produced reports as a source of information.

The land-grant university is obligated to provide relevant, factual, and timely information so that producers have all the resources at hand to make decisions. Through this study, it has been discovered that the majority of producers preferred to receive information via informal communication channels (friends and business associates). These sources may not be as valid and reliable as university-generated knowledge. How does CES climb the list to position itself as the number one source for information regarding crop production in this state?

Knowing producers' preferred sources of information gives CES educators a powerful tool for information dissemination. Also, the adult education literature points to a felt need on the part of the learner as the impetus for seeking out information (Merriam & Caffarella, 1999). Once the learner identifies a need for information, he turns to sources that are most familiar, in this case, the popular press magazines. County educators may benefit by spending more time with businesses, cooperative elevator operators, and chemical and seed dealers disseminating information. Faculty may consider publishing short articles in the most popular magazines and journals to reach more producers than CES fact sheets reach currently.

Faculty may also consider this data as base-line information for collecting stakeholder input in the future. Through the survey construction process, faculty were reflective of their educational programs and wanted to document the impact of years of disseminating information to producers. The Extension specialists were "not terribly surprised by the results," although they were somewhat unsettled by the data because educational programs had been in place for several years to encourage producers to adopt a variety of hard white wheat that had not been adopted at the time of the survey.

During the focus group meeting on March 8, 2002, the Extension specialists discussed the idea that more information from the university was being disseminated than producers may have reported in the survey. For example, a crop consultant interviewed for this study reported that he attended university-sponsored field days and had received the university produced Wheat Production manual. He diffused that information to retail outlets, cooperative elevator operators, and producers as a consultant. Reporters for newspapers and other media do not always give credit for research findings. Thus, information is being disseminated, but the connection to the land-grant university is not always made clear to the public.

University researchers may not always receive validation for their work, but it would be helpful if they had a better understanding of how information was diffused from the university to end users. Future research could focus on how stakeholders adopt innovations. It would appear that the this group functions as late majority adopters based on characteristics such as making little use of mass media and securing ideas from peers (Rogers, 1995). Deepening the understanding of clients' methods of adoption will allow CES educators to more effectively reach this group.

Developing a Model for Collecting Stakeholder Input

Land-grant universities have been directed to collect and implement stakeholder input when setting priorities for research, education, and Extension (AREERA, 1998) and to be more engaged with their constituents (Kellogg, 1999). Greene (1988) discovered that giving stakeholders a voice at the program-planning table increased the likelihood that they would use study findings to improve programs. Although including stakeholders in the priority setting process is inconvenient, costly, and time consuming, it is essential for fulfilling the land-grant mission.

This research explored a process for collecting stakeholder input using survey techniques that met the call for fairness, transparency, accountability, and inclusiveness (Dyer, Miller, & Leval, 1999). The process proved to be linear and cost-effective, and yielded high-quality data that was instrumental for one academic unit in gaining a deeper understanding of their constituents. Table 1 outlines the actors, decisions, and actions required for starting the process of engaging stakeholders.

Table 1.
Decision Makers, Decisions, and Actions Required for Collecting Stakeholder Input at the Land-Grant University

Actors	Decisions and Actions
Dean of Agriculture	Decision to collect stakeholder input for setting Extension priorities. Fund research initiative.
Department head	Preliminary analysis to determine needs for stakeholder input.
Department faculty, Extension educators, and/or survey team	Work collaboratively within unit to determine information needs from stakeholders. Identify legitimate stakeholders. Estimate sample frame and select subjects randomly. Work collaboratively to develop a survey that meets standards for collecting valid and reliable data. Analyze data and feedback to unit faculty.
Survey team	Present findings to all stakeholders. Negotiate findings and conclusions with all stakeholders.
Department head, faculty, and Extension educators	Implement data in setting future priorities for Extension activities. Recycle model as necessary.

The survey was developed following high-quality survey research techniques (Dillman, 2000) and input from the Extension specialists. The specialists were intimately involved in wording the questions and ensuring that all of their information needs were satisfied. The survey was evaluated by a panel of experts, pilot tested, and administered to the producers. Results were delivered to stakeholders 10 months following the initial interviews. The faculty were called together to negotiate the conclusions and recommendations of the study based on the findings.

The focus of the Extension specialists was to develop varieties and techniques for superior crop production in the state. Research was focused on selection of traits that were considered desirable by producers and consumers alike. The faculty were highly engaged in producer education and Extension activities, hosting demonstration plot field days around the state.

It's not surprising that the faculty have been highly engaged with their stakeholders, yet they learned much from going through the formal process of gathering stakeholder input using this model. For example, they learned that after promoting hard white wheat for 7 years, only 4% of the farmers had adopted this crop. They also learned that education efforts to determine the optimum time for removing cattle from wheat pasture to be used for grain had not been diffused thoroughly. The mailed-survey design allowed respondents to remain anonymous; thus, feedback may have been more honest than face-to-face data collection.

By using randomly selected producers, this model allowed access to underserved stakeholders, producers who have not engaged with the public university or benefited from its research. The faculty were able to learn how to better serve those who have remained in the shadows of the land-grant university by understanding their information needs and reasons for not using CES.

Other faculty groups at land-grant universities may desire to test the model developed in this study. Successful replication will serve to refine the model and prove its usefulness in collecting and using stakeholder input for setting CES priorities.

Acknowledgement

This research was paid for by the Oklahoma Agricultural Experiment Station, Division of Agricultural Sciences and Natural Resources, Oklahoma State University.

References

Agricultural Research, Extension and Education Reform Act of 1998, Pub. L. No. 105-185, 7 USC 7601, 112 Stat. 523 (1998). Available at http://www.reeusda.gov/part/areera/

Altschuld, J. W., & Zheng, H. Y. (1995). Assessing the effectiveness of research organizations: An examination of multiple approaches. Evaluation Review, 19(2), 197-216.

Ary, D., Jacobs, C. J., & Razavieh, A. (1996). Introduction to research in education (5 ed.). Fort Worth: Holt, Rinehart, and Winston, Inc.

Dillman, D. A. (2000). Mail and Internet surveys: The tailored design method (2 ed.). New York: John Wiley & Sons, Inc.

Dyer, J., Miller, B., & Leval, K. (1999). In the eye of the stakeholder: Who sits at the agricultural research decision-making table? Available at http://www.csare.org/pubs/dyer.htm

Greene, J. G. (1988). Stakeholder participation and utilization in program evaluation. Evaluation Review, 12(2), 91-116.

Kellogg Commission on the Future of State and Land-Grant Universities. (1999). Returning to our roots: The engaged institution. National Association of State Universities and Land-Grant Colleges. 1307 New York Ave., NW, Suite 400. Washington, D.C. 20005.

Kelsey, K. D., & Mariger, S. C. (2002). A survey-based model for setting research, education, and extension priorities at the land-grant university-A case study of Oklahoma Wheat Producers: Final report. Oklahoma State University, Agricultural Education, Communications, and 4-H Youth Development.

Kerlinger, F. N. (1986). Foundations of behavior research (3 ed.). Fort Worth: Holt, Rinehart and Winston, Inc.

Lindner, J. R., Murphy, T. H., & Briers, G. E. (2001). Handling nonresponse in social science research. Journal of Agricultural Education, 42(4), 43-53.

Merriam, S., B., & Caffarella, R., S. (1999). Learning in adulthood: A comprehensive guide. San Francisco: Jossey-Bass.

Rogers, E. M. (1995). Diffusion of innovations (4th ed). New York: The Free Press.

Silag, B., Schultz, A., Bishop, P., Dale, D., & King, J. (1998). Visions of change in higher education: W. K. Kellogg Foundation.