Delphi Survey of Needs for On-Farm Research: Forecasting Changes in a Farm Organization

Introduction

For farm organizations that conduct research, priorities change over time. Change is driven by the needs and values of an organization's members in response to their farm operations' dynamic environments. These needs and values are complex and diverse, and may be expressed as conflicting priorities. This article reports on a Delphi study conducted for a sustainable agriculture farmer organization to identify and prioritize research needs of its members. The study also aimed to demonstrate Delphi's feasibility and applicability for needs assessment in Extension. Results from the study included a list of highly ranked ideas for agricultural research. The study also uncovered a new area of emphasis for the organization and forecast changes in staffing, programming, and research topics 10 years later.

On-Farm Research

A Delphi study was implemented for an organization whose farmers conducted research themselves, on their own farms. On-farm research addresses problems particular to specific farms (Chambers, Pacey, & Thrupp, 1989; Exner, 1995). On-farm research features greater control of topics by farmers and, concomitantly, less control by Extension and university scientists (Foster, Norton, & Brough, 1995). The farmer-centered research movement gained strength in the United States and worldwide in the 1980s and 1990s.

Extension Challenges

The greater autonomy presumed in farmer-centered research may pose technical and political challenges to Extension staff, who typically are urged to focus on research lines favored by land-grant agricultural scientists (Blaine, 2005; McDowell, 2004). Farmer-centered research may also go against the thrust of Extension-accepted research agendas, for example, by focusing on organic agriculture or small cattle breeds for grass-fed livestock operations (Bell, 2004; Hassanein, 1999). Blaine (2005) notes that Extension educators are also encouraged to emphasize research that is widely applicable, whereas farmer-centered research may be less concerned about broad application. It may not be easy, therefore, to counsel such organizations on research agendas (Exner, 1995); a formal process like Delphi would be useful.

Study Overview

This study consisted of a two-round Delphi survey of on-farm research needs conducted by a team of university scientists, an Extension staff person, and director of the farm organization Practical Farmers of Iowa (PFI). PFI boasts a rich history of conducting farmer-centered research both independently and in cooperation with land-grant institutions, Extension, U.S. Department of Agriculture (USDA) scientists, private research facilities, and independent agricultural consultants (Exner & Thompson, 2008). The study was inspired by PFI's expressing a desire for a more formal needs assessment to identify high-priority research combined with researcher eagerness to test Delphi in an agricultural Extension setting. The team, consisting of the four authors of this article, planned, conducted, and reported the results. The study was conducted from February to August 2004.

PFI currently supports its members through an annual conference, a research cooperators' meeting, a quarterly newsletter, online webinars called "farminars," blog and social media presence, field days, and research reports on statistically valid research projects (Grudens-Schuck, Cramer, Exner, & Shour, 2003; T. Opheim, personal communication, April 30, 2015). Today, PFI's membership hovers around 2,500, and the organization supports a full-time staff of 14. In the 1990s, PFI expanded, leading to the hiring of an executive director in 2001—a crucial step toward organizational stability for a small-scale nonprofit organization. At the time of the study, PFI had access to an Extension staff person (agronomy) to supervise the research, housed at Iowa State University (ISU) in Ames, Iowa. Currently, the university position is gone, and a staff position within PFI oversees the research program.

Over 30 years, PFI has helped Iowa farmers conduct more than 1,100 on-farm research trials (Practical Farmers of Iowa [PFI], n.d.-a). Topics range from nitrogen management in field corn to management of parasites in swine. The on-farm research program began in 1987, and reports are available for research from 1989 to the present (PFI, n.d.-c). From 1989 through 1993, the only topic area for research was field crops. In 1994, a single livestock topic was added. The mix of field crops and livestock topics continued until 1996, when a few tree topics were added (termed "horticulture") (PFI, n.d.-c). Trees remained the only horticultural crop topic until 2000, when two market garden-type projects were added. The leadership of the organization desired improved processes for prioritizing on-farm research needs to adjust to the concomitant increase in research ideas. This is the era in the PFI organizational development during which the Delphi was introduced as a method for identifying high-priority research areas.

Delphi Study

The needs assessment survey employed the Delphi technique, a set of procedures characterized by iterative use of surveys over time with the same panel of respondents (Adams, 2001; Andranovich, 1989; Crance, 1987; Witkin & Altschuld, 1995). A successful process of setting priorities encourages broad participation, enables dialogue and reflection, and facilitates collective decision making. Delphi adheres to these principles and is regarded as a pragmatic, cost-efficient method of soliciting expert opinion and arriving at group consensus (Ludwig, 1997). Delphi's history, as well as its greatest strength, lies in its ability to foretell a likely future. Delphi is also considered to be effective in addressing pressing or controversial issues, generating ideas, and building consensus (Linstone & Turoff, 1979; Miller, Repinski, Hayes, Bliss, & Trexler, 2011; Osborne, Collins, Ratcliffe, Millar, & Duschl, 2003). Delphi method is defined, in part, as "a method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem" (Linstone & Turoff, 1979, p. 3; see also Fink, Kosecoff, Chassin, & Brook, 1984; Keil, Tiwana, & Bush, 2002). A noteworthy characteristic of Delphi is that it does not require face-to-face exchanges among group members. However, according to Witkin and Altschuld (1995), the opportunity for respondents to be engaged in generating, reviewing, and ranking ideas enhances the quality of the findings.

Survey Design

The design required that panel members (interchangeable with "respondents") reply to two questionnaires, administered consecutively. The process first asked panelists to provide ideas ("open ended") for on-farm research projects. The wording of the first question is crucial and was debated by the research team. The PFI team members (director and Extensionist) urged the team to use language that fit the current agricultural landscape, principally, "production agriculture," a term indicating field crops and livestock. There was a concerted effort to avoid language that encouraged research on vegetables or other market garden crops. These latter topics were considered to be sidetrack issues for the organization.

In Delphi, the list of topics generated by the first-round questionnaire becomes the outline for designing the second-round questionnaire (Linstone & Turoff, 1979). The second-round questionnaire was more structured and was sent to panel members who provided the original data in the first round. Panel members ranked the ideas in the second-round survey within farming enterprise categories determined by Delphi researchers. A follow-up phone call was made to nonrespondents for each round to improve response rate. Handwritten thank you notes were mailed to all respondents.

Sample Size and Composition

The highest percentage of the sample consisted of farmers with membership in PFI. Four university and Extension scientists were added to increase the variety of ideas generated. A small number of farmers were chosen because they were not PFI members, to counteract potential "group think." These enrichments are permitted in the Delphi process as long as the panel members are considered to be knowledgeable—"experts," per Delphi—in the topic area (e.g., on-farm research, sustainable agriculture research). Panel members also met the following four characteristics requirements, balanced for enterprise type and gender:

• Members must understand and be willing to express ideas for on-farm research.
• Members must have a commitment to, and knowledge of, sustainable farming.
• Members must have good (but not perfect) written communication skills.
• Members must be motivated and willing to participate in both rounds of the survey.

Procedures

One hundred four potential respondents were nominated by a newly-formed PFI-ISU Advisory Board (part of the expansion of responsibilities that the organization was experiencing). The 104 potential participants were narrowed to 43, balanced for commodity, gender, and region. University researchers and Extensionists (six of 43) were selected to provide a wide range of ideas across the disciplines (Table 1).

Table 1.
Composition of Delphi Survey Panel

Group	Potential participants	Selected participants
Crops	35	12
Livestock	8	7
Mixed crop/stock farms	8	6
Horticulture/agroforestry	10	6
University-Extension	11	6
Non-PFI	32	6
Total	104	43

Round 1: The first survey was sent to 43 individuals. Twenty-four usable surveys from the first round were returned, yielding a return rate of 56%. Livestock-only farmers (7), however, did not respond to the first round of the survey. The first round of the survey generated over 132 on-farm research ideas related to sustainable agriculture. Although the first-round survey question used language that encouraged production agriculture (field crops and livestock for Iowa), on-farm research ideas related to horticulture were returned in the mix.

Round 2: The second survey was sent to all 24 first-round respondents. A category for horticulture was included as part of the structure of the second-round survey on the basis of Round 1 results. The survey asked respondents to rank 77 research topics in six categories and to identify the top four to six research ideas. The rate of return for the second survey was 92%.

Top Research Categories

The study identified highly ranked ideas for research in six areas: soils, crops, pests and weeds, horticulture, marketing and management, and livestock health and feed. The topics were determined on the basis of Round 1 responses. Within these categories, panel members ranked ideas by importance.

Soils

In the soils category, the top idea for research focused on soil quality. The survey highlighted the need to know which cropping practices improved rather than degraded farm fields. Respondents wanted to compare sustainable cropping practices to conventional practices with an eye toward impacts on water quality and soil erosion. Research was needed to compare strategies for reducing movement of nitrogen and phosphorus into streams and lakes. Profits were also a critical concern here, evident across several categories.

Crops

In the crops category, respondents wanted research on how to increase profits from growing small grains on organic and sustainable farms. The survey emphasized research on production budgets to compare organic and sustainable crops with conventionally raised crops. The survey also indicated that responders wanted more research on identifying cover crops that succeed on Iowa farms and how to manage them.

Pests and Weeds

Developing strategies for controlling pests and weeds on farms using organic management practices was the top pick in this category. The second highly ranked idea focused on use of biological control agents, such as natural predators and parasites, to control agricultural pests and weeds. Specifically, respondents indicated that research was needed on the use of "buffers"—reservoirs that provide food and shelter for beneficial organisms. The third item provided reminders that Canada thistle and bean leaf beetle in soybeans continued as problems.

Horticulture

Panel members endorsed investigating new perennial food crops and other alternative crops (i.e., not corn or soybean). Research was needed to adapt machines and procedures to grow and process market garden crops on a small scale; to power greenhouses with renewable energy sources (not fossil fuels); and to pinpoint the value of organic soil amendments, such as compost, for use in vegetable production in Iowa, which was on the rise.

Marketing and Management

Marketing and management were also considered important to panel members. Research on the economics of alternative crops (such as triticale, kenaf, adzuki beans, and agroforestry crops) was featured in two of the top picks for this category. Panel members encouraged research on farm management strategies that conserved energy, increased local food production, and kept young people in agriculture—all at the same time. The impact of use of genetically modified crops was also endorsed as a focus for research, specifically the impact on sales of agricultural goods when non–genetically modified crops became contaminated.

Livestock Health and Feed

The livestock category included a focus on alternative feed grains for livestock. Research on grazing also was desired and included topics such as documenting the effects of raising cattle, sheep, and goats on the same pasture (e.g., impact of mixed-species grazing on livestock parasite populations). In addition, studies were needed to determine factors that influenced the quality of meat and milk from livestock grazed to market weight (i.e., "finished") on pasture.

Interpretation of Results

Weighing the Findings

Results of the research meant different things to different players in PFI and the university. The meaning of the results was vetted in several settings: a focus group event with about 50 PFI leaders and farmers in February 2005, postsurvey discussions with research team members, and informal follow-ups in 2004 and 2005. PFI members and leadership debated the meaning of the results with great interest.

Meaning of Results

Take-home lessons for Extension personnel in particular regarding the meaning of the Delphi results centered on expected results, surprising results, and the need to recognize that there is a difference between "farmer-centered" research and "farmer-only" research.

Expected Results

Some of panelists' most highly ranked ideas had been in place for more than 15 years. PFI members and staff observed specifically that the topics of soil health and water quality ranked high because they were part of the core mission of the organization, which focused on sustainable and organic farming practices. Several topics that ranked high had political and symbolic value in addition to representing an opportunity to generate knowledge of biophysical or economic value. Some of these topics, such as soil quality, were considered to be what PFI is "about." Members were not ready to "move on" or cease the research.

Surprising Results

While familiar and already valued topics rose to the top, so did topics that were newer or previously undervalued by PFI. Most notably, a number of panel members provided ideas that fell into a horticulture category, with some items on alternative marketing. Emphasis in the survey instrument had been placed on production agriculture, which typically signaled commodity crops and livestock and mixed crop–livestock farms. Moreover, the unexpected wellspring of ideas about horticulture and related endeavors forecast development within PFI of official member priorities that eventually included horticulture and local foods (PFI, n.d.-b). There was also an increase in research projects related to horticulture, with 15 ideas generated at the Cooperators' Meeting in 2014, compared with 10 ideas related to field crops. There were seven ideas provided by horticulture members and field crops members each as 2015 research priorities (PFI, n.d.-b). PFI also now supports a horticulture coordinator position.

Need to Recognize the Difference Between Farmer-Centered and Farmer-Only Research

PFI advocates for the right of farmers to conduct high-quality research and for the findings to be taken seriously. However, in practice, members of PFI told us during the face-to-face interpretation session that they carefully weighed the ability of farmers to conduct particular types of research. Farmers decided that some research ideas were too difficult, complex, expensive, or risky to conduct at the farm level. For example, trials that required controls might permit weeds that are difficult to control to flourish in some areas of a farm. Comparisons of conventional and organic management might require use of pesticides, risking certification. During the face-to-face data interpretation session, PFI members stated that they welcomed the suggestions of researchers and Extension staff who could be trusted to collaborate well. There were clearly opportunities for interested Extension and research staff. This lesson learned has implications for Extension staff who wonder whether to connect with a farmer-centered research organization or not. Potential openings are worth exploring.

Dissemination of Results

The survey results were presented at two forums. The first forum was an ISU College of Agriculture and Life Sciences Advisory Group meeting that included PFI leadership, college administrators, and faculty with research and Extension appointments. Rankings were shared, and individuals expressed strong interest in the results. A scientist working at the National Soil Tilth Laboratory (now, National Laboratory for Agriculture and the Environment) immediately shared the rankings with colleagues on state and national boards who were setting research agendas internally.

In 2006, the ISU College of Agriculture and Life Sciences moved to jointly promote partnerships between farmers and land-grant scientists on topics of mutual interest and to fund those ventures. The key Extension contact for the PFI organization (Exner) played a role as publicist and matchmaker for farmers and researchers. The program was offered in 2006 and 2007, demonstrating commitment of PFI and the college to further support on-farm research as part of the landscape of research endeavors in agriculture.

Weaknesses of the Study

The study had weaknesses. First, a three- or even four-round process might have produced higher quality data, yet there were resources for only a two-round survey. However, a two-round process was methodologically adequate according to literature cited herein. Second, the rate of return by producers who produced only livestock was low. Only farmers with mixed livestock and crops were available to suggest research ideas. Consequently, the study may have suffered in unknown ways from lack of research ideas about livestock.

Conclusions

The Delphi study contributed to PFI's organizational capacity building by providing the members and leaders of the organization with concrete, prioritized ideas about on-farm research needs. The findings related to horticulture and related topic areas accurately forecast important changes in the organization's programs, research, and staffing. The Delphi method has potential for expanded use within Extension, ultimately contributing to the methodological toolkit of Extension personnel. From the methodology point of view, Delphi is versatile, and it allows for qualitative and quantitative data collection (Rowell, Polush, Riel, & Bruewer, 2015). Its use could enhance the rigor of findings generated by Extension, especially in topic areas that are unfamiliar. The circumstances addressed well by Delphi that could be especially relevant to Extension include (a) addressing complex problem that "can benefit from subjective judgments on a collective basis" (Rowell et al., 2015, p. 4), (b) involving a broader representation and individuals with diverse backgrounds, (c) structuring communication that mixes face-to-face with online interactions, and (d) fostering the process of consensus building among a diverse group of stakeholders (Linstone & Turoff, 1979). As Barbara Ludwig encouraged in 1997, Extension and its clients could benefit from employing this "under-used methodology . . . to explore the future" (para. 3).

References

Adams, S. J. (2001, October). Projecting the next decade in safety management. Professional Safety, 46, 26–29.

Andranovich, G. (1989). Developing community participation and consensus: The Delphi technique. Los Angeles, CA: Author.

Bell, M. M. (2004). Farming for us all: Practical agriculture and the cultivation of sustainability. University Park, PA: Pennsylvania State University Press.

Blaine, T. W. (2005). Applied extension in an era of devolution. Journal of Extension [online], 43(2) Article 2COM1. Available at: http://www.joe.org/joe/2005april/comm1.shtml

Chambers, R., Pacey, A., & Thrupp, L. A. (Eds.). (1989). Farmer first: Farmer innovation and agricultural research. London, England: Intermediate Technology Publications.

Crance, J. H. (1987). Guidelines for using the Delphi technique to develop habitat suitability index curves (Report No. 82 [10.134]). Washington, DC: U.S. Department of the Interior.

Exner, D. N. (1995). Complementary abilities and objectives in on-farm research. In C. Francis, R. Janke, V. Mundy, & J. King (Eds.), Extension education materials for sustainable agriculture: Alternative approaches to on-farm research and technology exchange, Series Volume 3 (pp. 33–35). Lincoln, NE: University of Nebraska.

Exner, D. N., & Thompson, R. (2008). The paired-comparison: A good design for farmer-managed trials. Unpublished. Ames, IA.

Fink, A., Kosecoff, J., Chassin, M., & Brook, R. H. (1984). Consensus methods: Characteristics and guidelines for use. American Journal of Public Health, 74(9), 979–983.

Foster, J., Norton, G., & Brough, E. (1995). The role of problem specification workshops in Extension: An IPM example. Journal of Extension [online], 33(4) Article 4FEA1. Available at: http://www.joe.org/joe/1995august/a1.php

Grudens-Schuck, N., Cramer, J. A., Exner, D., & Shour, M. (2003). The new adult education: Bringing peer educators up to speed. Journal of Extension [online], 41(4) Article FEA2. Available at: http://www.joe.org/joe/2003august/a2.shtml

Hassanein, N. (1999). Changing the way American farms: Knowledge and community in the sustainable agriculture movement. Lincoln, NE: University of Nebraska Press.

Keil, M., Tiwana, A., & Bush, A. (2002). Reconciling user and project manager perceptions of IT project risk: A Delphi study. Information Systems Journal, 12(2), 103–119.

Linstone, H. A., & Turoff, M. (Eds.). (1979). The Delphi method: Techniques and applications. Reading, MA: Addison-Wesley Publishing.

Ludwig, B. (1997). Predicting the future: Have you considered using the Delphi methodology? Journal of Extension [online], 35(5) Article 5TOT2. Available from: http://www.joe.org/joe/1997october/tt2.php

McDowell, G. R. (2004). Is Extension an idea whose time has come—and gone? Journal of Extension [online], 42(6) Article 6COM. Available at: http://www.joe.org/joe/2004december/comm1.shtml

Miller, J. K., Repinski, S. L., Hayes, K. N., Bliss, F. A., & Trexler, C. J. (2011). Designing graduate-level plant breeding curriculum: A Delphi study of private sector stakeholder opinions. Journal of Natural Resources and Life Sciences Education, 40, 82–90.

Osborne, J., Collins, S., Ratcliffe, M., Millar, R., & Duschl, R. (2003). What ideas about science should be taught in school science? A Delphi study of the expert community. Journal of Research in Science Teaching, 40 (7), 692–720.

Practical Farmers of Iowa. (n.d.-a). Cooperators' Program. Accessed Sept. 8, 2015, from http://practicalfarmers.org/member-priorities/research-demonstration/

Practical Farmers of Iowa. (n.d.-b). Research priorities. Accessed Sept. 8, 2015, from http://practicalfarmers.org/news-events/events/cooperators-meeting/

Practical Farmers of Iowa. (n.d.-c). Research reports. Accessed Sept. 8, 2015, from http://practicalfarmers.org/farmer-knowledge/research-reports/

Rowell, L. L., Polush, E. Y., Riel, M., & Bruewer, A. (2015). Action researchers' perspectives about the distinguishing characteristics of action research: A Delphi and learning circles mixed-methods study. Educational Action Research, 23(2), 243–270.

Witkin, B. R., & Altschuld, J. W. (1995). Conducting needs assessment: A practical guide. Thousand Oaks, CA: Sage Publications.