Determining What Growers Need to Comply with the Food Safety Modernization Act Produce Safety Rule

Introduction

The Centers for Disease Control and Prevention (2016) has estimated that 48 million residents in the United States contract food-borne illnesses each year. An analysis of confirmed cases of food-borne illnesses reported between 1998 and 2008 indicated that 46% were associated with fresh produce and nuts (Painter et al., 2013). Partly due to concerns with fresh produce safety, the U.S. Food and Drug Administration Food Safety Modernization Act (FSMA) was signed into law. This regulation shifts the focus of ensuring safety of the U.S. food supply from a reactive stance (responding to contaminated foods) to a proactive stance (preventing contamination).

The FSMA consists of seven rules. One of these is the Produce Safety Rule (PSR). The PSR addresses six topics: agricultural water; biological soil amendments; worker training, health, and hygiene; domesticated and wild animals; sprouts; and equipment, tools, and buildings. The PSR establishes science-based minimum standards for the safe growing, harvesting, packing, and holding of produce. The PSR requires growers to mitigate risk of contamination and complete training developed by the Produce Safety Alliance (PSA), unless they meet exemption criteria. The PSA training is based on good agricultural practices. Programs targeting good agricultural practices have been taught by Extension educators for over 10 years across the United States (Mahmoud, Stafne, Coker, & Bachman, 2016; Nayak, Tobin, Thomson, Radhakrishna, & LaBorde, 2015).

The North Central Region (NCR) Center for FSMA Training, Extension, and Technical Assistance, referred to herein as the NCR FSMA, was established in 2015 as one of four regional centers focused on assisting Extension educators and growers with implementing the PSR. Extension's NCR consists of 12 states (Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, South Dakota, and Wisconsin). In this article, we describe how our team of NCR FSMA researchers used a modified Delphi approach (Dalkey & Helmer, 1963; Hsu & Sanford, 2007) to develop a research-based needs assessment identifying areas of the PSR that fruit and vegetable growers do not understand and what methods of education they prefer.

The Delphi method is a well-accepted technique for gathering consensus on a topic (Hsu & Sandford, 2007; Okoli & Pawlowski, 2004). One can use this approach without face-to-face contact, allowing participants to remain anonymous and preventing individuals from becoming dominant (Dalkey, Brown, & Cochran, 1969; Delbecq, Van de Ven, & Gustafson, 1975). The Delphi method involves collecting information via multiple rounds of questions with controlled feedback. The process ends when consensus among participants is reached (Hsu & Sandford, 2007). Using a modified Delphi approach, researchers can generate qualitative and quantitative data for assessing perceptions of stakeholders (Hasson, Keeney, & McKenna, 2000).

We collected data directly from growers in the NCR by asking the same subjects to respond to two rounds of a survey. Our primary focus with this article is to emphasize the importance of identifying needs of producers regionally and suggest an effective process for doing so. For this reason, we describe the process we used for both rounds of the survey, and we present results from the first round as context for how we crafted the content of the instrument used in the second round.

Methods

Survey Development and Implementation

We used the same process to develop and implement both rounds of our needs assessment survey (see Figure 1). We (a) drafted the survey instrument, (b) conducted pilot testing of the instrument with 30 Extension educators and produce growers in the NCR, (c) modified the instrument, (d) distributed the instrument, and (e) analyzed responses.

Figure 1.
Process of Developing and Deploying a Two-Round Delphi Approach Needs Assessment Survey

The round 1 survey instrument can be viewed at https://ncrfsma.org/needs-assessment-surveys. In this round, we used a 5-point Likert-type rating scale to assess growers' perceived levels of knowledge (1 = no knowledge, 5 = high knowledge) and need for information (1 = no need, 5 = high need) related to 22 topics addressed by the PSR. We used a similar scale to identify preferred methods of information delivery based on 12 listed options. All sections included space for comments.

We distributed the survey instrument via email through electronic mailing lists provided by NCR FSMA Extension educators and produce commodity groups. We also posted a link to the survey on the NCR FSMA website. We sent hard copies to Extension educators in the NCR to administer to attendees at face-to-face gatherings and populations that do not use technology (such as Plain populations).

In round 2, participants rank-ordered their perceived levels of understanding of the top four topics for which they needed more information as identified during round 1. We also used a 5-point scale (1 = very unlikely to use, 5 = very likely to use) to assess the likelihood of respondents using various educational delivery methods. The round 2 survey instrument can be viewed at https://ncrfsma.org/needs-assessment-surveys.

Data Analysis

Following guidance from our advising statisticians, we included in our analysis only data from respondents who had completed 33% or more of the survey instrument. We analyzed quantitative data using descriptive statistics in SPSS version 12.0. Four members of our team independently reviewed open-ended comments and reached consensus on themes from the qualitative data.

Results

Of the 348 responses to round 1, 299 (86%) met the completion criteria of 33% or more. Herein we describe these data, which we used in developing the instrument for the second round of the survey.

Quantitative Data

A profile of respondents from round 1 is shown in Table 1. Many respondents were young and female, with 40.3% under the age of 45 and 58.3% female. About half (49.7%) of respondents in round 1 could be defined as beginning farmers (farming for 10 or fewer years), and over a quarter (27.1%) reported farming for more than 20 years. The vast majority (82.2%) of respondents were growing produce on 5 or fewer acres.

Table 1.
Profile of Respondents to Survey Round One (n = 299)

Characteristic	Number of respondents	Percentage of respondents
Age
<45 years	114	40.3%
45–54 years	62	21.9%
55–64 years	69	24.4%
>64 years	38	13.4%
Total	283
Gender
Male	118	41.7%
Female	165	58.3%
Total	283
Number of years farminga
0–5 years	76	26.8%
6–10 years	65	22.9%
11–15 years	30	10.6%
16–20 years	36	12.7%
>20 years	77	27.1%
Total	284
Number of acres devoted to production of fresh producea
1–5 ac	227	82.2%
6–9 ac	17	6.2%
10–49 ac	22	8.0%
50–69 ac	2	0.7%
70–99 ac	2	0.7%
≥100 ac	6	2.2%
Total	276
aPercentages do not add to 100% due to rounding.

Respondents' self-reported knowledge was lowest for soil and water testing requirements of the PSR, with mean ratings below 2.49 on the 5.0 scale (5 = high knowledge) (Table 2). Respondents also indicated a high need for information on these topics, demonstrated by means above 3.57 on the 5.0 scale (5 = high need) (Table 2). We used the findings related to participants' perceived knowledge and information needs to craft content for the second round survey instrument. Specifically, the list of topics for which growers rank-ordered their perceived levels of understanding during the second round of the survey included topics related to soil and water test types, frequency, and analysis; protection of crops from fecal contamination; and worker training.

Table 2.
Topics from Produce Safety Rule Identified by Growers as Those for Which They Had Lowest Knowledge and Highest Information Need (n = 299)

Topic (no. of responses)	M±SD	Median
Perceived current knowledgea
Types of soil tests (275)	2.49±1.30	2.00
Frequency of soil testing (274)	2.49±1.30	2.00
Types of water tests required (274)	2.59±1.26	2.00
Analysis of water results (269)	2.67±1.27	3.00
Frequency of water testing (273)	2.69±1.27	3.00
Intervals for application of amendments (272)	2.72±1.30	3.00
Untreated soil amendments (274)	2.86±1.31	3.00
Treated soil amendments (274)	2.88±1.30	3.00
Water testing methods (275)	2.89±1.21	3.00
Soil exclusion practices (270)	3.03±1.26	3.00
Perceived need for informationb
Types of water tests (264)	3.68±1.24	4.00
Analysis of water results (258)	3.67±1.27	4.00
Types of soil tests required (260)	3.63±1.30	4.00
Frequency of soil testing (259)	3.60±1.31	4.00
Frequency of water testing (262)	3.57±1.26	4.00
Crop protection from fecal contamination (257)	3.26±1.29	3.00
Worker notification practices (255)	3.20±1.34	3.00
Worker training frequency (252)	3.17±1.40	3.00
Worker training content (252)	3.16±1.41	3.00
Condition/maintenance of equipment, tools, and facilities (257)	3.14±1.34	3.00
aKnowledge scale: 1 = no knowledge, 5 = high knowledge. bInformation need scale: 1 = no need, 5 = high need.

Respondents indicated how they would prefer to receive information about the PSR, as shown in Table 3. Checklists, Extension publications, printed handouts, face-to-face workshops, and web-based handouts were identified as preferred methods of educational delivery, with mean ratings of 3.72 or above on the 5.0 point scale (5 = definitely would use). Social media and videos (YouTube or DVD) received mean ratings of 3.03 or lower on the same scale, making them the least preferred methods of educational delivery. We did not see any notable differences in top preferences between males and females, older and younger farmers, or beginning and experienced farmers.

Table 3.
Preferred Methods of Educational Delivery for Produce Safety Rule Information

Method (no. of responses)	Ma±SD	Mediana
Checklists (276)	3.93±1.16	4.00
Extension publication/fact sheets (286)	3.92±1.04	4.00
Printed handouts (286)	3.84±1.14	4.00
Face-to-face workshops (290)	3.77±1.19	4.00
Web-based handouts (PDFs) (278)	3.72±1.33	4.00
Online modules/short courses (281)	3.50±1.48	4.00
One-on-one consultations (281)	3.40±1.33	3.00
YouTube or other Internet-supported videos (282)	3.34±1.55	4.00
Online interactive tools (278)	3.32±1.48	4.00
Videos (279)	3.03±1.48	4.00
DVDs (281)	2.79±1.50	3.00
Facebook/Twitter (281)	2.16±1.35	2.00
aPreferred method of delivery scale: 1 = would not use, 5 = definitely would use.

Qualitative Data

We identified four themes from qualitative information provided through open-ended comments about knowledge and information needs related to the PSR.

• Regulations and compliance. Respondents expressed a lack of understanding of food safety requirements in specific states, confusion about where to find information on safety regulations, and concerns regarding costs of compliance.
• Food safety best practices. Growers raised questions about food safety best practices, including practices related to in-field and postharvest handling, animal controls, organic farming, and storage and processing facilities.
• Challenges in worker training. Respondents questioned how to train and effectively communicate with farmworkers, particularly those with diverse backgrounds or those whose primary language differs from that of the trainer.
• Record keeping. Respondents indicated a need for information about practical record-keeping strategies.

We also identified themes related to preferred educational delivery methods.

• Importance of credible trainers. Respondents indicated that for a trainer to be credible, he or she must have experience growing produce commercially.
• Positive and negative aspects of technology. Respondents found online methods of educational delivery to be convenient, and they appreciated that online information can be easily updated. On the other hand, others disliked the lack of personal interaction concomitant with online delivery and the potential lack of accessibility for those who do not use or have access to computers or the Internet, particularly high-speed Internet service.
• Value of visual-based messages. Respondents suggested that we develop visual-based educational materials for diverse populations and nonnative English speakers.

Discussion and Implications

Our survey respondents most commonly were under 45 years of age, female, engaged in farming for more than 20 years or 5 or fewer years, and growing on a limited number of acres. This profile of growers in the NCR differs significantly from that suggested by data for produce growers in the U.S. Census of Agriculture. Compared with 15.3% identified by the U.S. Census of Agriculture, 40.3% of our respondents were under the age of 45; additionally, the majority of our respondents (58.3%) were female, compared with 12.2% of the fruit and vegetable producers nationally (U.S. Department of Agriculture, 2012). Nearly half of our respondents were beginning farmers (farming 10 years or less), suggesting relative inexperience, and a large majority were farming on a small scale (fewer than 5 ac).

The uniqueness of the grower population in the NCR highlights the importance of conducting an assessment to understand populations with whom we work. Had we used national data to understand produce growers, we likely would have targeted the wrong audiences. Based on our findings, we have identified a strategy for collaborating with programs for women farmers and programs for beginning farmer in sharing educational resources with fruit and vegetable growers.

In the NCR, there has been an influx of Bosnian, Burmese, and Hmong immigrant farmers. The NCR network of partners has observed that many of these growers have years of farming experience and currently grow on small plots of land. Because immigrants frequently work in agriculture-based enterprises in the United States (Camarota & Zeigler, 2015), language and literacy barriers may affect farmers' compliance with the PSR. The PSA developed training modules in English and Spanish; however, people who speak other languages also work in agriculture. Our qualitative results indicate that there is a demand for visual-based educational materials appropriate for diverse audiences. The idea of providing visual-based educational materials to convey food safety information is well supported. Researchers have found minimal-text visuals effective for conveying critical food safety messages to Spanish-speaking workers on farms (Justen, Haynes, VanDerZanden, & Grudens-Schuck, 2011) and in retail foodservice settings (Rajagopal, 2012).

Despite the fact that our respondents were younger than the average fruit and vegetable grower nationally, they indicated a preference for traditional Extension outreach methods, such as checklists, printed publications, and face-to-face trainings, over online modules, online interactive tools, and social media. The likelihood that respondents live in rural areas where Internet access may still be unreliable or low speed may explain these results. Comfort with use of technology may be a challenge as well. In the NCR, the population of Plain growers does not use technology; thus, Extension educators should consider using mixed methods to deliver information. However, Guenthner and Swan (2011) cautioned Extension educators from generalizing population differences as they found that electronic technology was more widely used by potato farmers than by some university students.

Another theme that emerged from our qualitative data suggested that growers believe other growers are more credible providers of PSR training than traditional educators. This is a challenge for Extension educators, whose experience in horticultural production may be more in the realms of research and education than operational experience with farms. To address this challenge, Extension educators could consider enlisting growers to become involved with PSR trainings. Extension educators may be able to do this by hosting PSR field days at local farms, asking local growers to deliver content during workshops, or including photos of local farms or testimonials from local farmers in printed materials.

We developed our survey to assess the needs of growers in the NCR regarding the FSMA PSR. However, Extension professionals who are working with similar grower populations can use our methods to guide development of materials on any educational topic.

Hearing directly from growers rather than making assumptions about their needs based on our own perceptions provided us with information that will be useful in helping fruit and vegetable producers in the NCR comply with the PSR. Extension educators are well positioned to guide producers in understanding safe food handling practices and elements of the PSR, but it is imperative that they hear the voice of growers.

Acknowledgment

A grant from the Food and Drug Administration (grant number 1R01FD005685-01) titled "Strategies for Successful Implementation of FSMA (Food Safety Modernization Act) in the North Central Region through Adoption of a Systems Approach and Stakeholder Engagement Framework" funded the project described in this article.

References

Camarota, S. A., & Zeigler, K. (2015). U.S. immigrant population hit record 42.4 million in 2014: Asian, Sub-Saharan African, and Middle Eastern immigrant populations grew the most since 2010. Retrieved from https://cis.org/Report/US-Immigrant-Population-Hit-Record-424-Million-2014

Centers for Disease Control and Prevention. (2016). Estimates of foodborne illness in the United States. Retrieved from https://www.cdc.gov/foodborneburden/2011-foodborne-estimates.html

Dalkey, N. C., Brown, B. B., & Cochran, S. (1969). The Delphi method: An experimental study of group opinion. Santa Monica, CA: Rand Corporation.

Dalkey, N. C., & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9(3), 458–467.

Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning. Glenview, IL: Scott, Foresman, and Co.

Guenthner, J. F., & Swan, B. G. (2011). Extension learners' use of electronic technology. Journal of Extension, 49(1), Article 1FEA2. Available at: https://joe.org/joe/2011february/a2.php

Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32(4), 1008–1015.

Hsu, C. C., & Sandford, B. A. (2007). The Delphi technique: Making sense of consensus. Practical Assessment, Research and Evaluation, 12(10). Retrieved from http://pareonline.net/pdf/v12n10.pdf

Justen, E., Haynes, C., VanDerZanden, A. M., & Grudens-Schuck, N. (2011). Insights from Spanish-speaking employees in the Iowa horticultural industry. Journal of Extension, 49(6), Article 6RIB8. Available at: http://www.joe.org/joe/2011december/rb8.php

Mahmoud, B. S. M., Stafne, E. T., Coker, C. H., & Bachman, G. R. (2016). Implementing good practices programs to encourage production of high-quality, safer produce in Mississippi. Journal of Extension, 54(4), Article 4IAW5. Available at: https://www.joe.org/joe/2016august/iw5.php

Nayak, R., Tobin, D., Thomson, J., Radhakrishna, R., & LaBorde, L. (2015). Evaluation of on farm food safety programming in Pennsylvania: Implications for Extension. Journal of Extension, 53(1), Article 1FEA9. Available at: https://joe.org/joe/2015february/a9.php

Okoli, C., & Pawlowski, S. D. (2004). The Delphi method as a research tool: An example, design considerations and applications. Information & Management, 42(1), 15–29.

Painter, J. A., Hoekstra, R. M., Ayers, T., Tauxe, R. V., Braden, C. R., Angulo, F. J., & Griffin, P. M. (2013). Attribution of foodborne illnesses, hospitalizations, and deaths to food commodities by using outbreak data, United States, 1998–2008. Emerging Infectious Diseases, 19(3), 407. doi:10.3201/eid1903.111866.

Rajagopal, L. (2012). Use of visuals for food safety education of Spanish-speaking foodservice workers: A case study in Iowa. Journal of Extension, 50(2), Article 2RIB6. Available at: https://www.joe.org/joe/2012april/rb6.php

U.S. Department of Agriculture. (2012). Census of agriculture. Retrieved from https://www.agcensus.usda.gov/Publications/2012/