Assessing Healthful Eating and Physical Activity Practices in Places Children Learn

Introduction

Schools and other places children spend time are important targets for health promotion (Centers for Disease Control and Prevention, 2009; Institute of Medicine, 2012; Story et al., 2009; U.S. Department of Agriculture [USDA], 2016) and among the most common settings for Supplemental Nutrition Assistance Program-Education (SNAP-Ed) interventions. Cooperative Extension programs partner with schools, out-of-school programs, and early care and education (ECE) programs to improve child health through SNAP-Ed policy, systems, and environment (PSE) interventions (Bergman et al., 2018; Lee et al., 2019; Schroeder et al., 2018).

Sensitive, easy-to-use tools that measure institutional practices related to healthful eating and physical activity are critical for identifying areas for improvement and measuring progress (Cates et al., 2014; Naja-Riese et al., 2019). Herein we describe the development and content validity and feasibility testing of questionnaires for initial assessment, annual reassessment, and evaluation of health practices in these three settings.

Site-Level Assessment Questionnaires

We developed site-level assessment questionnaires (SLAQs) as program planning and evaluation instruments for SNAP-Ed programs in elementary and secondary schools plus out-of-school and ECE programs. SLAQs assess institutional healthful eating and physical activity practices for each setting and were designed to be

• concise enough that sites could complete them annually,
• comprehensive regarding healthful eating and physical activity practices,
• demonstrative of face validity,
• reliable between respondents and time points, and
• sensitive enough to detect meaningful differences between sites and change over time within a site.

Items include a practice statement followed by a Likert-scale series of response options indicating degree of implementation; some items use yes/no responses. Likert-scale items are scaled 0–4, and yes/no items are coded as "no" = 0 and "yes" = 4; 4 represents a best practice. Questionnaires are completed as fillable Adobe pdfs, programmed with skips to reduce error, with autocalculated section and overall scores that allow respondents to see results immediately upon completion. Figure 1 shows an example portion of an SLAQ.

Figure 1.
Portion of Early Care and Education Site-Level Assessment Questionnaire

Development

To develop the SLAQs, we conducted a scan of existing environmental, policy, and practice assessments from the SNAP-Ed Evaluation Framework's recommended evaluation tools (USDA, 2016), the National Collaborative on Childhood Obesity Research Measures Registry (McKinnon et al., 2012), and other known sources. We adapted items from existing instruments to a standardized statement-and-response format; constructed new items as needed; and aligned items with state and national regulations and recommendations and the SNAP-Ed Evaluation Framework.

We developed instruments in parallel to maintain consistency across settings while keeping instruments true to setting-specific needs. Table 1 shows domains assessed by each instrument. Each questionnaire is organized into sections that we anticipated could be answered by the same staff person at a site to facilitate coordinated data collection among multiple staff members.

Table 1.
Wellness Domains Addressed on Site-Level Assessment Questionnaires

Domain	Instrument(s)	Sample question
Wellness policies and committee	All	Students' families are active members of the school level wellness committee.
Meal/snack program participation	All	School participates in the Fresh Fruit & Vegetable Program.
Meals and snacks	All	Students receive at least one fruit or vegetable with each meal.
Food environment/feeding practices	All	Students have at least 30 minutes for eating lunch, including time to get food and sit at the table.
Nonmeal foods (e.g., competitive foods, fundraisers)	All	Items sold as part of fundraisers are limited to non-food items or foods that meet the Smart Snacks criteria.
Physical activity	All	Staff actively facilitate physical activity during recess.
Sedentary/screen time	Out-of-school, early care and education	When entertainment screen time is offered, active alternatives are provided.
Physical education (PE)	School	At least 50% or more of PE class time is spent in moderate to vigorous physical activity.
Nutrition education	All	Nutrition education is taught with sequential curricula consistent with state or national standards.
Gardens	All	Produce from garden is used in meals or snacks.
Student involvement	School	Students take part in taste tests to determine food preferences for menu items.
Parent/family involvement	All	Nutrition education (workshops, activities, and take-home materials) is offered to parents.
Breastfeeding support	Early care and education	A welcoming and private space is provided for mothers to breastfeed or express breast milk.
Staff training	All	PE teachers complete annual professional development on PE or physical activity.
Environment and policy surrounding the school	School	Mobile vendors sell unhealthy food items within visual range of school entrances and within 30 minutes of school start or end times. a
a This item is reverse-coded. Most Site-Level Assessment Questionnaire items are phrased as best practices; however, we wrote a small number of items in the opposite direction when doing so was necessary to improve item clarity.

Content Review

Topic experts reviewed the questionnaires to assess content validity. Experts included Extension and SNAP-Ed professionals, education professionals with nutrition expertise, and academic researchers. Reviewers considered the goals of the SLAQs while providing feedback regarding missing content, items that would not contribute substantially to assessment, and clarity and logic of the items and response options.

Feasibility Testing

Questionnaires were tested at a convenience sample of sites: three elementary schools, three secondary schools, two out-of-school programs, and three ECE programs. Each site's administrator coordinated self-assessment, involving appropriate staff. Sites completed the SLAQ as a paper-and-pencil form and then answered a survey asking how many staff were needed to complete the SLAQ, how long it took to complete, the usefulness of the SLAQ for their program, and what challenges they faced when completing it. In addition, some testers provided substantial, question-by-question feedback.

Results and Discussion

Expert panel content review yielded feedback categorized as follows:

• language that may be confusing or difficult to interpret;
• need for inclusion of definitions and references, especially for state or federal guidelines;
• response options needing revision;
• items needing a "skip" or "N/A" option;
• unclear intent of items; and
• items that could be removed.

Revisions incorporating expert panel suggestions were made prior to feasibility testing.

Feasibility testing indicated that SLAQs are feasible and useful for program planning. All testers reported that the results would help them promote healthful eating and physical activity at their sites. However, testers expressed concern about questionnaire length and confusion around specific items. Responses and patterns of nonresponse revealed areas for improvement.

Feasibility testing revealed what resources were needed for the self-assessment. The school SLAQ was most resource intensive. Representatives of schools took 1–7 hr (M = 3.6 hr) to complete the questionnaire, with answering of questions divided among three to seven staff members (M = 4.5 staff); more staff were involved at secondary schools (four to seven staff) compared to elementary schools (three or four staff). Representatives of out-of-school programs completed the SLAQ in an average of 2 hr (answering of questions by three staff per site). Representatives of ECE programs completed the SLAQ in 1.5 hr on average (answering of questions by one or two staff per site). Issues encountered during feasibility testing are summarized in Table 2, along with remedies that addressed them. Questionnaire content did not change substantively with this revision.

Table 2.
Issues Identified During Feasibility Testing and Associated Remedies

Issue	Remedy
Items or associated response options on the school questionnaire were not relevant to specific grade level.	School questionnaire was divided into separate versions for elementary and secondary levels.
Testers indicated that wellness policy information is time-consuming to gather and report. Several testers left policy questions blank.	Item-by-item wellness policy question was replaced with one policy question per section.
Items or response options were unclear or confusing.	Items and/or response options were revised or deleted.
Terminology, such as program names, was unfamiliar or too technical.	Items were revised to simplify language. Where relevant, definitions were added to the questionnaires as endnotes, including web links for more information if available.
Items needed a not applicable (N/A) option.	N/A was added, or where relevant, questions were grouped, and a skip pattern was added.
Testers did not follow instructions on the questionnaire, including regarding skip patterns.	Questionnaire was reformatted where possible to reduce reliance on instructions. When moving from feasibility testing to the next project phase, respondents were encouraged to use the fillable pdf form, which automates skips.
Items contained within a single section were completed by multiple staff.	Items were reorganized into sections according to who completed them.
Testers desired a place to write in comments for each section, for additional information and for future reference.	Added a page at the end for comments by section.
Items asking about combined program participation were difficult to answer.	Where appropriate, items asking about program participation were separated.

Implications

SLAQs are novel tools that support Extension professionals working with schools, out-of-school programs, and ECE sites to assess needs, create action plans, and evaluate progress, while also offering high-quality, aggregable data for larger scale analysis. Schools that self-assess using SLAQs benefit from collecting data aligned with the Final Rule of the Healthy, Hunger-Free Kids Act of 2010 (Food and Nutrition Service, 2016), which outlines school wellness requirements for local educational agencies and schools participating in national school meal programs. This alignment means that schools using an SLAQ can measure and report how they meet each school wellness requirement of the Final Rule and can identify areas of Final Rule implementation that may be lacking.

A strength of the SLAQs, when compared to existing assessments, is that they were designed with the sensitivity to detect change between sites and across time points. Using aggregated scores from multiple sites, Extension professionals can associate the strength of measured institutional practices with nutrition and physical activity outcome data. When SLAQs are repeated over time, change in scores can be correlated with changes in measured outcomes.

Conclusion

SLAQs are feasible, comprehensive assessments of school, out-of-school, and ECE practices related to healthful eating and physical activity with usefulness for planning and evaluating PSE efforts in these settings.

Author Note

We thank the many stakeholders and expert reviewers who contributed to the development of the SLAQs. The California Department of Public Health Nutrition Education and Obesity Prevention Branch supported the effort. The project was funded by USDA's Supplemental Nutrition Assistance Program Education. Our institution is an equal opportunity employer.

Correspondence concerning this article should be addressed to Carolyn D. Rider. Email: cdrider@ucanr.edu

References

Bergman, J. J., Linnell, J. D., Scherr, R. E., Ginsburg, D. C., Brian, K. M., Carter, R., Donohue, S., Klisch, S., Lawry-Hall, S., Pressman, J., Soule, K., & Zidenberg-Cherr, S. (2018). Feasibility of implementing a school nutrition intervention that addresses policies, systems, and environment. Journal of Extension, 56(1), Article v56-1a6. https://joe.org/joe/2018february/a6.php

Cates, S., Blitstein, J., Hersey, J., Kosa, K., Flicker, L., Morgan, K., & Bell, L. (2014). Addressing the challenges of conducting effective Supplemental Nutrition Assistance Program Education (SNAP-Ed) evaluations: A step-by-step guide. https://snaped.fns.usda.gov/library/materials/addressing-challenges-conducting-effective-supplemental-nutrition-assistance

Centers for Disease Control and Prevention. (2009). Recommended community strategies and measurements to prevent obesity in the United States: Implementation and measurement guide. Morbidity and Mortality Weekly Report, 58(RR-7), 1–30.

Food and Nutrition Service. (2016). Final rule: Local school wellness policy implementation under the Healthy, Hunger-Free Kids Act of 2010. U.S. Department of Agriculture. https://www.fns.usda.gov/school-meals/fr-072916c

Institute of Medicine. (2012). Accelerating progress in obesity prevention: Solving the weight of the nation. The National Academies Press. https://doi.org/10.17226/13275

Lee, E., Smathers, C., Zubieta, A. C., Ginnetti, S., Shah, A., & Freedman, D. A. (2019). Identifying indicators of readiness and capacity for implementing farm-to-school interventions. Journal of School Health, 89(5), 373–381. https://doi.org/10.1111/josh.12747

McKinnon, R. A., Reedy, J., Berrigan, D., Krebs-Smith, S. M., & NCCOR Catalogue and Registry Working Groups. (2012). The National Collaborative on Childhood Obesity Research Catalogue of Surveillance Systems and Measures Registry: New tools to spur innovation and increase productivity in childhood obesity research. American Journal of Preventive Medicine, 42(4), 433–435. https://doi.org/10.1016/j.amepre.2012.01.004

Naja-Riese, A., Keller, K. J. M., Bruno, P., Foerster, S. B., Puma, J., Whetstone, L., MkNelly, B., Cullinen, K., Jacobs, L., & Sugerman, S. (2019). The SNAP-Ed Evaluation Framework: Demonstrating the impact of a national framework for obesity prevention in low-income populations. Translational Behavioral Medicine, 9(5), 970–979. https://doi.org/10.1093/tbm/ibz115

Schroeder, M., Grannon, K., Shanafelt, A., & Nanney, M. S. (2018). Role of Extension in improving the school food environment. Journal of Extension, 56(7), Article v56-7iw2. https://www.joe.org/joe/2018december/iw2.php

Story, M., Nanney, M. S., & Schwartz, M. B. (2009). Schools and obesity prevention: Creating school environments and policies to promote healthy eating and physical activity. The Milbank Quarterly, 87(1), 71–100. https://doi.org/10.1111/j.1468-0009.2009.00548.x

U.S. Department of Agriculture. (2016). The Supplemental Nutrition Assistance Program Education (SNAP-Ed) evaluation framework: Nutrition, physical activity, and obesity prevention indicators: Interpretive guide to the SNAP-Ed evaluation framework. https://snapedtoolkit.org/framework/index/