Cooking Healthy, Eating Smart: A Strategically Timed Formative Evaluation of a Community-Based Nutrition and Food Safety Program for Rural Older Adults

Introduction

Formative evaluation (FE) is essential to program development and improvement (Herman, Morris, Fitz-Gibbon, 1987; Joint Committee on Standards for Educational Evaluation, 1994; Patton, 1987). Trochim (2006) delineated five types of FE—needs assessment, evaluability assessment, structured conceptualization, implementation evaluation, and process evaluation—highlighting the need for FE at multiple points in time. In education, these types of FE can be completed before the development of a curriculum (needs assessment, evaluability assessment, or structured conceptualization) and after the curriculum has been pilot tested (implementation evaluation or process evaluation). We believe an FE should take place at each of these times due to the ongoing nature of curriculum improvement, but the most important (and least reported on) evaluation point is between development and pilot testing. The aim of this article is to present the results of a formative evaluation conducted at this point in time, using a focus group methodology.

Focus Groups

When seeking to tailor a community-based curriculum and prepare it for pilot testing, educators can use focus groups composed of members from the target population to formatively evaluate the intervention. Conducting focus group research according to Morgan and Krueger's methodology (Morgan, Krueger, & King, 1998) provides meaningful data that can set the stage for a more efficient and effective pilot test. Although focus group research is a common method of FE at other points in time, we found few articles in the literature reporting on focus group research conducted with community members between development and pilot testing of community-based curricula.

Many developers of community-based curricula convene focus groups before development (Benke et al., 2013; Dumbauld, Kalichman, Bell, Dagnino, & Taras, 2014; Healey, Reed, & Huber, 2013; Nguyen, Tran, Kagawa-Singer, & Foo, 2011; Robinson, Wong, Konzelmann, Bielamowicz, & Rodgers, 2008). Others use focus groups to evaluate curricula after implementation (Besculides, Trebino, & Nelson, 2012; Hayes et al., 2010; Pettman et al., 2008; Smith, George, Young, Meehan, & Enfield, 2004). Few articles reference formatively evaluating a curriculum between development and pilot testing (Coronado, Acorda, Do, & Taylor, 2008; Parsai, Castro, Marsiglia, Harthun, & Valdez, 2011; Zamora et al., 2012). We believe the literature should highlight the importance of FE at this point in time.

Formative Evaluation of a Curriculum

In our study, we formatively evaluated a nutrition and food safety curriculum targeting rural older adults at three points in time:

1. Before development, we spoke with staff of the South Carolina Lieutenant Governor's Office on Aging and with rural older adults about the nutrition and food safety education needs of rural older adults.
2. Between development and pilot testing, we conducted focus group research with a sample of rural older adults and used their input to further refine the curriculum.
3. After pilot testing, we collected feedback from program participants and the Extension educators who delivered the curriculum.

Each stage of FE was necessary, but in this article we highlight the method and findings of our second stage of FE, which occurred at the least commonly reported time—between curriculum development and pilot testing.

The Cooking Healthy, Eating Smart (CHES) curriculum was developed by investigators from Clemson University and the Medical University of South Carolina to teach rural, low-income older adults in the selection and preparation of safe, healthful foods within a budget. On the basis of the revised Bloom's taxonomy (Anderson et al., 2001), each of the eight lessons in the curriculum included an objective, learning questions, a lesson content summary, an activities chart, activity guides, a supply list, and recipe handouts. The lessons were titled: Less Fat, Less Salt, More Fiber, More Water, More Calcium, Protein, More Fruits and Vegetables, and Food Safety.

An Extension agent conducted four focus groups with members of South Carolina Family and Community Leaders (SCFCL)—a volunteer organization targeting older adults (South Carolina Family and Community Leaders, 2012)—to formatively evaluate the CHES curriculum prior to its pilot test. The focus group research was conducted in four rural South Carolina counties with representatives from eight SCFCL clubs. The agent asked scripted questions (Table 1) and followed a protocol guided by Morgan and Krueger's recommended methodologies (Morgan et al., 1998).

Table 1.
Focus Group Session Questions

Aspect	Question(s)
Lesson content	Does the lesson content use words that are easy to understand? Does the lesson include enough information, so that the participant wants to or will change their behaviors? Did we leave out anything that is pertinent? Any other suggestions related to the content?
Delivery method	Is this activity appropriate for older adults? Please explain your response. Do you think this activity will hold the interest of the audience? Is the take-home item appropriate for the audience? Any other suggestions?
Recipes	Do you think that each recipe is easy to chew and swallow? Do you believe that the ingredients to make this recipe are readily available locally and familiar? Are the instructions clear? Do you think most older adults would have the equipment to make this recipe? Do you think that the ingredients to make this recipe are too expensive?
Overall	Do you have any other thoughts or comments that you would like to share with us about the lessons? Have we missed anything?

Participants and Focus Group Results

Each focus group comprised nine to 18 participants, resulting in a total of 46 participants. Only one focus group had more than the maximum of 12 participants recommended by Morgan et al. (1998). All focus group participants were female, reflecting a female dominance in SCFCL groups. More participants were African American (59%) than White (41%). Table 2 provides additional information about the characteristics of the focus groups.

Table 2.
Focus Group Characteristics

Characteristic category	Group 1	Group 2	Group 3	Group 4
Date	January 18, 2010	January 19, 2010	February 8, 2010	February 16, 2010
Location	Laurens	Bamberg	Beaufort	Lexington
Number of SCFCL clubs represented	1	1	1	5
Number of participants	9	18	10	9
Race
African American	0	17	10	0
White	9	1	0	9
CHES lessons reviewed	Less Fat, Less Salt	More Fiber, More Fruits and Vegetables	More Calcium, More Water	Protein, Food Safety

The focus group participants critically evaluated the curriculum and suggested needs for revision that had been unapparent to investigators. Their suggestions resulted in improvements. For example, an entire activity is now devoted to lactose intolerance, teaching strategies for obtaining adequate amounts of calcium when consuming lactose-free foods. Also, each recipe included in the curriculum now produces only one to four servings in order to decrease amounts of leftover food. Additionally, the CHES curriculum now advises older adults to sip water throughout the day, drink "at will" while at home, and avoid consuming liquids within 3 hr of going to bed or going out in public.

On the basis of participants' suggestions, specific changes were made to six of the eight lessons. The strategic timing of this FE—between curriculum development and pilot testing—allowed community members to provide practical criticisms for improving the curriculum at a point at which changes still could easily be made. Obtaining specific criticisms is impossible before a curriculum has been developed, and waiting until after it has been pilot tested to identify necessary changes wastes time and resources (Breault & Gould, 1998). Thus, FE must occur between development and pilot testing in addition to at the more commonly reported times.

Conclusion

Formative evaluations of a curriculum should be conducted at various stages for continued improvement. To tailor a curriculum and prepare it for pilot testing, it is best to conduct focus group research with members of the target population after development and prior to pilot testing to improve program design and effectiveness. Extension professionals can use this example in planning formative evaluations of their curricula.

References

Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., . . . (Eds.) (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives: Abridged edition. New York, NY: Addison Wesley Longman, Inc.

Benke, C. J., Bailey, S. J., Martz, J., Paul, L., Lynch, W., & Eldridge, G. (2013). Developing a parent-centered obesity prevention program for 4-H families: Implications for Extension family programming. Journal of Extension [online], 51(3) Article 3FEA8. Available at: http://www.joe.org/joe/2013june/a8.php

Besculides, M., Trebino, L., & Nelson, H. (2012). Successful strategies for educating hard-to-reach populations: Lessons learned from Massachusetts' train-the-trainer project using the "Helping You Take Care of Yourself" curriculum. Health Education Journal, 71(3), 350–357.

Breault, J. L., & Gould, R. (1998). Formative evaluation of a video and training manual on feeding children with special needs. Journal of Nutrition Education and Behavior, 30(1), 58–61.

Coronado, G. D., Acorda, E., Do, H. H., & Taylor, V. M. (2008). Feasibility and acceptability of an English-as-a-second language curriculum on hepatitis B for older Chinese American immigrants. Journal of Health Disparities Research & Practice, 2(3), 121–131.

Dumbauld, J., Kalichman, M., Bell, Y., Dagnino, C., & Taras, H. L. (2014). Case study in designing a research fundamentals curriculum for community health workers: A university–community clinic collaboration. Health Promotion Practice, 15(1), 79–85. doi:10.1177/1524839913504416

Hayes, A., Morzinski, J., Ertl, K., Wurm, C., Patterson, L., Wilke, N., & Whittle, J. (2010). Preliminary description of the feasibility of using peer leaders to encourage hypertension self-management. WMJ: Official Publication of the State Medical Society of Wisconsin, 109(2), 85–90.

Healey, W. E., Reed, M., & Huber, G. (2013). Creating a community-physical therapy partnership to increase physical activity in urban African-American adults. Progress in Community Health Partnerships: Research, Education, and Action, 7(3), 255–262. doi:10.1353/cpr.2013.0038

Herman, J. L., Morris, L. L., & Fitz-Gibbon, C. T. (1987). Evaluator's Handbook. Newbury Park, CA: Sage Publications, Inc.

Joint Committee on Standards for Educational Evaluation. (1994). Sanders, J. R. (Chair), The program evaluation standards: How to assess evaluations of educational programs (2nd ed.). Thousand Oaks, CA: Sage Publications, Inc.

Morgan, D. L., Krueger, R. A., & King, J. A. (1998). The Focus Group Kit, Vols. 1–6. Thousand Oaks, CA: Sage Publications, Inc.

Nguyen, T. N., Tran, J. H., Kagawa-Singer, M., Foo, M. A. (2011). A qualitative assessment of community-based breast health navigation services for Southeast Asian women in Southern California: Recommendations for developing a navigator training curriculum. American Journal of Public Health, 101(1), 87. doi:10.2105/AJPH.2009.176743

Parsai, M. B., Castro, F. G., Marsiglia, F. F., Harthun, M. L., & Valdez, H. (2011). Using community based participatory research to create a culturally grounded intervention for parents and youth to prevent risky behaviors. Prevention Science, 12(1), 34–47. doi:10.1007/s11121-010-0188-z

Patton, M. Q. (1987). How to use qualitative methods in evaluation. Newbury Park, CA: Sage Publications, Inc.

Pettman, T. L., Misan, G. M. H., Owen, K., Warren, K., Coates, A. M., Buckley, J. D., & Howe, P. R. C. (2008). Self-management for obesity and cardio-metabolic fitness: Description and evaluation of the lifestyle modification program of a randomised controlled trial. International Journal of Behavioral Nutrition and Physical Activity, 5, 15.

Robinson, S. F., Wong, W. W., Konzelmann, K. L., Bielamowicz, M. K., & Rodgers, A. S. (2008). Designing a bone health and soy focus group discussion guide based on the health belief model. Journal of Extension [online], 46(1) Article 1RIB3. Available at: http://www.joe.org/joe/2008february/rb3.php

Smith, M. H., George, J. L., Young, J. C., Meehan, C. L., & Enfield, R. P. (2004). Improving county-based science programs: Bringing out the science teacher in your volunteer leaders. Journal of Extension [online], 42(6) Article 6FEA5. Available at: http://www.joe.org/joe/2004december/a5.shtml

South Carolina Family and Community Leaders (2012). South Carolina Family and Community Leaders History. Retrieved October 7, 2013, from http://www.scfcl.com/history.html

Trochim, W. M. K. (2006). Introduction to evaluation. Retrieved June 9, 2014, from http://www.socialresearchmethods.net/kb/intreval.php

Zamora, D. S., Blinn, C. R., Chura, D. T., Sagor, E. S., Coyle, L. D., & Domke, G. M. (2012). Converting face-to-face curricula for online delivery: Lessons learned from a biomass harvesting guidelines curriculum. Journal of Extension [online], 50(5) Article 5FEA5. Available at: http://www.joe.org/joe/2012october/a5.php