June 2016 // Volume 54 // Number 3 // Ideas at Work // v54-3iw2
Cooking Healthy, Eating Smart: A Strategically Timed Formative Evaluation of a Community-Based Nutrition and Food Safety Program for Rural Older Adults
The use of focus groups to formatively evaluate community-based curricula after development and before pilot testing is not highlighted in the literature. In the study discussed in this article, research with four focus groups, composed of 46 women aged 65 years and older and belonging to eight South Carolina Family and Community Leaders clubs, was conducted to evaluate the Cooking Healthy, Eating Smart (CHES) curriculum. The CHES curriculum was tailored based on suggestions from the older adults. Extension professionals can better prepare a curriculum for pilot testing in the community by conducting a formative evaluation using focus groups with community members at this particular stage of development.
Formative evaluation (FE) is essential to program development and improvement (Herman, Morris, Fitz-Gibbon, 1987; Joint Committee on Standards for Educational Evaluation, 1994; Patton, 1987). Trochim (2006) delineated five types of FE—needs assessment, evaluability assessment, structured conceptualization, implementation evaluation, and process evaluation—highlighting the need for FE at multiple points in time. In education, these types of FE can be completed before the development of a curriculum (needs assessment, evaluability assessment, or structured conceptualization) and after the curriculum has been pilot tested (implementation evaluation or process evaluation). We believe an FE should take place at each of these times due to the ongoing nature of curriculum improvement, but the most important (and least reported on) evaluation point is between development and pilot testing. The aim of this article is to present the results of a formative evaluation conducted at this point in time, using a focus group methodology.
When seeking to tailor a community-based curriculum and prepare it for pilot testing, educators can use focus groups composed of members from the target population to formatively evaluate the intervention. Conducting focus group research according to Morgan and Krueger's methodology (Morgan, Krueger, & King, 1998) provides meaningful data that can set the stage for a more efficient and effective pilot test. Although focus group research is a common method of FE at other points in time, we found few articles in the literature reporting on focus group research conducted with community members between development and pilot testing of community-based curricula.
Many developers of community-based curricula convene focus groups before development (Benke et al., 2013; Dumbauld, Kalichman, Bell, Dagnino, & Taras, 2014; Healey, Reed, & Huber, 2013; Nguyen, Tran, Kagawa-Singer, & Foo, 2011; Robinson, Wong, Konzelmann, Bielamowicz, & Rodgers, 2008). Others use focus groups to evaluate curricula after implementation (Besculides, Trebino, & Nelson, 2012; Hayes et al., 2010; Pettman et al., 2008; Smith, George, Young, Meehan, & Enfield, 2004). Few articles reference formatively evaluating a curriculum between development and pilot testing (Coronado, Acorda, Do, & Taylor, 2008; Parsai, Castro, Marsiglia, Harthun, & Valdez, 2011; Zamora et al., 2012). We believe the literature should highlight the importance of FE at this point in time.
Formative Evaluation of a Curriculum
In our study, we formatively evaluated a nutrition and food safety curriculum targeting rural older adults at three points in time:
- Before development, we spoke with staff of the South Carolina Lieutenant Governor's Office on Aging and with rural older adults about the nutrition and food safety education needs of rural older adults.
- Between development and pilot testing, we conducted focus group research with a sample of rural older adults and used their input to further refine the curriculum.
- After pilot testing, we collected feedback from program participants and the Extension educators who delivered the curriculum.
Each stage of FE was necessary, but in this article we highlight the method and findings of our second stage of FE, which occurred at the least commonly reported time—between curriculum development and pilot testing.
The Cooking Healthy, Eating Smart (CHES) curriculum was developed by investigators from Clemson University and the Medical University of South Carolina to teach rural, low-income older adults in the selection and preparation of safe, healthful foods within a budget. On the basis of the revised Bloom's taxonomy (Anderson et al., 2001), each of the eight lessons in the curriculum included an objective, learning questions, a lesson content summary, an activities chart, activity guides, a supply list, and recipe handouts. The lessons were titled: Less Fat, Less Salt, More Fiber, More Water, More Calcium, Protein, More Fruits and Vegetables, and Food Safety.
An Extension agent conducted four focus groups with members of South Carolina Family and Community Leaders (SCFCL)—a volunteer organization targeting older adults (South Carolina Family and Community Leaders, 2012)—to formatively evaluate the CHES curriculum prior to its pilot test. The focus group research was conducted in four rural South Carolina counties with representatives from eight SCFCL clubs. The agent asked scripted questions (Table 1) and followed a protocol guided by Morgan and Krueger's recommended methodologies (Morgan et al., 1998).
Participants and Focus Group Results
Each focus group comprised nine to 18 participants, resulting in a total of 46 participants. Only one focus group had more than the maximum of 12 participants recommended by Morgan et al. (1998). All focus group participants were female, reflecting a female dominance in SCFCL groups. More participants were African American (59%) than White (41%). Table 2 provides additional information about the characteristics of the focus groups.
|Characteristic category||Group 1||Group 2||Group 3||Group 4|
|Date||January 18, 2010||January 19, 2010||February 8, 2010||February 16, 2010|
|Number of SCFCL clubs represented||1||1||1||5|
|Number of participants||9||18||10||9|
|CHES lessons reviewed||Less Fat, Less Salt||More Fiber, More Fruits and Vegetables||More Calcium, More Water||Protein, Food Safety|
The focus group participants critically evaluated the curriculum and suggested needs for revision that had been unapparent to investigators. Their suggestions resulted in improvements. For example, an entire activity is now devoted to lactose intolerance, teaching strategies for obtaining adequate amounts of calcium when consuming lactose-free foods. Also, each recipe included in the curriculum now produces only one to four servings in order to decrease amounts of leftover food. Additionally, the CHES curriculum now advises older adults to sip water throughout the day, drink "at will" while at home, and avoid consuming liquids within 3 hr of going to bed or going out in public.
On the basis of participants' suggestions, specific changes were made to six of the eight lessons. The strategic timing of this FE—between curriculum development and pilot testing—allowed community members to provide practical criticisms for improving the curriculum at a point at which changes still could easily be made. Obtaining specific criticisms is impossible before a curriculum has been developed, and waiting until after it has been pilot tested to identify necessary changes wastes time and resources (Breault & Gould, 1998). Thus, FE must occur between development and pilot testing in addition to at the more commonly reported times.
Formative evaluations of a curriculum should be conducted at various stages for continued improvement. To tailor a curriculum and prepare it for pilot testing, it is best to conduct focus group research with members of the target population after development and prior to pilot testing to improve program design and effectiveness. Extension professionals can use this example in planning formative evaluations of their curricula.
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., . . . (Eds.) (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives: Abridged edition. New York, NY: Addison Wesley Longman, Inc.
Benke, C. J., Bailey, S. J., Martz, J., Paul, L., Lynch, W., & Eldridge, G. (2013). Developing a parent-centered obesity prevention program for 4-H families: Implications for Extension family programming. Journal of Extension [online], 51(3) Article 3FEA8. Available at: http://www.joe.org/joe/2013june/a8.php
Besculides, M., Trebino, L., & Nelson, H. (2012). Successful strategies for educating hard-to-reach populations: Lessons learned from Massachusetts' train-the-trainer project using the "Helping You Take Care of Yourself" curriculum. Health Education Journal, 71(3), 350–357.
Breault, J. L., & Gould, R. (1998). Formative evaluation of a video and training manual on feeding children with special needs. Journal of Nutrition Education and Behavior, 30(1), 58–61.
Coronado, G. D., Acorda, E., Do, H. H., & Taylor, V. M. (2008). Feasibility and acceptability of an English-as-a-second language curriculum on hepatitis B for older Chinese American immigrants. Journal of Health Disparities Research & Practice, 2(3), 121–131.
Dumbauld, J., Kalichman, M., Bell, Y., Dagnino, C., & Taras, H. L. (2014). Case study in designing a research fundamentals curriculum for community health workers: A university–community clinic collaboration. Health Promotion Practice, 15(1), 79–85. doi:10.1177/1524839913504416
Hayes, A., Morzinski, J., Ertl, K., Wurm, C., Patterson, L., Wilke, N., & Whittle, J. (2010). Preliminary description of the feasibility of using peer leaders to encourage hypertension self-management. WMJ: Official Publication of the State Medical Society of Wisconsin, 109(2), 85–90.
Healey, W. E., Reed, M., & Huber, G. (2013). Creating a community-physical therapy partnership to increase physical activity in urban African-American adults. Progress in Community Health Partnerships: Research, Education, and Action, 7(3), 255–262. doi:10.1353/cpr.2013.0038
Herman, J. L., Morris, L. L., & Fitz-Gibbon, C. T. (1987). Evaluator's Handbook. Newbury Park, CA: Sage Publications, Inc.
Joint Committee on Standards for Educational Evaluation. (1994). Sanders, J. R. (Chair), The program evaluation standards: How to assess evaluations of educational programs (2nd ed.). Thousand Oaks, CA: Sage Publications, Inc.
Morgan, D. L., Krueger, R. A., & King, J. A. (1998). The Focus Group Kit, Vols. 1–6. Thousand Oaks, CA: Sage Publications, Inc.
Nguyen, T. N., Tran, J. H., Kagawa-Singer, M., Foo, M. A. (2011). A qualitative assessment of community-based breast health navigation services for Southeast Asian women in Southern California: Recommendations for developing a navigator training curriculum. American Journal of Public Health, 101(1), 87. doi:10.2105/AJPH.2009.176743
Parsai, M. B., Castro, F. G., Marsiglia, F. F., Harthun, M. L., & Valdez, H. (2011). Using community based participatory research to create a culturally grounded intervention for parents and youth to prevent risky behaviors. Prevention Science, 12(1), 34–47. doi:10.1007/s11121-010-0188-z
Patton, M. Q. (1987). How to use qualitative methods in evaluation. Newbury Park, CA: Sage Publications, Inc.
Pettman, T. L., Misan, G. M. H., Owen, K., Warren, K., Coates, A. M., Buckley, J. D., & Howe, P. R. C. (2008). Self-management for obesity and cardio-metabolic fitness: Description and evaluation of the lifestyle modification program of a randomised controlled trial. International Journal of Behavioral Nutrition and Physical Activity, 5, 15.
Robinson, S. F., Wong, W. W., Konzelmann, K. L., Bielamowicz, M. K., & Rodgers, A. S. (2008). Designing a bone health and soy focus group discussion guide based on the health belief model. Journal of Extension [online], 46(1) Article 1RIB3. Available at: http://www.joe.org/joe/2008february/rb3.php
Smith, M. H., George, J. L., Young, J. C., Meehan, C. L., & Enfield, R. P. (2004). Improving county-based science programs: Bringing out the science teacher in your volunteer leaders. Journal of Extension [online], 42(6) Article 6FEA5. Available at: http://www.joe.org/joe/2004december/a5.shtml
South Carolina Family and Community Leaders (2012). South Carolina Family and Community Leaders History. Retrieved October 7, 2013, from http://www.scfcl.com/history.html
Trochim, W. M. K. (2006). Introduction to evaluation. Retrieved June 9, 2014, from http://www.socialresearchmethods.net/kb/intreval.php
Zamora, D. S., Blinn, C. R., Chura, D. T., Sagor, E. S., Coyle, L. D., & Domke, G. M. (2012). Converting face-to-face curricula for online delivery: Lessons learned from a biomass harvesting guidelines curriculum. Journal of Extension [online], 50(5) Article 5FEA5. Available at: http://www.joe.org/joe/2012october/a5.php