Outcomes of Individual vs. Group Instruction in EFNEP

Introduction

The focus of the Cooperative State Research, Education, and Extension Service (CSREES) is "to advance a global system of research, extension and higher education in the food and agricultural sciences and related environmental and human sciences to benefit people, communities, and the Nation." In Cooperative Extension, this is achieved through educational programs in many disciplines, including food safety, nutrition, and health.

Evaluation is recognized as an essential element in quality programming (Hamilton, Verma, & Burnett, 1996), particularly in the present economic climate as we strive to maintain, even improve, the outcomes of our work with shrinking resources. In order to improve, we must continually and critically assess the effectiveness of the education we are providing (both content and process), reflect on how these results can inform our programming, and adjust our methodologies to improve the impact.

The goal of the Expanded Food and Nutrition Education Program (EFNEP) is "to assist limited resource audiences in acquiring the knowledge, skills, attitudes, and changed behavior necessary for nutritionally sound diets, and to contribute to their personal development and the improvement of the total family diet and nutritional well-being (EFNEP, 2002). During the early years of EFNEP, nutrition education of participants was carried out one-on-one, primarily in the home.

Front-line educators were paraprofessionals, indigenous to the community in which they were working. When hired, front-line educators were not required to have had formal training in either nutrition or educational methodology. However, supervising Extension agents with training in both content and methodology were responsible for providing on-the-job training for their staff. This model of hiring paraprofessionals from the community who receive on-the-job training remains the norm.

In 1979, a Federal EFNEP evaluation was carried out (Chipman & Kendell,1989). The resulting report emphasized the costliness of the one-on-one delivery method and recommended that staff move toward a small group format to improve the cost effectiveness of the program. Given this report, an increased concern for staff safety in individual home environments, and, more recently, the advent of welfare reform and its impact on participant lifestyles, small classes became the preferred educational method, particularly in urban areas.

In the late 1970s and early 1980s, studies were done to investigate the use of more efficient program delivery methods. Reports of pilot projects developing the small-group delivery method cite challenges similar to those encountered in present situations, such as:

• The need for increased training of paraprofessionals in group facilitation skills,
• Language barriers,
• Follow-up for participants missing a class, and
• The difficulty in obtaining evaluation data in a group setting (Neilan, 1985).

In 1981, studies were conducted in 10 state EFNEPs that compared the outcomes of different program delivery methods, including structured one-on-one, small group, and small-group-plus-phone (Experimental Evaluation, 1984). No significant differences were observed when delivery methods were compared. However, in all methods, the first two lessons were delivered one-on-one, so that individual contact was made with each participant. Currently, most participants, at least in New York State (NYS), are educated using either the one-on-one method or the group method, with only about 6% involved in both group and individual lessons.

EFNEP has an on-going evaluation system built into the program, with behavioral outcome data collected on all participants. In NYS we have observed that behavioral outcomes vary by mode of program delivery. This article reports a comparison of behavioral outcome indicators of participants educated individually in their homes vs. those educated in small groups within the community. We suggest possible explanations that should be explored in order to make decisions about programmatic modifications necessary to keep the Program relevant to the target population (Taylor-Powell, Steele, Douglah, 1996). This process could be valuable to other Extension programs seeking to maximize impact with limited resources.

Methods

Data used in the present study represent responses from graduates of the NYS EFNEP for Federal Fiscal Years (FY)1999-2001, which were gleaned from the evaluation instrument that is federally mandated in EFNEP. In NYS, graduation from EFNEP is defined by completion of six or more lessons, completion of both pre- and post-assessment, and the participant's and the front-line educator's assessment that the participant has met goals established upon entry into the Program.

Demographic information, program delivery method (group, individual, both, or other), and responses to 10 self-reported behavior checklist items were used. The checklist items represented four constructs:

• Food safety (2 items),
• Diet quality (5 items),
• Management of food resources (4 items), and
• Food security (1 item), with some items representing more than one construct.

Responses to the checklist items were Likert scales with 5 choices ranging from "almost always" to "do not do." Numeric scores of 1 to 5 were assigned, with "almost always" being 5. Responses to individual items were summed across domains to produce scores and were analyzed as continuous data.

The 3-year data set included a total of 17,160 participants. Of these, 7636 were excluded from the analyses presented here because they did not respond to one or more checklist items or chose "not applicable." There were no differences in this group as compared to the group analyzed (n=9524) based on age, gender, race, income, household size, education, size of home community, or percentage receiving individual vs. group education.

County EFNEPs chose the program delivery method or methods that best suited the local situation. Typically, urban areas chose to deliver education in groups, while rural areas chose both one-on-one and groups, and were more likely to make this choice on the needs of the individual participant.

Front-line educators collected outcome data from participants. Data collection forms were available in English and Spanish. Because many participants have difficulty reading the forms, either because of low literacy skills or they speak and read a language other than English or Spanish, the items on the evaluation instrument were routinely read to participants, and assistance was provided to facilitate the completion of the forms. Local staff then entered the data into the Evaluation Reporting System (ERS4), the software package designed for EFNEP. The data were compiled electronically and submitted to the State EFNEP office, where data from all counties were compiled to represent the statewide evaluation.

Initial comparisons between outcomes from participants taught in groups and those taught individually were made using data for the entire state. A number of urban sites exclusively use groups for program delivery, and some rural sites primarily use individual education. Demographics differ between the urban and rural populations, and outcome data may be influenced by differing experience and skill levels of the front-line educators working with a given delivery method.

We therefore performed a second set of analyses of outcomes from sites providing a balance of programming using the two different delivery methods. Counties were selected from which at least 40% of graduated participants were taught individually and at least 40% were taught in groups. A total of 14 different counties were included in this second analysis; some counties met the inclusion criteria for two or three years, some for only one, such that there were 8, 6, and 8 counties represented, respectively, in each year.

Independent t-tests were used to compare those receiving group education with those receiving individual education (pre-education, post-education, and change) and dependent t-tests were used to compare pre- and post-education scores in the same individual. Changes in scores from pre- to post-education assessment were also calculated and compared.

Results

Table 1 reports results from the analyses of 3 years' data for graduates across the NYS EFNEP. On entry into the program (pre-education), self-reported behavior scores were slightly but significantly better in participants taught individually than in those taught in groups. There was also significantly greater improvement from pre- to post-education among those taught individually. Statistically significant improvement was seen in scores for each of the four constructs. There was no difference in number of lessons provided to those receiving individual (8.3+0.06) as compared to those receiving group (8.4+0.02) education.

The participants in the group receiving individual education were different demographically from those involved in group education. Place of residence, race, income, education level, and ratio of men to women were all significantly different (p<0.0001). Over 79% of participants receiving individual education were white, while 48% of those in groups were Hispanic and 29% were black. These differences reflect the dissimilarity between the Upstate New York population (primarily farms and towns), which is 86% white, and the NYC population which is only 45% white (U.S. Census Bureau, 2001).

Over 86% of those receiving individual instruction lived on farms or in towns of <50,000 population, with only 9% living in cities or suburbs. Alternatively, most participating in groups lived in cities (76%) or suburbs (4.5%), with slightly over 20% living in more rural areas. Self-reported family income was higher among those taught individually ($955+10) as compared to those taught in groups ($765+12). Household size was slightly but significantly larger among those taught in groups (4.3+0.02 vs. 4.1+ 0.03, p<0.02). Education level was slightly but significantly higher (p<0.0001) among those taught individually (last grade completed 10.6+0.07) vs. those taught in groups (10.1 +0.04).

Based on potential differences by county as well as the demographic differences in those involved in group as compared to individual education, we then completed a second set of analyses. These included behavior change among a sub-set of participants in 14 counties with approximately equal portions of participants involved in each of the two educational delivery methods (Table 2).

Scores were not different at entry into the program (pre-education) either by individual construct or overall summary score. All participants reported improved behavior post-education, but the change among those taught individually was greater than among those taught in groups. It is interesting to note that the mean number of lessons provided to individuals was greater than the number provided to groups, 8.1+0.1 vs. 6.9+0.1, respectively (p<0.0001). This is attributable to the increased flexibility inherent in deciding when an individual is ready for graduation as compared to a group for which nutrition education is occurring as a part of another agency's program.

Demographics were more similar between those participants taught in groups and those taught individually than was true for the statewide data, but there were still some differences. Place of residence, age, education, household size, and income were not significantly different. However, there were more white participants involved in individual education and more black participants in group education (p<0.0001). Thirteen percent of those educated in groups were men as compared to 6% of those educated individually (p<0.001).

Discussion

Across the United States in FY2000, 73,663 adults participated in a series of EFNEP lessons with completion of pre- and post-education behavioral outcome data, including a food behavior checklist and a 24-hour dietary recall (EFNEP FY00). Of these, 73% were taught in a group setting, usually in a community or agency center; 20% were taught individually, usually in the participant's home; 6% in a combination of small group and individual instruction; and 1% using "other" delivery methods (Montgomery, S., personal communication, 2001).

In NYS, the percentage of participants reached in groups has shown a steady increase, with 50% participating in group education in FY1998, 61% in FY1999, 67% in FY2000, and 70% in FY2001. Is this shift consistent with maintaining quality programming and expected outcomes?

The data reported here indicate that, as currently delivered, programmatic impact is suffering as a result of this shift. Previous EFNEP data indicating that outcomes were similar among participants taught individually and in groups were based on a methodology in which the groups were recruited and educated individually for two lessons, followed by a series of group lessons. This is not the current method in most venues.

EFNEP is normally delivered by paraprofessionals who are recruited to be indigenous to the communities being served and hence able to easily relate to the participants  (Brink, 2000). Intensive on-the-job-training is then conducted to provide staff with the skills necessary for program delivery. This model has been used since the inception of the program. However, political and social changes have resulted in a very different environment in many sites where the program is delivered, increasing the challenges both of recruitment of participants and of program delivery.

Participants in the EFNEP have historically been recruited in two primary ways, referral from other agencies providing services to the target population, such as WIC and Head Start, and word-of-mouth referral from previous participants or others in the community.

Reasons Education Has Shifted from Primarily One-on-One to Group

First, group education has been thought to be a more cost-effective method of program delivery in an era of sharply declining resources (Chipman & Kendall, 1989).

Second, safety of front-line workers became a consideration, particularly in urban areas. For example, in the early 1980's the Program in NYC moved to providing all education in group settings to ensure staff safety.

Third, recruitment of individual participants has become increasingly difficult in the last 5 years, since the advent of welfare reform, because adults are now in the work force or in work preparation rather than being available for program delivery in homes or in small groups organized specifically for EFNEP.

Instead, participants are most often found by recruiting agencies working with groups of the target population, and the program is delivered to a group that exists for another purpose, such as Welfare-to-Work programs, Head Start parent groups, groups in half-way houses for drug and alcohol rehabilitation, etc. The number of lessons provided, the number of participants per group, and the specific composition of the group are often dictated by the agency with varying degrees of negotiation possible.

Advantages to Reaching Participants Through Pre-Existing Groups

• Ease in recruiting participants who are not readily available otherwise,
• Decreased cost per participant because a larger number can be reached, and
• Reliable attendance in the case of programs such as Welfare-to-Work or halfway houses.

Challenges to Group Education

• It is more likely that the number of lessons to be provided will be dictated by the agency rather than by a needs assessment of the group coupled with on-going evaluation of goal achievement.
• The number of participants may be larger than ideal, particularly for paraprofessional staff with limited training in group facilitation methods.
• The composition of the group may provide special challenges, for example participants who need individual attention because of limited reading skills or language barriers.
• Group dynamics may be a problem because of the different languages and cultures represented in a group.

For example, in New York City, EFNEP groups average 12 participants, with two to four different languages spoken in a given group and few individuals speaking more than one language. One recent example occurred in a group of 12 in which 8 languages were spoken, including English, Spanish, Russian, French, Haitian Creole, Korean, Chinese, and an African dialect.

Participants in such groups are often new immigrants who desperately need the material taught in EFNEP and are grateful for the work done as a part of the program, but the paraprofessional, even if bilingual, will only speak English and Spanish or English and Haitian Creole. She cannot come close to being indigenous to this population; no one person can. She may also not have the skills necessary to facilitate a group of this complexity.

Decreased cost per participant does not necessarily translate into increased cost-effectiveness, and the data reported here question the cost-effectiveness of group education as compared to one-on-one education. However, the current political and community climate does not support reversal of the trend toward more groups.

If the increase in groups is inevitable, what changes are necessary to assure excellence in programming? Program leaders and educators need to carefully consider:

• Training needs of staff working with groups and, where needed, with culturally and linguistically diverse groups;
• Choice of collaborating agencies and agreements with these agencies that allow sizes of groups, programming flexibility, and individualization to meet participant needs that are in line with adult learning theory (Cantor, 2001); and
• Educational strategies that allow for individualization within the context of a group, such as telephone contact or, where possible, one or two individual sessions.

Conclusions

Cooperative Extension programs are delivered in a very different environment today than in the past. On-going evaluation is critical to assure that programs adjust with the changing times and continue to be effective in meeting the needs of participants.

EFNEP provides examples of the challenges faced throughout Extension. Societal and political changes mean that Extension educators are working with increasingly multi-ethnic populations.  We have turned more to program delivery in group settings, often with groups convened by another agency for another purpose.  Facilitation of the highly heterogeneous groups that result, particularly in urban areas, requires skills that educators may not possess. Further research will be important to fully understand:

• The barriers encountered in the educational process that limit outcomes in groups,
• Training necessary to equip staff to provide the most effective group education,
• Changes in organization of programming that are needed to provide more individualized help for participants
• The actual cost-benefit of groups vs. individual instruction.

Results from this research will provide the information necessary for staff to adjust program planning, staff training, and program delivery to better meet the needs of the target audiences of the 21st Century.

References

Brink, M. (2000). Expanded Food and Nutrition Education Program. A precedent-setting program. Cortland, NY: EasyWriter Publications.

Cantor, J.A. (2001). Delivering instruction to adult learners. Dayton OH: Wall and Emerson.

Chipman, H., & Kendell P. (1989). 20 years of EFNEP: changes and challenges. Journal of Nutrition Education 21:265-269.

CSREES. Learn about CSREES. Mission statement. [On-line]. Available at: http://www.reeusda.gov/1700/about/csreesa2.htm#mission

EFNEP [On-line]. Available at: http://www.reeusda.gov/f4hn/efnep/about.htm

EFNEP FY00 Adult Data Fact Sheet. [On-line]. Available at:  http://www.reeusda.gov/f4hn/efnep/00facts.PDF

Experimental evaluation of nutrition education methods. (1984). Menlo Park, CA: Stanford Research Institute International.

ERS4. Evaluation/Reporting System 4.0. [On-line]. Available at: http://www.reeusda.gov/ers4/home.htm

Hamilton, C.H., Verma, S., & Burnett, M. (1996). Extension education evaluation: An evolutionary perspective with implications for theory and practice. Paper presented at Extension Education Evaluation Topical Interest Group, American Evaluation Association Annual Meeting, Atlanta, Georgia. [On-line]. Available at: http://danr.ucop.edu/eee-aea/aeapaper.html

Neilan, A.M. (1985). EFNEP innovative projects: Evaluating the effectiveness of small group methods in nutrition education for young mothers receiving public food assistance. Blacksburg, VA: The Virginia Cooperative Extension Service.

Taylor-Powell, E., Steele, S., & Douglah, M. (1996). Planning a program evaluation. Madison, WI: University of Wisconsin-Extension, Cooperative Extension. [On-line]. Available at: http://cf.uwex.edu/ces/pubs/pdf/G3658_1.PDF

U.S. Census Bureau. Census 2000 data. [On-line]. Available at: http://www.census.gov/