Evaluation of Dietary Guideline Bulletins Revised for a Low Literate Audience

Despite recent advances in audio and video technology, the field of nutrition education in the United States is overwhelmingly print-oriented. In addition, most of the print materials available are written at a level that makes them inaccessible to low- to mid-literate individuals (Hynak-Hankison, 1989; Stemmerman, 1991).

The purpose of this study was to compare changes in knowledge that resulted from reading revised and unrevised nutrition print materials in a sample of lower-literate adults, i.e., adults who read at the 5th-8th grade level. The study and materials were developed by a collaborative, team approach that included specialists from Cooperative Extension, nutrition, reading and literacy, and adult learning.

The control bulletins were adapted directly from the Human Nutrition Information Service (1989a, 1989b, 1989c & 1989d) dietary guidelines. The content was unchanged in the control versions, but relatively small sections were selected out of the original set of four bulletins and reproduced in three shorter booklets ranging from 4-9 pages in length. The control bulletins focused on individual topics as opposed to the original brochures which addressed several different topics at once. Paper size was unchanged at 8 1/2 x 11 inches. The treatment bulletins or revised materials were reformatted and rewritten to improve readability and (theoretically) comprehension of the information. The reformatting was guided by standard recommendations for materials targeted to lower literate audiences (Collins & Read, 1989; Doak, Doak & Root, 1985; Nitzke, 1987). The smaller sections of content, larger lettering, and increased white space combine to make the materials easier to process and read (Gaston & Daniels, 1988; Van Horn, 1989). The reading level was reduced from grade 9 to a grade 4 level as assessed by a computerized readability program (RIGHTWRITER, 1992). Paper size was unchanged. The topics, number, and length of treatment bulletins matched the control bulletins.

Four focus groups were conducted during the formative stages of bulletin development with representatives of the target audience recruited from Adult Basic Education (ABE) classes. These focus groups helped the project team with the choice of illustrations and graphics, lay-out and format, content focus, and word choice used in the treatment bulletins. Both sets of bulletins addressed the identical issues including shopping and label reading, fast food restaurants, and snacks.

Study participants (n = 295) were recruited from both rural (48%) and urban (51%) areas from ABE classes and job retraining programs. Participants were randomly assigned into treatment or control groups.

A 35-item multiple choice test was administered immediately prior to reading either a treatment or control bulletin and again immediately after reading a bulletin. A short demographic survey was also administered with the pretest as well as the reading subtests of the Tests of Adult Basic Education (CTB/McGraw-Hill, 1987) to verify literacy levels. Descriptive statistics and analysis of variance for group comparisons were computed using SPSS (1988).

The age of the study participants ranged from 16-60+ years with the largest number (43.7%) of the sample in their twenties. Most (71%) were women, single (57.6%), had at least some high school education (63.1%), were unemployed (73.9%), and earned less than $7,500 per year (56.9%). Almost a third had children (32.9%) or other adults (30.5%) living with them. A majority (72.2%) reported no health problems and most (71.5%) performed their own food shopping. Only 6.8% were Expanded Food and Nutrition Program (EFNEP) participants. Twenty percent of the sample read at less than a 5th grade level (low-literate), 54.5% read at a 5th-8th grade level (mid-literate), and 25% read above the 8th grade level. There were no differences between treatment and control groups.

Mean scores for pre- and post-tests indicated no statistical difference between treatment and control bulletins overall or for the rural or urban subgroups. There was no difference in knowledge gain from pre- to post-intervention by reading level, although readers with a higher literacy level scored higher in knowledge on both pre- and post-tests regardless of any other factor (see Table 1). In short, every group made approximately a two-point gain from pre- to post-intervention and the revisions appeared to have a negligible effect.

Because these results were contrary to what was expected, we re-examined the data on an item-by-item basis to determine if there were more subtle changes made. Changes on any test question were unlikely from pre- to post-intervention unless an individual marked unsure to that question on the pretest. For example, one question read, "Which is not a type of sugar?" Almost 28% of the sample were not sure at pre-test, but only 11% were not sure at post-test. This 17% change accounted for almost all of the 22% gain in the correct answer at post-intervention. In a few cases, participants changed an incorrect answer to a correct answer. For example, 73% of the sample incorrectly said, "The healthiest margarine has: no cholesterol" in the pretest. This response dropped 13.6% at the post-test where as the correct answer, "The healthiest margarine has: polyunsaturated fatty acids listed first on the label," increased by 13.9% on the post-test. However, when participants changed their answer from pre- to post-test, it was not always in the proper direction. Thus, the overall scores did not change significantly from pre- to post-intervention.

We concluded that the lack of improvement in knowledge due to materials revision may have been due to a relatively weak intervention, an inappropriate sample, and/or an inappropriate evaluation. Since great care was taken in the development process to ensure that the revisions made to the bulletins reflected the best available knowledge on how to communicate to lower literate audiences, this explanation appeared unreasonable. Only half the sample actually tested at the reading level targeted (5-8th grade), but it was large enough to detect a change if there had been one. Moreover, there should have been a differential effect with the other reading levels and there was none. Much less time was spent on developing the evaluation instrument than on developing the reading materials. Thus, the test instrument itself may have interfered with the participants exhibiting the extent of their learning. If so, we would have expected to see a differential effect on gain that depended on reading level. Verbal interviews or other qualitative evaluations might have produced different results.

With no alternative explanation, we were forced to re-examine our materials and initial assumptions. We finally concluded that, in fact, it was not worth the effort of rewriting the original materials for a lower literate adult audience. Merely reducing the volume of content in the original bulletins appeared to be as satisfactory as rewriting and reformatting. However, the process of revision itself may have been the fundamental flaw. We recommend that in the future educators develop materials for this target audience from scratch rather than revise pre-existing materials. These materials should prove more effective, but this hypothesis should be tested as well!

The results of this research have direct implications for Extension outreach efforts. Literacy levels are a major concern for print brochures, especially since many of the readily available materials from USDA are written at a 9-12 grade reading level. Therefore, usefulness of these materials with many audiences at high risk has been questioned. Simultaneously, the effort to revise such materials is very time consuming and costly. This study implies that concern is indeed warranted; however, it also suggests that Extension should not invest limited resources in revision efforts, but develop new materials specifically for lower literate audiences when possible.

Collins, D. E., & Read, H. (1989). Plain English: A guide to standard usage and clear writing. Scarborough, NJ: Prentice-Hall.

CTB/McGraw-Hill. (1987). Tests of adult basic education. Monterey, CA: Author.

Doak, C., Doak, L., & Root, J. (1985). Teaching patients with low literacy skills. Philadelphia: J.B. Lippincott.

Gaston, N., & Daniels, P. (February, 1988). Guidelines for writing for adults with limited reading skills. Washington, DC: U.S. Government Printing Office.

Human Nutrition Information Service. (1989a). Preparing foods and planning menus using the dietary guidelines (USDA Publication No. HG 232-8). Washington, DC: U.S. Government Printing Office.

Human Nutrition Information Service. (1989b). Making bag lunches, snacks, and desserts using the dietary guidelines (USDA Publication No. HG 232-9). Washington, DC: U.S. Government Printing Office.

Human Nutrition Information Service. (1989c). Shopping for food and making meals in minutes using the dietary guidelines (USDA Publication No. HG 232-10). Washington, DC: U.S. Government Printing Office.

Human Nutrition Information Service. (1989d). Eating better when eating out using the dietary guidelines (USDA Publication No. HG 232-11). Washington, DC: U.S. Government Printing Office.

Hynak-Hankison, M. T. (October, 1989). Literacy crisis: Implications for nutrition and dietetic professionals. Topics in Clinical Nutrition, 4, 63-70.

Nitzke, S. (1987). Research into practice example: Reaching low-literate adults with printed nutrition materials. Journal of the American Dietetic Association, 87 (Suppl. 9), S73-77.

RIGHTWRITER. (1992). User's manual [computer program]. Carmel, IN: Que Software, A Division of Prentice Hall Computer Publishing.

Statistical Package for Social Sciences. (1988). SPSS-X User's Guide (3rd ed.) [computer program manual]. Chicago: SPSS, Inc.

Stemmerman, M. G. (November, 1991). Readability of selected public health information materials. Tri-State Literacy Council, Cabell County Public Library, 4559 9th Street Plaza, Huntington, WV 25701.

Van Horn, B. H. (1989). Readability. Institute for the Study of Adult Literacy. The Pennsylvania State University, 204 Calder Way, Suite 209, University Park, PA 16801-4756.

Author Notes

This research was supported, in part, by a CSRS approved Hatch Grant #3110 from the Agriculture Experiment Station, College of Agriculture, The Pennsylvania State University.