How to Ascertain the Impact of Writing a Newspaper Column

Introduction

The need to disseminate research-based information in a context of diminishing resources is a challenge faced by Extension today in both urban and rural populations and across all program areas (McDowell, 2001).

Extension educators face several barriers in reaching very large audiences when using traditional face-to-face delivery methods. Face-to-face delivery methods require extensive time from Extension educators, time for learners to travel and attend workshops, and funds for both the educator and the learner to travel. USDA research and Extension scholars point to the need for Extension to change its methods of delivering research-based information and use media to reach more audiences (Boldt, 1987; Seevers, Graham, Gamon, & Conklin, 1997). Utilizing a newspaper column as a teaching method has the potential of reaching more people at a relatively low cost and overcoming the barrier of time for both the educator and the learner.

The newspaper could be a valuable delivery method for Extension when educators need to reach a large target audience about new research findings in areas such as water quality, food safety, and health. The literature reveals the importance and use of the newspaper by the public in educating themselves about food and its relationship to preventing diseases in particular. "When citizens were asked if they'd found any one of five kinds of information (nutrition, personal relations, energy conservation, family finances, or child raising) in mass media, 81% of respondents indicated that they found nutrition information in newspapers" (Pounds, 1985, p. 4).

A more recent study, prepared for the American Dietetic Association in 2002, found that even fewer persons sought nutrition information from doctors than in 1999. At the same time, both newspaper and magazines increased in their popularity as sources of nutrition information (Wirthilin, 2002). Although citizens look to the newspapers for information on nutrition, further research indicates that journalists who do write on nutrition do not always write accurate and useful columns (Center for Media and Public Affairs, INFIC, 2005). Given the public's interest and desire for nutrition information from newspapers, Extension has an exciting opportunity to fill this gap and provide research based nutrition information.

The theoretical literature in distance education suggests that using a newspaper as a delivery method creates an environment suitable for many learners. Theorists suggest that if the environment provides the element of control, allowing the learner to influence and direct many decisions related to the educational process e.g., when and where to learn, such an environment can influence and determine the educational impact (Garrison & Baynton, 1987). Choosing to learn by reading a newspaper column can be very individualized and private. Without group observation, the learner can have a high comfort level and sense of control.

Based on research indicating that citizens use newspapers as source of nutrition information and on control theory in distance education suggesting the value of newspapers as a delivery method, Extension launched a new program utilizing a newspaper column, which is described below. But measuring the impact that may result from reading a newspaper column presents a great challenge because it is almost impossible to identify and obtain contact information for the people who are reading the newspaper column in order to include them in a study. One approach to capture Extension's impact from utilizing a newspaper column as a delivery method is also described along with the results.

Program Delivery Method

In Northeastern Pennsylvania, the Department of Health contacted Extension after identifying hypercholesterolemia in children. Clinical assessments conducted by health officials identified 63% of 116 elementary school students as having elevated cholesterol levels. Research suggested that both genetics and poor eating habits led to the high cholesterol levels among the children (USDA Health and Human Services, 1991). Knowing that changing genetics was not a viable option, Extension chose to address the eating habits of the population.

The preponderance of research-based information in nutrition and its link to health called for Extension to program with the most effective and efficient use of resources. The Extension educator decided to use the local newspaper, the Sunday Dispatch, as the delivery method, due to its wide community distribution, having a circulation of 8,000, with a little over 50% subscribers.

Over 3 years, 40 columns appeared in the newspaper. The columns reflected the program objectives: that readers, the learners, would increase their awareness of the amount of fruit, vegetables, and calcium-rich foods they need to eat in a day and that they would eat the recommended amount of fruit, vegetables, and calcium-rich foods in a day.

Evaluation Method

Rationale

The evaluation of the program had three purposes. First, Extension needed to understand the effectiveness of this delivery method (Goldberg et al. 2004). Although Extension has evaluated the effectiveness of face-to-face delivery methods (Peterson, 2000), satellite seminars (Kiernan, Turgeon, & Hoffman, 1995; Heinrichs, Kiernan, & Ishler, 1996), and Web sites (Kiernan & Alter, 2004), there has been, with few exceptions (Smith, 1984), little evaluation of the newspaper column as a delivery method in Extension.

Second, the Extension educator wanted to know the nature of the impact on the readers in terms of knowledge and behavior change. The driving question was, to what extent was the newspaper column influencing its readers? Additionally, "proving program impact is important to justify the investment of time and effort" (Diem, 2003, p.1).

Finally, the Extension educator wanted to demonstrate to the Managing Editor of the newspaper that the column was effective and to have it featured often and more consistently. Before the study, the Managing Editor would print the column only on a bimonthly basis.

Evaluation Strategy

Measuring impact created a challenge, as data about who reads a particular column are typically unknown. Newspaper subscription lists are not public information, and many people also purchase newspapers at a retail store. To obtain a valid list of recurring readers, Extension established a collaborative relationship with the newspaper's Managing Editor.

Once a collaborative relationship was established, Extension considered three approaches to disseminate the evaluation: first, to request a random sample of the subscriber list so that Extension could send out the evaluation; second, to give the evaluation to the Managing Editor and ask that the newspaper send the evaluation to a random sample of the subscriber list; and third, to request that several evaluation questions be added to the newspaper's next print survey or focus group with their subscribers.

The Managing Editor of the Sunday Dispatch, very interested in knowing the impact of the column, agreed to the first approach, to share the subscriber mailing list with the provision that the subscriber names not be used for other purposes. The mailing list provided a defined scientific population from which to draw a sample (Rossi, Freeman, & Lypsey, 1999), and the results could be generalized to subscribers as a whole.

As recommended in the research, the mail survey comprised an 8 1/2" x 11" booklet form (Dillman, 2000) and included 5 questions. The introductory question, a screening question, measured the extent to which the subscriber read the column within the past 6 months.

The next two questions measured the impact of the column on the knowledge and behavior change of the readers. The first of these questions measured the extent to which the reader had increased their awareness of the need to eat recommended portions of vegetables and fruit, and recommended amounts of calcium-rich foods. The second question measured the extent to which the reader increased their portions of vegetables and fruit, and the amount of calcium-rich foods before reading the column, and after reading it.

After the subscriber mailing list was reviewed for duplicates and missing addresses, 800 subscribers from the total of 4027 subscribers were chosen randomly for the study.

The Extension educator implemented the survey in four steps. The first was the announcement of the survey, because research has shown that response will increase if a respondent believes the survey has worth (Dillman, 2000). In this study, the Managing Editor of the Sunday Dispatch announced the survey in an article with a picture of the Extension educator in the newspaper a week before the survey was mailed. The article asked readers to respond should they be randomly chosen and receive the survey.

The second step was the initial mailing of the survey with a letter describing how respondents would remain anonymous and how their names became known. Included was a preprinted postage paid envelope, a research-based procedure to increase the response rate (Dillman, 2000). The envelope not only decreased the burden on the respondents, but it reduced the postage cost for Extension because only return postage had to be paid.

The third step was a broad reminder. At the Managing Editor's suggestion, the Extension educator wrote a "Letter to the Editor," reminding readers to return their survey. The final step was a second mailing of the survey with the postage paid envelope, the accompanying letter being shorter, but stronger.

Findings

Mail surveys were returned at a rate of 26%. Assuming a 95% confidence level, that is, identifying a risk of 1 in 20 that the actual/error is larger, the margin of error was ± 3%.

When asked, "To what extent have you been able to read the column "Nutrition Corner" in the last six months?" we found that of the subscribers who responded (N=215), fully 60% looked at the columns in some way, i.e., scan some, scan and read some, or, usually read all (Figure 1).

Figure 1.
To What Extent Have Your Read "Nutrition Corner?" N=215

When subscribers were asked about their awareness of the need to consume the recommended daily amount of fruit, vegetables, and calcium-rich foods, we found that among those who looked at the column in some way: (N=130-132)

• 41% increased awareness of fruit by "quite a bit" or "a great deal"


• 39% increased awareness of vegetables by "quite a bit" or "a great deal"


• 45% increased awareness of calcium by "quite a bit" or "a great deal"

The survey also asked subscribers about their eating behavior before and after reading the column (Figures 2,3,4). On average, 15% changed their eating behaviors about the consumption of recommended servings of fruits, vegetables, and calcium-rich foods from hardly ever and sometimes, to usually.

Figure 2.
Percent Eating Two Servings of Fruit per Day Before and After Reading Column (N=132)

Figure 3.
Percent Eating Three Servings of Vegetables per Day
Before and After Reading Column (N=132)

Figure 4.
Percent Consuming Recommended Amount of Calcium
Before and After Reading Column (N=130)

Discussion

The study demonstrated that the Sunday Dispatch nutrition column was being read. We can speculate, as did the Managing Editor, that if the subscribers who responded are representative of the entire subscriber readership of just over 4,000, then 2,416 readers were looking at the column in some way in the 6 months previous to the survey.

The survey demonstrated that 39-41% of the respondents increased their awareness of the need to consume the recommended daily amount of fruit, vegetables, and calcium-rich foods, and on average, 15% of the respondents changed their eating behaviors in a specific way: increasing their consumption of recommended servings of fruits, vegetables, and calcium-rich foods from hardly ever and sometimes to usually. We can speculate, as did the Managing Editor, that if the subscribers who responded are representative of the entire subscriber readership, from 1,571 to 1,812 subscribers increased their awareness of the need to consume the recommended daily portions of specific foods, 1,007 made general nutritional changes, and 605 readers changed their eating behavior about the consumption of recommended daily portions of specific foods to "usually."

As a result of these findings, the owners of the newspaper endorsed the value of the column. Additionally, the Managing Editor gave the column more recognition by placing it in a regular location within the paper and increased the Nutrition Corner column's frequency from biweekly to weekly to help increase readership, thus supporting the Extension program delivered by newspaper. Previous to this study, the column was placed when and where there was room for it.

The study, examining the impact of an Extension program delivered through a newspaper column, has implications for Extension, the readers, and the newspaper itself.

For Extension

With the crisis in health today, so many more people need high-quality research-based nutrition information that is accurately reported (Hackman & Gaile, 1999). Hence Extension educators need to find delivery methods that can reach and show impact on large numbers of people in the county in which the educator works. Newspaper columns can be the delivery method that achieves those benefits for the Extension educator. Additionally, when compared to face-to-face trainings, a newspaper column can reduce educator time and cost, because a newspaper column lessens the need for repetitive, on the road, face-to-face workshops and thus, the cost of travel.

For the Reader

The readers of an Extension nutrition newspaper column can benefit in three ways. First, education plays an important role in whether or not a person is more likely to be overweight or obese (Mancino, Biing-Hwan Lin, & Ballenger, 2004). Readers who make the recommended changes discussed in the columns can increase their overall health and quality of life. Second, readers can benefit from this delivery method because of the sense of control they may feel over their learning environment (Garrison & Baynton, 1987) because reading a newspaper column is very individualized and private, giving the learner a high comfort level without group observation. Third, readers of a newspaper column can save travel time and the cost of attending a face-to-face nutrition workshop, yet receive weekly reinforcement by picking up the newspaper column at their convenience several times during the week.

For the Newspaper

The newspaper can benefit by gaining knowledge about a segment of its readers from a scientifically designed study conducted by Extension. If readers find value, they are likely to continue to purchase the newspaper, and, if they share their successes with friends or family, additional customers may accrue for the newspaper.

Limitations

As in all studies, this one has its limitations. One limitation is that the reader could have received the information and motivation to change from another nutrition source during the time period of the newspaper columns. To fully evaluate that perspective without the benefit of a control group, one must consider the array of nutrition sources available to the readers at the time. In the study reported here, perhaps individual readers may have heard about ways to include fruits, vegetables, and calcium-rich foods in their diets, but it is unlikely that a large number of the readers had done so because the nutritional information had not yet been circulated within the newspaper media in the geographical area in the same depth and breath as the news columns written by the Extension educator.

Even if a reader did pick up information on a national TV show, magazine, or newspaper column, it would only have related to some of the nutrition information and dietary recommendations given in this newspaper column, and furthermore, journals at the time were highly criticized for the quality of information they reported. "Research has found that journalists are very good at telling the 'what' of a food and health story, but can leave off 'to whom' and 'how much' is needed" (Center for Media and Public Affairs, IFIC, 2005). Additionally, the readers in this study identified the source of the information about nutrition as the newspaper column in the local newspaper.

A second limitation to this study is that readers could have over- or underreported the knowledge they gained and the changes they made in behavior either as a result of memory loss or presentation bias. To counter these biases, survey questions were carefully constructed following current research guidelines to reduce bias (Bradburn, Sudman, & Wansink, 2004).

A third limitation is a seemingly low response rate of 26%. There are two reasons why this was an acceptable percentage in this situation in evaluating a newspaper column.

First, although a percentage of 26% limits confidence one may have in generalizing, it does not diminish the significance of the impact on the 215 persons who did respond. Indeed significance is measured in different ways (Rossi et al., 1999). Knowing that Extension can reach and influence this size of a group demonstrates the efficiency of educating via a newspaper column compared to conducting face-to-face workshops. Indeed, getting people out to workshops for this program had proven ineffective.

Second, the Managing Editor was pleased to see that the newspaper was having an impact on that many readers. It was on the basis of this widespread impact that the Managing Editor decided to increase the number of columns per month in the newspaper.

Despite the limitations, the study has important strengths. It is a study that demonstrates the effectiveness of a newspaper column as an Extension delivery method despite the fact that the degree to which the public reads a newspaper has changed dramatically. No other research has evaluated the impact of the use of this delivery method in Extension in almost 25 years.

Conclusion

Newspaper columns may be one method for reaching large number of persons with research-based nutrition messages that influence nutrition knowledge and behaviors. Collaborating with newspaper editors and using the sampling framework of the study reported here to evaluate Extension newspaper columns can provide benefits to Extension educators, which include more column space; increased frequency of the column; a familiar, regular location within the newspaper; and exposure for other programs within Extension. Newspaper columns can be evaluated in this manner if Managing Editors see the impact of evaluation of a column as beneficial to the newspaper and share the mailing list of their subscribers.

References

Boldt, W. G. (1987). Targeting audiences and using creative media approach. Journal of Extension [On-line], 25(1). Available at: http://www.joe.org/joe/1987spring/rb2.html

Bradburn, N., Sudman, S., & Wansink, B. (2004). Asking questions. San Francisco, CA: Jossey-Bass.

Center for Media and Public Affairs (2005). Food for thought VI. December International Food Information Council.

Diem, K. G. (2003). Program development in a political world--It's all about impact! Journal of Extension [On-line], 41(1). Available at: http://www.joe.org/joe/2003february/a6.shtml

Dillman, D. A. (2000). Mail and Internet surveys. Second Edition. New York: Wiley and Sons, Inc.

Garrison, D. R., & Baynton, M. (1987). Beyond independence in distance education: The concept of control. American Journal of Distance Education, 1(3), 3-15.

Goldberg, J. et al. (2004). The obesity crisis: Don't blame it on the pyramid. Journal of the American Dietetic Association, 104(7) 1141-47.

Hackman & Gaile. (1999). Evaluation of newspaper reports of nutrition-related research. Journal of the American Dietetic Association, 99(12), 1564-66.

Heinrichs, A. J., Kiernan, N. E., & Ishler, V. A. (1996). Keys to developing an effective dairy satellite Extension program. Journal of Dairy Science, 79, 1981-87.

Kiernan, N. E., & Alter, T. (2004). Can a Web site be used as a vehicle for organizational learning about evaluation? Journal of Higher Education Outreach and Engagement, 10(1), 121-134.

Kiernan, N. E. (2001). First steps in evaluating a newsletter. Likely objectives. Program Evaluation. Tipsheet #43. Retrieved June 11, 2008 from: http://www.extension.psu.edu/evaluation/pdf/TS43.pdf

Kiernan, N. E., Turgeon A. J., & Hoffman L. D. (1995). Satellite seminars: An alternative method for Extension educators? Journal of Natural Resources and Life Science Education, 24(1), 36-44.

Mancino, L., Lin, & Ballenger. (2004). The role of economics in eating choices and weight outcomes. Agriculture Information Bulletin, 791.

McDowell, G. R., (2001). Land-grant universities and Extension into the 21st century: Renegotiating or abandoning a social contract. Ames, Iowa: Iowa State University Press.

Peterson, R., & Prillaman. J., (2000). An evaluation of an interdisciplinary team approach to delivery of childcare provider training. Journal of Extension [On-line], 38(2). Available at: http://www.joe.org/joe/2000april/rb4.html

Pounds, D. (1985). Putting Extension information where people will find it. Journal of Extension [On-line], 23(4). Available at: http://www.joe.org/joe/1985winter/a6.html

Rossi, P. H., Freeman, H. E., & Lypsey, M. W. (1999). Evaluation: A systematic approach. Sixth Edition. Thousand Oaks, California: Sage Publication, Inc.

Seevers, B., Graham, D., Gamon, J., & Conklin, N. (1997). Education through Cooperative Extension. Albany, New York: Delmar Publishers.

Smith, M.F., Barber, L., & Walker, K. (1984). Practical benefits of evaluation: An example. Journal of Extension [On-line], 22(1). Available at: http://www.joe.org/joe/1984january/a1.html

USDA Health and Human Services (1991). Report of the expert panel on blood cholesterol levels in children and adolescents. NIH Publication No. 91-2732.

Wirthlin Worldwide. (2002). Nutrition & you: Trends 2002. American Dietetic Association.