Does Adding an Extra Educational Intervention Add Value? The DairyBeef Train-the-Trainers Program

Introduction

Extension education program planners consider both the type and number of educational interventions needed to bring about change in their audience. One important question is "How much is enough?" if program budgets as well as program impacts must be optimized. This article describes the results of a controlled educational intervention trial on the impacts of follow-up materials on participant practices.

One method of broadening the reach of an educational program is by training those who can, in turn, educate the ultimate users of the information. These "train-the-trainers" programs are designed to amplify an educational program's reach to a wider audience by multiplying the expertise that provides producer education. Training Cooperative Extension farm advisors as content educators is part of an Extension specialist's mission, but they also can train private consultants and veterinarians who regularly work with producers to extend a program's message. This "multiplier effect" has been shown to be effective with extensive (12-module) and intensive (2-3 days per module) certificate programs for veterinarians in dairy production medicine (Moore, Sischo, & Hutchinson, 1996; Moore, 1999).

Intensive certificate programs bring about substantive changes in veterinary practitioners as well as in the clients they serve. These long-term programs meet many of the requirements for behavior change, as described in the PRECEDE model: predisposing, enabling, and reinforcing factors (Green & Kreuter, 1991). Davis and others, reporting on numerous randomized controlled trials of continuing medical education (CME), found interactive and mulitphasic programs more likely to improve physician performance in practice (Davis, Thomson, Oxman, & Haynes, 1992; Davis, Thompson, Oxman, & Haynes, 1995; Davis et al., 1999; Cauffman et al., 2002). Extension educators could benefit from knowing just how much reinforcement is required to elicit behavior change and, with respect to a train-the-trainers program, how much information is transferred to producers, the ultimate clientele.

Methods

The DairyBeef: Maximizing Quality & Profits educational program was designed to provide consistent food safety and dairy market cattle quality messages throughout the dairy industry (Moore et al., 2004). This producer-focused program was developed by faculty from seven western state land-grant universities with expertise in dairy production, veterinary medicine, and meat science. It was created as a Web-based educational program for producers as well as a "Train-the-Trainers" classroom program in CD-ROM format for dairy cooperative Extension advisors and other dairy consultants to deliver and provide the consistent dairy market cattle food safety and quality messages to producers.

Dairy veterinarians, dairy consultants, and dairy Cooperative Extension farm advisors in California were invited by mailed brochure to participate in one of four no-cost train-the-trainer workshops held in different locations. Veterinarians could receive continuing education credits towards their license renewal, and dairy consultants could receive continuing education credits towards their American Registry of Professional Animal Scientists membership. The list of potential participants came from a continuing education participant database maintained by the School of Veterinary Medicine, University of California, Davis.

Before the course began, participants were asked to complete an assessment of their learning stage with respect to four problem scenarios they should be able to handle after completion of the workshop. The learning stage theory was used as the basis of the scenario evaluation and the four responses: a. "I would refer this problem"; b. "I can handle this problem--no need to update"; c. "I have decided to update my skills & knowledge to address this problem"; and d. "I've recently updated my skills/knowledge required to address this problem" reflect the evaluation stage (a and b), the learning stage (c), and the gaining experience stage (d) (Slotnick et al., 2002).

After completing the pre-assessment, participants engaged in a 5-hour workshop covering the core DairyBeef educational segments for producers as well as modules on milk quality. The program agenda included the following.

The Objectives of the Program

Milk Quality, Antibiotic Residue Avoidance & Prudent Drug Use

• Milk Quality Standards in California, United States and the European Union
• Residue Avoidance Educational Materials
• Prudent Antibiotic Use

Hazards of On-farm Sales of Raw Milk

DairyBeef: Quality & Condemnations

• Why was the cow condemned?
• Preventing Carcass Defects

Reducing Pathogens in Market Cattle

Making Decisions to Cull Cows

Elements of Adult Learning and Relevance to Practice

Guidelines for Effective Oral Presentations

Creating Discussion Questions and Leading Discussions

Practice Developing and Giving a Presentation

(a "teach-back" in their own words, a summary of one of the learning segments using the educational materials provided)

All participants were given the trainer's version of course materials on CD-ROM, a notebook of the materials presented in the workshop, a flipchart, easel, and set of marking pens. At the end of the workshop, participants were asked to complete responses to the same practice problem scenarios, answer questions about program content, and commit to a plan for educating producers and using the materials. The last four digits of their social security number were requested on all evaluation forms in order to track and match individual responses without asking for any personal identifiers.

Once all the workshops were concluded, the names of all program attendees were randomly allocated to one of two groups using a random number generator in Excel©. The control group received no additional information. The intervention group received a follow-up letter in mid-November, 2004, that contained four printed handouts:

• Assessing the Risks, an on-farm tool to start a conversation with dairy producers on dairy beef quality;


• Educational Program Plan, a template that highlighted important steps in planning and delivering educational programs;


• Key Leadership Solutions for dealing with challenging behavior in meetings, a table highlighting some of the challenging behaviors seen in group meetings and some solutions to dealing with them; and


• Ensuring a Quality Educational Experience: A Summary, a six-page supplement to the trainer's notebook that covered additional tips on adult learning, effective presentations, and effective discussions.

All handouts except the last one were included on the original trainers' CD-ROM as electronic files.

Following Dillman's method for mail surveys (Dillman, 1978) a follow-up survey (Appendix C) was sent to all program attendees in February, 2005. A reminder postcard was mailed within 2 weeks, followed by an additional survey and an additional postcard. The same practice problem scenario questions were included in the follow-up, as well as questions on the use of the course materials and future plans to use the materials. Our hypotheses were that: (1) after the workshop, dairy veterinarians and consultants would change from their pre-program learning stage to a different learning stage; (2) participants would report confidence with providing education programs to producers; and (3) trainers who received additional materials would: (a) be more likely to use the materials with clients and (b) would report a greater level of comfort with giving presentations and leading discussions than participants not provided additional materials.

Outcomes included responses to the practice scenarios (reflecting a change in learning stage) before and after the course and at the follow-up survey, Likert scale responses to attitudinal questions, and binomial responses to use of the educational materials with producers. Ordinal and categorical data were analyzed by Chi-Square contingency table analysis, stratified analysis, and Fisher's Exact Test. Outcomes data were analyzed without knowledge of treatment group.

Results

Changes in Learning Stages

Not all participants (N=28) completed every survey or could be tracked. Only complete data (N=23) were evaluated for group differences. Participants started at different learning stages with regards to the different scenarios (Table 1) and changed with regards to learning stage from one time period to another. For the first scenario about market cattle residue violations, about one-half of the participants were in the evaluation stage (would refer or could handle the problem) and a little less than half were in the learning stage. Fifty percent moved to gaining experience immediately after the program (Table 1). Most participants (about 60%) were in the learning stage with regards to being able to increase market cattle value (Scenario 2) before the program and moved predominantly to gaining experience after the program. Four months later, there was still about 50% at the gaining experience stage. Most were in the learning stage with regards to dairy client education programs (Scenario 3) and moved into gaining experience. Similar results appeared for Scenario 4 (putting together an employee meeting).

Table 1.
Prevalence of Learning Stage for Program Participants of the DairyBeef Train-the Trainers Program from Before, After, and at 4-Month Follow-Up

1. Within your set of dairy clients, several have had a residue violation in cows sent to slaughter in the past four months. The packer would like to be assured that this is unlikely to happen again.

Survey/Stage	Evaluation	Learning	Gaining Experience
Before	54%	42%	2%
After	43%	0%	57%
4-Months Later	52%	10%	38%

2. More than one dairy producer has asked you to provide information on increasing the market value of their cull cows.

Survey/Stage	Evaluation	Learning	Gaining Experience
Before	35%	62%	3%
After	26%	2%	62%
4-Months Later	48%	4%	48%

3.Your employer/partners have decided that monthly dairy client education programs are essential. You are assigned next month's program.

Survey/Stage	Evaluation	Learning	Gaining Experience
Before	41%	48%	11%
After	26%	26%	48%
4-Months Later	58%	21%	26%

4. After reading a book on leadership, a dairy producer requests that you put together a meeting for his employees so that he can find out what they think might make the farm run more efficiently and improve milk quality.

Survey/Stage	Evaluation	Learning	Gaining Experience
Before	19%	73%	8%
After	24%	19%	57%
4-Months Later	48%	19%	33%

Because most of the surveys could be individually identified and tracked, individual changes in responses to the practice scenarios could be evaluated (N=23). For Scenario 1, 58% changed learning stage immediately after the program, and 48% changed after 4 months. After the program, 31% changed learning stage with regards to Scenario 2, and 38% changed stage after 4 months. Twenty-six percent changed learning stage for Scenario 3 after the program, and 63% changed after 4 months, significantly different than the change that occurred immediately after the course (P=0.02).

There was a tendency for more individual change between the end of the program and the 4-month follow-up among participants receiving the mailed materials for Scenario 1--a problem on residue violations (P=0.10, Fisher's Exact Test). However, there were no significant differences between the intervention groups between the end of the program and the 4-month follow-up in frequency of individual learning stage change for any of the other three scenarios ( 2: P=0.19; 3: P=0.49; 4: P=0.63, Fisher's Exact Test).

Confidence in Providing Education to Producers

As a result of the training, 100% of participants agreed (67%) or strongly agreed (33%) that they could provide a consistent message to dairy producers about milk and meat safety at the conclusion of the course (Table 2). To identify what they learned, participants were also asked to list three issues packers have regarding incoming market cattle and five farm-specific risks for food safety or quality. Ninety-six and 81%, respectively, answered these questions correctly and completely.

All participants were provided information on adult learning principles and practices in the workshop. At the end of the course, 100% agreed that they learned new ideas for conducting education programs for clients and new ideas for leading discussions (Table 2). All but one reported improving their ability to plan client education programs. Over half (52%) reported a date when they planned to conduct an educational session with producers using the materials.

Table 2.
Responses to Post-Program and 4-Month Follow-Up Surveys After a Train-the-Trainers Program on Dairy Market Cattle Food Safety and Quality

Immediately Post-Program	Strongly Agree	Agree	Disagree
Provide a consistent food safety message	33%	67%	0%
Learned new ideas for conducting educational programs	37%	63%	0%
Learned new ideas for conducting discussions	19%	81%	0%
Learned new ideas on adult learning	26%	70%	4%
Improved ability to plan educational programs	22%	70%	8%
Planned to do an educational program in the future	Yes = 52%		No=48%
4-Month Follow-Up	Strongly Agree	Agree	Disagree
Provide a consistent food safety message	39%	61%	0%
More confident to provide client education	26%	70%	4%
More confident to effectively facilitate discussions	26%	65%	9%
Used new ideas of adult learning	17%	57%	26%
Improved ability to plan educational programs	22%	70%	8%
Can better plan client education programs in the future	22%	69%	9%

Because learning stage might affect whether an individual planned to conduct a client education program or not, we evaluated the association between learning stage and participant plans. For the scenario regarding client education meetings (No. 3), learning stage before the workshop was not associated with a decision to conduct a client education program using the DairyBeef materials (P=0.77, Fishers Exact Test). However, there was a tendency for more individuals in the "learning" stage at the end of the workshop on planning a client education program compared to those in the evaluation or gaining experience stages (P=0.12).

4-month Follow-up and Differences Between Intervention Groups

To identify a baseline difference in educational experience among participants, they were asked if they had conducted dairy producer education programs within the year prior to attending the trainer's course. Sixty-one percent had done producer education before the course, but there was no difference in responses between the intervention groups (P=0.81).

At the 4-month follow-up, all participants continued to report confidence in their ability to provide a consistent food safety message (Table 2). There was no difference in reported confidence to provide the consistent messages between the two intervention groups (P=0.30). All but one person agreed that they were more confident in providing client education and effectively facilitating discussions. Seventy-two percent reported using new ideas about adult learning, but there was no difference between the intervention groups (P=0.83). All but two participants felt confident in planning client education programs.

Specific questions were asked about the course materials provided. Only 30% reported viewing the CD-ROM in the 4 months after the course, and there was no difference between the intervention groups (P=0.80). The only difference between intervention groups was a tendency for more control participants (3/10) to report visiting the Web site compared to those receiving addition materials (0/12) (P=0.06, Fishers Exact Test). There were no differences between groups in the percentage sharing the CD-ROM with colleagues or producers or in using the materials for client education. However, individuals who reported conducting training sessions with producers or farm labor in the previous year were more likely to report using the course materials for client education (P=0.02).

Discussion

The DairyBeef Train-the-Trainers Program is the first dairy quality assurance "train-the-trainers" program reported and evaluated in the literature. This half-day workshop did result in learning stage change of individual participants. With regards to the four practice problems provided in the surveys, most participants started the program in the evaluation or learning stage and moved to learning or gaining experience (Slotnick et al., 2002; Moore, 2003). However, the study indicates that the simple provision of information, even in the way of a reminder and materials, is insufficient to influence use of materials by individuals asked to provide the information to others. In addition, this short workshop, regardless of the interactivity, provision of materials, and follow-up, may not be have been sufficient to overcome all the factors necessary to result in behavior change in a complex topic such as food safety producer education, particularly when participants may not be rewarded for these services.

Previous work indicated that although dairy veterinarians saw a potential market in offering food safety services, they were unsure of their role and how they would be paid (Moore, Sischo, & Wilson, 2000). The best predictor of use of workshop materials for client education was the participant's report of conducting any kind of client education in the previous year. Thus, previous experience in education was the most important driver of new information delivery.

The reason why many participants went back to the evaluation stage at the 4-month follow-up can only be speculated upon. However, using the staged theory of learning, these individuals, having not used the materials to any great degree, may have gone back to evaluating the scenarios as new problems for them. In addition, when returning home, there were likely not enough enabling or reinforcing factors to make the change (conducting client education in dairy beef quality).

Using the PRECEED Model of behavior change (Green et al., 1991), the predisposing factors in adult education include moving the individual to awareness that the topic is important for them either to solve a problem they have or address some other individual need (Slotnick, 1996). Many educational programs can move individuals to awareness, but they may or may not move them to make the next step, which is the motivation to engage in the kinds of activities that will enable them to make behavior change, and may or may not reinforce the change in their practice life.

The trainers' program provided some predisposing and enabling factors and attempted to test reinforcement. The program increased awareness about the issues of dairy beef food safety and quality and the participant's role in providing education to their clients. The participants were enabled to spread the information to producers by being provided all the necessary educational materials with which to train, the tools and skills with which to educate, and gained practice in educational delivery.

One major difference between the success of a certificate program (Moore et al., 1996) and this 1-day workshop is likely due to its brevity or lack of continuity, thereby not providing on-going reinforcement. Our conclusion is that we may not have satisfied all the requirements to get participants engaged in having a market cattle quality conversation with their clients nor how they could be rewarded for providing client education in food safety. Those that had done educational programs before the workshop may have already developed a mechanism of remuneration, or were predisposed to conducting new educational programs.

Suggestions for Future Train-the-Trainers' Programs

Train-the-trainers' programs in food safety, quality assurance, biosecurity, agroterrorism, or any number of other topics for which there are not concrete, immediate "rewards" for either the trainer or producer may not result in the desired change in the industry unless predisposing, reinforcing, and enabling factors are in place to make it successful. Many successful trainers' programs provide education and training for certification or recertification, such as HACCP trainers' program for the retail/food service industry (Martin, Knabel, & Mendenhall, 1999). These kinds of programs deliver training on the organization's needs for participants to fulfill job requirements.

To make a difference with an educational program, there must be awareness that the issue to be addressed is an issue important to participants. In a study using the learning stage theory for needs assessment, large numbers of physicians indicated they could handle cases provided in the scenarios, but expert reviewers disagreed, indicating unperceived learning needs for the doctors (Slotnick et al., 2002). Based on this, marketing materials for educational programs must highlight not just what is going to be taught but why the skills and knowledge are necessary for their professional practice. Thus, the true value to the individual must be well articulated.

Single educational program events do not sufficiently enable participants to encourage practice with a new skill or spread new knowledge once they return home. Although interactivity and practice, such as the "teach-back" technique for trainers, can help enable their change, some individuals will require more practice than others. Continuous or multi-stage, interactive programs may help enable individuals to practice new skills in a safe environment. Single, short-term programs can be effective, in the short term, but the maintenance of the change may fall off (Backhaus et al., 2002).

Another method to encourage change is the use of a "commitment to change" contract (Mazmanian, Daffron, Johnson, Davis, & Kantrowitz, 1998; Mazmanian & Mazmanian, 1999; Dolcourt, 2000). However, even with the use of a commitment to change contract, in one study, only 35% reported implementing the change they had committed to (Halbur & Vandagriff, 2002). A true commitment might require a signature, which has been shown to enhance behavior change rates among physicians (Mazmanian, Johnson, Zhang, Boothby, & Yeatts, 2001).

Industry-wide "carrots" or "sticks" (such as monetary incentives or deductions from packers) are needed to motivate change among producers. To best evaluate progress with quality assurance programs, there should be some distinct endpoints, such as the frequency of residue violations (Gibbons-Burgener, Kaneene, Lloyd, & Erskine, 1999), real costs and benefits associated with the endpoints, and ability to monitor events on the farm that predispose to lower quality.

In addition, if veterinarians and dairy consultants are to engage in the process of providing information or education to producers, they need suggestions on how to approach their clients about the issue and how to motivate clients to pay for their educational services. The latter may have been one reason the workshops did not draw a large number of participants. Drawing attention to a topic not in the headlines or perhaps not directly relevant requires creative marketing.

Conclusions

A short-term "train-the-trainers" program may not be enough to effect behavior change among participants, even if reinforced with additional, mailed materials. Extension educators need to consider careful selection of reinforcement techniques to encourage behavior change and refer to predisposing, enabling and reinforcing factors in their programs.

Acknowledgements

The authors wish to thank the program participants. This research was funded by a grant from the California Dairy Research Foundation through the California Dairy Quality Assurance Program.

References

Backhaus, J., Junghanns, K., Mueller-Popkes, K., Broocks, A., Riemann, D., Hajak, G. et al. (2002). Short-term training increases diagnostic and treatment rate for insomnia in general practice. Eur Archives Psychiatry Clinical Neuroscience, 252, 99-104.

Cauffman, J. G., Forsyth, R. A., Clark, V. A., Foster, J. P., Martin, K. J., Lapsys, F. X. et al. (2002). Randomized controlled trials of continuing medical education: what makes them most effective? Journal of Continuing Education in the Health Professions, 22, 214-221.

Davis, D. A., Thompson, M. A., Oxman, A. D., & Haynes, B. (1995). Changing physician performance: a systematic review of the effect of continuing medical education strategies. Journal of the American Medical Association, 274, 700-705.

Davis, D. A., Thomson O'Brien, M. A., Freemantle, N., Wolf, F. M., Mazmanian, P. E., & Taylor-Vaisey, A. (1999). Impact of formal continuing medical education. Journal of the American Medical Association, 282, 867-874.

Davis, D. A., Thomson, M. A., Oxman, A. D., & Haynes, R. B. (1992). Evidence for the effectiveness of CME. Journal of the American Medical Association, 268, 1111-1117.

Dillman, D. A. (1978). Mail and telephone surveys: The complete design method. New York: John Wiley & Sons.

Dolcourt, J. L. (2000). Commitment to change: a strategy for promoting educational effectiveness. Journal of Continuing Education in the Health Professions, 20, 156-163.

Gibbons-Burgener, S. N., Kaneene, J. B., Lloyd, J. W., & Erskine, R. J. (1999). Evaluation of certification in the Milk and Dairy Beef Quality Assurance Program and associated factors on the risk of having violative antibiotic residues in milk from dairy farms in Michigan. American Journal of Veterinary Research, 60, 1312-1316.

Green, L. W. & Kreuter, M. W. (1991). Health Promotion Planning: an educational and environmental approach. (2nd ed.) Mountain View, CA: Mayfield Publishing Co.

Halbur, B. & Vandagriff, M. (2002). Commitment to change contract. Almanac: Alliance for CME, 24, 7-8.

Martin, K. E., Knabel, S., & Mendenhall, V. (1999). A model train-the-trainer program for HACCP-based food safety training in the retail/food service industry: an evaluation. Journal of Extension, [On-line],37(2).Available at: http://www.joe.org/joe/1999june/a1.html

Mazmanian, P. E., Daffron, S. R., Johnson, R. E., Davis, D. A., & Kantrowitz, M. P. (1998). Information about barriers to planned change: a randomized controlled trial involving continuing medical education lectures and commitment to change. Academic Medicine, 73, 882-886.

Mazmanian, P. E., Johnson, R. E., Zhang, A., Boothby, J., & Yeatts, E. J. (2001). Effects of a signature on rates of change. Academic Medicine, 76, 642-646.

Mazmanian, P. E. & Mazmanian, P. M. (1999). Commitment to change: theoretical foundations, methods, and outcomes. Journal of Continuing Education in the Health Professions, 19, 200-207.

Moore, D. A. (1999). Changes in skills, changes in abilities: an evaluation of a continuing education program in dairy production medicine. Journal of Veterinary Medical Education, 25, 17-20.

Moore, D. A. (2003). Evaluating the stages of veterinary practitioner learning for continuing education needs assessment and program evaluation. Journal of Veterinary Medical Education, 30, 50-56.

Moore, D. A., Kirk, J., Klingborg, D. J., Garry, F. B., Wailes, W., Dalton, J. et al. (2004). DairyBeef: maximizing Quality and profits - a consistent food safety message. Journal of Dairy Science, 87, 183-190.

Moore, D. A., Sischo, W. M., & Hutchinson, L. J. (1996). Effect of participation by veterinarians in a dairy production medicine continuing education course on management practices and performance of client herds. Journal of the American Veterinary Medical Association, 209, 1086-1089.

Moore, D. A., Sischo, W. M., & Wilson, D. (2000). Continuing education needs assessment for on-farm food safety services. Journal of the American Veterinary Medical Association, 217, 479-484.

Slotnick, H. B. (1996). How doctors learn: the role of clinical problems across the medical school-to-practice continuum. Academic Medicine, 71, 28-34.

Slotnick, H. B., Passin, S. M., Mejicano, G., Lowenstein, D. F., Jensen, F. H., Pizzoferrato, S. et al. (2002). Estimating prevalences of physicians' learning needs. Almanac: Alliance for CME, 24, 4-6.