Pilot Study: A Guide to Equine Welfare Assessment

Introduction

Much attention is being generated in the area of animal welfare science. There is growing societal concern about animal welfare and growing interest in animal welfare assessment. This is illustrated through recent pieces of U.S. Legislation, The Horse Transportation Safety Act of 2009 and The Prevention of Equine Cruelty Act of 2009. Animal agriculture producers, educators, researchers, politicians, and the general public are all becoming increasingly aware of the importance of adequate and responsible animal care (Heleski, Mertig, & Zanella, 2004).

Educating youth regarding optimal animal management practices and associated societal concerns is crucial because they will comprise these professional communities in the near future. Youths need to expand their knowledge beyond the doors of their local community to become responsible and contributing global citizens (Tapscott, 2009). A Guide to Equine Welfare Assessment aims to help Michigan's youth learn more about equine welfare and management, which is an area that currently lacks available curriculum. In June of 2000, 1,760 Needs Assessment Questionnaires were sent to Michigan 4-H leaders and county Extension staff with 4-H Horse and Pony project responsibilities (Sherlund, Waite, & Doumit, 2001). A need for additional youth curriculum was cited by participants of the survey. A telephone survey of 130 MSUE 4-H staff conducted by Michigan 4-H Youth Development in 2007 verified that this continues to be a need (unpublished survey data). A Guide to Equine Welfare Assessment will begin to fill this void.

Background

A guide for 4-H club leaders titled A Guide to Equine Welfare Assessment was developed by Andersen, Wickens, Siegford, Heleski, and Waite (2008) with four accompanying equine welfare assessment exercises. This guide, with its activities and exercises, is designed for 4-H adult leaders to use with their youth members. The welfare of agriculture animals has become an increasingly prevalent topic in public discourse, as consumer awareness and general concern for animal welfare increases. Welfare science is an area gaining attention, and it is paramount that the future leaders of the nation's horse industry be well-informed. Analytical thinking and clear communication, which can potentially be gained from participation in welfare judging activities, may prevent problems related to managing farms. Critical thinking and communication are practical, useful life skills. These skills were essential objectives emphasized while developing A Guide to Equine Welfare Assessment. Through the completion of welfare assessment exercises, youth are prepared to act as liaisons between university faculty producing scientific information and area farm managers who are implementing the information (Andersen, Wickens, Siegford, Heleski, & Waite, 2006).

This leader guide has yet to be incorporated into an active curriculum program available to horse leaders throughout the state and nation. However, based on response surveys completed at the 2004, 2008, and 2009, Michigan 4-H Teen and Adult Horse Leaders' Conferences, interest in equine welfare evaluation is high (unpublished evaluation data). The next step is to test the efficacy and efficiency of A Guide to Equine Welfare Assessment and its accompanying interactive disc; hence a pilot study was developed.

Hypothesis

A Guide to Equine Welfare Assessment and its accompanying interactive disc will provide the horse club leader with the tools and information to confidently inform youth on the basics of equine welfare assessment. Success will be demonstrated by increased post-test scores versus pre-test scores by participating youth and leaders, as well as overall positive feedback on evaluations completed by leaders.

Materials and Methods

At the 2008 and 2009 Michigan 4-H Horse Leaders' Conferences, 31 leaders indicated interest in participating in the curriculum-testing effort after hearing a description of the pilot study for A Guide to Equine Welfare Assessment. A second contact was made to each of these leaders, and from that group, 15 4-H groups volunteered to participate in the pilot study. As illustrated in Figure 1, the 15 groups represented 12 counties throughout Michigan. In total, these 15 groups were comprised of 45 adult leaders and 192 youth participants.

Figure 1.
Participating Michigan Counties

Ideally, regular scheduled club meetings would provide a structured time to work with this material. This structured environment was helpful in determining the amount of time needed to complete A Guide to Equine Welfare Assessment. If clubs focused on one section at each club meeting, it would take eight club meetings to finish the entire curriculum.

Content in A Guide to Equine Welfare Assessment is separated into eight sections.

1. Animal Welfare: an introduction
2. Behavioral Indicators
3. Physiological Indicators
4. Production Measures
5. Immunological Measures
6. Human and Animals Interacting
7. Animal Ethics
8. Evaluating Welfare

Each section has a similar format.

1. Objectives
2. Life Skills (gained through using materials in section)
3. Materials Needed (resources leaders need for section activities)
4. Student Activities
5. Resource Information (reference material for leaders)
6. Answers to Activities
7. Let's Talk About It (structured discussion to wrap-up section concepts)

In an effort to measure transfer of knowledge, before beginning work with the leader guide, youth and leaders evaluated four sets of welfare assessment exercises (scenarios) provided on the interactive disc. The youth and leaders rated the welfare of the horses in each scenario, using a Likert Scale of 1-5 (1: Poor - 5: Excellent). Youth and leaders were also encouraged to supplement their ratings with reasons supporting each decision. Upon completion of the entire leader guide, youth and leaders again evaluated the welfare in each of the scenarios. For the second welfare evaluation attempt it was expected that participants would provide more detailed reasons that would be rooted more in the concrete facts provided throughout the leader guide.

Additionally, youth and leaders were instructed to complete a series of pre- and post-tests for each of the eight sections described above. The pre- and post-tests consisted of 10 questions each: three true/false and seven multiple choice. Each of the eight sections was preceded by a pre-test to assess background knowledge on section-specific subject matter. Each of the eight sections was followed by the same quiz. It was expected that post-test scores would have a higher percentage of correct answers when compared with the same section's pre-test scores.

In order to further determine success of the leader guide section, evaluations were distributed to leaders. These evaluations were opportunities to provide feedback on the specific section. This feedback helped determine if the necessary resources, including time, were available to deliver the curriculum. The evaluations also determined perceived youth interest. This feedback helped establish usability of the leader guide, as determined by the leaders.

Each leader was provided with the following resources.

1. One copy of the interactive CD which included;
   1. Four welfare exercise/scenario sets
   2. PDF version of A Guide to Equine Welfare Assessment
2. One spiral bound, color copy of A Guide to Equine Welfare Assessment
3. Evaluation materials
   1. Participation consent forms for leaders, parents, and youth
   2. To be completed by leader
      1. Leader demographic sheet
      2. Eight section evaluations
   3. To be completed by youth and leaders
      1. Pre- and Post-Welfare exercise/scenario assessments
      2. Eight section specific pre-tests
      3. Eight section specific post-tests

To assist leaders, appropriate copies of materials were made ahead of time, according to size of club.

After initial distribution of materials, contact was regularly made with leaders. A variety of methods were used (email, phone, and mail) to check the progress of work with the leader guide. This contact served as an opportunity to resolve problems that arose with the material and offer support and encouragement to leaders. One week following shipment of materials, contact was made to be sure that leaders received the shipment. Nearing the 6-week mark of the study, an email was sent to participating leaders to check progress and resolve any remaining questions. Throughout this process, leaders were invited to call with questions, and several leaders did take advantage of that opportunity.

At the 3-month (or half-way) mark, leaders were sent a letter that included pre-paid postage as well as instructions on how to send completed materials back to researchers. In this mailing, it was mentioned that additional postage would be sent for the remaining half of the materials. Approximately 3 weeks after pre-paid postage was sent, no completed materials had been returned. At that time, researchers made follow-up contact with each leader through both phone and email, reminding leaders to send back any completed materials. Through this contact it was discovered that many clubs had been unable to begin the pilot study, but that interest remained. After several more weeks (5 months into pilot study), only four of the 15 involved clubs returned any completed material. Due to the low response rate, a survey was developed using www.surveymonkey.com to further determine reasoning for the lack of responses.

Results

Of the 15 participating groups, completed materials were returned from 4 (26.67% response rate). These four groups consisted of 18 leaders and 60 youth. One group completed materials from two sections, one completed materials from three sections, and two groups were able to complete the entire pilot study. The results were analyzed using an analysis of variance approach (ANOVA). The independent variables were group (1-4), Section (1-8), and Training (pre or post). The dependent variables were Scores (0-100). First, the pre- and post-tests scores from each section were analyzed. These data can be seen in Table 1.

Analysis of the mean pre-test scores versus mean post-test scores, allowed researchers to determine whether there was a significant change due to the training (i.e. instruction using A Guide to Equine Welfare Assessment). A significant change was seen in post-test scores, when compared to pre-test scores, for sections 1, 2, 3, and 8 (Table 1). There was not a significant change between pre- and post-test scores for sections 4, 5, 6, and 7 (Table 1). It is important to note that for sections 4-8 there was only data from two clubs.

Table 1.
Pre- and Post-test Mean Score Differences

Section		Mean Pre-test Score*	Mean Post-test Score*	Mean Difference	p value	Significant Change
1	Animal Welfare	55.7139	83.5607	27.8468	<.0001	Yes
2	Behavioral Indicators	71.7877	95.6054	23.8177	<.0001	Yes
3	Physiological Indicators	68.3073	86.224	17.9167	0.0009	Yes
4	Production Measures	92.9366	92.1033	-0.8333	0.8029	No
5	Immunological Measures	83.77	91.27	7.5	0.1783	No
6	Human & Animals Interacting	72.9366	82.1033	9.1667	0.1352	No
7	Animal Ethics	80.4366	91.272	10.8354	0.1026	No
8	Evaluating Welfare	72.9366	87.9366	15	0.0136	Yes
*Scores are represented as points correct out of a possible total of 100 points.

Eight leaders completed the online survey that was conducted to determine reasoning for response rate (Table 2).

Table 2.
Online Survey Responses

How much of the material did you complete with your group?
	%	Count
None	0.0%	0
1-3 Sections	75.0%	6
4-7 Sections	0.0%	0
All	25.0%	2
Of the material that you and your group completed, how useful did you find it to be in terms of improving knowledge of assessing equine welfare?
	%	Count
Very Useful	75.0%	6
Somewhat Useful	25.0%	2
Not Useful at all	0.0%	0
N/A	0.0%	0
Do you feel that leaders' personal commitments made it difficult to work with the leader guide?
	%	Count
Strongly Disagree	0.0%	0
Disagree	37.5%	3
Neutral/Not sure	0.0%	0
Agree	37.5%	3
Strongly Agree	25.0%	2
Do you feel that youth participants' personal commitments made it difficult to work with the leader guide?
	%	Count
Strongly Disagree	0.0%	0
Disagree	0.0%	0
Neutral/Not sure	25.0%	2
Agree	62.5%	5
Strongly Agree	12.5%	1
How would you (as a leader) prefer to receive this material?*
	%	Count
Hard Copy	50.0%	4
A disk only	37.5%	3
All online	12.5%	1
Other	37.5%	3
* Survey respondents were able to select more than one answer. 3 respondents also selected other and described a combination of online and hard copy materials as their preferred method.
What would you say is the largest factor affecting your inability to complete the project?
		Count
None**		1
Scheduling/Time**		5
Too much on meeting agendas**		3
Technological Difficulties (with personal equipment)* *		1
Respondent skipped question		1
**As these answers were in the form of open-ended responses, categories have been abbreviated to encompass survey responses. Some leaders cited more than one of these factors as the largest contributing factor.
Do you feel that you would use this material with your group in the future?
	%	Count
Yes	87.5%	7
No	0.0%	0
Neutral/Not Sure	0.0%	0
Respondent skipped question	12.5%	1

Discussion

Referring to Table 1, significant increases in post-test scores versus pre-test scores were seen in four of the eight sections. For several of the sections with no significant increase in scores, there was data from only two clubs, and, additionally, the pre-test scores were high (>80% correct), leaving little room for improvement. This data suggests that information covered in these sections is an area of strong knowledge for participants of the pilot study. The data may also suggest that the information in these sections was too basic for this audience and that possible revisions are needed.

To evaluate factors affecting usability and subsequent incorporation into the 4-H curriculum, a researcher-designed online survey was developed. Through analyzing the results of this survey, investigators noticed an interest in a more interactive format of the material. At least 50% of survey respondents described either an all online or combination of hard-copy and online as their preferred means of receiving the materials (Table 2). One such format could include a hybrid of the standard hard-copy with online resources. Although technology was included in the form of an interactive CD, no online curriculum was utilized in the study reported here.

Black (2002) compared various methods of course delivery and concluded "that students enrolled in hybrid courses perceived the combination of classroom and Internet course delivery modes to be superior to either internet- or classroom-alone modes." Of survey respondents who wished to have a hybrid of the materials, several mentioned that having the hard-copy would provide a "back-up" in case of difficulties with Internet access or other technological issues. This is an important consideration because many in higher education see a paradigm shift in the way in which today's youth are learning. While many students are increasingly savvy about using technology, many instructors (i.e., volunteer leaders in 4-H) are more comfortable having a more traditional hard-copy version, particularly as a fail-safe option. Tapscott (2009) quotes L. Prozena, president of the University of Akron, when he notes that although

the Industrial Age model of education will be hard to change…'the time has come for some far reaching changes to our model of pedagogy, how we operate, and our relationship to the rest of the world. We need to listen to these Net Generation [individuals born January 1977-December 1997] students to see the way forward.'

Another benefit of a hybrid mode of delivery is that students who would be reticent to contribute in a face-to-face setting are just as—or more—likely to participate in an online discourse as are the most gregarious students (Black, 2002). Kids who have grown up digital expect to talk back, to have a conversation. They want a choice in their education, in terms of what they learn, when they learn it, where, and how (Tapscott, 2009). There are challenges present when the Internet is introduced into the learning environment. A careful analysis of the course material must include determining what content can be used online (Black, 2002).

A possible way to increase participation would be including an incentive-based approach to encourage completion of the leader guide. Offering small prizes or certificates at specific intervals throughout the exercises and upon final completion may give youth more reasons to participate. It has been shown that incentives have affected the response rate of mail surveys. Church (1993) notes that the, "average increase in response rates over control [non-incentive] conditions [ranged from] 19.1 percent to 7.9 percent." Introducing incentives could be further investigated to determine if response rates would be affected.

Increasing one's knowledge base may not be incentive enough for youth to remain involved in educational programs such as this. However, life skills development has been proven to be a factor affecting teen involvement.

Participation choices of 4-H youth are influenced by their desire to improve life skills such as: public speaking, organizational skills, and leadership skills. Extension educators and 4-H leaders should continue to design programs and activities that promote personal growth and life skill building" (Gill, Ewing, & Bruce, 2010).

To further enhance life skills development "youth need to feel like an integral part of facilitating the organization" (Gill et al. 2010). Incorporating a teen-leader portion of the leader guide could provide this opportunity. Having a teen leader work with the club's adult leader to help instruct a particular section of the guide would be one method of integrating teen leadership into the program.

As previously mentioned, a need for more youth curriculum was cited by Michigan 4-H Horse Leaders in a survey performed in June of 2000 (Sherlund et al., 2001). Much of the information provided in the leader guide would be of great use to a horse bowl or hippology team, and perhaps the marketing and promotion of the leader guide could be additionally geared towards that audience. Including welfare sections in these educational contests could further provide opportunities for youth to use this knowledge of welfare assessment. At volunteer-training events such as the Michigan 4-H Horse Leaders' Conferences, information could be shared with leaders and horse bowl and hippology coaches to inform them of the opportunities within A Guide to Equine Welfare Assessment. 

It is recognized that supplemental information, such as Body Condition Scoring posters for the Production Measures section, could also enhance the learning experience. Furthermore, a possible revision to the guide could involve improving the list of objectives at the start of each section. These objectives could be improved through the use of Bloom's taxonomy. Originally published in 1956, Bloom's Taxonomy continues to be one of the most often cited references in education. The six levels of learning identified by Bloom from lowest to highest are: knowledge, comprehension, application, analysis, synthesis, and evaluation. These different levels have become a useful guide for teachers in planning classroom lesson plans and classroom objectives. "In almost all circumstances when an instructor desires to move a group of students through a learning process utilizing an organized framework, Bloom's Taxonomy can prove helpful" (Forehand, 2005). Using this method to classify objectives will give the leader clearer expectations for each section outcome. For example, the current objectives for Section I are:

1. Learn basic equine welfare concepts.
2. Identify different aspects of equine welfare.
3. Explore the difference between animal welfare and animal rights.

After revision, these objectives would read:

1. Compare and contrast animal welfare versus animal rights.
2. Define the Five Basic Freedoms.
3. Evaluate how animal welfare affects how you care for your animals.

These revisions will provide leaders with clearer direction as they present each section to their clubs.

Conclusions

Regarding the hypothesis on whether the materials included in A Guide to Equine Welfare Assessment provided the horse club leader with the necessary tools and information to confidently inform youth on the basics of equine welfare assessment, one must look to a variety of areas for the answer. Success was partially defined as increased post-test scores versus pre-test scores by participating youth and leaders. Judging by the information available in Table 1, it does appear that post-test scores were improved in several of the sections. Further examination of differences across sections may be useful in determining effectiveness of each section. It is also worth noting the difference in pre-test scores across sections, suggesting there may be significant differences between youths' base knowledge on subject areas.

Success was also defined as overall positive feedback on evaluations completed by leaders. Referring to Table 2, 100% of online-survey respondents found that, in terms of improving knowledge of assessing equine welfare, A Guide to Equine Welfare Assessment was either very useful or somewhat useful. Additionally, on the same survey, 100% of respondents said that they do plan to use the materials in the future, suggesting that using the leader guide was a positive experience.

Possibly the most revealing finding from the completion of a pilot study on A Guide to Equine Welfare Assessment is the overall low response rate. Results from the online survey are useful in determining why leaders were unable to complete the project. Overall time constraints, for both youth and leaders, were among the most commonly cited deterrents to completion. Survey responses also suggested that following a hybrid (of hard-copy and online instruction) may increase usability and effectiveness of A Guide to Equine Welfare Assessment.

References

Andersen, K., Wickens, C., Siegford, J., Heleski, C., & Waite, K. (2008). A guide to equine welfare assessment. East Lansing, MI: Michigan State University Department of Animal Science.

Andersen, K., Waite, K., & Heleski, C. (2006). 4-H animal welfare assessment: Does it work? Journal of Extension [On-line], 44(6) Article 6IAW7. Available at: http://www.joe.org/joe/2006december/iw7.php

Black, G. (Spring 2002). A comparison of traditional, online and hybrid methods of course delivery. Journal of Business Administration Online, 1(1). Retrieved from: http://www.atu.edu/business/jbao/Spring2002/black.pdf

Church, A. (1993). Estimating the effect of incentives on mail survey response rates: a meta-analysis. Public Opinion Quarterly, 57:62-79.

Forehand, M. (2005). Bloom's taxonomy: Original and revised. In: M. Orey (Ed.). Emerging perspectives on learning, teaching, and technology. Retrieved from: http://projects.coe.uga.edu/epltt/

Gill, B. E., Ewing, J. C., & Bruce, J.B . (2010). Factors affecting teen involvement in Pennsylvania 4-H programming. Journal of Extension [On-line], 48(2) Article 2FEA7. Available at: http://www.joe.org/joe/2010april/a7.php

Heleski, C. R., Mertig, A. G., & Zanella, A. J. (2004) Assessing attitudes toward farm animal welfare: A national survey of animal science faculty members. Journal of Animal Science, 82:2806-2814.

Sherlund, S. M., Waite, K. L., & Doumit, S. A. (May 2001). The use of technology among Michigan 4-H horse leaders. Proceedings from 17th ENPS Symposium.

Tapscott, D. (2009). Grown up digital: How the net generation is changing your world. New York, NY: McGraw-Hill.