How Do We Know if Our Contests Are "Fair"?

Introduction

A Balanced Approach to Competition

A major tenet of Extension work is that we should listen and respond to client input. If clients voice a strong concern, the first response of Extension workers is often to jump in and make changes. But is there a problem just because someone says there is? In 4-H work, leaders and parents are particularly outspoken about contests and whether judging is "fair." How can we determine if their concerns are valid? Are they just complaining because the "right kid" (usually their kid) didn't win? Or is there a real bias?

"Fairness" is not only a common concern from participants and volunteers but an important consideration for program planners and evaluators as well (Fetsch & Yang, 2002; Radhakrishna, Everhart, & Sinasky, 2006). An unintended bias in contest results is likely to skew evaluation data and prevent the attainment of desired outcomes. A perceived problem with fairness can also invalidate certain survey questions or lead to a low measurement of client satisfaction with a program. Therefore, it is a good idea for evaluators to consider the mechanics of the contest itself and be prepared to answer client concerns.

Contests have long been commonplace in youth organizations, including 4-H (Weber & McCullers, 1986), as a method to encourage skill acquisition. Research suggests that competitive events can also build character if parents and coaches encourage an attitude of winning as experience, rather than winning as consequence (Chandler & Goldberg, 1990). In an evaluation of 4-H Animal Science members at county fairs in Oregon, youth reported that competition contributed to the development of "sportsmanship" and "responsibility" (Arnold, 2007). In another study of 4-H participants, Radhakrishna et al., (2006) report that contests help youth learn new things, develop life skills, set goals, and strive for excellence. However, those youth also had some concerns that competition could lead to excessive parental involvement, unethical practices, and other unhealthy behaviors.

To keep the positive outcomes of competition in the forefront, organizations must carefully review their contests in order to avoid unintended negative effects and to keep them in line with current research findings (Fetsch & Yang, 2002). In 4-H, the most common tradeoff in contest design is between a Competitive (competing against one's peers) and a Cooperative (competing against standards of excellence) emphasis. Although many 4-H members report a positive response to both Competitive and Cooperative approaches (Fetsch & Yang, 2002), an individual contest will ultimately have to be one or the other.

One unifying aspect of studies of youth competition and fairness is that they all rely on perceptions: of youth participants, of youth non-participants, of parents, and of youth workers. This leads to an entirely subjective definition of "fairness." While a subjective definition is enlightening and many times useful, there are times when a more objective criterion is desired. Such objective methods can be found by turning to basic tenets of probability and independence of events. This researcher developed an evaluation program for the 4-H Master Showmanship Contest that incorporates both subjective data and objective, quantitative data to form a holistic picture of the contest's fairness.

The Master Showmanship Contest

Many livestock shows offer a Master Showmanship Contest in which youth show several animal species and receive a score for their showmanship abilities with each one. The participant with the highest total score is the Champion Showman (making this an example of a contest that is Competitive, although the earlier tiers of the competition are usually Cooperative).

In Clackamas County, Oregon, livestock leaders voiced growing concerns about the Livestock Master Showmanship Contest, particularly about whether each goat project area should be represented. However, it was primarily the leaders of non-goat projects saying there was a problem and goat leaders saying there wasn't. To solve the "Great Goat Bias" standoff, an objective approach was needed.

The first part of the study reported here measures the perspectives of youth and judges to see if they, too, perceived biases in the contest. Youth are able to articulate whether contests are fair once they can differentiate skill and luck, which is established by about age 12 (Thorkildsen & White-McNulty, 2002). Because the participants in the contest ranged from age 11 to 18, this researcher accepts the summarized youth responses to questions about fairness as valid. Recent research suggests that adults are interested in having youth take a larger role in community projects (Brennan, Barnett, & Baugh, 2007), so if the youth had a different perception of the contest it could influence adults who previously perceived a bias. Judges are generally regarded as authorities, and their perceptions carry weight with clientele as well.

The second, and most important, part of the study uses contest results to empirically determine whether any particular project animal causes a bias in the contest. Defining fairness is a complicated issue, but this researcher hypothesizes that if the contest is fair, the scores should be independent of the type of animal with which the member qualified. Likewise, members from each project area should have an equal chance to win the contest.

In one analysis, this researcher compared the scores of goat members and non-goat members in the county's Master Showmanship Contest. If goat members have an advantage, they should score higher than their non-goat counterparts. The other analysis used a list of winners and their project areas (animals) from the State Fair's Master Showmanship Contest. In a "fair" contest, the probability of a particular project area winning should not differ from random chance.

Methods

Youth participants and judges in the Clackamas County 4-H Master Showmanship Contest completed surveys at the 2005-07 county fairs (approved by IRB #3006). Participant surveys were administered after the contest, either immediately before or following announcement of the winners. Judges were given surveys when they completed their judging. Most survey questions asked participants to respond using a five-point Likert-type scale (5 = strongly agree; 1 = strongly disagree), and mean response scores were calculated. Additional items were presented as open-ended or check-box questions. Open-ended responses were categorized by the researcher, and attribution to more than one category was possible. The questions summarized here are only those that were linked to the fairness of the contests.

Total scores (the sum of the scores from each youth participant) were available from the county contest for 2004-2007. In all years, it was possible to determine which scores belonged to youth with goat projects and which to those who showed other livestock. The "round-robin" rotation of animals in 2004 and 2005 included Beef Cattle, Dairy Cattle, Dairy Goats, Fiber Goats, Llamas, Pygmy Goats, Sheep, and Swine. The 2006 and 2007 contests also included Meat Goats. The scoring rubric was changed for the latter 2 years. Due to these differences, it was not possible to combine all 4 years of data together for analysis. Therefore, a single-factor ANOVA was used on each of the two-year samples, comparing the goat and non-goat scores.

The winners of the Oregon State Fair 4-H Master Showmanship Contest were available for most years from 1999-2007. The State Fair rotation included Beef Cattle, Dairy Cattle, Dairy Goat, Pygmy Goat, Sheep, and Swine showmen through 2004. Meat Goat showmen were added in 2005. Defining a "winner" as a youth who was either champion or reserve champion showman, a chi-square test compared the actual and predicted frequencies of winning for each project area. The results were computed separately for contests held before and after the addition of the Meat Goat project.

Results

Youth surveys were collected from 52 youth over a 3-year period (2005-07) at the county contest. Three questions were of particular relevance to the topic of fairness. The two questions that used a Likert-type scale asked whether the contest was fair and whether the "best" showmen were chosen. Youth responses indicated that they were only slightly in agreement with these statements, and the large standard deviations show that there was a lot of variation in their opinions (Table 1).

Table 1.
Youth Participant Survey Responses

	5 = strongly agree; 1 = strongly disagree; n = 52
Statement	Mean	Standard Deviation
The Master Showmanship Contest does a good job in recognizing the best overall 4-H showmen.	3.7	1.3
The judging of the Master Showmanship Contest is fair.	3.8	1.3

An open-ended question asked youth who they thought had an advantage in the contest (Table 2). Forty percent of the youth said goat members had an advantage. Twenty-nine percent of youth said that no youth had an advantage or left the space blank.

Table 2.
Youth Participant Open-Ended Responses

Question: "Do you think that it is easier for some members than others to win the Master Showmanship Contest? For which members is winning easier?" (n = 52, may have multiple responses)
Blank or didn't know	10
Goat members (various reasons)	21
Those enrolled in more animal projects	9
Those who spent more time or effort studying	4
Past participants in the contest	4
Other reasons	5
No	5

From 2005-07, 18 judges returned surveys. A small number of these surveys may be from the same judges from different years, but each survey was treated as an independent response for analysis purposes. The judges' surveys included the same three questions as the youth survey. The judges, however, agreed rather strongly that the contest was fair and that it did a good job recognizing the "best" showmen (Table 3).

Table 3.
Judge Survey Responses

	5 = strongly agree; 1 = strongly disagree; n = 18
Statement	Mean	Standard Deviation
The Master Showmanship Contest does a good job in recognizing the best overall 4-H showmen.	4.2	0.9
The judging of the Master Showmanship Contest is fair.	4.6	0.6

The judges' explanations for who was likely to win were varied (Table 4). Thirty percent left the question blank or said no, 20% indicated that goat members had an advantage, and 20% said that time or effort studying made winning more likely.

Table 4.
Judge Open-Ended Responses

Question: "Do you think that it is easier for some members than others to win the Master Showmanship Contest? For which members is winning easier?" (n = 18, may have multiple responses)
Blank	4
Goat members (various reasons)	4
Those in more animal projects	2
Those who spent more time or effort studying	4
Other reasons	4
No	2

The analysis of the participants' scores was used to test the hypothesis that goat showmen score higher than other showmen. No significant difference was found in either 2004-2005 (Table 5, p>0.1) or 2006-2007 (Table 6, p>0.1).

Table 5.
2004-2005 County Contest

SUMMARY
Groups	n	Sum	Average	Variance	Low	High
Non-goats	30	19675	655.8333333	2080.281609	562	734
Goats	16	10819	676.1875	2549.895833	599	768
ANOVA
Source of Variation	SS	df	MS	F	P-value	F crit
Between Groups	4323.048007	1	4323.048007	1.92960707	0.171792	4.061706
Within Groups	98576.60417	44	2240.377367
Total	102899.6522	45

Table 6.
2006-2007 County Contest

SUMMARY
Groups	n	Sum	Average	Variance	Low	High
Non-goats	28	8634.5	308.375	1907.011574	189	386
Goats	21	6821.5	324.8333333	622.7833333	259	387
ANOVA
Source of Variation	SS	df	MS	F	P-value	F crit
Between Groups	3250.520833	1	3250.520833	2.389155195	0.128887	4.0471
Within Groups	63944.97917	47	1360.531472
Total	67195.5	48

Finally, a chi-square test of the State Fair results showed that no project area won the Master Showmanship Contest more often than predicted by chance (p>0.5). This was true both before and after the addition of the Meat Goat project to the rotation (Table 7).

Table 7.
Frequencies of Winning for Each Project Area

	1999, 2000, 2001, 2003		2005, 2007
Project Area	Observed	Expected	Observed	Expected
Beef	0.25	0.33	0.25	0.29
Sheep	0.38	0.33	0.25	0.29
Swine	0.38	0.33	0.25	0.29
Dairy goat	0.63	0.33	0.75	0.29
Pygmy goat	0.38	0.33	0.25	0.29
Dairy cattle	0.00	0.33	0.25	0.29
Meat goat	n/a	n/a	0.00	0.29
TOTAL	2.00	2.00	2.00	2.00
p value (from the χ2 test)	0.98683		0.983132
Note: The actual frequencies at which each project area's representative was a winner, compared to those expected by chance. Totals vary slightly due to rounding.

Discussion

Fairness of This Contest

The empirical results are clear: there is not a bias in favor of goat project areas. This is true for both county and state contests, even after the addition of the Meat Goat project to the line-up. But before we can lay the "Great Goat Bias" to rest, we need to examine the perception data.

The youth participants' and judges' surveys paint very different pictures of what is going on. The judges are far more satisfied with the fairness and results of the contest than the youth (Tables 1 and 3). When asked, without prompting, which youth had an advantage, 40% of youth (but only 20% of judges) suggested that goat members had an edge. Because the portion of youth who see a goat bias is roughly equal to the number of adult clients who see a bias (about 50%, by personal observation), this researcher surmises that youth and adult clients have discussed this matter on their own and come to their conclusions prior to contest participation and the survey. The researcher believes that the perspective of the judges, who are not involved or only peripherally involved in the county livestock advisory committee, is a more impartial and reliable assessment of contest bias, because they were not a part of certain contentious meetings.

What if the data had shown that goats do have an advantage? Some may argue that because a few judges cited a bias for goats (Table 4), average scores are slightly higher for goat showmen than other members (Tables 5 and 6) and because Dairy Goat members win the state contest with relatively high frequency (Table 7) there may be a bias present.

It would be premature to drop goats from Master Showmanship without doing additional research. One hypothesis is that Pygmy Goat and Dairy Goat (and Dairy Cattle) showmen would be more likely to win the State Fair contest: those youth with market projects may have sold their best showmanship animal at a junior livestock auction before the State Fair. In a study of limited duration, a dynasty of one family or one club could move through and skew the results. Or there could be a correlate to showing goats; for example, if it were found that goat members were more academically minded. The data collected for the study reported here are insufficient to test these hypotheses, but if the trends in winning frequency were to continue for several more years, it would be judicious to take a deeper look.

So, if showing goats isn't a predictor of winning the Master Showmanship Contest, what is? Are we giving out huge silver belt buckles based on chance? Hopefully not. The next step in the research project will be to look at factors that, by logic and the contest's logic model, are expected to produce winners. Likely correlates of winning are the amount of time spent studying, the number of years the participant has been showing, and the number of different animals that the participant regularly shows. Indeed, these reasons were all given by the youth and judges on their surveys (Tables 2 and 4).

In any case, 3 years of data is enough to convince this researcher that the Master Showmanship Contest is fair and that the addition of other goat projects has not introduces a bias into the contest. The researcher feels justified in extrapolating these results to any new type of animal: if there was reason enough to declare it a separate project area, then there should not be any concern about adding it to a round-robin style showmanship contest.

Communicating Results to Clientele

The final step in the research project will be to present these results to the county livestock committee. Ay, there's the rub. Interestingly, when presented with emerging results from past years indicating that there was not a bias, some 4-H leaders and parents refused to accept the research results, still insisting that goat members have an unfair advantage. This rigid adherence in the face of hard evidence is an intriguing characteristic. It also raises concerns about the effects that adult perceptions can have on youth participants.

A national report on 4-H competition highlighted the intense attention that parents can attach to winning when their children are involved (Allen et al., 1988). The likely key to gaining client acceptance of the results will not be in simply dismantling their old paradigm of "goat bias" but will be replacing it with a new understanding of actual predictors of winning.

As a side note, the researcher has learned that being clear and consistent with terminology can help lessen the perception of bias. It is common to refer to different kinds of livestock as "species." First of all, this is a misnomer. Beef and Dairy Cattle, taxonomically, are the same species, as are various domestic goats. Second, some clientele tried to build on this flawed foundation by claiming that since all goats are the same species, they should have a "run-off" before Master Showmanship so that only one goat member would enter. To avoid confusion and fallacious arguments, the researcher has adopted the nomenclature of "project area" instead of "species" and recommends that other Extension personnel make this change as well.

Numerical misinterpretations by clientele also must be addressed. Several clients justified the statement "goat kids always win" by grouping three or four different project areas together under the label of "goats," resulting in a logical error when it came to calculating probabilities of winning. Naturally, if you group four of nine participants into one "goat" category, a "goat" will win more often than a single swine entry.

Implications

The broad implications of this research for other professionals in Extension and/or youth work are in how they respond to client complaints and in how they conduct contests. If Extension professionals listen only to client input without conducting their own investigation, they may be following false assumptions and perceptions down a blind road. It can be difficult to distinguish between a situation where a few "squeaky wheels" are complaining and a true, widespread problem. Youth participants, in particular, are important voices to pursue. In this case, if the researcher had followed client concerns, some 4-H goat members would have been erroneously weeded out when they would otherwise have had the chance to be recognized as top showmen. If Extension professionals truly believe in a mission of presenting research-based information, then it is crucial that they research the facts and follow up decisions with timely testing of outcomes.

Any organization that uses contests as an educational tool should consider the results presented here. A proactive approach to contest planning should include an analysis plan with objective measures of fairness, not solely client perceptions of fairness. The researcher has found no other research that purports to mathematically test the fairness of a contest and believes the approach presented here is wholly unique. The basis of the method—testing to see whether a variable that should be independent really is—is widely adaptable and applicable to many different contests.

Acknowledgments

I would like to thank Dr. Mary Arnold of the Oregon 4-H Program for her review of the study design and surveys and her assistance in analyzing data. I also would like to acknowledge the youth who served on the Master Showmanship Task Force and supplied crucial input to a redesign of our county contest in 2005. The task force was organized by Lynette Black (now agent for the Wasco County, Oregon, 4-H Program). Tim Thacker, the volunteer Master Showmanship Coordinator for 2005-2007, aided by ensuring the contest was run to appropriate standards and facilitated data collection. Finally, I would like to express my appreciation to the members of the Clackamas County 4-H Livestock Advisory. Opinionated as they may be, they nonetheless are dedicated to our youth and bring important insight and good ideas to the program.

References

Allen, J., Iyechad, T., Mayeske, G., Parsons, J., Rodriguez, J., Singh, F., Swiney, K., Tolley, M. J., & Butterfield, M. (1988). National 4-H competition task force report. (Available from the National 4-H Council, 7100 Connecticut Ave., Chevy Chase, MD 20815.)

Arnold, M. E. (2007). 2006 4-H Animal Science Fair regional program evaluation report. Corvallis, OR: 4-H Youth Development Education Department, Oregon State University.

Brennan, M. A., Barnett, R. V., & Baugh, E. (2007).Youth involvement in community development: Implications and possibilities for Extension. Journal of Extension [On-line], 45(4) Article 4FEA3. Available at: http://www.joe.org/joe/2007august/a3.php

Chandler, T. J. L., & Goldberg, A. D. (1990) Building character through sports: Myth or possibility? Counseling and Values. 34(3):169-76.

Fetsch, R. J., & Yang, R. K. (2002). The effect of competitive and cooperative learning preferences in children's self-perceptions: A comparison of 4-H and non-4-H members. Journal of Extension [On-line], 40(3) Article 3FEA5. Available at: http://www.joe.org/joe/2002june/a5.php

Radhakrishna, R. B., Everhart, L., & Sinasky, M. (2006). Attitudes of 4-H participants about 4-H competitive events. Journal of Extension [On-line], 44(6) Article 6RIB3. Available at: http://www.joe.org/joe/2006december/rb3.php

Thorkildsen, T. A., & White-McNulty, L. (2002). Developing conceptions of fair contest procedures and the understanding of skill and luck. Journal of Educational Psychology. 94(2):316-26.

Weber, J. A., & McCullers, J. C. (1986). The blue ribbon: An American way of life. The effects of competition and rewards in 4-H. Journal of Extension [On-line], 24(3) Article 3FEA1. Available at: http://www.joe.org/joe/1986fall/a1.php