Findings of 4-H Impact Studies in Six Western States

Introduction/Theoretical Framework

For more than a century, the 4-H Youth Development program has provided opportunities and experiences for youth to plan their own learning, work with caring adults, and develop life skills.

According to Hamilton, Hamilton, and Pittman, (2004 [as cited in Heck & Subramaniam, 2009, p. 1]), the term "youth development" can be described in three different approaches, "1) a natural process through which youth grow into adults; 2) a set of principles underlying youth programs that encourage thriving among youth; or 3) a set of practices that foster the development of young people." Thus, positive youth development (PYD) refers to the developmental characteristics that lead to positive outcomes and behaviors among young people (Heck & Subramaniam, 2009). The framework of PYD as reported in "The Positive Development of Youth" views young people as resources to be developed rather than as problems to be managed" (Lerner, Lerner, & Phelps, 2008).

Early PYD programs focused on prevention or the effort of supporting youth before risky behaviors occur. Ali-Coleman (2006) has stated that prevention is not enough when we talk about youth development. He suggested that even if youth development professionals and volunteers are successful in preventing high-risk behaviors, they must also help young people prepare for the future not only by helping them to understand life's challenges and responsibilities, but to also teach them the life skills necessary for success. The absence of youth problems does not guarantee positive youth development.

Today's youth are faced with many choices. They have the choice to participate or not to participate in many risky behaviors. Astroth and Haynes (2002) found that time spent out-of-school is the largest block of time in a 24-hour period in a child's life. The hours directly after school, when many children are left unattended, is a time when these negative behaviors and activities double (Astroth & Haynes, 2002). After-school activities and out-of-school programs that emphasize youth development, such as 4-H, are ways of involving young people while giving them somewhere to be in the hours following school. Keeping youth active and involved in positive and meaningful activities can help decrease negative behaviors.

4-H program leaders are accountable to many stakeholders and need to show the effect of the program on the youth it serves. Program effectiveness or the ability of a program to make a positive difference in the youth that it reaches is a way in which accountability is determined (Boyd, Herring, & Briers, 1992). A goal of The National 4-H Strategic Plan (2001) was to "collect national impact and accountability data that fully demonstrates the impact of 4-H on youth, their families and communities" (p.13). Since 2000, six of the western states have chosen to survey students in the 5th, 7th, and 9th grades in schools across numerous geographically, socioeconomically, and culturally diverse areas of their respective states to determine the impact 4-H has on the lives of 4-H members.

Purpose/Objectives

The purpose of the project described here was to illustrate the impact that participating in 4-H had on youth in six western states. The analysis addressed the following objectives:

1. To describe 4-H and non 4-H youth participation by state, grade level, and self-reported academic performance, and


2. To compare 4-H and non 4-H youth on self-reported engagement in risk behaviors and the following social characteristics: level of personal identity; social competency; relationship with adults; helping others; leadership roles; and responses to character, self-confidence, and empowerment statements

Methods/Procedures

Six western states (Montana, Idaho, Utah, Colorado, Nevada, and New Mexico) individually conducted 4-H impact studies comparing 4-H youth and non-4-H youth between 2000 and 2007. Each state developed a separate impact report of findings. Data summaries from these reports were used to compare findings among the six states. It is important to note that while there is an 8-year span between studies, each study is a snapshot of a point in time in each state.

Each state used the same population of students in the 5th, 7th and 9th grades in public schools across numerous geographically, socioeconomically, and culturally diverse areas of their respective states. Public schools in each state were contacted by the county Extension agent and/or a representative of the state 4-H office. In almost all states, schools selected were a convenience sample of those willing to participate. Over 13,700 youth were surveyed from the six states. Idaho had the largest participation (n=3,601), with 26% of participants enrolled in 4-H. Nevada had the smallest (n=1,492), with only 11% of participants enrolled in 4-H.

The survey instrument was developed by Astroth and Haynes (2002) for the first of the six studies conducted in Montana. Although each subsequent state modified the instrument to focus on some unique aspects of youth programming in that state, the main themes of enrollment, academic performance, engagement in risk behavior, communication, leadership, helping others, and self esteem were common and allowed for a comparison across the states. Question formats for each of the domains investigated were not altered, allowing for comparisons across states. Each state assessed face and content validity using a panel of experts in the areas of youth development, education, health, research, and statistics. Reliability of the various subscales was established by Goodwin et al. (2005). Cronbach alphas for the subscales ranged from .57-.85. No other analysis was conducted. Data summaries from the six impact reports were analyzed and compared. Simple descriptive statistics were calculated on most questions.

Results/Findings

The results that are reported are from published or unpublished summaries from each state that participated in the studies. No additional manipulations were done on the data because no raw data was collected from the states.

4-H Enrollment

Of the students surveyed, roughly 25% self-reported participating in 4-H for at least 1 year. In the case of Montana, the number was about 36%, while New Mexico and Nevada reported the lowest numbers between 12.5 and 20%. New Mexico had the highest percentage of ninth-grade students, Colorado seventh-grade students and Utah fifth-grade students. Colorado had the lowest percentage of ninth graders, Utah seventh graders, and New Mexico fifth graders. Data were not available from Montana regarding percentages from each grade level.

Academic Performance

The data in Table 1 shows that overall, participants in 4-H had higher self-reported grades than non 4-H members. An average of 12.3 percentage points separated 4-H youth with mostly A's from their non-4-H member peers. Margins were much closer in the A's and B's category, with non-4-H members actually performing slightly better in Utah and New Mexico. Nevada did not compare grades between 4-H and non-4-H members.

Table 1.
Performance in School for 4-H and Non-4-H Youth in Percentages

		Mostly A's	A's and B's	Mostly B's	B's and C's	Mostly C's	C's and D's	Mostly D's	Less Than D's
Montana	4-H	33.4	37.5	9.7	13.5	2.0	2.4	0.07	0.07
	Non 4-H	19.6	37.7	9.1	20.8	4.2	5.6	1.3	1.9
Idaho	4-H	33.9	38.9	5.9	13.6	2.1	4.2	0.5	0.8
	Non 4-H	22.8	37.9	8.0	17.8	3.7	6.1	1.3	2.4
Utah	4-H	46.1	28.3	8.4	10.4	3.2	2.2	0.4	1.0
	Non 4-H	33.2	34.5	8.5	13.9	2.5	4.4	1.0	2.1
Colorado	4-H	38.2	36.2	5.5	11.4	2.5	4.7	0.5	1.0
	Non 4-H	26.0	32.3	6.1	18.8	3.1	8.8	2.4	2.5
New Mexico	4-H	30.1	28.1	9.3	16.8	6.6	6.1	1.0	2.0
	Non 4-H	17.9	31.6	7.9	23.7	4.1	9.9	1.7	3.0

Engagement in Risk Behaviors

4-H members in all states were less likely to report they had participated in shoplifting, drug use, driving after consuming alcohol, smoking cigarettes, and engaging in sexual activity than non 4-H members (Table 2). In Utah, Montana, and Colorado, students enrolled in 4-H were more likely to report having cheated on a test than their peers. 4-H members were less likely to report drinking alcohol. Utah 4-H members were the only member group with data available that reported a higher incidence of damaging property and skipping class than non-members. New Mexico 4-H youth self-reported a higher rate of smokeless tobacco use than non-4-H members. Utah students had no reported difference between 4-H members and non-members in the use of smokeless tobacco. Data for all categories were not reported by every state. Although most states opted to use percentages, Nevada chose to use the means of the Likert-type results to show the differences in self reported risk behaviors between 4-H and non 4-H youth. Nevada reported no or very minimal difference between 4-H members and non-4-H youth in risky behaviors.

Table 2.
Risk Behaviors by State and Participation in Percentages

	Utah		Montana*		Idaho*		Colorado		New Mexico
Risk Behavior	4-H	Non 4-H	4-H	Non 4-H	4-H	Non 4-H	4-H	Non 4-H	4-H	Non 4-H
Cheat on a test	27.4	20.7	35.8	26.6	25.1	27.8	25.5	23.8	46.7	56.2
Drink alcohol	5.5	6	18.5	22.6	10.9	20.5	16	15.9	22.6	32.1
Shoplift	1.8	5.5	6.9	14.5	5.8	11.5	5.2	6.5	16.2	24.5
Use drugs	4.3	4.4	8.8	18.9	5.7	15.4	4.3	8.1	14.9	21.9
Drive after consuming alcohol	0.9	1.5	4.0	9.0	1.7	5.9	**	**	33.2	32.8
Damage property	9.9	9.3	13.1	19.6	9.7	17.1	9.5	13.7	20.9	28.9
Smoke cigarettes	10.2	11.7	14	22.6	6.9	2	5.6	9.3	21.2	24.5
Use smokeless tobacco	3.4	3.4	**	**	2.3	17.5	**	**	12.3	7.6
Engage in sexual activity	2.8	2.4	**	**	**	**	**	**	16.6	24.0
Skip class	11.2	10.4	**	**	**	**	**	**	18.9	24.7
**Data not available

Parent Communication

In most states, 4-H members reported they discussed important items with their parents/guardians at a higher rate than non- 4-H members, excluding one exception in Nevada, where 4-H youth did not to speak to parents about drugs. Utah and Nevada had the smallest difference in percentages between 4-H youth and non-4-H youth. Idaho and Montana had some of the biggest gaps between groups.

Leadership positions held

4-H members were more likely to hold leadership positions, particularly as a committee member or chair. In all six states, 4-H members reported higher levels of elected leadership, having held a leadership position, being a committee chair, and being a committee member.

Helping Others

In all six states, 4-H members reported higher rates of helping others by being involved in a help project, giving time or money to charity, and helping the poor, sick, and others than non-members.

Positive Identity

Only Utah, Idaho, Colorado, and New Mexico chose to report data related to positive self-identity (Table 3). The identity statements included the following: 1) When things don't go well for me, I am good at finding a way to make things better, 2) *I have little control over the things that will happen in my life, 3) On the whole I like myself, 4) *At times, I think I am no good at all, 5) All in all, I am glad I'm me, 6) *I feel I do not have much to be proud of, 7) *Sometimes I feel like my life has no purpose, and 8) When I am an adult, I'm sure I will have a good life.

In every state, 4-H members responded positively to every identity statement at a higher percentage than non 4-H members, with the exception of two instances in Utah. 4-H participants in Utah were less likely to respond positively to statements 4 and 7 than non-4-H members. Nevada opted to use the means from the Likert-type responses to determine the degree of positive identity in the students that completed the survey. 4-H students were more likely to have mean scores corresponding to having a positive identity for every statement listed except for statements 5 and 6 (I'm glad I am me, and I don't have much to be proud of).

Table 3.
Percentage of Students Who Responded Positively to the Identity Statements

Question Number	Utah		Idaho		Colorado		New Mexico
	4-H	Non 4-H	4-H	Non 4-H	4-H	Non 4-H	4-H	Non 4-H
1	66.9	63.7	67.0	62.2	59.8	58.5	68.6	59.8
2*	54.8	45.7	52.9	47.7	58.7	48.0	45.1	41.7
3	78.4	77.7	81.8	75.9	81.1	74.2	71.2	68.2
4*	37.2	37.4	39.8	38.6	41.1	40.69	44.5	41.4
5	82.0	80.8	85.1	82.6	86.7	81.8	79.8	77.2
6*	72.1	67.3	77.1	61.5	78.4	67.2	66.5	61.87
7*	61.2	61.9	66.9	59.6	67.9	60.2	66.7	57.2
8	75.9	74.9	77.3	74.1	**	**	**	**
(1= Strongly Disagree - 5= Strongly agree) * negatively stated items ** Data not available

Character, Self-Confidence, Empowerment

4-H youth in four states were more likely to report having higher feelings of character, self-confidence, and empowerment than non-4-H members. Utah, Colorado, and New Mexico chose to include these data in their research reports by percentages, and Nevada reported mean scores.

In 29 of the 33 comparisons, 4-H members responded more positively to the character, self-confidence, and empowerment statements than non-4-H members. 4-H youth in both Utah and New Mexico responded less positively than-non 4-H members for statements 1 and 4. Table 4 shows the response rate of those students who were asked questions relevant to character, self-confidence, and empowerment. The results from the Nevada study are again recorded in means of the response value of the Likert-type scale as opposed to percentages. 4-H members in Nevada reported higher positive means to all statements than non-4-H members, with the exception of statement 3 (Ten years from now, I think I will be happy).

Table 4.
Percentage of Students Who Responded Positively to Character, Self-Confidence, and Empowerment Statements

Statements	Utah		Colorado		New Mexico
	4-H	Non 4-H	4-H	Non 4-H	4-H	Non 4-H
1. I can do things on my own	79.8	80.4	85.6	77.1	68.5	74.3
2. I set goals	80.7	74.5	77.4	68.7	72.8	62.7
3. Ten years from now, I think I will be very happy	82.9	81.6	82.3	77.1	79.5	72.3
4. I am responsible for my actions	89.2	89.5	86.1	86.6	80.4	82.4
5. Adults in my town or city make me feel important	67.2	61.7	54.7	45.6	52.3	39.7
6. Adults in my town or city listen to what I have to say	55.5	52.6	44.7	38.2	48.9	34.0
7. Adults in my town or city don't care about people my age *	56.2	55.4	60.0	55.1	62.3	49.4
8. In my town or city, I feel like I matter to people	61.5	59.6	53.0	44.7	56.3	42.7
9. In my family, I feel useful and important	77.8	78.1	77.6	71.8	77.1	67.2
10. I'm given lots of chances to help make my town or city a better place to live	50.0	48.9	47.4	34.7	48.2	32.3
11.Students help decide what goes on at my school	58.6	58.2	56.6	53.4	54.5	44.8
* Strongly disagree and disagree responses are counted here because this is a negatively worded statement

Conclusions/Recommendations

Although there are other possible explanations for the results, these data show that participation in the 4-H Youth Development Program makes a positive difference in the lives of those surveyed. In comparing the data from the six western states, these data show that youth involved in 4-H programs overall had higher positive response values than youth not in 4-H in almost every category.

• 4-H members in all six states had higher self-reported grades than non-4-H members.


• In all states, students involved in 4-H were less likely to participate in risky behaviors.


• In all states, 4-H members reported discussing important items with their parents/guardians at a higher rate than non-4-H members, excluding one exception in Nevada.


• 4-H members were more likely to hold leadership positions, particularly as a committee member or chair. In all six states, 4-H members reported higher levels of elected leadership, having held a leadership position, being a committee chair, and being a committee member.


• In all six states, 4-H members reported higher rates of helping others by being involved in a help project, giving time or money to charity, and helping the poor, sick, and others than non-members.


• In every state, 4-H members responded positively to identity statements at a higher percentage than non-4-H members.


• 4-H members were more likely to report having higher feelings of character, self-confidence and empowerment than non 4-H members.

Data from the six impact studies supports that the 4-H Youth Development Program is making a contribution to individuals, families, and communities in which these youth are engaged and supports the notion that 4-H is by this definition engaged in positive youth development.

Demonstrating the significance of successful youth development programs like 4-H should be a priority in the minds of administrators, educators, and legislators. 4-H activities do not simply teach youth skills in agriculture and home economics, but include non-formal, experiential, educational programs that teach youth valuable life skills (Boyd, Herring, & Briers, 1992). The data gathered from the research reported here will have value to educators, parents, legislators, and other program leaders in articulating 4-H program impacts. The results of the study can be used to market and promote 4-H membership and volunteer recruitment opportunities.

Several possible long-term outcomes are that 4-H members are developing into mature adults who will be productive members of society because they are less likely to engage in activities that are illegal and/or socially unacceptable. These students are more likely to be successful throughout life as they build on the successes they have achieved while in the 4-H program, either scholastically or within the program itself. As 4-H members become adults and active participants within their communities, they will have stronger personal relationships and perhaps volunteer for 4-H in the future. It is also likely that because of the leadership development and opportunities that arise in the program that former members will go on to participate and hold other leadership positions both civically and politically.

Documenting program impacts and being accountable are essential tools for program success and continuation. In an era of increased competition for limited resources, funders expect evidence that a program is making a difference. These studies have merit individually because they provide a snapshot of the 4-H Youth Development Program in each state. Collectively, they suggest that 4-H is a positive youth development program. The similarities in the findings from six different state programs support that the results are not random. Caution in generalizing the results to larger populations should be taken because in almost every situation a convenience sampling procedure was used. It must also be recognized that factors other than membership in 4-H may account for some of the differences found.

More research would benefit the 4-H program on both the local and national level. Major areas that further research should focus on are:

• Regional data differences could be compared for similarities and differences.


• A standardized survey should be used for every state studied so that the all the results can be compared


• A longitudinal study should be conducted with the same students to provide insight into the effects of 4-H on the lives of those students over time to determine if 4-H is really making the difference that were have found.

These variables might provide insight into factors related to the differences between 4-H and non-4-H members. These results are very positive for the 4-H program. Van Horn, Flannigan, and Thompson (1999) summed it up in saying, "If the 4-H program wants to be a force in the future, it needs to be progressive and adaptive to new trends and ideas... It needs to continue to address the issues that face today's youth such as drug, tobacco, and alcohol abuse" (p. 6).

References

Ali-Coleman, K. (2006). Youth Development Assertions: Youth Development theory believes five things to be true. Retrieved from http://youthdeveopment.suite101.com/article.cfm/youth_development_assertions

Astroth, K. A., & Haynes, G. W. (2002, August). More than cows & cooking: Newest research shows the impact of 4-H. Journal of Extension [On-line], 40(4) Article 4FEA6. Available at: http://www.joe.org/joe/2002august/a6.php

Boyd, B. L., Herring, D. R., & Briers, G. E. (1992). Developing life skills in youth. Journal of Extension [On-line], 30(4). Article4FEA4. Available at: http://www.joe.org/joe/1992winter/a4.php

Goodwin, J., Barnett, C., Pike, M., Peutz, J. Lanting, R., & Ward, A. (2005). Idaho 4-H impact study. Journal of Extension [On-line], 43(4). Article 4FEA4. Available at: http://www.joe.org/joe/2005august/a4.php

Hamilton, S. F., Hamilton, M. A., & Pittman, K. (2004). Principles for youth development. In S. F. Hamilton & M. A. Hamilton (eds). The youth development handbook: Coming of age in American communities. (p. 3-22). Thousand Oaks, CA: Sage Publications, Inc.

Heck, K. E., & Subramaniam, A. (2009, winter). Youth development frameworks. Retrieved from http://fourhcyd.ucdavis.edu

Learner, R. M., Lerner, J. V., Phelps, E. (2008). The Positive development of youth. Report of the findings from the first four years of the 4-H study of positive youth development. Institute for Applied Research in Youth Devleopment.

Lee, C. D., Beard, R., & Straquadine, G. S. (2003). Description of Utah 4-H club members' perceived level of critical elements in the 4-H experience. Paper presented at Western Region Teaching Symposium. Logan, UT: Utah State University.

Lewis, S. R. (2007). Impact assessment of the Nevada 4-H program: An examination of public school students' perceptions and behavior. Proceedings of the 2008 Western Region AAAE Research Conference, Volume 27 [CD-Rom].

National 4-H Strategic Plan. (2001). The power of youth in a changing world. Retrieved from http://www.national4-hheadquarters.gov/library/strategic.pdf

Singletary, L., & Smith, M. (2004). Nevada 4-H statewide impact assessment. Reno, NV: University of Nevada Cooperative Extension, EB-04-01, 1-110.

Van Horn, B. E., Flannigan, C. A., & Thompson, J. S. (1999, February). Changes and challenges in 4-H (Part 2). Journal of Extension [On-line], 37(1) Article 1COM. Available at: http://www.joe.org/joe/1999february/comm1.php