The Impact of the 4-H Program on Nevada Public School Youth

Introduction

4-H has a rich history of helping youth grow into productive citizens. Through participation in 4-H, youth learn life skills that they can further shape and use as adults. Federal, state, and county dollars fund 4-H programs coordinated by land-grant college and Cooperative Extension systems. Accountability for these funds is of paramount importance to ensure continued support. Furthermore, evaluation of 4-H youth development programming helps determine how 4-H affects youth and in what ways it needs to improve.

For much of the 4-H history, it was simply assumed that the youth development program was effective in helping youth avoid at-risk behaviors (Singletary & Smith, 2004). This level of accountability was deemed adequate for nearly 100 years, until conveying program worth through anecdotal success stories became seen as an unreliable method of program evaluation (Goodwin, Carroll, & Oliver, 2005). Within the past couple of decades, Cooperative Extension systems have increasingly recognized the need to discover more quantifiably defensible impact information (Gruidi & Hustedde, 2003; Karr, Keith, Vaugh, & Lockaby, 2001; Seevers, Dormody, & Clason, 1995; Scholl & Lago, 1994; Boyd, Herring, & Briers, 1992).

The numerous and varied approaches utilized suggest that no one measurement strategy to assess the impact of youth development programs has been commonly accepted. Adoption and replication of quantitative impact measurement approaches has been slow. Nevertheless, 4-H impact measurement is the responsibility of Cooperative Extension systems, and demands for quantitative outcome measures of programmatic impact continue to escalate.

Purpose/Objectives

One of the goals of The National 4-H Strategic Plan (2001) is to "collect national impact and accountability data that fully demonstrates the impact of 4-H on youth, their families, and communities" (p. 13). 4-H program managers and administrators are continually searching for improved methods of determining impact. Efforts to measure 4-H impact are numerous, and impact measurement strategies vary in focus and approach.

The study reported here sought to replicate a 4-H impact evaluation conducted in Montana (Astroth & Haynes, 2001), Idaho (Goodwin et al., 2005), Colorado (Goodwin, Carroll, & Oliver, 2005), and Utah (Tubbs, 2005). Replication of one instrument over time and across states should establish consistency and bolster accountability. The study contributes a Nevada perspective to those provided from youth in Montana, Colorado, and Utah. The study also provides Nevada stakeholders state-level accountability data.

The objectives of the study were to:

1. Measure impacts of the 4-H experience on the lives of Nevada youth and
2. Provide impact data for accountability and improvement of University of Nevada Cooperative Extension's 4-H youth development programming.

Methods/Procedures

Although these data are part of a larger causal comparative study, this particular portion of the study was descriptive/correlational and inferential in nature. The in-person, in-class written questionnaire approach was the same used in previous studies (Astroth & Haynes, 2001; Goodwin et al., 2005; Goodwin, Carroll, & Oliver, 2005; Tubbs, 2005).

The instrument consisted of 67 questions/items. The format included yes-no, multiple choice, level of agreement, fill-in-the blank, and short essay questions. The instrument collected various types of student information, including extracurricular activity involvement, risky or negative behavior, personal identity, positive identity, self confidence, character and personal empowerment, close relationship with parents/guardian and other adults, school leadership positions held, caring for others, general demographics, and 4-H membership information.

A stratified random sampling technique used two strata, urban and rural. Two school districts were grouped in the urban category and 15 in the rural category. School districts in urban and rural Nevada counties were randomly prioritized using the Research Randomizer (2006). Researchers contacted school districts and requested participation in the study in this random order. The sampling population for the study consisted of 5th, 7th, and 9th grade students enrolled in Nevada public schools. The sampling unit was Nevada public elementary, middle, and high schools.

Nevada school principals requested a total of 4,368 instruments, with 1,492 instruments returned, resulting in a 34.2% response rate. The potential for error due to Dillman's (2007) Coverage and Nonresponse threats limit the generalizability of the findings. It was impossible to accurately differentiate the amount of coverage and nonresponse error. Principals estimated the number of students matching the sample selection criteria and selected classes they thought would provide the greatest access to students in each age group. Without direct access to the students in each school, the researchers were unable to control this source of coverage error. If a school principal over-estimated the number of students matching the selection criteria, that would artificially inflate the estimate of coverage error.

Although the reader should use caution when transferring these results to other populations, the authors feel confident that the findings are reflective of anecdotal and observational insights, particularly those held by one of the authors, who has spent 24 years of his career in the state working with youth.

Age groups, gender, 4-H participation, and population density were the independent variables. Age groups consisted of 10-12-, 13-14-, and 15-18-year-old students. Survey question #50, "are you female or male," determined gender. Survey question #56, "have you ever belonged to a 4-H club that meets formally outside of school, " determined 4-H participation. The location of each respondent's school was used to determine population density, rural or urban.

Results/Findings

Over one-half, 52.3%, of the 1,492 survey participants were female, and 47.7% were male. And 63.1% of the survey respondents were from the urban school district, while 36.9% were from the rural districts. One hundred sixty-six students, 11.8% of the respondents, indicated they had been involved in 4-H. The majority of students, 88.2%, had never belonged to 4-H. The sample population was evenly distributed across three age groups, with 39.7% of the students 10-12 years old, 28.8% 13-14 years old, and 31.5% 15-18 years old.

A unique approach that the authors of the study took compared to the previously cited studies in Montana, Arizona, and Utah, was that they aggregated data on the dependent variables into constructs and indices. Summated construct scores were calculated for extracurricular activity involvement; school leadership positions held; close relationships with adults; caring for others; amount of negative behavior; personal identity; positive identity; and self-confidence, character, and empowerment. Data were transformed and recoded into new variables that represented composite dependent constructs.

Unlike previous studies in Montana, Arizona, and Utah in which data were analyzed from univariate and bivariate perspectives, researchers involved in the study reported here used more complex multivariate statistical analyses in an attempt to drill deeper into the complex relationships among variables. ANOVA for the extracurricular activities construct by age groups, gender, 4-H participation, and population density resulted in statistical significance for the variables, age groups (F = 3.974, p = .019), 4-H participation (F = 49.881, p < .001), and population density (F = 7.826, p = .005), in the amount of extracurricular activities youth engage in during the school week. As a rule, those who were most active in extracurricular activities were in the younger age group, 4-H members, and living in rural areas.

The ANOVA for the leadership positions construct by age groups, gender, 4-H participation, and population density resulted in statistical significance for the variables age groups (F = 6.459, p = .002), 4-H participation (F = 5.009, p = .025), and population density (F = 4.287, p = .039). Those who were most active in school leadership roles were younger youth, living in rural areas, and having 4-H experience.

The ANOVA for the caring for others construct by age groups, gender, 4-H participation, and population density resulted in statistical significance for a single variable in predicting the likelihood of youth helping others in need, 4-H participation (F = 13.198, p ≤ .001). Youth claiming to practice the most care for others were those having 4-H experience.

The ANOVA for the self-confidence, character, and empowerment construct by age groups, gender, 4-H participation, and population density resulted in statistical significance for a single variable, 4-H participation (F = 8.155, p = .004). Youth with highest self-confidence, character, and empowerment tended to be those with 4-H experience.

Conclusions/Recommendations

The previous studies compared the impact of 4-H youth programming among Extension districts or regions, while the study reported here used population density as an independent variable. Differences in research design make direct comparison of results between studies difficult; however, when examined at higher elevations, the study contributes to the greater regional effort to assess the impact of 4-H programming.

Results indicate that youth who have been involved in Nevada 4-H programming may have some character and behavior traits that differ from youth who have never been involved in 4-H. In particular, youth involved in 4-H are more likely to engage in other organized activities in and out of school, participate in more school leadership roles, care and contribute to the well-being of more people in need, and have higher self-confidence, character, and empowerment than youth who have never been involved in 4-H.

In the study reported here, 4-H involved respondents did not differ from non 4-H respondents with regard to amount of negative behavior practiced, closeness of relationship with adults, and levels of personal and positive identity. Again, the readers should use caution when transferring the results to other populations due to potential for Dillman's (2007) Coverage and Nonresponse error threats. One could conclude that 4-H programming is making a notable impact on the lives of Nevada youth, but room for improvement remains.

One strategy might be to further develop the program areas identified as strengths, such as leadership skill building, propensity to get involved in organized activities, and interest in caring for others. Safrit and Auck (2003) made sound recommendations to improve 4-H programming by capitalizing upon those areas in which 4-H seems to excel, leadership, and community service. Their recommendations included the following.

• Encourage volunteers to conduct community service and connect project work with service opportunities.
• Youth development professionals develop and share community service learning materials.
• Link statewide events to learning opportunities in volunteerism, community service, and service learning.
• Develop partnerships with schools and youth organizations by sharing curriculum and community service opportunities and essentially bring community service learning to all youth not just those identified as 4-H members.

In addition to Safrit and Auck's (2003) recommendations, the results of the study reported here should be summarized and made available to legislators, school officials, and community leaders. It is the responsibility of the University of Nevada Cooperative Extension to disseminate accountability information. It is critical that community decision-makers are aware of how 4-H programming investments pay long-term dividends in the growth and development of our youth.

There is need for a coordinated regional impact study. The studies in Montana, Idaho, Colorado, and Utah used different methodologies by randomly selecting schools within state Extension regions. Additionally, the present study included a multivariate approach in an effort to examine the phenomena in greater depth. Future researchers should consider more sophisticated multivariate or modeling techniques in an effort to provide yet a deeper understanding of this complex topic. The findings in Nevada were not entirely consistent with those in Montana, Idaho, Colorado, and Utah. Finally, adequate funding must be made available to support long-term regional assessments to examine every aspect of our Extension programs. To truly assess program impact, we must begin to use multiple measures over multiple periods of time to more scientifically estimate the true impacts of our programs.

References

Astroth, K. A., & Haynes, G. W. (2001). Final report of the Montana public school students' out-of-school time study. Bozeman, MT: Montana State University, 3-22.

Boyd, B. L., Herring, D. R., & Briers, G. E. (1992). Developing life skills in youth. Journal of Extension [On-line], 30(4) Article 4FEA4. Available at: http://www.joe.org/joe/1992winter/a4.php

Dillman, D. (2007). Mail and Internet surveys: The tailored design method 2007 update with new Internet, visual, and mixed-mode guide, 2^nd Ed. Pullman, WA: Washington State University.

Goodwin, J., Barnett, C., Pike, M., Peutz, J., Lanting, R., & Ward, A. (2005). Idaho 4-H impact study. Journal of Extension [On-line], 43(4) Article 4FEA4. Available at: http://www.joe.org/joe/2005august/a4.php

Goodwin, J., Carroll, J. B., & Oliver, M. (2005). Public school students' out-of-school time study: Measuring the impact of Colorado's 4-H youth development program. Unpublished manuscript, Fort Collins, CO: Colorado State University.

Gruidl, J., & Hustedde, R. (2003). Evaluation of capacity-building programs: A learning organization approach. Journal of Extension [On-line], 41(5) Article 5FEA1. Available at: http://www.joe.org/joe/2003october/a1.php

Karr, K., Keith, L., Vaugh, P., & Lockaby, J. (2001). The self-perceived impact of participation in the Texas 4-H council by former council members. Texas 4-H Research Review: 2000-2001. Office for Texas 4-H and Youth Research. College Station, TX: Texas A&M University.

National 4-H Strategic Plan. (2001). The power of youth in a changing world. Retrieved October 17, 2007, from: http://www.national4-Hheadquarters.gov/library/strategic.pdf

Research randomizer. (2006). Retrieved October 17, 2007, from: http://www.randomizer.org

Safrit, R. D., & Auck, A.W. (2003). Volunteerism, community service, and service-learning by Ohio 4-Her's in grades 4-12. Journal of Extension [On-line], 41(4) Article 4RIB6. Available at: http://www.joe.org/joe/2003august/rb6.php

Scholl, J., & Lago, D. (1994). Using non-reactive methods to study and improve 4-H programs. Journal of Extension [On-line], 32(3) Article 3TOT3. Available at: http://www.joe.org/joe/1994october/tt3.php

Seevers, B. S., Dormody, T. J., & Clason, D. L. (1995). Developing a scale to research and evaluate youth leadership life skills development. Journal of Agricultural Education, 36(2), 28-34.

Singletary, L., & Smith, M. (2004). Nevada 4-H statewide impact assessment. Reno, NV: University of Nevada Cooperative Extension, EB-04-01, 1-110.

Tubbs, T. M. (2005). Utah public school students' out-of-school time study to measure the impact of the Utah 4-H program. Unpublished Master of Science Thesis, Utah State University, Logan.