Idaho 4-H Impact Study

Background

In 2002, many states passed the 100-year mark for the existence of the 4-H program. Will there be a 200-year mark observed for the 4-H program in your state in 2102? The answer to that question lies in the ability of the 4-H program to demonstrate to citizens and elected officials the continued relevance and worth of this 100-plus-year old institution. In the past, the value of 4-H has been assessed and conveyed by anecdotal success stories and the "gut-feeling" of those familiar with the program. Those methods are no longer adequate to ensure its continued impact and existence. This study described here was designed to provide sound, research-based information to the public about the effect of 4-H membership on the lives of young people.

Does the 4-H program and the associated activities these young people become involved in just give them something to do in order to keep them out of trouble--or are life skills being developed to help them become successful, contributing, and competent adults? This question seems to be answered in the study conducted by Astroth and Haynes (2001). Astroth and Haynes reported that their research showed that 4-H participants were more likely than other youth to:

• Succeed in school, getting more A's than other kids
• Be involved as leaders in their school and community
• Be looked to as role models by other youth
• Help others in the their community (p. 9)

Furthermore, Astroth and Haynes say that the 4-H kids surveyed told them that they were less likely than other kids to:

• Shoplift or steal (3 times less likely)
• Use illegal drugs of any kind to get high (2 times less likely)
• Smoke cigarettes
• Damage property for the fun of it (2 times less likely)
• Skip school or cut classes without permission (p. 10)

Methodology and Data Analysis

The Idaho 4-H Impact Study is actually a replication of the Astroth and Haynes study. The study was conducted in a nearly identical format, with only minor adaptations made for the situation in Idaho. The strengths of the Astroth and Haynes study were that it provided a simple method to acquire a random sample of 4-H and non-4-H members in a population of youth that represent the state.

One perceived weakness in the Astroth and Haynes study was that there were no ethnic demographic data collected. This was because, given the low ethnic minority populations in Montana, the researchers felt that there would not be a large enough number of surveys completed by this audience to conduct an accurate statistical analysis of the data. A valid reason for not asking the ethnic demographic question is that anonymity of the subjects could be comprised if there were only one or two ethnic minorities present in a classroom where data were collected. For these reasons, no ethnic demographic question was present in the Idaho study, either.

Organization of methods in this study were intended to determine the awareness and attitudes of students in 5th, 7th, and 9th grades within the state of Idaho concerning the depth of the perception of their own development and the impact 4-H may have had on them. These grades were chosen because they are a broad representation of the audience served by the 4-H program.

The populations used in this study consisted of a stratified random sample of 18 selected counties of the 44 in the state of Idaho. A stratified random sample was used to ensure an equal distribution of counties in all four Extension Districts in the state.

In each county, four schools were randomly selected. The list of schools was retrieved from the 2001-2002 Idaho Educational Directory. Using the complete list of schools that had grades 5th, 7th, and 9th within their populations; a computer random generator was utilized to determine the random selections. This group composed a total of 6,334 possible students in all three grades.

The questionnaire used to assess the ways young people spent their time out-of-school and how that use of time may affect their academic, emotional, social, and cognitive development was patterned after the Astroth and Haynes (2001) model, which was used by the Montana State 4-H Office for a similar study. A request was made to The Search Institute to use some of their questions, and that approval was granted. The questionnaire included basic demographic and family-oriented questions as well as questions categorized into seven subscales:

• Positive identity--personal power, self esteem
• Social competency-¨leadership, planning and decision making, resistance skills
• Relationships with adults
• Self confidence
• Empowerment
• Kindheartedness
• Skills

The student questionnaire was divided into seven sections and consisted of 73 questions. In addition to an English version of the survey, a Spanish version was available for those pupils who had a hard time reading or comprehending English. The format of the questions consisted of fill-in-the-blank, yes/no, essay, circle those that apply, and multiple-choice. The reliability of the Likert-type items in this survey was assessed by Cronbach's standardized coefficient alpha, a measure of the extent to which a summated rating scale can be formed meaningfully from a set of correlated items. Cronbach's (1951) standardized alpha has a maximum value of 1 and a minimum that can be less than one (when items have opposite valence, that is, some are worded negatively and others positively).

In these results, with one exception, the alpha values are strong. For items q4 through q22 (measuring positive identity and social competency), alpha = .8591; for items q32 through q44 (measuring self-confidence, character, and empowerment), alpha = .8257; for items q48 through q51 (measuring skills learned), alpha = .5729; and for items q63 through q69 (measuring the impact of 4-H), alpha = .8658. Items on the questionnaire addressed several areas. The survey included basic demographic and family-oriented questions as well as questions categorized into seven subscales: 1) Positive identity--personal power, self esteem, 2) Social competency--leadership, planning and decision making, resistance skills, 3) Relationships with adults, 4) Self confidence, 5) Empowerment, 6) Kindheartedness, and 7) Skills.

The Idaho questionnaire was pilot tested in the Filer School District in April 2002. The pilot test was done to ensure that the survey was both readable and understandable and to determine time constraints, if any. This was also when the Spanish version of the survey was discovered as necessary. A total of 185 surveys were gathered at this time.

Questionnaires were not coded in advance. However, when each class finished within the participating grade, the instruments were wrapped with an accompanying tracking sheet completed by the facilitator with pertinent information.

Collection of the Data

The questionnaires were given to the corresponding county Extension educators within the randomly selected counties. Packets of instructions on how to administer the instrument were sent to all county educators within these randomly selected counties. Initial contact was made by the educators to the school superintendents and/or principal to gain permission to conduct the survey in their schools. Actual execution of the questionnaires was conducted in the fall of 2002 and early spring of 2003. The parental "opt out" letters were sent previous to the actual administration of the study. On April 3, 2003, the survey portion of the study was declared complete. Response rates are reported in Table 1.

The following table shows the actual numbers of responding districts and the corresponding schools:

Logistic regression using the chi-square (χ²) test was performed for analysis. Frequencies and percentages were run on the questions regarding extracurricular participation, negative behaviors, positive identity, social competency, contact with adults, self confidence, empowerment, caring about others, skills learned, skipping school, age, grade, gender, household size, family types, and impact of 4-H for members.

Findings and Discussion

Respondents were asked to indicate a number of background characteristics to help with statistical analysis. Of the 3,601 respondents who participated in the study, 50.2 % were male, and 49.8 % were female. Of the 3,601 respondents to the questionnaire, 935 reported they had been involved in the 4-H program for at least one year. Of the 935 respondents who had been in 4-H, 555 of them had been a member for 2 years or more. (Table 3) Data analysis of 4-H membership in these findings is confined to 4-H membership of 2 or more years.

Students were asked to specify which grade they were in at the time of the survey: 5th, 7th, or 9th. Of the participants, 43.3 % indicated the 5th grade, 25.3 % indicated the 7th grade, and 28.6 % indicated the 9th grade. (Table 4)

All respondents were asked to designate where they lived at the time of the study. Of the 3,601 students, 15.2 % lived on a farm, 33.8 % lived in the country but not on a farm, 47.4 % lived in town, and 128 did not indicate where they were living.

To determine the involvement in the identified at-risk behaviors, respondents answered yes or no to the wording on the survey instrument as follows:

During the past year did you . . .

• Cheat on a test?
• Drink any alcohol without parental permission?
• Shoplift?
• Use any drugs like marijuana, methamphetamines, or cocaine; or sniffed glue or other fumes to get high?
• Drive a car when you've been drinking?
• Carry a gun to school?
• Smoke cigarettes?
• Use smokeless tobacco (such as Copenhagen/Skoal)?

As 4-H membership versus non-4-H membership was compared with at-risk behaviors, the following differences are observed. When all grades (5th, 7th, and 9th) are in the population, 4-H members were less likely to engage in shoplifting, drug use, damaging property just for the fun of it, and smoking (Table 5).

The argument could be made that many of the 5th and 7th grade respondents have not yet been exposed to many of the at-risk behaviors asked about in the study. For example many 5th and 7th graders have not had the opportunity to drive, much less drink and drive. Perhaps a more realistic picture emerges when only 9th grade responses are examined for these at-risk behaviors (Table 6). 4-H Students in the 9th grade were less likely to drink alcohol, shoplift, use drugs, damage property, or smoke cigarettes than their non-4-H classmates.

As shown in Table 7, 4-H members were more likely to report that they received mostly A's than non-4-H students.

This research revealed that 4-H members are more likely to succeed in school (Table 7), help others within their communities (Table 8), and be involved as leaders in their school (Table 9). By being active in leadership roles, these 4-H members are preparing themselves to become productive citizens within their community and world.

Data shown in Table 10 below indicate Idaho 4-H youth have better relationships with adults than those students who are not active in 4-H. Youth were asked if there was an adult with whom they felt comfortable going to for help when they had an important question about their life. Seventy-six percent of active 4-H members said they did have someone they could go to other than a parent. Additionally, 79.2 % of Idaho 4-H youth said they have had a good conversation in the past month with one of their parents that had lasted longer than 10 minutes. Comparatively, when asked if they had a good conversation that had lasted longer than 10 minutes with someone besides their parents, 63.8 % of Idaho 4-H members said they had.

When asked if they could go to their parents to discuss issues related to drugs, alcohol, sex, or any other serious issue, 4-H members were more likely to do so than were non-4-H members (Table 11).

There were 33 questions on the survey that related to the respondents attitude or perceptions about the world around them. The authors have chosen to use the term "world view" to describe this series of questions as a whole. This series of questions are further broken down into five groups of:

• Positive Identity--personal power, self-esteem, sense of purpose, and positive view of the future
• Social Competency--planning, interpersonal competency, and resistance skills
• Social Competency--leadership
• Self-confidence, Character, and Empowerment
• Skills Learned

Responses to these world-view questions were on a Likert-type scale: strongly agree, agree, neutral, disagree, and strongly disagree. As the data is reviewed, please note that five of the questions are reported in the opposite direction of the rest of the questions and are reported as disagree and strongly disagree. Four of these questions are noted in Table 14 with a superscript "a," while the fifth is noted in Table 16. The first group of Positive Identity Statements shows that there was significant positive difference in the way 4-H members responded to the following statements (Table 12).

The next group of social competency questions also revealed that 4-H members were more likely to respond in a positive manner to the statements of:

• I'm good at planning ahead.
• I feel really sad when one of my friends is unhappy.
• I am good at making and keeping friends.
• I know how to say "no" when someone wants me to do things I know are wrong or dangerous.

The data in Table 14 shows that 4-H members were more likely to respond in a positive manner to 10 out of 13 of the Self Confidence, Character, and Empowerment statements than non-4-H members.

The data reported in Table 15 show that 4-H members were more likely to volunteer in class to lead activities and that they feel other kids look up to them and follow their example (Table 16).

The data in Table 16 suggest that 4-H members are more comfortable at public speaking, organizing, and at managing money.

Conclusions and Recommendations

The data collected in the study reported here support the conclusions made by Astroth and Haynes (2001). Authenticating the significance of successful youth development programs like 4-H should be a priority in the minds of facilitators, legislature, and educators. 4-H activities don't simply teach youth skills in agriculture and home economics, but include nonformal, experiential educational programs that teach youth valuable life skills (Boyd, Herring, & Briers, 1992).

The study supports the conclusions made by Boyd et al. (1992). It is hoped that the study will also be useful in the future support of the 4-H program in Idaho as well as other states. Federal, state, and county government budgets to deliver mandated programs have been stretched to their limits in many areas of the country. The future existence of non-mandated programs such as Cooperative Extension and 4-H educational efforts in the future will depend on informing elected officials, potential 4-H families, and potential donors about the value of this program. The 4-H program must demonstrate the positive impact that it has on youth and families if it is to remain a relevant educational program in this country. Data such as that obtained in the research study reported will help to fill this need.

One of the major recommendations of the principle investigators of the study is that other states replicate it. As a larger group of states duplicate the original Montana Out of School Time/4-H Impact Study, those data could be compiled into a larger and much stronger research project to benefit 4-H nationwide.

Another recommendation is to communicate these data in a systematic and effective manner to the elected officials, opinion leaders, school officials, and citizens of the state of Idaho. Such an effort could have the following results:

• Increase awareness and support for the 4-H program at the local and state level.
• Enhance 4-H volunteer leader recruitment and retention.
• Increase parent interest in having their kids involved in the 4-H program.
• Enhance 4-H support from University of Idaho faculty who are not directly involved in the program.

"I have developed presentation skills, work skills, social skills, organization skills, and participation skills."
9th grade 4-H member, Bingham County, ID

The bottom-line observation by the investigators is that 4-H does make a difference in the lives of young people. Now it is time to share this information with others.

"I have learned better organizing and money management skills. I have learned how to work hard and be dedicated to my work."
9th grade 4-H member, Oneida County, ID

References

Astroth, K. A., & Haynes, G. W. (2001). Final report of the Montana public school students' out-of-school time study. Bozeman, MT: Montana State University.

Astroth, K. A., & Haynes, G. W. (2002). More than cows & cooking: Newest research shows the impact of 4-H. Journal of Extension [On-line], 40(4). Available at: http://www.joe.org/joe/2002august/a6.shtml

Boyd, B. L., Herring, D. R., & Briers, G. E. (1992, Winter). Developing Life Skills in Youth. Journal of Extension [On-line], 30(2) Available at: http://www.joe.org/joe/1992winter/a4.html

Idaho Educational Directory (2001-2002). Compiled and produced by Idaho's State Department of Education. Available On-line at: http://www.sde.state.id.us/admin/eddirectory/