Evaluation of Juntos 4-H: A Wraparound Program Helping Latinx High Schoolers Succeed

Introduction

Considering the emergence of the Latinx population as the largest minority group in the United States (U.S. Bureau of the Census, 2017), 4-H and other youth-serving organizations have sought ways to engage more Latinx youths. Detailed focus group research, needs assessments, and community asset mapping activities conducted around the state of North Carolina in 2006 and 2007 demonstrated that Latinx families considered helping their youths succeed in school and attain a higher education the biggest priority for their communities (Behnke, 2008). Research has shown that Latinx youths have numerous strengths, and over the last two decades Latinx students have been performing better and better in school each year; however, research continues to show that Latinx youths drop out of high school at higher rates than their peers (Gramlich, 2017). Those who drop out are more likely to experience long-term economic and academic inequalities, long-term poverty, low-paying employment or unemployment, dependence on government assistance, increased incidence of incarceration, mental health issues, and negative physical health outcomes (e.g., Aud, KewalRamani, & Frohlich, 2011).

Established in 2007, the Juntos 4-H program empowers Latinx youths to achieve high school graduation and attend higher education (Behnke & Kelly, 2011). Based on studies that show the effectiveness of wraparound interventions in helping Latinx students stay in school, Juntos 4-H is comprised of four wraparound components: (a) family engagement, which includes a 5-week middle school or 6-week high school Juntos Family Workshop Series followed by bimonthly family nights to increase parent involvement and school communication; (b) monthly one-on-one success coaching by a local Juntos 4-H site coordinator to help students with their academic progress; (c) afterschool Juntos 4-H club meetings and activities twice a month throughout the school year; and (d) summer programming, including a weeklong summer college experience, full-day family college events, soccer tournaments, and other educational events (Behnke, Gonzalez, & Cox, 2010; Engle, Bermeo, & O'Brien, 2006; Schargel & Smink, 2014; Zeller, Carpenter, Lacefield, & Applegate, 2013).

A focus on academic behavior change in formative and summative evaluations has benefited Juntos 4-H, leading to real-world changes in participants' lives and producing a better understanding of how to serve and meet the needs of Latinx youths and their families (Behnke, Bodenhamer, McDonald, & Robledo, 2019; Behnke & Kelly, 2011). The goals of Juntos 4-H are (a) to increase Latinx student success by improving student attendance and grades and prospects of achieving high school graduation, (b) to increase the percentage of Latinx students attending higher education, (c) to increase family engagement that leads to students' educational success, and (d) to increase the sense of belonging among Latinx students and families in their schools and communities.

Thanks to funding from the New York Life Foundation, the National 4-H Council, the U.S. Department of Agriculture's Children, Youth, and Families at Risk grant program, and numerous other funding sources, Juntos 4-H expanded from North Carolina into 11 other states. Considering the dramatic growth of Juntos 4-H, evaluation has also grown in importance. As a first step to addressing this circumstance, we undertook a study to assess the impact of the North Carolina Juntos 4-H program on participating youths and their families. (Only student data are included herein.)

Method

Participants

Our study included 241 students who participated in all four components of Juntos 4-H during the 2016–2017 school year. All respondents in the study were of Latinx background. Three quarters (75%) were born in the United States, and about half (51%) were female. Teachers and school administrators had identified for participation in Juntos 4-H students who came from limited-resource households and/or struggled academically; thus the large majority of study participants had started the program with these characteristics. The largest proportion of respondents were in eighth grade (32%), with substantial representations from ninth grade (21%) and 10th grade (27%). We disseminated surveys to 312 active Juntos 4-H'ers, and 241 surveys were returned with complete data (for a 77% completion rate). The 241 surveys represented programs from five North Carolina counties (two rural, one suburban, and two urban) that were selected because they consistently included all four components of Juntos 4-H.

Measures

We created an instrument with 89 quantitative and seven qualitative questions (pilot tested with thousands of youths and improved from 2012 through 2016) assessing grit, academic success, belonging, social supports, parent engagement, college-going attitudes, and so on. To establish content validity, we asked a panel of experts (including members of National 4-H Council) to determine whether the instrument was appropriate for assessing the impact of Juntos 4-H (Henerson, Morris, & Fitz-Gibbon, 1987). The panel indicated that only minor changes to the survey were needed and signified that the various subscales were balanced. North Carolina State University's institutional review board approved the research; passcode-protected Microsoft Excel files allowed data to remain confidential; and data were matched across surveys according to student school ID numbers, which were eliminated after matching.

Besides basic demographic questions, we used only a small subsample of the 89 original quantitative questions in our study. The questions included 22 statements to which the respondents were asked to indicate level of agreement via a 4-point Likert scale (1 = strongly disagree to 4 = strongly agree). Twelve behavioral statements measured via a 5-point Likert scale (1 = never to 5 = all of the time) assessed changes in participants' actual behavior. Almost identical survey instruments were administered during the first month of the school year (pretest) and last month of the school year (posttest). The only differences between the pretest and posttest involved the addition of four questions on the posttest whose answers required involvement in the program.

Trained, bilingual graduate student data collectors traveled to the five counties to administer the surveys to study participants, using Qualtrics in a computer lab at each site. English and Spanish versions of the survey were available at the click of a button on participants' screens (though only 1% of respondents used the Spanish option). The Spanish versions were created and validated through back translation with three reviewers. The data collectors administered 293 pretests and 247 posttests. Less than 1% of administered surveys were incomplete (leaving 282 out of 293 pretests and 241 out of 247 posttests). Due to the nature of our evaluation, we analyzed the responses to quantitative questions using Wilcoxon signed-rank tests and basic descriptive statistics in SPSS 25.

Results

The first goal of Juntos 4-H is to increase Latinx student success by improving student attendance and grades and fostering achievement of high school graduation. Our findings showed that participating students experienced improved behavior toward achieving academic success. For example, 94% felt that participation in Juntos 4-H motivated them to get good grades in school, 87% felt it caused them to improve their grades to prepare for college, 78% felt it increased their motivation to graduate from high school, and 68% felt it improved their school attendance. Additionally, the ultimate behavior change of interest is that 100% of 12th-grade study participants graduated from high school on time.

The second goal of the program is to increase the percentage of Latinx students preparing for and attending higher education. Improvement in college preparation was measured via six matching pretest and posttest questions, the results of which we found to be statistically significant by using Wilcoxon signed-rank tests (see Table 1).

Additional posttest questions showed that 91% of respondents felt confident that they would continue their education after high school. Students also indicated what they planned to do after graduation: almost half of the respondents (47%) planned to complete a bachelor's degree at a university, and a third (33%) planned to pursue graduate studies. The remaining respondents planned to attend community college to seek an associate's degree or certificate (11%) or to complete high school without pursuing higher education (9%).

Results showed that Juntos 4-H consistently met the third goal of increasing family engagement that leads to students' educational success. For instance, 85% of students felt that their parents had learned how to help them do well in school because of the program. Additionally, four items demonstrated statistically significant behavior changes in the area of family engagement. For example, 96% of respondents reported that their parents had talked with them about their grades (27% higher than on the pretest), and 78% indicated that their parents had talked with them about college options (20% higher than on the pretest). Moreover, 91% of posttest respondents reported that their parents monitored their homework, and 83% reported that their parents had attended school events numerous times throughout the school year.

The final goal of the program is to increase the sense of belonging among Latinx students and families in their schools and communities. Related to this goal, 95% of respondents felt they belonged in their communities, and 94% felt they belonged at their schools. Additionally, 77% of respondents felt able to advocate for themselves at school, a statistically significant increase of 18% from pretest to posttest. Furthermore, in response to one assessment of behavior, 79% of respondents reported having participated in community service as part of their involvement in Juntos 4-H.

Conclusions and Implications

The results from our study provide empirical support that the four goals of Juntos 4-H are being met in North Carolina. In particular, these basic findings demonstrate the utility of working with Latinx youths on academic success and using 4-H to achieve the goals. The results can inform professionals working in Extension 4-H and other youth development programs in a number of ways as they seek to engage and retain this critical audience.

First, Extension programs for Latinx youths could be focused on higher education as a central theme that supports the youths in working toward their personal goals. Focusing on academic success and belonging were important to youths in our study and could be similarly important to youths in other Extension programs around the country. Because the behavioral objectives of the program were emphasized with program staff, at the conclusion of the program year the objectives appear to have been transferred to actual improvements in students' behaviors. Consistently reminding staff of key behavioral objectives can have long-term impacts on program success.

Second, because Latinx families continue to put a high level of emphasis on their children's academic success (LeFevre & Shaw, 2012), this primary focus of Juntos 4-H is consistent with the needs and wants of the families, and our findings indicate that the program is making an impact in this realm. In North Carolina, we have observed that Juntos 4-H also acts as a gateway to other 4-H programming and other Extension offerings (e.g., the Expanded Food and Nutrition Education Program; financial literacy programs; science, technology, engineering, and math activities), where numerous youths and families can further educate themselves. We also have noted that in some counties, Latinx youths now represent the majority of 4-H members, whereas prior to Juntos 4-H there were only a handful Latinx 4-H'ers. The popularity of Juntos 4-H demonstrates the need to have bilingual Extension professionals who can deliver varied, culturally appropriate content to Spanish monolingual families (Herndon, Behnke, Navarro, Daniel, & Storm, 2013).

Third, in our study, youths reported that most parents were very involved in the Juntos 4-H program and were doing more to be engaged in their academic lives. In other studies, parent surveys, focus groups, and attendance data have shown that Juntos 4-H families are committed to the long-term success of the program and that over time parent engagement increases alongside academic success (Behnke et al., 2019; Behnke & Kelly, 2011). Such studies show that deliberate measures to engage parents and families are an effective way to assure the long-term engagement of Latinx youths in 4-H (Hobbs, 2004). Because Latinx parents have minimal experience with the concept of afterschool clubs and even less with volunteer-led 4-H clubs, Extension-led programs for Latinx youths should be intentional in the involvement of parents to assure long-term engagement in 4-H. As we have observed literally hundreds of groups take part in the Juntos workshop series, serving thousands of Latinx youths and their parents, we have learned that parents are crucial to the success of Juntos 4-H.

Fourth, as evidenced in our findings around belonging, there is value in bringing Latinx youths together within 4-H. In our experience, many Juntos 4-H youths feel a sense of isolation in their experience as Latinx youths in their communities. Through Juntos 4-H, students have found lasting friendships with other Latinx students who often have shared cultural, familial, and language-related intersectional identities. Though this may not be the case everywhere, clubs such as these can offer youths ways to engage in their ethnic identity, be empowered, and make a difference in their communities. As our findings indicate, participating youths feel more connected to their schools and their communities, and their involvement in community service and community events makes a difference in those communities.

A fifth implication involves the evaluation of programs such as Juntos 4-H. Over the last decade, we have developed evaluation and tracking tools that have been used in numerous settings. The evaluation model, process, and tools we have created may benefit other Extension programs, and collaborations across programs could lead to increased validity of the measures.

There are limitations to our study. The relatively small sample size and selectivity of the sample necessitates caution when interpreting its results. Our findings reveal an overwhelmingly positive attitude toward Juntos 4-H that could be the product of self-selection bias as involvement in the program and the related research was voluntary. Students may be affected by confirmation bias due to their commitment to the program and the length of time they committed to the study.

Our study provides a window into future research needs related to Juntos 4-H. It is our intention to have an external evaluator conduct a randomized control trial study of the North Carolina Juntos 4-H program to establish with greater certainty the program's effectiveness. We recommend that evaluators develop greater depth in understanding Juntos 4-H by (a) examining the outcomes of program alumni, (b) exploring through longitudinal methods the impact of dosage on program participants' behaviors and outcomes, (c) using focus group and interview methodologies to better understand the impacts of the program, (d) comparing evaluation data across the numerous states that are conducting Juntos 4-H, and (e) working with North Carolina Department of Public Instruction and/or local school districts to study how Latinx students in the program compare to their Latinx peers not in the program.

References

Aud, S., KewalRamani, A., & Frohlich, L. (2011). America's youth: transitions to adulthood (NCES 2012-026). U.S. Department of Education, National Center for Education Statistics. Washington, DC: U.S. Government Printing Office.

Behnke, A. O. (2008). Expanding the Latino market niche: Developing capacity and meeting critical needs. Journal of Extension, 46(5), Article 5RIB5. Available at: http://www.joe.org/joe/2008october/rb5.php

Behnke, A. O., Bodenhamer, A., McDonald, T., & Robledo, M. (2019). The impact of the Juntos program: A qualitative evaluation. Hispanic Journal of Behavioral Sciences, 41(1), 63–84.

Behnke, A. O., Gonzalez, L., & Cox, R. (2010). Latino students in new arrival states: Factors and services to prevent youth from dropping out. Hispanic Journal of Behavioral Sciences, 32(3), 305–317.

Behnke, A. O., & Kelly, C. (2011). Creating programs to help Latino youth thrive at school: The influence of Latino parent involvement programs. Journal of Extension, 49(1), Article v49-1a7. Available at: https://www.joe.org/joe/2011february/a7.php

Engle, J., Bermeo, A., & O'Brien, C. (2006). Straight from the source: What works for first-generation college students. Washington, DC: Pell Institute for the Study of Opportunity in Higher Education.

Gramlich, J. (2017). Hispanic dropout rate hits new low, college enrollment at new high. Pew Research Center Fact Tank, September. Retrieved from https://www.pewresearch.org/fact-tank/2017/09/29/hispanic-dropout-rate-hits-new-low-college-enrollment-at-new-high/

Henerson, M. E., Morris, L. L., & Fitz-Gibbon, C. T. (1987). How to measure attitudes. London, UK: Sage.

Herndon, M. C., Behnke, A. O., Navarro, M., Daniel, J. B., & Storm, J. (2013). Needs and perceptions of Cooperative Extension educators serving Latino populations in the South. Journal of Extension, 51(1) Article v51-4tt5. Available at: http://www.joe.org/joe/2013february/a7.php

Hobbs, B. (2004). Latino outreach programs: Why they need to be different. Journal of Extension, 42(4), Article 4COM1. Available at: https://www.joe.org/joe/2004august/comm1.php

LeFevre, A. L., & Shaw, T. V. (2012). Latino parent involvement and school success: Longitudinal effects of formal and informal support. Education and Urban Society, 44(6), 707–723.

Schargel, F. P., & Smink, J. (2014). Strategies to help solve our school dropout problem. New York, NY: Routledge.

U.S. Bureau of the Census. (2017). Population estimates program (PEP). Retrieved from http://quickfacts.census.gov/qfd/states/00000.html

Zeller, P. J., Carpenter, S., Lacefield, W. E., & Applegate, E. B. (2013). Graduation coaching in a rural district school. International Journal for Leadership in Learning, 1(1), 106–126.