 |
June 2018
|
June 2018 // Volume 56 // Number 3 // Tools of the Trade // v56-3tt3
Addressing Retention in Youth Programs: A Survey for Understanding Families' Experiences
Abstract
Research on retention in the 4-H youth development program has consistently shown that one of the primary indicators for youths' dropping out of 4-H is being a first-year member. Extension 4-H professionals from California, Idaho, Wyoming, and New Jersey formed a team to study this issue. Our team surveyed first-year members and their parents/guardians to better understand why youths were not reenrolling in 4-H after their first year. This article introduces the survey used to assess the first-year experience and intent to reenroll. We discuss the survey development process, survey testing, lessons learned, and conclusions related to its future use.
Background
4-H clubs have been a staple in quality educational program delivery since Extension's inception. Youths who participate in 4-H earn higher grades, have higher levels of civic engagement, and engage in less risky behavior (Lerner & Lerner, 2013). Although impressive, such positive impacts can be made only if participants remain in the program (Pratt & Bowman, 2008). The most recent available U.S. Department of Agriculture 4-H enrollment reports indicated that 31 of the U.S. states and territories experienced declines in club enrollment (U.S. Department of Agriculture, 2010). Research has revealed that dropout occurs because (a) youths are busy with sports or other organizations, (b) youths are unhappy with their clubs or projects, and (c) parent involvement is low (Harder, Lamm, Lamm, Rose, & Rask, 2005; Hartley, 1983; Ritchie & Resler, 1993). Demographic factors also influence dropout (e.g., gender, age at entry) (Astroth, 1985; Defore, Fuhrman, Peake, & Duncan, 2011; Harder et al., 2005; Ritchie & Resler, 1993). Further, a primary indicator for dropping out is being a first-year member (Astroth, 1985; Hamilton, Northern, & Neff, 2014; Harder et al., 2005; Hartley, 1983). Examination of enrollment data from California showed similar trends (Lewis, Horrillo, Worker, Miller, & Trzesniewski, 2015).
Our multistate team consisting of California, Idaho, Wyoming, and New Jersey 4-H professionals formed to study retention. We developed a survey for first-year 4-H members and their parents/guardians to better understand why youths were not reenrolling in 4-H after their first year. Here we introduce the survey used to assess first-year experience and intent to reenroll.
Survey Development and Testing
Our effort began with county-based 4-H professionals conducting interviews via phone with several first-year families whose children had not reenrolled in 4-H. We used feedback from the interviews as well as similar surveys from other states to develop the questions for our survey. We developed parallel versions of the survey for youths and adults to gain both perspectives.
We pilot-tested the survey in summer 2015 in a subset of counties in California and all counties in Idaho. To establish baselines regarding motivation to join and expectations of the program, we asked survey takers why they had joined 4-H and how they had heard about 4-H. We then asked two open-ended questions about their expectations of 4-H and whether those expectations were met (from adults only). We asked whether (a) youths intended to reenroll in the program, (b) they had other family members in 4-H, and (c) their parents/guardians had been in 4-H as children; the latter two questions helped us understand prior history with the program. To assess the experience of being in 4-H, we asked "What was the best part of your experience in 4-H last year?" and "If you could change one thing about 4-H, what would it be?" We also asked youths to indicate how much they agreed with 10 statements related to program features that maximize youth development (Eccles & Gootman, 2002). Institutional review boards at each university approved the study with online consent; responses to the survey were anonymous.
In summer 2017, we added items to the adult survey that addressed involvement of other children in the family and their own involvement in 4-H in order to gain a better understanding about the family's history with 4-H and how that history may have influenced their expectations of and experience in the program. Further, 4-H teens in California and New Jersey reviewed the survey and suggested asking about barriers not previously identified by our team, allowing for a more comprehensive view of program experiences.
In summer 2017, we emailed a Qualtrics survey link to all first-year families in California, Idaho, Wyoming, and New Jersey. In addition to the demographics presented in Table 1, we collected enrollment data (project participation, state, county, and club name). The survey consisted of 28 items and took on average 11 min to complete. The mean youth age from the youth reports was 10.06 years (with a standard deviation of 2.58) and from adult reports (on the child's age) was 9.83 years (with a standard deviation of 2.45).
| Demographic category | Youth reports % | Adult reports % |
| Gender | ||
| Female | 62.4 | 59.2 |
| Male | 37.4 | 39.0 |
| Gender fluid/gender of multiple children given | 0.2 | 1.8 |
| Ethnicity | ||
| Non-Hispanic or Latino | 82.4 | 83.1 |
| Hispanic or Latino | 17.6 | 16.9 |
| Race | ||
| White | 81.5 | 80.4 |
| Black or African-American | 1.2 | 1.2 |
| Asian | 2.7 | 2.2 |
| American Indian or Alaska Native | 1.4 | 0.9 |
| Native Hawaiian or other Pacific Islander | 0.2 | 0.1 |
| Other | 5.3 | 7.6 |
| Multiple races | 7.8 | 7.6 |
| Residence type | ||
| Farm | 18.9 | 18.8 |
| Town (nonfarm, rural, population <10,000) | 28.4 | 28.0 |
| Town or city (population 10,000–50,000) | 30.8 | 28.4 |
| Suburb of city (population >50,000) | 9.8 | 12.0 |
| Central city (population >50,000) | 12.2 | 12.8 |
| Family involvement in 4-H | ||
| Family has another child in 4-H | — | 40.3 |
| Parent/guardian was in 4-H as a child | — | 31.1 |
| Plans to enroll in 4-H again | 75.0 | 86.0 |
| Note. Youth N = 751; adult N = 931. Percentages are based on the total of those who answered the question. Missing data ranged from 5.0% to 28.0%. A dash (—) indicates that data were not collected from the source. | ||
Table 2 provides a list of 10 statements youths rated on program features that maximize youth development. We used a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Exploratory factor analyses (Santos & Clegg, 1999), within and across states, showed that the 10 items loaded onto one overall "experience" factor. A one-factor solution explained 63% of the scale variance for youths, the scree plot indicated a one-factor solution, and the scale had excellent reliability (0.93). The 10 items together measure an overall "experience" in 4-H during the first year, despite items tapping different experiences (such as by asking about clubs, projects, and relationships with others). This is a useful scale that can be used to assess an overall sense of feelings toward a program.
| Item | Factor loading—Youths | M (SD) |
| 1. Adults who worked with me were caring. | 0.73 | 4.58 (0.87) |
| 2. Club leaders helped me figure out what was going on in the club. | 0.83 | 4.08 (1.26) |
| 3. My 4-H club made me feel important. | 0.84 | 4.00 (1.24) |
| 4. My project leader(s) were helpful. | 0.83 | 4.20 (1.21) |
| 5. I made new friends. | 0.66 | 3.99 (1.27) |
| 6. I learned new things in my 4-H project(s). | 0.78 | 4.43 (1.01) |
| 7. My project(s) were interesting. | 0.76 | 4.37 (1.02) |
| 8. I enjoyed my club meetings. | 0.80 | 3.93 (1.30) |
| 9. Adults and other members listened to what I had to say. | 0.80 | 4.13 (1.10) |
| 10. I had the opportunity to serve my community. | 0.65 | 4.16 (1.12) |
| 10-item scale | 4.20 (0.90) | |
| Note. Each question had missing data; the N across questions ranged from 630 to 672. The N for the scale was 568. | ||
Finally, youths reported on reasons why they might leave the program. We used a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Mean responses were highest for not feeling connected to other youths, not having time, and not knowing when the club or project met. Table 3 provides the means and standard deviations for these questions.
| Item | M (SD) |
| 1. I was bullied by other youth in my club or project. | 1.14 (0.60) |
| 2. I was bullied by adults in my club or project. | 1.14 (0.63) |
| 3. I didn't feel connected to other youth in the club. | 1.97 (1.32) |
| 4. I didn't feel welcomed in my club. | 1.56 (1.09) |
| 5. 4-H isn't for kids like me. | 1.21 (0.64) |
| 6. Parents in 4-H don't get along. | 1.30 (0.82) |
| 7. I don't have time for 4-H because of my other activities (like sports, church, Boy Scouts, Girl Scouts, FFA). | 1.92 (1.23) |
| 8. 4-H is too expensive. | 1.47 (0.95) |
| 9. I didn't know when my club or project met. | 1.62 (1.20) |
| Note. Each question had missing data; the N across questions ranged from 659 to 668. The N for all items was 628. | |
Lessons Learned
Through implementing the survey and from analyzing the data, we learned several lessons that are summarized in Table 4.
| Lesson learned | Implication for 4-H |
| Implementing the survey | |
| We did not want to burden people who had already completed the survey with reminders. We used the Qualtrics "Contacts" feature to send out the initial email with the survey. | This helped us track the survey distribution. |
| Paper surveys are needed for families without Internet access or for whom email addresses are missing. | This will increase the response and help us get a better response rate. |
| We advertised the survey often and broadly (e.g., newsletters, meetings, events). | This helped increased participation, showed how the data will help the program, and shared previous findings. |
| Analyzing the survey results | |
| Results from the survey can be useful in planning marketing and recruitment activities. | Learning how most people hear about 4-H may help direct resources to more effective means. |
| Data from the survey can be used to inform procedures and activities to help improve youth programs and increase retention. | States have used data from the first 3 years to develop a handbook for new families to help them navigate 4-H. |
| Youths and families express reasons for dissatisfaction with the program that our team had not previously identified. | Information from the survey may also inform the development of an exit survey for families who do not plan to return. |
| The collection of qualitative data supplements the collection of quantitative data by allowing participants to expand on their experiences. | The analyzed qualitative data can be used to help identify areas of improvement and resources that are needed. |
Conclusions
The survey we developed provides useful information about the 4-H program and aids understanding of the challenges new members and families may face. By better understanding the perspectives of first-year families, 4-H can develop materials and means to support participants and their retention in the program. The 10-item experience scale provides an overall indicator of one's experience in 4-H. Other Extension programs may choose to adopt or adapt this survey tool to get similar perspectives about their programs in order to make informed programmatic changes. The survey tool can be accessed be contacting the first author.
References
Astroth, K. A. (1985). The challenge of retaining 4-H members. Journal of Extension, 23(3), Article 3FEA4. Available at: http://www.joe.org/joe/1985fall/sa4.php
Defore, A., Fuhrman, N. E., Peake, J. B., & Duncan, D. W. (2011). Factors influencing 4-H club enrollment and retention in Georgia. Journal of Youth Development, 6(2). Retrieved from http://www.nae4ha.com/assets/documents/JYDfinal0602.pdf
Eccles, J., & Gootman, J. A. (2002). Community programs to promote youth development. Washington, DC: The National Academies Press.
Hamilton, S. F., Northern, A., & Neff, R. (2014). Strengthening 4-H by analyzing enrollment data. Journal of Extension, 52(3), Article 3FEA7. Available at: http://www.joe.org/joe/2014june/a7.php
Harder, A., Lamm, A., Lamm, D., Rose, H., & Rask, G. (2005). An in-depth look at 4-H enrollment and retention. Journal of Extension, 43(5), Article 5RIB4. Available at: http://www.joe.org/joe/2005october/rb4p.shtml
Hartley, R. S. (1983). Keeping 4-H members. Journal of Extension, 21(4). Available at: http://www.joe.org/joe/1983july/a4.php
Lerner, R. M., & Lerner, J. V. (2013). The positive development of youth: Comprehensive findings from the 4-H Study of Positive Youth Development. Chevy Chase, MD: National 4-H Council. Retrieved from http://www.4-h.org/About-4-H/Research/PYD-Wave-9-2013.dwn
Lewis, K. M., Horrillo, S. J., Worker, S. M., Miller, J., & Trzesniewski, K. (2015). Retaining youth: An examination of California 4-H youth enrollment trends. Paper presented at the American Evaluation Association, Chicago, IL.
Pratt, C., & Bowman, S. (2008). Principles of effective behavior change: Application to Extension family educational programming. Journal of Extension, 46(5), Article 5FEA2. Available at: http://www.joe.org/joe/2008october/a2.php
Ritchie, R. M., & Resler, K. M. (1993). Why youth drop out of 4-H. Journal of Extension, 31(1), Article 1RIB3. Available at: http://www.joe.org/joe/1993spring/rb3.php
Santos, J. R. A., & Clegg, M. D. (1999). Factor analysis adds new dimension to Extension surveys. Journal of Extension, 37(5), Article 5RIB6. Available at: http://www.joe.org/joe/1999october/rb6.php
U.S. Department of Agriculture. (2010). Organized 4-H Clubs REEIS Report. Retrieved from https://reeis.usda.gov/content/organized-4-h-clubs