Evaluating the Mentor-Mentee Relationship in the 4-H Tech Wizards Program

Introduction

Volunteer mentors have immensely significant responsibilities as role models, coaches, and friends for youth. A growing number of evaluations provide confirmation that volunteer mentoring programs positively influence a range of youth outcomes, including improved peer and parental relationships, academic achievement, self-concept, and behavior (DuBois, Portillo, Rhodes, Silverthorn, & Valentine, 2011; Grossman & Bulle, 2006; Rhodes, Reddy, Roffman, & Grossman, 2005). Similar in structure to other youth development programs, mentorship programs are characterized by: a close relationship with a caring adult; supervision; life skill development; academic tutoring; and community service. Mentorship programs provide emotional support, advice, and guidance about subjects that youth might feel uncomfortable, apprehensive, or fearful discussing with their parents (DuBois, Holloway, Valentine, & Cooper, 2002; Grossman & Bulle, 2006). Positive relationships with caring adults, which are associated with promoting resiliency among youth, are a key element of youth development programs in Extension (Astroth, 2014; Coller & Kuo, 2014; & Riggs, Marshall, Serfustini, & Bunnell, 2006).

Mentorship programs target at-risk youth, who are often underserved by more traditional youth programs (Jekielek, Moore, & Hair, 2002). The core of the mentoring model is the mentor-mentee relationship. Several factors foster strong relationships between the mentors and youth. The most successful relationships are those in which the mentors and youth are engaged in friendship-oriented social activities. These rapport-building activities may include talking or having lunch together and setting goals with the youth (DuBois et al., 2002; Herrera, Vang & Gale, 2000). Traditionally, youth mentoring programs are based on a 1:1 mentor-mentee ratio. However, this leads to limitations in the number of youth that can benefit from a mentoring relationship because volunteers are a limited resource. Small-group mentoring programs have become effective solutions to leverage the impact of volunteers and engage more youth. Further, group mentoring programs foster increased diversity in both mentors and mentees (Herrera et al., 2002). In addition to the development of relationships with their mentors, these group scenarios allow friendships to be formed between peers, which is equally as beneficial to the youth.

The Tech Wizards program is a small group mentoring program that targets at-risk youth. Oftentimes, at-risk youth are underrepresented in the fields of Science, Technology, Engineering, and Math (STEM). The mentor-mentee match is based upon STEM interest of the mentor and mentees. Mentors, youth, and staff identify the STEM interests for the program. Youth self-select their STEM interest, and program staff work with mentors to match their interests with a small group of youth. The target mentor-mentee ratio for the program is 1:4. Mentor-Mentee groups within the Tech Wizards program meet weekly for 2 to 3 hours for 12-15 months. The number of small group mentoring programs has increased recently, and little research is readily available to evaluate the mentor-mentee relationship. The purpose of the study reported here was to evaluate the quality of the youth-mentor relationship in a small group mentoring program. The objectives of the study were to:

• Monitor the quality of individual youth-mentor relationships;
• Gain insight into the perceived strengths and weaknesses of the youth-mentor relationship; and
• Establish a benchmark for measuring changes in the quality of youth-mentor relationships over time.

Methods

The study reported here included youth participating in an afterschool mentoring program (n=107). The afterschool site was in a community-disadvantaged area where more than 70% of the U.S. population has a higher standard of living. Participants were between 12 and 16 years old and included 57 females and 50 males. Volunteer mentors were recruited from local communities, schools, and businesses. Each mentor completed the mandatory application, background screening, and orientation process. Mentors were assigned a group of youth based upon similar STEM interests. Finally, all volunteer mentors received a minimum of 6 hours of training. The study was approved by the Institutional Review Board (IRB) for behavior research involving humans.

Instrumentation

The Mentor-Youth Inventory survey was used in the study (Jucovy, 2002). This survey instrument was based upon the youth mentor relationship research of Rhodes, Reddy, Roffman, and Grossman (2005). The questionnaire included 19 questions that measured three domains of the youth-mentor relationship (Table 1). These domains include: the extent to which the relationship is centered on youth (YC); the youth's emotional engagement with the mentor (EE); and the extent to which the youth is dissatisfied with the mentor relationship (YD). To test whether the three domains of the mentor-mentee relationship were consistent, confirmatory factor analysis was used. Reliability for the constructs ranged from .74 to .85 (Rhodes et al., 2005).

Youth-Centered Relationships

Youth-centered relationships occur when youth feel their mentors are attuned to their opinions and preferences. Youth are more likely to show improvement in their behaviors and attitudes with engaged mentors than are youth who feel their mentors are less interested in them. The questionnaire included five questions to measure youth-centered relationships. The Cronbach's alpha for the five questions used to measure youth centered relationships was .81. This was consistent with Rhodes et al. (2005) and indicates a high degree of internal consistency for the items.

Emotionally Engaged Youth

Emotionally engaged youth feel better about being around their mentor and are more likely to show improvement in their behaviors and attitude than are youth who feel less positive about their mentor-mentee relationship. The questionnaire included eight questions to measure youth who are emotionally engaged. The Cronbach's alpha for the eight questions used to measure emotional engagement was .85. This was consistent with Rhodes et al. (2005) and indicates a high degree of internal consistency for the items.

Youth Dissatisfaction

Youth who are dissatisfied with their mentor are less likely to show improvement in their behaviors and attitudes compared to youth with more favorable impressions. The questionnaire included six questions to measure the level of dissatisfaction of youth participants. The Cronbach's alpha for the six questions used to measure youth dissatisfaction was .91. This was consistent with Rhodes et al. (2005) and indicates a high degree of internal consistency for the items.

Data Collection and Analysis

At the start of the mentoring program, mentors and youth mentees were provided an orientation related to the goals and objectives of the program. To assess the quality of the mentoring relationship, the researchers collected data at two different time points, approximately 12 months apart. Three months into the program (Time Period 1), youth were provided with a survey and given instructions to provide feedback related to their experience in the Tech Wizards program. Approximately 12 months after the first survey (Time Period 2), youth were provided with a survey and instructions to contribute feedback about their experience in the Tech Wizards program. This second survey was administered 1 week prior to the end of the program, as a conclusive study of the duration of the program.

Table 1.
The Youth Survey: Measuring the Quality of Mentor-Youth Relationships

Youth-Mentor Relationship Domain	Item #	Questions	Not True at all	Not Very True	Sort of True	Very True
YD	1	My mentor makes fun of me in ways I don't like.	①	②	③	④
YC	2	My mentor almost always asks me what I want to do.	①	②	③	④
EE	3	When I'm with my mentor, I feel special.	①	②	③	④
YD	4	Sometimes my mentor promises we will do something; then we don't do it.	①	②	③	④
YC	5	My mentor is always interested in what I want to do.	①	②	③	④
EE	6	When I'm with my mentor, I feel excited.	①	②	③	④
YD	7	When my mentor gives me advice, it makes me feel stupid.	①	②	③	④
YC	8	My mentor and I like to do a lot of the same things.	④	③	②	①
EE	9	When I'm with my mentor, I feel sad.
YD	10	I feel I can't trust my mentor with secrets—my mentor would tell my parent/guardian.	①	②	③	④
YC	11	My mentor thinks of fun and interesting things to do.	①	②	③	④
EE	12	When I'm with my mentor, I feel important.	①	②	③	④
EE	13	When I'm with my mentor, I feel bored.	④	③	②	①
YD	14	I wish my mentor asked me more about what I think.	①	②	③	④
YC	15	My mentor and I do things I really want to do.	①	②	③	④
EE	16	When I'm with my mentor, I feel mad.	④	③	②	①
YD	17	I wish my mentor knew me better.	①	②	③	④
EE	18	When I'm with my mentor, I feel disappointed.	④	③	②	①
EE	19	When I'm with my mentor, I feel happy.	①	②	③	④

The mean for each domain was calculated by adding the responses to the questions associated with the domain and then dividing this sum by the total number of questions. For example, the domain for Youth-Centered Relationships added the responses to questions 2, 5, 8,11, and 15 and then divided by five. Standard deviations were calculated for the mean of each domain. A range of scores was also calculated. A paired t-test, including the effect size, was used to compare the two time periods for the extent to which the relationship is centered on youth (YC); the youth's emotional engagement with the mentor (EE); and the extent to which the youth is dissatisfied with the mentor relationship (YD).

Results

Over the course of the study, the range of responses for each of the domains showed that youth indicated the program was more youth centered; youth were more emotionally engaged with their mentor; and less dissatisfied with their mentor (Table 2). Of particular interest is the finding that the number of youth reporting the lowest level on each scale significantly improved from Time Period 1 to Time Period 2. For example, the number of youth reporting they were high dissatisfied improved from 35 youth (32.7%) in Time Period 1 to only 9 youth (8.4%) in Time Period 2.

Table 2.
Range of Individual Responses in Measuring the Quality of Mentoring Relationships

	Time Period 1		Time Period 2
	# Youth	Percent	# Youth	Percent
Youth Centered
Very Youth Centered	10	9.3%	10	9.3%
Moderately Youth Centered	22	20.6%	61	57.0%
Not Youth Centered	75	70.1%	36	33.6%
Emotionally Engaged
Highly Engaged	6	5.6%	45	43.0%
Moderately Engaged	27	25.2%	26	24.3%
Not Engaged	74	69.2%	15	14.0%
Dissatisfied with Mentor
Highly Satisfied
Moderately Satisfied	16	15.0%	33	30.8%
Dissatisfied	35	32.7%	9	8.4%

The findings showed that at Time Period 1, youth reported that the mentoring program was not very youth centered (M = 2.52, SD =.73) and not emotionally engaging (M = 2.34, SD =.70), and youth were not particularly satisfied with the mentor relationship (M = 2.06, SD =.69). At Time Period 2, youth reported that the mentoring program had become more youth-centered (M = 3.01, SD =.37); more emotionally engaging (M = 3.07, SD =.39), and they were less dissatisfied with their mentor relationship (M = 1.65, SD =.42) (Table 3). A dependent t-test for paired variables compared the mean differences in the three domains in Time Period 1 and Time Period 2 and established that the results were statistically significant (Table 3). Furthermore, the effect size was determined for each of the domains. For Youth-Centered programs, Cohen's effect size value (d = .69) suggested a moderate to high practical significance. For Emotionally Engaged youth, Cohen's effect size value (d = .54) suggested a moderate to high practical significance. For Dissatisfaction with Mentor, Cohen's effect size value (d = .34) suggested a small to moderate practical significance (Table 3).

Table 3.
Contrast of Time Period 1 with Time Period 2 for Measuring the Quality of Mentoring Relationships

	Time Period 1		Time Period 2
	M	SD	M	SD	t(106)	p	Eta
Youth Centereda	2.52	0.73	3.01	0.37	7.12	<.001	0.69
Emotionally Engagedb	2.34	0.70	3.07	0.39	37.59	<.001	0.54
Dissatisfied with Mentorc	2.06	0.69	1.65	0.42	-5.52	<.001	0.34
a 1.0 – 2.99 (not youth-centered) 3.00 – 3.49 (moderately youth-centered) >3.49 (youth-centered) b 1.0 – 2.99 (not very engaged) 3.00 – 3.49 (moderately engaged) >3.49 (highly engaged) c 1.0 to 1.49 (highly satisfied) 1.5-2.49 (not satisfied\dissatisfied) 2.5 or higher (highly dissatisfied)

Discussion

Rhodes et al. (2005) suggested that the success of a mentoring program is measured by the development of a positive relationship between youth and a mentor. Consistent with Rhodes et al. (2005), the youth-mentor relationship in the mentoring program developed over time. Within the Tech Wizards program, the degree to which the youth-mentor relationship became more youth centered improved from Time Period 1 to Time Period 2 by 19% from (m= 2.52) to (m= 3.01). Initially, youth mentees expressed that they were not very engaged with their mentor or program (m=2.34). Engagement improved by 31% (m=3.07), providing further evidence that mentoring relationships evolve over time. Further, dissatisfaction with a young person's mentor decreased by 25% over the duration of the program from (m=2.06) to (m= 1.65). The researchers acknowledge that the research study did not employ a control group. In the researchers' opinion, having a control group would have had a negative impact on the youth-mentor relationship. As such, the lack of a control group may be a limitation of the study.

The study was designed to measure the development of the mentoring relationship. The user-friendly instrument, the Mentor-Youth Inventory, provided an opportunity for the Tech Wizards group mentoring program to assess and understand the mentor-youth relationship. In each domain, higher numbers of youth reported that the program was more youth centered, they were more emotionally engaged with their mentor, and they were less dissatisfied with their mentor by the end of the program. This is an important factor for the retention of youth in the 4-H Tech Wizards program. DuBois et al. (2011) stressed the importance of ensuring program quality and effectiveness. In the study reported here we were able to identify the nature of the mentoring relationship from a youth perspective. Areas of improvement were also identified through this process. The ability of a program to monitor progress is important for continuous quality improvement.

Implications

Other professionals could benefit from the strategies used in the program. Creating a youth-centered program is contingent upon well-defined roles and responsibilities. Decreasing dissatisfaction with a mentor requires building trust between a youth and a mentor. Maintaining a sense of belonging is necessary for increasing emotional engagement by youth. None of this, however, could have been discovered without measuring and benchmarking the nature of the youth-mentor relationship in the beginning. By taking the initiative to examine the development of youth-mentor relationships over time, those responsible for mentoring programs will be well positioned for success.

The study reported here showed that the Mentor-Youth Inventory survey instrument provided the researchers the ability to establish a benchmark and monitor youth-mentor relationships. This is important and necessary for the future advancement of mentoring programs. Enhanced efficacy of mentorship programs could translate into the revitalization of countless communities and the cultivation of more confident and motivated youth.

Acknowledgments

Funding for Florida 4-H Tech Wizards is a collaborative partnership with National 4-H Council, Office of Juvenile Justice Delinquency Prevention, and University of Florida IFAS Cooperative Extension (4HNMP2). Correspondence should be addressed to Dr. Bryan D. Terry, University of Florida, PO Box 110310, Gainesville, FL 32611-0310; email: terrys1@ufl.edu

References

Astroth, K. A. (2014). Interdependence: Ninth and newest critical element for 4-H positive youth development. Journal of Extension [On-line], 52(6) Article 6COM2. Available at: http://www.joe.org/joe/2014december/comm2.php

Coller, R. J., & Kuo, A. A. (2014). Youth development through mentorship: A Los Angeles school-based mentorship program among Latino children. Journal of community health, 39(2), 316-321.

DuBois, D., Holloway, B., Valentine, J., & Cooper, H. (2002). Effectiveness of mentoring programs for youth: A meta-analytic review. American Journal of Community Psychology, 30(2), 157-197.

DuBois, D. L., Portillo, N., Rhodes, J. E., Silverthorn, N., & Valentine, J. C. (2011). How effective are mentoring programs for youth? A systematic assessment of the evidence. Psychological Science in the Public Interest, 12(2), 57-91.

Grossman, J., & Bulle, M. (2006). Review of what youth programs do to increase the connectedness of youth with adults. Journal of Adolescent Health, 6, 788-799.

Herrera, C., Vang, Z., & Gale, L. Y. (2002). Group mentoring: A study of mentoring groups in three programs. Philadelphia, PA: Public/Private Ventures.

Jekielek, S., Moore, K. A., & Hair, E. C. (2002). Mentoring programs and youth development: A synthesis. Washington, D.C.: Child Trends.

Jucovy, L. (2002). Measuring the quality of mentor-youth relationships: A tool for mentoring programs. Philadelphia: Public/Private Ventures.

Reddy, R., Roffman, J. G., Grossman, J. B., & Rhodes, J. E. (2001). The development and validation of a youth mentoring relationship inventory. Boston, MA: University of Massachusetts.

Riggs, K., Lee, T., Marshall, J. P., Serfustini, E., & Bunnell, J. (2006). Mentoring: A promising approach for involving at-risk youth in 4-H. Journal of Extension [Online], 44(3) Article 3FEA5. Available at: http://www.joe.org/joe/2006june/a5.php

Rhodes, J., Reddy, R., Roffman, J., & Grossman, J. B. (2005). Promoting successful youth mentoring relationships: A preliminary screening questionnaire. Journal of Primary Prevention, 26(2), 147-167.