Reliability Analysis of the Adult Mentoring Assessment for Extension Professionals

Introduction

Prior to 2015, there existed no formal tool with which to assess the quality of Extension educators' mentoring of other Extension educators. I developed the Adult Mentoring Assessment for Extension Professionals to help mentors of adult learners develop accurate profiles of their actual mentoring style (for those with prior experience) or probable mentoring style (for those with little or no prior experience).

The assessment complements a formal Mississippi State University Extension Service training program that was also developed in 2015 and prepares veteran Extension agents to successfully mentor newly hired agents for the first 12 to 18 months of employment. Both the assessment and the training program are based on the theoretical foundation of transformative learning, with the premise that Extension is a learning organization and its personnel are adult learners (Denny, 2016). The training focuses on (a) understanding the characteristics and motivations of adult learners and (b) applying transformative learning theory in the context of mentoring adult learners. This theoretical application is realized in the form of six constructs related to behaviors of mentors of adult learners: relationship, information, facilitation, confrontation, modeling, and vision.

The Adult Mentoring Assessment for Extension Professionals allows users to determine the efficacy of their demonstration of the aforementioned six constructs on the basis of the frequency with which they engage in behaviors related to each when serving in a mentor role. The scores on the various constructs help mentors identify specific proficiencies to maintain or strengthen and deficiencies to overcome. The overall score on the assessment is a general indicator of competence as a mentor of adult learners. By completing the assessment prior to starting a formal mentoring relationship, mentors should have a better understanding of how they can best contribute to the success of their mentees.

Instrument and Sample

The Adult Mentoring Assessment for Extension Professionals was adapted from an assessment developed by Norman Cohen (1995) for mentors in general workplace settings. Whereas the six constructs remained the same, the total number of statements was reduced to 38, the wordings of the statements were adjusted to better fit the culture and nature of Extension as an organization, and the response scale was reduced from 5 points to 4. Evaluation specialists at Mississippi State University assessed the modified instrument for content validity. Additionally, the instrument underwent reliability testing because the total number of statements per construct decreased from the original instrument.

Following institutional review board approval, the assessment was distributed via Qualtrics to all district/regional Extension directors for the southern region land-grant universities. The directors were instructed to identify agents who were currently mentors or had the potential to be effective mentors on the basis of a list of criteria I provided and to request that those individuals complete the assessment online.

Between December 2015 and February 2016, 135 assessments were submitted, and 97 of those were complete. Respondents represented 10 land-grant institutions in Alabama, Arkansas, Florida, Georgia, Louisiana, Mississippi, Tennessee, and Texas. Participants responded to 38 behavioral statements by indicating the frequency—never, seldom, sometimes, or often—with which they engaged in a behavior when serving in the role of mentor (formally or informally).

Analysis and Results

Reliability is a concern when a tool is used to measure behavior (Rosenthal & Rosnow, 1991). I sought to measure the reliability of the six constructs based on the 38 statements of mentor behaviors. I analyzed each construct for internal consistency using Cronbach's alpha procedures (Drost, 2011).

All six constructs passed the minimum standard for reliability that is acceptable in the social sciences (Cortina, 1993). Coefficient alphas for the six constructs range from .71 to .88, as shown in Table 1.

Conclusion

It is my hope that other Extension systems and similar organizations that use formal mentors will see the value in this tool for training and development purposes. Users of the tool can now be confident in its reliability and validity. Persons interested in obtaining the assessment may contact me via email (mdd269@msstate.edu).

References

Cohen, N. H. (1995). Mentoring adult learners: A guide for educators and trainers. Malabar, FL: Krieger Publishing Company.

Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98–104.

Denny, M. D. (2016). Mentoring adult learners: Implications for Cooperative Extension as a learning organization. Journal of Extension, 54(3), Article 3FEA2. Available at: http://www.joe.org/joe/2016june/a2.php

Drost, E. A. (2011). Validity and reliability in social science research. Education Research and Perspectives, 38(1), 105–123.

Rosenthal, R., & Rosnow, R. L. (1991). Essentials of behavioral research: Methods and data analysis (2nd ed.). New York, NY: McGraw-Hill Publishing Company.