Extension Educators' Views of Scholarship and Performance Evaluation Criteria

Introduction

The core values of land-grant universities are historically based on research and outreach. Enhancing scholarly activity within Extension is one way to bolster the land-grant mission while continuing its legacy. Boyer (1990) considers scholarship as having four distinct yet overlapping functions: discovery, integration, application, and teaching. In this view, scholarship is more than engaging in original research. It is also stepping back from a study to search for relationships, build connections between theory and practice, and effectively communicate new knowledge to others. Boyer's depiction of scholarship may require scholars to redefine their current view of scholarship.

To ensure that this contemporary definition of scholarship would be taken seriously, Boyer (1990) challenged scholars to develop standards and evaluation approaches for scholarly work. This challenge was addressed by Glassick, Huber, and Maeroff (1997), who assert that guidelines for scholarship evaluation need to be developed, clarified, and understood among colleagues. In other words, there needs to be a shared understanding of scholarship among colleagues and a clear process for evaluating scholarly work. This understanding needs to reflect contemporary views as well as the core values of the institution. From that base, annual review, promotion evaluations, and scholarship assessment reform are possible.

The faculty senate at Oregon State University (OSU) undertook the challenge to define and articulate characteristics of scholarship that apply across academic disciplines and department missions to provide a theoretical foundation for reviewing and revising tenure and promotion guidelines (Weiser & Houglum, 1998). While Boyer's (1990) work influenced its development, the OSU model has a stronger emphasis on outputs to validate scholarship. For instance, OSU's promotion and tenure criteria assess the extent to which scholarly achievement is "original, significant, and useful to others" (Weiser & Houglum, 1998, p. 3).

The University of Wisconsin-Extension applied Boyer's (1990) broader thinking of scholarship (also see Lynton & Elman, 1987; Lynton, 1995) to promotion and tenure criteria in the hope that the scholarship documentation would better match the probationary faculty's portfolio development (Wise, Retzleff, & Reilly, 2002). As a part of that effort, outreach was described as "a particular and distinct form of scholarly activity deeply embedded in the University's mission to create, integrate, transfer, and apply knowledge" (University of Wisconsin-Madison Council on Outreach, 1997, p. 35). University of Wisconsin-Extension faculty members also adopted the following revised definition of scholarship: "creative intellectual work; reviewed by the scholar's peers who affirm its value; added to our intellectual history through its communication; and valued by those for whom it was intended" (UWEX Articles of Faculty Governance, Appendix I.B. 2001).

One of the core values of the University of Minnesota Extension Service (herein referred to as Extension) is that "Scholarship and research guide our educational programs" (Casey, Morse, & Markell, 2004, p. 28). The organization has a historic legacy of research-based programming and audience engagement. To become a more responsive organization with higher quality programs that have better documented impact, a task force was formed in 2005 to develop a shared understanding of scholarship in relation to promotion standards and assessment tools. This article reports the results of a survey of Regional Extension Educators (REEs) within Extension that solicited thoughts on their definition of scholarship, extent of its use in their everyday work, and its importance in performance evaluation. Documenting this information was critical to the success of bolstering our organization's ability to enhance scholarship.

Methods

A survey containing both categorical and open-ended questions was developed and conducted on-line using Survey Monkey (Survey Monkey) from September 7 - 16, 2005. The survey included the following sections: demographics, defining scholarship, current use of scholarship within Extension, how scholarship was related to staff and faculty's work, and scholarship and performance evaluation. The development of some survey questions was influenced by Boyer's (1990) model of scholarship. Scholarship was not defined for study participants either before or within the survey.

The analysis presented here focuses on how scholarship was currently defined and conducted by REEs, as well as their perspectives about the importance of various factors that may influence performance evaluation. REEs, who have statewide responsibility, are the largest group of faculty within the University of Minnesota Extension Service. REEs hold professional and academic positions with academic ranks (i.e., Assistant Extension Professor, Associate Extension Professor, and Extension Professor) without tenure. Administratively, they are organized into five Capacity Areas based on broad subject matter areas: Agriculture, Food, and Environment; Community Vitality; Family Development; Natural Resources and Environment; and Youth Development.

REEs are the focus of this article because they are the only group within Extension that is subject to new performance evaluation and promotion guidelines that add scholarship as one of the primary criteria. (Extension also includes local Extension educators as well as administrative, civil service, and other professional and academic staff members but they are not included in this analysis since they are not subject to these new guidelines.)

Survey responses were analyzed using SAS (SAS Institute). Fisher exact tests were used to evaluate independence of responses in cross-tabulations to test the strengths of relationships between responses to survey questions (SAS Institute; Rosner, 1995). Likert scores were averaged and used to rank responses within questions and t-tests were used to determine and compare the strength of answers.

Results

Ninety-one (91) useable surveys were completed by REEs in five Capacity Areas (Table 1). These 91 REEs represented 75% of the potential total number of REEs in these five capacity areas at the time of the survey. These REEs had worked in Extension an average of 14.6 years; the minimum was 0.25 years and the maximum was 34 years. They all had 100% Extension appointments.

Definition of Scholarship

When asked to provide open-ended comments about what came to their mind when thinking about scholarship as it related to their work, common themes were that it 1) is a research-based or discipline-based approach that grounds the foundation of our work, 2) is a focused area of study or research, 3) is a process (e.g. research, programmatic, teaching), 4) provides valued results (e.g. published work, public), and 5) is a field of study.

The REEs indicated that being valued by the intended audience was the most essential criteria for determining whether Extension work is scholarship (Table 2). The response average was significantly (p<0.05) higher than all other averages. However, other criteria also received high rankings: communicated to others, contributes to a body of knowledge, and creative intellectual work. Review by peers was the lowest ranked criteria. This lowest ranking is reinforced by the fact that when tested with a t-test, its average score (3.40) was significantly (p<0.05) lower than the other criteria's average scores.

Even though scholarship was not defined prior to or within the survey, the REEs overwhelmingly agreed with Boyer's (1990) classification of scholarship as discovery, integration, application, and teaching (Table 3).

Use of Scholarship in Extension Work

On average, REEs said that 29% of their work was currently dedicated to scholarship; the median was 20%. Not all capacity areas had similar levels of scholarship. However, they thought that 37% of their work should be dedicated to scholarship; the median was 25%. Fisher's exact test shows a strong relationship (p<0.0000) between the current percent of time dedicated to scholarship and the amount of time that these respondents felt should be dedicated to this endeavor (Table 4). Respondents who spent less than 30% of their time currently dedicated to scholarship were most strongly supportive of increasing their role in this area. For example, of the 27 REEs who indicated they currently dedicate 0-10% of their time to scholarship, 16 indicated they should increase their time on scholarship.

In terms of the role of scholarship within their Extension work, the statement with the highest response average was that REEs used others' scholarship in their work (Table 5). Being aware of scholarship in their field and sharing their scholarship with intended audiences were also frequently cited. The response average was significantly greater than "Some" application (p<0.001) for these three criteria. Contributing and participating in scholarship, sharing scholarship with peers, and, generating scholarship were less frequently cited by the REEs.

REEs were most strong in their belief that improving scholarship was important to the sustainability of Extension (Table 6). They overwhelming indicated that scholarship should be expected of all those with academic rank (i.e., Assistant Professor, Associate Professor, Professor) and of themselves. They were not as strong in their agreement that scholarship should be expected of people in local educator positions; the average response for this statement, while significant by itself, was significantly (p<0.05) lower than for other statements.

When asked through an open-ended question about the barriers that hinder their ability to increase their amount of scholarship, the following themes emerged from REE responses: 1) time constraints (i.e., too much paperwork and travel, would take away from program delivery), 2) lack of money (i.e., to attend professional meetings, to do the necessary research), and 3) lack of incentives, support, and direction (i.e., Extension culture does not reward scholarship, supervisor does not support scholarship, need practical guidance from supervisors). It is important to note that time constraints was the dominant theme depicting the types of barriers followed by the other concerns that may impede one's ability to increase levels of scholarship.

The Relationship of Scholarship to Performance Evaluation

The REEs selected several factors that should have significant or great influence on (or even dominate) their performance evaluation within Extension: program development, program evaluation, program management and delivery, scholarship, service to Extension committees and work teams, and service to field or discipline (Table 7). While scholarship was one of those factors, it was not the most important one noted. Program management and delivery received a higher response average and was significantly greater (p<0.001) than the average for scholarship. Program development was also considered more important than scholarship (p<0.05). The importance of scholarship was not significantly different from program evaluation and service to the field or discipline. Service to Extension committees and work committees and service to community had statistically equal scores (p<0.10). Revenue generation and seniority, years of service, or academic rank were not ranked highly by the REEs for influencing or impacting their performance evaluations.

Conclusions

Regional Extension Educators overwhelmingly agreed with statements about scholarship in each of the four categories described by Boyer (1990): discovery, integration, application, and teaching. This broad conceptualization of scholarship also meshed with themes that emerged from open-ended comments about what came to their mind when thinking about scholarship as it related to their work.

Even though scholarship was not formally defined in the survey or its introductory note, REEs thought they were engaged in scholarship although their level of engagement was not uniform across capacity areas. Although they were very strong in their belief that improving their scholarship was important to the sustainability of Extension, they did not think they should increase their scholarship work to the exclusion of other work. In fact, the importance of program management and delivery may be a barrier to increasing the amount of scholarship generated by REEs. An organizational challenge may lie in bridging program and scholarship in everyday work so that both areas receive adequate investment.

Furthermore, it is important to note that REEs indicated that time constraints, lack of money, and lack of organizational support and structure were other significant barriers. These are important considerations that may require further exploration in order to build a culture that supports Extension scholarship. Their most important factors for determining whether Extension work is scholarship are whether the work is valued by the intended audience and it contributes to a body of knowledge. These factors reflect the core values of outreach and research found in land-grant universities and provide a foundation for enhancing Extension's scholarly activity.

Interestingly, the REEs indicated that while it was important, scholarship was not the most important factor for influencing or impacting their performance evaluations. The most important factors were program management and delivery and program development. So even with the increased emphasis on scholarship, it was not seen as important as the long-established Extension activities of program management, delivery, and development.

Developing a shared understanding of scholarship through ongoing support by administration, developing and communicating a clear process of evaluating scholarly work, and aligning that effort with performance evaluation and demonstrating it as an important priority in performance evaluation are important first steps toward enhancing scholarly activity. Last, if Extension wants to increase the importance of scholarship in REEs' work and performance evaluations, these results show the need to spend considerable time building a culture that has greater support, removing barriers, and providing both monetary and non-monetary reward systems for scholarship.

References

Boyer, E. L. (1990). Scholarship reconsidered: Priorities of the professoriate. Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching.

Casey, C. H., Morse, G. W., & Markell, J. (2004). Strategic planning framing concepts. St. Paul, MN: University of Minnesota Extension Service. Available at: http://www.extension.umn.edu/jump/compact05/summary.html

Glassick, C. E., Huber, M. T., & Maeroff, G. I. (1997). Scholarship assessed: Evaluation of the professoriate. A Special Report to the Carnegie Foundation for the Advancement of Teaching. San Francisco, CA: Jossey-Bass Publishers.

Lynton, E. A. (1995). Making the case for the professional service. Washington DC: American Association for Higher Education.

Lynton, E. A., & Elman, S. E. (1987). New priorities for the University: Meeting society's needs for applied knowledge and competent individuals. San Francisco, CA: Jossey-Bass Publishers.

Rosner, B. (1995). Fundamentals of biostatistics. (4th ed.). Belmont, CA: Duxbury Press.

SAS Institute Inc, (c) 2002-2003. SAS 9.1, Cary, NC.

Survey Monkey. Available at: http://www.surveymonkey.com

University of Wisconsin-Extension Articles of Faculty Governance. Appendix I.B, Criteria for Faculty appointment and Promotion in UW-Extension. (2001). Retrieved March 20, 2006, from http://www1.uwex.edu/secretary/policies/section8/

University of Wisconsin-Madison, Council on Outreach. (1997). Commitment to the Wisconsin idea: A guide to documenting and evaluating excellence in outreach scholarship. Madison, WI: Office of Outreach Development, University of Wisconsin-Madison.

Weiser, C. H., & Houglum, L. (1998). Scholarship unbound for the 21st century. Journal of Extension [On-line], 36(4). Available at: http://www.joe.org/joe/1998august/a1.html.

Wise, G., Retzleff, D., & Reilly, K. (2002). Adapting scholarship reconsiders and scholarship assessed to evaluate University of Wisconsin-Extension outreach faculty for tenure and promotion. Journal of Higher Education Outreach and Engagement, 7(3):5-17.