Tennessee Extension Agents' Perceptions of Performance Appraisal

Introduction

Very few studies of Extension agent perceptions of their performance appraisal system have been completed. Performance appraisal has received only a peripheral glance, being mentioned briefly in studies of job satisfaction, management, and employee retention. Clearly, more understanding is needed. A search of 46 years (1963 – 2009) of issues of the Journal of Extension found only eight studies of performance appraisal systems (Davis & Verma, 1993; Heckel, 1978; Kuchinke, Correthers & Cecil, 2008; Patterson, 1987; Peterson & McDonald, 2009; Rice, 2001; Terry & Israel, 2004; Zoller & Safrit, 1999). Of these eight studies, only one (Davis & Verma, 1993) was exclusively from the agent's viewpoint.

Davis and Verma (1993) compared Extension agents' views of their numeric performance appraisal to the agents' perception of the ideal performance appraisal system in a seven-state study of 602 agents. The study found that agents perceive the ideal performance appraisal as one in which their appraisers had adequate instruction and plan of work (or personal job objective) incorporated into the appraisal.

Research has suggested that agents prefer a performance appraisal system in which a team of appraisers is used rather than a single appraiser (Davis & Verma, 1993; Zoller & Safrit, 1999). Davis and Verma (1993) suggested that the trio of appraisers be the county directors, regional director, and state/regional specialist.

Performance Appraisal Process

In Tennessee, both the University of Tennessee (UT) and Tennessee State University (TSU) use the same appraisal system for Extension agent and county director positions. The performance appraisal system includes a rubric delineating 27 criteria in the broad categories of program development, program accountability and professionalism (Figure 1). County directors have an additional category of nine criteria describing administrative performance, such as guidance of personnel and financial management (Figure 1). For each criterion, except those relative to program development, the rubric delineates performance as exemplary (E), achieves expectations (A), and unsatisfactory (U). The ratings on the program development section are either achieves expectations (A) or unsatisfactory (U), with the exemplary rating not used.

Figure 1.
Tennessee Extension Agent Performance Appraisal Criteria

Program Development Needs Assessment Communicates Program Issues Networking Funding Appropriate Delivery Methods Clear Implementation Steps Plans for Evaluation (Tools/Methods) Reaches Diverse Audience/CR Parity Outcomes/Impacts Clearly Defined Program Accountability Impacts/Outcomes/Goals from Plan Effectively Marketed Programs Teaching Method Effectiveness Utilized Planned Evaluation Methods Reported Program Progress Reported Accomplishments Networking Utilized Available Funding Sources Civil Rights Parity and Diversity

Professionalism Effective Internal Relationships Effective External Relationships Trained in High Priority Program Areas Seeks Appropriate Professional Development Work Habits Knows/Follows Policy and Procedure Oral and Written Communication Effective Teaching Skills Markets UT/TSU Extension County Directors Total County Program Leadership Guidance of Personnel Evaluation of Personnel Financial Management Management of Facilities/Equipment Public Relations Communications/Office Management Knows Policies/Meets Deadlines EEO/CR/AA/ADA Policy and Implementation

In November, the Extension agent prepares materials to demonstrate competence and performance in the criteria during the past 11 months. The Extension agent may attach any number of documents or any amount of text to his/her electronic appraisal form. The appraisal form is due to the county director on or about December 1 of every year. The county director completes the rating form in a one-on-one meeting with the Extension agent. The county director then forwards the rating form and materials prepared by the Extension agent to the regional director, who makes the final rating in one-on-one consultation with the county director. The exception is that every third year, the regional director meets directly with the Extension agent (Byrd, 2009).

Purpose/Research Questions

The purpose of the study reported here was to determine Extension agents' perceptions of their performance appraisal system. Specific research questions were:  

1. What are the Extension agents' perceptions of the current performance appraisal system?
2. What are the Extension agents' perceptions of any needed improvements to the performance appraisal system?

Methods

The research reported here used a survey instrument created by Donaldson (2011) that measured the Extension agents' perceptions of the current appraisal system. The instrument employed Likert-type questions to measure these constructs, reflecting that the study is evaluative in scope and purpose (Colton & Covert, 2007). The items used six response categories: strongly disagree, disagree, neither agree nor disagree, agree, strongly agree, and don't know. The instrument and research questions were reviewed by an expert panel of eight state and regional Extension administrators who validated the instrument's face validity and found that the instrument supported the research questions.

The population of Extension agents employed by the UT Extension and TSU Extension in January 2010 was 312. A simple random sample of 39 agents (12% of the population) was drawn for the pilot test. The pilot test survey was available for 4 weeks in early 2010. Of the 39 pilot test members, 28 completed the survey, for a 72% response rate. The researcher noted no problems in survey completion. The responses to the question "Is there anything else you would like to share about the performance appraisal system, please type it in the box below" were analyzed. Because no common statements were made by the pilot test respondents, the comments were not incorporated into the survey items.

The instrument was sent to the study population (with pilot test members omitted) in spring 2010. It was available for 4 weeks. The data were combined into a single data file for analysis. The total completed responses were 218 for a completed response rate of 69%.

The data set was constructed and analyzed using the Statistical Package for the Social Sciences (SPSS, 2008). The researcher ignored missing values through pairwise deletion, deleting only missing values for a case rather than excluding the case from the entire data set. Descriptive statistics used were mean, mode, range, and percentage. Because the study was a survey of a population, inferential statistics were not used for data analysis.

Findings

The average years of employment for respondents in the study were 16.7 years, with a range of less than 1 year to 38 years; one in five respondents (20.9%) had worked 5 years or less. The mode was 2 years of employment for 15 (6.9%) respondents. Regarding institutional status, 202 (93.9%) respondents were employed by UT, and 12 (5.6%) respondents were employed by TSU, while one (0.5%) was employed exclusively (100% of salary paid) by a Tennessee county government.

Research Question One—What Are the Extension Agents' Perceptions of the Current Performance Appraisal System?

The instrument had a total of 14 items targeting perceptions of the current appraisal system representing fairness, job description, multiple appraisers, professional development, and overall effectiveness. Percentages for item responses are shown in Table 1. For the purpose of this discussion, the data have been collapsed, so that disagreement is represented by strongly disagree and disagree responses, and agreement is represented by strongly agree and agree responses. A sixth answer category for "don't know" was provided on the instrument. Respondents were also allowed to skip questions (provide no answer). In calculating the percentage of responses, the "don't know" and no answer responses were collapsed and included.

Table 1.
Respondents' Perceptions of the Current Appraisal System

Perceptions (N=218)	%Strongly Disagree	%Disagree	%Neither Agree  Nor Disagree	%Agree	%Strongly Agree	%Don't Know/No Answer
Needs to be improved.*	0.0	3.6	15.6	40.8	38.1	1.8
Is fair.	9.2	23.9	16.5	42.6	6.4	1.3
Has discouraged me.*	5.9	22.9	22.0	28.4	18.8	1.8
Has helped me improve my professionalism.	9.2	22.9	20.1	38.5	7.3	1.8
Reflects my major job responsibilities.	7.3	23.4	22.5	42.2	3.2	1.3
Helps me understand my job duties.	7.8	23.9	22.5	38.5	6.4	1.0
Is implemented fairly.	9.2	23.9	22.0	38.1	5.0	1.8
Represents what I do on the job.	13.8	28.4	15.6	36.2	5.0	1.0
Would be more accurate if a team of the county directors, regional director and subject matter specialist served as raters.*	12.4	30.0	16.5	26.6	9.1	5.5
Causes me confusion about job responsibilities.*	4.1	35.3	27.9	26.2	4.5	1.8
Is unbiased.	12.4	28.9	27.1	26.1	3.2	2.3
Would be more accurate if a subject matter specialist served as a rater.*	17.4	31.7	24.0	16.9	4.5	5.5
Would be more accurate if county directors did NOT serve as raters.*	28.4	36.0	19.2	7.3	6.4	2.7
Is close to ideal.	18.8	41.4	28.0	7.3	1.3	2.7
Note. Row percentages may not total 100.0 due to rounding. *Items with an asterisk have reverse polarity whereby disagreement is the positive response.

Appraisal System with Overall Effectiveness

The item with the strongest agreement among all items regarding satisfaction with the current appraisal system was, "the performance appraisal system needs to be improved" (78.9%). The majority of respondents disagreed that the performance appraisal system is "close to ideal" (60.2%).

Appraisal System with Multiple Appraisers

The majority of respondents (64.4%) expressed approval for having the county director as an appraiser by disagreeing that the appraisal system "would be more accurate if county directors did NOT serve as raters." For the item, "would be more accurate if a subject matter specialist served as a rater," 49.1% disagreed; and for the item, "would be more accurate if a team of the county directors, regional director and subject matter specialist served as raters," 42.4% disagreed.

Appraisal System Fairness

Regarding fairness, 49.1% agreed that the performance appraisal system "is fair," and 43.1% agreed that the appraisal system "is implemented fairly." Conversely, four in 10 respondents (41.3%) disagreed that the appraisal system "is unbiased."

An Appraisal System Based on Job Description

Regarding job description, four in 10 agreed that the appraisal system "reflects my major job responsibilities" (45.4%) and "helps me understand my job duties" (44.9%). About equal numbers of respondents agreed (41.3%) and disagreed (42.2%) that the appraisal system "represents what I do on the job."

An Appraisal System that Provides Professional Development

Four in 10 respondents (45.8%) agreed that the performance appraisal system "has helped me improve my professionalism."

The percentage of respondents who neither agreed nor disagreed with the statements regarding their perceptions of the current performance appraisal system ranged from 15.6% to 28%. More than one-fourth of respondents neither agreed nor disagreed that the appraisal system "causes me confusion about job responsibilities" (27.9%), "is unbiased" (27.1%), and "is close to ideal" (28.0%).

Research Question Two—What Are the Extension Agents' Perceptions of Any Needed Improvements to the Performance Appraisal System?

The last open-ended question on the survey asked, "If there is anything else you would like to share about the performance appraisal system, please type it in the box below." Of the 218 respondents, 96 (44%) provided written comments. Of these 96 commenters, 68 (71%) of 218 respondents suggested improvements. In coding, 126 different comments were noted, and 72 comments fit six major themes. Analysis of the written comments revealed the following key themes relative to an improved performance appraisal system.

• Inefficiency Issues—Seventeen respondents noted that the current appraisal process is too time consuming and inefficient.
• Rubric More Closely Reflecting Job—Twelve respondents described the need to make the rubric more representative of the job.
• Appraisal Interview—Eleven respondents noted that Extension agents should have a direct appraisal interview with the regional director, rather than being represented by the county director at this appraisal interview.
• County Directors' Appraisal Skills—Twelve respondents discussed how county directors needed additional skills in performance appraisal.
• Merit Raises—Eleven respondents described the need to implement, improve, and/or strengthen merit raises.
• Input for the County Directors' Appraisal—Nine respondents wrote about the need to allow Extension agents to have input into the appraisal of the county directors.

Inefficiency Issues

Seventeen respondents noted that the current appraisal process is too time-consuming and inefficient. Some respondents felt that the appraisal process required the agent to compile too much information. Others felt that they were not provided adequate time to discuss the ratings received from the appraisal interview between regional directors and county directors. They indicated that they were asked to sign their appraisal, yet there was not a formal interview scheduled in which they could receive feedback and understand how to improve. One respondent commented:

It is entirely too time consuming. We, as county based agents, should not be required to write a "thesis" on ourselves each year. Our supervisors know the level that we are performing, and they should be able to fairly evaluate us using a very simple form.

Rubric More Closely Reflecting Job  

Twelve respondents described the need to make the rubric more representative of the job. The most common need identified was to expand recognition for work, as one respondent described, "…beyond the priority program area". Another respondent described the need this way:

The appraisal system does not always seem to accurately reflect the work that we do. We have to spend much of our time planning and conducting events that sometimes do not accurately feed into the appraisal, and while it may be easy to say we can cut some things out, that may not be possible in the county in which we work, if we want to continue to have a positive relationship with the people and agencies in our counties.

Appraisal Interview

Eleven respondents noted that Extension agents should have a direct appraisal interview with the regional director, rather than being represented by the county director at this appraisal interview. Respondents indicated that the county directors could not be knowledgeable about all Extension programs conducted in the county. Some respondents noted that UT Extension formerly used direct appraisal interviews between regional directors and Extension agents. One respondent described the following:

I think employees should be able to represent themselves at Performance Appraisals with their Regional Director, versus the County Director trying to justify your work. There is NO WAY for the County Director to be able to be familiar with everything that everyone in his/her office has done throughout the year.

County Directors' Appraisal Skill

Twelve respondents described the need to improve performance of the county directors in conducting the appraisal. The respondents expressed that the role of the county directors should focus on improving performance, especially in encouraging high -quality performance, yet they felt that many times the county directors place emphasis on finishing the appraisals. Representative comments included the following:

I have a County Director who has never met with me before or after performance appraisals. I simply receive the form and am told to sign it. Thank goodness for Regional staff who look out for me.

In the eyes of our County Director he puts more importance on having something in the boxes versus helping us evaluate and improve our job performance.

Merit Raises

Eleven respondents described the need to implement, improve, and/or strengthen merit pay raises. Some described the need to implement a merit pay system that would recognize performance with pay, as one respondent described, "Receiving an E on the performance appraisal offers the same opportunities as receiving a U." Respondents also questioned the fairness of past merit pay implementations. One respondent described the past implementation this way:

Although there have not been merit raises in a long time, that is what I find most unfair about the process. How can I score the highest (E), my co-worker score an E, and she gets a raise and I not because "somebody", "somewhere" assigned points to determine who gets the raises and who does not.

Input for the County Directors' Appraisal

Nine respondents wrote about the need to allow Extension agents to have input into the appraisal of the county directors. One respondent expressed:

I feel that the county agents and other workers should have an opportunity to provide feedback about the County Directors. In a past life this was called a 360 review and gave the supervisor an opportunity for confidential feedback from direct reports, peers and other co-workers. This is badly needed in Extension.

Conclusions and Discussion

Extension agents reported both positive and negative aspects to the Tennessee Extension Agent Performance Appraisal System. The vast majority of Extension agents (78.9%) agreed that "the performance appraisal system needs to be improved," and the majority disagreed (60.2%) that the present system is "close to ideal".

In contrast, respondents also perceive the most positive aspects of the current performance appraisal system to be the involvement of county directors as appraisers, fairness, and the improvement of the Extension agents' professional development by having participated in the appraisal system. The majority (64.4%) expressed approval for having the county director as an appraiser by disagreeing that the appraisal system "would be more accurate if county directors did NOT serve as raters." The respondents disagree with using subject matter specialists as appraisers. This is inconsistent with previous research by Zoller and Safrit (1999) and Davis and Verma (1993), which found Extension agents favor a team of appraisers that includes a subject matter specialist.

Nearly one-half of respondents (49%) agreed that the appraisal system is fair. Four in 10 respondents (45.8%) agreed that the performance appraisal system "has helped me improve my professionalism." Extension may be able to capitalize on this perception by setting employee development as an organizational goal. Doing so may improve both the employee performance and organizational effectiveness (Daley, 1992), and this emphasis may lead to greater satisfaction with the appraisal system.

Employees who feel that performance dimensions used to assess their job are relevant to their actual job responsibilities have a more positive perception of their performance appraisal system (Dipboye & de Pontbriand, 1981; Narcisse & Harcourt, 2008). One–third (30.7%) of the participants in the reported here study disagreed that the appraisal system reflected their major job responsibilities.

The number of respondents who neither agreed nor disagreed with the statements regarding their perceptions of the current performance appraisal system ranged from 15.6% to 28%. While the reason for this is not known, this is an interesting finding. It may be due to the fact that one in five respondents (20.9%) had worked 5 years or less, and these respondents have had limited experience with the performance appraisal system.

Recommendations for Improving Performance Appraisal

The following recommendations should be viewed with a caveat. The key themes from the written comments represent a relatively small group of respondents. In addition, one in five respondents (20.9%) had worked 5 years or less under this appraisal system, and their perceptions may represent limited experience with it.

• Appraisal Interview—It is recommended that county directors continue to contribute to the annual performance appraisal of Extension agents, but, as the findings of the study suggest, Extension administrators should explore approaches to annual appraisal interviews of Extension agents by regional directors.
• Performance Dimensions—The performance appraisal system, in its present format or in a new format, should have validation and reliability studies. This is an essential step for the success of an appraisal system, because it verifies the criteria (French & Malo, 1987) and adds credibility to the appraisal process in the eyes of those subject to it.
• Merit Pay Raises—Extension administrators should consider providing employees professional development in the entire appraisal process, including how the final ratings are used for merit pay purposes.
• Performance Appraisal System Efficiency—Extension administrators should explore strategies to require less effort on the part of the Extension agent to prepare the appraisal materials.
• Training in Conducting Performance Appraisal—It is recommended that every appraiser have professional development courses in performance appraisal so that their implementation of the performance appraisal system will be more efficient and more effective, consistent with research findings of Middlewood (2001) and Davis and Verma (1993). Comprehensive training of appraisers will be essential if merit pay decisions are based on appraisal results, as French and Malo (1987) have pointed out.
• Employee Contributions to County Director Performance Appraisal—It is recommended that Extension administrators identify competencies needed to be an effective county Extension director. These competencies should be used as the basis for an instrument that would collect evaluative data regarding county director performance from Extension agents.

A Final Note

Valid, reliable performance appraisal systems are important to any profession. They should be the basis for professional development of individuals, improvement of the services rendered, and summative decisions made with regard to an employee (merit pay, job placement, promotion, or termination, etc.). However, the steps necessary to ensure valid, reliable performance appraisal systems are often overlooked when a system is initially developed, and regular review of a system by those subject to it and those charged with implementing it is not conducted often enough. We in Extension can do better, as can professionals in most fields.

References

Byrd, H. (2009). UT Extension proposed regional review schedule (version 2). University of Tennessee Extension. Retrieved from: http://agweb.ag.utk.edu:8090/eesd/EESD.nsf/PerformanceAppraisalReviewSchedule.pdf

Colton, D., & Covert,  R.W. (2007). Designing and constructing instruments for social science research and evaluation. San Francisco: John Wiley & Sons, Inc.

Daley, D. M. (1992). Performance appraisal in the public sector: Techniques and applications. Westport, CT: Quorum Books.

Davis, W. L., & Verma, S. (1993). Performance appraisal: How extension agents view the system. Journal of Extension [On-line], 31(4) Article 4FEA3. Available at: http://www.joe.org/joe/1993winter/a3.php

Dipboye, R. L., & de Pontbriand, R. (1981). Correlates of employee reactions to performance appraisals and appraisal systems. Journal of Applied Psychology, 66(2) 248-251.

Donaldson, J. L. (2011). A survey of Tennessee Extension agents' perceptions of the Tennessee Extension agent performance appraisal system (Doctoral dissertation). University of Tennessee, Knoxville.

French, R. L., & Malo, G. E. (1987). Developing performance evaluation systems for career ladders. Thresholds in Education, 13(1) 16-19.

Heckel, M. C. (1978). Measuring the performance of extension educators. Journal of Extension [On-line], 16(6). Available at: http://www.joe.org/joe/1978november/78-6-a1.pdf

Kuchinke, K. P., Correthers, G., & Cecil, K. (2008). Managing performance in Extension: Redesigning the performance evaluation system at Illinois. Journal of Extension [On-line], 46(4) Article 4FEA5. Available at: http://www.joe.org/joe/2008august/a5.php

Middlewood, D. (2001). Managing teacher appraisal and performance: A comparative approach. London:  Routledge/Falmer.

Narcisse, S., & Harcourt, M. (2008). Employee fairness perceptions of performance appraisal: A Saint Lucian case study. The International Journal of Human Resource Management, 19(6), 1152-1169.

Patterson, T. F. (1987). Refining performance appraisal. Journal of Extension [On-line], 25(4) Article 4FEA5. Available at: http://www.joe.org/joe/1987winter/a5.php

Petersen, B., & McDonald, D. A. (2009). A focused interview study of 4-H volunteer performance appraisals. Journal of Extension [On-line], 47(5) Article 5RIB4. Available at: http://www.joe.org/joe/2009october/rb4.php

SPSS, Inc. (2008). mrInterview fact sheet. Retrieved from: http://www.spss.com/mrInterview

Rice, R. D. (2001). An examination of the relationships between the Alabama Cooperative Extension System assessment center ratings and subsequent county agent-coordinators' job performance ratings. Journal of Extension [On-line], 39(2) Article 2RIB1. Available at: http://www.joe.org/joe/2001april/rb1.php

Terry, B. D., & Israel, G. D. (2004). Agent performance and customer satisfaction. Journal of Extension [On-line], 42(6) Article 6FEA4. Available at: http://www.joe.org/joe/2004december/a4.php

Zoller, C., & Safrit, R. D. (1999). Ohio State University extension agents' perceptions of agent support teams. Journal of Extension [On-line], 37(1) Article 1RIB4. Available at: http://www.joe.org/joe/1999february/rb4.php