Appraising the Appraiser: Extension Agents' and County Directors' Perceptions of Their Appraisers

Introduction

For most of the 20th century, research in performance appraisal focused primarily on performance measurement and accuracy, with an emphasis on the instruments used for appraisal. Few research resources were invested in studies that explored the human aspects of appraisal, specifically, employee perceptions (Lopez, 1968; Meyer, Kay, & French, 1965). Today, performance appraisal research has expanded from a primary focus on performance measurement and accuracy to include more human factors, especially employee perceptions (Fletcher, 2001). An effective performance appraisal system should be viewed as such by both supervisors and employees (Schuman & Olufs, 1988). Gaby (2004) asserts that ascertaining employee perceptions toward performance appraisal is useful for scientific and practical reasons.

Performance Appraisal of County Extension Agents

The literature regarding the performance appraisal of Extension agents includes both commentary and research. The commentaries specific to Extension personnel reflect the development of performance appraisal practices and concepts across disciplines and organizations. The first commentaries on the subject were made more than 40 years ago by Warner (1967), who proposed that Extension programs would be strengthened if supervisors would provide frequent feedback to employees, and Bruce and Carter (1967), who noted the need for employee recognition and self-appraisal through program statistics. This thought was echoed by Durfee (1970), who implored Extension administrators to adopt a management by objectives approach and to more effectively coach Extension agents for better performance. Buford (1990) posited that Extension performance appraisal systems were subjective and ambiguous. In an information age, Buford proposed, Extension needed to devote more effort to employee recognition and career advancement opportunities. Likewise, Boone (1990) noted that Extension performance appraisal systems completely ignored interdisciplinary work, despite the fact that modern-day societal problems need solutions from interdisciplinary teams.

Ladewig and Shiao (1983) described the historical approaches to appraise county Extension agents: personal interviews, subjective ratings, and knowledge testing. In 1977, the United States Department of Agriculture contracted with the American Institutes for Research (AIR) to develop instruments to assist land-grant universities in appraising Extension agent performance. AIR managed a nationwide study that yielded a "Performance-Against-Standards Form" with 200 items in six broad categories: program planning, program promotion and public relations, program implementation, program support, interpersonal and personal, and supervisory performance. In a follow-up study with Ohio State University Extension agents, Ladewig and Shiao (1983) found about 20% of the items not applicable to an Extension agent's job. Additional reliability analysis (using item score to total score correlation) reduced the instrument to 60 items (Ladewig & Shiao, 1983).

Kuchinke, Correthers, and Cecil (2008) studied 16 performance appraisal systems used by state Extension organizations. They found that it was acceptable for the regional director to conduct the annual performance appraisal and the performance appraisal interview with Extension agents. Yet they also viewed this lack of input from multiple appraisers as a weakness of the performance appraisal system used for 10 years by the University of Illinois Extension. They also found a need for alignment between the performance appraisal system and the Extension organization's strategic goals.

Extension Agents' Perceptions of Performance Appraisal

Wolford (1985) surveyed the population of 380 Extension agents employed by Virginia Cooperative Extension Service and conducted 12 interviews with agents to analyze perceptions toward performance appraisal, especially relevancy and value. Wolford found that agents perceive a lack of communication with their supervisors.

Vogt and Van Tilburg (1989) studied employee perceptions of performance appraisals conducted by the county Extension directors through a questionnaire completed by a sample of 54 county directors and 71 agents. At the time of this research, Extension agent appraisal by county Extension directors was a significant change from appraisals being conducted by the regional director. The researchers measured satisfaction with this new approach to performance appraisal. The county directors were significantly more satisfied than the Extension agents.

Tennessee Extension Agent Performance Appraisal Process

Donaldson and French (2013) fully discuss the Tennessee Extension appraisal process and rating criteria. After the year has passed, the employee prepares materials to demonstrate competence and performance in the appraisal criteria during the past year. The employee may attach any number of documents or any amount of text to his/her electronic appraisal form. The appraiser completes the rating form in a one-on-one meeting with the employee. Appraisal interviews of Extension agents are completed by county directors, and appraisal interviews of county directors are completed by regional directors. It should be noted that every county director in the state is also an Extension agent with programmatic responsibilities in one or more base programs (4-H youth development, agriculture and natural resources, family and consumer sciences, and/or community resource development).

Purpose

The purpose of the study reported here was to determine Extension agents' and county directors' perceptions of the roles and behaviors of their appraisers (specific to the performance appraisal process) as the literature review showed the benefits of understanding employee perceptions toward performance appraisal. As previously stated, appraisal interviews of Extension agents are completed by county directors, and appraisal interviews of county directors are completed by regional directors.

Methods

The study reported here draws on the same data used for Donaldson (2011) and Donaldson and French (2013). The panel review, instrument pilot test, and data analysis are fully discussed in Donaldson and French (2013). The instrument was reviewed by an expert panel and pilot tested with a group of 39 Extension agents. The instrument was deployed online to the study population in spring, 2010. The population of county Extension agents and county directors at the time of this study was 312; 217 were Extension agents not serving as county directors and 95 were county directors.

To measure the respondent's perceptions of the role and behavior of their appraiser (appraiser performance), seven questions were developed from eight studies, as shown in Table 1. The studies indicated the major factors relative to appraiser performance were:

• J—Appraisers' Judgment/Fairness/Trust (Mani, 2002);
• U—Appraisers' Understanding of the Job Being Appraised (Mani, 2002); and
• S—Appraisers' Skill/Instruction in Performance Appraisal (Broadwell, 1985; Daley, 1992; French & Malo, 1987; Grote, 2008; Guinn, 1998; Krayer, 1987; Rasch, 2004).

The survey used six response categories: 1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, 5=strongly agree, and 6=don't know. Respondents were also allowed to skip questions (provide no answer). In calculating the percentage of responses, the "don't know" and no answer responses were collapsed and included. Branching was used in the survey instrument so that Extension agents who were not county directors described their perceptions of county directors in conducting appraisals, and county directors described their perceptions of regional directors in conducting appraisals.

The study had different response rates between county directors and Extension agents. While the overall response rate to the survey was 69%, 135 Extension agents responded (62% of the 217 Extension agents who are not county directors) and 83 county directors responded (87% of 95).

Table 1.
Theoretical Framework for Satisfaction with Appraiser Performance Items

Appraiser Performance Items (Donaldson, 2011)	Factora	Theoretical Framework/Source(s)
1. Exercised good judgment in making appraisal ratings.	J	Mani, 2002
2. Was unbiased in making appraisal ratings	J	Mani, 2002
3. Gave an honest assessment of my job performance.	J	Mani, 2002
4. Understands my work better than anyone else in this organization.	U	Mani, 2002
5. Needed more instruction in performance appraisal.	S	Broadwell, 1985; Daley, 1992; French & Malo, 1987; Grote, 2008; Guinn, 1998; Krayer, 1987; Rasch, 2004
6. Provided confusing instructions about the appraisal.	S	Broadwell, 1985; Daley, 1992; French & Malo, 1987; Grote, 2008; Guinn, 1998; Krayer, 1987; Rasch, 2004
7. Showed no appreciation for the work I do.	U	Mani, 2002
aThe factors were abbreviated as follows: J – Appraisers' Judgment/Fairness/Trust, U – Appraisers' Understanding of the Job Being Appraised, and S – Appraisers' Skill/Instruction in Performance Appraisal.

Findings

In the following report of findings, the strongly disagree and disagree responses were collapsed to show disagreement, and agree and strongly agree responses were collapsed to show agreement. (This is a common method for simplifying and describing responses.) The majority of Extension agents (68.9%) and county directors (73.5%) expressed their positive perception of appraisers' performance by disagreeing with the statement, "showed no appreciation for the work I do." Likewise, the majority of Extension agents (62.7%) and county directors (68.8%) agreed that their appraiser "was unbiased in making appraisal ratings." Equal proportions of Extension agents (66.7%) and county directors (66.3%) perceived that their appraiser "gave an honest assessment of my job performance." Percentages for item responses are shown in Table 2.

The biggest difference between Extension agents and county directors regarding perceptions of their appraisers was for the item, "needed more instruction in performance appraisal." Among Extension agents, 37.8% disagreed that their county director "needed more instruction in performance appraisal," compared to 53.0% of county directors, who disagreed that their regional director needed more instruction. Among Extension agents, 30.6% disagreed that their county director "understands my work better than anyone else in this organization," compared to 43.4% of county directors, who disagreed regarding their regional director. Nearly 10% more county directors (76.0%) than Extension agents (67.4%) agreed that their appraiser "exercised good judgment in making appraisal ratings."

As shown in Table 2, the percentage of Extension agents who neither agreed nor disagreed with the appraiser performance statements ranged from 11.8% to 23%. One in five Extension agents neither agreed nor disagreed that their county director "needed more instruction in performance appraisal" (23%) and "provided confusing instructions about the appraisal (20%). The percentage of county directors who neither agreed nor disagreed with the appraiser performance statements ranged from 12% to 23%. One in five county directors neither agreed nor disagreed that their regional director "understands my work better than anyone else in the organization," (23%) and "needed more instruction in performance appraisal" (23%).

Table 2.
Respondents' Perceptions of the Roles/Behaviors of Appraisers by Job Title

Perceptions	% Strongly Disagree	% Disagree	% Neither Agree Nor Disagree	% Agree	% Strongly Agree	% Don't Know/ No Answer
Extension Agents (N=135)
Exercised good judgment in making appraisal ratings.	3.7	8.2	16.3	42.2	25.2	4.4
Gave an honest assessment of my job performance.	4.4	11.9	12.6	43.0	23.7	4.4
Was unbiased in making appraisal ratings.	6.7	6.0	17.0	37.8	25.2	7.4
Understands my work better than anyone else in this organization.	15.0	15.6	17.0	28.2	20.7	3.7
Needed more instruction in performance appraisal.*	6.7	31.1	23.0	20.7	14.8	3.7
Provided confusing instructions about the appraisal.*	10.4	49.0	20.0	8.9	8.9	3.7
Showed no appreciation for the work I do.*	37.0	31.9	11.9	9.6	6.7	3.7
County Directors (N=83)
Exercised good judgment in making appraisal ratings.	4.8	7.2	12.0	59.0	17.0	0.0
Was unbiased in making appraisal ratings.	6.0	10.8	14.5	58.0	10.0	0.0
Gave an honest assessment of my job performance.	4.8	13.3	15.7	50.6	15.7	0.0
Understands my work better than anyone else in this organization.	13.3	30.1	23.0	25.3	7.2	1.2
Needed more instruction in performance appraisal.*	12.0	41.0	23.0	18.1	6.0	0.0
Provided confusing instructions about the appraisal.*	10.8	51.8	16.9	19.3	1.2	0.0
Showed no appreciation for the work I do.*	36.1	37.4	13.3	7.2	6.0	0.0
Note. Row percentages may not total 100.0 due to rounding. *Items with an asterisk have reverse polarity whereby disagreement is the positive response.

Conclusions

The majority of Extension agents have positive perceptions of the performance of the county directors in conducting the performance appraisal, and the majority of county directors have positive perceptions of the performance of the regional directors in conducting the performance appraisal. Appraisers are viewed as exercising positive judgment and fairness, and they are trusted by their subordinates. The majority of respondents (70.6%) agreed that the appraisers "exercised good judgment in making appraisal ratings." Six in 10 respondents (66.5%) agreed that appraisers "gave an honest assessment of their job performance" and were "unbiased in making appraisal ratings" (65.1%).

County directors and Extension agents differ in perceptions of the performance appraisal instruction needed by their appraisers, with more Extension agents than county directors expressing that their appraiser needs more instruction. Almost one-fourth (24.1%) of county directors showed agreement that their appraiser "needed more instruction in performance appraisal" compared to 35.4% of Extension agents.

Discussion and Recommendations

Instruction

Almost one-fourth (24.1%) of county directors and more than one-third (35.4%) of Extension agents showed agreement that their appraiser "needed more instruction in performance appraisal." Therefore, it is recommended that every appraiser have professional development courses to implement the performance appraisal system more efficiently and more effectively, consistent with research findings of Middlewood (2001) and Davis and Verma (1993).

The literature points to instruction for appraisers as a key to a fairer and more reliable performance appraisal system. The literature points to training appraisers in three concepts: employee behaviors/practices representing the appraisal standards; valid, reliable assessment procedures; and interpretation of data (Broadwell, 1985; Daley, 1992; Grote, 2008; Guion, 1998; Rasch, 2004; Vasset, Marnburg, & Furunes, 2010). Daley (1992) suggests that instruction is effective at reducing appraiser errors. Regarding performance appraisals in higher education, Rasch (2004) proposed that appraisers needed instruction on understanding continuous improvement and leadership. Krayer (1987) described effective appraiser training as one in which communication skills for the performance appraisal interview were stressed. Krayer further described that appraisers ought to role play both the interviewer (appraiser) and interviewee role as part of their instruction.

Recommended resources for appraisers include an appraiser manual, hands-on practice in rating sets of materials and role play (Krayer, 1987), and reliability checking through the use of the practice materials. Once improvements to the system are made, every employee ought to receive an "appraisal manual" detailing the standards for every position, the evaluation procedures, feedback procedures, and a section on appraiser training. This would create more trust in the system and show Extension agents what has been improved.

Periodical Reviews

The instrument used in the study reported here could be used for benchmarking employee satisfaction with their appraiser. It could be administered every 5 years, or whenever the appraisal system is changed (if more often than 5-year intervals). This practice would be consistent with research by Davis and Verma (1993) that showed agents have a more positive perception of the performance appraisal process when the system itself is periodically reviewed. The instrument would need appropriate field testing and reliability and validation studies.

References

Boone, E. J . (1990). Crossing lines. Journal of Extension [On-Line], 28(3) Article 3TP1. Available at: http://www.joe.org/joe/1990fall/tp1.php

Broadwell, M. M. (1985). Supervisory handbook: A management guide to principles and applications. New York: John Wiley & Sons.

Bruce, R. L., & Carter, Jr., G L. (1967). Administrative climate. Journal of Extension [On-line], 5(1). Available at: http://www.joe.org/joe/1967spring/1967-1-a1.pdf

Buford, Jr., J. A. (1990). Extension management in the information age. Journal of Extension [On-line], 28(1) Article 1FUT2. Available at: http://www.joe.org/joe/1990spring/fut2.php

Daley, D. M. (1992). Performance appraisal in the public sector: Techniques and applications. Westport, CT: Quorum Books.

Davis, W. L., & Verma, S. (1993). Performance appraisal: How Extension agents view the system. Journal of Extension [On-line], 31(4) Article 4FEA3. Available at: http://www.joe.org/joe/1993winter/a3.php

Donaldson, J. L. (2011). A survey of Tennessee Extension agents' perceptions of the Tennessee Extension agent performance appraisal system (Doctoral dissertation). University of Tennessee, Knoxville.

Donaldson, J. L., & French, R. L. (2013). Tennessee Extension agents' perceptions of performance appraisal. Journal of Extension [On-line], 51(3) Article 3FEA10. Available at:http://www.joe.org/joe/2013june/a10.php

Durfee, A. E. (1970). Helping others improve. Journal of Extension [On-line], 8(2). Available at: http://www.joe.org/joe/1970summer/1970-2-a2.pdf

Fletcher, C. (2001). Performance appraisal and management: The developing agenda. Journal of Occupational and Organizational Psychology, 74, 473-487.

French, R. L., & Malo, G. E. (1987). Developing performance evaluation systems for career ladders. Thresholds in Education, 13(1) 16-19.

Gaby, S. H. (2004). Summarizing and measuring participants' perceptions related to performance appraisal effectiveness (Doctoral dissertation). University of Tennessee, Knoxville.

Grote, D. (2008). Passing judgment: Why we still can't get performance appraisal right. The Conference Board Review, pp. 36-43.

Guion, R. M. (1998). Assessment, measurement, and prediction for personnel decisions. Mahwah, NJ: Lawrence Erlbaum Associates.

Krayer, K. J. (1986). Using training to reduce role conflict and ambiguity. Training & Development Journal, 40(11), 49.

Kuchinke, K. P., Correthers, G., & Cecil, K. (2008). Managing performance in Extension: Redesigning the performance evaluation system at Illinois. Journal of Extension [On-line], 46(4) Article 4FEA5. Available at: http://www.joe.org/joe/2008august/a5.php

Ladewig, H. & Shiao, K.S. (1983). Development of personnel appraisal procedures to measure the job performance of county Extension Agents. Ohio State University Department of Agricultural Education, 35 pages. Retrieved from ERIC database. (ED 230712)

Lopez, Jr., F. M. (1968). Evaluating employee performance. Chicago: Public Personnel Association.

Mani, B. G. (2002). Performance appraisal systems, productivity, and motivation: A case study. Public Personnel Management, 31(2), 141-160.

Meyer, H. H., Kay, E., & French, Jr., J. R. P. (1965). Split roles in performance appraisal. Harvard Business Review, 123-129.

Middlewood, D. (2001). Managing teacher appraisal and performance: A comparative approach. London: Routledge/Falmer.

Rasch, L. (2004). Employee performance appraisal and the 95/5 rule. Community College Journal of Research and Practice, 28, 407-414.

Schuman, D., & Olufs, III, D. W. (1988). Public administration in the United States. Lexington, MA: D.C. Heath and Company.

Vasset, F., Marnburg, E., & Furunes, T. (2010). Employee's perceptions of justice in performamce appraisals. Nursing Management—UK, 17(2), 30-34.

Vogt, B., & Van Tilberg, E. (1988). Ohio Cooperative Extension Service agents perceptions of the agent rating method of the performance appraisal system. Journal of Agricultural Education 30(1), 60-67.

Warner, W. K. (1967). Feedback in administrative. Journal of Extension [On-line], 5(1). Available at: http://www.joe.org/joe/1967spring/1967-1-a5.pdf

Wolford, A. K. (1985). An analysis of the effects of race and gender in scoring Extension agent performance standards (Doctoral dissertation). Virginia Polytechnic Institute and State University, Blacksburg.