Practical Tips for Evaluators and Administrators to Work Together in Building Evaluation Capacity

New relationships among government and institutions, professionals, and service providers have greatly increased the demand and opportunities for evaluation and the development of evaluation systems (Norris & Kushner, 2007). In Extension, many organizations employ full- or part-time evaluators to work with faculty and staff to help build their capacity for evaluation. In Extension, building evaluation capacity generally means helping individuals and teams develop the knowledge, skills, and motivation to evaluate their programs and communicate the results. Currently, 170 evaluators subscribe to the Extension Education Evaluation topical interest group listserv of the American Evaluation Association. More than 5,000 professionals are currently members of the American Evaluation Association, and the demand for evaluation services is currently growing and expanding.

One source of expertise on building evaluation capacity emphasizes that evaluators should be aware of their organizational contexts, power hierarchies, and administrative processes (Volkov & King, 2007). Volkov and King recommend that "key leaders of the organization support and share responsibility for evaluation capacity building."

Many Extension evaluators report to administrators such as deans and directors, and work for the organization, at least in part, through administrators who are "champions" for evaluation. Evaluation champions are leaders who have influence in the organization and who support and value evaluative thinking. Champions are key to the success of evaluation capacity building efforts in Extension (Taylor-Powell & Boyd, in press). While evaluation champions may or may not have formal leadership positions, many formal leaders in Extension nationwide appear to be evaluation champions.

Administrators and evaluators who work together to create and communicate evaluation expectations can contribute positively to the organizational climate for evaluation and evaluation capacity building. Clarifying policies and changes in this dynamic area of organizational life can reduce frustration and barriers for field faculty and specialists who are not yet convinced that formal evaluation is an important and useful dimension of their professional work.

Evaluators and administrators together can: 1) involve stakeholders-field faculty and specialists-in what is realistic and what is possible; 2) set a positive tone for what can be done; and 3) communicate that both program improvement and program results are important findings in evaluation processes.

The following are topics that administrators and internal evaluators should consider as they orient themselves to common goals and specific evaluation expectations for members of the organization. They should engage in focused conversation about these topics before major evaluation efforts, as well as during ongoing organizational processes.

Organizational History, Including Budgetary Influences on Resource Distribution

• History of the institution, including its nature as an 1862, 1890, 1994 or other land-grant institution
• Purpose of the land-grant system
• Roles relative to federal reporting and accountability information for Cooperative State Research Education and Extension Service

Administrative Principles That Drive Decisions

• Mission, vision, and goals for the organization, and how program evaluation can support those
• Who makes programming decisions, who makes budgeting decisions, as well as the level of centralization
• What "program evaluation" has historically meant in the organization and the messages that have been delivered

Appropriate Roles During an Evaluation Process and Expectations of Each Other

• How evaluators and administrators will involve people early in processes, connect key people, and be champions of the effort
• Where in the organization the results of evaluation work might be productive or, at least, influential
• What makes an evaluation effort feasible, appropriate, useful, and accurate-the four main standards of program evaluators per the American Evaluation Association

As basic as it sounds, frequent, productive communication between evaluators and administrators regarding evaluation progress in the organization can help to create the conditions where evaluation capacity can grow. Administrators who attend evaluation professional development opportunities as learners or observers send a message that they care and that they want to know whether there are new trends or influences on practical, relevant evaluation in community-based education evaluation. In addition, administrators who complete and disseminate reflective work with evaluators demonstrate that evaluation is a scholarly activity in its own right and that it deserves the attention of land-grant faculty across the nation.

Acknowledgements

Thank you to Laura Hoelscher, Susan Paddock, Meg Gores, Nancy Franz, and Michael Lambur for their feedback on earlier versions of this piece.

References

Norris, N., & Kushner, S. (2007). The new public management and evaluation. Advances in Evaluation: Dilemmas of Engagement: Evaluation and the New Public Management. 10: 1-16.

Russ-Eft, D., Bober, M., de la Teja, I., Foxon, M., & Koszalka, T. (2008). Evaluator competencies: Standards for the practice of evaluation in organizations. John Wiley & Sons: San Francisco.

Taylor-Powell, E., & Boyd, H. (in press). Evaluation capacity building in complex organizations. New Directions in Evaluation.

Volkov, B., & King, J. (2007). A checklist for building organizational evaluation capacity. Retrieved January 12, 2008, from: http://www.wmich.edu/evalctr/checklists/