The Latest Take on Utilization-Focused Evaluation

Introduction

Michael Quinn Patton's (2008) Utilization-Focused Evaluation, 4th Edition, emphasizes the importance of creating evaluations with the end user in mind. Stakeholders are central to Extension work because they connect program priorities to the communities they serve (Barker, 2007). The majority of Extension programs utilize stakeholder input throughout program development to ensure that clients are receiving the education they want and will use. Patton (2008) believes evaluations should be created in the same way. By utilizing stakeholder input to develop evaluations, Extension professionals can get information their clients want and will use. From this perspective, it is easy to see why a book on Utilization-Focused Evaluation would be a handy tool for Extension professionals as they strive to include stakeholders in all phases of program planning and evaluation.

The first edition of this book was reviewed in the Journal of Extension in 1983 (Warner, 1983). The previous review gives an overall discussion of what the book includes, emphasizes the main points important to Extension professionals, and gives the reader an idea of what to expect. Because an overall discussion of the book already exists in this journal, this book review focuses on what has changed through the last three revisions, the strengths and weaknesses of the current edition, and how the book is being currently used to enhance Extension work.

Changes from Previous Editions

Patton's (2008) user-friendly evaluation practices suggests evaluation studies are only relevant when decision makers and end users are involved in the planning process. Through utilization-focused evaluation, decisions about how the information will be used are determined prior to any data collection (Warner, 1983). At the time of the first edition, in 1978, utilization-focused evaluation was a controversial topic, and the book primarily explained and defended this method as a proper practice. In the second edition, the book's focus was adjusted to enable the identification of intended users. By the third edition, utilization-focused evaluation had become widely understood, if not used. That edition included a section examining the effects of engaging stakeholders in the evaluation process in addition to utilizing them when writing useable recommendations from results.

Over the last 30 years evaluation methods have evolved with society, creating dramatic changes. Most recently, the need for global competency has become increasingly important (Zhai & Scheer, 2002). The fourth edition is filled with international examples emphasizing the importance of global perspectives. As diversity of clientele increases, so, too, has the field of evaluation. Patton does a nice job of using recent research activities to emphasize his main points and provides examples of how diversity effects evaluation planning.

New material also includes a section on innovative uses for evaluation and the effects politics can have when evaluators are faced with ethical decisions. An entire chapter is devoted to distinguishing between the use of evaluation for political reasons as accountability requirements continue to increase and evaluation for the sake of learning. He emphasizes the need for increased communication skills on the part of the evaluator throughout the process, but, most important, when insisting the results are used appropriately.

Strengths and Weaknesses

Patton's background is ideal for writing a book of this type. He has been involved with evaluation in a variety of settings, including formal and non-formal educational programs. He uses this to his advantage throughout the text, drawing on his extensive experience to bring ideas to life through examples multiple audiences can relate to.

In addition, he uses parables and stories at the beginning of each chapter to encourage higher level thinking by the reader prior to approaching the in-depth material. For example, at the beginning of a chapter on intended process use, Patton (2008) tells a story about a known smuggler traveling across the border by donkey. They search him every time, but never find anything illegal and let him pass. Years later, the border guard runs in to him and asks what he could possibly have been smuggling. He informs the guard he had been smuggling illegal donkeys. This parable emphasizes the need to keep the big picture in sight when working towards intended impact.

While engaging, these parables do not always directly apply to the subject matter. They also add to the length of the book. At 636 pages, it is a bit overwhelming to the casual reader. Those interested in a brief overview of utilization-focused evaluation will find it difficult to locate answers quickly and efficiently. The chapters are written to be read in their entirety and cannot be easily skimmed to absorb the general content. This is not a book for someone who hopes to gain insight into utilization-focused evaluation in one sitting.

Current Extension Use

While this book may not be for those needing a quick overview, it is a good resource on how to conduct high-quality evaluations. Extension professionals are being asked to evaluate multiple programs annually (Rennekamp & Arnold, 2009). The June 2009 issue of the Journal of Extension focused solely on evaluation as it exists and is being used within Extension. Throughout that issue, the ongoing debate on how Extension should be using evaluation continues. While some feel Extension should evaluate to prove programmatic worth, others feel evaluation should discover new information and improve programs (Rennekamp & Arnold). Patton (2008) asks the question: Why can't it do both? Patton encourages the creation of evaluations that offer new information and improve programs while serving stakeholders in the process. In addition, the useable information can provide evidence of programmatic worth for accountability purposes.

While Patton's book will assist Extension professionals in becoming better user-oriented evaluators, it is safe to say many are already headed in this direction. In fact, throughout the Journal of Extension, numerous programmatic evaluations show results that assist in future program enhancement and also serve accountability requirements (Braverman & Engle, 2009; Jayaratne, Harrison, & Bales, 2009; Jones, Jacobs, Yarrow et al., 2008). In order to continue this effort, new Extension professionals need to be educated and knowledgeable about utilization-focused evaluation, and Patton's book is an excellent resource for this endeavor.

References

Barker, B. S. (2007). Stakeholders' input on 4-H science and technology program area exploratory study. Journal of Extension [On-line], 45(2) Article 4RIB6. Available at: http://www.joe.org/joe/2007april/rb6.php

Braverman, M. T., Engle, M. (2009). Theory and rigor in Extension program evaluation planning. Journal of Extension [On-line], 47 (3). Available at: http://www.joe.org/joe/2009june/a1.php

Jayartne, K. S. U., Harrison, J. A., & Bales, D. W. (2009) Impact evaluation of food safety self-study Extension programs: Do changes in knowledge relate to changes in behavior of program participants. Journal of Extension [On-line], 47(3) Article 3RIB1. Available at: http://www.joe.org/joe/2009june/rb1.php

Jones, W. D., Jacobs, K. M., Yarrow, G. K., & McPeake, R. (2008). Using workshops to educate landowners about developing natural resource enterprises to diversity income on the family farm. Journal of Extension [On-line], 46(5) Article 5FEA4. Available at: http://www.joe.org/joe/2008october/a4.php

Patton, M. Q. (2008). Utilization-focused evaluation. Thousand Oaks, CA: Sage Publications, Inc.

Rennekamp, R. A., & Arnold, M. E. (2009). What progress, program evaluation? Reflections on a quarter-century of Extension evaluation practice. Journal of Extension [On-line], 47(3) Article 3COM1. Available at: http://www.joe.org/joe/2009june/comm1.php

Warner, P. D. (1983). Utilizing results. Journal of Extension [On-line], 21(5). Available at: http://www.joe.org/joe/1983september/83-5-tt11.pdf

Zhai, L., & Scheer, S. D. (2002). Influence of international study abroad programs on agricultural college students. Journal of International Agricultural and Extension Education, 9(3), 23-29.