Tools for Formative Evaluation: Gathering the Information Necessary for Program Improvement

Introduction

Program accountability and improvement are two major functions of Extension evaluation. Program improvement is considered one of the most important tasks of evaluation. However, this function of evaluation has been overlooked by some Extension educators due to their overemphasis on accountability. This situation can be attributed to the increased demand for accountability in Extension services (Hachfeld, Bau, Holcomb, & Craig, 2013).

The program improvement function of evaluation is called formative evaluation (Scriven, 1991). The intent of formative evaluation is to "help form or shape the program to perform better" (Rossi, Lipsey, & Freeman, 2004, p. 34). As the name indicates, formative evaluation makes Extension educators more informed about the changes needed to improve their programs. Formative evaluation helps Extension educators determine what worked well and what went wrong in their educational programs. Also, formative evaluation helps Extension educators determine the changes needed to further improve their programs.

Cooperative Extension services in the United States are experiencing more resource limitations (Monroe, McDonell, Hermansen-Báez, Long, & Zipperer, 2007) and are being forced to optimize their program effectiveness. Under current resource limitations, there is no room for programs to fail. For Extension educators operating under this demanding situation, it is critical to determine all necessary improvements at the pilot stage of programs and to make necessary changes before the next round of programming. This approach prevents potential program failures and increases the cost effectiveness of Extension.

Extension educators are having to develop more effective programs than ever before and should be prepared to better their programs continuously by determining needed improvements. This continuous improvement process contributes to the effectiveness of Extension programs and maximizes program outcomes and impacts. Also, this process contributes to improving the professional outlook of Extension educators and to achieving client satisfaction. The latter factor is a desirable condition for gaining public support for Extension programs.

Normally, new Extension educators experience a steep learning curve when developing Extension programming. They have to learn educational programming within a very short period of time and without failing. New Extension educators can accomplish this task if they pay due attention to formative evaluation and use formative evaluation as a helpful tool in determining the changes needed to further improve their educational programs.

The purpose of this article is to discuss how to develop a formative evaluation tool useful for gathering the information needed for program improvement.

Information Needed for Formative Evaluation

All the information needed for formative evaluation can be categorized into three broad groups: (a) negative factors associated with the program, (b) positive factors associated with the program, and (c) the changes needed for further improvement of the program.

Gathering Information About Negative Factors

If an Extension educator is aware of the weaknesses, problems, and barriers associated with a program, he or she can find alternatives to overcome those issues in the next round of programming. For this reason, it is important to ask questions to identify weaknesses, problems, and barriers associated with a program. The following questions can be posed to program participants to gather this information:

• What do you consider to be the most significant weakness of the program?
• Did you meet your learning expectations from this program? (yes or no)
• If you did not meet your learning expectations, why not?

Gathering Information About Positive Factors

If an Extension educator is aware of the strengths of an educational program, he or she can capitalize on those strengths when building future programs. The following questions are helpful for gathering information about program strengths:

• What do you consider to be the most significant strength of the program?
• Which part of the program did you like the most?

Gathering Information for Making Needed Changes

Program participants are the best group from whom to solicit suggestions for further improvement of any educational program. Adult audiences, especially, are capable of identifying what needs to be done to improve a program. For this reason, Extension educators can gather useful information from participants for further improvement of their programs. The following questions can be used to gather such information:

• What do you suggest could be done to improve this program?
• What would be the best way to spread the word about this program to potential participants in future?

In addition to the aforementioned questions, provide participants with scaled items for assessing their satisfaction with the program (see Figure 1).

Practical Application

To design more useful evaluation tools, Extension educators should include formative evaluation questions in addition to outcome evaluation questions on evaluation instruments. In assessing Extension programs, questions such as those suggested above can be used to gather formative information for making changes necessary for program improvement. Such questions help Extension educators determine the strengths and weaknesses associated with their programs. When Extension educators are aware of the strengths of their programs, they can capitalize on those strengths to overcome weaknesses, thereby building better programs for the future. Additionally, when Extension educators have information that helps them identify changes needed for program improvement, they can make informed decisions about necessary program revisions.

References

Hachfeld, G. A., Bau, D. B., Holcomb, C. R., & Craig, J. W. (2013). Multiple-year Extension program outcomes and impacts through evaluation. Journal of Extension [Online], 51(1) Article 1FEA2. Available at: http://www.joe.org/joe/2013february/a2.php

Monroe, M. C., McDonell, L., Hermansen-Báez, L. A., Long, A. J., & Zipperer, W. (2007). Building successful partnerships for technology transfer. Journal of Extension [Online], 45(3) Article 3TOT6. Available at: http://www.joe.org/joe/2007june/tt6.php

Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach (7th ed.), p.34. Thousand Oaks, CA: Sage Publications.

Scriven, M. (1991). Evaluation thesaurus (4th ed.). Newbury Park, CA: Sage Publications.