February 2008 // Volume 46 // Number 1 // Tools of the Trade // 1TOT3

Previous Article Issue Contents Next Article

A User-Friendly Evaluation Resource Kit for Extension Agents Delivering Financial Education Programs

Inadequate evaluation tools and limited evaluation capacity prevent many Extension agents from effectively assessing program impact. A user-friendly and reliable resource kit is now available to help agents evaluate their financial education programs. This resource kit has an online evaluation manual and a database. The manual is available to help educators understand basic evaluation concepts and learn how to use the database. The database is available to help agents design customized evaluation instruments based on their specific program needs. A reliable evaluation instrument can be created within about 10 minutes.

K. S. U. Jayaratne
State Leader for Program Evaluation and Assistant Professor
Department of Agricultural and Extension Education
North Carolina State University
Raleigh, North Carolina

Angela C. Lyons
Associate Professor
University of Illinois at Urbana-Champaign
Urbana, Illinois

Lance Palmer
Assistant Professor
Department of Housing and Consumer Economics
The University of Georgia
Athens, Georgia


Financial education programs play an important role in helping people achieve their financial well-being (Gorham, DeVaney, & Bechman, 1998). Cooperative Extension provides educational programs to address these issues. The Family and Consumer Sciences Extension agents deliver financial education programs at the local level. These programs vary with the financial needs of local citizens.

There are two main barriers agents face when trying to document the impact of financial education programs. The first is that many agents do not have the necessary evaluation knowledge and skills. The second is that user-friendly and reliable evaluation tools do not exist to help them assess the impact of their programs (Lyons, Palmer, Jayaratne, & Scherpf, 2006). To address these barriers, the National Endowment for Financial Education (NEFE) provided funding to create a resource kit that could be used to evaluate financial education programs.

An evaluation toolkit was subsequently developed by the authors. This tool kit was tested with Extension agents and found to be user-friendly and reliable for designing instruments to evaluate financial education programs.

Evaluation Resource Kit

The resource kit is available online to Extension agents at: <http://www.nefe.org/eval/license_agreement.html>. The toolkit has two components, an evaluation manual and an online database for designing customized evaluation instruments.

The Evaluation Manual

The purpose of the evaluation manual is to help Extension agents understand basic evaluation concepts and apply those concepts to document program impact. There are five parts to the manual. The first two parts provide an overview of financial education and the basics of program evaluation. The third part explains the evaluation planning process and the identification of impact indicators. The fourth part explains how to use the database to design evaluation instruments. The final part explains the processes associated with collecting, summarizing, and presenting evaluation data. The manual is available online and in printable form.

The Online Evaluation Database

The online database is a user-friendly resource for designing customized evaluation instruments. Users select from one of four specific evaluation formats as well as from a pool of impact indicators. Users can also add their own questions and statements and are not limited to the indicators provided. The indicators are categorized under six major financial themes. Within these six themes, the indicators are further categorized under sub-themes, and the user has the option to select questions from the topics of interest.

Use of the Evaluation Resource Kit

To access the toolkit, the user first needs to accept NEFE's license agreement. Once the user clicks on agree, a new Web page opens, and the user is directed to the evaluation manual and database. It is advisable to review the manual before using the database. The online version of the manual is ideal for quickly reviewing needed content.

When the user clicks on the database, the process for creating a customized evaluation instrument begins. There are six to 10 steps to designing an evaluation tool, depending on the type of options selected by the user. The first step is deciding which evaluation option is best. There are four choices, "post evaluation only" for one-time short presentations, "pre and post evaluation" for one-time relatively long workshops, "stages to change evaluation" for multi-session programs, and "train-the-trainer evaluation" for instructor trainings.

Once the appropriate evaluation option is selected, a new Web page opens, and the user is prompted to enter general program information such as the name of the program and organization. At this stage, the user also has the option to create an information-recording sheet for the program and a follow-up evaluation instrument.

The user then moves the curser to the "Next" button and clicks on it. From here, the subsequent steps vary with respect to the type of evaluation option selected. For example, the "post evaluation only" and the "pre and post evaluation" options have 10 steps. The "post evaluation only" option documents participants' levels of knowledge, skills, and aspirations at the end of the program. The "pre and post evaluation" option captures actual changes in participants' knowledge, skills, and intended behavior. If the follow-up option is selected, participants' actual behavior change can be documented.

The "stages to change evaluation" option has nine steps and is appropriate for documenting participants' progress towards actual behavior change. The "train-the-trainer evaluation" option has 10 steps and documents the impact that the training has on instructors' subject matter knowledge, training skills, personal financial management skills, and confidence levels in delivering the program.

For all of the above options, a user can add to an evaluation instrument questions that capture qualitative and demographic information. Once the user is familiar with the evaluation database, and has a clear understanding of the program objectives, he or she is able to design a customized and reliable evaluation instrument within about ten minutes.

Applications and Recommendations

Overall, this evaluation toolkit helps Extension agents delivering financial education create reliable evaluation instruments. In using this resource, agents can do a better job of evaluating their programs. The train-the-trainer evaluation option of the database is appropriate for financial education Extension specialists to assess their in-service training programs. The manual contains general evaluation information that can be used by any Extension professional who wants to develop a better understanding of program evaluation.


This project was funded by the National Endowment for Financial Education® (NEFE®). Project number 001-10-2003


Gorham, E. E., DeVaney, S. A., & Bechman, J. C. (1998). Adoption of financial management practices: A program assessment. Journal of Extension [On-line], 36 (2). Available at: http://www.joe.org/joe/1998april/a5.html

Lyons, A. C., Palmer, L., Jayaratne, K. S. U., & Scherpf, E. (2006). Are we making the grade? A national overview of financial education and program evaluation. The Journal of Consumer Affairs, 40 (2), 208-235.