October 2007 // Volume 45 // Number 5 // Tools of the Trade // 5TOT1

Previous Article Issue Contents Previous Article

SAMMIE: Using Technology for a One-Stop Program Evaluation Resource

Whether evaluating impact of community-based programs is new to you, or you are an experienced evaluator, SAMMIE can help you expand your skills. SAMMIE represents Successful Assessment Methods and Measurement In Evaluation. It is a one-stop Web portal to valuable impact documentation resources. Through SAMMIE you can: [1] Access resources on 21 evaluation related topics; [2] Read the best literature on the Web related to program evaluation; [3] Ask an Expert your questions about program evaluation; and [4] Develop a personalized program evaluation plan. SAMMIE is available free of charge to anyone who has Web access Go to: <http://www.sammie.osu.edu>

Thomas M. Archer
Leader, Program Development and Evaluation
Ohio State University Extension
Columbus, Ohio

Karen Bruns
Ohio State University Extension
Columbus, Ohio

Catherine A. Heaney
Associate Professor, Teaching
Stanford University Prevention Research Center
Stanford, California

A Timely Approach to Extension Program Evaluation Education

The day of the Extension, 2-day, statewide, evaluation in-service workshop has long passed. No, wait--lengthy, evaluation in-services have always been limited! Face-to-face professional development sessions on any topic are becoming antiquated. But the fact is that the vast majority of Extension program personnel have NEVER been overly interested in developing formal evaluation skills.

There are several reasons for this. First, most Extension employees are hired because of their discipline expertise and their interest in teaching. Evaluation is not a part of their academic arsenal, and it may not be as high of a priority when it comes to professional development.

Then there is the ever-present, and sometimes oppressive, issue of budget constraints. Traveling halfway across the state for a 5-hour professional development workshop takes financial resources. Smaller budget mean fewer people, which means those who are left do more--programming, not evaluation.

But a new era exists for those conducting programs with public or foundation funds. No longer can one expect continued financial support just by doing good work. One must now show results. Even though there may be little interest, time, or money to acquire adequate program impact evaluation skills necessary to sustain their "doing good work," more and more Extension Educators are realizing the importance, and even the necessity, of acquiring impact evaluation skills.

Distance Education and the Web have provided alternatives to face-to-face in-services for sharing knowledge and skills. There are millions of resources on the Web, but can program educators afford the time needed to search and verify the legitimacy of these Web sites? For example, a recent Web search on "How to Conduct a Focus Group Interview" yielded 1.98 million hits; a similar search on "How to Design a Questionnaire" netted more than 16.1 million hits.

Portals to the Web

SAMMIE is a one-stop Web site that gives users ready access to impact evaluation resources and tools. As an acronym, SAMMIE stands for Successful Assessment Methods and Measurement In Evaluation.

Whether evaluating the impact of educational programs is new to the potential SAMMIE user, or s/he has evaluated programs and just wants to quickly refresh knowledge on specific evaluation techniques, SAMMIE is the Web portal to help. Visitors to the site are encouraged to do the first six modules in order, but can jump from one module to another based on their specific needs.

As one goes through the modules, the SAMMIE user can:

  1. Access resources. Web-based readings and resources are clustered around the 21 learning modules on topics ranging from ethics in evaluation to designing and using surveys, focus groups, and other evaluation methods.

  2. Read the best and latest literature on the Web related to program planning and evaluation.

  3. "Ask an Expert" questions about program planning and evaluation and engage in discussion forums. These tools provide added information and insight into specific questions the user may have.

  4. Develop a personalized program with an evaluation plan. By using a wiki, colleagues can develop an evaluation plan on-line and open it for feedback and critique from other SAMMIE participants.

  5. Use learning tools, including videos and self-assessments to help expand knowledge.

Learning Modules Available

All these features are available for each of the 21 SAMMIE modules. Below is a list of the module topics:

  1. What Can Evaluation Do for You?

  2. Stages of Evaluation

  3. Getting Everybody on Board

  4. Developing a Logic Model

  5. Asking Evaluation Questions and Setting Standards for Success

  6. Ethical Issues

  7. Evaluation Design I

  8. Evaluation Design II

  9. Data Collection and Analysis Strategies

  10. Quantitative Methodologies--Survey Methodology

  11. Quantitative Methodologies--Questionnaire Design

  12. Quantitative Methodologies--Sampling

  13. Quantitative Methodologies--Descriptive Statistics

  14. Quantitative Methodologies--Techniques for Comparing Groups

  15. Qualitative Methodology--Individual Interviews

  16. Qualitative Methodology--Focus Groups

  17. Qualitative Methodology--Observations

  18. Qualitative Analysis

  19. Interpreting Your Findings

  20. Reporting Your Results

  21. Enhancing the Use of Your Evaluation Results

Potential Use

This Web portal has a multitude of potential uses in program planning and designing/conducting impact evaluations, as well as analyzing, reporting, and using the results of those evaluations.

SAMMIE is available free of charge to anyone who has Web access. Go to <http://www.sammie.osu.edu> to learn more. In just a few minutes potential participants can create an account. After creating an account, participants will want to go through the orientation module, which showcases all the features of the SAMMIE modules and gives an overview of how to use the various features. Check it out!


SAMMIE was made available through The Great Lakes Center for Agricultural Safety and Health, The Ohio State University Extension Program Development and Evaluation Unit, The Ohio State University Office of Outreach and Engagement, OSU CARES/OSU Extension, and as the result of 2003-2006 grant support by cooperative agreement #U50OH008108 from Centers for Disease Control & Prevention.