SAMMIE: Using Technology for a One-Stop Program Evaluation Resource

A Timely Approach to Extension Program Evaluation Education

The day of the Extension, 2-day, statewide, evaluation in-service workshop has long passed. No, wait--lengthy, evaluation in-services have always been limited! Face-to-face professional development sessions on any topic are becoming antiquated. But the fact is that the vast majority of Extension program personnel have NEVER been overly interested in developing formal evaluation skills.

There are several reasons for this. First, most Extension employees are hired because of their discipline expertise and their interest in teaching. Evaluation is not a part of their academic arsenal, and it may not be as high of a priority when it comes to professional development.

Then there is the ever-present, and sometimes oppressive, issue of budget constraints. Traveling halfway across the state for a 5-hour professional development workshop takes financial resources. Smaller budget mean fewer people, which means those who are left do more--programming, not evaluation.

But a new era exists for those conducting programs with public or foundation funds. No longer can one expect continued financial support just by doing good work. One must now show results. Even though there may be little interest, time, or money to acquire adequate program impact evaluation skills necessary to sustain their "doing good work," more and more Extension Educators are realizing the importance, and even the necessity, of acquiring impact evaluation skills.

Distance Education and the Web have provided alternatives to face-to-face in-services for sharing knowledge and skills. There are millions of resources on the Web, but can program educators afford the time needed to search and verify the legitimacy of these Web sites? For example, a recent Web search on "How to Conduct a Focus Group Interview" yielded 1.98 million hits; a similar search on "How to Design a Questionnaire" netted more than 16.1 million hits.

Portals to the Web

SAMMIE is a one-stop Web site that gives users ready access to impact evaluation resources and tools. As an acronym, SAMMIE stands for Successful Assessment Methods and Measurement In Evaluation.

Whether evaluating the impact of educational programs is new to the potential SAMMIE user, or s/he has evaluated programs and just wants to quickly refresh knowledge on specific evaluation techniques, SAMMIE is the Web portal to help. Visitors to the site are encouraged to do the first six modules in order, but can jump from one module to another based on their specific needs.

As one goes through the modules, the SAMMIE user can:

1. Access resources. Web-based readings and resources are clustered around the 21 learning modules on topics ranging from ethics in evaluation to designing and using surveys, focus groups, and other evaluation methods.


2. Read the best and latest literature on the Web related to program planning and evaluation.


3. "Ask an Expert" questions about program planning and evaluation and engage in discussion forums. These tools provide added information and insight into specific questions the user may have.


4. Develop a personalized program with an evaluation plan. By using a wiki, colleagues can develop an evaluation plan on-line and open it for feedback and critique from other SAMMIE participants.


5. Use learning tools, including videos and self-assessments to help expand knowledge.

Learning Modules Available

All these features are available for each of the 21 SAMMIE modules. Below is a list of the module topics:

1. What Can Evaluation Do for You?


2. Stages of Evaluation


3. Getting Everybody on Board


4. Developing a Logic Model


5. Asking Evaluation Questions and Setting Standards for Success


6. Ethical Issues


7. Evaluation Design I


8. Evaluation Design II


9. Data Collection and Analysis Strategies


10. Quantitative Methodologies--Survey Methodology


11. Quantitative Methodologies--Questionnaire Design


12. Quantitative Methodologies--Sampling


13. Quantitative Methodologies--Descriptive Statistics


14. Quantitative Methodologies--Techniques for Comparing Groups


15. Qualitative Methodology--Individual Interviews


16. Qualitative Methodology--Focus Groups


17. Qualitative Methodology--Observations


18. Qualitative Analysis


19. Interpreting Your Findings


20. Reporting Your Results


21. Enhancing the Use of Your Evaluation Results

Potential Use

This Web portal has a multitude of potential uses in program planning and designing/conducting impact evaluations, as well as analyzing, reporting, and using the results of those evaluations.

SAMMIE is available free of charge to anyone who has Web access. Go to <http://www.sammie.osu.edu> to learn more. In just a few minutes potential participants can create an account. After creating an account, participants will want to go through the orientation module, which showcases all the features of the SAMMIE modules and gives an overview of how to use the various features. Check it out!

Acknowledgment

SAMMIE was made available through The Great Lakes Center for Agricultural Safety and Health, The Ohio State University Extension Program Development and Evaluation Unit, The Ohio State University Office of Outreach and Engagement, OSU CARES/OSU Extension, and as the result of 2003-2006 grant support by cooperative agreement #U50OH008108 from Centers for Disease Control & Prevention.