The Skillathon: Program Evaluation Can Be Fun!

Introduction

Program evaluation continues to be a popular topic in this age of accountability. The good news is that the discussion has transitioned from "why" to "how." Extension educators are becoming comfortable using paper and pencil assessments, but oftentimes these instruments do not capture changes in participants' skills. A skillathon can be used as a pre/post assessment tool and is ideal for documenting changes in skills that cannot otherwise be measured using traditional methods. It is especially useful with low literacy and special needs audiences.

The Skillathon as Educational Method

Skillathons are used often as experiential learning tools. A well-designed skillathon provides participants with opportunities to increase subject matter knowledge and skills. They are easy to implement, and participants have fun while learning!

A skillathon is simply a set of individual learning stations that each host a specific activity to complete or problem to solve. Participants learn as they use discovery and reflection to accomplish the task. Skillathons are often designed for several participants to work together. This enhances team building and reinforces communication and cooperative decision-making while increasing knowledge. (See Rusk, Egger, Machtmes, & Richert, 2002; Rusk & Machtmes, 2002 for examples. See the 4H 101 Handbook for information about conducting educational skillathons.)

The Skillathon as Assessment Tool

The skillathon can also be used as an assessment tool. Properly executed, a skillathon provides excellent program evaluation data, gathered in an exciting setting that minimizes "test" anxiety. This tool can virtually eliminate literacy difficulties that often plague paper and pencil assessments.

Like an educational skillathon, an assessment skillathon is a set of individual stations with a specific activity for participants to complete or problem to solve. The station evaluator ensures participants understand the instructions, and scores individuals based upon predetermined guidelines.

An assessment skillathon is "administered" twice, once at the beginning and again at the end of the program. The first administration (pre-test) provides a baseline measure of skills participants have before a program. The second administration (post-test) will document increases (think positively!) in the same skills. The pre/post data are then compared to reveal changes that may be attributed to program participation.

How to Design an Assessment Skillathon

It is not difficult to implement a fun-filled skillathon. Preparation is critical, however, if the purpose of the skillathon is to collect evaluation data. The following resources are required (examples from RCE Youth Farmstand Program):

1. Program objectives. Evaluations cannot be planned without clearly stated objectives. Objectives should articulate skills participants will gain from the program.


2. Station tasks. Activities at each station should be based upon skills identified in the program objectives.


3. Time. Adequate time is needed for two (2) "administrations" (pre-test and post-test). The length of time each skillathon takes depends upon participants, number of stations, available facilities and personnel, etc.


4. Fun, fun, fun. Incorporate fun elements into the assessment design. This is particularly important when dealing with audiences who may be fearful of terms such as "test" and "evaluation."


5. Station instructions. Clearly written instructions and evaluation criteria (Figure 1) should be available for each station evaluator. Information should include:
   • Supply list

   • Situation description

   • Task description

   • Scoring criteria
   
   Scoring must be easy to understand and should not require a calculator! Participants may appreciate an index card adhered to the table with simple instructions (Figure 2).

Figure 1.
Sample Station Instruction Sheet

Figure 2.
Sample Participant Instruction Card

6. Participant scoring and tracking system. Participants and scores must be tracked accurately using a system that ensures:
• Participant scores are recorded accurately at each station (Figure 1).

• Participants complete each station (Figure 3).

• All scores are recorded in a Master List for ease of analysis (Figure 4).

Figure 3.
Sample Participant Station Tracking Card

Figure 4.
Sample Master Score Sheet

7. Trained station evaluators. The importance of properly trained personnel cannot be over-emphasized. Station evaluators must understand the activity and evaluation and scoring criteria. Evaluators must know how to assist (or not) participants. It is strongly suggested the same individual serve as station evaluator for both pre-test and post-test skillathons to ensure scoring continuity.


8. Station supplies. Items will be needed to complete station tasks. The Information and Score Sheet (Figure 1) should include a list of required items. A "pilot" test of each activity is strongly suggested to ensure that all required items are available in suitable quantities.

Data Management and Analysis

Scores should be transferred from the Information and Score Sheets (Figure 1) to a Master Score Sheet (Figure 4) immediately after each skillathon. The data should then be entered into a spreadsheet or statistical software analysis program. There will be two (2) sets of identical variables (pre-test, post-test) for each participant. The easiest way to differentiate the two sets of variables is with descriptive variable labels, for example, scale1 (pre-test) and scale2 (post-test), register1 and register2, etc.

Once the data are entered and checked for accuracy, the best way to reveal whether statistically significant differences exist between pre-test and post-test scores is to use a paired T-test analysis. This test compares the averages of the pre-test and post-test variables. It is beyond the scope of this article to provide detailed instructions on data analysis, but several easy to understand books are available commercially, and many universities have staff to help with data analyses.

When to Use an Assessment Skillathon

A skillathon assessment can be used for programs in all disciplines and with a variety of audiences. Figure 5 provides general guidelines to help you decide if a skillathon would be appropriate for your program.

Figure 5.
When to Use an Assessment Skillathon

The size of the group should also be considered. Large groups should be divided, with perhaps one group engaged in a separate learning experience while the other group participates in the skillathon.

Conclusion

Extension educators have used educational (and fun!) skillathons successfully for decades. The skillathon also lends itself readily for use as an evaluation instrument. An assessment skillathon may be the perfect tool for Extension professionals in search of innovative methods for capturing changes in participants' skills. Who says evaluation has to be boring?

References

Cooperative State Research Education and Extension Service , USDA. (2003). 4-H 101 handbook [On-line]. Available at: http://www.national4-hheadquarters.gov/library/4H101

Rusk, C. P., Egger, T., Machtmes, K., & Richert, B. T. (2002). Junior pork day: A family experience. Journal of Extension [On-line], 40(4). Available at: http://www.joe.org/joe/2002august/iw3.shtml

Rusk, C. P., & Machtmes, K. (2002). Teaching youth through 4-H animal science workshops. Journal of Extension [On-line], 40(5). Available at: http://www.joe.org/joe/2002october/iw7.shtml