Use of Interactive Electronic Audience Response Tools (Clickers) to Evaluate Knowledge Gained in Extension Programming

Introduction

Extension programming is built on the principle of providing education that impacts knowledge, behavior, or condition (Figure 1). Program evaluation, and in particular, measuring impact can be challenging as evaluation response rate and ease of data analysis may serve as barriers to educators engaging in program evaluation (Lekies & Bennett, 2011). Moreover, as changes in behavior and condition are often the focus of program evaluation, short-term outcomes such as changes in knowledge can be overlooked in the evaluation process. To truly measure knowledge change, baseline knowledge of program participants must be established. The burden associated with collecting baseline data often results in educators substituting Likert scale questions into end-of-meeting evaluations to gauge how much knowledge the participant feels that they gained as a result of the program. While this perceived change in knowledge can be useful to educators, it still does not effectively measure change in knowledge.

Figure 1.
Logic Model Used for Extension Program Development and Evaluation (Taylor-Powell & Henert, 2008).

Inputs		Outputs			Outcomes – Impact
		Activities	Participation		Knowledge (Short Term)	Behavior (Medium Term)	Condition (Long Term)

One potential and perhaps less invasive method for collecting pre- and post-program participant knowledge is through the use of interactive electronic audience response tools (clickers). In an attempt to broaden the means of data collection, Parmer, Parmer, and Struempler (2012) reported on the potential relevance of clicker use in Extension programming. Likewise, Bird and McClelland (2010) identified that clicker portability was well suited for Extension and that such technology allows educators to better engage participants as well as permits broader participation in educational settings. Furthermore, when collecting potentially sensitive data, clickers may better facilitate anonymity and increase response rate in a group environment (Ginter, Maring, Paleg, & Valluri, 2013; Carlson, 2014).

Clickers present an opportunity for educators to quickly collect demographic data as well as change in knowledge resulting from a program without the need for end-of-session, written surveys that often result in suboptimal response rates. Moreover, the accompanying clicker software (Turning Point 5; Turning Technologies, Youngstown, Ohio) may be easily evaluated by individual session or aggregated across many programs.

The use of clickers in a formal educational setting such as college classrooms has become commonplace. However, the potential benefits of clickers when used in Extension programming to document knowledge change as well as participant attitude towards use of this technology is limited. This article aims to address these gaps in understanding.

Clicker Data Collection

A statewide educational series on beef heifer development was conducted at 12 locations across Iowa. Prior to the beginning of each program, clickers were activated and synchronized with the Turning Point 5 software. Clickers were used by all participants, but clicker identification was not affiliated with individual participants.

At initiation of the meeting, as part of the Microsoft® PowerPoint® presentation, a series of six questions, including participant age, size of farming operation, and operational challenges important to heifer development, were posed via Turning Point software. Depending on the question length and whether or not multiple responses were allowed, participants were allowed between 15 and 60 seconds to answer each question with their clicker. The subsequent educational program intermittently asked three questions both before and after key concepts related to heifer development were discussed. Recorded participant answers were later compiled to determine changes in knowledge of those key concepts.

At the end of the presentation, participants used their clickers to respond to the comment "I like providing input using clickers as a partial substitute to written evaluations" on a Likert scale with options of strongly agree, agree, neutral, disagree, or strongly disagree.

Results

In total, there were 309 attendees of the state-wide program, 245 of which were unique participants not affiliated with implementation of the program. The use of the clicker technology was successful in tracking a change in knowledge. Specifically, the correct response rate to baseline questions was 47.1%, and the correct response rate to identical follow-up questions was 87.2% (see Table 1).

Of the 245 participants at the meetings, 100% answered at least 1 clicker question through the Turning Point software and 90.6% (222/245) answered all 16 Turning Point questions using the clicker. In addition, 94.7% (231/244) of participants had a positive attitude towards use of the clicker technology, 4.5% of participants were indifferent to the technology, and only 0.8% did not like the use of clickers during the program.

Conclusion

Data collected during this educational series validated clicker technology as a useful tool for collecting anonymous demographic information and measuring knowledge change of attendees. Furthermore, clicker-based technology yielded was viewed in an overwhelmingly positive manner by participants. Thus, clicker technology is well-suited as an evaluation tool in Extension programming and in particular an effective means by which to track changes in knowledge (short-term outcomes).

References

Bird, C., & McClelland, J. (2010). Have you used clickers in programming? Journal of Extension (On-line), 48(5), Article 5TOT9. Available at: http://www.joe.org/joe/2010october/tt9.php

Carlson, B. (2014). Using Turning Point to conduct an Extension needs assessment. Journal of Extension (On-line), 52(1), Article 1TOT3. Available at: http://www.joe.org/joe/2014february/tt3.php

Ginter, A. C., Maring, E. F., Paleg, B., & Valluri, S. (2013). Using clicker technology with rural, low-income mothers: Collecting sensitive data anonymously. Journal of Extension (On-line), 51(4), Article 4FEA3. Available at: http://www.joe.org/joe/2013august/a3.php

Lekies, K. S., & Bennett, A. M. (2011). The evaluation attitudes and practices of 4-H educators. Journal of Extension (On-line), 49(1), Article 1RIB2. Available at: http://www.joe.org/joe/2011february/rb2.php

Parmer, S. M., Parmer, G., & Struempler, B. (2012). Testing a new generation: Implementing clickers as an extension data collection tool. Journal of Extension (On-line), 50(5), Article 5TOT5. Available at: http://www.joe.org/joe/2012october/tt5.php

Taylor-Powell, E., & Henert, E. (2008). Developing a logic model: Teaching and training guide. Benefits, 3, 22.