Using Clicker Technology with Rural, Low-Income Mothers: Collecting Sensitive Data Anonymously

Introduction

This article is based on a multi-state study of health message development with rural mothers using an empowerment framework. The projected was funded through the USDA National Institute of Food And Agriculture's (NIFA) Rural Health and Safety Grant Program. A multi-year, multi-state, USDA-funded study known as "Rural Families Speak" (RFS) (2010-46100-21791) provided the research base for the study reported here. Researchers were interested in transferring knowledge gained from RFS findings to the development of constructive health messages for rural mothers.

Clickers are a relatively new technology, particularly within the realm of Extension, university classrooms, and other community groups. Officially known as "audience response systems," these handheld devices are linked to Power Point software that allows an audience to "click" an answer to a particular question (e.g., "In general, would you say your health is: 1) Excellent, 2) Very good, 3) Good, 4) Fair, 5) Poor"). All participants' answers are displayed on the screen for everyone to view. Beyond the advantage of audience viewing, this technique preserves the responses as research data for analysis with additional software. In the view of the research team, other benefits of clickers include the ability to track individual responses, poll anonymously, and ensure general understanding of a particular idea. Clickers are becoming popular in classrooms, and scientists have seen advantages to use in research settings as well. This article presents a case study of how clickers were used to engage participants in crafting physical health messages.

Literature Review

Clicker Technology in the Classroom

To date, clickers have frequently been used in college classrooms, particularly in large freshman survey courses. In part, this is attributable to the recognition that college students belong to a tech-savvy generation (the "Net Generation," Robinson, 2006) and may respond well to an electronic medium for discussion (Bode, Drane, Kolikant, & Schuller, 2009). Using clickers allows all students, whether there are 20 or 500, to make their "voice" heard while maintaining anonymity (Morse, Ruggieri, & Whelan-Berry, 2010). This is especially useful for more reserved students. In fact, the ensured anonymity of clickers may be a large draw for students and may guarantee greater amounts of feedback (Jones, Connolly, Gear, & Read, 2001; Sharma, Khachan, Chan, & O'Byrne, 2005; Beekes, 2006; Freeman, Blayney, & Glinns, 2006) than other methods that require students to speak in front of the class.

Some educators are hesitant to use clickers, wondering whether use will take up valuable instruction time (Morse, Ruggieri, & Whelan-Berry, 2010) or if they truly aid in learning (Lantz, 2010). However, feedback from students is generally positive (Cue, 1998; Graham, 2007; Lantz, 2010), reflecting the belief that clickers actually help them learn in the classroom. One caveat that Morgan (2008) presented was that students might be unaccustomed to this type of interaction with their professors; they may be used to more face-to-face communication.

Using Clickers for Research

Clickers are being used to gather data in multiple contexts. Waltz, Maniccia, Bryde, Murphy, Harris, and Waldenmaier (2010) used clickers to train public health professionals in New York State about emergency preparedness following September 11, 2001 and the subsequent anthrax attacks. They pointed out that when participants are coming from diverse professional backgrounds or discussing sensitive material, clickers "can be a mechanism for equalizing the audience by allowing all participants to provide input while protecting their anonymity" (p. 66).

Extension's Support of Clicker Usage

Bird and McClellan (2010) make a case for using clickers, explaining that participants often desire to view others' responses. By seeing other participants' answers, they are more likely to feel involved with the group and are thus encouraged to participate further. Moreover, the authors note that by using clickers, participants can engage in rich discussion about their answers. In the case of our study, we wanted participants to see, discuss, and understand the differences or similarities of their viewpoints before they engaged in the creation of physical health messages. Participants noted when they had similar or varying responses to specific questions, and they often discussed their reasons for a particular response.

The use of clickers in focus groups provides an opportunity for discussion, once everyone has cast their vote. Also appealing, votes are kept anonymous, which is especially attractive if there is a personal question being asked (Gustafson & Crane, 2005). In its effort to enhance educational programs, Extension relies on local information, which is often personal. For example, in our study, focus group facilitators asked about participants' level of confidence in filling out medical and dental forms. This could be perceived as private or embarrassing. Therefore, it was useful to have a method of gathering data in a group setting that was both anonymous and that displayed the information to everyone and elicited further discussion.

Empowerment Theory

First proposed by the seminal theorist Rappaport (1987), empowerment theory provides a context for understanding women's agency, control, and self-efficacy. From this perspective, the mothers become the experts when discussing their real-life issues. In asking them to use their real-life experience in developing health messages, we acknowledged the mothers as collaborators (Rappaport, 1987). We used a participatory process (Bergsma, 2004) by encouraging the mothers to create the message that was most appealing to them. By playing an active role in the development of health messages, mothers may gradually feel empowered to make good health-related decisions, and perhaps this will extend to their outlook on life. 

Methods & Sample

In 2011, eight focus groups were conducted in the states of Hawaii, Iowa, Maryland, Massachusetts, South Dakota, Tennessee, and Washington. Participants were rural, low-income, ethnically diverse mothers. Each mother had at least one child under the age of 12, with an income of 185% or less of the federal poverty level. Thirty-six mothers participated. Focus groups provide the opportunity for participants to learn from each other and produce reactions to core messages that may differ from individual interviews.

Procedure

Pre-Focus Group Preparation

Developing questions to be answered with clickers was a team effort. There were several components of the question set, including the general tone and clarity, and whether or not the questions seemed biased. These components were addressed in several settings: in the research team weekly meetings, during a multistage review process, and through the facilitator training. The questions were pilot-tested with two focus groups, which led to question refinement. The resulting protocol was approved by each state university's IRB.

To use clickers with our focus groups, it was essential that our research team include at least one member with clicker experience. Our Extension Learning Technologist demonstrated clicker technology to our multi-state team, including an overview of how clickers extract useful information and general utility. The technologist also facilitated the pilot-test focus group that preceded the multi-state training and was available to consult with state facilitators regarding technical issues.

The research team held one training session using a Web-based distance technology with facilitators from the 10 initial participating states (Hawaii, Iowa, Maryland, Massachusetts, North Carolina, South Dakota, Tennessee, and Washington) to ensure comfort with the clickers. (Due to extenuating circumstances, only eight states conducted focus groups.) Following the training session, research team members held follow-up training with individual state facilitators as needed. The research team created a document with troubleshooting questions, which was distributed to all facilitators.

Focus Groups

In the eight focus groups, facilitators distributed the hand-held clickers, then explained their use, and provided two warm-up questions to allow participants to practice how to use the clickers. Participants were told that after being asked a particular question (e.g., "How comfortable are you with filling out dental or medical forms on your own?"), they would use the clicker to select a number 1-6 (low numbers = not comfortable; high numbers = comfortable). That number would indicate their individual response. After everyone had replied, their answers would be displayed on a chart. The responses were anonymous: there was no way to link the response to the individual participant.

Questions targeted three main categories:

1. Participants' perceived health and level of health literacy,
2. Participants' perceived level of control over their own and their family members' health decisions, and
3. Appeal of particular health messages

Each of these clicker-answered questions was followed by a discussion. This provided a nice balance of quantitative and qualitative data, which will be reported in other research papers. At times, the facilitators referenced the answers on the screen when encouraging participants to discuss their responses to a particular question or to discuss their experiences more generally. These answers also prepared participants to create their own messages about physical health.

A key concept for use of clickers in focus group research is "equalizing the audience." Although our participants were all rural, low-income mothers with at least one child under the age of 12 and were at least 185% under the federal poverty level, they have unique experiences with regard to family composition, financial and relationship struggles, and physical and mental health. If participants felt shame or embarrassment about their personal challenges, they might have been uncomfortable having their answers linked to them. However, because their answers were anonymous, they might have felt freer to "click" their true answer. In this sense, clickers offer researchers a valuable tool for reaching limited-income audiences. Similarly, Benavente, Jayaratne, and Jones (2009) note that it is imperative to implement educational methods that will better reach marginalized audiences.

Observations

Observers were present in each focus group to record observations using a standardized observation sheet. Following the focus group, all observers and facilitators submitted an observation form. Members of the research team analyzed the observation forms to discuss our methodology, draw conclusions, and state lessons learned.

Lessons Learned and Implications

In conducting the eight focus groups, we learned several lessons about using clicker technology with the participating mothers. These lessons may be applicable to other, similar groups or to groups with different participants. Lessons include:

1. Participants will come to the focus group with varying levels of technology expertise. Care must be taken to ensure they are comfortable with the clickers before beginning the session.
2. Time lapse between staff/facilitator training and focus group session matters. Retention of training information was marginal for state facilitators who did not conduct focus groups until 4-6 weeks after clicker training.
3. Using clickers may dampen facilitators' creative license, as they will be unable to change the order of the interview questions. This must be clearly stated by the trainers to the facilitators. Because ours was a research study, it was important that the questions be asked in order.

Discussion

Using clickers to facilitate focus groups with rural low-income, mothers was an enlightening experience. Given that this group might have been uncomfortable sharing openly about their perceived health and health-related experiences, using clickers enabled participants to share their responses anonymously. The mothers were very interested in seeing the group responses. The immediate graphical output facilitated the discussion and was an advantage for both breaking the ice in the focus groups and enriching discussion needed for the greater study of health message development. We used an empowerment framework to guide our study of rural, low-income mothers' perceptions of health and health messages. As such, we hope the participants were empowered to discuss their perceptions of personal health issues, amount of control they possessed over health care decisions, and preference for particular health messages in a relatively safe environment.

As with all research, there are a number of limitations. First, there is the lack of a body of knowledge in the literature regarding use of clicker technology with rural, low-income mothers and as a research technique. Thus, our study pioneered the technique with the targeted audience. Another limitation was the skill set of the facilitators and the time lapse between clicker training and focus groups. Retention of understanding between training and conducting the focus groups was marginal among those whose focus groups occurred 4-6 weeks after training, as evidenced by missing or disorganized data (e.g., questions skipped or asked out of order). In order to follow the protocol, it was crucial that facilitators not ask questions out of order. Based on data gleaned from observation reports, this restriction may have inhibited the creative license of facilitators. Not all facilitators followed these guidelines, which led to some clicker questions being omitted in the dataset.

The study was also limited by its small sample size. Future research and/or use of clickers as a teaching tool with rural, low-income mothers needs to address the limitations of the study reported here. One way to improve upon the methodology used in the study is to determine how focus group facilitators could be better equipped to teach clicker use to participants and how to troubleshoot clicker problems during focus groups. This kind of facilitation skill set is important for researchers and Extension educators desiring to incorporate clicker technology into their programming.

Conclusion

This article describes a case study of the use of clicker technology to conduct a research study with rural, low-income mothers. We concluded that the clickers did provide data we could use in the larger study and that it met our requirements: easy handling; anonymity for participants' responses; quick display of answers; and ability to store and analyze data. Participants were able to view and comment on each other's answers, thereby allowing us to collect both qualitative and quantitative data. This pioneering case study is adding to the limited body of knowledge about the use of clicker technology in general and in research settings with rural, low-income mothers in particular. Findings from this case study and from future studies will provide the evidence base needed for sound Extension programming.

Acknowledgments

This study was funded by a subcontract from the USDA National Institute of Food and Agriculture's Rural Health and Safety Grant. The grant, number 2010-46100-21791, is titled: "Core Health Messages: A Strategy to Improve the Health and Well-Being of Rural, Low-Income Families."

The authors thank Dr. Bonnie Braun, grant PI, for conceptual and editorial guidance.

References

Beekes, W. (2006). The 'millionaire' method for encouraging participation. Active Learning for Higher Education 7, 25–36. doi: 10.1177/1469787406061143

Benavente, L. M., Jayaratne, K. S. U, & Jones, L. (2009). Challenges, alternatives, and educational strategies in reaching limited income audiences. Journal of Extension [On-line], 32(2) Article 6RIB2. Available at: http://www.joe.org/joe/2009december/rb2.php

Bergsma, L.J. (2004) Empowerment education: The link between media literacy and health promotion. American Behavioral Scientist, 48, 152-164. doi: 10.1177/0002764204267259

Bird, C., & McClelland, J. (2010). Have you used clickers in programming? Journal of Extension (On-line), 48(5), Article 5TOT9. Available at: http://www.joe.org/joe/2010october/tt9.php

Bode, M., Drane, D., Kolikant, Y.B-N., & Schuller, M. (2009). A clicker approach to teaching calculus. Notices of the AMS, 56(2), 253-256. Retrieved from: http://www.ams.org/notices/200902/rtx090200253p.pdf

Cue, N. (1998). A universal learning tool for classrooms? In Proceedings of the "First quality in teaching and learning conference," Kowloon, Hong Kong, China. 10–12, December.

Freeman, M., Blayney, P., & Glinns, P. (2006). Anonymity and in class learning: The case for electronic response systems. Australasian Journal of Educational Technology 22, 568–580. Retrieved from: http://www.ascilite.org.au/ajet/ajet22/freeman.html

Graham, C. R., Tripp, T. R., Seawright, L., & Joeckel, G. (2007). Empowering or compelling reluctant participators using audience response systems. Active Learning in Higher Education, 8(3), 233-258. doi: 10.1177/1469787407081885

Gustafson, C., & Crane, L. (2005). Polling your audience with wireless technology. Journal of Extension [On-line], 43(1) Article 6TOT3. Available at: http://www.joe.org/joe/2005december/tt3.php

Jones, C., Connolly, M., Gear, A., & Read, M. (2001). Group interactive learning with group process support technology. British Journal of Educational Technology, 32, 571–586. Retrieved from: 10.1177/1350507606067173

Lantz, M. E. (2010). The use of 'clickers' in the classroom: Teaching innovation or amusing novelty? Computers in Human Behavior, 26(4), 556-561. doi: 10.1016/j.chb.2010.02.014

Morgan, R. K. (2008) Exploring the pedagogical effectiveness of clickers. Insight: A Journal of the Center for Excellence in Teaching and Learning, 3, 31-36. Retrieved from: http://www.eric.ed.gov/PDFS/EJ888407.pdf

Morse, J., Ruggieri, M., & Whelan-Berry, K. (2010). Clicking our way to class discussion. American Journal of Business Education, 3(3), 99-108. Retrieved from: http://web2integration.pbworks.com/f/Clicking+Our+Way+To+Class+Discussion.pdf

Rappaport, J. (1987). Terms of empowerment/Exemplars of prevention: Toward a theory for community psychology. American Journal of Community Psychology, 15(2), 121-148. doi: 10.1007/BF00919275

Robinson, S. (2006). Using games and clickers to encourage students to study and participate. Cullowhee, 11(2) 25-30.

Sharma, M. D., Khachan, J., Chan, B., & O'Byrne, J. (2005). An investigation of the effectiveness of electronic classroom communication systems in large lecture classes. Australasian Journal of Educational Technology 21, 137–154. Retrieved from: http://www.ascilite.org.au/ajet/ajet21/sharma.html

Waltz, E. C., Maniccia, D. M., Bryde, R. L., Murphy, K., Harris, B. R., & Waldenmaier, M. N. (2010). Training the public health workforce from Albany to Zambia: Technology lessons learned along the way. Public Health Reports, 125, 61-89. Retrieved from: http://www.publichealthreports.org/issueopen.cfm?articleID=2535