December 1999 // Volume 37 // Number 6 // Ideas at Work // 6IAW1

Previous Article Issue Contents Previous Article

How to Evaluate a Conference Informally with "Listening Posts"

Abstract
Extension agents often ask, "Isn't there an informal but somewhat structured way to get feedback at a conference or workshop without using a survey?" This article describes the use of 'Listening Posts' and the author gives a number of practical tips for putting this qualitative strategy to use. Benefits include: quality feedback, high participation and enthusiastic support from conferees and the chance to build program ownership among conference workers. Deficits: could exclude very shy persons or result in information most salient to participants.


Nancy Ellen Kiernan
Program Evaluation Specialist
College of Agricultural Sciences
Pennsylvania State University
University Park, Pennsylvania
Internet address: nekiernan@psu.edu


Extension staff often ask, "Isn't there an informal yet structured way to get feedback at a conference without using a survey?" One technique is to create "Listening Posts," a selected group of conference participants who agree to post themselves in the corridors and break rooms at a conference to record on a clipboard what the other participants tell them about the conference as it unfolds.

In the conference's introduction, the chair discusses the evaluation and 1-2 questions related to the conference goals, for example: "We want to know two things: What did you think of the idea of bringing Extension and research together at this conference? What is one idea or suggestion that you found useful for your professional work? Then, tell us anything else about the conference you feel is important for us to know. We asked 12 registered participants to circulate in the halls outside the meeting rooms and record your ideas. These Listening Posts have a blue badge and clipboard. You can't miss them.

"We asked participants, people like yourself, instead of the conference planners, to be the Listening Posts so you'll have no reluctance in talking about your experience at the conference. After you give your ideas to a Listening Post, you receive a sticker with 'Someone Listened to Me' printed on it. Put the sticker on your conference badge to show others you have contributed to the evaluation. The sticker will remind others to participate. After the conference, the feedback will be summarized for planning future programs."

Scientific Benefits of the Listening Post Technique

Array of Participants

Listening Posts insure feedback from many participants and especially from participants who tend to leave early, don't like surveys, and have low reading skill or language problems.

Quality Data

Typically, at the end of a conference when evaluations are often circulated, participants' minds are numb, making it hard to remember one session from another, never mind what was learned. Giving participants a chance to talk to Listening Posts throughout the conference enhances the validity of data.

Oriented to the Client's Perspective

Listening Posts provide conference planners with rich, qualitative insights about all aspects of the conference. They learn what's important to the participants.

Program Ownership

You can build ownership in the program by selecting leaders or important people from your target audience to be Listening Posts. Their participation should increase their interest and concern in the long-term success of the program.

User Friendly

This evaluation strategy puts little response burden on participants. Indeed it can be fun for both participants and Listening Posts. Experience shows in one state that the strategy gets repeated at other conferences due to popular demand.

Scientific Limitations of the Technique

Shy Persons

Participants who are extremely reluctant to talk may not contribute. Work to overcome this possibility in the opening conference remarks by mentioning the need for everyone to contribute their positive and negative feelings at least once.

Selected Information

Participants will reveal some things they learned or found confusing, but obviously they can't discuss everything. Participants will tend to comment on what is most salient to them.

How is this Technique Different from what Extension Agents do all the Time -- listen for Participants' Reactions?

First, because participants in this technique are invited to contribute, the percentage of participants who give their reactions will be far greater than those who would normally react in an ad hoc fashion. Research has shown that many people participate if asked, but are reluctant otherwise.

Second, participants will feel more free to comment because someone other than a planning committee member is recording their reactions. Participants are often reluctant to give criticism directly to people on a planning committee.

Tips for Putting the Listening Post Technique to Use

Have a program committee decide on a schedule for the Listening Posts, who to ask to volunteer, and where to meet at the conference. Training is then done to ensure reliability within and across volunteers regarding two points: (a) Listening Posts are true to their name; they listen, record participants' views but never react to what they hear and (b) except to repeat the broad questions outlined in the conference introduction if requested, Listening Posts do not ask questions.

Have the committee call the Listening Posts in plenty of time and send them a conference agenda so they can review ahead of time, when and where they will circulate at the Conference.