A Tool for Developing Questionnaire Content

Identifying the content of data collection instruments can be one of the most challenging aspects of conducting a community survey or needs assessment. This article describes a tool used by CD-DIAL (Community Development--Data Information and Analysis Laboratory). CD-DIAL is an Iowa State University Extension unit that provides technical assistance and training to non-profit and government organizations attempting to collect primary data from their constituents, clients, or organization members.

The Goal

In our work with organizations, our primary goal is to assist the organization in collecting valid, reliable information. Our second goal is that the information will be used to make decisions leading to enhanced community or organizational capacity. As we begin new projects, we often see examples of past surveys and focus groups results that were never used for program or community improvement. Completion of the survey became the end of the process, instead of the first step in community or organizational improvement.

Because we work with communities and organizations on a variety of topics in which we do not necessarily have expertise, we rely on the knowledge and expertise of local members. In order to build on this local expertise, we facilitate a dialogue with a team of community/organization members using three statements that provide the information needed to advise the community/organization on survey methodology.

The Setting

The community or organization brings together a team of 10 to 20 persons. Members of the team are from organizations requesting the assistance and often include others such as clients, users of services, and citizen representatives to provide a broader range of perceptions and concerns. The meeting room is comfortable; chairs are arranged around a table so that participants can see each other and the facilitator.

The facilitator introduces the process and provides each participant with a worksheet containing three statements. Team members are given approximately 5 minutes to complete each statement, and responses are written on a flip chart. The atmosphere is open and nonjudgmental. Participants are encouraged to explain and discuss their statements without criticizing or evaluating others because the primary purpose of the process is to understand the range of issues and concerns that need to be addressed.

The Statements

Three statements are used so that the team and facilitator can reach a shared understanding of the team's vision for the future, perceptions of how the information will be used, and a set of predictions about data collection results (Figures 1, 2, and 3).

Figure 1.
Statement Eliciting Team's View of Where the Community Will Be in the Future

In this first statement, we explore a basic understanding of the team's view on where the community should be at some point in the future. Note that the statement is written in present tense, as though the date has arrived and the condition exists. An example of statements we have seen generated from this first exercise is "In the year 2005, Pottawattamie County children have increased access to quality early childhood education programming and are ready to learn when they enter school."

Figure 2.
Statement Eliciting Team's View of How They Will Achieve the Ideal Condition

The second statement assists the team in exploring potential strategies that could lead to community or organizational change. Note that "conducting a survey" is not a satisfactory answer to this statement. As participants share their phrases, discussion often ensues regarding methods to be explored and challenges to existing notions about boundaries on personal and organizational change. In response to this statement, we have seen phrases such as "Our team will create the ideal condition for residents by breaking down the barriers to accessible and understandable services, or teaching residents about a healthy lifestyle."

Figure 3.
Statement Eliciting Team's View of What the Survey Will Show

Much like the popular television show that asks teams of participants to compete in predicting survey results, this statement provides a list of predictions or hypotheses that drive the development of survey instruments and help to clarify the target population for the data collection process. Some examples of phrases resulting from this statement are "Seniors have transportation problems," and "Residents are not aware of county conservation facilities."

These statements also are useful in the summaries we provide to organizations when data collection is complete. We use survey results to either confirm or disprove beliefs about the target population.

In our work with community teams, we frequently find examples of professional bias, often when team members work exclusively with certain populations. Responses to this third statement allow identification and exploration of those biases, leading to better question construction. For example, in one session a participant predicted the survey would show that single parents who work did not really want to be at home, raising their children. This participant believed that single parents should be home doing exactly that. The process allowed members of the team to discuss the participant's perceptions and clarify the team's goals for program development.

The Outcome

In our work with over 100 communities and organizations, asking teams to complete these statements has typically resulted in several pages of phrases. Phrases are compiled and shared with the team as part of the survey development process. The exercise has assisted in clarifying the target population for the survey and team member training regarding sound question construction and survey methodology. Use of the exercise has consistently provided our clients and our staff with a basic framework for building and implementing a well-designed survey process that results in useful data and often results in positive community or organizational change.