Handling Survey Data

The survey is a much-used Extension tool of the trade. Some tricks of the trade in handling data increase validity. Here are some tips for handling: (1) missing data, (2) nonsense or error responses, (3) scaling concerns, (4) "other" categories of responses, and (5) "not applicable" responses.

If more than 20% of the respondents failed to answer a specific item, it's usually best to eliminate that item. If a particular individual fails to complete a substantial part of the survey, eliminate that individual's entire questionnaire. These are just rules of thumb that should be established before conducting the survey. Careful pilot testing to eliminate confusing or low-response items can reduce the problem of missing data.

When respondents give answers that don't make sense or indicate an answer that's not a choice (as in circling a space between choices), eliminate that response. Nonsense data must not be interpreted, but reported verbatim. Don't guess at an individual's meaning.

Learn from your mistakes. For example, if your scale has no NEUTRAL category (SD D A SA) and individuals make their own, next time, include the NEUTRAL choice (SD D N A SA).

The choice to assign zeros and negatives to numbered scales is up to the evaluator. What numbers will best represent your variables and aid interpretation? For example: Will a score of .52 be meaningful? Will a score of -.12 be understood related to attitude or knowledge?

What about the scale's length? One simple consideration is that a scale with more than nine points will need two spaces for each response when data are entered into a computer.

Most respondents choose a portion of the scale's continuum and use it. For example, if the scale is 1-11, then an individual may choose 6 through 9; if the scale is 1-5 that same individual will probably use 3 and 4. Either scale will produce about the same amount of variability.

The dilemma of analyzing "other" category responses needs to be determined after data collection. Choices include: (1) treating all respondents who marked "other" the same, (2) entering each separate response to the "other" category, or (3) combining them into meaningful groupings.

Make the decision based on the importance of knowing how many respondents: (1) marked "other," (2) made specific responses, or (3) responded within a specific grouping. For example, is it important to know that 25% marked the "other" category regardless of what they wrote in, or is it more important to know that three people gave a specific response in the "other" category?

When more than 15%-20% of the respondents indicate the item "doesn't apply" to their situations, the validity of that item should be questioned. When items have more than 20% N/A, either (1) eliminate the item or (2) eliminate the N/A responses from the analysis.

If some questions won't apply to everyone, provide N/A responses. Remember that N/A means "this item does not apply to my situation," while N (NEUTRAL) means "I am neutral in my opinion." Be sure you're clear in your instructions about their meanings. One way to add clarity is to include the NEUTRAL category on the continuum and place the N/A response to one side:

SD D N A SA N/A

Concerns related to returned data can be handled by making logical choices and explaining your decisions.