Converting Qualitative Feedback into Quantifiable Categories

When gathering evaluative data or programmatic information, researchers typically employ numeric data, checklists, weighting scales, or other quantitative measures. Often, there is a need to ask open-ended questions to probe subject areas or topics more fully. This situation is likely to occur in areas which are relatively unresearched, where response categories cannot be predicted, when the researcher needs to identify an alternative method of collecting data or information in order to improve response rate, or reduce potential for researcher induced bias.

Converting raw, open-ended data from large sample sizes into meaningful categories that the researcher can utilize to quantify the results presents the next challenge. This paper presents a method utilized by the authors to convert and quantify qualitative data from open-ended survey questions into meaningful, quantitative statistics. The richness of qualitative findings, when augmented by quantitative statistics, may collectively provide a clearer answer to the research question being studied.

The purpose of the study was to identify motivational factors; retention characteristics; and demographic descriptors which differentiate between continuing and non-continuing 4-H volunteers. In the study, involving 1138 current and 123 former 4 -H volunteers, both open-ended and forced choice questions were utilized on a mailed questionnaire.

The purpose of the open-ended, or qualitative component, of this study was to gather information from adult 4-H volunteers whose responses could not be accurately predicted categorically. The qualitative component consisted of seven open-ended questions which focused on the motives to continue to volunteer for 4-H; their motives to terminate their volunteer service in 4-H; determining their unmet needs while volunteering; identifying ways to make their volunteer experience more positive; and qualifying their ratings of their local, county and state 4-H programs.

In order to fully investigate a variety of topics without biasing the responses by presenting a limited number of response choices, seven open-ended questions were asked on a four page, sixty-item questionnaire. The questions probed topics that had few published reports represented in the literature. Their responses, therefore, could not be objectively predicted.

Thirteen of the state's 92 counties were randomly selected, with three stratification layers being represented. These included county population (urban, suburban and rural); Extension educator longevity (high, moderate and low) and Cooperative Extension Service area. All current and former (those serving three years or less within the past five years) 4-H volunteers from these 13 counties were sampled comprehensively. After data collection and non-respondent follow-up, 494 current and 40 former volunteers returned surveys.

At this time, the issue of converting open-ended responses into categories was addressed. For the first question, the researcher took the first 10 questionnaires from the first county and grouped the responses into similar categories. The first 10 questionnaires from the second county were distributed in a similar manner, with new categories emerging as necessary. This process was repeated for each of the 13 counties involved in the study, although new categories no longer emerged after the fifth county. This process was repeated for the remaining six qualitative questions.

In order to eliminate researcher induced bias, each response was reviewed by three raters who worked through this process collectively, and reached consensus on allocating responses to each category. Twenty-six descriptors were identified for each open-ended question. The descriptors for each open-ended question, were grouped into six to ten broad categories per question. The categories for each open-ended question received an alphabetical code, after which both forced choice and open-ended questions were coded for data entry. Data were analyzed using descriptive statistics, chi-square, and t-tests.

From the qualitative component of this study, it was determined that 4-H volunteers are primarily motivated to continue volunteering due to affiliation motives. These affiliation motives included "Youth" (61.12%), "The 4-H Organization" (20.84%), and "Feeling Needed" (10.54%). Primary negative motives contributing to volunteer resignation included both an affiliation and achievement motive. These included "Being Unwanted, Unappreciated, and a Lack of Help" (34.05%), and "A Lack of Time, a Job Change, or Occupational Retirement" (29.51%), and "Disappointment with Extension Staff" (18.40%). Finally, 4-H volunteers identified two key ways to make their volunteer position more positive. These included "Additional Training Opportunities" (24.73%) and "Additional Support" (20.16%).

This study, which researched factors affecting volunteer longevity, included 53 quantitative questions which provided demographic information as well as data on recognition, support, program quality, and initiation motives. In addition, the topics of continuing and terminating motives, unmet needs and making the volunteer experience more positive were probed qualitatively in greater detail. Although the process utilized by the researcher and described herein was time consuming, it did provide a means of converting meaningful qualitative information into quantitative statistics. (Chi-square and t-tests were the statistical measures of choice.)

This study reiterated that both quantitative and qualitative research can be blended in order to strengthen an investigation. Qualitative research is more time consuming, however, it more deeply probes a topic, particularly when the results cannot be accurately predicted or supported in the literature.