The Use of Focus Groups to Evaluate the Volunteer Conference of Southern States

Introduction & Review of Literature

Both volunteer and paid staff devote considerable time, energy, and resources to developing, planning, and staging local, area, district, state, multi-state, and regional volunteer forums or conferences (Culp, 2000). The purpose of regional volunteer forums is to increase the capacity of volunteer and salaried staff to contribute to the achievement of the mission of 4-H Youth Development Education and the Cooperative Extension System as a whole (Curtis et al., 1991). Similarly, the purpose of a state, area, or district volunteer conference or workshop is to provide educational and technical resources needed to deliver the 4-H Youth Development program and to meet the needs of 4‑H youth (Culp, 2000). Determining the best way to evaluate these programs is a critical initial component in program evaluation.

The first step in sustaining a volunteer program is evaluation (Culp, 2013). Evaluation is an important component of Extension and volunteer programs (Culp et al., 2009). Extension professionals need to evaluate programs to determine whether programs they conduct are making a positive and productive impact (Galloway, Peterson, & Dalton, 2006). In the current climate of decreased operating budgets, increased time constraints, and reduced staff, Extension professionals are more challenged than ever to provide programs that produce reportable impacts (Diem, 2002.) Formal evaluation will be more successfully implemented if volunteers are involved in developing the process, establishing evaluative criteria and, setting goals (Peterson & McDonald, 2009).

The purpose of conducting a focus group is to listen and gather information. Focus groups provide a way to understand how people feel or think about an issue, product, or service. Focus groups are used to gather opinions (Krueger & Casey, 2009). Focus groups are an efficient and effective assessment tool and are especially useful when time and resources are limited (White, Arnold & Lesmeister, 2008). Focus groups are an informal, qualitative assessment capable of capturing the breadth and depth of a situation without quantification (Washington & Fowler, 2005).

Evaluating the level of success of programmatic objective accomplishment and gathering input and insight about the program are two benefits (Nordstrom, Wilson, Kelsey, Maretzki, & Pitts, 2000). Unlike the one-way flow of information in a one-on-one interview, focus groups generate data through the give and take of group discussion—not just about what they think, but why they think the way they do (American Statistical Association, 1997).

Market researchers originally used focus groups to evaluate the responses of potential customers to new products. The use of focus groups has spread to a variety of organizations that are interested in soliciting the opinions of current or prospective clientele about proposed, new or established programs (Krueger & Casey, 2009).

Focus groups are useful for identifying needs and constraints that might be missed by other assessment methods. Proposed programs for new or established clientele can be revised before costly mistakes are made. Additionally, programmatic objectives can be re-directed or retooled, and recruitment strategies may be adjusted to better meet the needs and interests of potential volunteers (The Community Tool Box, 2014; Gamon, 1992).

The first Volunteer Conference of Southern States (VCSS) was held October 3-6, 2013. Because the VCSS was essentially a Phoenix that was rising from the ashes of the former Southern Region 4-H Volunteer Forum, it was important to conduct an in-depth assessment of the event and gain a critical perspective of the conference attendees. This perspective would become vitally important when evaluating the success of the event and planning the 2014 VCSS. Focus groups were used to gain an understanding about the event in order that conference planners could make informed choices (Krueger & Casey, 2009). Focus groups were used to collect qualitative data from a homogeneous population in a group situation through a focused discussion (Krueger & Casey).

Focus Group Methodology

A series of seven questions was developed by the Southern Region 4-H Volunteer Advisory Group (SR4-HVAG) (Culp, Edwards, & Jordan, 2014), in collaboration with the SR4-HVAG facilitators and the conference coordinator immediately prior to the 2013 Volunteer Conference of Southern States (Culp, Edwards, & Jordan). The series of seven questions was posed to conference participants at a networking luncheon on October 5, 2013 and was used to evaluate the conference. The SR4-HVAG was engaged in the process in order to obtain an unbiased volunteer perspective on regional programming (Culp, Edwards & Jordan, in press.)

Facilitators played an important role in determining the success of focus groups. As identified by Masadeh (2012), facilitators were identified who possessed good interpersonal skills and personal qualities, were friendly, had a sense of humor, were genuinely interested in the participants, and were good listeners, non-judgmental and adaptable. Members of the SR4-HVAG served as focus group facilitators. Pregroup preparation of moderators is a critical factor in focus group success as both social-interaction and research tools (Stewart, Shamdasani, & Rook, 2007). Moderators received instruction and developed facilitation skills at a pre-conference meeting 2 days prior to conducting the focus groups. Advisory group members represented all of the states in attendance and were demographically similar to conference attendees.

A total of 144 conference participants were randomly assigned to sit at one of 18 round table focus groups that included seven people and a facilitator. One participant at each round table volunteered to capture and record the information in a Word document on a Mac Notebook. The seven questions in the series were randomized in order that the same amount of time could be devoted to each question, regardless of when it was discussed in the series.

The series of seven questions included:

• Thinking about your experience at this conference, what will you do with this information and resources, once you return home?
• From your perspective, what were the three most important benefits of the conference?
• In what ways did this conference fulfill your expectations of those three benefits?
• What could have been done programmatically, to fulfill any expectations that were unmet?
• If you were recruiting new volunteers to attend the 2014 Volunteer Conference of Southern States, what would be your "talking points"? What message would you deliver to "sell them" on attending?
• What will motivate you to return to the 2014 Volunteer Conference of Southern States?
• If you had the opportunity to meet with the planning committee of the 2014 Volunteer Conference of Southern States, what would you stress?

The Mac Notebooks were networked, and all responses were grouped by question. Confirmability and trustworthiness (Shenton, 2004) were established in three primary ways. First, 18 facilitators, none of whom included the team of researchers, conducted each of the focus groups. Second, the data was gathered from each focus group by the series of networked Mac notebooks and was assembled into a dataset by a UGA staff member, unrelated to and unfamiliar with the study. Finally, qualitative findings were coded by keyword identification using the process of three raters identified by Culp and Pilat (1998). The three raters included the two SR4-HVAG facilitators and the conference coordinator, all of whom were state volunteerism specialists. The three raters used consensus to categorize data. The data set was analyzed using descriptive statistics. Following the conference, findings were validated by sharing the categorical trends and results with the Volunteer Advisory Group and the state program leaders in the Southern Region.

Results

The first question was "thinking about your experience at this conference, what will you do with this information and resources, once you return home?" Their responses are summarized in Table 1.

Table 1.
How Will You Use This Information and Resources, After Returning Home?

Use of Information After the Conference	Number of Responses	Rank
Share informally with others	46	1
Apply the information	31	2
Teach or present the information formally	10	3

Forty-six (46) respondents indicated that they planned to "share (informally) with others." "Others" included other volunteers, youth, and Extension professionals. Thirty-one (31) participants planned to "apply" their information by "using the information in my club, implementing resources into my program, and implementing resources into church and other organizations." Ten (10) people indicated that they would "teach or present the information (formally) by presenting workshops in my club, or at the county and state levels."

Because the number of responses in each category was smaller for this question as compared to others, it may not be readily apparent to volunteers how to use the information and resources provided by the conference.

The second question was "from your perspective, what were the three most important benefits of the conference?" The four primary response categories that emerged are listed in Table 2.

Table 2.
The Most Important Benefits of the Volunteer Conference of Southern States

Identified Conference Benefits	Number of Responses	Rank
Education / Knowledge / Learning	70	1
Networking	59	2
Self-Satisfaction	19	3
Logistics	6	4

Participants identified the educational activities they experienced at the conference as being primarily important, with networking opportunities second. The two primary benefits are important to note, particularly networking. Conference planners generally devote a considerable amount of time, energy, and resources to educational programming, including workshops, general sessions, intensive learning labs, and service learning activities. However, it is important to note that conference participants list networking opportunities nearly as often as educational opportunities in their responses. For many volunteers, the opportunity to share information with other conference participants is an important aspect of the conference.

The third question discussed was "in what ways did this conference fulfill your expectations of those three benefits?" The four primary response categories that emerged are listed in Table 3.

Table 3.
How the Volunteer Conference of Southern States fulfilled my Expectations

Fulfilled Expectations of the Conference	Number of Responses	Rank
Education / Knowledge / Learning	59	1
Networking	40	2
Self-satisfaction	11	3
Logistics	4	4

Respondents identified high expectations for educational (59) and networking (40) opportunities and indicated that the conference had fulfilled these expectations. Additionally, 11 participants experienced self-satisfaction through their conference experience.

The fourth question asked "what could have been done programmatically, to fulfill any conference expectations that were unmet?"

Table 4.
Programmatic Changes that would have Fulfilled Unmet Conference Expectations

Programmatic Changes	Number of Responses	Rank
Logistics (Scheduling, State Meetings, Food, Lower Costs)	92	1
Education / Knowledge / Learning	36	2
Additional Information & Communications	32	3
Offer Funshops	18	4
Provide coffee all day	12	5

The primary programmatic change, as identified by 92 focus group participants, was a category of responses related to "conference logistics." Conference logistics included adjustments to the conference schedule (22), the request to hold state meetings (17), issues related to food choices and meals (10), and reduced registration costs (2). The second and third most frequently suggested changes to the program were similar in frequency. These included suggested changes made to the educational program and learning activities (36) and additional information and more frequent pre-conference communication be sent (32). Eighteen (18) people requested that "funshops" (crafting workshops) be reinstated (they had been dropped from the conference schedule), and 12 people requested that coffee be provided all day.

The fifth query posed to focus groups was a two-part question. "If you were recruiting new volunteers to attend the 2014 Volunteer Conference of Southern States, what would be your 'talking points'"? What message would you deliver to 'sell them' on attending?" The responses are listed in Table 5.

Table 5.
Talking Points to Recruit New Volunteers to Attend the Volunteer Conference of Southern States

Programmatic Changes	Number of Responses	Rank
Logistics (location/facilities, food, adults only, other, new cabins)	73	1
Education / Knowledge / Learning	54	2
Networking Opportunities (Share ideas and information, meeting people, developing friendships)	52	3
Personal Satisfaction (fun, opportunities, relaxing, reinvigorate, belonging)	27	4

Focus group participants identified four broad categories of talking points that they thought would encourage volunteers to attend the next conference. The first categorical response, identified by 73 people, was "logistics."

Logistics included the location and facilities at Rock Eagle 4-H Center (23), food and meals (17), an adults-only conference (12), "other" (11), and the allure of new cabins (10). It is important to note that 12 focus group participants (all of whom were adults) preferred attending an "adults only" conference, especially because they work directly with youth in their role as a 4-H volunteer. Additionally, during the 2013 Conference, Rock Eagle was in the process of replacing the cabins built in the 1950's with state-of-the-art cottages, each of which contained six sleeping rooms and a centralized great room.

The second category of responses, identified by 54 focus group participants, was "Education / Knowledge / Learning." Following closely with 52 responses was "Networking Opportunities." This category included "sharing ideas and information, meeting people, and developing friendships. " The final category (27) involved "Personal Satisfaction" and included fun, opportunities, a relaxing environment, the opportunity to energize and reinvigorate, and a sense of belonging.

The sixth focus group question asked respondents to identify the factors that would motivate them to return for the 2014 VCSS. Responses are indicated in Table 6.

Table 6.
Factors That Will Motivate Participants to Attend the 2014 VCSS

Motivational Factors	Number of Responses	Rank
Logistics (Costs, Pre-Conference communication, adults only, Held at Rock Eagle)	66	1
Education / Knowledge / Learning	51	2t
Networking Opportunities (Share ideas and information, meeting people, developing friendships)	51	3t
Personal Satisfaction (fun, opportunities, relaxing, reinvigorate, belonging)	30	4
t = tied response

Participants identified four broad categories of factors that would motivate them to attend the 2014 conference. The factors most frequently identified were grouped into a category labeled "logistics." These factors included the economical costs of attending (26), pre-conference communication (18), appreciation for a conference targeting adults (9), and the location of the conference (held at Rock Eagle 4-H Center) (9).

The second and third categories of factors were both tied, with 51 responses each. The first was "Education / Knowledge / Learning." "Education" included high-quality workshops and information, hands-on activities, and materials that are ready to use and teach. The second category (also with 51 responses) was "Networking Events and Opportunities." The final group of factors was a group of intrinsic motives classified as "Self-Satisfaction," with 30 responses.

The final question posed to focus group participants asked for suggestions for the 2014 Conference Planning Committee. The responses to this question are listed in Table 7.

Table 7.
Suggestions for the 2014 Conference Planning Committee

Suggestions	Number of Responses	Rank
Logistics (Planning Suggestions, Hospitality, Schedule adjustments, Food Concerns)	117	1
Workshops	32	2
Communication (Registration confirmations, Presenter information, pre-conference information)	31	3
Networking Events & Opportunities	17	4
Schedule Funshops	17	5

---

The majority of suggestions for the 2014 conference focused on "logistical issues." These included: conference planning suggestions (77), increased hospitality (30), schedule adjustments (24) and food concerns (14). "Workshops" were the second most frequently identified suggestion. Participants made suggestions as to specific types of workshops that they would like to see presented at the 2014 conference. This category was closely followed by "Communication" as identified by 31 participants. Specific items in this category focused on improved pre-conference communication (13), presenter information (12) and pre-conference information (6). The final two categories identified by 17 respondents each included the need to schedule "Networking Events and Opportunities" and Funshops in the 2014 Conference Program of Events.

Implications and Discussion

Focus groups proved to be an informative method of gathering evaluation data on the conference. Participants were very open, forthcoming, and willing to share their input, thoughts, expectations, and suggestions. Using a bank of networked Mac Notebooks was an effective and efficient method of compiling the data into a single document that could be coded.

The role of both the focus group facilitator and recorder were central to the process. There was a great deal of variation among groups regarding the amount of time required to discuss the seven questions. Although they had received facilitation training 2 days previously, for most of the facilitators, this was the first time that they had served in this role. Likewise, several recorders needed a few extra minutes to become acquainted with and comfortable in using the Mac Notebook.

The input received through the focus group round table discussions was instrumental in planning the 2014 conference. Substantial modifications were made to the conference and schedule that incorporated suggestions and input gleaned through the focus group process.

The following suggestions, collected during the 2013 focus groups, were implemented by the 2014 conference planning committee:

1. Conference participants were unsure how to implement information and use resources gathered at the 2013 conference. Therefore, at the conclusion of each session, 2014 workshop presenters were asked to make suggestions regarding how to use the information, apply it into their own 4-H club or program, and share it with other 4-H members, volunteers and Extension professionals.
2. Additionally, 2014 conference planners developed a strategy that would facilitate the implementation of this information and resources. This included a list of "talking points" and a "homework assignment" including a suggestion list or a series of "take-home" activities for conference participants to use when they returned to their homes, communities, and 4-H programs.
3. Because respondents identified the two primary benefits of the conference as education and networking, a conscious effort was made to recruit meaningful workshops and devote a significant amount of time on the schedule each day to networking and sharing events.
4. For many volunteers, the opportunity to share information with other conference participants is an important aspect of the conference. Therefore, a sufficient amount of time was devoted to facilitating networking opportunities when planning volunteer development activities.
5. The conference accomplished its primary objectives of delivering a quality educational program for 4-H volunteers and providing significant networking opportunities.
6. The 2014 conference was adjusted to fulfill unmet expectations of participants. Adjustments included modifying the schedule, holding state delegation meetings, recruiting quality educational workshops, and devoting one evening to scheduling "funshops."
7. Additional steps have been taken to improve communication among the states and between volunteer specialists and volunteers in each state/commonwealth. A stronger presence on the conference website and Facebook page was a priority.

Focus groups will be used to evaluate future conferences. Findings will be compared to those from 2013 with results contrasted for similarities and differences. Appropriate adjustments will be made to future conference schedules and programs to incorporate findings collected from the focus group round table discussions.

Finally, when asked to identify the greatest benefits of the conference, attendees identified "education/knowledge/learning" as most important (70 responses) and "networking" as second most important (59). "Self-satisfaction" was the third most frequently cited benefit, with 19 responses, and "logistics" was fourth with six (6). Because the two primary objectives for the VCSS were education and networking, the conference planners determined that the event had successfully achieved its goals. The use of round table focus groups to evaluate the conference strengthened and reinforced both benefits. Information was readily exchanged among participants during the round table focus groups, which have proven to be a useful networking event.

References

American Statistical Association (1997). What are focus groups? Alexandria, VA: Section on Survey Research Methods, American Statistical Association. Retrieved from: http://www.prm.nau.edu/prm447/asa%20brochures/focusgroups.pdf

Culp, III, K. (2000). Planning educational volunteer forums: Steps to success. Journal of Extension [On-line], 38(6) Article 6FEA3. Available at: http://www.joe.org/joe/2000december/a3.php

Culp, III, K. (2013). Sustaining members, volunteers and leaders in community organizations. Journal of Extension [On-line], 51(2) Article 2TOT6. Available at: http://www.joe.org/joe/2013april/tt6.php

Culp, III, K., Edwards, H. C., & Jordan, J. W. (2014). Creating the southern region 4-H volunteer advisory group. Journal of Extension [On-line], 52(6) Article 6IAW9. Available at: http://www.joe.org/joe/2014december/iw9.php

Culp, III, K., Edwards, H. C., & Jordan, J. W. (in press). Utilizing an advisory group to obtain volunteer perspective for regional programming. Journal of Extension.

Culp, III, K., & Pilat, M. (1998). Converting qualitative feedback into quantifiable categories. Journal of Extension [On-line], 36(5) Article 5IAW3. Available at: http://www.joe.org/joe/1998october/iw3.php

Diem, K. (2002). Making program choices when resources are limited: Using a self-assessment tool with stakeholders. Journal of Extension [On-line], 40(4) Article 4TOT3. Available at: http://www.joe.org/joe/2002august/tt3.php

Galloway, R., Peterson, K., & Dalton, M. (2006). Student focus groups reveal impacts of 4-H program. Journal of Extension [On-line], 44(4) Article 4TOT2. Available at: http://www.joe.org/joe/2006august/tt2.php

Gamon, J. A. (1992). Focus groups: A needs assessment tool. Journal of Extension [On-line], 30(1) Article 1TOT2. Available at: http://www.joe.org/joe/1992spring/tt2.php

Krueger, R. A., & Casey, M. A. (2009). Focus groups: A practical guide for applied research, 4th ed. Thousand Oaks, CA: Sage Publications, Inc.

Masadeh, M. A. (2012). Focus group: Reviews and practices. International Journal of Applied Science and Technology, 2(10)63-68.

Nordstrom, P. A., Wilson, L. L, Kelsey, T. W., Maretzki, A. N., & Pitts, C. W. (2000). The use of focus group interviews to evaluate agriculture education materials for students, teachers and consumers. Journal of Extension [On-line], 38(5) Article 5RIB2. Available at: http://www.joe.org/joe/2000october/rb2.php

Shenton, A .K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22(2004)63-75. IOS Press, Northumbria University, Newcastle upon Tyne, NE1 8ST, UK.

Stewart, D. W., Shamdasani, P. N., & Rook, D. W. (2007). Focus groups: Theory and practice 2^nd ed. Thousand Oaks, CA: Sage Publications, Inc.

The Community Tool Box (2014). Retrieved from: http://www.ctb.ku.edu/en/table-of-contents/assessment/assessing-community-needs-and-resources/qualitative-methods/main

Washington, R., & Fowler, S. (2005). Systematic assessment of resistance to Extension organizational change: Evidence from the Alabama Cooperative Extension System. Journal of Extension [On-line], 43(2) Article 2RIB2. Available at: http://www.joe.org/joe/2005april/rb2.php

White, D. J., Arnold, M. E., & Lesmeister, M. (2008). Using focus groups to evaluate youth development program direction. Journal of Extension [On-line], 46(6) Article 6RIB3. Available at: http://www.joe.org/joe/2008december/rb3.php