Participant Comfort with and Application of Inquiry-Based Learning: Results from 4-H Volunteer Training

Introduction

How can Extension staff ensure that participants in a training workshop are able to apply what they learn—especially if the participants are not familiar or comfortable with the workshop's content? Designing training to support learning transfer is a complex process in general but is perhaps even more complex when designing training for program volunteers. This article addresses two concepts: (a) how intentional efforts to increase trainees' comfort with workshop content affected 4-H volunteers' comfort levels with that content and (b) the extent to which comfort with tools and techniques learned during training affects application of those tools and techniques in the future.

A 2012 statewide Minnesota 4-H volunteer training program—Wonder! Question! Discover! Expanding 4-H Learning—focused on equipping adult volunteers to support youth in inquiry-based learning (I-BL), especially with science activities. Training was intentionally designed to develop volunteer competence in and comfort with engaging more young people in I-BL experiences and increasing science literacy (significant goals in Minnesota and nationally). Participants were surveyed immediately after training and 1 year later to explore

• whether they became more comfortable in guiding I-BL as a result of the training,
• the extent to which they applied the tools and techniques they experienced during the training, and
• the degree to which their comfort level influenced their application of those tools and techniques in the year following the training.

Background

Research suggests that nonformal/out-of-school educational settings, such as 4-H, are important in addressing science, technology, engineering, and math (STEM) learning needs (Bell, Lewenstein, Shouse, & Feder, 2009; Krishnamurthi, Ballard, & Noam, 2014; Meyer, Bevan, & Garza, 2010; Smith, Meehan, Enfield, George, & Young, 2004) and that STEM training is needed for 4-H staff and volunteer development (Nippolt, 2012; Smith, 2008; Smith et al., 2004). Professional development increases confidence and competence in facilitation of STEM programs (Junge & Manglallan, 2011). Furthermore, incorporating learning-transfer strategies into training is often key to achieving program outcomes (Caffarella, 2002). Broad (1997) defines learning transfer as "the effective and continuing application by learners . . . of knowledge and skills gained in learning activities" (p. 2). Merriam and Leahy (2005) indicate that "learning transfer is influenced directly by variables within the design of the program. Activities that attend to transfer, such as action plans, coaching and a variety of instructional techniques . . . seem to make a difference in ensuring for some transfer" (p. 9). Modeling methods for facilitation during training has been found to improve learner effectiveness in implementation and to increase confidence and reduce anxiety about teaching science (Konen & Horton, 2000). Use of hands-on, experiential learning approaches—the mainstay of the 4-H youth educational model—can also be valuable with adults learners (Torock, 2009).

This research undergirds University of Minnesota 4-H volunteer training. The Extension Center for Youth Development offers many ongoing local and online volunteer training opportunities. It also provides annually a statewide 90-minute face-to-face training designed to reach at least one volunteer in every 4-H club. Topics are chosen for the purposes of addressing state priorities, responding to identified needs of club leaders, and motivating and preparing adult and youth volunteers to co-lead high-quality club experiences. Leading I-BL in a STEM context emerged as an important topic that was new to most staff members and volunteers.

Methods

Training Design and Objectives

The 2012 statewide volunteer training was intentionally designed to

• train facilitation staff on both content and facilitation in order to better support volunteers following the training,
• equip participants with a basic understanding of I-BL (the inquiry process and tools and techniques for expanding learning),
• increase volunteer comfort level in guiding I-BL by providing opportunities for volunteers to engage in two science investigations that involve using open-ended questioning and the "I Wonder" Board tool, and
• deliver consistent content and educational experiences through multiple methods of instruction.

The design team trained regional and local staff to lead the workshops. Content included information about defining inquiry, the importance of wonder, key science abilities aligned with science process skills (Harlen & Jelly, 1997), and open-ended questioning. Hands-on experiences included using the "I Wonder" Board (Stevenson, 2013), guiding inquiry with Polymer Orbs and Mysterious M&M's^® investigations, and practicing questioning techniques. (Note: In this article, the word tool refers to learning activities, such as Polymer Orbs and Mysterious M&M's^® investigations and use of the "I Wonder" Board. The word technique refers to the use of open-ended questioning.) The training also included discussion of and an action plan tool for the application of training concepts to a variety of 4-H learning topics and contexts. Each participant received an Inquiry and Science Learning Activities Booklet (LAB), which contains strategies for leading guided inquiry in nine activities, including the two used during training (see Table 2 for list); handouts on questioning techniques and experiential learning; and a bag of polymer orbs for use with program youth. Materials for this training are available at http://z.umn.edu/wonder.

Initial Training Evaluation

In total, 1,902 4-H volunteers, staff, and youth (representing 87% of 4-H clubs) were trained by 4-H staff facilitators at 84 sites across Minnesota in fall 2012. The end-of-session evaluations completed by 958 adults and 334 youth indicated that the training achieved the stated outcomes, and 99% of participants rated the overall quality of the training as "good" to "excellent" (mean 4.3 on a 5-point scale). Participants generally agreed or strongly agreed that they had "an understanding of how inquiry expands learning" and that they learned to use questioning techniques and the "I Wonder" Board to help youth expand learning (means 3.72 and 3.71, respectively, on a 4-point scale).

When asked what they found "particularly interesting or useful" in the training, participants at all sites highlighted new ideas and the LAB, hands-on learning and participation in several of the experiments, Polymer Orbs and Mysterious M&M's^® investigations, "I Wonder" Board, open-ended questioning with youth, and ways to better engage youth. One volunteer participant commented, "I am not good at science so the ideas and questions were very helpful. Then I felt more confident to go back and help the kids."

Approximately 30 responses indicated that greater access to supplies would be helpful in applying the training tools, so a few months later, 15 LAB activity kits were created and made available at regional offices and promoted to volunteers via email and newsletters.

Follow-Up Survey

Survey Sample

The authors sought to determine how participants used the training content and resources in the year following the training. In November 2013, a follow-up survey was emailed to all training participants (youth and adults) who provided an email address (n = 1,131). Although email was undeliverable to 81 of these addresses, 361 participants responded, for an overall response rate of 31.9%. Responses came from 198 4-H adult volunteers, 12 staff, and 39 youth (other respondents were not defined). Because adult volunteers were the target audience of the training, only responses from adult volunteers were analyzed for this article. 4-H volunteer experience varied: 8% reported volunteering 1–2 years; 28%, 3–5 years; 28%, 6–10 years; 24%, 11–20 years; and 12%, 20-plus years.

Measures

Survey respondents were asked which training tools they had used (see Table 2 for a list of training tools), where they had used them, and with how many adults and youth they had used them. They also were asked about their use of open-ended questioning techniques, with five response options that ranged from "a lot more than" to "less than" before the training. Comfort level in guiding I-BL was evaluated by using retrospective measures of comfort from "extremely comfortable" to "not at all comfortable" at three time points described as "before" and "immediately after" the training and "now, a year later."

Results

Comfort Level in Guiding I-BL

Follow-up survey data analysis revealed that the training significantly increased volunteers' mean comfort level in guiding I-BL, from 2.86 before the training to 3.46 immediately afterward (see Table 1). Furthermore, that comfort level was essentially perceived to be sustained 1 year later; the slight decline to 3.38 was not significant. Although 62% of volunteers indicated that they felt some level of comfort in guiding I-BL even "before training," this increased to 85% "immediately after training," and the percentage who indicated that they were extremely or very comfortable increased from 25% to 48%. Although many volunteers indicated no change (49%) or only slight decreases in comfort (2%) from before to after the training, almost half (48%) reported increases in comfort.

Table 1.
Retrospective Perspectives on the Levels of Comfort in "Guiding Inquiry-Based Learning"

Level of Comfort	"Before Training"	"Immediately after Training"	"Now, a Year Later"
Extremely Comfortable	12%	15%	16%
Very Comfortable	13%	33%	25%
Comfortable	37%	37%	43%
Only a Little Bit Comfortable	27%	13%	13%
Not at All Comfortable	11%	2%	3%
Mean Level of Comfort	2.86	3.46	3.38
Difference from Before Training Level		0.60	0.52
Significance		t(170) = 9.44, p < .001	t(170) = 9.54, p < .001

Looked at from a change perspective, the results indicate that 24% of volunteers were "extremely comfortable" or "very comfortable" before training and simply stayed that way. Another 24% became "extremely comfortable" or "very comfortable" as a result of the training. Similarly, 19% started as "comfortable" and stayed that way, and 18% became "comfortable" during the training. Only 15% never felt or became comfortable at all.

To better understand the changes in comfort, the correlation between years of experience as a 4-H volunteer and comfort level was examined. Years of volunteer experience was not related to comfort level at any of the three time points: Experienced 4-H volunteers were not any more or less comfortable guiding I-BL than new volunteers. Interestingly, however, the change in comfort from before to immediately after the training was significantly and slightly negatively correlated with volunteer experience (r = −.139, p < .10). In essence, less experienced volunteers were slightly more likely to report a change in comfort in guiding I-BL following the training. Volunteers with 3–5 years of experience particularly seemed to gain in comfort from the training—64% experienced a gain in comfort—whereas only 27% of those with 20-plus years saw positive gains in comfort.

Use of the Tools, Techniques, and Kits

Volunteers were asked about the tools and techniques they had used since the training. Each of these is discussed, in turn, and identified in Table 2. First, the tools that volunteers experienced during the training (Polymer Orbs, Mysterious M&M's^®, and "I Wonder" Board) were, by far, the tools that most of them used in the year following training: 54% used Polymer Orbs; 39% used Mysterious M&M's^®; and 24% used the "I Wonder" Board. Overall, 73% reported using at least one of these three tools (not shown in Table 2). Only 5% or fewer of the volunteers used additional tools available after the training (in the LAB activity kit but not demonstrated during the training), except for Oobleck, a commonly used 4-H activity that was used by 15% of the training participants in the year following training. Altogether, the volunteers reached a large number of people through the use of tools experienced during their training. The mean numbers of youth and adults reached per volunteer using the tools are reasonable but highly variable (as indicated by the high standard deviations).

Table 2.
Use of I-BL Tools by 4-H Volunteers Within a Year After Training

Tools	% Using Tool (N = 198)	Total Number of Youth Reached by Volunteers Using Tools	Mean Number of Youth Reached per Volunteer Using Tools	S.D.	Total Number of Adults Reached by Volunteers Using Tools	Mean Number of Adults Reached per Volunteer Using Tools	S.D.
Tools Experienced in Training
Polymer Orbs	54%	1325	13.3	(14.22)	503	5.2	(10.41)
Mysterious M&M's®	39%	911	12.7	(8.41)	296	4.3	(2.94)
"I Wonder" Board	24%	474	11.0	(6.95)	176	4.3	(3.54)
Tools Available After Training
Oobleck	15%	419	16.1	(11.46)	100	4.0	(2.86)
Big Floating Bubbles	5%	125	15.6	(7.91)	27	3.4	(3.34)
Bouncing Balloons	4%	95	11.9	(8.71)	31	3.9	(4.19)
Take It in Stride	3%	53	10.6	(4.39)	24	4.8	(2.77)
Balloon-Powered Cars	3%	85	14.2	(3.43)	23	3.8	(2.79)
High Speed Mystery Foods	2%	12	6.0	(2.83)	6	3.0	(1.41)
Pinhole Cameras	2%	11	3.7	(2.08)	7	3.5	(2.12)

Almost 21% did not use any of the tools. These 38 volunteers cited a lack of "fit" with club interests, lack of time to prepare or add activities to the preplanned club meeting structure, or the training topic's not being applicable to their specific volunteer role as reasons for not using the tools. About 55% of those who had not used tools said they intended to (5%) or at least may (50%) do so in the next 6 months (see Table 3).

Table 3.
Use and Intended Future Use of Tools, Open-Ended Questioning Techniques, and LAB Activity Kits

Tools, Techniques, and Activity Kits	% of Volunteers Using During Year After Training	% of Volunteers Who Did Not Use			Estimated % of Volunteers Using or Indicating Possible Use over 18 Months
		But Intend to in Next 6 Months	But May in Next 6 Months	And Do Not Intend to in Next 6 Months
Any of the 3 Demonstrated Tools	78%
Any of the Tools	84%	5%	50%	45%	90%
Open-Ended Questioning Technique	55%a	35%	52%	13%	94%
Activity Kits	13%	7%	71%	21%	81%
aIndicates % using open-ended questioning "a lot more" or "a little more" than before training.

Over 56% of volunteers surveyed indicated that they used open-ended questioning a lot more (16%) or a little more (40%) since the training. About 39% reported using it at about the same level as before the training. Only 6% indicated that they used it less than before. Forty-one respondents who reported similar use shared that they already used the technique and appreciated the reminder to use it. Over 87% of those who had not increased their use of open-ended questioning said they intend to (35%) or at least may (52%) do so in the next 6 months (see Table 3).

Thirteen percent of volunteers surveyed used activity kits, which were not part of the training but were made available regionally during the year following training. When asked to indicate why they did not use the kits, 11% indicated that the kits were not easily accessible, 25% indicated that they did not know the kits were available, 33% indicated that they did not remember that the kit was available, and 16% indicated that they used their own or their counties' kits. Note that these kits did not exist at the time of training; they were created after the training on the basis of postsession surveys indicating need. Of the 87% of volunteers who did not use kits, 78% indicated that they intend to (7%) or may (71%) in the next 6 months (see Table 3).

Taken together, the estimated actual use and intended future use of tools, open-ended questioning, and activity kits ranged from 81% to 94%—a vast majority of volunteers in each case (see Table 3).

Influence of Comfort Level in Guiding I-BL on Use of Tools, Techniques, and Kits

This section explores the extent to which volunteers' levels of comfort in guiding I-BL influence whether they use different tools, techniques, and activity kits as well as the number of youth and adults with whom they report using them. Because of the low percentage of use of some tools, use of those tools is not analyzed further in this section.

As seen in Tables 4 and 5, few significant relationships exist between use of tools and either level of comfort immediately following the training or the type of change in comfort experienced during the training (although the trend is often in that direction). The comfort levels in guiding I-BL did not generally predict use of tools, percentage of volunteers using any tools, or number of youth or adults reached. Neither did the change in comfort level from before to after the training relate significantly to these.

Interestingly, there was a significant relationship between both measures of comfort in guiding I-BL and the use of open-ended questioning. Fifty-six percent of those who were highly comfortable after the training used the technique compared to only 44% of those who were "not at all comfortable" or only "a little bit comfortable." Furthermore, the four individuals who indicated that they were "not at all comfortable" did not use any of the tools. Similarly, those who experienced a gain in comfort during the training were much more likely to use open-ended questioning than others (76% vs. 43% and 35%). Gains in comfort during training were also weakly but significantly related (at < .10 level) to the mean number of the 10 tools that people reported using and to the number using Polymer Orbs in particular.

Table 4.
Levels of Tool, Technique, and Kit Use by Level of Comfort After Training

Use of Tools, Techniques, and Activity Kits	Level of Comfort After Training			Sign. Level
	Not at All Comfortable or Only a Little Comfortable (N = 25)	Comfortable (N = 4)	Extremely or Very Comfortable (N = 82)
Use of Tools
Mean # of 10 Tools Used per Volunteer	1.2	1.4	1.9	NS
% Using Any of 10 Tools Mentioned	69%	86%	88%	NS
Mean # of 3 Demonstrated Tools Used per Volunteer	1.3	1.5	1.5	NS
% Using Any of 3 Demonstrated Tools	62%	70%	70%	NS
% Using Polymer Orbs	54%	63%	63%	NS
Mean # of Youth Reached with Polymer Orbs	9.50	14.70	14.30	NS
Mean # of Adults Reached with Polymer Orbs	6.20	4.50	9.00	NS
% Using Mysterious M&M's®	27%	38%	52%	NS
Mean # of Youth Reached with Mysterious M&M's®	13.40	11.80	13.80	NS
Mean # of Adults Reached with Mysterious M&M's®	4.20	4.60	5.00	NS
% Using "I Wonder" Board	19%	23%	21%	NS
Mean # of Youth Reached with "I Wonder" Boards	10.40	14.70	10.40	NS
Mean # of Adults Reached with "I Wonder" Boards	4.00	6.40	4.90	NS
% Use of Open-Ended Questioning	44%	59%	56%	< .10
% Use of Activity Kits	12%	10%	13%	NS

Table 5.
Levels of Tool, Technique, and Kit Use by Perceived Changes in Comfort from Before to After Training

Use of Tools, Techniques, and Activity Kits	Perceived Changes in Comfort from Before to After Training			Chi-Squared Sign. Level
	Little or No Change but Not Comfortable	Little or No Change But Some Comfort	Gain in Comfort and Extremely or Very Comfortable
Use of Tools
Mean # of 10 Tools Used per Volunteer	1.27	1.89	1.71	< .10
% Using Any of 10 Tools Mentioned	85%	92%	85%	NS
Mean # of 3 Demonstrated Tools Used per Volunteer	1.30	1.38	1.61	NS
% Using Any of 3 Demonstrated Tools	64%	65%	75%	NS
% Using Polymer Orbs	55%	54%	68%	< .10
Mean # of Youth Reached with Polymer Orbs	13.3	11.9	15.8	NS
Mean # of Adults Reached with Polymer Orbs	5.6	5.2	4.6	NS
% Using Mysterious M&M's®	27%	49%	68%	NS
Mean # of Youth Reached with Mysterious M&M's®	13.6	12.5	13.6	NS
Mean # of Adults Reached with Mysterious M&M's®	3.3	4.5	4.0	NS
% Using "I Wonder" Board	21%	30%	25%	NS
Mean # of Youth Reached with "I Wonder" Boards	10.9	10.3	13.6	NS
Mean # of Adults Reached with "I Wonder" Boards	5.6	5.3	5.6	NS
% Use of Open-Ended Questioning	43%	35%	76%	< .001
% Use of Activity Kits	13%	11%	17%	NS

In sum, the level of comfort in guiding I-BL did not generally influence the use of most tools, even two out of the three tools demonstrated during the training. Nor did it influence the use of the activity kits that were made available later. The more consistently significant influence of comfort level was on use of open-ended questioning.

Discussion and Implications for Extension

Follow-up survey data indicated that the training was highly successful in increasing and sustaining comfort levels in guiding I-BL and in teaching and preparing 4-H volunteers to apply tools and open-ended questioning techniques in their communities.

The more comfortable people became in guiding I-BL during the training, the more they used open-ended questioning afterward. This correlation supports the idea that when adults learn a technique and become comfortable using it to guide learning, they will apply that technique later in their community contexts. Thus, Extension staff may find it particularly worthwhile to spend training time getting participants comfortable with techniques they will need to apply later.

Some survey respondents applied open-ended questioning more often in the year after the training because the training "had been a good reminder" of the technique. This finding supports Smith's (2013) assertion that episodic training can reinforce existing knowledge and skills. Barker, Grandgenett, and Nugent (2009) suggest that an ongoing training model may help participants better develop knowledge and skills necessary to effectively facilitate STEM content; however, volunteers in this one-time training indicated high levels (73%) of use of new training tools, and they were clearly most likely to use tools they had experienced during the training. As long as they were at least "a little bit comfortable," they applied these tools without apparent need for ongoing training. The data about application also support the idea that "learning by doing" is not just a pithy 4-H concept but a sound approach to learning and teaching for people of any age (Torock, 2009). Extension staff would do well to design trainings so that participants have hands-on experiences with the tools they will need to apply later.

Ensuring that participants had easy access to supplies likely added to their use of the tools. Polymer orbs were given directly to participants and were used by more than half of volunteers at any level of comfort. Additionally, use of the LAB activity kits may have been improved if they had been available during the training. Extension staff would do well to ensure easy access to supplies and hands-on exploration of kits at training.

Further, as noted earlier, participants were given time during the training to create an action plan for implementation of the content. Although action planning was not a focus of the follow-up survey research, this design component (Merriam & Leahy, 2005) may have increased the likelihood that volunteers would apply their learning and, thus, may be useful for Extension staff when designing training.

Regarding limitations, this research relied on participants' retrospective ratings of comfort level before and just after the training; using a true preassessment and postassessment of comfort levels in guiding I-BL may have garnered more accurate data. Asking respondents to rate their specific levels of use of open-ended questioning before and after training (instead of asking only about relative use after training) would have been more valuable for gauging application and change. Quality and effectiveness of volunteers' application of their learning was not assessed; this could provide better insight into training outcomes. A final limitation is the 31% response rate for the follow-up survey and the inability to link responses to original postsession survey responses.

Conclusion

In designing Extension training, educators must intentionally incorporate learning strategies that actively engage participants in the content, skills, or behaviors that educators expect them to apply. This study found that it is possible to increase participants' long-term comfort levels in guiding I-BL with a one-time training intentionally designed to support learning transfer. Also, participants apparently do not need very high comfort levels in order to use tools; they can engage in a learning session, be "only a little bit comfortable," and still apply tools from the training. The most helpful tactic is providing them with opportunities to experience tools and build skills by practicing techniques that will allow them to effectively guide learning in the future.

References

Barker, B. S., Grandgenett, N., & Nugent, G. (2009). A new model of 4-H volunteer development in science, engineering, and technology programs. Journal of Extension [Online], 47(2) Article 2IAW4. Available at: http://www.joe.org/joe/2009april/iw4.php

Bell, P., Lewenstein, B., Shouse, A., & Feder, M. (2009). Learning science in informal environments: People, places, and pursuits. Washington, DC: The National Academies Press.

Broad, M. L. (1997). Transfer concepts and research overview. In M. L. Broad (Ed.), Transferring learning to the workplace (pp. 1–18). Alexandria, VA: American Society of Training and Development.

Caffarella, R. S. (2002). Planning programs for adult learners: A practical guide for educators, trainers, and staff developers (2nd ed.). San Francisco, CA: Jossey-Bass.

Harlen, W., & Jelly, S. (1997). Developing science in the primary classroom. Essex, England: Addison Wesley Longman, LTD.

Junge, S. K., & Manglallan, S. S. (2011). Professional development increases afterschool staff's confidence and competence in delivering science, engineering and technology. In A. Subramaniam, K. Heck, R. Carlos, & S. Junge (Eds.), Advances in youth development: Research and evaluation from the University of California Cooperative Extension (2001–2010) (pp. 70–78). Davis, CA: University of California Agriculture and Natural Resources. Retrieved from http://4h.ucanr.edu/Research/4HPublications/

Konen, J., & Horton, R. L. (2000). Beneficial science teacher training. Journal of Extension [Online], 38(2) Article 2RIB1. Available at: http://www.joe.org/joe/2000april/rb1.php

Krishnamurthi, A., Ballard, M., & Noam, G. (2014). Examining the impact of afterschool STEM programs. Washington, DC: Afterschool Alliance. Retrieved from http://www.afterschoolalliance.org/STEM-PUBLICATIONS.cfm

Merriam, S., & Leahy, B. (2005). Learning transfer: A review of the research in adult education and training. Pennsylvania Association for Adult Continuing Education Journal of Lifelong Learning, 14, 1–24. Retrieved April 24, 2015, from http://www.iup.edu/page.aspx?id=17489

Meyer, R. L., Bevan, B., & Garza, P. (2010). Museums afterschool: Principles, data, and design. Poster presented at University of Minnesota Extension Fall Program Conference. St Paul, MN: Regents of the University of Minnesota.

Nippolt, P. L. (2012). 4-H science: Evaluating across sites to critically examine training of adult facilitators. Journal of Youth Development, 7(4), 5.

Smith, M. (2008) Volunteer development in 4-H: Constructivist considerations to improve youth science literacy in urban areas. Journal of Extension [Online], 46(4) Article 4IAW2. Available at: http://www.joe.org/joe/2008august/iw2.php

Smith, M. (2013). Findings show lesson study can help be an effective model for professional development of 4-H volunteers. California Agriculture, 67(1), 54.

Smith, M., Meehan, C., Enfield, R., George, J., & Young, J. (2004). Improving county-based science programs: Bringing out the science teacher in your volunteer leaders. Journal of Extension [Online], 42(6) Article 6FEA5. Available at: http://www.joe.org/joe/2004december/a5.php

Stevenson, A. (2013). I Wonder . . . "I Wonder" boards serve as springboard for scientific investigations. Science and Children, 50(6), 74.

Torock, J. (2009). Experiential learning and Cooperative Extension: Partners in non-formal education for a century and beyond. Journal of Extension [Online], 47(4) Article 6TOT2. Available at: http://www.joe.org/joe/2009december/tt2.php