Closing the Science Comprehension Achievement Gap Through the Use of an Extension Youth Development Science Comprehension Model

Introduction and Conceptual Framework

National education achievement trends indicate that traditional teaching methods are not successfully preparing students in kindergarten through grade 12 in science (U.S. Department of Education, 2009) and that transformative approaches to teaching science are needed (National Research Council, 2007, 2012). Reforming science education includes enhancing the experiences of youths by engaging them in high-quality programs (Skelton & Dormody, 2009). Given the emphasis on high-stakes testing and declining student achievement in science, the Cooperative Extension System has an important role to play in improving youth academic achievement through youth development enrichment programs and special interest topics.

The Memorial Middle School Agricultural Extension and Education Center (MMSAEEC) is a youth science center where inquiry-based learning and experiential education are emphasized (Skelton & Seevers, 2010). The MMSAEEC is a partnership between the New Mexico State University Cooperative Extension Service and a public school district in Las Vegas, New Mexico. It was established for the purpose of enhancing learning opportunities for a predominately underserved Hispanic population. The mission of the MMSAEEC is to deliver educational programs in agriculture and natural resource science to youths attending Memorial Middle School. A basic premise of the mission involves development of a teaching and learning model of excellence for agriculture and natural resource science that complements in-class instruction by providing context to content through hands-on learning opportunities (Skelton & Dappen, 2015). Science, technology, engineering, and math education is delivered through the teaching of earth, life, and physical science; sustainable agriculture; and natural resource conservation systems. Through MMSAEEC programs, students are engaged in the classroom, lab, and field, conducting experiments and research by using scientific practices that are representative of what agricultural and natural resource scientists do.

The conceptual framework for the study reported here was based on a model for improving youth science comprehension (Skelton, Seevers, Dormody, & Hodnett, 2012). This youth science comprehension model focuses on the development of and interactions among science knowledge, science skills, and reasoning abilities that form a broader contextual understanding and improved comprehension of science subject matter by youths (Figure 1). The model incorporates features of inquiry-based learning and experiential education that research suggests are essential for effective science learning (Blair et al., 2004; Bourdeau, 2004; Smith, 2008; Clarke, 2010). Furthermore, higher level thinking skills, experiential education, and inquiry-based learning are all essential if science is to become concrete in the minds of students (Center for Science, Mathematics, and Engineering Education, 1996). The model describes a promising teaching methodology that promotes student learning and effects the engagement of learners in scientific investigation, where students learn to ask meaningful and researchable questions, develop hypotheses, and conduct experiments. In short, youths begin to develop the skills necessary to analyze and interpret data, cite evidence, and communicate findings. Research has demonstrated that this approach leads to greater understanding of science (National Research Council, 2000).

Figure 1.
Conceptual Model for Improving Youth Science Comprehension

In the study reported here, we tested the model to understand whether youth development educational impacts are occurring as theorized, recognizing that if so there would be broad implications related to the potential effects of Extension programs on youth development in the sciences. The research addressed whether programs delivered through the MMSAEEC led to improved outcomes in agriculture and natural resource science for the predominantly Hispanic and economically disadvantaged population. Research testing the model is needed to validate and enhance its intellectual merit and impact and to make important contributions to the research base and practice of teaching and learning through Extension programs.

Purpose and Research Objectives

We sought to determine by assessing potential student growth whether applying the features of the youth science comprehension model to a youth development program led to improved student academic achievement. We also sought to determine on the basis of students' academic performance levels (at grade level or below) whether the model affected student achievement. Specifically, the study focused on the following objectives:

1. Describe selected demographics of the participating students.
2. Determine student growth within each of two programs on the basis of data from pretest and posttest assessments.
3. Compare program outcomes with regard to students' academic achievement levels.

Methods and Procedures

The study was a census of six classes of sixth-grade students in earth science and five classes of eighth-grade students in physical science. Sixth-grade students participated in a soil pH program and eighth-grade students in a water chemistry program delivered as school enrichment programs through the MMSAEEC. We developed a guided inquiry framework for program content and greenhouse experiments to engage students in scientific investigation.

We designed pretest and posttest assessments to measure students' science knowledge, skill development, reasoning abilities, and overall science comprehension growth as a result of each program treatment. The assessments contained nine items, with three questions used to measure each of the three subdimensions (science knowledge, skill development, reasoning abilities). Reliability of the pretest and posttest program assessments was established with a test-retest method where instruments were administered at the onset of a program and immediately after program completion. We developed valid and reliable instruments that reflected content delivered through MMSAEEC programs based on New Mexico agriculture, food, and natural resource content and performance standards (Castillo, 2003). A panel of experts reviewed the instruments to establish face and content validity.

The differences between the pretest and posttest assessment scores (paired by student) were used to determine the students' growth as a result of the program treatment. Additional data were collected from the school district on student performance levels (i.e., whether students were performing at or below grade level). Means and standard deviations were calculated for the student pretest and posttest assessment scores. A t-test was used to determine whether there was a significant difference between growth scores for students who were performing at grade level and students who were performing below grade level at the time of assessment. Students were removed from the analyses if they did not attend all program sessions, data were incomplete on either the pretest or the posttest, or performance level data were not obtainable (n = 21). We determined that members of the group removed from the analyses were demographically similar to the students included in the analyses. The data were analyzed through the use of SPSS version 22.

Results

The first objective of the study was to describe selected demographics of participating students (Table 1). The study took place at the MMSAEEC, which is located in rural northern New Mexico. The student population was primarily Hispanic, and a majority of the student population was classified as economically disadvantaged. The youths in the study were drawn from sixth- and eighth-grade science classes from the participating school. The sixth-grade classes had 88 students, of which 41 were female and 47 were male. Fifty-two students were performing below grade level. The eighth-grade classes had 43 students, of which 20 were female and 23 were male. Fourteen students were performing below grade level.

Table 1.
Demographic Characteristics

Characteristic	Soil pH program (6th-grade students, N = 88)	Water chemistry program (8th-grade students, N = 43)
Hispanic ethnicity	89.772%	83.721%
Gender (female)	46.590%	46.512%
Below-grade-level performance	59.091%	32.558%

The second objective of the study was to determine student growth scores within each program through the use of data from pretest and posttest instruments (Table 2). Mean scores for students performing at grade level on the soil pH program were 48.150% (SD = 18.037) on the pretest and 79.956% (SD = 13.493) on the posttest. Average student growth for students performing at grade level on the soil pH program from the pretest to the posttest was 31.806%. Mean scores for students performing below grade level on the soil pH program were 34.806% (SD = 14.932) on the pretest and 64.119% (SD = 15.973) on the posttest. Average student growth for students performing below grade level on the soil pH program from the pretest to the posttest was 29.313%. Mean scores for students performing at grade level on the water chemistry program were 52.874% (SD = 19.621) on the pretest and 68.966% (SD = 17.412) on the posttest. Average student growth for students performing at grade level on the water chemistry program from the pretest to the posttest was 16.092%. Mean scores for students performing below grade level on the water chemistry program were 29.365% (SD = 13.508) on the pretest and 63.492% (SD = 18.194) on the posttest. Average student growth for students performing below grade level within the water chemistry program from the pre-test to the post-test was 34.127%.

Table 2.
Student Grade Level Performance and Test Scores by Program Area

Program	Grade level performance	N	Pretest score		Posttest score
			Mean % correct	SD	Mean % correct	SD
Soil pH	At grade level	36	48.150	18.037	79.956	13.493
	Below grade level	52	34.806	14.932	64.119	15.973
Water chemistry	At grade level	29	52.874	19.621	68.966	17.412
	Below grade level	14	29.365	13.508	63.492	18.194

The third objective of the study was to compare program outcomes relative to students' academic achievement levels (Table 3). A t-test was used to determine differences between mean score differences of students performing at grade level and students performing below grade level for each program. It was determined that there was no significant statistical difference in the mean student growth scores for the soil pH program (t = 0.624, p > .05). However, significant statistical difference in the mean student growth scores was found for the water chemistry program (t = −4.053, p < .05).

Table 3.
Grade Level Performance Percentage Growth Scores

Program	Grade level performance	Growth scores (% difference)		t	df	Sig.
		M	SD
Soil pH	At grade level	31.806	18.822	0.624	86	.535
	Below grade level	29.313	18.162
Water chemistry	At grade level	16.092	13.474	−4.053	41	.000*
	Below grade level	34.127	14.098
*Means significantly different, p < .05.

Conclusions, Implications, Recommendations

Results from the study indicate that the model by Skelton et al. (2012) holds promise for improving youth science comprehension. Results show that youths exhibited growth across both sixth- and eighth-grade programs. A lack of significant difference between student groups on the soil pH program might not be surprising. Scores on the pretest were low for both groups. This could be due to the fact that science is not taught regularly at the elementary schools and that as a result these concepts were more abstract, causing all students to struggle initially. Of particular promise is the growth exhibited by youths who were performing below grade level. Posttest assessment scores for the soil pH program demonstrate that those students made very similar gains to their peers performing at grade level. Posttest assessment scores on the water chemistry program demonstrate that the students performing below grade level made up considerable ground and were within 5.5 percentage points of students performing at grade level. It is critical that students performing below grade level not be further left behind.

There are several important implications of the research for the delivery of Cooperative Extension System programs. The results indicate that programs developed by Extension personnel for youth development school enrichment or special interest topics can benefit from (a) the incorporation of features of inquiry-based learning and experiential education and (b) a focus on the three contributing subdimensions of science comprehension: science knowledge, science skills, and reasoning abilities. Methods that link knowledge to skill development and reasoning abilities must be used to achieve the broadest understanding by learners. The results reported here also are consistent with the need for enhanced learning opportunities and improvement of minority student achievement scores through nontraditional programs that facilitate career pathways. New approaches to teaching and learning science are of the utmost importance to reverse current education trends for minority students, in particular Hispanic students. Minority students face numerous academic barriers to achievement, both in the classroom and outside of school (Barton, 2003). Proficiency in science is low among Hispanic students, and these deficiencies persist into undergraduate education, graduate education, and the workforce (Gasbarra & Johnson, 2008; National Science Foundation, 1994).

Further research is needed to determine whether different types of programs affect student academic growth similarly through the use of the model. Researchers could also expand the number of questions used to measure each subdimension of science comprehension. To enhance generalizability of the results, the study reported here should be replicated in different populations and through the use of various Extension youth development science programs to determine whether the model affects academic achievement similarly. The findings from our study indicate that there is an opportunity for Extension youth development science programs at the elementary grade level (i.e., 4-H novice school enrichment programs or afterschool special interest topic programs). Finally, Extension programs can play a vital role in improving science achievement in the United States. It appears that programs following this model provide effective preparation for future scientists because the model matches the scientific method they will employ throughout their careers.

References

Barton, P. (2003). Parsing the achievement gap. Princeton, NJ: Educational Testing Service.

Blair, R. B., Meyer, N., Rager, A. B., Ostlie, K., Montgomery, K. L., & Carlson, S. (2004). Best practices for environmental field days: Structuring your event for fun and learning. Journal of Extension, 42(5) Article 5TOT4. Available at: http://www.joe.org/joe/2004october/tt4.php

Bourdeau, V. D. (2004). 4-H experiential education: A model for 4-H science as inquiry. Journal of Extension, 42(5) Article 5TOT3. Available at: http://www.joe.org/joe/2004october/tt3.php

Castillo, S. (2003). New Mexico agriculture, food, and natural resources content standards and benchmarks. Retrieved from http://www.nmffa.org

Center for Science, Mathematics, and Engineering Education. (1996). From analysis to action: Undergraduate education in science, mathematics, education and technology. Washington, DC: National Academies Press.

Clarke, K. C. (2010). A science, engineering and technology (SET) approach improves science process skills in 4-H animal science participants. Journal of Extension, 48(1) Article 1IAW3. Available at: http://www.joe.org/joe/2010february/iw3.php

Gasbarra, P., & Johnson, J. (2008). Out before the game begins: Hispanic leaders talk about what's needed to bring more Hispanic youngsters into science, technology, and math professions. Retrieved from http://www.publicagenda.org/files/pdf/outbefore.PDF

National Research Council. (2000). Inquiry and the national science education standards: A guide for teaching and learning. Washington, DC: National Academies Press.

National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academies Press.

National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts and core ideas. Washington, DC: National Academies Press.

National Science Foundation. (1994). Women, minorities, and persons with disabilities in science and engineering. Retrieved from http://www.nsf.gov/statistics/wmpdse94/chap2/minorits.htm#fn1back

Skelton, P., & Dappen, P. (2015). Memorial Middle School Agricultural Extension and Education Center [Website]. Retrieved from http://mmsaeec.nmsu.edu

Skelton, P., & Dormody, D. (2009). Adding context to content: The Memorial Middle School Agricultural Science Center model. The Agricultural Education Magazine, 81(5), 26-28.

Skelton, P., & Seevers, B. (2010). A new Extension model: The Memorial Middle School Agricultural Extension and Education Center. Journal of Extension, 48(6) Article 6IAW5. Available at: http://www.joe.org/joe/2010december/iw5.php

Skelton, P., Seevers, B., Dormody, T., & Hodnett, F. (2012). A conceptual process model for improving youth science comprehension. Journal of Extension, 50(3) Article 3IAW1. Available at: http://www.joe.org/joe/2012june/iw1.php

Smith, M. H. (2008). Volunteer development in 4-H: Constructivist considerations to improve youth science literacy in urban areas. Journal of Extension, 46(4) Article 4IAW2. Available at: http://www.joe.org/joe/2008august/iw2.php

U.S. Department of Education. (2009). The facts about science achievement. Retrieved from http://www.ed.gov/nclb/methods/science/science.html