August 2008 // Volume 46 // Number 4 // Research in Brief // 4RIB6
 |
 |
 |
Evaluation of a Wildlife Education Exhibit for Youth
Abstract
Understanding the ability of educational exhibits to communicate information effectively is important in all Extension programs. We evaluated the influence of a table-top exhibit entitled "Threats to Pennsylvania's Wildlife." In fair and classroom settings, participants in grades 5-7 (N=698) were randomly assigned to one of two experimental groups. One group participated in a pre- and post-survey; the other completed only a post-survey after viewing the exhibit. Results demonstrate that table-top exhibits are useful in conveying basic information and facilitating learning; however, the overall effectiveness of this exhibit was only about 10% and may have been influenced by additional confounding factors.
Introduction
Extension professionals frequently design and create exhibits and displays for conveying educational information to the general public or specific target groups. Historically, most studies involving exhibits have looked at demographics and visitor responses rather than gauging the effectiveness of the exhibit itself (Eason & Linn, 1976). Because many Extension programs use table-top educational displays, all program areas should have an interest in conveying the intended information and evaluating communication effectiveness.
Attempts to measure the educational value of exhibits have long been challenging. Shettel, Butcher, Cotton, Northrop, and Slough (1968) [as cited in Wells & Smith, 2000] pioneered one of the first studies that attempted to assess the effectiveness and usefulness of exhibits. They found that exhibits can be evaluated using a wide variety of techniques and that this line of research can improve exhibit effectiveness in the future.
Adams, Thomas, Lin, and Weiser (1989) measured the educational value of exhibits as a method of transferring wildlife information to high school students in Texas. Their results showed that students scored higher on the post-test after viewing the exhibit than the pre-test, suggesting a knowledge gain associated with using the exhibit. In another study, Klevans (1990) [as cited in Wells & Smith, 2000] reported that elementary students learn effectively and show a significant knowledge gain when presented with an interactive exhibit about endangered species. Finally, Falcao et al. (2004) suggested that a combination of basic background material, to allow for more simple comprehension, and complex analytical material may be presented through exhibits for an effective increase in knowledge gain.
Educational evaluation is often overlooked and avoided, but it is necessary to understand whether the information provided in the exhibit effectively reaches the audience (Heffernan, 1998). Allowing students to read and explore an exhibit on their own gives them control over their own learning and promotes individual comprehension and the formation of connections between new material and life experiences (Eason & Linn, 1976). Because the evaluation of exhibit and display effectiveness has frequently been identified as an area requiring further research (Landsittel, Murphy, Kiernan, Hard, & Kassab, 2001), and Cooperative Extension invests considerable resources into the production of these tools, the study reported here was carried out to assess the use of a specific table-top exhibit as a teaching tool for school-aged children in grades 5-7.
An additional objective was to compare and evaluate the use of an exhibit in two very different educational settings, in the classroom (formal setting) and at a fair (non-formal setting). The educational content of the exhibit covered the three greatest threats to wildlife in Pennsylvania (pollution, habitat loss and destruction, and invasive species). The information obtained from the study was sought to provide better insights into future use, planning, and exhibit design efforts.
Methods
The first segment of data collection took place at a large agricultural fair, Ag Progress Days, (APD) organized by Penn State Cooperative Extension. This event is staged each August in Rock Springs, Pennsylvania, and attracts more than 30,000 visitors over 3 days. While APD focuses on production agriculture, it is open to the public and provides an opportunity for families, children, and school groups to learn about agriculture and natural resources. We chose to target children ages 10-12 (grades 5-7) as these are the largest portion of youth visitors that attend APD. Three hundred and seventy-five children consented to participate in the study and completed surveys at the Threats to the Wildlife of Pennsylvania exhibit in August, 2005.
Second, four elementary schools (5th grades) and one middle school (7th grade) in State College Area School District (SCASD), State College, Pennsylvania, were also involved in the study from October through November, 2005. Three hundred and twenty-three youth participated in the study in the classroom setting. Prior to the school visits, an informed consent letter and form was sent home to the parents of children in the participating classes. The research was explained in the letter, and parents were given the opportunity to exclude their child from the study if they chose to do so. Data was collected from 698 participants at APD and SCASD combined.
Rockwell and Kohn (1989) suggested a "post-then-pre" method to accurately assess behavioral changes associated with Extension programs. Because the tests in our study were knowledge-based, we assigned participants to one of two experimental groups designed to account for potential bias. Group 1 was the "pre & post-survey group," and Group 2 was the "post-survey only group." The groups were allowed to view the exhibits at separate times. Children in both groups were permitted to take their time in viewing the exhibit and answering the survey questions. All the surveys were identical and served as the "tests" used for the study. This quasi-experimental design was used to observe and control for the "test-effect" of the Group 1 youth who saw the topics of interest on the pre-survey before viewing the exhibit.
The survey collected demographic information as well as general knowledge on items that threaten wildlife. Viewing the exhibit for approximately 5 minutes was the "treatment" between the pre- and post-surveys of Group 1 and before the post-survey for Group 2. The exhibit consisted of a free-standing, four-panel display with 12 color photos and brief captions about the subject. No formal instruction was given to the children before the surveys except that they should complete the survey on their own, and students' questions regarding subject matter were only fielded after all surveys were completed.
Data analysis consisted of a series of Chi-square and t-tests to identify differences between groups and test venue. We chose to use these evaluation methods to make simple comparisons that would be more understandable and applicable to all disciplines. We used t-tests to compare nominal data and Chi-square analysis for noting differences in categorical data, such as demographic information. We recognized alpha levels of 0.05, 0.01, and 0.001 in order to rank relative significance of differences detected.
Results and Discussion
Demographics and Test Effect
Descriptive statistics on the demographics of youth surveyed indicated significant differences in age and background between the children at the two venues (Table 1). Children attending APD (mean age= 10.49) were slightly younger than the students in SCASD (mean age= 10.85) [t= 241.58, sig=<0.001]. Children at APD were also more likely to report living on a farm than those in SCASD (APD= 42%; SCASD= 2.8%; χ2= 148.18, sig=< 0.001). Most of the children surveyed in the school district identified their home as being located in town (83%). Additionally, a higher percentage of girls were surveyed at APD (57% female participants), whereas the gender distribution was more equivalent in the school district (50% female participants). Although there was a 7% difference in gender between the venues, this difference was not statistically significant (χ2=3.28, sig= 0.068).
| Demographic | APD (n=375) | SCASD (n=323) |
| Mean age (years) | 10.49 | 10.85 |
| Female (%) | 57 | 50 |
| Male (%) | 43 | 50 |
| Farm residence (%) | 42 | 3 |
| Town residence (%) | 34 | 83 |
Finally, because of these demographic differences, the results from the two venues were analyzed separately, to avoid confounding interactions. There were no significant demographic differences observed between Groups 1 and 2 within each venue.
As mentioned in the methods section, the experimental design was constructed so as to control for the test effect associated with Group 1. Because these participants saw the questions before viewing the exhibit, they had a better idea of what information to focus on. This was shown in an analysis of Group 1 post-scores with Group 2 post-scores, where numerous significant differences were observed. To avoid this test effect, the pre-survey scores of Group 1 were used to represent the current knowledge of children about wildlife and natural resources, and these results were compared with the post-survey scores from Group 2 in the analysis for both locations.
Ag Progress Days
When the APD pre-survey data from Group 1 was compared with the post-survey data from Group 2, significant increases in knowledge were demonstrated after viewing the exhibit (Table 2). The three largest threats to Pennsylvania wildlife (habitat loss and destruction, pollution, and the spread of invasive species) were consistently chosen as correct answers by a higher percentage of children after they viewed the exhibit. In fact, many children already knew that habitat loss and destruction (43%) and pollution (64%) threaten wildlife in Pennsylvania before viewing the exhibit.
Several other choices that were widely (>18%) thought to be serious threats to wildlife in Pennsylvania were litter, hunting, forest fires, cars, diseases, and timber harvesting. Although these choices are all items that may affect wildlife, they are not considered among the top three "biggest threats" identified by the exhibit. The percentage of children selecting these options all decreased after viewing the exhibit (9% on average), and for three of these misconceptions (hunting, cars, and diseases) the decrease was statistically significant. In this regard, the knowledge gain by viewing the exhibit was corrective in nature.
The survey also asked participants to identify the correct definition of an "invasive species"; however, the percentages of children who correctly defined this term from both groups at APD indicated that this topic may not have been well addressed in the exhibit. A full 67% of participants (Group 1) correctly defined "invasive species" on the pre-survey, but only 66% were able to correctly define the term "invasive species" after viewing the exhibit (t=0.294, sig=0.769). It appears the exhibit did nothing to change youth understanding about what an invasive species is, but did increase their recognition (28%) of them as a threat to wildlife (Table 2).
| a | b | c | d |
| Group 1 (n=194) | Group 2 (n=181) | Increase/Decrease Between Columns b and c | |
| Choose the three items that most threaten PA wildlife. | pre-survey | post (only) survey | |
| invasive species | 2% | 30% | 28% *** |
| habitat loss and destruction | 43% | 62% | 19% *** |
| hunting | 31% | 16% | -15% ** |
| cars | 25% | 12% | -13% ** |
| pollution | 64% | 75% | 11% * |
| diseases | 21% | 10% | -11% * |
| forest fires | 26% | 19% | -7% |
| timber harvesting | 18% | 12% | -6% |
| cold weather/deep snows | 7% | 2% | -5% |
| railroads | 2% | 4% | 2% |
| tornadoes | 11% | 9% | -2% |
| litter | 33% | 31% | -2% |
| ATVs | 5% | 3% | -2% |
| bird watching | 0% | 1% | 1% |
| hurricanes | 5% | 6% | 1% |
| flooding | 4% | 5% | 1% |
| hiking | 1% | 1% | 0% |
| fishing | 2% | 2% | 0% |
| Notes: Percentages are organized by absolute change (column d) in the table, but were randomly listed in the survey. We used t-tests to identify significant differences. *p < .05, **p < 0.01, ***p < 0.001. | |||
State College Area School District
As seen with participants at APD, significant increases in correct answers were observed after viewing of the exhibit in the classroom (Table 3). The three largest threats to Pennsylvania wildlife were consistently chosen as correct answers by a higher percentage of children after they viewed the exhibit (habitat loss and destruction, pollution, and invasive species). As also observed at APD, many children at the schools already had prior knowledge of threats to wildlife in Pennsylvania such as habitat loss and destruction (57%) and pollution (68%) before viewing the exhibit.
| a | b | c | d |
| Group 1 (n=151) | Group 2 (n=172) | Increase/Decrease Between columns b and c | |
| Choose the three items that most threaten PA wildlife: | pre-survey | post (only) survey | |
| invasive species | 7% | 30% | 23% *** |
| pollution | 68% | 86% | 18% *** |
| timber harvesting | 20% | 10% | -10% * |
| diseases | 14% | 5% | -9% * |
| forest fires | 21% | 13% | -8% * |
| hunting | 27% | 20% | -7% |
| habitat loss and destruction | 57% | 63% | 6% |
| flooding | 8% | 2% | -6% * |
| litter | 29% | 25% | -4% |
| hurricanes | 7% | 10% | 3% |
| railroads | 3% | 1% | -2% |
| ATVs | 2% | 0% | -2% |
| fishing | 1% | 2% | 1% |
| cars | 20% | 21% | 1% |
| cold weather/deep snows | 4% | 4% | 0% |
| hiking | 1% | 1% | 0% |
| tornadoes | 7% | 7% | 0% |
| bird watching | 2% | 2% | 0% |
| Notes: Percentages are organized by absolute change (column d) in the table, but were randomly listed in the survey. We used t-tests to identify significant differences. *p < .05, **p < 0.01, ***p < 0.001 | |||
Several other choices that were widely (>14%) perceived to be serious threats to wildlife in Pennsylvania prior to viewing the exhibit were litter, hunting, forest fires, cars, timber harvesting, and diseases. The percentage of children selecting these options significantly decreased after viewing the exhibit (7% on average), and for three of these misconceptions (forest fires, timber harvesting, and diseases) the decrease was shown to be statistically significant. Again, similar to the APD results, the knowledge gain by viewing the exhibit was corrective.
Most of the school children (75%) correctly defined "invasive species" on the pre-survey; however, only 71% correctly defined this term after viewing the exhibit. The decrease (4%) was not statistically significant and, again, suggests that the definition of an invasive species was not clearly presented in the exhibit (t=0.800, sig=0.424). However, the school students did increase their understanding about the threat of invasives to wildlife (23% change) after viewing the exhibit (Table 3).
Venue Comparisons
Because there were so many demographic differences in the children surveyed at each site, comparisons between APD and SCASD could not be made. However, some broad comparisons can still be drawn. Children in the school environment tended to be slightly better educated about some wildlife topics prior to completing the survey. For example, more SCASD children were already aware of habitat loss and destruction and its effect on wildlife (57%) as compared to the APD participants (43%). This might be attributed to a stronger science curriculum in the school district or a better classroom learning environment. Further, most of the school children identified their home as a town and may have had more direct experience with development and loss of important wildlife habitat than their rural counterparts at APD.
SCASD students showed slightly less overall change in their knowledge after viewing the exhibit (average of 6%) than the participants at Ag Progress Days (average of 7%). Children surveyed in SCASD exhibited less change in their knowledge because they were more generally informed about the main things that are threatening to wildlife prior to completing the survey. Introducing participants to the threat of invasive species was the most significant change brought about by viewing the exhibit in both locations.
Conclusions
Our findings suggest that the children's knowledge gain from viewing the exhibit was consistent, but modest (about 10% overall), and that the exhibit was most effective in teaching participants the three biggest threats to wildlife in Pennsylvania. Almost half of the children were already aware that pollution and habitat loss and destruction were significant threats to wildlife. However, many of the participants were not familiar with the threat of invasive species. The results suggest that, while children learned that invasive species were a threat to wildlife in Pennsylvania, they were not proficient in defining what invasive species were, after viewing the exhibit. Because of the broad context of invasive species, including animals, plants, and diseases, the average person may not understand how to identify these species and their impacts on the environment. Our findings suggest that more education on invasive species is needed to effectively teach this broad concept.
Participants in both locations most frequently selected litter, hunting, forest fires, cars, timber harvesting, and disease as one of the three biggest threats to wildlife prior to viewing the exhibit. The percentage of children selecting these and most other "incorrect" answers decreased or remained constant after they viewed the exhibit in both locations. The middle school children in this study seemed to be misinformed about the overall impact of these items on wildlife populations in Pennsylvania.
There were a few confounding issues with the experimental design and data collection methods. At the Ag Progress Days site, some children were accompanied by parents, and they may have been rushed or distracted. One way to control for distraction in the fair setting might be to have an exhibit, activity, or resting place to occupy parents while youth were participating in the study. These were not issues in the classroom setting. The classroom may have provided a slightly better, and more controlled learning environment for the use of this exhibit as compared to the fair-like setting at APD.
Further research could be developed around assessing effectiveness of different types of exhibits, primarily those that include an interactive component for children to better relate to the issue at hand. The research could also be expanded to encompass a wider age range, because the study reported here focused on upper elementary youth in grades 5-7. More evaluation of exhibit effectiveness at various age levels could provide additional useful information to extension educators. The effectiveness of the exhibit itself may have been improved by including larger print and pictures, ensuring a quiet venue for viewing the exhibit and taking the survey, and separating the three main topic panels of the exhibit to reinforce the main concepts. Also, research completed at various venues may give Extension educators more information about how children learn and what types of strategies are most effective with different groups at different locations.
By understanding how and what people learn from exhibits, Extension professionals can gain better insights to their audiences and effective communication efforts. It is especially important to properly educate children about wildlife so they can understand the importance of the threats and issues affecting their conservation. Because children are the future of natural resource conservation, providing accurate information to them in an educational way is essential in both formal and informal settings.
References
Adams, C. E., Thomas, J. K., Lin, P. C., & Weiser, B. (1989). Promoting wildlife education through exhibits. Journal of Research in Science Teaching, 26(2), 133-139.
Eason, L. P., & Linn, M. E. (1976). Evaluation of the effectiveness of participatory exhibits. Curator, 19(1), 45-62.
Falcao, D., Colinvaux, D., Krapas, S., Querioz, G., Alves, F., Cazelli, S., Valente, M. E., & Gouvea, G. (2004). A model-based approach to science exhibition evaluation: a case study in a Brazilian astronomy museum. International Journal of Science Education, 26(8), 951-978.
Heffernan, B. M. (1998). Evaluation techniques for the Sandy Point Discovery Center, Great Bay National Estuarine Research Reserve. Journal of Environmental Education, 30(1), 25. Retrieved June 30, 2006 from http://www.proquest.com/
Klevans, M. (1990). An evaluation of an interactive computer exhibit in a museum setting. Unpublished doctoral dissertation. University of Texas, Austin. [as cited in Wells & Smith]
Landittel, D. P., Murphy, D. J., Kiernan, N. E., Hard, D. L., & Kassab, C. (2001). Evaluation of the effectiveness of educational interventions in the Pennsylvania Central Region Farm Safety Pilot Project. American Journal of Industrial Medicine, 40, 145-152.
Neathery, M. F. (1998). Informal learning in experiential settings. Journal of Elementary Science Education,10(2), 36-49.
Rockwell, S. K., & Kohn, H. (1989). Post-then-pre evaluation. Journal of Extension [On-line], 27(2). Available at: http://www.joe.org/joe/1989summer/a5.html
Shettel, H. H., Butcher, M, Cotton, T. S., Northrop, J., & Slough, D. S. (1968). Strategies for determining exhibit effectiveness. Report No. AIR E95 4/66-FR. American Institute for Research, Washington D.C. [as cited in Wells & Smith]
Wells, M., & Smith, L. (2000). The effectiveness of nonpersonal media used in interpretation and informal education: An annotated bibliography. National Park Service, United States Department of the Interior. Harpers Ferry Center, Harpers Ferry, West Virginia.