Differences Between Parent and School Personnel Observations of Extension Service Literacy Program's Impact with Children

Purpose

The purpose of the Energy Express Qualitative Research Study was to understand the program's impact with the children served from the perspective of their parents and school personnel. The research questions were:

1. Did the parents observe a change in children's reading as a result of participating in Energy Express?
2. Did the school personnel observe changes in children's reading as a result of participating in Energy Express?
3. Were there differences between the parents' and school personnel's observations?

Energy Express Description

Energy Express is a six-week literacy program designed to promote the school success of children living in low-income communities by maintaining their reading skills over the summer months. It aims to meet the twin challenges - erosion of skills that makes summer time costly for new readers and the nutritional decline faced by students accustomed to school meals (USDE, Every Child a Reader, 1999). College student mentors work with small groups of eight children, who attend five days a week for three and half-hours. A site coordinator, who is a schoolteacher, manages the Energy Express site.

Access to quality reading material should continue throughout a child's school years (USDE, Raising Readers, 1999). Using an integrated, literature based language arts curriculum focused on reading, the mentors guide the children through enriching experiences to make reading meaningful in their lives. Activities include shared reading, individual reading, writing, drawing, and creative arts projects.

Children stay in their small groups for family-style breakfast and lunch, practicing social skills and learning good nutrition. The meals help to maintain their growth over the summer and ensure that the children are ready to learn. The West Virginia University Extension Service coordinates this statewide program, and local collaborative partners coordinate and implement the program in Mineral County.

Quantitative Evaluation Results

Reading is the foundation for all learning. In West Virginia, 4 of every 10 fourth-grade students experience difficulty in reading, and among the remaining 6, only 40% are above a functional reading level (NAEP, 1998). Eighty-eight percent of children who have difficulty reading at the end of first grade display similar difficulties at the end of fourth grade (Juel, 1998). Children's school success is greatly diminished if they cannot read well by the end of third grade. Poor readers at the end of fourth grade comprise an overwhelming percentage of school dropouts, juvenile delinquents, and prison inmates (Kingery, 1999). Energy Express was designed to address these issues and the challenge of children losing ground during the summer.

Energy Express has exceeded its goal of children maintaining reading skills over the summer months. Significant differences in the reading achievement of participating Energy Express children were documented using pre- and post-Woodcock-Johnson Reading Achievement Subtest scores. Seventy percent of the children tested maintained or improved their scores, with the average "hypothetical child" gaining 1 month in word identification and 3 months in comprehension skills, at a time when nonparticipating peers may lose reading skills (Butera & Phillips, 1998).

A reading attitude pre- and post-survey with Energy Express youths found significant positive differences in children's attitudes about reading. Children experienced gains in how they feel about reading a book during free time, how they feel about reading a story with other students, how they feel about writing stories, and how they feel about making art project based on a story in a book (Butera, 1998).

Gains in reading scores justify the program's continuation, but what do the parents and school personnel think about the benefit to the children? Did they observe changes with the children that the quantitative data could not capture? Sustainability of Energy Express is dependent upon acceptance of the program among parents and school personnel within the community. A research study was conducted to learn about Energy Express's impact with the children from the perspective of their parents, teachers and principals.

Research Design

A qualitative evaluation research design was implemented. Data collection involved conducting interviews with a sample of parents and school personnel from each of the three Mineral County Energy Express sites, Elk Garden, Keyser Primary Middle, and Wiley Ford Schools. The Elk Garden School site serves a small school population of 180 students who are a part of the same school for grades K-8. Both the Keyser and Wiley Ford school sites serve much larger populations, 1100 and 764, respectively, with students coming from different feeder schools. A combined 160 children participated in the Mineral County Energy Express sites.

The sample represented the population of children and was 8% of the persons who could have been interviewed. Eight parents, eleven teachers, and two principals were interviewed, for a total of 21 persons from three sites.

The qualitative analysis method used involved the division of the data into relevant and meaningful groups while maintaining a connection to the whole (Gredler, 1996). A set of codes was developed to help identify initial categories of responses. The frequency of the code's occurrence helped to identify patterns and relationships emerging across the data. A matrix design was used to organize and analyze the information to make general statements about the data. 

Errors in the data translation may have occurred. Typed transcripts from hand-written notes could have had simple typing errors, and there may have been gaps in the information when the writer was unable to keep up with the speaker. Typed transcripts from audiotapes could have had errors from poor taping quality, the clarity of a speaker's diction, or unfamiliarity with a speaker's accent or speech patterns.

Major Research Findings

Parents' Perception of Program Impact

All of the parents interviewed felt Energy Express had positive influences with their children. Some of the parents noticed improvement in their child's reading that they attributed to Energy Express. One parent said it definitely helped my daughter read this year, and another said, "it helped to bring her son's reading up and that he is doing better in school this year than last year." Several parents said that their children were reading more independently, and one parent said that her daughter was reading longer books.

One parent noticed a change in the amount of the son's reading at home, indicating an increased motivation to read. "I noticed my son reading hunting magazines or he will grab the newspaper and go to the sports section and read the sports section. He is a West Virginia fan. He's got to read about West Virginia!... So he is now reading more than he did before."

One parent's goal was met in that her daughter had opportunities to read and maintained her skills. She said, "As far as I could tell her grades, from one year to the next year, they stayed about the same. That's what I was hoping for. Her not getting tired out from having that dry spell during the summer. Just to keep her grades up and her mind going... because in the summer time, if she wouldn't have been in the program, she wouldn't have wanted to read at all."

A parent mentioned her child's need to gain confidence in his ability to read. "Any extra reading that I can get him where he can get confidence (is important) and it has helped. (It is) important that he now realizes he can read. He really picked it up and is doing a lot better."

Several parents mentioned the social benefits for their children learning how to play together, read together, and create together in a positive small group environment. A parent said, "(It) helped her playing with other kids, (provided a) chance to play and have fun."

All parents interviewed felt that Energy Express was a good experience for their children. None of the parents described a negative impact of the program. "(I) loved the activities they did," said one person. A number of parents commented that the activities integrated with the reading were very good. One parent said, "They did a lot of projects based on the booksŠ" and another parent said, "(I) liked the interactive learning, (the program) involves so many things - hands on, music, and reading."

School Personnel's Perception of Program Impact

"Students loved the program!" said one teacher. Overall, teachers and principals were very positive about Energy Express and the program benefits. "It's a very different way of schooling. It's so much fun! It's so much hands on along with the literature." Another teacher said, "I've just been impressed with the whole idea of the program when I heard about it. ... I just think this is a wonderful program. What an opportunity for the children whether they are having reading difficulties or not."

Some teachers were aware that Energy Express goes beyond social development to help support a child's reading skills. "They do incorporate some skill lessons...which I thought was more appropriate because we have such low-test scores and it needed to be done." One principal said, "It (Energy Express) is a balance between cognitive, social and cultural (learning)."

A teacher observed increased confidence among children who were familiar with a story used in the classroom. "Whenever I've read When I was Young in the Mountains, or it could be any book, one of the kids will say 'I got that book when I was in Energy Express,' so they've already read it and they have something to talk about and that's good for self confidence and self-esteem."

Each of the teachers was shown a list of the students who attended Energy Express and, with one exception, knew at least one child on the list; most knew approximately three children who participated.

School Personnel's Observations from the Smaller School

School personnel from the smaller school noticed and discussed seeing changes among the children who attended Energy Express. Three of the teachers noted that the children either maintained or gained academic skills. "It seems that those students who attended Energy Express either maintained where they are, or they may even have moved up on the Johnsons (standardized reading test) because they have been reading ... all summer with instruction."

In comparing those students who attended Energy Express to those who did not, one teacher noted an increased interest in reading, and another teacher observed that Energy Express children seemed more eager to learn. "The achievement of academic skills (is one difference). Reading is fun (to them). They do a lot of reading. If they are finished they will find something to do. They don't just stare off into space."

School personnel noted improved social skills among children who participated as compared to those who did not attend Energy Express. "They are very social. I think they enjoy other people more than some others do," said one teacher. Another person said, "I noticed that the children who attended are very social, and cooperate more with others than some that did not participate."

A teacher mentioned that one child had shown great improvement, especially in working in a school environment. "He's improved a lot. More maturity wise. I know last year in first grade he had a lot of trouble with (a) structured environment and just basically getting into the gusto of school and going through the summer, even though it wasn't an instructional, sit down type situation, as opposed to school. It has helped him a lot in that and skill wise I think he is interested in achieving more."

Another person summarized, "Social and general behavior (changes were) shown most. (They) interact and work with their peers better. Reading and academic gains were slight, ...but (the) program (is) good for peer interaction."

Teachers from the smaller school stressed the value of having Energy Express available for children who live in geographically isolated areas. "Students here live so far away from one another, it certainly is good for them to get together and practice those social skills . . . and those kids have access to library books and books all summer long."

School Personnel's Observations from the Larger Schools

Most teachers and principals from the two larger school communities, while they responded to the questions, found it difficult to address whether there were differences among children who participated in Energy Express. There is very little opportunity for teachers to observe the same student from year to year. "I don't think that you ever know the difference ... related to the fact that you don't know them before," said one teacher. Another teacher said, "(It is) hard to tell, (I) don't see children again. (I) would like to think it helps. (One boy's) skills were still very low, now in February he is able to do words ... it could have helped him get ready to learn ... he did come in very low." Another teacher said, "(I) think those who did participate were in a position to benefit both academically and socially."

Some stated that the students who attended needed the program. "Most of the ones who participated here were ones who need an academic boost." Three students, one teacher noted, had severe learning issues, and all three really struggled. "I can't personally say that there was (a difference). But I've seen some of the ones I did have last year, they needed something to help carry them over. I'm sure they benefited from it because they would be the ones that as soon as they walked out the door it would be gone. . . . (I) would tend to believe that it would help them maintain."

A reading support instructor observed improvement in some children, and attributed the change to both programs. "Now some of them I do recognize they are in the first grade now and they are doing fairly well. A few are going to be brought up for learning disabilities. They would have those anyway. A couple of them I do recognize that are excellent readers now and that I have dropped them from my reading program because they have done so well." She also observed a specific benefit to one child. "His mother probably had to fight with him to get him to go. I see now that there's a difference. It has helped with him being positive."

Comparison of Indicators Between Parents and Teachers

Table 1 summarizes the difference and similarities in perception among the two groups and serves to answer the research questions.

Indicator	Parents	School Personnel
Reading Skills	More positive changes in reading skills and improvement in reading among some children	Fewer changes in reading skills and possible improvement in reading with a few children
Reading Independently	More reading on their own at home	More reading on their own between assignments during school
Motivation to Read	Increased interest in reading	Increased interest in reading
Socialization	Some socialization improvement	Significant socialization improvements
Group Skills	Did not comment on group skill changes	Group skills improvements
Confidence	Increased self-confidence	Did not comment about change in confidence

Research Question 1: Did the parents observe a change in children's reading as a result of participating in Energy Express? Parents did observe positive changes in their child's reading skills and more interest in reading.

Research Question 2: Did the school personnel observe a change in children's reading as a result of participating in Energy Express? School observed some changes in a child's reading and an increased interest in reading.

Research Question 3: Were there differences between the parents' and school personnel's observations? Parents observed more increases in reading skills and increases in self-confidence, whereas most teachers did not observe changes in these areas. Both groups discussed an increased interest in reading and more independent reading. Both noticed improved socialization; however, teachers noticed significant improvement including better group skills.

Conclusions

The major findings were that the parents value Energy Express and that some observed improvements in their children's reading. School personnel value Energy Express more for the program's holistic approach to child development and less for the program's impact on reading scores.

Based on Parent's Perception

Parents attribute a variety of changes to participation in Energy Express: improvement in reading, maintained reading, more reading at home, increased confidence in reading, and better social interaction skills. Some stated that Energy Express contributed to their child's improved school success.

Based on School Personnel's Perception

School personnel from the smaller school felt that Energy Express reached the goal of helping children to maintain their reading scores over the summer months, and they observed gains among some of the children. Because the school is smaller and the community close knit, teachers and students tend to be better connected. School personnel from the larger schools, and the more disconnected communities, had little opportunity to observe changes in children because they do not see them in consecutive years. Connections and relationships between teachers and children are significantly weaker in larger school environments.

Among school personnel, the greatest gains observed were improved socialization and increased interest in reading. Teachers were more apt than the principals or parents to note improvements in social skills and reading interest. Because teachers work with children in a classroom setting, there are more opportunities for them to observe group interaction skills and reading motivation. Some teachers noted a significant difference between children who had been a part of the Energy Express small groups compared to those who were not in the program. Those who had the small group experience could function better in group settings.

Implications

The following are implications for the Extension Service and school partners.

• Qualitative research is a complement to quantitative data collection, because it captures increased interest in independent reading at home and in the school environments.
• The qualitative research is consistent with the quantitative data. Parents and school personnel confirm a positive impact on children's reading.
• Future Extension studies should collect data from different groups. Parents and school personnel have different opportunities to observe children's development. Parents are more likely to observe a child's individual response to the program, whereas teachers have the opportunity to observe a child's response in the context of a group of children.
• Socialization and group skills are a benefit from summer reading programs that use a small-group approach.
• There are significant differences among teachers' connections with children in larger schools as compared to smaller school communities. Teachers from larger schools have little contact with the same children from year to year.
• Future Extension programming could be particularly valuable in larger school communities. If the teachers are disconnected from the children over time, Extension programs could provide this connection.
• Program implications are that Energy Express should be continued and expanded to other states as a holistic literacy model that contributes to children's academic success.

References

Butera, G., & Phillips, R. (1998). Energy Express program indicator report on pre and post Woodcock-Johnson Reading Achievement Subtest scores. West Virginia University Extension Service, Morgantown, WV.

Butera, G. (1998). Garfield reading attitude survey - Keyser Primary Middle School results. West Virginia University, Morgantown, WV.

Gredler, M.E. (1996). Program evaluation: Data analysis and interpretation. Englewood Cliffs, NJ: Prentice-Hall, Inc., 15, 295-307.

Juel, C. (1998). Learning to read and write: A longitudinal study of fifty-four children from first through fourth grade. Journal of Educational Psychology, 80, 437-447.

Kingery, B. (1999). Reading, English and language arts. Charleston, WV: West Virginia Department of Education.

Miltenberger, M. (1999). Energy Express evaluation research study Mineral County report. West Virginia University Mineral County Extension Service, Keyser, WV.

NAEP. (1998). The NAEP 1998 reading report card for the nation. Washington D.C.: U.S. Department of Education, Office of Educational Research and Improvement, National Center for Educational Statistics.

National Institute for Literacy. (1998). Fast facts in literacy & fact sheet on correctional education. Washington, D.C.: National Institute for Literacy.

U.S. Department of Education: America Reads Challenge. (July 1999). Every child a reader: How citizens, public leaders, and communities can help [Online]. Available at: http://www.ed.gov/pubs/startearly/ch_4.html.

U.S. Department of Education: America Reads Challenge. (July 1999). Raising readers: The tremendous potential of families [Online]. Available at: http://www.ed.gov/pubs/startearly/ch_1.html.