Helping Children by Educating Their Caretakers: An Evaluation of Knowledge Gain and Practice Implementation

Introduction

In 1940, Emily Post wrote: "The shortest part of life is the childhood. The training and guidance received during that period, however, determine the remainder of life" (p. 1). Early childhood is a critical time in human development (e.g., Ramey & Ramey, 1999). In the U.S., nearly one third of children under age five receive care in an organized care facility or family home day care (U.S, Census, 2002). It is imperative that child-care centers provide optimal experiences for these children. An important way that Cooperative Extension contributes to this pressing issue is through child care-provider training that promotes high-quality child care.

What is high-quality child care? Research reveals high-quality child care environments share several elements. Two of those elements are the child-care provider and care environment. According to Boschee and Jacobs (1998), important aspects of these elements include:

• A child-care provider who is warm, nurturing, and sensitive to children's needs; discipline that is positive and helps teach social skills.
• A care environment with numerous materials that are developmentally appropriate to children; play opportunities that enhance all aspects of development-physical, cognitive, social, and emotional.

The benefits of high-quality care are as astounding as the consequences of low-quality care are disheartening. Researchers have consistently found evidence that high-quality child care can enhance cognitive, language, and social skills (e.g., Burchinal, Cryer, Clifford, & Howes 2002; Love et al., 2003). Low-quality child care may result in social and emotional problems (Belsky et al., 2007). While research is limited, some studies have revealed that one way to improve child-care quality is with informal or specialized education (e.g., Haupt, Larsen, Robinson, & Hart, 1995; Kaplan & Conn, 1984; Snider & Fu, 1990). One study found child caregivers were motivated to improve their child-care setting and increase their involvement with children following a 20-hour training session (Kaplan & Conn, 1984). Researchers found that child-care teachers displayed an increase in developmentally appropriate beliefs and practices following an in-service training (Haupt et al., 1995). It is essential that efforts be made to create optimal child-care experiences.

Program Development and Description

In Louisiana in 2005, nearly 150,000 children were provided care in approximately 12,700 regulated facilities across the state (Nagle & Terrell, 2005). Thus, child care-provider training has the potential for serving a wide audience and producing beneficial impact. Initiated in 2000, Louisiana Child Care Provider Training Program (CCPTP) helps fulfill state-mandated training of child-care professionals. Training typically occurs in 3-hour sessions. Twelve hours of continuing education are required each year for child caregivers.

In 2004, several members of the CCPTP team (state and field faculty) met to create a new curriculum series to be implemented statewide. Concurrent to curriculum development activities, an evaluation plan for the program was developed. The team based the curriculum on Right from Birth, a book focused on parenting children ages 0 to 18 months using sensitive and appropriate care practices that enhance development. Mississippi Public Broadcasting (MPB) developed a video parenting series and workshop curriculum based on the book. With the authors' permission, the team created a new training series for child care providers that included the MPB parenting videos, original PowerPoint presentations, hands-on activities, and a multi-method evaluation design.

The LSU Right from Birth series consisted of four 3-hour sessions. Session topics included:

• Session 1: Brain development, social and emotional development, learning, and intelligence.
• Session 2: Elements of high-quality care and promoting school success.
• Session 3: Best-care practices for 1- to 6-month-old infants.
• Session 4: Best care practices for 7- to 18-month-old infants.

Several field agents conducted the classes throughout Louisiana. In keeping with the elements of quality care as defined by Boschee and Jacobs (1998), the program goals included child care providers' knowledge and use of developmentally appropriate, sensitive care practices. The specific learning objectives were that in following the Right from Birth training participants demonstrate increased knowledge and use of:

1. Responsive care practices that promote social and emotional development,
2. Developmentally appropriate discipline and limit-setting techniques,
3. Developmentally appropriate sensory environments, and
4. Developmentally appropriate discipline and limit-setting techniques.

Evaluation Methods

Procedure

The focus of this article is a program evaluation conducted between August 2004 and June 2005. The program evaluation assessed the LSU Right from Birth program's accomplishment of its objectives. There were three parts in the evaluations. First, to assess knowledge gain, a pre-test was given to all participants at the beginning of each Right from Birth training session and each participant completed a post-test at the end.

Two methods were used to assess practice implementation. Four weeks following the last training session, a survey was sent to participants who attended all sessions in the series. The purpose of the survey was to assess participants' use of practices recommended in the trainings. Participants who returned the survey by mail were contacted by telephone for a brief interview to assess practice implementation in more detail. Participants were asked to recall what particular changes they made in their care of children as a result of what they learned at the Right from Birth trainings.

Participants

The participants in the study reported here were child-care providers, including child care-center directors, lead teachers, assistant teachers, and family home day-care owners. The completion of the Right from Birth sessions fulfilled annual state-mandated training requirements for child-care professionals.

Participants signed up for each Right from Birth session individually with agents who worked in their geographic areas. Thus, the number of individuals participating in each workshop varied. For the present evaluation, 70 individuals completed Session 1, 57 completed Session 2, 164 completed Session 3, and 114 completed Session 4. Forty-four participants completed the entire training series and were invited to participate in the delayed post practice implementation survey.

Instruments

Pre- and Post-Tests

The LSU Right from Birth team members developed four pre- and post-test assessments. Pre- and post-tests consisted of an average of 10 - 12 questions each. Questions were developed based on overall program goals and learning objectives developed for each workshop and came directly from the workshop content. An expert in child development not directly associated with the program reviewed the questions for face and content validity. The program objectives addressed in each workshop pre- and post-test varied with the content of the workshops. See Table 1 for more detailed information on the items addressed in each workshop.

Table 1.
Pre- and Post-Test Questions Addressing Program Objectives

Session	Objective	Number of Questions
1	Responsive Care (Objective 1)	7
	Discipline and Limit Setting (Objective 2)	3
	Stimulating Environment (Objective 3)	3
2	Responsive Care (Objective 1)	4
	Stimulating Environment (Objective 3)	4
	Language and Literacy (Objective 4)	3
3	Responsive Care (Objective 1)	6
	Stimulating Environment (Objective 3)	6
4	Discipline and Limit-Setting (Objective 2)	6
	Stimulating Environment (Objective 3)	4

The pre- and post-tests were presented in a game format to be more appealing and less intimidating to participants.

Delayed Post Self-Report Survey

Several members of the LSU Right from Birth team developed a 17-item self-report survey. Participants were asked to indicate whether they performed a specific practice related to child care more, less, or the same as before the training. The survey items were developed based on overall program objectives and a thorough review of the training content. Critical child caregiving concepts and practices related to program objectives were identified in each training session.

Five items assessed objective 1; four items assessed objective 2; three items assessed objective 3; and five items assessed objective 4. Four items were reverse scored. An expert in child development not directly associated with the program reviewed the survey items for face and content validity. An example item includes: "I respond to a baby as soon as he starts crying," assessing objective 1.

Delayed Post Telephone Interview

A telephone interview protocol was developed to assess caregivers' use of practices they were taught in the training sessions. Questions were developed based on the overall program objectives. Participants were also asked free-recall questions to help determine what themes were most important to them.

Data Analyses

Data gathered from the pre- and post-test assessments were analyzed with SPSS software using paired-sample t-tests. Data from each session were calculated separately; they were not analyzed between sessions. After data from each session were recorded, answers to questions that addressed each learning objective were summed. Using SPSS analysis software, paired-sample t-tests were conducted to compare responses to items related to specific learning objectives for pre- and post-tests.

Data gathered from the delayed post survey were analyzed by totaling the number of responses to each of the 17 items. Data gathered from the telephone interview were transcribed and analyzed. QSR NUD*IST software for qualitative data analysis was used to identify and code major themes.

Results

Knowledge Gain

Based on the pre- and post-test assessments for all four training sessions, participants showed statistically significant knowledge gains in each of the sessions. The largest increase from pre- to post-test was in the first training session. A summary of the overall findings is included in Table 2.

Table 2.
Knowledge Gain in the Right from Birth Training

Session	Pre-Test		Post-Test		t-Value	df	P-Value
	Mean	SD	Mean	SD
1 (n = 70)	6.96	2.67	9.81	1.86	-7.73	69	<.001
2 (n = 57)	7.84	2.95	9.05	3.40	-2.74	56	.008
3 (n =163)	9.46	1.41	11.25	1.15	-13.17	162	<.001
4 (n = 112)	8.73	1.37	9.72	1.36	-6.64	111	<.001
n = the number of child care providers completing the pre- and post-tests.

Paired sample t-tests were also calculated for the main topics addressed within each workshop session. These tests also showed statistically significant knowledge gain for each of the four learning objectives. A summary of the findings related to questions addressing specific topics appears in Table 3.

Table 3.
Knowledge Gain in Right from Birth by Learning Objective

Responsive Care
Session	Pre-Test		Post-Test		t-Value	df	P-Value
	Mean	SD	Mean	SD
1	4.66	1.80	6.14	1.07	-6.13	69	<.001
2	3.28	1.11	3.46	1.02	-1.09	56	.279
3	4.67	.88	5.63	.66	-11.87	162	<.001
Discipline and Limit-Setting
Session	Pre-Test		Post-Test		t-Value	df	P-Value
	Mean	SD	Mean	SD
1	1.69	.88	2.21	.85	-4.61	69	<.001
4	4.31	.92	4.61	.86	-2.87	111	.005
Stimulating Environment
Session	Pre-Test		Post-Test		t-Value	df	P-Value
	Mean	SD	Mean	SD
1	1.14	.99	2.39	.69	-9.37	69	<.001
2	2.84	1.32	3.53	1.51	-4.161	56	<.001
3	4.74	1.30	5.64	.66	-8.47	162	<.001
4	3.36	.78	3.72	.54	-5.10	111	<.001
Language and Literacy
Session	Pre-Test		Post-Test		t-Value	df	P-Value
	Mean	SD	Mean	SD
2	1.81	.97	2.19	1.09	-2.36	56	.002

Practice implementation

Seven participants initially responded to the follow-up survey. Surveys were re-sent to those who did not respond the first time; however, no more surveys were received.

Those participants who responded to the survey overwhelmingly reported performing the recommended practices more or at least the same as before the training and the inappropriate practices less. Participants reported implementing recommended practices related to language development and literacy more so than other practices. Because of the small sample size, these results cannot be generalized. Additional evaluation of the series is necessary to verify any trends.

Six attendees indicated on their survey that they were interested in participating in a post-training telephone interview. Four participants actually were interviewed; the remaining did not answer the phone or continuously rescheduled. Again, the small sample limits the interpretation of the data. However, respondents did report that several of their practices changed as a result of the training series. Example responses to the interview questions appear in Table 4.

Table 4.
Sample Responses to Interview Questions

Topic	Response
Responsive Care	"I stopped letting him cry because I thought he would get spoiled...but I learned in class that at a certain age they can't get spoiled."
Sensory Stimulation	"I changed the entire décor of the infant room to include more black and white."
Language Development	"I stopped using 'baby talk.' I listened to the way the parents talked with their children and told parents and other workers what I learned." "I learned it is never too early to start reading."

Participants reported reading more to the children in their care more frequently than any other specific activity. Additionally, developmentally appropriate sensory stimulation activities were commonly reported. None of the four interviewees reported recalling any practices related to guidance and discipline learned in the training.

Conclusions

The evaluation of the Right from Birth series contributes to Extension in several ways. The information gathered is relevant not only to the LSU Right from Birth team, but also to Extension professionals across the country. Furthermore, some of the "lessons learned" in the present study may benefit Extension evaluation in family and consumer sciences and other areas.

First, the results of the Right from Birth evaluation provide evidence that child-care providers became more knowledgeable about appropriate care practices. This information indicates that a primary goal of the program was accomplished. Second, while the data are insufficient to draw strong conclusions, the results suggest that informal training may precipitate meaningful changes in the practices of child care providers. The content of the results may be useful to other family and consumer sciences agents interested in child care research. Finally, the multi-method approach is relevant to all areas of Extension. While the methods of this study require improvement, the basic design may serve as a model for evaluations in other areas of Extension.

The study had some limitations. The low responses to survey and telephone interview were a primary limitation, especially because they limit how generalizable the results are. Additional follow-up in the study was limited; home telephone numbers were only collected when participants returned surveys, restricting access to participants who did not return surveys. Future researchers might consider collecting participants' home addresses and telephone numbers to contact them directly rather than through their work locations. Alternatively, future methods might involve scheduling appointments with child-care centers and completing surveys in groups. Another method might include conducting focus groups at child-care centers or at a central location in order to get more participation and rich data.

The results of this evaluation are encouraging. The evaluation reported here adds to the growing body of research that shows that informal training can result in significant knowledge gain. While further verification through research is needed, the evaluation also suggests that training child-care providers on specific practices known to benefit children may lead to adoption of those practices.

Childhood is a critical time for the development of cognitive, social, and emotional health. Researchers have discovered methods of care that are best for children. Adult educators are teaching child-care providers those practices. Child care-provider training may help prevent children in care from experiencing developmental problems, improve school readiness, and promote positive mental health (Burchinal et al., 2002). Future research that evaluates practice implementation more rigorously could provide further evidence about the effectiveness of child care-provider training in improving child-care practices and environments.

References

Belsky, J., Burchinal, M., McCartney, K., Vandell, D. L., Clarke-Stewart, K. A., & Owen, M. T. (2007). Are there long-term effects of early child care? Child Development, 78, 681-701.

Boschee, M. A., & Jacobs, G. M. (1998). Ingredients for quality care. National Network for Child Care. Retrieved May 27, 2007, from: http://www.nncc.org/Choose.Quality.Care/ingredients.html.

Burchinal, M. R., Cryer, D., Clifford, R. M., & Howes, C. (2002). Caregiver training and classroom quality in child care centers. Applied Developmental Science, 6, 2-11.

Grace, C. (Writer), Lindsey, G. (Writer), Lymberis, T. (Director), Hartman, D. (Director), & Lemons, A. (Director). (2000). Right from birth: A parenting series. Jackson, MS: Mississippi Public Broadcasting & The Early Childhood Institute.

Love, J. M., Harrison, L., Sagi-Schwarts, A., van IJzendoorn, M. H., Ross, C., Ungerer, J. A., Raikes, H., Brady-Smith, C., Boller, K., Brooks-Gunn, J., Kisker, E. E., Paulsell, D., & Chazan-Cohen, R. (2003). Child care quality matters: How conclusions may vary with context. Child Development, 74, 1021-1033.

Nagle, G., & Terrell, D. (2005). Investing in the child care industry: An economic development strategy for Louisiana. Louisiana Department of Social Services. Retrieved May 27, 2007, from: http://www.dss.state.la.us/Documents/OFS/Investing_In_The_Chi1.pdf.

Post, E. (1940). Children are people. New York, NY: Funk and Wagnalls Company.

Ramey, C. T., & Ramey, S. L. (1999). Right from Birth. New York, NY: Goddard Press.

U. S. Census (2002). Child care arrangements. Retrieved May 27, 2007, from: http://www.census.gov/prod/2005pubs/p70-101.pdf.