Drawings as a Method of Program Evaluation and Communication with School-Age Children

As interest in community-based and locally-directed prevention programs have grown, so has the competition for resources and the accompanying need to document program effectiveness. For childhood prevention programs that hope to include children's attitudes and feelings into the evaluative process, evaluators often must be creative in their search for methods that are developmentally appropriate and valid.

Unfortunately, program evaluators traditionally have struggled with how to incorporate children's perceptions into program evaluation plans. Research has shown that evaluation plans and instruments that are not appropriate for their audience can yield results which are tenuous and often unusable (Cronbach, 1982). The projective drawing technique described here to aid in program evaluation is based in part on the work of Koppitz (1983). She believes that drawing is a natural mode of expression for young children: "During the elementary school years, boys and girls can express their thoughts and feelings often better in visual images than in words" (p. 2).

In addition, this technique also reflects the work of others, including Buck (1948), Machover (1949), Burns and Kaufman (1970), and Knoff and Prout (1985), who have developed conceptual frameworks to interpret children's drawings. Many investigators have demonstrated that children's drawings can reflect self- concept, attitudes, wishes, and concerns (Golomb, 1992; Burns, 1982; Klepsch & Logie, 1982; Koppitz, 1968).

Although art activities have long been associated with children's programming, little has been written about using drawings as an evaluation tool of children's programs. Several authors (Koppitz, 1983; Rubin, 1984; Burns, 1982; Allan, 1978) have developed methods with which to interpret information from children's art work and drawings. These methods, as well as the use of other projective techniques have been used mainly for individual diagnostic purposes in clinical or educational settings. In such contexts, children's drawings have been used for a variety of assessment purposes, including intellectual development (Harris, 1963; Goodenough, 1926), learning disabilities (Cox & Howarth, 1989), personality (Prout, 1983; Wade, Baker, Morton & Baker, 1978; Hulse, 1951; Machover, 1949), and emotional adjustment (Koppitz, 1968).

The drawing activity described here, while conceptually linked to such individual diagnostic purposes, focuses on program effectiveness rather than individual assessment. The activity is one element of a multi-modal childhood education program evaluation that includes qualitative and quantitative components in an effort to comprehensively assess the program's objectives.

The program is a five year, federally-funded, locally developed and directed school-age educational program to teach life skills to limited resource families in Washoe County, Nevada. It is a collaborative venture between Nevada Cooperative Extension and several community agencies including the local housing authority, city recreation division, school district and Salvation Army. More than 1,000 children and their families participated in the first two years of the program at seven sites at housing authority projects, homeless shelters, and local schools. Programs have included an after school program, a summer day program, parent education workshops, a parent newsletter, family activities, and training youth service providers, staff, and volunteers.

Curricula for youth highlights life skills such as decision- making, communication, cooperation, conflict resolution, self- responsibility, literacy, nutrition and consumer education. The parent and staff curricula focuses on the same life skills as well as child development and guidance. In addition, the staff received training on the effects of poverty and homelessness, and how to recognize and report child abuse and neglect.

The premise of the program evaluation plan was based on the need to use multple sources and times for gathering data (Patton, 1982, 1990). Multiple sources and times allow compilation of a more comprehensive picture of program impact (Cronbach, 1982). Evaluators gathered information from school records, parents, teachers, program staff, and the youth themselves at various times each year of the program.

Data collected for the treatment group included school records (including academic grades, citizenship/academic behavior, citizenship/social behavior, and attendance); teacher, parent, and staff ratings (these were social skills rating inventories completed on each child [the Social Skills Rating System, Teacher and Parent Elementary Forms, Gresham & Elliot, 1990]); staff notes (staff documented case study vignettes as part of the qualitative data collection process); child ratings (children completed a short attitude survey regarding school, family, friends, and the after-school program); and the drawing activity--in which children engaged in an art activity drawing pictures about their thoughts and feelings regarding (a) school, (b) their family, and (c) the after-school program.

All of these data collection activities (except staff notes, which were done weekly, and school records, which were collected only at the end of the school year) were administered at the beginning and end of the school year for each child. In addition, the program evaluation described here was conducted only with those children enrolled in the after-school component of the program (about 100 children per year).

The evaluation attempted to determine the effectiveness of the program's objectives with regard to (a) increasing children's grades, school attendance, and perceptions towards school and the after school program, and (b) increasing childrens' life skills and positive perceptions toward family and friends. As part of the formative evaluation process, program staff also wanted to know what aspects of the program the participants liked or didn't like and why.

Only school records and teacher, parent and children ratings were collected for the control group. All instruments selected for the evaluation were normed on similar age populations. In addition, all letters, consent forms and questionnaires were translated into Spanish for those participants whose first language was Spanish. Program staff, who were bilingual, presented the projective drawing activity in Spanish as necessary.

This activity was conceived as a qualitative data collection process as well as an educational process--for children to learn about feelings and ways to appropriately express them. Staff were trained in the projective drawing technique by the authors who also helped initially administer the activity to children at each program site. The process for the art activity was as follows:

1. Staff led a discussion on feelings, asking such questions as "What are feelings?" and "What are some feelings that all of us have?"

2. With staff assistance, children listed as many feelings as they could. Staff helped in the identification of feelings and elicited examples from the children of the meaning of the feelings that they generated. This list of feelings then was written on poster paper and positioned so that all the children could see it.

3. Using colored markers and crayons, the group of children then decided together which color best represented each of the feelings they had identified. As each match was made, a scribble of that color was placed next to each feeling, creating a color/feeling template.

4. Using the color/feeling template that the group developed, the children then drew pictures about themselves and one of the aforementioned topics (family, school and the after- school program). They were told, however, that instead of drawing with colors, they would be using the color/feeling template to draw with feelings (e.g., if the children had decided that blue meant sad, and the topic of the drawings for that day was how they felt about school, a student doing poorly in school might draw her or himself with a blue marker sitting alone in a classroom).

5. Staff then asked the children, who wished, to tell the others about their pictures and describe their picture using feelings.

6. While the children described their picture, a staff member wrote down the child's comments and attached them to the child's drawing for later review.

7. Staff then collected the pictures and recorded the names, date, and color/feeling template for later discussion and review with the program evaluators and project director.

Although this activity was conceptualized principally as a qualitative formative assessment, the evaluation plan also proposes to have the sets of children's drawings from the final year of the project (when we anticipate having the largest number of matched sets of drawings) evaluated by three independent raters trained in child development and this drawing activity.

This will be based on a point system according to the following criteria: (a) the color/feelings template meaning to the drawing, (b) the drawing content, (c) children's comments about their picture, and (d) age of child. Once completed, inter- rater reliability will be established. These results, as well as short written reports on selected sets of drawings will then be compiled and summarized with the other program evaluation data.

For the drawing activity, children were grouped according to age. It is important to developmentally group children for this activity, since younger children (first and second graders) in our sample generated fewer feelings and needed more help with the concept of drawing with feelings. Older children (grades three through seventh), produced more sophisticated drawings and needed less guidance from program staff (thus, in Piagetian terms, the authors recommend that for this structured drawing activity children be at least in the concrete operational stage [ages 7 to 11], since children not yet in this stage will have difficulty with the concept of drawing with feelings instead of colors [Ginsberg & Opper, 1979]).

In general, all age groups exhibited growth in the number of feelings they could generate and accurately describe over the span of the program. However, because the control group did not engage in the drawing activity, we do not know if this is a result of the program or maturational processes.

As program developers, we believe that the greatest benefit of this art activity lies in the immediate and direct information and feedback the drawings can provide to program staff. Rich qualitative data have emerged with regard to the children's attitudes and feelings about the program and their social world.

Specifically, program staff report that the drawings have been tremendously helpful in conveying changes in family structure and environment, areas of concern, conflict or success at school, and attitudes about various aspects of the program.

In addition, this activity has been used as a measure of how effective the program has been in enhancing participant's life skills across social settings and as a screening device to detect abuse or neglect. This has helped program staff to be more responsive to individual children's needs and problems, and to modify program content and curriculum accordingly.

Since portfolio evaluation methods use art work, project reports, and/or journal entries, we believe this activity also could be useful in conjunction with these methods of evaluation which stress self-directed learning (Paulson, Paulson & Myer, 1991).

As with any data gathering activity involving children, protocols should be in place to address confidentiality issues and indications of abuse and neglect that may emerge. In addition, despite the seeming simplicity of this drawing activity, we caution that the individual interpretation of projective drawings requires extensive training and knowledge of psychological assessment and child development. We advise those interested in pursuing this activity for the purposes of program evaluation to consult with a professional who has the requisite graduate preparation and training.

Evaluators of children's educational programs must be creative in their attempts to meet the challenges posed by working with at-risk audiences. We found the projective drawing activity to be a developmentally appropriate method to gather data and to help with many of the evaluation challenges presented by our program's target audience: low literacy, primary languages other than English, varied educational levels, and diverse cultural backgrounds.

This drawing activity was developed as a fun, structured exercise to bring young children's views and feelings into the program evaluation process. As developers and evaluators of children's programs, we have found this activity useful as an educational and evaluative technique. If used as a component in a multi-modal design, childhood program evaluators can compare the projective drawing activity findings with other assessment sources. This allows for a richer and more sensitive assessment of a program's efficacy, as well as a better understanding of the children who participate in education programs.

Allan, J. (1978). Serial drawing: A therapeutic approach with young children. Canadian Counselor, 12(4), 223-228.

Buck, J. N. (1948). The H-T-P technique: A qualitative and quantitative scoring manual. Journal of Clinical Psychology, 4, 317-396.

Burns, R. C. (1982). Self-growth in families. New York: Brunner/Mazel.

Burns, R. C., & Kaufman, S. F. (1970). Kinetic family drawings (K-F-D): An introduction to understanding children through kinetic drawings. New York: Brunner/Mazel.

Cox, M. V., & Howarth, C. (1989). The human figure drawings of normal children and those with severe learning difficulties. British Journal of Developmental Psychology, 7, 333-339.

Cronbach, L. (1982). Designing evaluations of educational and social programs. San Francisco: Jossey-Bass.

Ginsberg, H., & Opper, S. (1979). Piaget's theory of intellectual development. Englewood Cliffs, NJ: Prentice-Hall.

Golomb, C. (1992). The child's creation of a pictorial world. Berkeley and Los Angeles: University of California Press.

Goodenough, F. L. (1926). Measurement of intelligence by drawings. New York: Harcourt, Brace, & World.

Gresham, F., & Elliott, S. (1990). Social skills rating system. Circle Pines, MN: American Guidance Service.

Harris, D. B. (1963). Children's drawings as measures of intellectual maturity. New York: Harcourt, Brace and World.

Hulse, W. C. (1951). The emotionally disturbed child draws his family. Quarterly Journal of Child Behavior, 3, 152-174.

Klepsch, M., & Logie, L. (1982). Children draw and tell. New York: Brunner/Mazel.

Knoff, H. M., & Prout, H. T. (1985). Kinetic drawing system for family and school: A handbook. Los Angeles: Western Psychological Services.

Koppitz, E. M. (1968). Psychological evaluation of children's human figure drawings. New York: Grune & Stratton.

Koppitz, E. M. (1983). Psychological evaluation of human figure drawings by middle school pupils. New York: Grune & Stratton.

Machover, K. (1949). Personality projection in the drawings of the human figure. Springfield, IL: Thomas.

Patton, M. J. (1982). Practical evaluation. Beverly Hills, CA: Sage.

Patton, M. J. (1990). Qualitative evaluation and research methods. Beverly Hills, CA: Sage.

Prout, H. T. (1983). School psychologists and social- emotional assessment techniques: Patterns in training and use. School Psychology Review, 12, 377-383.

Rubin, J. A. (1984). Child art therapy. New York: Van Nostrand Reinhold.

Wade, T. C., Baker, T. B., Morton, T. L., & Baker, L. J. (1978). The status of psychological testing in clinical psychology: Relationships between test use and professional activities and orientation. Journal of Personality Assessment, 42, 3-10.