How to Design Better Programs: A Staff-Centered Stakeholder Approach to Program Logic Modeling

What Is Program Design?

Program Design is both a product and a process. The process results in a theoretical framework for describing the effects and consequences of a program as they are related to its development and implementation.

The process employs two main concepts:

1. Program logic modeling and
2. Stakeholder viewpoints.

In the former, models are developed in schematic form of key aspects of how the program plan will be carried out in a sequential manner, by what staff, and with what consequences. In the latter, viewpoints are obtained from persons who have a special interest in or influence over the problem area being addressed in order to better inform the modeling process.

These two concepts are implemented through the efforts of a team of 6 to 15 persons who have expertise in developing and delivering programs related to the problem area of interest. This is called a "staff-centered approach." It has proven especially useful in bringing together and developing a consensus among persons who are separated due to boundaries established by geography, organization, disciplines, and, in some cases, even personalities.

This staff-centered approach is activated by a group facilitator(s) who directs and moderates the efforts of the group as they work their way through a sequence of disciplined steps in a series of workshop sessions spaced over a period of days, weeks, or months. The products from these sessions are codified and put in more readable form by an organizational contact person. Unlike the design team participants, the facilitator(s) does not need to be an expert in the subject matter under consideration. Indeed, such expertise might conflict with the conduct of his or her duties.

These efforts result in a program plan that is an agreed-to product of the design team's efforts based on their collective knowledge and experience. It enhances the likelihood of success of what will be done because those who are part of the team and/or their colleagues–those who must carry out the plan–have an explicit, agreed-to guide to action.

What Does the Process Entail?

Typically, two, 2-day sessions are required, with most of the work being done during the work group sessions However, with some complex topics such as Water Quality, Youth Development, or Leadership Development, a third or even fourth session might be required. The design team might also elect to hold a verification session in which program providers and subject matter specialists who were not part of the team are brought in to see how the modeling results conform to their thinking and experience.

Selection of the Design Team

The selection of design team members is an absolutely critical decision that will affect the success of the entire effort. The decision has two aspects: who and how many.

There is no hard and fast number for the latter. For experience and organizational mixture, 6 to 12 persons are usually involved. It is an absolute requirement that some of the members, preferably a majority, are program providers who work directly with clientele or potential clientele, in the topical area under consideration.

The remainder of the group is comprised of subject-matter specialists for the topic of concern and administrative staff. The involvement of an upper-level administrator may demonstrate the importance attached to the effort and hence motivate of the design team even more (provided, of course, that his or her presence does not inhibit the group's functioning). The program providers on the group serve as "reality filters" to ensure that what is proposed is practical or "do-able."

Finally, a person should be selected to serve as the workshop facilitator. This person should be experienced in the program design process and preferably have some training and/or experience in program evaluation. The facilitator must be a third party to the topic of concern and preferably should be a third party to the organization itself. Experience has shown that work group members are more inclined to attend to the tasks at hand if the facilitator is not "one of their own." Then, too, by being a non-expert, the facilitator can ask many "dumb" questions that can be revealing or even challenging without threatening the design team members. (Two facilitators reduce fatigue and increase variety for the team.)

Initiation of Activities

Once the design team has been convened, the activities are initiated. A handout of materials is used by the facilitator to introduce the group to the process. The first sheet contains an agenda for the entire workshop and is discussed by the facilitator in sufficient detail to provide some clarification and incentive.

Next, major concepts are introduced and explained through illustrations of actual and generic models, stakeholder identification and generic questions, interview guidance, etc. A brief history of the program design process and benefits of the process is presented. An actual example report is then given, also as a handout, that serves to familiarize the group with one of the major products.

The steps involved in program modeling are as follows.

1. A set of major or main events can be identified that comprise the program, its effects, and consequences, and that are sequentially and causally related to one another so that if one event fails to occur, then all of those succeeding it in the causal chain also fail to occur (the program logic model).
2. For each main event of the program, a set of activities with a corresponding set of resources, can be identified that must be accomplished in order for the main event to occur (the functional component).
3. For each activity in the functional component, one or more sources of evidence of the occurrence of that activity can be identified (the indicator component).
4. Things happen that can perturb or disrupt the causal relationships (barriers) but can perhaps be overcome by special efforts (barrier reductions).
5. Things happen once the program effects have occurred that perturb or prevent the consequences from taking place and are difficult or impossible to overcome by special efforts (intervening events); and.
6. For the occurrence of each main event in the program logic model, unplanned effects may also occur that can be positive or negative, known or unknown (spinoffs).

In order to inform this modeling process as well as other aspects of the program design process, information is collected from a judgmental sample (Patton, 1990; Henry, 1990) of stakeholders concerning their views about the nature of the problem, issue, or need and how it should be addressed, and by whom.

Once the preceding concepts have been discussed, the facilitator introduces the concept of stakeholders and gives a working definition, "an individual or group who has a special interest in or influence over the topic or program-to-be and who can provide information that will be useful for the design, development, implementation, and evaluation of the program," and asks the group to identify some general categories of stakeholders. This helps map the environment or delineate spheres of influence/concern for the topical area or program-to-be. It usually produces too many categories, some of questionable relevance or utility. The facilitator usually waits until later when some of the modeling has been completed and the group has a better sense of what they are about.

Matrix of Educational Effects

The modeling is initiated by starting with the development of the Matrix of Educational Effects. The group identifies the target audience(s) for the program to be and systematically completes the cells of the matrix, using categories from the Bennett hierarchy (1979) of: Knowledge, Attitudes, Skills, Aspirations, and Behavior (or practice) change. This matrix of effects (E) is then used as the basis for identifying their consequences (C) and antecedents (A). That is, what events logically follow as a result of E and what events must logically precede E in order to insure that E occurs, depicted as:

and read:

Once all of these events have been identified, the activities/resources are identified for the A events, and the indicators are identified for all of the events in the program logic model.

The modeling process is interrupted in order to deal in more detail with stakeholders. Some generic questions are reviewed and adapted, or new, more appropriate ones are developed. These may be further refined or tailored to different categories of stakeholders once they have been identified. A list of specific stakeholders is then identified. A contact letter and other interview procedures are developed.

Relevant documents, such as task force reports, evaluation studies of related topics, program plans, etc., are identified at this point. If there is a need to review them, some members of the group should be given the assignment of orally reporting them at their next meeting so that the results can be used by the group in their deliberations. Upon completion of this step, the team resumes the modeling process until it is time to recess.

The recess period usually lasts 6 to 12 weeks, during which time the organizational contact person sees to it that the interviews are conducted and transcribed. The length of this period is usually determined by the need for time to complete the stakeholder interviews and by the calendars of the group members.

On meeting again, the group divides up the interview results into groups of stakeholders, with at least two members reviewing each of the subgroups, discussing them with one another to reach agreement on their meaning, and making some cryptic summary notes. These summaries are then reported to the full group and entered into a matrix format of question answers by stakeholder groups (on banner paper posted on the wall) by the facilitator[s]).

After reviewing and discussing these summaries, the group makes some general thematic observations about their results and implications. The group then reviews and completes the modeling started in the first session. Finally, the group makes some conclusions and recommendations for administration, and a report of all the group's work is prepared, usually by the organizational contact. A briefing of the top-level administrator(s) by the team may be included in these efforts. Among other uses, this report serves as the "blueprint" for program development and implementation

What Are the Benefits from the Program Design Process?

The results of this process increase the likelihood that later efforts will be successful by:

• Involving staff in the process by giving them the time and the opportunity to meet together and reach a consensus on the "blueprint";
• Providing an explicit causal framework for articulating the program's nature, effects, and consequences;
• Identifying things that can go wrong and what might be done about them before they occur;
• Identifying unplanned results from carrying out the program in a particular way;
• Specifying sources of evidence that can be examined and/or obtained to judge adequacy of program implementation and degree of program impact;
• Involving persons of influence with respect to the topic and obtaining guidance from them before the program is developed; Clarifying to administration the nature of their commitment, especially with regard to resources;
• Increasing the likelihood of resource commitment through an explicit, agreed-upon plan that encompasses stakeholder viewpoints;
• Providing an effects oriented guide for program development;
• Providing a framework to communicate the program to others;
• Providing an agreed-upon "blueprint" for future action;
• Increasing the acceptance of measured effects once the program is implemented.

The process also has some direct benefits to the staff in terms of improving their program-planning and evaluation skills. In addition, the process can benefit the organization by increasing its visibility with stakeholders for the topic of concern.

References

Bennett, C. (1975) Up the hierarchy. Journal of Extension 13(2) pp. 7-12.

Henry, G. (1990) Practical sampling. Newbury Park, CA: Sage.

Mayeske, G. & Lambur, M. How to design better programs: a staff centered stakeholder approach to program logic modeling. Crofton, MD.: The Program Design Institute

Patton, M. (1990) Qualitative evaluation & research methods. Newbury Park, CA: Sage.