Program Integrity: A Powerful Organizing Construct or Just More Jargon?

Introduction

Extension educators typically are action-oriented individuals who turn first to proven, experience-based approaches for developing and documenting programs. Structured, deliberate programming approaches can be a tough sell, particularly if imposed by administrators. Articles describing efforts to promote alternative approaches among educators repeatedly note resistance to change. Examples include "shying away" from use of formal evaluation methods (Diem, 2002), "resistance to organizational change" (Washington & Fowler, 2005), "resistance to scholarly approaches" (McGrath, 2006), and even "the Dragon of Resistance" (Franz & McCann, 2007).

Yet, in their communities, day in and out Extension educators serve as influential change agents. It can't be that change per se is the issue. The challenge may well lie with those of us promoting intentional approaches for Extension programming.

More often than not, program development is parsed out into component elements such as curriculum development, marketing, accountability, evaluation, and impact communication, with little or no emphasis on an overall, rational and convincing framework. It's not surprising that the resulting stream of new initiatives typically is met by an attitude of "this too shall pass" as new priorities emerge and sanctioned approaches and organizational leadership change. Theory and models are often covered, but it seems we typically don't quite get to the basic motivation. Through recent development and introduction of a program leadership curriculum we clearly exposed the need for a compelling, integrating theme and believe we have identified a viable candidate—program integrity.

The origins of the approach are humble. During a review of program development and assessment tools for a statewide gathering of county Extension directors, one frazzled participant asked, "Given the volatility of the local program environment, why should we invest in such deliberate and time-consuming approaches?" We responded: "to ensure the integrity of the programs we offer." It was one of those rare moments when a concept resonated with the audience and conveyed fully the intent of the approaches being offered. The seed for a working concept of program integrity was planted.

That seed germinated when we embarked on a comprehensive Program Leadership Certification effort to support those in program leadership roles. The curriculum included introduction to leadership models and approaches, logic models, program advisement and governance, accountability and evaluation, and managing commitments supplemented by an applied project. In other words, we generated yet another laundry list of program development topics similar to many that have come before. As we reviewed materials, we realized that the foundational components we were seeking were found in instructional design, ethics, and public accountability and that those, in turn, comprised the basis for a statement of program integrity.

Program Integrity Defined

We drafted our definition and then reviewed literature from multiple fields to avoid reinventing a basic concept. We found that program integrity in the public sector often is associated with fraud prevention and public accountability. Terms such as "accountability," "honesty," and "trustworthiness" were cited frequently. For example, the Pennsylvania Department of Public Welfare (2009) frames program integrity as addressing fraud and abuse, ensuring that clientele receive quality services and that feedback is provided to enhance program performance. Our colleagues in the United States Department of Agriculture Risk Management Agency (2009) apply the concept of program integrity in terms of compliance with procedures and specifications. Fisheries and Oceans Canada (2005) defines program integrity in terms of relevance to program and client priorities and the degree to which objectives are met. Program integrity sometimes is defined in terms of quality of program delivery (e.g., Hansen, Graham, Wolkenstein & Rohrbach, 1991). Related concepts such as program fidelity and implementation fidelity (e.g., Fixsen, Naoom, Blase, Friedman & Wallace, 2005) were also explored, but none of the definitions quite fit our intent.

Following literature review, we engaged the first two cohorts of our leadership certification program in refining our definition. The cohorts provided a robust test of our approach. Comprised of Extension educators with substantive leadership responsibilities in all major program areas, they represented local Extension units from our smallest to our largest, and their experience ranged from 6 months to 25 years. In addition to working with leadership program participants, we also shared our definition with senior organization leadership. Using their feedback and agreement regarding the need for and value of the concept, we developed an updated version as follows.

Program integrity is the practical concept that programs should be organizationally and structurally sound. For Cornell Cooperative Extension, a program can be said to have integrity when it is: driven by organization mission, developed through ethical and valid processes, documented to be providing intended benefits, conducted in a manner respectful of program participants, consistent with current program quality standards, and deemed a responsible use of funding received. Key elements of program integrity include the following.

Congruence with Mission

• Driven by the Cornell Cooperative Extension mission
• Documented to deliver benefits as intended
• Meets equal program opportunity obligations and practices

Process Integrity

• Employs defensible processes for gaining input about needs and opportunities
• Uses shared decision criteria and open processes for determining program investments

Evaluation

• Active monitoring and feedback for all major program investments
• Data-based decisions about program improvement and the future of program investments
• Mix of program investments aligned with priority needs and opportunities and consistent with program quality criteria

Stewardship

• Program development conducted in an ethical and open manner
• Inputs required for conducting major programs are known and documented
• Relative cost and return considered when selecting among delivery alternatives

Program Integrity Applied

The program integrity definition became the guide for developing and implementing our program leadership effort. We reassessed the curriculum to determine if the components selected adhered to the concept of program integrity and provided comprehensive coverage of key elements. During instruction, we opened each of the program leadership modules with a discussion of program integrity to explore its relationship to the topic at hand and model how we were applying it to our own work. For example, we included in our program logic model workshop a discussion of how using logic models can help ensure program integrity by exposing program assumptions, specifying intended outcomes, and inventorying inputs. In our session on program governance and advisement structures and processes, we focused on process integrity, equal program opportunity and stewardship. Our session on managing program commitments picked up on mission congruence, evaluation, and stewardship.

By consistently citing our definition of program integrity, we expanded our collective understanding of the key elements and tested the comprehensiveness of our curriculum. In short, use of program integrity as an integrating theme provided a conceptual anchor and continuity for participants in our leadership program. It also served as a clear guide as we developed and refined the curriculum. Participants recognized the value of the concept in integrating and reinforcing the various aspects of their complex roles and in developing a comprehensive understanding of the Cornell Cooperative Extension System. We knew we were really onto something when educators began to cite program integrity in reference to their own work. We have since incorporated program integrity in new educator orientation sessions focused on exploring quality programming and professional roles. It continues to play well.

Conclusion

What do you think? Can a call to program integrity foster professional responsibility? Might it provide a needed framework and motivation for structured, deliberate approaches to program development? Or have we just added to the jargon jungle? Let us know if you have found other solutions, or if you have suggestions for strengthening our approach.

References

Diem, K. G. (2002). Using research methods to evaluate your Extension program. Journal of Extension [On-line], 40(6) Article 6FEA1. Available at: http://www.joe.org/joe/2002december/a1.php

Fisheries and Oceans Canada, Audit and Evaluation Directorate. (2005). Evaluation of program integrity -- SAR component. Retrieved March 13, 2008 from: http://www.dfo-mpo.gc.ca/communic/cread/evaluations/05-06/60251_e.htm

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: a synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

Franz, N. K., & McCann, M. (2007). Reporting program impacts: slaying the dragon of resistance. Journal of Extension [On-line], 45(6) Article 6TOT1. Available at: http://www.joe.org/joe/2007december/tt1.php

Hansen, W. B., Graham, J. W., Wolkenstein, B. H., & Rohrbach, L. A. (1991) Program integrity as a moderator of prevention program effectiveness: results for fifth-grade students in the adolescent alcohol prevention trial. Journal of Studies on Alcohol and Drugs, 52, 568-579.

McGrath, D. M. (2006). The scholarship of application. Journal of Extension [On-line], 44(2) Article 2FEA8. Available at: http://www.joe.org/joe/2006april/a8.php

McGrath, D. M., Conway, F. D. L., & Johnson, S. (2007). The Extension hedgehog. Journal of Extension [On-line], 45(2) Article 2FEA1. Available at: http://www.joe.org/joe/2007april/a1.php

Pennsylvania Department of Public Welfare. (2009). Bureau of program integrity. Retrieved March 13, 2008 from: http://www.dpw.state.pa.us/About/OMAP/AboutOMAP/003671318.htm

U.S. Department of Agriculture, Risk Management Agency. (2009). FCIC program integrity: Short reference for state and county FSA offices, and RMA regional offices. Retrieved March 13, 2008 from: http://www.rma.usda.gov/aboutrma/fields/4_rm/

Washington, R., & Fowler, S. (2005). Systematic assessment of resistance to Extension organizational change: Evidence from the Alabama Cooperative Extension System. Journal of Extension [On-line], 43(2) Article 2RIB2. Available at: http://www.joe.org/joe/2005april/rb2.php