Using Diffusion of Innovation Concepts for Improved Program Evaluation

Introduction

The diffusion of innovations theories, developed over a half century ago, have provided a popular framework to explain how new ideas and technologies are spread and adopted in a community (Rogers, 2003). The framework has been used for program planning, it has been empirically tested, and it has undergone critique from various perspectives since its inception in the 1950s (Yates, 2001). Throughout the years, it has remained instrumental to Extension professionals, scholars, and students alike and continues to be useful in countless other fields, including medicine, telecommunications, information technology, and social marketing (Rogers, 2003).

Although the framework has provided practitioners in Cooperative Extension and other fields with an overview of how information is diffused and adopted, its potential as a tool in the area of program evaluation has not been fully examined. The field of program evaluation is complex and comprehensive, but one of its basic goals is to determine whether program participants adopt a particular practice promoted by the educational program.

Adoption rates are often expressed as a simple percentage: for example "34% of participants adopted practice 'X' based on a follow-up survey conducted 'Y' months after the program." A number reflecting ultimate adoption of practice, however, does not indicate what role education may or may not have had in the decision to adopt or reject a particular practice.

For example, what do we know about those participants who did not adopt a particular practice? Was the proposed practice too expensive to implement? Was it too complex to comprehend and implement? Was it too risky? Did the program presenter provide inadequate information? Answers to these and other questions can provide educators with a better understanding of their role in influencing the adoption of practice. Future programs could then be designed to accommodate these factors and yield higher rates of adoption.

Extension educators and evaluation specialists have in fact included questions of this nature in post workshop surveys (Rollins, 1993) but not in a way that isolates the relative and absolute effects of the educational program. This article provides a theoretical argument for using classical diffusion of innovation concepts and theories along with modern multivariate statistical procedures such as regression analysis to gain a more robust understanding of the factors that influence the adoption of practice. In addition, a planned empirical test of this concept is discussed. Gaining this more detailed insight is essential for today's Extension professional who is interested in affecting change.

Diffusion of Innovations Background

"Diffusion is the process by which an innovation is communicated through certain channels over time among members of a social system" (Rogers, 1963). The diffusion framework is a fairly involved framework that includes several "sub-theories" or concepts. These concepts together provide insight into human and social nature, including how new information is accepted (or not accepted) by potential users. Because of this, the diffusions framework draws heavily from the fields of psychology and rural sociology (Beal & Bohlen, 1957).

Components of the classic diffusion framework include the innovation-decision theory, the individual innovativeness theory, the theory of rate of adoption, and the theory of perceived attributes (Rogers, 2003). In addition, the diffusion framework includes theories relating to communication aspects and channels. Only the individual innovativeness theory, the theory of perceived attributes, and concepts relating to communication channels are covered here because these are the most relevant to adoption of practice by individuals. Finally, some factors that are external to the diffusion framework but which affect adoption of practice are discussed.

The theory of individual innovativeness suggests that in most social systems there are innovators, early adopters, early majority adopters, late majority adopters and "laggards." These five categories are often visually represented as S- and bell-shaped curves (Rogers, 2003). The S-shaped curve indicates the cumulative number of adopters from innovators to laggards; the bell-shaped curve represents the resulting normal distribution.

The S-shaped curve illustrates the fact that there are relatively few adopters at first but that, as the technology, concept, or practice is picked up by innovators and early adopters, their influence will have an impact on the later adopters that make up a majority of potential adopters. The resultant bell-shaped curve graphically represents the different types of adopters and roughly reflects categories corresponding to standard deviations. That is, early and late majority adopters are often time statistically shown to be one standard deviation "above" the mean (average adopter), and the innovators, and early adopters, and laggards are two to three standard deviations "below" the mean (Rogers, 2003). Figure 1 depicts this classic graph.

Figure 1.
Bell-Shaped Graph Depicting Levels of Adopters (from Rogers, 2003)

Another diffusion theory, the theory of perceived attributes, focuses on how the program participant views characteristics of the practice under investigation. These have been typically categorized as those that relate to the complexity, compatibility, trialability, relative advantage, and observability of a practice or technology (Rogers, 2003). A brief description of each of these attributes follows.

• Complexity involves the degree of difficulty of understanding and implementing the practice from the perspective of the potential adopter.


• Compatibility concerns itself with a host of factors relating to the degree to which the practice is compatible to current objectives and philosophies of the program participant. Compatibility factors can be something as simple as not having the right resources (land or otherwise) to implement the practice or could involve more philosophical concepts such as concern over applying pesticides or fertilizers to an organic crop.


• Trialability deals with the potential to experiment with the practice on a smaller, less intensive scale. The expectation is that if an owner can implement the new practice on a trial basis he or she can possibly even modify the potential practice further to meet their specific needs.


• Relative advantage speaks to the possibility of increased income, reduced cost, or other factors that may make adopting this practice advantageous over other alternatives, including doing nothing.


• Observability relates to the degree to which the potential adopter has had the opportunity to see the practice implemented or see the results of the implemented practice. Some practices are obviously more observable than others (planting trees vs. preparing an estate plan, for example) and therefore might be adopted by individuals more quickly.

In summary, all things being equal, the more profitable, understandable, personally compatible, observable, and testable the participant considers the innovation, the higher the potential for adoption.

Other concepts relevant to the diffusion of innovations framework and having influence on adoption/rejection decisions include those relating to the communication channel, social networks, and external factors (Rogers, 2003). Communication channels include the change agent or agency and attributes of the communication program (for example, educational program type or means of disseminating information). Social networks and systems include support systems such as a local farmer/forest owner organization or association and the type and amount of interaction with professionals following the educational intervention (such as county agents or foresters). Finally, external factors such as markets, weather, natural disasters, policy, and unanticipated events all affect adoption of practices.

Diffusion of Innovation Framework for Evaluation

The diffusion of innovations approach to program evaluation recognizes that a variety of factors influence the adoption of a practice. This approach provides insight into why educational program participants adopt or reject a practice on which they have received information or training.

Diffusion of innovations research can provide information, for example, on barriers and motivations external to the educational program that may have strong influences on the decision to adopt or reject a practice. These factors may include the individuals' personality, socio-demographic characteristics, networks, and prior knowledge of the topic. Other influences may stem from the five perceived attributes associated with the practice or innovation under question (complexity, compatibility, trialability, relative advantage, and observability) or from the participant's social network and the availability of information and assistance from other sources.

Empirical examples that use this framework for evaluation purposes within the U.S. Extension System are not common. Research by King and Rollins (1995) indicated that change agent's attitude, participant's economic concerns, and technical information influenced the adoption of an agricultural innovation by participants who received information from a training program. Another study investigated factors that influence the adoption of practice and the participation in educational outreach of integrated pest management (IPM) in Utah. Characteristics that influenced farmers and producers to adopt practices or participate in educational programs included major source of income (on-farm or off-farm), farm size, market destination (in-state or out-of-state), diversity of crop produced, past intensity of IPM outreach efforts, and development of commodity organizations (Alston & Reding, 1998 ).

Research specifically linking the diffusion of innovations theories with program evaluation has been undertaken in the health field. Cervero and Rottet (1984) created an instrument designed to empirically test a diffusion-innovation-evaluation model hypothesized previously by Cervero Figure 2). The study sought to analyze the impact of the training program (Continuing Professional Education or CPE Program) on behavior change and performance (client outcomes) and to determine the extent of non-education program factors (proposed change, individual behavior, social system) on employee adoption of practice (measured as a change in performance).

A 51-item survey instrument was designed, tested for validity, and implemented. Data collection included the review of charts, interviews with participants and supervisors, and personal observation by the researchers themselves. Results indicate that a model based on Rogers's diffusion of innovations explains between 39 and 81% of the variance of the dependent variables (Rogers, 2003). This finding suggests that this design and a modified framework could be extremely useful as an evaluation tool in Extension and other educational fields.

Figure 2.
The Cervero Program Evaluation Model

From Theory to Practice in Extension

The Cervero model informs a way to test an evaluation model based on diffusion of innovations concepts. Such an empirical test of this model within the Extension System is underway for a regional forestry short course (The Master Tree Farmer Series). Past program participants will be surveyed to estimate the influence of various factors on the adoption of practice. Table 1 illustrates a sample of the proposed variables that will be used to study the influence of educational and other variables on the adoption of practice. These are strictly a sampling of the types of questions that will be asked.

A group of professional foresters and educators will provide more input into the reliability and validity of these questions and others before the actual survey is designed and delivered. As with the Cervero model, the study covers four categories of independent variable. These include those that relate to the educational program of interest, the individual participating in the program, the proposed practice (trialability, complexity, observability, relative advantage and compatibility), and the social system or network surrounding the participant. A sampling of subvariables and questions are also listed.

Other factors that may explain the variation in the dependent variable include location (are participants in some states more likely to apply a practice than those in others?) and time elapsed since attending the course (landowners may have had time to implement practices that were problematic immediately following their participation). While some of these variables may be correlated with others, the collection of this data at this time will be useful for descriptive purposes (for example, key stakeholders may wish to know if adoption of practice occurs in their state to a greater extent than in others).

To More Robust, Defensible Extension Evaluations

While the diffusion of innovations concepts were developed during the years of rapid agricultural innovation and dissemination, the theories have evolved over the years to incorporate user/client-based needs (Rogers, 2003). User/client needs-based models incorporate end-users into research design, implementation, and technology transfer activities. The diffusion of innovations framework for program evaluation should lead to a better understanding of any barriers or issues surrounding adoption of a practice by incorporating end user needs and obstacles.

As with the diffusion of innovations framework, the area of program evaluation is vast, diverse, and still developing. In Extension, evaluation has historically been one-dimensional. Measurement has focused solely on the impact of program participation in terms of change in behavior or adoption of practice. Characteristics of the educational program and influences of external factors have not normally been accounted for in an inclusive framework such as this. Using diffusion of innovations techniques to guide an Extension program evaluation can provide an opportunity to investigate what is going on inside "the black box" of program impact determination (Bush, Mullis, & Mullis, 1995). In addition, it can also move Extension evaluation toward more causal modeling.

Today's Extensionist is living in a complex world where social, economic, and environmental factors all influence adoption of practice (Clements, 1999). Applying statistical inferences to study these factors in a systematic intentional manner can yield a better understanding of the relative impact of education and information. This may be a particularly important investment in major, multi-year, or multi-state Extension programs.

Such research may also uncover valuable information for providing leadership among public service providers who also may influence adoption of practice. Examples in the forestry community, for instance, include state forestry agencies and associations, private forestry consultants, and others who provide support and assistance to private owners. Finally, this type of program evaluation model can be adopted for many disciplines within Extension and can be used with stakeholders and potential funding agencies to provide for more holistic, credible evaluations. The results of applying these concepts in an actual program evaluation are currently underway and will provide insight into the practical nature of such a model.

References

Abrussese, R. S. (1987). The Cervero model. The Journal of Continuing Education in Nursing, 18(1), 22-23.

Alston, D. G., & Reding, M. E. (1998). Factors influencing adoption and educational outreach of integrated pest management. Journal of Extension [On-line], 36(3). Available at: http://www.joe.org/joe/1998june/a3.html

Andrews, M. (1983). Evaluation: An essential process. Journal of Extension [On-line], 21(5). Available at: http://www.joe.org/joe/1983september/index.html

Beal, G. M., & Bohlen, J. M. (1957). The diffusion process. Ames: Iowa State University of Science and Technology.

Bennett, C. (1977). Analyzing impacts of extension programs (Slightly rev. July l977. ed.). Washington: Deptartment of Agriculture Extension Service.

Broderick, S. H., Snyder, L. B., & Tyson, C. B. (1996). Selling stewardship within the community: A social marketing approach. In M. J. Baughman & N. Boodman (Eds.), Proceedings: Symposium on nonindustrial private forests: Learning from the past, prospects for the future (pp. 255-263). St. Paul: University of Minnesota, Minnesota Extension Service Special Programs.

Bush, C., Mullis, R., & Mullis, A. (1995). Evaluation: An afterthought or an integral part of program development. Journal of Extension [On-line], 33(2). Available at: http://www.joe.org/joe/1995april/a4.html

Cervero, R. M., & Rottet, S. (1984). Analyzing the effectiveness of continuing professional educational: An exploratory study. Adult Education Quarterly, 34(3), 135-146.

Clements, J. (1999). Results? Behavior change! Journal of Extension [On-line], 37(2). Available at: http://www.joe.org/joe/1999april/comm1.html

King, R. N., & Rollins, T. (1995). Factors influencing the adoption decision: An analysis of adopters and nonadopters. Journal of Agricultural Education, 36(4), 39.

Lamble, W., & Seaman, D. (1994). Diffusion and adoption: Basic processes for social change. In D. J. Blackburn (Ed.), Extension handbook: Processes and practices. San Francisco: Thompson Educational Publishing.

Rogers, E. M. (1963). The adoption process: Part I. Journal of Extension [On-line], 1(1). Available at: http://www.joe.org/joe/1963spring/index.html

Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.

Rollins, T. (1993). Profile of farm technology adopters. Journal of Extension [On-line], 31(3). Available at: http://www.joe.org/joe/1993fall/rb1.html

Yates, B. L. (2001, May 24-28, 2001). Applying diffusion theory: Adoption of media literacy programs in schools. International Communication Association Conference. Retrieved February 8, 2005, from: http://www.westga.edu/~byates/applying.htm