The Value in Evaluating and Communicating Program Impact: The Ohio BR&E Program

Introduction

Evaluating program impact is important (Diem, 2003; Roucan-Kane, 2008). Communicating program value is even more important in today's political environment (Stup, 2003). This article briefly describes the origins of a long-time Extension program, referred to as "Business Retention and Expansion" (BR&E). Recent impact evaluation efforts and findings, and how such information has been communicated in marketing, informational, and justification efforts with program partners are also discussed.

Origins

The Ohio BR&E Program was conceived in the early 1980s as a tool local leaders could employ to identify barriers to and opportunities for local economic development. Born out of need, like most Extension programs, the program was designed to help local communities stimulate economic growth during a time when little was happening economically. Additionally, it was a response to widespread and growing use of state-based personal and real property tax incentive programs for attracting new industrial investment to local communities, a practice whose critics believed pitted struggling communities against one another (Morse, 1990). The BR&E program was designed to serve as a highly visible and recognized Extension program in the area of economic development at a time when Ohio State University Extension had little visibility in this programming area. Through in-person and mail surveys, the program originally focused on helping communities identify and address business concerns (referred to as "red flags") and communicate a pro-business attitude. The program later evolved to include identifying issues for strategic planning (Blaine, Hudkins, & Taylor; 1999).

From the beginning, the program was a collaborative effort designed to help communities understand ways in which existing resources could be employed to bring about positive economic change. The Ohio Department of Development helped secure funding to hire a program manager. Utility company representatives served in a "consultant" role providing program guidance to local Extension staff and community volunteers. Extension professionals developed a certification program to train volunteers who would collect survey data during their visits with businesses in their local community.

Community partners readily welcomed the program's "self-help" approach to economic development and its availability rapidly spread by word of mouth. Interest was sufficient to hold a statewide conference featuring some of the most successful community programs. Program demand skyrocketed. Twenty-five years later, more than 145 community BR&E programs have been conducted in 79 of Ohio's 88 counties (Ohio BR&E Program Report, 2010). The programs have involved partnerships that included utility companies, state and local development officials, businesses, community residents, and Extension.

In general, BR&E programs are designed to help local communities learn how to systematically gather information critical to understanding local development issues such as needed roadway or infrastructure improvements or workforce training (Coleman, 1991; Loveridge & Smith, 1992; Morse, 1990; Phillips, 1996; Smith, Morse, & Lobao, 1992). While program objectives specific to the OSUE approach have not changed much since 1986, the methods by which they are achieved have evolved significantly with the advent of the Internet and more advanced information and communication technologies.

Beginning in 2001, program materials became Web-based, and the use of computer software became an integral program component. Furthermore, the program was offered in an "annual membership" format, which involved 12 months of hands-on training for a flat one-time fee. The goal: to build capacity of community partners to carry out a BR&E program from start to finish independent of outside assistance.

Programming has involved participants at a variety of community levels: neighborhoods, villages, cities, counties, and multi-county regions. Most programs have involved a committee of volunteers led by a local coordinator. Various community professionals such as Extension educators, Chamber of Commerce directors, neighborhood or regional non-profit directors, or municipal or county officials have served as local coordinators.

While program partners have cited business and job creation and retention since 1986 (Ohio BR&E Program Report, 2010), it wasn't until 2008 that a formal annual Web-based evaluation was initiated with program partners. The goal: to better describe the effectiveness of the BR&E program, including the types of community development impacts that program participants attribute to their involvement in the BR&E program.

Program Evaluation

Since 2008, local coordinators have been invited to share local program impact data. The information has been collected annually via a Web-based evaluation. Twelve individuals have served in the role of local coordinator since 2004 and have been invited to participate in the web-based evaluation annually.

In addition to fixed-choice items, local coordinators have been invited to provide detailed narrative input related to program impact. For example, the Web-based questionnaire has collected program evaluation data helpful in assessing the extent to which:

• Community volunteers have participated in delivery of the BR&E program,
• Data relevant to the local economy were employed by local decision makers to inform strategies designed to foster economic growth and development,
• Relationships were established and/or cultivated with existing businesses,
• Existing businesses created jobs and/or retained jobs, and
• Personal income was created and/or retained.

For example, local coordinators were invited to respond to questions such as "Your BR&E program efforts consist of surveys, in-person visits, roundtables, etc. Approximately how many businesses participated in this full range of BR&E activities over the past year?" Another example read, "Considering that your local BR&E program may have very well identified and aided a struggling business contemplating employee layoffs or closure if it weren't for your assistance, please estimate the number of existing jobs that were saved over the past year as a result of these local efforts."

Evaluation Findings and Discussion

Of the 12 local coordinators invited to provide evaluation input, six have responded annually since 2008. Because no identifying information was requested in the questionnaire, it is unknown if the same six local coordinators have responded each year. Highlights from this annual evaluation effort conducted over the past 3 years are aggregated in Table 1.

Local coordinators provided evaluative feedback indicating that community interests have taken the opportunity to volunteer and engage in gaining a better understanding of their local economy. Businesses have willingly shared their needs, concerns, and points of pride with development officials and interested community volunteers. Local decision-making has been better informed by local data. And ultimately, jobs have been created and saved, yielding income for residents of Ohio communities and adding value to the economy.

Local coordinators also provided narrative evaluative input. Key themes gleaned from the input shared included the following.

1. The BR&E program has helped to foster and strengthen collaborations. Local coordinators indicated the BR&E program helped to leverage other economic development initiatives in the community.
2. The BR&E program has enabled the collection and analysis of valuable local data. For example, one respondent indicated that the findings generated by their local BR&E program were critical to informing the community's economic development planning and marketing efforts.
3. Program partners perceive the reputation and resources of Ohio State University have added value to their local BR&E program. "OSU personnel are well received by our city officials and taskforce members and are one of the major reasons the city continues to partner with OSU", said one local coordinator.

Conclusions and Suggestions

Evaluating program impact enables us to better communicate program value. According to Rennekamp and Arnold (2009), such information has value in justifying continued investment in program development, delivery, and evaluation. Such evaluation data have also been useful in tailoring the BR&E program to better meet community needs.

Helping local communities learn how to systematically gather information critical to understanding local development needs is a key program objective. Involving community volunteers in this process helps to ensure that capacity to carry out such work is developed in a broad cross-section of community interests, rather than in one or two key development personnel. The number of community officials using locally collected data to inform community decision-making also provides evidence that capacity to engage in such community work has been improved. Further, data provided by local coordinators on an annual basis supports the notion that these community partners have been able to carry out a BR&E program from start to finish without outside assistance (another key program objective).

While evaluation data collected over the years have informed efforts to enhance program effectiveness, they have also been critical for communicating program value to prospective program partners. Over the past few years, print media and video productions highlighting BR&E program impact data have been the focus of a coordinated promotional campaign aimed at enhancing awareness of the program. Such materials have been distributed to local opinion leaders and community officials in a variety of formal and informal venues to demonstrate the program's value to community partners; a program arguably even more valuable during difficult economic times.

The program impact data have also been of critical value in supporting the development of targeted messages strategically communicated to Extension program underwriters. Anecdotal evidence suggests the information has been very useful to the organization's legislative affairs and communications units as they rally to respond to ever-increasing funding threats at federal, state, and local levels.

The BR&E program was born out of need to help communities better understand and affect their local economy. Many lasting and worthwhile partnerships were forged throughout the state in pursuit of this overall goal. Evaluation data collected over the years have provided program direction and communicated program value to prospective program partners. Communicating such information to program underwriters has been critically important to Extension's partner communities, including the businesses, families, and wage earners that comprise them.

References

Blaine, T. W., Hudkins, S. A., & Taylor, C. R. (1999). Taking R&E to the next level. Journal of Extension [On-line]. 37(6) Article 6FEA2. Available at: https://www.joe.org/joe/1999december/a2.php

Coleman, B. (1991). Minnesota business retention and expansion program. Economic Development Review, 23-25.

Diem, K. G. (2003). Program development in a political world—It's all about impact! Journal of Extension [On-line], 41(1) Article 1FEA6. Available at: https://www.joe.org/joe/2003february/a6.php

Loveridge, S., & Smith, T. R. (1992). Factors related to success in business retention and expansion programs. Journal of the Community Development Society. 23(2), 66-78.

Morse, G. W. (1990). A conceptual model of retention and expansion business visitation programs, In The retention and expansion of existing businesses: Theory and practice in business visitation programs, ed. George W. Morse. Iowa: Iowa State University Press.

Ohio BR&E Program Report. (2010). The Ohio State University Business Retention & Expansion Initiative. The Ohio State University.

Phillips, P. (1996). Business retention and expansion: Theory and an example in practice. Economic Development Review, 14(3), 19.

Rennekamp, R. A., & Arnold, M. E. (2009). What progress, program evaluation? Reflections on a quarter-century of Extension evaluation practice. Journal of Extension [On-line], 47(3) Article 3COM1. Available at: https://www.joe.org/joe/2009june/comm1.php

Roucan-Kane, M. (2008). Key facts and key resources for program evaluation. Journal of Extension [On-line], 46(1) Article 1TOT2. Available at: https://www.joe.org/joe/2008february/tt2.php

Smith, T. R., Morse, G. W., & Lobao, L. M., (1992). Measuring impacts of business retention and expansion visitation programs. Journal of the Community Development Society, 23(1), 123-143.

Stup, R. (2003). Program evaluation: Use it to demonstrate value to potential clients. Journal of Extension [On-line], 41(4) Article 4COM1. Available at: https://www.joe.org/joe/2003august/comm1.php