Measuring Economic Impact Through Adoption: A Study of the Multi-County New Landowners Educational Series

Introduction

Extension is a publicly funded entity, and as recipients of this public funding, Extension professionals are expected by county, state, and federal funding partners to report program impacts (Richardson, 1996). Further, program impacts must be reported to key decision makers and other stakeholders for Extension to remain relevant (O'Neill & Richardson, 1999). Government-supported programs such as Extension must demonstrate sufficient public benefit to make them worthy of continued financial support. Reporting program impacts is vital to Extension, and when possible, dollar impact figures should be obtained. As O'Neill (1998) noted, ". . . money talks in Extension evaluation. Incorporating economic impact assessments into program planning should be a high priority for every Extension educator" ("Conclusion"). That suggestion is as true today as ever before. Seemingly, all entities that receive public funds are being held accountable and are attempting to attach dollar values to their efforts.

Measuring the economic impact of Extension programming efforts can be accomplished in several ways. One method of measuring economic impact involves placing monetary value on certain practices that are adopted. For example, through the adoption of a specific management practice taught in the Texas Beef Quality Assurance program, participants increased gross returns of an estimated $7.2 million in 2011 (McCorkle, 2012). Another example of positive economic benefit relates to money saved by local health care systems when people apply newfound knowledge associated with developing more healthful lifestyles, such as knowledge gained from the Walk Across Texas Campaign. An impact study associated with this program indicated that potential lifetime health care cost savings was expected to be $63,210 for females and $57,230 for males (McCorkle, 2012). When avoidance of lost wages was included, the potential lifetime economic benefit for 2011 participants totaled $265 million (McCorkle, 2012). The Multi-County New Landowners Educational Series (MCNLES) is an Extension education program offered in Texas across the counties of Austin, Colorado, Fayette, and Washington that provides new landowners with instruction regarding management practices and agricultural enterprises that could be best suited for their properties (Matheney, 2014). The purpose of our study was to add to the body of research related to measuring economic impacts of Extension programs by assessing the impact of the MCNLES.

Methods

Participants

The population for the study comprised participants of the 2006–2010 MCNLES. These individuals were selected due to the premise that they would have had sufficient time to implement best management practices (BMPs) taught in the course. A contact list was developed from prior registration information on file in the Austin County Extension office. The selection of participants was narrowed to include only individuals who had attended at least eight sessions of the program.

We used an online survey to collect the evaluation data and provided 162 individuals the opportunity to participate in the study. An email that announced the study and included the survey link was sent to these individuals in July 2012. A total of 61 individuals (38%) initially responded to the survey, resulting in a response rate below the desired rate of 50%. Following an approach suggested by Lindner, Murphy, and Briers (2001) for handling nonresponse in social science research, we sought additional potential participants from the list of nonrespondents. An additional 27 individuals completed the survey as a result of this process. In total, 88 individuals (54%) completed the survey.

Survey Instrument

The electronic survey instrument consisted of 60 questions organized in three sections. Questions on the survey instrument investigated the adoption of BMPs taught during the MCNLES. For the purposes of this article, we focus on items from the first and second sections of the survey instrument.

The first section of the questionnaire began with notification of the opportunity to consent to participate in the study. This notification was followed by a yes-no question respondents used to confirm their participation in the MCNLES and an item for identifying the year they took the course, with the response options "2006," "2007," "2008," "2009," "2010," and "do not remember." Next, respondents were asked how they had heard about the program; response options were "Extension website," "word of mouth," "brochure/mailing," "newspaper/radio," "realtor," and "other" with a box for supplying specific information. This section closed with seven questions addressing respondent demographic information, including age, gender, ethnicity, education level, annual income, and agricultural background.

The second section of the survey instrument included questions for determining the perceived economic benefits resulting from participation in the MCNLES. It is important to note that during the MCNLES sessions, instructors explained how practices presented could result in economic benefits for participants' agricultural operations (e.g., how soil testing could improve fertilizer application and, thus, forage production, potentially resulting in an increase in income from livestock or hay production; how maintaining agricultural valuation for tax purposes could result in lower property taxes). A yes-no question addressed whether participation in the course had caused the respondent to experience an economic benefit on his or her agricultural operation. The exact wording of the question was "Do you believe that the MCNLES course provided an economic benefit to you in your agricultural operation?" Respondents who answered no were directed to the last section of the survey. If a respondent answered yes, the next item asked the respondent to estimate the amount of economic benefit that resulted from participation in the MCNLES. Respondents were asked to base this estimate on land management practices changed or adopted as a direct result of participation in the program and to estimate the monetary benefit on a per-acre basis by choosing from "$1–$9," "$10–$29," "$30–$49," and "more than $50." This question was followed by a yes-no question addressing whether participation in the course had caused the respondent to experience an increase in the value of his or her property. Respondents who answered no were directed to the last section of the survey. Respondents who selected yes were asked to estimate the per-acre property value increase. This questioning sequence allowed identification of respondents' perceptions of property value increase as a direct result of implementing practices learned in the MCNLES.

Data Collection and Analysis

For data collection, we followed Dillman's (2007) procedures for email surveys. Members of the target population (N = 162) were contacted via email and provided a web link to the survey instrument. After a total of 8 weeks, the survey was closed, and 61 individuals (38%) had completed the survey. Because the response rate was not at the desired 50% or greater for initial respondents, we sought additional participants from the list of nonrespondents. A random sample of 27 nonrespondents agreed to complete the survey by phone. We compared these respondents to the initial respondents by following the third method of addressing nonresponse suggested by Lindner et al. (2001). We recorded the answers of these 27 additional respondents and analyzed them for validity against the answers of the initial respondents. We used a cross-tabulation of nominal level data for independent and dependent variables to determine whether significant differences existed between the 27 respondents who completed the survey by phone and the initial respondents who completed the survey sent via email. No significant differences were found relative to economic benefit per acre or the year the participant took the course. There were also no significant differences found with regard to gender, education level, and income level. Significant differences were found (p < .05) with regard to age; 51.9% of later respondents were over age 65, whereas only 26.2% of initial respondents were in this age category. Significant differences also were found with regard to the practice of selectively clearing unwanted brush species (a practice taught in the course), with 87.5% of later respondents indicating they did so often or always as compared to only 53.7% of initial respondents. Because these two characteristics were the only significant differences, the two groups were combined and treated as one population.

Results

Eighty-eight participants (54%) completed the survey. The largest portion of the participants (25.3%) indicated that they did not remember the year they attended the MCNLES; 21.8% reported attending the course in 2009, and 20.7% reported attending in 2008. The largest percentage of the participants (n = 34, 38.6%) had heard about the course through a personal invitation they received in the mail. Of the 88 participants, almost half (46.6%) were between the ages of 56 and 65, 59% were male, and 94.3% were Caucasian (non-Hispanic). The majority of participants (86.4%) had either a college or postgraduate degree, and 67.1% had annual income levels higher than $100,000. Almost two thirds of the participants (63.6%) did not have an agricultural background; however, 59.1% lived on a farm or ranch at the time of the study. (For the purposes of the study, a farm or ranch was defined as 10 ac or more.) Additionally, 75% of the participants owned or managed 100 ac or less, and 23.9% owned or managed 101 to 400 ac. Only one participant reported owning more than 400 ac. See Table 1 for complete respondent demographic information.

Table 1.
Demographics of Respondents Who Participated in the Multi-County New Landowners Educational Series Conducted by the Texas A&M AgriLife Extension Service (n = 88)

Variable	f	%
Age
≤55	17	14.8
56–65	41	46.6
≥66	30	34.1
Gender
Male	52	59.1
Female	36	40.9
Education
No college degreea	12	13.6
College degreeb	41	46.6
Postgraduate degreec	35	39.8
Income leveld
<$100,000	26	32.9
>$100,000	53	67.1
Ethnicity/race
White (non-Hispanic)	82	94.3
Hispanic	2	2.2
African American	1	1.1
Other	2	2.3
Agriculture background
Grew up in agriculture	32	36.4
Did not grow up in agriculture	56	63.6
Amount of land owned or managed
≤100 ac	66	75.0
101–400 ac	21	23.9
≥401 ac	1	1.1
aRespondents having less than a high school degree, a high school diploma or GED equivalent, or some college. bRespondents having a bachelor's degree or an associate's degree only. cRespondents having any degree higher than a bachelor's degree, such as master's degree, PhD, MD, and so on. dIncome levels were grouped into two categories on the basis of logical break points for responses received.

A majority of the participants (89.8%) perceived that participation in the MCNLES had provided them with an economic benefit relative to their agricultural operations. Of those participants, 26.7% estimated the economic benefit to be $1–$9 per acre, 33.3% estimated it to be $10–$29 per acre, 12.0% estimated it to be $30–$49 per acre, and 28.0% estimated it to be more than $50 per acre. We determined the grand mean for all participants on the basis of the midpoint value in each estimated value-per-acre range, with the exception of the final category, for which we used the lowest possible value in an effort to remain conservative. We calculated the grand mean by using the summated-value total divided by the frequency-of-response total. We determined the grand mean for all participants to be $26.57 per acre, as shown in Table 2.

Table 2.
Estimated Economic Benefit on Agricultural Operation for Participants of the Multi-County New Landowners Educational Series Conducted by the Texas A&M AgriLife Extension Service (n = 75)

Estimated value-per-acre range	Estimated value-per-acre grand mean calculation valuea	f	%	Summated value (grand mean calculation value × frequency)
$1–$9	$5.00	20	26.7	$100.00
$10–$29	$19.50	25	33.3	$487.50
$30–$49	$39.50	9	12	$356.00
≥$50	$50.00	21	28.0	$1,050.00
Totals		75		$1,993.00
Grand mean economic benefit/acreb				$26.57
aThe grand mean was determined on the basis of the midpoint value in each estimated value-per-acre range, with the exception of the final category, for which the lowest possible value was used for the purpose of remaining conservative. bTotal summated value divided by total frequency equals grand mean.

A majority of the participants (93.1%) perceived increases in the values of their properties as a result of practices they changed or adopted after participating in the MCNLES. Forty-two participants (48%) provided an estimate of the increase to property value. Their estimates ranged from an increase of $0 per acre to an increase of $4,000 per acre (see Table 3); the average increase was $648.69 per acre. We calculated the average value by using actual numbers reported by respondents. The average number of acres owned or managed by participants (n = 87) was 103.3 ac, with the smallest property being 10 ac and the largest being 1,649 ac. Thus, the average total economic benefit per participant, based on the average property value increase (i.e., $648.69) and the average number of acres owned or managed (i.e., 103.3 acres), was $67,009.68.

Table 3.
Estimate of Per-Acre Property Value Increase for Participants of the Multi-County New Landowners Educational Series Conducted by the Texas A&M AgriLife Extension Service (n = 42)

Dollar value increase per acre	f	%
$0–$50	9	21.4
$51–$100	9	21.4
$101–$250	4	9.5
$251–$500	6	14.2
$501–$1,000	8	19.0
$1,001–$4,000	6	14.2

Conclusions, Implications, and Recommendations

On the basis of our study findings, we concluded that the participants in the MCNLES valued the information gained through participation in the program and perceived an economic benefit. Given that this program is a fee-for-service program, we also concluded that participants believed the education obtained via the program was worth the cost of registration and, possibly more importantly, their time. If this were not the case, their estimations of economic benefit would have been negative.

Data we collected in an effort to describe the study population revealed that the audience for the MCNLES consisted mostly of college-educated individuals with above average incomes. This finding has implications for program planning and administration.

The method we used to quantify the economic impact of the MCNLES on participants' agricultural operations involved asking the participants to place a dollar value on economic benefit per acre. According to their responses, the MCNLES had a positive economic impact on their agricultural operations. In fact, a majority of the participants reported that they had benefited economically from their participation in the series. We calculated the grand mean estimated value-per-acre benefit as $26.57. Although this number is a representation of participants' opinions and beliefs rather than a number derived from a balance sheet, the data answer the call by O'Neill (2008) to collect estimated dollar values of practices implemented as a result of participation in Extension programming. Continued evaluation of participants' perceptions of the economic impact of Extension programs is needed to document impact through valid and reliable data. Collection of economic impact estimations can be a sensitive process; thus, careful collection of data over time should be compared and contrasted for the purpose of adding rigor to the evaluation process.

Identification of specific practices that had the most impact on participants and documentation of the level at which participants adopted such practices were not addressed in our study. Future studies should include questions that place an economic value on specific practices introduced by the MCNLES. For instance, how much does the adoption of soil testing affect the value of a property? In addition, future studies should include questions related to the economic value of Extension programs. For example, how much would a program participant estimate having to pay for similar property management advice from a fee-based consultant? Extension personnel should continue to attempt to place dollar values on the impacts of their programs by estimating the monetary value of behaviors changed. Future studies also could assess associated levels of participation in other Extension-sponsored programs, especially those that provide public value benefit (Kalambokidis, 2004). For example, do individuals become involved in a master volunteer program, such as master gardeners or master naturalists, after attending the MCNLES? Additional investigation into questions such as these would enable improved Extension programming efforts stemming from an understanding of the effect of one Extension program on another.

Of the 81 individuals who perceived that they experienced an increased property value as a result of their participation in the program, only 42 provided an estimated dollar value per acre. One can only speculate as to why many individuals (39) chose not to do so. Of course, it is possible that the mere association with a monetary value caused nonresponse to the question. However, it is also possible that participant age or income level, year the course was taken, or other factors influenced respondents' willingness to estimate economic benefit per acre. Or perhaps participants had a difficult time quantifying the value or thought the question was too difficult to answer. The relatively low percentage of participants who provided an estimate of increase to property value leads us to consider whether information could be provided during the MCNLES that would assist participants in understanding how to estimate economic benefit as a result of various practices. The need for studies that document economic benefit of Extension educational programming is great; however, research that includes questions related to financial issues can cause uneasiness or distrust and can lead to study attrition. Future studies should explore methods of documenting economic impact beyond the use of mere survey responses.

References

Dillman, D. A. (2007). Mail and Internet surveys: The tailored design method (2nd ed.). New York, NY: John Wiley and Sons.

Kalambokidis, K. (2004). Identifying the public value in Extension programs. Journal of Extension, 42(2), Article 2FEA1. Available at: http://www.joe.org/joe/2004april/a1.php

Lindner, J. R., Murphy, T. H., & Briers, G. (2001). Handling nonresponse in social science research. Journal of Agricultural Education, 42(4), 43–53. doi:10.5032/jae.2001.04043

Matheney, K. J. (2014). The Multi-County New Landowners Educational Series. Retrieved from http://austin.agrilife.org

McCorkle, D. (2012). Economic impact briefs. Retrieved from http://agrilifeextension.tamu.edu/about/strategic‑plan‑and‑impacts/economic‑impact‑studies/

O'Neill, B. (1998). Money talks: Documenting the economic impact of Extension personal finance programs. Journal of Extension, 36(5), Article 5FEA2. Available at: https://www.joe.org/joe/1998october/a2.php

O'Neill, B. (2008). Calculating the economic impact of health education programs: Five tools for Extension educators. Journal of Extension, 46(1), Article 1TOT4. Available at: http://www.joe.org/joe/2008february/tt4.php

O'Neill, B., & Richardson, R. G. (1999). Cost-benefit impact statements: A tool for Extension accountability. Journal of Extension, 37(4), Article 4TOT3. Available at: http://www.joe.org/joe/1999august/tt3.php

Richardson, J. (1996). Extension accountability (Fact sheet AEE 96-02). Raleigh, NC: North Carolina Cooperative Extension Service.