Self-Evaluation of a Statewide Conservation Planning Program: Perceptions of an Extension Partner

Introduction

The study reported here is an example of how a statewide organization can use information gathered from its personnel to evaluate agency performance regarding methods for reaching goals and objectives. As Extension personnel continue to work closely with the Natural Resource Conservation Service (NRCS) to reach natural resource goals and objectives, they can improve methods of information dissemination by learning about the conservation planning process.

Program evaluation commonly involves assessment of individual programs by internal personnel (Roucan-Kane, 2008; Archer, Bruns, & Heaney, 2007; Arnold, 2002; Bailey & Deen, 2002). The use of logic models to improve program effectiveness is encouraged and has been the feature of recent Extension program evaluation articles (Rennekamp & Arnold, 2009). The study reported here was intended to evaluate the conservation planning process and the perceptions of a state agency regarding their strategies and client response to those strategies. Extension and NRCS personnel are linked by common goals and objectives. Extension personnel are integral to the programming needs of NRCS and can use information from state NRCS personnel to inform "accurate and timely information and educational materials" provided to NRCS for the management of natural resources (Ishler et al., 2006).

Conservation planning affects the quality of our most precious natural resources: water, air, soil, wildlife, and plant material as well as every aspect of agriculture, one of the largest industries in the United States. Important factors in the development of programs encouraging farmers to adopt conservation practices are the perceptions of personnel who advise the farmers on how to implement the practices and where a farmer might receive financial support. NRCS is one of many government entities working to conserve soil, water, and other natural resources. NRCS partners with farmers to develop plans for resource management and, similar to Extension programs, provides valuable information to farmers about methods of managing resources. Through program evaluation, NRCS and county Extension personnel are able to determine training needs and new approaches to achieving the conservation of natural resources (Radhakrishna & Martin, 1999).

NRCS was chosen as the sample population for the study because of its involvement and expertise in conservation planning. The results of the study reveal the conservation practices that are most often recommended in contrast to those that are actually implemented by the farmers and landowners and the reasons given by agents as to why they feel farmers adopt or fail to adopt the practices. These results will be useful to anyone working with farmers to implement conservation measures.

Purpose and Objectives

The study assessed conservation planning activities, training activities, and perceptions of Mississippi NRCS personnel regarding the adoption of conservation practices by farmers/landowners. Our research question: How do conservation planners obtain and disseminate information to users, and what is the perceived response of those users to the information?

The first objective was to determine the status of conservation planning education statewide for both NRCS and the farmers it serves. To achieve this objective, respondents were asked to identify the regularity of training and the type of training method that they felt was most effective.

The second objective was to identify the primary resource concerns and incentive programs being recommended by NRCS in Mississippi, as well as the regularity of enrollment in programs when recommended. Knowledge of resource concerns and incentive programs informs trainers and information providers (e.g., Extension personnel) of the subject areas that are of most concern to NRCS and the farmers and landowners that they serve. The information is generated primarily from research on methods for achieving conservation goals such as water quality, biodiversity, and soil health, among others. (See Table 4 for a list of Resource Concerns.)

The third objective was to determine the reasons for non-adoption of conservation practices. Knowing the reasons for non-adoption allow NRCS and Extension personnel to overcome obstacles that prevent conservation practices from being implemented by their constituents.

Methods and Materials

Many studies have shown that the same design principles used for mail surveys may also be used for the design of email and Web surveys (Couper, 2000; Crawford, Cooper, & Lamias, 2001; Denscombe, 2006; Dillman, 2007; Kiernan, Kiernan, Oyler, & Gilles, 2005). The design and construction of the self-administered online questionnaire in the study reported here followed the Total Design Method (Dillman, 2007). Question Pro was selected as the host website for the questionnaire. The values of using Question Pro for Web surveys include easy design for questionnaires, the ability to view real-time reports, and the inclusion of an analysis software program for data export. The first mailing for this survey, a pre-notice letter, was emailed to a total of 330 Mississippi NRCS agents on Tuesday, May 22, 2007, and was followed by two successive survey request letters.

Results

The final number of useable responses was 162 employees, which generated a final response rate of 49% of the original 330 employees who received the initial e-mail (162/330).

Demographics

Of the 162 respondents, the majority was male (83%). A total of 85% of the respondents were between 30 and 59 years of age, with the largest age group being between 50 and 59 years old (35%). The educational backgrounds of the respondents were largely agriculture-based degrees.

NRCS personnel were asked about their years of service with both NRCS (including the Soil Erosion Service and Soil Conservation Service) and the Mississippi NRCS. The largest percentage of respondents had been with NRCS for 1 to 5 years (25%), and an equal number of respondents had worked for NRCS 16 to 20 years (19%) and 26 to 30 years (19%).

Conservation Planning

The survey included two questions regarding the general involvement of respondents in the conservation planning process, while the remainder of the survey dealt with specific elements of conservation planning, such as conservation practices, incentive programs, and water quality. The first of the general questions asked if the respondent works directly with or provides technical assistance on the development of conservation plans for farmers or landowners in the state. Seventy-nine percent of the respondents stated that they provided technical assistance on conservation plans in the state of Mississippi. The second question asked how many conservation plans the employees had consulted on in the past year. A majority of respondents (52%) had consulted farmers on over 30 conservation plans in the past year (Table 1), indicating that the study population had a significant influence on conservation planning in the state of Mississippi.

Table 1.
Total Number of Conservation Plan Consultations in the Past Year

# of Conservation Plans (n = 155)	f	%
None	28	18.1
0-10	21	13.5
11-20	11	7.1
21-30	15	9.7
> 30	80	51.6

Conservation Practices

In order to identify the major conservation foci of NRCS as well as identify areas in need of increased education, training, or focus, a section of the survey was dedicated specifically to training associated with Conservation Practices as well as the recommendation and implementation of specific Resource Concerns. Of the 162 respondents, 85% regularly provided technical assistance to farmers, landowners, or other NRCS personnel about the use of conservation practices. 90% stated that they were "very comfortable" or "somewhat comfortable" with the design or implementation of conservation practices. When asked if farmers or landowners were well educated about the benefits of the conservation practices applied or installed on their properties, 73% believed that the farmers were well educated on conservation practices.

Conservation practice training available to NRCS agents fall into one of two general categories: formal or informal. Formal training includes organized training sessions and workshops, while informal training refers to on-the-job training. Approximately 67% of the respondents claim that they received formal training "once every six months" (Table 2) or "annually" (32%). In contrast to formal training opportunities, 71% claim to have received on-the-job training regarding conservation practices on a weekly, monthly, or bi-annual basis.

Table 2.
Regularity of Formal/Organized and Informal/On-the-job Conservation Practice Training Sessions

Regularity of Formal Training (n=156)	f	%
Monthly	7	4.5
Every six months	55	35.2
Annually	50	31.8
Every two years	19	12.2
Every five years	5	3.2
Never	20	12.8
Regularity of Informal Training (n=154)
Weekly	39	25.5
Monthly	36	23.5
Every six months	33	21.6
Annually	18	11.7
Every two years	10	6.5
Every five years	4	2.6
Never	14	9.1

When asked which programs are most effective in the education of farmers and landowners, 76% answered that workshops were the best type of training for farmers or landowners to communicate the benefits of conservation practices (Table 3). Conferences/seminars, training videos, and online training were other beneficial training methods that were ranked highly by respondents. Three respondents answered that no training was necessary for farmers or landowners to better understand conservation practices.

Table 3.
Preferred Training Methods for Farmers/Landowners to Learn the Benefits of Conservation Practices

Training Type	f	%
Workshops	123	75.9
Conferences/Seminars	79	48.8
Training Videos	48	29.6
Online Training	43	26.5
None is necessary	3	1.9

The frequency with which the respondents recommended the conservation practices associated with each of the 20 resource concerns identified by NRCS is summarized in Table 4. Based on the responses, 94% of the respondents claimed "always" or "often" when asked how often they recommend conservation practices associated with soil erosion, making them the most frequently recommended resource concern. This was followed by water quality and plant condition associated with plant health and vigor. The least recommended resource concerns were those associated with the depletion of fossil fuel resources, air quality associated with greenhouse gases, and air quality associated with ozone precursors.

Table 4.
Regularity of Recommendation of Conservation Practices Related to Primary Resource Concerns

	Regularity of Recommendation (%)
Resource Concern	Always	Often	Occasionally	Seldom	Never
Soil erosion	53.0	41.0	2.7	2.0	1.3
Soil health	14.3	35.4	33.3	12.2	4.8
Soil contaminants	3.4	22.0	35.6	32.2	6.8
Soil structure	8.8	17.6	33.1	34.5	6.0
Water quantity (non-irrigated lands)	16.4	34.3	26.0	15.8	7.5
Water quantity (irrigated lands)	10.8	20.9	16.9	26.4	25.0
Water quality	24.7	50.7	18.5	4.7	1.4
Plant condition (plant health and vigor)	19.7	46.3	17.0	12.2	4.8
Plant condition (wildfire hazard)	6.2	26.0	27.4	28.1	12.3
Plant pests	11.6	34.9	28.8	18.5	6.2
Fish and Wildlife (inadequate habitat quality)	10.9	44.9	26.5	14.3	3.4
Plant, fish, and wildlife (declining species)	4.7	29.3	35.4	25.9	4.7
Livestock nutrition and husbandry	11.5	30.4	30.4	14.9	12.8
Air quality (particulate matter)	2.0	6.8	24.0	37.7	29.5
Air quality (ozone precursors)	0.0	1.4	18.5	39.0	41.1
Air quality (greenhouse gasses)	0.0	2.0	18.6	36.6	42.8
Air quality (ammonia)	1.4	9.6	19.9	34.9	34.2
Air quality (odor)	1.4	14.2	26.4	29.0	29.0
Depletion of fossil fuel resources	0.0	0.7	17.2	35.9	46.2
Energy production	0.7	4.8	26.9	37.9	29.7

Water Quality

In addition to the questions relating to conservation practices associated with water quantity and water quality, two questions relating specifically to water quality testing were included in the survey. Water quality data collection is another way that conservation planners obtain information about the effectiveness of the conservation programs and practices that they recommend and further familiarizes them with a practice and how it works.

These two simple questions yielded informative results. When asked if water quality testing was a regular function at farms with conservation plans, 88% of the employees responded that water quality testing did not take place on those properties (Figure 1). However, when they were asked if water quality testing would improve conservation planning, 58% responded that it would improve conservation planning (Figure 1). Many respondents commented that testing is expensive and that it would only be useful in certain situations, but the responses indicate a desire on the part of the respondents to have data that evaluates the effectiveness of the conservation practices that are implemented.

Figure 1.
Water Quality Testing and Potential Improvement of the Planning Process by Collecting Water Quality Data

Incentive Programs

The first set of questions relating to specific incentive programs included a Likert scale question for each of the 12 incentive programs, inquiring how frequently (always, often, occasionally, seldom, or never [Table 5]) the agents recommend the incentive programs to farmers or landowners developing conservation plans. The most frequently recommended program was EQIP (Environmental Quality Incentives Program), followed by CTA (Conservation Technical Assistance) and CRP (Conservation Reserve Program). Nearly 89% of the respondents claimed to "never" or "seldom" recommend CCPI (Cooperative Conservation Partnership Initiative), making it the least recommended program, followed by HFRP (Healthy Forests Reserve Program)and GRP (Grassland Reserve Program).

Table 5.
Regularity of Recommendation and Regularity of Enrollment in Incentive Programs by Farmers/Landowners Developing Conservation Plans

	Regularity of Recommendation			Regularity of Enrollment
Program	Always/Often	Occasionally	Seldom/Never	Always/Often	Occasionally	Seldom/Never
EQIP	89.6	4.9	5.6	92.8	5	2.1
CTA	86.4	4.3	9.2	85.6	9.8	4.5
CRP	64.8	23.2	12.0	68.9	26.7	4.5
WHIP	59.7	31.3	9.0	57.2	36.2	6.5
EWP	31.2	31.2	37.6	32	26	42
WRP	25.3	24.6	50.0	28.7	18.4	52.9
GLCI	15.7	23.6	60.7	14.3	27	58.8
CSP	14.3	22.3	63.3	8.7	25.4	65.9
RC&D	13.4	29.8	56.8	15.5	27.1	57.4
HFRP	7.3	13.1	79.6	9.7	14.5	75.8
GRP	7.1	16.4	76.4	7.8	19.4	72.8
CCPI	4.5	6.7	88.8	4.9	9	86.1
CCPI - Cooperative Conservation Partnership Initiative CRP - Conservation Reserve Program CSP - Conservation Security Program CTA - Conservation Technical Assistance EQIP - Environmental Quality Incentive Program EWP - Emergency Watershed Program GLCI - Grazing Lands Conservation Initiative GRP - Grassland Reserve Program HFRP - Healthy Forest Reserve Program RC&D - Resource Conservation & Development Program WHIP - Wildlife Habitat Improvement Program WRP - Wetlands Reserve Program

In order to identify relationships between the frequencies of recommendation versus the frequency of enrollment, the employees were asked with what regularity farmers or landowners actually enroll in the incentive programs when advised. Almost 93% of the respondents believed that farmers or landowners "always" (34%) or "often" (59%) enroll in EQIP when recommended, making EQIP the most commonly adopted program (Table 5). The second and third most adopted programs, as perceived by the respondents, were CTA and CRP. CCPI was the least adopted program, followed by HFRP and GRP.

Relationships between high recommendation rates and the perceived high rates of enrollment in EQIP and CTA programs are best described by the nature of those programs. EQIP and WHIP pay for 75% of the installation costs of conservation practices. Neither of these programs requires the farmer to take land out of production rotation. CTA is a program designed to provide conservation planning technical assistance to farmers. The CRP program requires the farmer to remove land from production, which would be a barrier to adoption, but it is a well-known program that provides monetary incentive to remove farmland from agricultural rotation.

NRCS personnel opinion about the farmers' lack of participation in the incentive programs were identified by asking the reasons that farmers choose not to enroll in the available programs. Respondents were asked to categorize the reasons for not participating as economic, cultural, or social, and then to elaborate with an open-ended response. An overwhelming majority (64%) answered that economics played a role in the farmers' lack of participation, while social and cultural reasons were split at 18% each (Figure 2).

Figure 2.
NRCS Perception of Farmer Reasons for Choosing Not to Participate in Conservation Programs

The open-ended responses that accompanied the previous response generally fell into one of the following four categories (Table 6):

1. A culture of mistrust of the government;


2. economic reasons on the part of the government;


3. economic reasons on the part of the landowner; and


4. social reasons associated with the application process.

Table 6.
Statements Supporting NRCS Personnel Perceptions of Nonparticipation by Farmers in Conservation Programs

Reasons for Non-Participation	Statement
Cultural - mistrust of the government	"Don't want to get involved with the Government", "Some are afraid of the government" "Not totally trusting in the Government" "Some mistrust of the Government" "...Some folks just don't trust the government still" "Some farmers are leery of the government knowing so much about their farms..."
Economic - reasons on the part of the government	"Those listed, other than CRP, EQIP, and WHIP, are not funded in our area to any degree..." "Often times the incentive program may not be economically beneficial in the long run. I believe that the agency should pay 100 to 110 percent of conservation practices that are needed." "No funding available in the County" "The cost share rates are not up-dated to reflect the actual cost of the material" "Not all conservation programs are available..." "Some programs are not funded in our county and others are not adequately funded such as EQIP and WHIP"
Economic - reasons on the part of the landowner	"Rapidly developing county. Property is too valuable to tie up in long term contracts." "Incentive programs do not address small scale limited resource producers" "Just don't see if they can afford to!" "Without good information and demonstrations, too many farmers do not realize the true economic benefits of participation in various programs" "Some just look at direct cost versus long-term benefits and economic returns."
Social - the time-consuming application process	"Lack of understanding the application process" "...Some do not like the requirements and red tape they have to commit to get incentives" "Many times they get frustrated with the paperwork/length of time it takes to apply/provide requested documentation it takes to get in a program..." "...Others simply do not want to go through all of the hoops that is required for cost share programs" "Too much red tape. Too many requirements. Programs ask landowners to make too many long range decisions in a world that changes daily."

Conclusions

Conservation programs represent a partnership between the public and governmental entities with the common goal of promoting biodiversity maintenance, a goal that is worth promotion by Extension professionals (Gerlach, 2009). In addition, a collaborative partnership exists between Extension and NRCS whereby Extension is relied upon to provide accurate and timely conservation information to its agency partners (Ishler et al., 2006).

As farm operators become more environmentally committed, they will depend on Extension personnel for sustainable business models (Harrison, 2002), and NRCS will be a conduit for much of the incentive funding that helps to promote the implementation of conservation practices. As such, an evaluation of an Extension partner such as NRCS provides useful information for setting communication and outcome goals that address the flow of information from Extension to the partners. The evaluation also provides valuable background information to Extension regarding the demographics and daily practices of its partners. A statewide evaluation is appropriate because Extension and NRCS operate administratively at that scale.

Personnel at the Mississippi NRCS are overwhelmingly male, with a fairly even distribution of age classes. However, the most common age class at the agency is between 50 and 59 years of age, leading to turnover within the next decade. As new employees are brought into the agency, training will continue to be an important aspect of their duties. Extension programs and publications can provide a portion of this training.

Our research found that almost half of Mississippi NRCS employees take advantage of informal, in-the-field training opportunities on at least a monthly basis. A majority of employees also take part in formalized conservation planning training at least annually. An important function of Extension personnel will be their availability to meet in the field to discuss conservation practices and bring new information to NRCS. Our study showed that workshops are overwhelmingly favored by NRCS personnel for the transfer of information to their constituents. We presume that workshops for NRCS personnel would also be favored.

Transferring conservation planning information to farmers by NRCS personnel comes in the form of formal training and the recommendation of incentive programs implementing conservation practices on farms. In this regard, the duties of information transfer and training are very similar to those of county and state Extension personnel. Workshops, conferences, and seminars are perceived to be the most effective methods of disseminating conservation information to farmers. Workshops offer opportunities for farmers and conservation planners to discuss field methods with a high level of "give-and take," which has proven useful in developing conservation innovations (Hagman, Chuma, & Murwira,1996).

Three incentive programs (EQIP, CTA and CRP) that lead to the implementation of conservation practices dominate recommendations to farmers. These programs are the most comprehensive and well funded. Other programs are dependent on specific resource factors and funding levels that preclude them from widespread recommendation. Many of the conservation planners surveyed feel that in-field evaluation of the outcomes associated with the implementation conservation practices, for example with water quality monitoring programs, would improve the conservation planning process. Extension should be aware of these programs and how they can provide their expertise when a program is adopted by a landowner.

Air quality, fossil fuel depletion, and energy production are resource concerns of the USDA that are not being addressed on a regular basis by the Mississippi NRCS. These resource concerns have been, by comparison to other concerns, recently added to the lexicon of the conservation planner (Michele Laur, pers. communication, June 2009). The addition of these resources to the NRCS Strategic Plan (USDA, 2005) encourages evaluation of the contribution of farms to meeting air quality standards and will receive greater scrutiny in the future. However, at present no Mississippi counties are designated as "non-attainment" areas (not meeting national air quality standards) by the U.S. Environmental Protection Agency (USEPA 2005), which is obviously a factor in their low recommendation rate.

We recommend that Extension agencies study the methods and perceptions of other entities that rely on them for information, materials, and programs. In this case, the perceptions of what programs are commonly recommended by NRCS, enrolled in by farmers and reasons for farmers not enrolling are useful in program development and the production of educational materials. For example, knowing that financial reasons are perceived to affect adoption of conservation practices more than other factors should lead to increased information about alternative income streams associated with some practices, particularly wildlife related practices.

The Natural Resource Enterprise Demonstration Center is an example of a cooperative program between Extension, NRCS, and the Mississippi Agriculture and Forestry Experiment Station that is working to determine the potential economic impact of conservation-based programs that produce alternative income streams. This type of program directly addresses farmer and landowner concerns about the economics of government incentives and alternative sources of income and benefits from information about potential barriers to adoption and strategies for overcoming them.

Demographic data, preferences with regard to educational programming and training frequency, and farmer/landowner response to programs all inform the conservation planning process. Extension and other agencies are training and planning partners. These inquiries represent assessments that will help both agencies attain their conservation goals and objectives.

References

Andrews, M. (1983). Evaluation: An essential process. Journal of Extension [On-line] 21(5). Available at: http://www.joe.org/joe/1983september/83-5-a1.pdf

Archer T. M., Bruns, K., & Heaney, C. A. (2007). SAMMIE: Using technology for a one-stop program evaluation resource. Journal of Extension [On-line], 45(5) Article 5TOT1. Available at: http://www.joe.org/joe/2007october/tt1.php

Arnold, M. E. (2002). Be "logical" about program evaluation: Begin with learning assessment. Journal of Extension [On-line], 40(3) Article 3FEA4. Available at: http://www.joe.org/joe/2002june/a4.php

Bailey, S. J., & Deen, M. Y. (2002). A framework for introducing program evaluation to extension faculty and staff. Journal of Extension [On-line], 40(2) Article 2IAW1. Available at: http://www.joe.org/joe/2002april/iw1.php

Couper, M. P. (2000). Review: Web surveys: A review of issues and approaches. The Public Opinion Quarterly 64(4): 464-94.

Crawford, S., Couper, M., & Lamias, M. (2001). Web surveys. Social Science Computer Review 19(2): 146-62.

Denscombe, M. (2006). Web-based questionnaires and the mode effect: An evaluation based on completion rates and data contents of near-identical questionnaires delivered in different modes. Social Science Computer Review 24(2): 246-54.

Dillman, D. A. (2007). Mail and Internet surveys: The Tailored Design Method. 2nd ed. Hoboken, New Jersey: John Wiley and Sons, Inc.

Gerlach, J. D. (2009). The Conservation Reserve Program: A tool for public participation in biodiversity management and conservation. Journal of Extension [On-line], 47(3) Article 5TOT3. Available at: http://www.joe.org/joe/2009october/tt3.php

Hagmann J., Chuma, E., & Murwira, K. (1996). Improving the output of agricultural extension and research through participatory innovation development and extension; experiences from Zimbabwe. The Journal of Agricultural Education and Extension 2(4) 15-23.

Harrison, J. D. (2002). Managing for sustainable agriculture. Journal of Extension [On-Line], 40(4) Article 4FEA5. Available at: http://www.joe.org/joe/2002august/a5.php

Ishler, V., Dodd, A., Abdalla, C., Martin, G., Meinen, R. J., Mikesell, R. E., & Weld, J. L. (2006). Agricultural environmental programming in Pennsylvania: Increasing visibility and relevancy of Extension. Journal of Extension [On-line], 44(4) Article 4FEA7. Available at: http://www.joe.org/joe/2006august/a7.php

Kiernan, N. E., Kiernan, M., Oyler, M. A., & Gilles, C. (2005). Is a Web survey as effective as a mail survey? A field experiment among computer users. American Journal of Evaluation 26(2): 245-52.

Morford, S., Kozak, R., Murari, S., & Innes, J. (2006). Factors affecting program evaluation behaviours of natural resource extension practitioners - motivation and capacity building. Journal of Extension [Online], 44(3) Article 3FEA7. Available at: http://www.joe.org/joe/2006june/a7.php

Radhakrishna, R., & Martin, M. (1999). Program evaluation and accountability training needs of Extension agents. Journal of Extension [On-line], 37(3) Article 3RIB1. Available at: http://www.joe.org/joe/1999june/rb1.php

Rennekamp, R. A., & Arnold, M. E. (2009). What progress, program evaluation? Reflections on a quarter-century of Extension evaluation practice. Journal of Extension [On-line], 47(3) Article 3COM1. Available at: http://www.joe.org/joe/2009june/comm1.php

Roucan-Kane, M. (2008). Key facts and key resources for program evaluation. Journal of Extension [On-line], 46(1) 1TOT2. Available at: http://www.joe.org/joe/2008february/tt2.php

United States Department of Agriculture - Natural Resource Conservation Service. (2005). Productive Lands, Healthy Environment - NRCS Strategic Plan. Retrieved May 23, 2009 from: http://www.nrcs.usda.gov/ABOUT/strategicplan/index.html

United States Environmental Protection Agency. (2005). Counties Designated "Non-attainment" for Clean Air Act's National Ambient Air Quality Standards (NAAQS). Retrieved May 23, 2009 from: http://www.epa.gov/oar/oaqps/greenbk/mapnpoll.html