Process and Reality: Working with a Local Watershed Organization to Develop a Restoration Effectiveness Monitoring Program

There is a critical need for information on the relative effectiveness of measures taken to restore or rehabilitate riparian and aquatic habitats and associated upland watersheds. In California, alone, millions of dollars are spent every year for implementing restoration projects. However, in the rush to implement restoration, few dollars have been expended on evaluation of the effectiveness of restoration projects or techniques for achieving specified objectives. Many restoration projects have been undertaken without clearly defined objectives or provisions for intermediate to long-term monitoring. If done at all, monitoring is typically limited to the implementation (i.e., construction) phase.

Restoration is often undertaken under the direction of local watershed organizations and Resource Conservation Districts. Working with such stakeholder groups is a key opportunity for Extension professionals.

In the Feather River watershed of northeastern California there has been an extremely active restoration program for the past 15 years. The Feather River Coordinated Resource Management group (CRM), a consortium of 21 local, state, and federal agencies and private landowners has implemented over 30 different stream and riparian restoration projects since 1985. Some of these involve rather extensive modifications to stream channels; others involve changing land use practices (exclusion or management of grazing) or simple revegetation.

The stated goal of the CRM is: "to maintain in perpetuity the stability, vitality, and diversity of the Feather River watershed and its communities." In general, the CRM's restoration projects have been undertaken in an opportunistic manner, responsive to cooperative landowners and availability of funding. Monitoring, however, has been short-term and limited.

In 1996, with the intention of stimulating local interest, the University of California Cooperative Extension (UCCE) held a technical workshop on ecological and environmental monitoring in the Feather River watershed. It was well attended and generated much local discussion on the need for monitoring restoration activities. Later in the year, with the support of the CRM, UCCE obtained funding for beginning the process to establish a "demonstration monitoring watershed" wherein the focus would be evaluating the effectiveness of restoration in "improving water quality and quantity" because that has been the principal aim of past restoration projects. Our objective was to designate a watershed and obtain the commitment of the Feather River CRM agencies to implement a long-term monitoring program.

In this article, we report on the methods and results of that effort. What we found working with an existing group for the purposes of establishing a monitoring program may be helpful to other Extension workers.

The Process in Theory

Our past work with the Feather River CRM had led us to conclude that its overall monitoring goal was to assess the effectiveness of both riparian and upland restoration practices. Some typical CRM riparian restoration activities include installation of check dams, stream channel realignment, streambank stabilization, and revegetation. Upland activities include road rehabilitation and forest thinning. In our proposal for funding to select a demonstration watershed we stated our assumed goal as follows:

"The goal of the Feather River Demonstration Watershed will be to provide scientific monitoring of the efficacy of watershed restoration efforts in terms of water quality as well as yield and timing and other attributes as identified."

On the basis of this assumed goal, we designed a process by which a demonstration watershed could be selected and a monitoring plan prepared. Two committees, one comprised of CRM representatives and the other comprised of outside peer reviewers, would develop and prioritize criteria for watershed selection through a DELPHI process.

In DELPHI, an initial list of criteria is given to a group. Each member ranks the criteria according to priority on a scale of 1-5 and suggests deletions or additional criteria. The analyst averages scores to develop a second list representing the ordered criteria. This list is given to the group; each member re-ranks the criteria and returns it. The reiterative process repeats until the group reaches a final consensus on priority. The final criteria would then be applied to existing data for Feather River sub-basins (approximately 5-10,000 acres in size), and one or more of these would be selected for the establishment of a monitoring program. The final step in our project would be the preparation of an effectiveness-monitoring plan, including provisions for baseline data collection. We expected to complete this project in one year.

As is usual in DELPHI, we planned to do most of the work on criteria development by mailing ranked and re-ranked lists to participants. Peer review would be accomplished primarily through the media of electronic mail. We anticipated two to three meetings of the CRM panel over the course of the project and our funding included travel costs for panel members. We did not anticipate a need for any face-to-face meetings with the peer panel.

Results

Criteria Development and Goal Formulation

We met with the Feather River CRM Management Committee to present our proposed research agenda and solicit members for the policy panel. The CRM nominated a 29-member Technical Advisory Committee (TAC) that included representatives from most member agencies, local County supervisors, private landowners, and restoration project workers. We solicited membership on the scientific review panel from academic and agency researchers. Representatives of the University of California and University of Nevada, the USDA-Forest Service, and the California Department of Forestry and Fire Protection agreed to serve on the peer panel. Of the 29 nominated TAC members, as many as 20 attended one or more of the organized workshops.

An initial questionnaire was sent to the TAC. (Copies of questionnaire forms used in this project are available on request from the authors.) We suggested the following general criteria for consideration by TAC members and asked them to rank them in terms of their importance to selection of a monitoring watershed:

Accessibility: for public involvement and education.
Representativeness: to ensure transferability of results to other watersheds.
Reference conditions: at least some parts of the watershed represent target conditions for restoration.
Existing or proposed restoration projects: the watershed has or will have projects that will be available for effectiveness monitoring.
Detailed data are available: the watershed has been subject to detailed environmental studies in the past or is proposed for studies in the future.
Land cover proportions: vegetation types, water, urban uses. Geomorphology: watershed physical and hydrologic conditions.
Habitat: diversity of aquatic, riparian and terrestrial habitats.

TAC members ranked these criteria and suggested some others related to land ownership, restrictions on uses, and historical land use. After the results of the first questionnaire were obtained, we synthesized the ranking and returned a second questionnaire to the TAC for re-ranking.

When the second questionnaire was returned to us, several things became evident. First and most important, respondents indicated a lack of consensus on our assumed goal "to evaluate effectiveness of upland and riparian restoration." There seemed to be a split in the group between the desirability of effectiveness monitoring versus "demonstration" of restoration techniques. Given that, responses on other criteria were questionable. For example, criteria related to land ownership (preference for all-public, all-private, or mixed ownership) were all ranked similarly low, indicating no clear direction. Criteria that were ranked the highest were existence of multiple land and resource management problems, representativeness, accessibility, existence of historical data, and existence of varied habitat and environmental conditions. There was no clear emphasis on effectiveness monitoring in the selection and ranking of criteria. We determined that it was necessary to backtrack and develop consensus on the goals for establishing and conducting a monitoring program.

We met with the TAC twice to develop goals. The group eventually consented to the following two-part goal statement for the demonstration watershed-monitoring program:

1) Evaluate effectiveness of restoration in reducing erosion and sedimentation, while considering effects on water quantity and flow timing, wildlife, fisheries and forage at the project, reach, and small and large watershed scales, and 2) provide opportunities for education, training and technology transfer pertinent to monitoring.

This goal was formally adopted by the TAC and by the CRM Management Committee. The first part generally corresponded to our initial understanding. The second part reflected a lesser degree of interest in effectiveness monitoring on the part of some TAC members.

During the course of developing this goal statement, several issues arose. The question of the appropriate scale for monitoring became especially important and one on which we consulted extensively with our scientific review panel. If the aim was to monitor the effectiveness of restoration in reducing erosion and sedimentation while considering effects on other values, such as wildlife, fish, etc., then the literature and our scientific panel suggested that there are different possibilities at different scales.

At the large-watershed scale (>10,000 acres), it would be possible to conduct trend monitoring of water quantity, water quality (sediment yield, sedimentation rates in reservoirs), and land cover changes. There are no documented cases of restoration effectiveness monitoring at the large-watershed scale.

The literature and our peer review panel suggested that effectiveness monitoring would be feasible at the small-watershed scale (<1000 acres) at which most studies of watershed effects from land use treatments have been conducted. Possibilities include paired experiments in geomorphically similar basins to evaluate watershed-level restoration effects on stormflow sediment, stream temperature, channel pattern, bank stability, mass wasting, and wildlife/fish/invertebrate populations. At the project-site scale (1-5 acres), monitoring would focus on implementation (i.e., was the project done as planned?), longevity (did the project survive?), and site-specific effectiveness (was a site-specific problem solved?). This type of monitoring is already being done but tends to be short-duration (1-3 years).

The realization of the effects of scale on monitoring expectations was important to the Feather River CRM because much of its hope rests on large-scale watershed benefits. The literature suggests that the likelihood of demonstrating such cumulative benefits from many individual restoration projects is low, especially in the short-term because of natural variability and the relatively small proportion of a large watershed that may be economically feasible to restore.

Related to the scale issue was the suggestion by our peer panel that the monitoring program should be based on a conceptual model of restoration effectiveness. A search of the literature indicated that there were neither models of properly functioning Sierra Nevada watersheds nor models indicating how management might alter processes to restore functions in degraded systems. Model development has since become a major part of our ongoing research agenda.

Other issues that arose included the need for a vigorous outreach program to get private landowners to cooperate with a monitoring program. This issue underlay the ambiguity in preference for land ownership in a demonstration watershed. In view of uncertainty, the TAC concluded that it would be advantageous to confine the monitoring program to public lands (i.e., National Forests) while seeking ecological and environmental representation of private lands. Also, the manner in which restoration projects had been implemented in the past (opportunistically, subject to availability of funding and landowner willingness) was seen as an obstacle to effectiveness monitoring at the watershed scale. Finally, there was a feeling in the TAC that monitoring efforts should not substitute for on-the-ground restoration activities due to limited staff and funding.

Watershed Selection

When we began this project, some TAC members had definite ideas on which sub-basins within the greater Feather River watershed would be most suitable for demonstration monitoring. The goal that was acceptable reflects the division within the group between monitoring for purposes of assessing restoration project effectiveness and demonstration for educational purposes.

While engaged in our watershed selection process, two things happened that ultimately affected the outcome. One CRM agency representative (Regional Water Quality Control Board) developed a proposal to establish 22 stations for trend monitoring throughout the Feather River watershed. This proposal became one element of what was eventually to be a multi-scale monitoring concept, including trend, effectiveness, and implementation monitoring.

The second important development was the proposal by other CRM representatives (USDA-Forest Service, primarily) to focus future restoration efforts in the 90,000-acre Last Chance watershed, the easternmost watershed that still drains to the Sacramento River. The intent for Last Chance was to emphasize restoration of meadows and associated streams and groundwater aquifers to enhance water storage, restore wet meadow communities, reduce downstream flood peaks, and augment summertime baseflows. Anecdotal evidence has indicated that check dams, grazing and vehicular exclosures, revegetation, and streambank stabilization in combination may achieve most of these effects, at least at the local level.

After developing a final list of selection criteria, the TAC used the selection criteria to screen candidate watersheds. The Last Chance basin was chosen after comparative analysis with other basins (Table 1). It was assumed that effectiveness monitoring would primarily occur at the small-watershed scale using a paired basin approach. Results of the screening indicated that Last Chance did indeed meet many criteria that are pertinent to restoration effectiveness monitoring, but the remoteness of the basin made it relatively unsuitable for public education and technical training.

The TAC concluded at this point that two different watersheds were needed--Last Chance and another, which is more accessible. By consensus, it was decided that a decision on an educational demonstration monitoring watershed would be temporarily deferred. We also deferred choice of sub-basins for comparative effectiveness monitoring until such time as restoration projects are actually proposed. The TAC concluded that detailed watershed analysis would be needed to select sub-basins.

Table 1
Final Criteria for Selecting a Demonstration Monitoring Watershed and Results of Application to Last Chance Watershed

Criteria	Suitability of Last Chance Watershed
Potential for forest health restoration	High
Potential for riparian restoration	High
Potential for riparian restoration	High
Potential for road restoration	Moderate
Existence of many land and resource management issues	Moderate
Accessibility to population center	Low
Seasonal accessibility	Moderate
Representativeness	Moderate (i.e., Eastside Sierra)
Local interest to community	Moderate
Land use restrictions	High (i.e., low restriction)
Availability of historical data	High
Availability of streamflow data	Moderate
Detailed studies planned or Underway	High
Wide range of habitats	High (i.e., diverse habitats)
Variability in geomorphology	High (i.e., diverse geomorphology)
Likelihood of pairable watersheds	High
Climate and microclimate	High
Past and present human uses	High
Land cover proportions	High (i.e., diverse habitats)

The products of this project included a clear statement of goals, an understanding of the effects of scale on monitoring, a final choice of an effectiveness-monitoring watershed, and a proposal (now funded) for watershed trend monitoring. Coincidentally, funding was obtained for establishing some reference monitoring sites within the Last Chance watershed. The project forever changed the understanding of and support for environmental monitoring among members of the Feather River CRM. Increased awareness has created a foundation for a continuing effort. Currently, we are seeking funding to do research on meadow hydrology in the Last Chance basin.

The forwarding of recommendations to the Feather River CRM Management Committee concluded the project. Below, we discuss how the process actually used in this project compares to the process we proposed at the outset. We feel that the differences between theory and reality are an important lesson for other Extension workers. We suggest a modified process based on our experiences.

Discussion

What actually happened in this project differs from what we had anticipated in three fundamental ways.

Our assumptions about the goal for monitoring were wrong, and we had to step back and spend considerable time developing an acceptable goal.

The differences in opinion among TAC members led to the determination that effectiveness monitoring and educational objectives could not be achieved within a single watershed.

The development of selection criteria and the application of those were modified from our expectations. Although we had anticipated applying the criteria to all possible sub-basins >10,000 acres within the Feather River watershed, local knowledge and professional judgment reduced the choices a priori. Ultimately, professional judgment of the TAC was much more important in selecting the watershed than the abstract criteria.

In view of these lessons, we propose a modified process for working with a local group to select a watershed for monitoring (Figure 1). The process outlined below could be adapted to any similar project with a local watershed organization, such as restoration project site prioritization and site selection or prioritization of watersheds for conservation and protection. In our view, this modified process presents greater potential for success.

Figure 1
Modified Process for Selecting a Demonstration Monitoring Watershed

Step 1: Problem Formulation/Project Initiation (3 months):
Procure start-up funding
Establish peer and local interest panels
Preliminary data gathering
Literature search
Step 2: Establish Project Goal(s) (1 month):
Define project area
Develop overall goal statement
Peer review
Step 3: Define Monitoring Scales and Issues/Objectives (2 months):
Peer review
Step 4: Develop Conceptual Model for Watershed (1 month):
Prepare flow/process diagram for watershed
Develop sub-models indicating relationships between issues/activities and processes/results
Identify key monitoring variables for each scale/issue
Peer review
Step 5: Assemble Data Base and Determine Selection Criteria for Candidate Basins (3 months):
Delineate candidate basins
Procure data for candidate basins
Create data management system
Develop and prioritize criteria for basin selection
Step 6: Data Analysis for Basin Selection (3 months):
Develop statistical profiles for each basin based on criteria
Step 7: Select Demonstration Basin (1 month):
It is assumed here that a 10-20,000 acre basin will be selected that is representative of processes at larger scales and wherein monitoring at other finer scales can be performed.
Present results of Step 6 to local panel
Conduct selection process
Peer review
Step 8: Conduct Issue-Related Watershed Analysis for Selected Basin (6 months):
Procure funding
Generally, the goal is to determine environmental and ecological conditions relative to important issues.
Step 9: Present Results of Watershed Analysis (1 month):
Local panel review
Peer review
Step 10: Design Monitoring Plan(s) for Selected Basin (3 months):
Develop specific methodologies for monitoring at relevant scales to answer key questions and address issues. This may include an overall basin-level monitoring approach and/or specific approaches to effectiveness monitoring at paired sub-basin, reach and project level. This will depend on whether or not projects are proposed for which monitoring plans can be prepared. If no projects are proposed, specific monitoring plans may be prepared in the future. But the overall plan will be prepared and will guide monitoring at other scales.
Local panel review
Peer review
Step 11: Pilot Monitoring Program (6 months):
Procure funding
Test monitoring protocols
Revise monitoring plan(s) as necessary
Local panel review
Peer review
Step 12: Implement Long Term Monitoring (multiple years):
Procure funding

Acknowledgments

This project was funded by the University of California, Division of Agriculture and Natural Resources Competitive Grant program. The authors sincerely appreciate the willingness and determination of the Feather River Coordinated Management Group.