October 2003 // Volume 41 // Number 5 // Research in Brief // 5RIB2
 |  |
 |
Concordance Among Extension Workers, Researchers, and Professional Arborists in Rating Landscape Trees
Abstract
Formulas developed to determine the monetary value of landscape trees
require a species rating value. Polling Extension professionals, researchers,
and arborists is a common procedure to derive species rating value. Due
to differences in professional training and background, and the variability
of landscapes in any given area, professional agreement on the rating of
a species is difficult to reach but is necessary to be useful. The Delphi
method and W criterion were used to determine the strength of concordance
among extension professionals. We hope that other Extension professionals
can use this method in developing species rating in their respective areas.
Introduction
The extent of urban expansion across the U.S. has tripled since 1950 (Dwyer, et al., 2000), and about 75% of the population now lives in urban areas. There are several reasons why this trend is relevant to the role and work of Cooperative Extension professionals. Most notably, the demographics and needs of clientele have changed. In particular, the client base is increasingly urban and interested in a wide range of natural resource topics rather than a narrow agricultural focus (Rodewald, 2002). This increased interest in natural resources has in turn resulted in increased public awareness of the importance of trees in urbanized and developing landscapes.
Landscape trees provide a variety of benefits for urban dwellers, including monetary value. The need to know the monetary value of landscape trees is no longer limited to real estate transactions, lawsuits, insurance claims, and tax purposes, but also holds true for tree inventories and tree care investment decisions.
Formulas are the most commonly used method for determining the value of trees. Formulas and methods developed over the years are described in the 8th edition of the Guide for Plant Appraisal by the Council of Tree and Landscape Appraisers (CTLA). The most widely used is the trunk formula method developed for trees too large to be physically replaceable. Using the formula requires the determination of size, condition, location, and species rating value based solely upon species characteristics. Watson (2000) compared the trunk formula method with other formulas and concluded that the major differences in appraised values using the trunk formula are attributed to species rating values.
Cooperative Extension specialists, particularly foresters and horticulturists, are often asked to provide species rating value. Traditionally, they have been using species values that came with the Guide. These species rating values were subjectively assigned by a group of experts in 1970 (Lewis, 1970). The current species rating values are in five categories of 20% class intervals. The Guide classifies trees according to species and varieties, varying according to different geographical areas in the country.
Problems with Current Species Rating Procedures
The traditional process of polling experts to arrive at species rating values has inherent problems. First, the likelihood exists for high incidences of missing values due to differences in level of familiarity of the experts with the species in question. Second, the process allows for a wide range of values assigned to every species. This could be attributed to the differences in training and background of the experts and the wide variability of landscapes across a locale, state, or region.
Both problems could result in weak agreement or lack of agreement among professionals and thus compromise the utility and credibility of species ratings generated through such a process. Although it is essential that a group of experts representative of end users of the species rating values be formed to determine species rating values that are acceptable and useful, it is equally important that there is strong agreement among them on the resultant species rating values.
In an attempt to address these problems, Blanche, Guidry, and Wefel (1997) developed a more objective approach to establishing a single value for a given species using tree characteristics that are quantifiable and inherent, such as: flood tolerance, pollution tolerance, specific gravity, longevity and others. However, their approach remains the exception, and polling judgments and opinions of tree experts remains the most popular approach for determining species rating value.
Purpose of Present Study
In the study described here we used the Delphi method to collect data on species rating and used the Kendall coefficient of concordance to test the strength and quality of agreement of a panel of Extension professionals among themselves and in comparison with other professional groups.
The Delphi Method
An under-utilized method combining quantitative and qualitative opportunities to poll expert judgments for species rating values is the Delphi method. The Delphi, an iterative process for evoking expert opinion, has three primary features: (1) anonymity, (2) controlled feedback, and (3) statistical group response (Dalkey, 1969).
Thus, the Delphi offers several advantages for collecting species rating values from experts with varying background and levels of experience. It reduces the psychological pressures of face-to-face confrontation and the possibility of an especially vocal or powerful individual dominating the final outcome. Also, improved agreement increases with the number of experts and with additional iteration. Last, there is potentially greater acceptance of the group opinion by the individual participants using Delphi than there is with face-to-face procedures.
Today, the Delphi is used in a wide range of applications in industry, governmental agencies, and other organizations. Ludwig (1997) described the use of the Delphi by Extension professionals to help clientele or customers determine where programs or applied research should focus.
Kendall Coefficient of Concordance (W)
The Kendall coefficient of concordance (W) is used to determine the association among k sets of species rating values. W is an index of the divergence of the actual agreement shown in the data from the possible perfect agreement. Values of W can range from 0 to 1, with 0 indicating perfect disagreement, and 1 indicating perfect agreement (Landis & Koch, 1977).
Materials and Methods
The Panel of Experts
The study took place in Louisiana through an Urban and Community Forestry grant from the Louisiana State Office of Forestry. Nineteen individuals from each major geographical region of the state were initially identified and asked to independently provide expert opinion on the species rating value of landscape plants in Louisiana. Each individual was considered an "expert" based on their roles and previous experiences, and knowledge of species rating. These experts included six foresters, three professional/consulting arborists, one university horticulture professor, one landscape architect, and eight Cooperative Extension Service personnel. Four of the experts failed to return their respective responses by the due date; thus, the analysis was based on the rating provided by the remaining 15 experts.
List of Species
A list of 205 plant species was sent to each expert accompanied by a short explanation of the objective of the project and a copy of Chapter 5 of the Guide. Chapter 5 describes the factors to consider in arriving at a species rating value based upon species or cultivar characteristics without regard to condition and location factors.
For purposes of this article, results associated with the 63 species in the International Society of Arboriculture (ISA) list of Louisiana trees are shown.
The Delphi Method and Test of Agreement
During the first round of the Delphi, experts were also invited to comment on their rationale for the rating and add additional species considered to be important and included in the list. The results were compiled, and basic descriptive statistics, including range, average, and median percent rating for each tree species were calculated.
During the second round, the experts were asked to review the species rating results from the first round of the Delphi. Experts were asked to reconsider their respective ratings with reference to the ratings of the other experts.
The degree of agreement as measured by Kendall coefficient of concordance (W) among the 15 professionals in assigning the plant species among the five classes was determined for the results of each round.
Results and Discussion
Species Experts Failed to Assign Value Rating
The number of species each expert failed to assign a value rating reflects the first problem identified above. Two of the experts, a forester and a horticulturist, rated every species on the entire list. The former works with the NRCS Plant Data Center, and the latter is the oldest expert in the group. Their familiarity with the species could be explained by their work and experiences. Overall, the arborists failed to assign between 10% and 48% of the species; the foresters, between 14% and 31%; the horticulturist, 1% and 49%; and the landscape architect, 11%. This clearly indicates the differences in familiarity of the species by the experts.
Species Rating Values Collected
Table 1 illustrates problem 2. The assigned species rating values varied substantially among the professional groups. Species values ranged from 40.0% to 95.8% for foresters, 30.8% to 88.6% for horticulturists, and 10.0% to 95.0% for consulting arborists. Six of the 10 highly rated tree species are common to the three professional groups. These six species are Ilex opaca, Quercus alba, Quercus falcata var"pagodaefolia", Quercus shumardii, Quercus virginiana, and Taxodium distichum. In contrast, only two of the 10 tree species which rated the least favorable are common to all three professional groups; these are Salix nigra and Morus rubra.
|
Expert |
|||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Scientific Name |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
11 |
12 |
13 |
14 |
15 |
|
Ilex opaca |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
|
Quercus alba |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
|
Quercus falcate var "pagodaefolia" |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
. |
1 |
1 |
1 |
1 |
1 |
|
Quercus shumardii |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
|
Quercus virginina |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
|
Taxodium distichum |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
|
Magnolia grandiflora |
1 |
2 |
1 |
1 |
1 |
2 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
|
Magnolia acuminata |
2 |
2 |
2 |
2 |
2 |
. |
2 |
3 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
|
Magnolia virginiana |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
1 |
2 |
2 |
2 |
2 |
2 |
|
Fagus grandifolia |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
1 |
2 |
1 |
|
Carpinus carolinia |
2 |
2 |
2 |
4 |
2 |
2 |
2 |
2 |
2 |
2 |
1 |
2 |
2 |
2 |
2 |
|
Ostrya virginiana |
2 |
2 |
2 |
3 |
2 |
1 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
|
Oxydendrum arboreum |
2 |
2 |
2 |
3 |
2 |
. |
2 |
3 |
2 |
2 |
2 |
2 |
. |
2 |
2 |
|
Pinus taeda |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
2 |
1 |
2 |
2 |
1 |
1 |
2 |
2 |
|
Prunus caroliniana |
3 |
3 |
3 |
3 |
3 |
2 |
3 |
3 |
3 |
3 |
2 |
3 |
2 |
3 |
3 |
|
Celtis occidentalis |
3 |
3 |
3 |
3 |
4 |
4 |
3 |
4 |
3 |
3 |
3 |
3 |
. |
3 |
3 |
|
Gleditsia triacanthos |
3 |
3 |
3 |
3 |
3 |
4 |
3 |
5 |
2 |
3 |
3 |
3 |
3 |
3 |
3 |
|
Cedrus atlantica |
2 |
. |
2 |
3 |
. |
. |
2 |
3 |
2 |
2 |
2 |
2 |
. |
2 |
1 |
|
Halesia carolina |
. |
2 |
2 |
2 |
2 |
1 |
2 |
2 |
2 |
2 |
2 |
2 |
3 |
2 |
1 |
|
Koelreuteria paniculata |
3 |
3 |
3 |
3 |
3 |
. |
3 |
5 |
. |
4 |
3 |
3 |
4 |
3 |
3 |
|
Quercus lyrata |
2 |
2 |
2 |
1 |
2 |
2 |
2 |
2 |
1 |
1 |
2 |
2 |
1 |
2 |
2 |
|
Acer negundo |
4 |
4 |
4 |
5 |
4 |
5 |
4 |
5 |
. |
4 |
4 |
4 |
4 |
5 |
4 |
|
Acer rubrum |
1 |
2 |
2 |
2 |
2 |
1 |
2 |
2 |
. |
2 |
2 |
1 |
2 |
2 |
1 |
|
Salix nigra |
4 |
5 |
4 |
5 |
4 |
5 |
4 |
4 |
4 |
. |
4 |
4 |
4 |
5 |
4 |
|
Chionanthus virginicus |
2 |
2 |
2 |
2 |
2 |
1 |
2 |
2 |
3 |
2 |
2 |
2 |
1 |
1 |
2 |
|
Pinus echinata |
2 |
2 |
2 |
4 |
2 |
2 |
2 |
2 |
4 |
2 |
2 |
2 |
3 |
3 |
2 |
|
Nyssa sylvatica |
2 |
2 |
2 |
2 |
2 |
3 |
2 |
3 |
2 |
2 |
2 |
1 |
2 |
4 |
2 |
|
Ulmus pumila |
4 |
4 |
4 |
5 |
4 |
5 |
4 |
5 |
4 |
5 |
4 |
4 |
4 |
5 |
4 |
|
Acer palmatum |
2 |
3 |
2 |
3 |
2 |
2 |
2 |
1 |
. |
2 |
2 |
2 |
2 |
2 |
1 |
|
Maclura pomifera |
3 |
3 |
3 |
4 |
3 |
3 |
3 |
4 |
. |
2 |
3 |
3 |
2 |
3 |
3 |
|
Morus rubra |
3 |
3 |
3 |
5 |
3 |
4 |
3 |
3 |
4 |
. |
3 |
3 |
2 |
3 |
3 |
|
Cedrus deodara |
2 |
2 |
2 |
3 |
. |
. |
2 |
3 |
2 |
2 |
2 |
2 |
2 |
3 |
1 |
|
Acer saccharinum |
4 |
4 |
4 |
5 |
4 |
5 |
4 |
5 |
4 |
3 |
4 |
4 |
4 |
5 |
4 |
|
Catalpa bignonioides |
3 |
3 |
3 |
5 |
3 |
4 |
3 |
4 |
4 |
3 |
3 |
3 |
3 |
4 |
3 |
|
Cercis cadensis |
1 |
2 |
2 |
3 |
2 |
3 |
2 |
3 |
2 |
2 |
2 |
2 |
3 |
2 |
2 |
|
Liquidambar styraciflua |
2 |
2 |
2 |
3 |
2 |
3 |
2 |
3 |
2 |
1 |
2 |
2 |
2 |
2 |
3 |
|
Pinus palustris |
2 |
2 |
2 |
2 |
2 |
1 |
2 |
2 |
1 |
1 |
2 |
2 |
1 |
4 |
2 |
|
Prunus persica |
3 |
3 |
3 |
5 |
3 |
3 |
3 |
4 |
3 |
4 |
3 |
3 |
4 |
3 |
4 |
|
Carya illinoensis |
1 |
3 |
2 |
2 |
2 |
2 |
2 |
3 |
2 |
2 |
2 |
2 |
1 |
3 |
2 |
|
Fraxinus Americana |
1 |
2 |
2 |
3 |
2 |
2 |
2 |
3 |
2 |
3 |
2 |
1 |
2 |
2 |
2 |
|
Liriodendron tulipifera |
1 |
2 |
2 |
2 |
2 |
1 |
2 |
2 |
1 |
1 |
1 |
1 |
1 |
2 |
1 |
|
Quercus marilandica |
3 |
3 |
3 |
5 |
3 |
5 |
3 |
3 |
4 |
3 |
3 |
4 |
4 |
3 |
3 |
|
Quercus stellata |
2 |
2 |
3 |
3 |
2 |
3 |
2 |
3 |
3 |
3 |
2 |
2 |
2 |
3 |
3 |
|
Salix babylonica |
4 |
4 |
3 |
3 |
3 |
4 |
3 |
3 |
3 |
4 |
4 |
3 |
4 |
4 |
3 |
|
Sassafras albidum |
2 |
2 |
2 |
4 |
2 |
2 |
2 |
3 |
2 |
1 |
2 |
2 |
3 |
2 |
3 |
|
Cornus florida |
1 |
2 |
2 |
2 |
2 |
2 |
2 |
3 |
2 |
2 |
2 |
1 |
1 |
1 |
1 |
|
Pyrus calleryana |
2 |
2 |
2 |
2 |
4 |
2 |
2 |
2 |
3 |
3 |
2 |
3 |
2 |
3 |
3 |
|
Lagerstroemia indica |
1 |
2 |
2 |
2 |
2 |
2 |
2 |
3 |
1 |
2 |
1 |
2 |
1 |
3 |
2 |
|
Diospyros virginiana |
2 |
3 |
3 |
4 |
3 |
2 |
3 |
4 |
3 |
3 |
3 |
3 |
2 |
3 |
4 |
|
Melia azedarach |
3 |
4 |
3 |
5 |
3 |
4 |
3 |
4 |
4 |
3 |
3 |
3 |
4 |
4 |
4 |
|
Quercus laurifolia |
2 |
2 |
2 |
3 |
3 |
1 |
2 |
2 |
1 |
1 |
2 |
2 |
3 |
2 |
2 |
|
Robinia pseudoacacia |
4 |
4 |
3 |
5 |
3 |
5 |
3 |
3 |
5 |
4 |
3 |
3 |
3 |
3 |
3 |
|
Fraxinus pennsylvanica |
2 |
1 |
2 |
4 |
2 |
1 |
2 |
3 |
4 |
2 |
2 |
1 |
2 |
2 |
2 |
|
Cryptomeria japonica |
2 |
2 |
2 |
3 |
. |
. |
2 |
3 |
2 |
3 |
3 |
2 |
. |
3 |
1 |
|
Quercus phellos |
2 |
2 |
2 |
1 |
3 |
2 |
2 |
1 |
1 |
1 |
2 |
2 |
1 |
3 |
2 |
|
Juglans nigra |
2 |
3 |
2 |
2 |
2 |
3 |
2 |
3 |
1 |
1 |
2 |
2 |
2 |
3 |
1 |
|
Platanus occidentalis |
2 |
3 |
2 |
3 |
2 |
4 |
3 |
3 |
2 |
1 |
2 |
3 |
2 |
3 |
2 |
|
Ginkgo biloba |
2 |
1 |
1 |
2 |
2 |
1 |
2 |
2 |
4 |
2 |
2 |
2 |
3 |
3 |
1 |
|
Quercus nigra |
3 |
2 |
2 |
4 |
3 |
4 |
3 |
3 |
1 |
3 |
2 |
2 |
3 |
3 |
2 |
|
Populus heterophylla |
2 |
3 |
3 |
5 |
3 |
4 |
4 |
4 |
4 |
2 |
3 |
3 |
. |
5 |
3 |
Notes: |
|||||||||||||||
Agreement Among Professions
Each species was assigned to a class based on the rating by each expert. In the Delphi rounds, the degree of agreement among the three professional groups as measured by W (Tables 2 - 5). Overall, W was 0.55. Yet, within groups, the highest degree of agreement was obtained from the horticulturists, (W = 0.77), followed by the foresters (W = 0.60) and arborists/landscape architect (W = 0.48). A highly significant value of W might also imply that the experts (within the same professional groups) applied similar standards in assigning value to the species under study. These standards as listed in the instructions provided them from the CTLA Guidelines.
|
Professional Group |
W |
p |
|---|---|---|
|
Foresters |
0.39 |
0.001 |
|
Extension Horticulturists |
0.35 |
0.001 |
|
Arborists/Landscape Architect |
0.48 |
0.001 |
|
Overall |
0.30 |
0.001 |
|
Professional Group |
Class 1 (100%) |
Class 2 (80%) |
Class 3 (60%) |
Class 4 (40%) |
Class 5 (20%) |
|---|---|---|---|---|---|
|
Foresters |
0.21 |
-0.02 |
0.01 |
0.09 |
0.13 |
|
Extension Horticulturists |
0.14 |
-0.05 |
0.003 |
-0.21 |
0.13 |
|
Arborists/Landscape Architect |
0.08 |
0.03 |
0.03 |
0.19 |
-0.04 |
|
Overall |
0.18 |
0.02 |
0.02 |
0.04 |
0.14 |
|
Professional Group |
W |
p |
|---|---|---|
|
Foresters |
0.60 |
0.001 |
|
Horticulturists |
0.77 |
0.001 |
|
Arborists/Landscape Architect |
0.60 |
0.001 |
|
Overall |
0.55 |
0.001 |
|
Professional Group |
Class 1 (100%) |
Class 2 (80%) |
Class 3 (60%) |
Class 4 (40%) |
Class 5 (20%) |
|---|---|---|---|---|---|
|
Foresters |
0.52 |
0.29 |
0.24 |
0.25 |
0.06 |
|
Horticulturists |
0.55 |
0.36 |
0.26 |
0.21 |
0.15 |
|
Arborists/Landscape Architect |
0.55 |
0.22 |
0.20 |
0.09 |
0.32 |
|
Overall |
0.54 |
0.36 |
0.27 |
0.25 |
0.16 |
Another striking aspect is the degree of agreement indicated by the Kappa statistic (K) (Table 5). K measures the quality of agreement among the groups as to the species assigned to each of the five classes. With K= 0.54, there is moderate agreement among the professional groups over the species assigned to Class 1. However, there is only a fair agreement among the groups over the species assigned to Classes 2, 3, and 4, with K = 0.36, K = 0.27, and K = 0.25, respectively. The poor agreement for Class 5 (K = 0.16) reflects strong differences of opinion on what species should be included in this category/class.
Testing W
The mean difference in W between rounds is positive (0.19), and the probability of the difference occurring by chance is 0.0218, assuming pooled and equal variances, and 0.0271 under unequal variances. The improvement in W between round 1 and round 2 of the Delphi was highly significant (p = 0.0284).
Comparison Between Species Rating from Delphi and the Guide
Table 6 shows the species ratings of the ISA Louisiana trees by each professional group and the Guide. The mean species rating [2.88, (0.1857)] based on the Guide was compared with the mean species rating obtained from each professional group. The results (Table 7) show that the mean species ratings obtained through our method were significantly lower than those provided in the Guide. This may be a reflection of the appreciation of the local values of species compared with species values at the regional level.
|
Scientific Name |
The Guide |
Foresters |
Horti-culturist |
Landscape Architect/ |
|---|---|---|---|---|
|
Acer negundo |
5 |
3 |
4 |
3 |
|
Acer palmatum |
3 |
1 |
2 |
2 |
|
Acer rubrum |
1 |
1 |
2 |
2 |
|
Acer saccharinum |
5 |
3 |
4 |
4 |
|
Carpinus carolinia |
3 |
2 |
2 |
3 |
|
Carya illinoensis |
2 |
2 |
3 |
2 |
|
Catalpa bignonioides |
5 |
3 |
3 |
2 |
|
Cedrus atlantica |
1 |
1 |
3 |
3 |
|
Cedrus deodara |
3 |
2 |
2 |
3 |
|
Celtis occidentalis |
3 |
2 |
3 |
4 |
|
Cercis cadensis |
5 |
2 |
2 |
2 |
|
Chionanthus virginicus |
3 |
1 |
2 |
2 |
|
Cornus florida |
1 |
1 |
2 |
2 |
|
Cryptomeria japonica |
4 |
3 |
2 |
2 |
|
Diospyros virginiana |
4 |
2 |
3 |
3 |
|
Fagus grandifolia |
1 |
1 |
2 |
2 |
|
Fraxinus pennsylvanica |
3 |
1 |
2 |
2 |
|
Ginkgo biloba |
1 |
2 |
2 |
2 |
|
Gleditsia triacanthos |
2 |
3 |
3 |
3 |
|
Halesia carolina |
3 |
1 |
2 |
2 |
|
Ilex opaca |
1 |
1 |
1 |
2 |
|
Juglans nigra |
2 |
2 |
3 |
2 |
|
Koelreuteria paniculata |
3 |
3 |
3 |
3 |
|
Lagerstroemia indica |
2 |
1 |
2 |
1 |
|
Liquidambar styraciflua |
3 |
2 |
3 |
2 |
|
Liriodendron tulipifera |
1 |
1 |
2 |
2 |
|
Maclura pomifera |
4 |
3 |
3 |
2 |
|
Magnolia acuminata |
2 |
2 |
2 |
2 |
|
Magnolia grandiflora |
1 |
1 |
1 |
1 |
|
Magnolia virginiana |
3 |
2 |
2 |
2 |
|
Melia azedarach |
5 |
3 |
4 |
3 |
|
Morus rubra |
5 |
2 |
3 |
3 |
|
Nyssa sylvatica |
2 |
2 |
3 |
3 |
|
Ostrya virginiana |
3 |
2 |
2 |
2 |
|
Oxydendrum arboreum |
3 |
1 |
2 |
2 |
|
Pinus echinata |
4 |
2 |
3 |
2 |
|
Pinus palustris |
1 |
1 |
3 |
2 |
|
Pinus taeda |
1 |
1 |
2 |
2 |
|
Platanus occidentalis |
3 |
3 |
2 |
2 |
|
Populus heterophylla |
5 |
2 |
4 |
3 |
|
Prunus caroliniana |
4 |
2 |
3 |
3 |
|
Prunus persica |
5 |
3 |
4 |
3 |
|
Pyrus calleryana |
2 |
3 |
2 |
2 |
|
Quercus alba |
1 |
1 |
1 |
2 |
|
Quercus falcate |
2 |
1 |
2 |
2 |
|
Quercus laurifolia |
2 |
2 |
2 |
2 |
|
Quercus lyrata |
1 |
1 |
2 |
2 |
|
Quercus marilandica |
4 |
3 |
3 |
3 |
|
Quercus nigra |
4 |
3 |
2 |
3 |
|
Quercus phellos |
1 |
2 |
1 |
2 |
|
Quercus shumardii |
1 |
1 |
1 |
1 |
|
Quercus stellata |
4 |
2 |
2 |
3 |
|
Quercus virginiana |
1 |
1 |
1 |
1 |
|
Robinia pseudoacacia |
5 |
2 |
3 |
3 |
|
Salix babylonica |
3 |
3 |
3 |
2 |
|
Salix nigra |
5 |
3 |
4 |
3 |
|
Sassafras albidum |
3 |
2 |
2 |
3 |
|
Taxodium distichum |
2 |
1 |
1 |
1 |
|
Ulmus pumila |
5 |
3 |
4 |
4 |
|
Professional Group |
Mean Rating (std error) |
p |
|
|---|---|---|---|
|
Foresters |
1.93 (0.1015) |
0.001 |
|
|
Extension Horticulturists |
2.44 (0.1109) |
0.0429 |
|
|
Arborists/Landscape Architect |
2.33 (0.0897 |
0.0079 |
|
Conclusions and Recommendations
In this article, our analysis reveals that the Delphi method as a means for developing species value ratings is workable. The good degree of agreement among members of different professional groups in rating individual landscape trees supports the reliability of the method for developing species rating. Committees for developing species rating can be constituted using Cooperative Extension members of the same or different tree or natural resource-related professions without fear that this will unduly influence the outcome of the species ratings.
However, in light of our findings we recommend the following to avoid some of the difficulties others might face in developing a species rating for a state:
- Carefully screen and select expert participants. Identify a pool of potential
experts from different groups interested in and/or users of the species rating,
who are both willing and have the time to participate. Require each expert
to submit a resume, including a description of previous experiences relative
to species rating, and familiarity with the Guide procedures for species
rating.
- The size of the panel should be manageable but reasonable. With large panels,
it is more difficult to achieve agreement; yet, groups too small may not
be representative panels.
- Analyze agreement of the experts before assigning individual species to their respective classes or rating values. The W and K criteria are effective for this purpose.
Overall, our experience in using the Delphi method demonstrates that some experts have limited knowledge of and familiarity with the species. This suggests that the selection of expert participants who have experienced species rating and who are also knowledgeable about the CTLA tree appraisal methods is of utmost importance.
The Delphi could be used to develop species rating more appropriate to conditions at local levels rather than to continue dependence on species rating based on regional level or larger like the Guide. The method could be easily replicated, allowing more flexibility and ready updates when special circumstances arise and not requiring waiting for long-term updates on the Guide. We hope that other Extension professionals can use this method in developing species rating in their respective areas.
Finally, the disparity between the Guide and the results of the Delphi may indicate the possibility of further effort to reconcile species values between those developed at state levels and those developed at the regional levels.
Acknowledgements
The authors wish to thank the following experts for their time and effort in the development of the species rating: Don Baker, Scott Courtright, Jim Culpepper, Severn Doughty, Dan Gill, Malcolm Guidry, Steve Hotard, Ricky Kilpatrick, Ken Maki, Lincoln Moore, Paul Orr, Jerry Robertson, Steve Shurtz, Carlos Smith, Jr., Robert Souvestre, Bob Thibodeaux, and Joe Walker White. We also would like to thank J. J. Muchovej for his constructive editorial suggestions.
References
Blanche, C. A., Guidry, M., & Wefel, M. (1997). Tree species rating for Louisiana: A novel and objective approach. Currently unavailable.
Dwyer, J., F., Nowak, D., et al. (2000). Connecting people with ecosystems in the 21st century: An assessment of our nation's urban forests. USDA Forest Service, Gen. Tech. Rep. PNW-GTR-490, 483.
Dalkey, N., & Helmer, O. (1963). An experimental application of the Delphi Method to the use of experts. Management Science 9(3), 458-467.
Landis, J. R., & Koch, G. G. (1977). The Measurement of observer agreement for categorical data. Biometrics 33, 159-174.
Lewis, C. E. (ed.). (1970). Shade tree evaluation. International Shade Tree Conference Inc., 43.
Ludwig, B. (1997). Predicting the future: Have you considered using the Delphi Methodology? Journal of Extension [On-line]. 35(5). Available at: http://www.joe.org/joe/1997october/tt2.html
Rodewald, A. (2002). Interagency collaboration on wildlife management issues: Opportunities and constraints. Journal of Extension [On-line]. 40(2). Available at: http://www.joe.org/joe/2002april/rb3.html
Watson, G. (2002). Comparing formula methods of tree appraisal. Journal of Arboriculture. 28(1), 11-18.