Post-Course Evaluation of a Grape Management Short Course

Introduction

The grape and wine industry has increased substantially in the United States (U.S.) in recent years, particularly in non-traditional areas of wine grape production. It has been estimated that the total economic impact of the U.S. grape industry is approximately $162 billion annually (MKF Research, 2007). Commercial grape production has expanded to every state in the continental U.S., and local wine industries have created significant agri-tourism opportunities that are making important contributions to rural economies.

Wine has an aura of urbane culture, and many wine grape producers are drawn from non-agrarian backgrounds. As a whole, the land-grant philosophy and original intentions of that philosophy have changed over time as the once agrarian-dominated society has become more urbanized. New growers are often undereducated in viticulture and enology practices. Unfortunately, this lack of agricultural education can result in large economic losses due to the high start-up costs associated with grape production (Stafne, 2007).

Most available information on viticulture has been generated using traditional grape production areas as a model; however, those regions may or may not have the same production conditions. Issues such as site selection and cultivar choice are crucial for success in any regions, and new growers are often ignorant of these critical aspects of optimal vineyard production. Oklahoma and other states where grape production is a growing industry have begun conducting courses to give current and prospective producers the basics needed to be successful, and evaluation of these courses is critical to determine impact. Several investigators have recently conducted similar studies examining the effectiveness of Extension programming (Higginbotham & Kirk, 2006; Blaine, Hall, Downer, & Ebert, 2008; Brown, Gibson, & Stewart, 2008; Meyer & Foord, 2008).

Currently, several states have "schools" or short courses that provide instruction in the basics of viticulture. Many new growers have no background in agriculture or horticulture, thus necessitating an introduction to topics of basic biology, plant pathology, entomology, and other related disciplines. A short course program is designed to prepare potential clients for a more advanced knowledge base that will not only prepare them for vineyard management, but also alleviate workload and utilize the expertise of university Extension scientists in a more efficient manner.

Grape Management Short Course Description

The Oklahoma State University (OSU) Grape Management Short Course (GMSC) is a Cooperative Extension-based course taught for seven to eight class periods, once per month for 4 hours, leading to a total of 28 to 32 contact hours. Attendance is capped at 70 students per year to maximize instructor-student interaction. Topics of instruction include: vineyard site selection, pruning and training, canopy management, disease and insect management, cultivar and rootstock selection, harvest considerations, economic feasibility, and various other related issues. Most lectures are delivered in an electronic slide presentation format, and students receive copies of all lecture material. Each student also receives a notebook filled with relevant educational materials from Oklahoma State University as well as other institutions. Students are encouraged to make use of the Oklahoma State University grape Web site <www.grapes.okstate.edu> to increase their knowledge base.

Post-Course Evaluation Methods

In December 2007, a post-course evaluation was sent to participants of the GMSC. Former students were contacted by e-mail and asked to complete the evaluation in the Internet-based tool SurveyMonkey via an embedded link, as this method had the advantage of reduced cost (O'Neill, 2004; Malone, Herbert, & Kuhar, 2005). A single follow-up e-mail was sent in early January 2008 asking potential respondents to finish the evaluation tool by January 31, 2008. A total of 306 participants who attended the course from 2001 to 2007 were asked 10 questions pertaining to their course experience. Fifty-eight e-mail addresses were returned as non-functional, thus 248 participants were contacted.

Responses were gathered and statistically analyzed in JMP version 7.0 (SAS Institute, Inc.) for one-way analysis of variance and Pearson product-moment correlations in the multivariate pairwise correlations procedure. Nonresponse error was handled as described by Miller and Smith (1983) and Lindner and Wingenbach (2002), where comparison of early to late respondents was done through using "days to respond" as a regression variable. "Days to respond" was treated as an independent variable to which major response variables where regressed (Lindner & Wingenbach, 2002).

Results and Discussion

A total of 70 evaluations were returned from the initial e-mail to 306 potential respondents, for a response rate of 28.2%. Reasons for lack of response are unknown, but could be postulated as previous researchers have done, including lack of experience with Internet evaluations (O'Neill, 2004) and security issues (Malone, Herbert, & Kuhar, 2005). Another possible reason is perceived lack of confidentiality by respondents (O'Neill, 2004). Even though responses were gathered anonymously, the grape industry in Oklahoma is very young, and growers may have suspicions concerning any data collected. It is possible that they believed information given might be shared with "competitors," thus potentially putting them at a competitive disadvantage. A final possibility is that students did not plant grapes or have yet to plant grapes.

Comparison of early to late respondents using the "days to respond" method yielded no statistically significant results, as indicated in Table 1, and therefore respondents and nonrespondents were considered not to have differed. Evaluation questions follow, with a shortened variable name in brackets for use in Table 2.

Table 1.
Nonresponse Error Regression Analysis Using "Days to Respond" as an Independent Variable

Variables	R2	Significance Probability (P<0.05)
Days to respond and Effective	0.0069	0.4923
Days to respond and Price	0.0158	0.3040
Days to respond and Impact	0.0138	0.3407
Days to respond and Money	0.0581	0.1064
Days to respond and Aware	0.0004	0.8678
Days to respond and Recommend	0.0233	0.2107

Question 1. What year(s) did you attend the OSU GMSC? [Year] (n=70)

The options included any year from 2001 through 2007. The responses were skewed toward more recent years (2005, 2006, and 2007; 68.1%). However, year was not a significant dependent variable, indicating that responses were similar over all years and thus pooled for analysis.

Question 2. Did you have grapes planted when you started the course? (n=70)

The options were Yes (1) or No (2). The number of No responses was predominant (75.7%). This result indicates that students were mainly in the beginning stages of exploring grape production while attending the course. It is also important in understanding the probable level of knowledge and practical experience a student had attained prior to the course.

Question 3. What year did you plant your vineyard? (n=63)

This was a follow-up to question 2. A total of 43 responses were given, with answers ranging from 1985 to 2008. The responses (81.4%) indicated that a majority of vineyards were planted within the last 5 years, which is consistent with estimates of recent industry growth (Stafne, 2007).

Question 4. How effective was the OSU GMSC in educating you about grape growing? [Effective] (n=70)

The options were Poor (1), Fair (2), Good (3), Great (4), and Excellent (5). No respondents gave Poor as a response. Only 7.1% said Fair, whereas in the remaining responses Great received the highest percentage (41.4%).

Question 5. In retrospect, was the OSU GMSC worth the price you paid to attend? [Price] (n=69)

The options were No (1), Could have been better (2), Satisfactory (3), A good deal (4), and Definitely worth it and more (5). The majority of respondents indicated that the GMSC was either A good deal (43.5%) or Definitely worth it (23.2%). No one replied that the GMSC was not worth the price. The price has varied over the years from $100 in the early years to the current price of $250 per person.

Question 6. How do you think the OSU GMSC impacted your grape growing efforts? [Impact] (n=68)

The options were Negatively (1), Could have been better (2), Not much impact (3), Positive (4), and Tremendous help (5). The majority responded Positive (63.2%), and another 25.0% responded Tremendously. Only 4.4% replied that the impact could have been better, and another 7.4% stated that the course did not have much impact.

Question 7. How do you believe the OSU GMSC impacted your grape growing operation financially? [Money] (n=46)

Respondents were given the option of replying that they saved money due to the course or lost money due to the course within five categories: $1 to $5,000, $5,000 to $10,000, $10,000 to $50,000, $50,000 to $100,000, and $100,000+. Forty-seven total respondents indicated that they saved money. Three respondents said they had lost money.

Categories were kept broad intentionally for two main reasons. A cost of establishing one acre of grapes is roughly $5,000 and many small-scale growers in Oklahoma do not keep track of exact figures. Most of those who saved money (74.5%) were in the $1 to $5,000 category. Another 17.0% indicated they saved between $5,000 and $10,000. Three respondents (6.3%) indicated that they had saved between $10,000 and $50,000, and one respondent (2.1%) stated that between $50,000 and $100,000 had been saved. The three who replied that they had lost money were in the $1 to $5,000 category. Respondents were also asked to provide an estimate of savings or loss, but few complied. The potential financial impact of the course to the grape growers of Oklahoma could thus be estimated between $120,032 (low) to $490,000 (high).

Question 8. What do you perceive as the biggest obstacle to overcome for successful grape production in Oklahoma? (n=68)

The options were Disease, Insects, Climate, Liquor laws, and Marketing. This question was an attempt to ascertain what students had gained from the course as well as to understand where their interest was focused (grape production vs. wine production). Not surprisingly, a good number of responders (38.2%) stated that liquor laws were an obstacle to successful grape production. While certainly a concern for the future of the industry, grapes can be successfully grown in Oklahoma regardless of liquor laws. Other factors (e.g., disease, insect, and climate) are of much more concern than liquor laws or marketing for production of grapes. An equal percentage of respondents (38.2%) indicated climate as the number one obstacle. Disease was third (13.2%), followed by marketing (7.4%), and insects (2.9%).

Question 9. Are you aware of other grape-related offerings by OSU? [Aware] (n=70)

The options were No (1), A few (2), Some (3), Most (4), and All (5). A large number of respondents indicated that they were aware of some or most of the grape-related materials from OSU (62.9%); however, another 30.0% said they knew of few or no offerings. The remaining 7.1% said they were aware of all OSU grape-related offerings.

Question 10. Would you recommend the OSU GMSC to others? [Recommend] (n=69)

The options were No (1), Unsure (2), Probably (3), Yes (4), and Strongly Yes (5). None of the responses indicated that they would not recommend the course. A small portion was unsure if they would recommend the course (2.9%). Most respondents indicated yes or strongly yes (84.1%), indicating that attendees were receiving the viticulture education that they required and expected from GMSC instructors.

Pearson product-moment pairwise correlations were performed to determine if relationships could be discerned to improve the course in the future (Table 2). Year of attendance was not significantly correlated to any variable, so students, regardless of when they attended the course, had similar thoughts about the education they received. The value of the price paid to attend the course was significantly correlated to the respondents' perceived effectiveness of the course as well as to the perceived impact on grape growing efforts. The perceived impact was significantly correlated to the perceived effectiveness of the course, as one would expect, and the amount of money a grower saved was also significantly correlated to perceived effectiveness and perceived impact.

Table 2.
Pairwise Correlations of Variables from 2001 to 2007 Attendees of the Oklahoma State University Grape Management Short Course

Variables	Correlation (r)z	Responses (n)	Significance Probability (P<0.05)
Effective and Year	0.1106	70	0.3622
Price and Year	0.0292	69	0.8117
Price and Effective	0.6628	69	<.0001*
Impact and Year	0.0704	68	0.5684
Impact and Effective	0.5329	68	<.0001*
Impact and Price	0.5683	67	<.0001*
Money and Year	0.0882	46	0.5600
Money and Effective	0.3192	46	0.0306*
Money and Price	0.1559	46	0.3009
Money and Impact	0.3076	45	0.0398*
Aware and Year	-0.0570	70	0.6394
Aware and Effective	0.2546	70	0.0334*
Aware and Price	0.2198	69	0.0696
Aware and Impact	0.1746	68	0.1544
Aware and Money	0.0191	46	0.8996
Recommend and Year	0.0619	69	0.6136
Recommend and Effective	0.6907	69	<.0001*
Recommend and Price	0.7157	68	<.0001*
Recommend and Impact	0.5932	67	<.0001*
Recommend and Money	0.2955	45	0.0487*
Recommend and Aware	0.2542	69	0.0351*
zDavis' Descriptors (Davis, 1971), where degree of relationship is none = 0.00-0.09, low = 0.10-0.29, moderate = 0.30-0.49, substantial 0.50-0.69, high 0.70-0.89, and very high 0.90-1.00.

Awareness of OSU programming correlated to perceived effectiveness, but not price paid, perceived impact, or money saved. Recommendation of the course correlated with perceived effectiveness, price paid, perceived impact, money saved, and awareness of OSU programs.

A one-way analysis of variance was conducted for responses from questions 2, 3, and 8, which had discrete rather than continuous variables. Whether a student had already planted vines or not was not significantly related to any other variable. The only significant effect related to year of planting was that of money saved (P = 0.0027). More recent attendees believed they had saved more money than less recent attendees.

Possible reasons for this result may be related to the curriculum evolution since the inception of the course or possibly the change in student demographic. Earlier courses often taught students who had already established their vineyards, whereas most attendees in recent courses had not yet planted grapes. Although not statistically significant (F = 2.1951;P = 0.1456), students who stated they had not yet planted before attending the course indicated that they saved more money than those who had already planted ($13,387 vs. $5,333). Students often decide not to plant grapes at all after taking the course or greatly reduce the vineyard size. These decisions reduce initial expenditures, as installation for one acre of vineyard can cost more than $12,000 (Poling, 2007).

In terms of obstacles, the only significant relationship was that of obstacle and perceived impact (P = 0.0490), with those who indicated Insect as an obstacle reporting the least perceived impact, whereas those that responded Climate as an obstacle had the highest perceived impact (although not statistically different from Disease, Liquor laws, or Marketing). Insects can be detrimental but are often easily controlled. The responses that indicated Insect as the biggest obstacle to grape production may signify an incomplete understanding of how grapes can be produced in Oklahoma, thus likely leading to the lack of perceived impact.

Conclusions

The results from the evaluation indicate a large, positive financial impact to grape producers and potential grape producers within Oklahoma. High recommendation levels also indicate the course is meeting the educational needs of beginning grape growers; however, many growers are unaware of other Oklahoma State University Cooperative Extension grape-related programmatic materials, such as a Web site, newsletter, fact sheets, research publications, and workshops. Therefore, a more directed effort must be made to heighten the awareness of new and established grape producers regarding the grape-related educational efforts of Oklahoma State University Cooperative Extension.

References

Blaine, T. W., Hall, F. R., Downer, R. A., & Ebert, T. (2008). An assessment of agricultural producers' attitudes and practices concerning pesticide spray drift: implications for Extension education. Journal of Extension [On-line], 46(4) Article 4FEA8. Available at: http://www.joe.org/joe/2008august/a8.php

Brown, A. S., Gibson, J. D., & Stewart, D. L. (2008). An assessment of Virginia cooperative Extension's new Extension agent training program. Journal of Extension [On-line], 46(4) Article 4FEA7. Available at: http://www.joe.org/joe/2008august/a7.php

Davis, J .A. (1971). Elementary survey analysis. Englewood Cliffs, NJ. Prentice Hall.

Higginbotham, G. E., & Kirk, J. H. (2006). Survey results from participants of a short course for dairy herdsman. Journal of Extension [On-line], 44(2) Article 2RIB4. Available at: http://www.joe.org/joe/2006april/rb4.php

Lindner, J. R., & Wingenbach, G. J. (2002). Communicating the handling of nonresponse error in Journal of Extension Research in Brief articles. Journal of Extension [On-line], 40(6) Article 6RIB1. Available at: http://www.joe.org/joe/2002december/rb1.php

Malone, S., Herbert, Jr., D. A., & Kuhar, T. P. (2005). An on-line survey process for assessing impact of an email-delivered pest advisory. Journal of Extension [On-line], 43(5) Article 5RIB2. Available at: http://www.joe.org/joe/2005october/rb2.php

Meyer, M., & Foord, K. (2008). Consumer preferences and perceptions of gardening information. HortTechnology 18(1):162-167.

Miller, L. E. & Smith, K. L. (1983). Handling nonresponse issues. Journal of Extension [On-line], 21(5). Available at: http://www.joe.org/joe/1983september/83-5-a7.pdf

MKF Research. (2007). The impact of wine, grape and grape products on the American economy 2007: Family business building value. Retrieved September 16, 2008 from: http://www.ngwi.org

O'Neill, B. (2004). Collecting research data online: implications for Extension professionals. Journal of Extension [On-line], 42(3) Article 3TOT1. Available at: http://www.joe.org/joe/2004june/tt1.php

Poling, E. B. (2007). The North Carolina winegrape grower's guide. North Carolina Cooperative Extension Service publication AG-535.

Stafne, E. T. (2007). Profile and challenges of the emerging Oklahoma grape industry. Oklahoma Cooperative Extension Service Circular E-999. Retrieved June 16, 2009, from: http://www.grapes.okstate.edu/PDFs/ProfilesChallengesEmergingGrapeIndustryE999.pdf