Colorado's AgrAbility Project's Effects on KASA and Practice Changes with Agricultural Producers and Professionals

Introduction

Disability rates resulting from work-related injuries remain steadily high among farmers and ranchers (Deboy, Jones, Field, Metcalf, & Tormoehlen, 2008; Field & Jones, 2006). The National AG Safety Database reported that approximately 288,000 farmers and ranchers with one or multiple disabilities work in U.S. agriculture (Willkomm, 2001). In conjunction with this population being highly susceptible to becoming disabled, there is the added vulnerability of being underserved by health service groups (Schweitzer, Deboy, Jones, & Field, 2011). Some of the reasons for the lack of health services within this population are in part due to transportation and geographical limitations, low financial budgets towards health care, and negative perceptions of health care services (Schweitzer et al., 2011). This lack of services has cascading effects of increasing the propensity for mental and behavioral health issues and in turn the likelihood of obtaining a subsequent injury.

As a means of addressing this gap in services, USDA implemented AgrAbility nationally. Specifically, AgrAbility is a United States initiative whereby Extension and non-profit professionals provide information, education, and service to individuals with disabilities within the agricultural population. For instance, individuals with physical disabilities are provided with ideas for assistive technology and machinery modifications to gain or regain independence. Participating individuals are also offered workshops and programs designed to address how one can manage stressful situations and other mental/behavioral issues. A common means of program evaluation within Extension is the use of Bennett's hierarchical model (Bennett, 1975, 1976; Rockwell & Bennett, 2004).

Bennett's Hierarchical Model

Bennett (1976) stated that in order to properly evaluate an Extension program there should be an implementation of a hierarchical model. This model incorporates the assessment of inputs (e.g., time, money, and investments), which should provide valuable activities (e.g., workshops) and desirable reactions from participants. One key assessment of the program evaluation involves understanding how likely an individual will change their knowledge, attitude, skills, and aspirations (KASA) upon completion of a program (Bennett, 1975, 1976). In the Targeting Outcomes of Programs (TOP) model, changes in KASA have been theorized to occur and be applied in order for practice change to occur (Rockwell & Bennett, 2004). Studies have highlighted the usefulness of assessing KASA and practice changes as a means of understanding the usefulness of an Extension program and the response of other change strategies such as needs assessments (Harder & Strong, 2010).

We reviewed the literature to find what was different about those programs that were most successful at meeting multiple objectives, such as increased knowledge, improved attitudes, and improved behavioral changes. Those that were more successful in meeting multiple goals tended to have multiple meeting times (generally at least two) and used multiple learning styles (e.g., presentation styles, participatory exercises) (see Fenwick, Vassilas, Carter, & Haque, 2004; Moffat & Tung, 2004; Saketkoo, Anderson, Rice, Rogan, & Lazarus, 2004). Those workshops used teaching modalities like lecture, video vignettes, case studies, experiential learning, problem-based learning, didactic presentation, handouts, overheads, question-and-answer periods, group activities, participation, support, and discussion.

Many Extension programs have been evaluated using Bennett's (1975, 1976) hierarchical model. Workman and Scheer (2012) conducted a meta-analysis, assessing how many program evaluations used each hierarchy in Bennett's (1976) hierarchical model. Of the 302 program evaluations assessed, 90% examined KASA change, and 82% examined practice change. Workman and Scheer highlighted that although a program's true impact goes beyond the assessment of process outcomes, assessment of KASA and practice changes are keys in the overall assessment of a program and its impact on the participant and society as a whole.

Purpose

Using a KASA process outcome, the primary purpose of the current study was to report descriptively the influence that participating in an AgrAbility workshop changed professionals' and farmers/ranchers with disabilities' knowledge, attitudes, aspirations, behaviors, and practice changes. Because community professionals (i.e., Division of Vocational Rehabilitation Counselors, occupational therapists, physical therapists, Emergency Medical Technicians, chiropractors, county health nurses, community college health professionals, and Extension agents) play key roles in the dissemination and facilitation of Extension educational programs (Lockett, Moore, & Wingenbach, 2014), the second purpose of the study was to compare the responses of the farm/ranch participants with those of professionals to see if one group benefitted more.

Method

Participants

Participants included farm and ranch families with disabilities (n = 401 individuals) who were recruited along with AgrAbility professionals (n = 401) who work with and want to learn ways of better assisting farm and ranch families with disabilities from the Colorado AgrAbility Project (CAP). CAP is one of 20 state or regional projects currently funded to provide information, education, and service to farmers and ranchers with disabilities and professionals who work with them. The study focuses on a 16-year project, starting in 1998. Because our overall goal was to reach new people annually and increase the number of new clients, longitudinal impact was not evaluated. Instead, we focused on the pooled sample across all 16 years.

Approximately 226 (56.4%) of the farmers and ranchers with disabilities were male. Age ranged from under 19 to over 70, with a mean age group between 51 and 60 years. Our sample was predominantly Caucasian (90%). It was an educated sample, with 35.2% reporting some college/technical school and 35.7% reporting being college graduates or higher. The average number of farmers and ranchers participating in each workshop was 5.2; the range was 1-13.

Of the 401 professionals who participated in the study, 260 (64.8%) were female. The age ranged from under 19 to over 70, with a mean age group of 41-50 years old. The sample was Caucasian (78.6%) and Hispanic (13.5%). The majority of professionals were college graduates or higher (77.1%). The average number of professionals participating in each workshop was 6.1; the range was 1-18.

Materials and Measures

Knowledge and aspirations were assessed using the 13-item AgrAbility Participant Satisfaction Survey Help Us Help You (HUHY)1 (Fetsch, 2013a) for the post-test survey immediately following the workshops. Knowledge, attitudes, behaviors, and practice changes were assessed via the 13-item AgrAbility Program Evaluation Survey Help Us Help You (HUHY)2 (Fetsch, 2013b) for the medium-range follow-up survey. Both surveys are self-reports. A sample item for HUHY1 included "As a direct result of participating in the AgrAbility program, my knowledge increased" from very little (1) to very much (9). A sample item for HUHY2 included "As a direct result of participating in this program, I have done something differently that I consider an improvement" from very little (1) to very much (9). Because CAP is a federally funded project, satisfaction levels with AgrAbility and tax dollar support levels were also assessed both times using HUHY1 and HUHY2 via the following dichotomous item (Fetsch, 1996): "This program was supported either totally or in part by your tax dollars. Do you want your tax dollars to continue supporting this type of program? Yes or No."

Procedures and Analyses

Each year for 16 years, an Extension state specialist and agents collaborated to market, present, and evaluate between three and 11 3-hour workshops using multiple learning styles and teaching modalities (i.e., lecture, case studies, Power Point presentations, success stories, and discussion) as delivery methods with both farmers and ranchers with disabilities as well as professionals (Fetsch, 2014). The educational workshops were guided by current research-based information to inform farmers and ranchers with disabilities about cutting-edge assistive technology and adaptations that agricultural producers could fabricate or purchase to increase their independence and reduce stress levels. In addition, unique information was provided to address the current needs of agricultural producers and professionals.

The CAP increased awareness of AgrAbility by providing 150 educational workshops across Colorado. Each of the 16 years CAP worked closely with three-10 Colorado State University Extension (CSUE) agents who marketed the program to ranchers and farmers with disabilities and to local professionals who worked with them. CAP and the Extension agents distributed 5,000+ flyers via direct mailings, wrote news releases, aired radio spots, and made personal contacts with agricultural families who could benefit from AgrAbility.

Surveys were administered by the first author, who was not involved with directly providing AgrAbility services. Participation was voluntary, and University Institutional Review Board procedures were followed related to informed consent and confidentiality. All participants took part in the intervention workshop. Baseline data was collected immediately following the workshops. Follow-up surveys with stamped return envelopes were mailed to participants on average 2.8 months following the workshop (range = 2-33 months), with a second mailing to non-respondents 3-4 weeks later.

In order to assess KASA and practice changes of the sample, a series of independent t test analyses were performed.

Results

Over 16 years (1998-2013), the number of farm/ranch workshop participants increased from 12 to 45 per year, and the number of professionals increased from 16 to 75 per year. The first objective of the study was to report descriptively the influence that participating in an AgrAbility workshop changed professionals' and farmers' and ranchers' with disabilities knowledge, attitudes, aspirations, behaviors, and practice changes. Immediately following the workshop, almost all of the participants reported having increased knowledge, attitudes, and satisfaction with AgrAbility (Table 1).

Table 1.
requency of Immediate Outcomes from 401 Ranchers/Farmers and 401 Professionals

Outcomes	% Ranchers/Farmers	% Professionals
Reported increased knowledge	99.8	98.8
Reported increased satisfaction levels with AgrAbility	97.5	99.0
Plan to use the information	97.8	99.3
Want their tax dollars to continue supporting AgrAbility	94.3	94.8

Medium-range follow-up results were received approximately four months following the workshops (M = 114.4 days; SD = 54.04 days; N = 496). Of those who returned their follow-up surveys, high percentages reported increased knowledge, improved attitudes and behaviors, satisfaction levels, and tax dollar support levels (see Table 2).

Table 2.
Frequency of Follow-up Outcomes from 277 Farmers/Ranchers and 386 Professionals

Outcomes	% Ranchers/Farmers	% Professionals
Reported increased knowledge	94.6	98.2
Reported improved attitude/outlook	89.5	94.8
Reported doing something to improve their lives	84.5	77.2
Reported increased satisfaction levels with AgrAbility	91.3	93.0
Want their tax dollars to continue supporting AgrAbility	94.2	95.9

In order to test the second objective, which assessed group differences in KASA and practice changes between the farmers and ranchers with disabilities and those with professionals, we conducted several independent sample t tests. Table 3 shows that ranchers and farmers with disabilities rated the educators significantly higher (M = 8.01) than did professionals (M = 7.85). The effect size d is approximately .17, which is a somewhat smaller than typical size for effects in the behavioral sciences. Farmers and ranchers did not differ significantly from professionals on knowledge increased immediately (p = .243), on their plans to use the information (p = .057), on their satisfaction levels with AgrAbility (p = .364), nor on their tax dollar support levels (p = .083). When the results are not significantly different, then there is no point in calculating the size effect (d).

On their follow-up surveys, results revealed that professionals' average knowledge levels were significantly higher than those of farmers and ranchers with disabilities (p = .046). The effect size d is approximately .16, which is a small or smaller than typical size for effects. Attitudes improved were also statistically significantly higher among professionals than farmers and ranchers with disabilities (p = .006). The effect size d is approximately .23, which is a small or smaller than typical size for effects in the behavioral sciences. Professionals did not differ significantly from ranchers and farmers with disabilities on behaviors improved (p = .222) nor on satisfaction levels increased (p = .507; see Table 3).

Table 3.
Comparison of Farmers and Ranchers with Disabilities with Professionals on Immediate and Follow-up Surveys

Variable	M	SD	n	df	t	p	d
Immediate Surveys.
Rate educator				788	2.28	.023	.17
Farmers	8.01	1.01	390
Professionals	7.85	0.93	400
Knowledge increased	1			798.11	-1.17	.243
Farmers	7.88	1.17	401	400
Professionals	7.98	1.13
Plan to use information				798	1.91	.057
Farmers	7.82	1.31	399
Professionals	7.64	1.27	401
Satisfaction increased	1			789	1.91	.364
Farmers	8.06	1.05	392
Professionals	7.99	1.08	399
1 The t and df were adjusted because variances were not equal.

Discussion

In the present Extension educational program evaluation study, high percentages of farmers and ranchers with disabilities and professionals who work with them reported high KASA levels as well as had a high desire for continued tax dollar support for AgrAbility. Consistent with previous literature (e.g., Holland, Greenberg, Tidwell, & Newcomer, 2003; Hughes, Nosek, Howland, Groff, & Mullen, 2003), high levels of behavioral changes (77%-84%) were reported approximately 4 months following participation in AgrAbility workshops. The findings of the present study build on and support the findings of the other empirical evaluations of the AgrAbility Project in the literature (Meyer & Fetsch, 2006).

Farmers and ranchers with disabilities reported overall lower KASA and tax dollar support levels than professionals on both their immediate and follow-up surveys. In particular, we observed significant differences in knowledge and attitudes on the medium range follow-up surveys between farmers and ranchers with disabilities and professionals. One explanation for this difference could be attributed to differences in sample characteristics (i.e., gender, age, educational level). The ranchers and farmers with disabilities were predominantly male, Caucasian, and less educated than the professionals. It is possible that these characteristics contributed to the differences observed between the two groups. In addition, farmers and ranchers tend to be independent and to think pragmatically in devising quick solutions to concrete problems to improving their functionality as well as economic efficiency of their farms and ranches (Willock et al., 1999). It may be more challenging for farmers and ranchers to readily adopt and use some of the knowledge gained and attitudes improved by the AgrAbility workshops. Professionals, however, may be receptive and willing to apply readily the information received in the workshops because they can use strategies to address their clients' needs. However, more work is needed to delineate such differences that contribute to KASA and practice changes.

Our AgrAbility Project educational programs with ranchers/farmers with disabilities and professionals who work with them showed some improvement in participants' knowledge, and the knowledge they gained was retained for up to approximately 4 months. Both groups reported similar levels of improvements as a result of participating in the AgrAbility educational workshops. It appears to contribute to attitudinal and behavioral changes and impacts. A limitation of the present study is the absence of a control group. Without a control group it is very difficult to analyze the workshop's true effect in initiating and sustaining KASA and practice changes. Another limitation is the use of a two post-test survey design. While a pretest-posttest design is common in assessing changes, we opted to use two post-test surveys because some studies have shown retrospective designs to correct the pretest-posttest limitation (Rockwell & Kohn, 1989).

The study underscores the need for Extension and AgrAbility to continue working towards improving KASA and behaviors of farmers and ranchers with disabilities as well as professionals. Even though KASA and practice change levels decreased for professionals as well as farmers and ranchers with disabilities, 90%-98% of participants reported KASA improvements 4 months afterwards. In addition, the study showed that similar workshops can be implemented for both professionals and farmers and ranchers with disabilities. Future researchers would do well to consider integrating KASA evaluation of their Extension programs from the beginning.

Implications for Other Extension Programs

By working collaboratively throughout the program design, development, delivery, and evaluation process, Extension agents and state specialists can document KASA and practice changes. They can test empirically participants' self-reported changes and compare different groups' results both immediately following the workshops and months later. Reporting results can increase public awareness of the benefits participants gain from taking part in well-designed Extension educational programs. Understanding how likely individuals are to change their knowledge, attitudes, skills, and aspirations (KASA) is key to the overall assessment and its impact on participants and society.

Acknowledgement

A preliminary version of this article was presented at the National Training Workshop of the National AgrAbility Project, Charleston, WV on October 13, 2010. The authors gratefully acknowledge Leah Beamer, Rosie Gomez, and David Haralson who are graduates of the Department of Human Development & Family Studies at Colorado State University for their preliminary work on the literature review. This research was supported by grants from the U.S. Department of Agriculture, National Institute of Food and Agriculture 2010-41590-20750, 2006-41590-03405, 2002-41590-01350, and 1998-EDFA-1-0267. We acknowledge and appreciate the good work that the following Colorado State University Cooperative Extension Agents and another professional did to promote and market the Colorado AgrAbility Project workshops over 16 years (1998-2013): Tom Hooten, Dale Edwards, Marvin Reynolds, Dean Oatman, Bruce Bosley, Dennis Kaan, Myong Frye, Bill Hancock, Eleanor West, Sheila Gains, Luann Boyer, Jerry Alldredge, Bill Ekstrom, Elisa Shackleton, Debbie Alpe, Ernie Marx, Robert Mathis, Tom McBride, Yvette Henson, C. J. Mucklow, Jan Nixon, Kenny Smith, Carolyn Valdez, Jane Crow, Todd Hagenbuch, Frank Sobolik, Tommy Covington, Pat McCarty, Wendy Rice, Kim Fabrizius, Bruce Fickenscher, Robbie Baird LeValley, Janet Sennhenn, Kurt Jones, Janet Benavente, Gale Loeffler, David Colburn, Jim Smith, Karen Crumbaker, and Bob Hamblen.

References

Bennett, C. (1975). Up the hierarchy. Journal of Extension [On-line], 13(2). Available at: http://www.joe.org/joe/1975march/1975-2-a1.pdf

Bennett, C. F. (1976). Analyzing impacts of extension programs. Extension service work: (Adobe Digital Edition version). Retrieved from: https://ia800909.us.archive.org/34/items/analyzingimpacts57benn/analyzingimpacts57benn.pdf.

Deboy, G. R., Jones, P. J., Field, W., Metcalf, M., & Tormoehlen, R. L. (2008). Estimating the prevalence of disability within the U.S. farm and ranch population. Journal of Agromedicine, 13, 175-190. doi: 10.1080/10599240802371763

Fenwick, C. G., Vassilas, C. A., Carter, H., & Haque, M. S. (2004). Training health professionals in the recognition, assessment and management of suicide risk. International Journal of Psychiatry in Clinical Practice, 8, 117-121.

Fetsch, R. J. (1996). Tax dollar support levels are strong for family living programs. Journal of Extension [On-line], 34(2) Article 2IAW1. Available at: http://www.joe.org/joe/1996april/iw1.php

Fetsch, R. J. (2013a). AgrAbility Participant Satisfaction Survey: HUHY1-Help Us Help You. (Available from the primary author, Department of Human Development and Family Studies, Colorado State University, Fort Collins, CO 80523-1570).

Fetsch, R. J. (2013b). AgrAbility Participant Satisfaction Survey: HUHY2-help us help you. (Available from the primary author, Department of Human Development and Family Studies, Colorado State University, Fort Collins, CO 80523-1570).

Fetsch, R. J. (2014). AgrAbility farm/ranch adaptations [PowerPoint slides]. (Available from primary author, Department of Human Development & Family Studies, Colorado State University, Fort Collins, CO 80523-1570).

Field, W., & Jones, P. (2006). Disability in agriculture. In J. E. Lessenger (Ed.) Agricultural medicine: A practical guide. (pp. 70-80). Springer: NY. doi: 10.1007/0-387-30105-4_7

Harder, A., & Strong, S. (2010). An analysis of outcomes associated with conducting county program reviews in Cooperative Extension. Journal of Southern Agricultural Education Research, 60, 79-89.

Holland, S. K., Greenberg, J., Tidwell, L., & Newcomer, R. (2003). Preventing disability through community-based health coaching. Journal of the American Geriatrics Society, 51, 265-269.

Hughes, R. B., Nosek, M. A., Howland, C. A., Groff, J. Y., & Mullen, P. D. (2003). Health promotion for women with physical disabilities: A pilot study. Rehabilitation Psychology, 48(3), 182-188.

Lockett, L., Moore, L., & Wingenbach, G. (2014). A global worldview among Extension professionals: A case study of best practices for study abroad programs. Journal of Extension [On-line], 52(4) Article 4FEA3. Available at: http://www.joe.org/joe/2014august/a3.php

Meyer, R. H., & Fetsch, R. J. (2006). National AgrAbility Project impact on farmers and ranchers with disabilities. Journal of Agricultural Safety and Health, 12(4), 275-291.

Moffat, J., & Tung, J-Y. (2004). Evaluating the effectiveness of culture brokering training to enhance cultural competence of independent living center staff. Journal of Vocational Rehabilitation, 20, 59-69.

Rockwell, K., & Bennett, C. (2004). Targeting outcomes of programs: A hierarchy for targeting outcomes and evaluating their achievement. Retrieved from: http://digitalcommons.unl.edu/aglecfacpub/48

Rockwell, S. K., & Kohn, H. (1989). Post-then-pre evaluation. Journal of Extension [On-line], 27(2) Article 2FEA5. Available at: http://www.joe.org/joe/1989summer/a5.php

Saketkoo, L., Anderson, D., Rice, J., Rogan, A., & Lazarus, C. J. (2004). Effects of a disability awareness and skills training workshop on senior medical students as assessed with self-ratings and performance on a standardized patient case. Teaching and Learning in Medicine, 16, 345-354.

Schweitzer, R. A., Deboy, G. R., Jones, P. J., & Field, W. E. (2011). AgrAbility mental/behavioral health for farm/ranch families with disabilities. Journal of Agromedicine, 16, 87-98. doi: 10.1080/1059924X.2011.554766

Willkomm, T. (2001). Farming and ranching with a disability. National AG Safety Database. Retrieved from: http://nasdonline.org/document/1832/d001767/farming-and-ranching-with-a-disability.html

Willock, J., Deary, I. J., McGregor, M. M., Sutherland, A., Dent, B., Grieve, R.,… Austin, E. (1999). Farmers' attitudes, objectives, behaviors, and personality traits: The Edinburgh study of decision making on farms. Journal of Vocational Behavior, 54, 5-36.

Workman, J. D., & Scheer, S. D. (2012). Evidence of impact: Examination of evaluation studies published in the Journal of Extension. Journal of Extension [On-line], 50(2) Article 2FEA1. Available at: http://www.joe.org/joe/2012april/a1.php