Examining a Professional Development System: A Comprehensive Needs Assessment Approach

Introduction

In today's rapidly changing world, knowledge is quickly outdated. An organization with knowledge development and education as its base needs to have processes in place to continually develop its intellectual capital (Van Buren, 2001). With the aging baby boomer population and increasing diversity of the workforce, the 21st century organization must be skilled at developing capacities of personnel.

Determining professional development needs in any organization is a challenging task--especially when there are 1,400 personnel in 100 locations! The task becomes more complex with the diversity of job responsibilities: conducting Extension programs, teaching, providing office support, and serving as technical subject matter resources.

Mincemoyer and Kelsey (1999) studied in-service attendance and satisfaction level in Pennsylvania State University's Cooperative Extension program. The top reasons cited by county-based faculty for not attending in-services were:

1. Previous commitments (56%);
2. Too much time away from the office (53%);
3. Conflict with local programming (45%);
4. Conflict with another in-service (43%);
5. Work/family conflicts (42%); and
6. In-service not relevant to program in county (41%).

Since district and state Extension specialists are a primary source of information for county agents, it is important that they understand their needs and be inclusive in their in-services (Radhakrishna & Thompson, 1996; Shih & Evans, 1991). State Extension faculty may not fully understand their role in the programming process, especially in developing resource materials, providing in-services, and in evaluating programs relative to timely issues identified as county needs (Baker & Villalobos, 1977).

Methodology

In-service training is an important component of professional development provided by Ohio State University Extension (OSU Extension). In autumn 2000, a team of OSU Extension professionals conducted a comprehensive needs assessment process using four instruments:

1. To determine factors impacting personnel participation in in-service opportunities.
2. To identify barriers coordinators faced in providing professional development and to identify support needed in this role.
3. To determine both technical subject matter and process skill developmental needs of program and support personnel.

Several expert panels established content and face validity for each of the research instruments. The instruments were field tested, and a reliability coefficient was calculated using Cronbach's alpha (r = .94). Software entitled "EventHandler" (GAP Consulting, 2000) was piloted with the research instruments. Because results of the field tests indicated the software worked best with short research instruments, mail surveys were used for both the program personnel and support staff. Multiple mail and e-mail reminders encouraged response to the electronic and mailed surveys. To control for non-response error (Miller & Smith, 1983), early respondents and late respondents were compared. No statistically significant differences were found between early and late respondents for the major variables of the study. This comparison was not possible with the electronically administered surveys.

Key Findings

Coordinator's Survey

Seventy-three (36%) coordinators responded, including state specialists, Extension associates, district specialists, program area team leaders, and administrators. Eighty-eight percent indicated they are expected to provide training as a part of their job description, with 81% coordinating at least one in-service per year. The primary target audience was Extension agents (87%). Though office support staff compose 28% of the workforce, they were targeted by only 16% of the coordinators.

Barriers commonly faced by in-service coordinators were:

• Too much competition for audiences (71%),
• Low attendance (34%),
• Inadequate facilities (26%),
• Too many responsibilities (25%), and
• Expenses of outside resource people (22%).

Coordinators cited assistance needed to be effective in providing in-service:

• Using electronic registration process (41%),
• Using new formats for in-services (40%),
• Support in implementing distance education strategies (39%), and
• Start-up funds (34%).

Coordinators most frequently evaluated training programs at a reaction level. Less than half (41%) reported evaluating at the skill or knowledge acquisition level. Over half indicated they would use, if provided, a standardized evaluation instrument with components to assess knowledge gained, intentions to apply knowledge, teaching effectiveness, and reactions of participants.

User's Views

Of approximately 1,400 Extension employees, one third responded to a survey examining the professional development infrastructure. Only 305 (22%) responses were usable due to problems administering the electronic survey that resulted in partially submitted data. Though the mean years of employment with Extension was 10.3 years, 20% of the respondents had worked 2.25 years or less. Respondents included Extension agents (31%), support staff (30%), and program assistants (19%). Twenty percent were state and district specialists, Extension associates, and state program coordinators.

Five primary barriers limiting participation in in-service education:

• Difficulty in taking time from job,
• Scheduling conflicts,
• Lack of in-services relevant to job,
• Too far to travel to in-services, and
• Too much time on the road.

During the past year, personnel spent a median of 4 days in technical knowledge/skills training at Extension-sponsored in-services and 2 days at in-services provided by other sources. They spent a median of 1 day in process skill training (communication, leadership, teamwork, conflict management, etc.) at Extension-sponsored in-services and had not attended similar trainings outside of Extension. Support staff and state administrative/professional (A&P) staff tend to attend statistically significant fewer OSU Extension in-services (knowledge/skills and process skills) than Extension agents do. Support staff attend statistically significant fewer outside in-services (knowledge/skills) than agent and state A&P staff.

Methods of Delivering Training

Do coordinators provide training in the way the users most prefer? Table 1 shows a comparison of the methods coordinators currently use to provide training compared to the users' preferences. With the exception of face-to-face delivery, coordinators were under-utilizing methods preferred by users.

Table 1:
Comparison of Methods Used to Deliver Training and Users' Preferences

Methods of Delivering In-Service	Coordinators' Current Use	Users' Preferences
Face to face Workshops/Seminars	93%	92%
Coaching/Mentoring	36%	Not on survey
Reading Materials	32%	44%
Telephone Coaching /Mentoring	29%	41%
E-mail Coaching/Mentoring	25%	53%
Study Tours	25%	Not on survey
Self-Instructional Materials	25%	37%
Conference Calls	23%	52%
Satellite Television	22%	65%
Web-Based Training	12%	69%
CD-ROM Based Training	7%	63%
E-mail Chat Rooms	7%	32%
Interactive Video	5%	54%
Videotapes	Not on survey	56%

Priority Training Needs

A Westinghouse model called GETNA (1997) was adapted for use in conjunction with Borich's model (Borich, 1980) to determine training needs. Competencies were grouped into process skills and subject matter areas. Process skills for all personnel included: communications, information and technology, personal/professional development, and leadership and management skills. In addition, the program personnel survey included program development and planning and evaluation. Subject matter areas were grouped by program area including: 4-H Youth Development (4-H), Family and Consumer Sciences (FCS), Agriculture and Natural Resources (ANR) and Community Development (CD). Respondents were instructed to complete all four process skill areas and only the subject matter skill areas relevant to their jobs. The support staff instrument included competency areas for support tasks and financial functions in addition to the process skill competencies.

Using a five-point Likert scale ranging from "0" not proficient/relevant to "4" extremely proficient/relevant, respondents indicated their perceived level of proficiency and relevance to job role for each competency. Discrepancy scores were calculated through the use of the Borich (1980) formula and used to identify priority needs. The possible range of scores was between -4 (lowest priority) and +16 (highest priority). The higher the number, the greater the need for training.

Fifty-nine percent (468) of OSU Extension employees with program responsibilities responded to the survey examining perceived relevance and proficiency on key competencies. Seventy-one percent of the respondents were field personnel, including Extension agents, program assistants, and nutrition educators. The remaining respondents were state specialists, Extension associates, and district specialists. Some of these respondents coordinated in-services, so responded to that instrument as well. Twenty-nine percent of the respondents had FCS as the primary program area assignment, followed by ANR (22%), 4-H (22%), and CD (8%). The majority of the respondents (75%) have worked for OSU Extension less than 17 years, with a mean of 10.8 years. Three hundred three (48%) support staff responded, the majority (75%) having worked for OSU Extension less than 14 years, with a mean of 9.33 years. Table 2 lists top 10 priority topics identified through a ranking of discrepancy scores.

Table 2.
Rank Order of Top 10 Topics Identified as Highest Professional Development Priorities
(Scale -4 to +16)

Topic	Mean Score Program Personnel	Rank	Mean Score Support Staff	Rank
Using Presentation Software	3.62	1	1.81	10
Planning for Retirement	3.56	2	3.30	1
Achieving Work/Life Balance	3.42	3	1.98	6*
Managing Stress	3.17	4	2.51	2
Working with Legislators, Community Leaders, and Funding Sources	2.80	5
Writing and Managing Grants	2.79	6
Developing Web Pages	2.78	7	1.98	7*
Facilitating Career Growth and Renewal	2.72	8
Using Computers for Program Development and Delivery	2.64	9
Documenting Teaching Effectiveness	2.59	10
Understanding University Benefits			2.30	3
Dealing with Difficult People/Situations			2.03	4
Maintaining a Positive Work Attitude			1.99	5
Understanding Staff Performance Review			1.96	8
Using Database/Spreadsheet Software			1.89	9
* Indicates tied ranking

Comparison of Priorities and Competency Area Needs for All Personnel

Items in the program personnel and support staff instruments were grouped into key competency areas. Rankings were calculated from 1 (Highest) to 5 (Lowest). For all personnel, the personal and professional development competency area was identified as the most needed, while technical subject matter competency areas (program personnel) and technical job knowledge areas (support staff) were least important. Five of the 10 highest priority needs were the same for all personnel (Table 2).

Discussion and Implications

The top four barriers to attending in-service training were consistent with results from Mincemoyer and Kelsey (1999):

• Difficulty taking time from job,
• Scheduling conflicts,
• In-services viewed as irrelevant to job, and
• Distance to training sites too far, requiring too much travel time.

Face-to-face approaches were the most frequently used delivery method for in-services and were preferred by most personnel. However, to address the barriers of time and travel to participate, other approaches are needed to deliver in-services. Personnel indicated overwhelmingly (90%) that they would be willing to participate in in-service education via distance education, yet few coordinators (22%) are currently using distance formats for delivery. Incentives are needed for coordinators to use a wider variety of methods. Training for coordinators and start-up funds could be used as incentives to support creative and futuristic delivery of in-services.

Budget constraints faced by Extension may mandate use of new delivery methods. Coordinators may overlook the total expense to the organization when planning in-services. They consider costs such as location, meals, speaker and materials, but the major expense for field-based personnel is reimbursement of travel expenses. Findings ways to minimize training inputs while addressing critical needs is essential. Coordinators may need to use distance and/or self-study methods and offer in-services at locations geographically convenient to the participants rather than the presenter.

Though personal and professional development topics were ranked as most needed for training, findings indicated that respondents attended a median of 6 days of technical knowledge/skills training and only 1 day of process skills training. The results show interest in developing process skills, but personnel are not attending the trainings. More pressing job demands may cause individuals to put aside their personal and professional development needs. Individuals will need supervisor encouragement to participate in process skill development. The organization needs to address creative ways to implement process skill development in the context of subject matter development.

In this era of accountability, educational organizations must evaluate program outcomes. In-service coordinators would benefit from standardized evaluation instruments that emphasize high-level outcomes or impacts and that can be adapted to the specific in-service.

OSU Extension is not unique in facing the challenges of designing a professional development system to meet the educational needs of a diverse population of employees and a geographically dispersed workforce. The question continues to be what is the right professional development model for complex organizations? Research could be conducted nationally to examine what models states are using to overcome the challenges facing professional development of Extension personnel.

References

Baker, M., & Villalobos, H. (1997). Perceptions of county faculty of the professional development needs of specialists. Journal of Extension [On-line], 35(4). Available at: http://www.joe.org/joe/1997august/a1.html

Borich, G. (1980). A needs assessment model for conducting follow-up studies. Journal of Teacher Education 31(3).

Gap Consulting, LLC (March, 2000). "WWO Contests Features-Professional version 3.1". GAP Consulting. LLC. Available at: http://www.wolfcountry.com/scripts/

Miller, L. E., & Smith, K. L. (1983). Handling non-response issues. Journal of Extension, 21(5) 45-50. Available at: http://www.joe.org/joe/1983september/83-5-a7.pdf

Mincemoyer, C., & Kelsey, T. (1999). Assessing in-service education: Identifying barriers to success. Journal of Extension [On-line], 37(2). Available at: http://www.joe.org/joe/1999april/a3.html

Radhakishna, R. B., & Thomson, J. S. (1996). Extension agent's use of information sources. Journal of Extension, [On-line], 34(1). Available at: http://www.joe.org/joe/1996february/rb2.html

Shih,W., & Evans.J.,F. (1991). Where field staff get information--approaching the electronic times. Journal of Extension [On-line],29(3). Available at: http://www.joe.org/joe/1991fall/a5.html

Stone, B., & Bieber, S. (1997). Competencies: A new language for our work. Journal of Extension [On-line], 35(1) Available at: http://www.joe.org/joe/1997february/comm1.html

Van Buren, M.E. (2001). State of the industry report 2001. Alexandria, Va.: American Society for Training and Development

Westinghouse Electric Corporation Technology Transfer Program (1997). GETNA-General employee training needs analysis: a paper-and-pencil tool for determining common denominator training needs. Department of Energy/Carlsbad Area Office.