Organizational Capacity Survey: Capturing an Extension System's Current State and Pinpointing Areas for Improvement

Extension systems are often in transition due to the requirement to be responsive to factors that include stakeholder needs; local, state, and federal budgets; technological changes; legislative priorities; and personnel changes. Such organizational transitions provide an opportune time for organizational learning. Mississippi State University (MSU) Extension has been engaged in an intentional process to reassess system priorities, evaluate efficiency and effectiveness, and focus on resource management with two primary goals in mind: (a) enhance connections between campus and counties and (b) re-establish an integrated program planning framework and process.

When broad system changes are put into motion, several activities can help ensure success (Betts, Peterson, & Roebuck, 2003):

• Explicitly declare the desired changes and organizational supports needed.
• Write operational definitions of strategies and desired outcomes so that they can be measured.
• Evaluate system change through data collection at multiple time points (e.g., baseline, mid-course, end).
• Use the evaluation data to provide feedback, inform stakeholders, make changes during the process, and plan for sustainability.
• Provide organizational support for staff through in-service training, administrative support, and other relevant resources.

Development and Implementation of an Organizational Capacity Survey

MSU Extension evaluation specialists recognized that an organizational capacity survey (OCS) could fill the need they had to understand MSU Extension employees' perceptions of the state of the organization. An OCS is one type of tool that can be used for documenting system changes in times of transition. The MSU Extension evaluation specialists used the OCS to explore the current state of MSU Extension as perceived by its employees in terms of (a) assumptions on which organizational support for programs are based; (b) the ways in which organizational values, vision, and strategic planning are communicated and understood across organizational levels; (c) identification of areas in which capacity needs to be developed; and (d) celebration of areas of success (Betts, Peterson, Marczak, & Richmond, 2001).

The online OCS was conducted from May through June 2014. The MSU OCS was loosely based on an OCS developed to document the current states of and changes in abilities of state Extension systems that had received Children, Youth, and Families at Risk (CYFAR) funding from the U.S. Department of Agriculture (Betts, Marczak, Peterson, Sewell, & Lipinski, 1998). This foundational OCS is referred to herein as the CYFAR OCS. (CYFAR funding was to be used to develop and sustain effective programs for children, youth, and families at risk. For more information regarding the CYFAR OCS, see "Graduation: From National Initiative to Base Programs" in the February 2003 issue of the Journal of Extension.) Whereas the CYFAR OCS was used with paid Extension professionals who worked directly or indirectly with children, youth, and families, the MSU OCS targeted MSU Extension administrators, specialists, agents, and instructors across all four program areas (agriculture and natural resources, community resource development, family and consumer sciences, and 4-H youth development).

Similar to the CYFAR OCS, domains assessed in the MSU OCS were MSU Extension's vision and commitment, plan of work, working relationships, diversity and pluralism, public value, personnel knowledge and expertise, training needs, and demographics. Plan of work and public value were not included in the CYFAR OCS, but were added as new domains to the MSU OCS. Table 1 provides sample survey questions. The full instrument is available from the first author.

The majority of MSU OCS questions were phrased to assess perceptions of MSU Extension currently and perceptions of how MSU Extension should be ideally. This format allowed for the calculation of discrepancy or gap scores between paired items to identify areas in which capacity development was needed and/or desired. Specifically, discrepancy scores were calculated for questions about organizational vision and commitment, plan of work, working relationships, diversity and pluralism, and public value by subtracting the "ideal" response from the "current" response to demonstrate the size and direction of the gap.

The processes for establishing validity and reliability of items in the CYFAR OCS were described in the initial report of results (Betts et al., 1998). Since the plan-of-work items in the MSU OCS were modeled after the mission and vision questions in the CYFAR OCS, validity and reliability should mirror that of the CYFAR survey. Face validity of the public value scale was established through expert review, with minor revisions made on the basis of expert recommendations. Cronbach's alpha was .782 for the "current" public value items and .865 for the "ideal" public value items.

Conclusions

The MSU OCS demonstrates the utility of the method for informing organizational learning. Findings from an OCS can reveal gaps in what an organization is doing and what an organization's employees perceive the organization should be doing. Identified areas of need can then be addressed through state-based professional development. Such professional development should address relevant structural factors (e.g., mechanisms and procedures that allow an organization to systematically collect, disseminate, and use information) and cultural factors (e.g., shared professional values, vision, and leadership) (Rowe, 2010).

Extension systems in other states could adopt and/or adapt the MSU OCS to identify areas in which their respective organization's capacity could be strengthened to effectively support staff and programs, and meet clientele needs. If multiple Extension systems used the MSU OCS questions, gaps that occur across states could be identified, perhaps resulting in regional or national capacity-building efforts. Also, if an OCS is repeated every 3–5 years, Extension systems can track changes over time and document trends in organizational capacity.

References

Betts, S. C., Marczak, M. S., Peterson, D. J., Sewell, M., & Lipinski, J. (1998). Cooperative Extension's capacity to support programs for children, youth and families at risk: National results of the Organizational Change Survey. Retrieved from http://ag.arizona.edu/sfcs/cyfernet/cyfar/

Betts, S. C., Peterson, D. J., Marczak, M. S., & Richmond, L. S. (2001). System-wide evaluation: Taking the pulse of a national organization serving children, youth, and families at risk. Children's Services: Social Policy, Research, and Practice, 4(2), 87–101. doi:10.1207/S15326918CS0402_3

Betts, S. C., Peterson, D. J., & Roebuck, J. C. (2003). Graduation: From national initiative to base programs. Journal of Extension, 41(1) Article 1FEA4. Available at: https://www.joe.org/joe/2003february/a4.php

Rowe, E. (2010). Looking at Extension as a learning organization. Journal of Extension, 48(4) Article 4RIB1. Available at: https://www.joe.org/joe/2010august/rb1.php