Knowledge, Attitudes, and Commitment Concerning Evidence-Based Prevention Programs: Differences Between Family and Consumer Sciences and 4-H Youth Development Educators

Introduction

Over the past several decades, increasing numbers of preventive interventions targeting youth and families have been developed and packaged for distribution. Many, but not all, of these interventions or programs are considered "evidence-based" in that they are built on a sound theoretical base, and their effectiveness has been assessed using rigorous evaluation methodologies (Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2004; Flay et al., 2005; National Research Council and Institute of Medicine, 2009; Olson, 2010; Small, Cooney, & O'Connor, 2009). While the relative effectiveness of individual programs varies, we can have general confidence that implementing evidence-based interventions or programs will lead to stated outcomes when they are implemented with fidelity and with similar populations. This reasoning is clearly illustrated in that funding to implement prevention programs from most government sources is contingent on selecting programs from government supported "best practice" lists (Hill & Parker, 2005; Olson, 2010).

In light of trends toward the use of pre-packaged evidence-based and other prevention programs, various scholars have called on Extension administrators, faculty, and staff, especially within the 4-H Youth Development and Family and Consumer Sciences (FCS) areas, to follow suit. These scholars argue that adopting evidence-based practices (EBPs) will help FCS and 4-H educators strengthen the effectiveness of their outreach efforts, be more accountable to their funders, and demonstrate positive outcomes to stakeholders (Dunifon, Duttweiler, Pillemer, Tobias, & Trochim, 2004; Fetsch, MacPhee, & Boyer, 2012; Hill & Parker, 2005; Molgaard, 1997; Spoth & Greenberg, 2005; 2011). Extension's involvement in delivering EBPs can also have a positive effect on its reputation as an important resource for children, youth and families in the community. (Mincemoyer et al., 2008). Furthermore, implementing an existing, packaged prevention program, whether evidence-based or not, may be more efficient than developing a new program (Olson, 2010).

In light of these trends, the purpose of the research described in this article was to examine differences and similarities across county-level 4-H and FCS Extension educators in attitudes towards and knowledge of two separate, but related types of interventions. Specifically, our research focused on attitudes and knowledge related to both evidence-based practices at a general level and also pre-packaged prevention programs that may or may not be evidence-based.

Use of Evidence-Based and/or Prevention Programs Within Extension

While research on evidence-based and other prevention programming within Extension is limited, several studies have examined the extent to which Extension professionals are open to incorporating such efforts into their work. For instance, Hill and Parker (2005) surveyed a sample of FCS and 4-H faculty and educators to determine their attitudes toward the use of evidence-based and other prevention programs. The results revealed a majority of study participants perceived a need for prevention programming to address issues such as: substance abuse, school dropout, and suicide. Moreover, between one-half and two-thirds of participants believed that evidence-based prevention programming could be effective in addressing negative outcomes. However, nearly as many participants expected that traditional Extension programs would be equally effective. Participants perceived numerous barriers to implementing evidence-based and/or prevention programs, which included: limited resources, collaboration difficulties, fear of losing traditional Extension programs, and a perception that prevention programs focus on deficits of program participants (Hill & Parker, 2005).

A similar study with a sample from Pennsylvania and Iowa compared attitudes and knowledge of prevention programs among FCS and 4-H educators and community-based human service providers (Perkins, Mincemoyer, & Lillehoj, 2006). In that study, results indicated both Extension educators and human service providers felt connected to their communities, and both groups had similar experiences with collaborations. However, while Extension educators reported more knowledge of the risk and resiliency model, they reported less awareness of actual community-based prevention programs. These findings may be the result of educators' biases toward traditional Extension programs, as indicated in Hill and Parker's (2005) findings. Similarly, they may reflect the common 4-H focus on positive youth development, which could be viewed as not encompassing a risk-reduction approach employed in some prevention programs. Whatever the causes, these results suggest several factors that may discourage Extension professionals from implementing pre-packaged prevention programs, whether evidence-based or not.

Differences in Evidence-Based Approaches Across Program Areas

Although most calls for increased use of evidence-based programs have been directed at the FCS and 4-H program areas, there is some evidence that receptiveness towards such approaches differs between them. Hamilton, Chen, Pillemer, and Meador (2013) examined use of research by Extension educators from different program areas in New York State. Their findings indicate that compared to educators based in the agriculture and family and consumer sciences program areas, those from 4-H reported more difficulty accessing relevant research, less involvement in conducting research, and less familiarity with evidence-based programs.

While the reasons for differences in usage of research-based information among Extension educators is not entirely known, a perception exists that 4-H educators are more likely to administer traditional club programming than to implement packaged evidence-based youth or family-focused programs (Hill & Parker, 2005; Scholl & Paster, 2011). While 4-H club activities are usually based on solid educational theory and can be tailored to the unique needs of stakeholders, these youth development programs are typically not subject to rigorous evaluation.

While a study by Bikos and colleagues (2011) indicated that 4-H faculty, educators, and volunteers expressed a general interest in using research to inform program development, they reported numerous barriers that limited 4-H personnel capacity to act on such interests. For example, study participants reported partial knowledge on where to access relevant research, limited time to incorporate research-based knowledge into their work, and limited funding. Furthermore, some saw value in continuing their traditional Extension programming because they felt it could be more responsive and tailored to the unique needs of their stakeholders. Pre-packaged prevention programs, whether evidence-based or not, may be viewed as less flexible and thus less in line with stakeholder needs.

Research on the use of evidence-based programs among FCS educators is limited. However, in recent years, several indicators suggest that FCS educators have been relatively successful in creating and implementing evidence-based strategies. For example, the common Extension practice of using age-paced newsletters for parents has been found to effectively promote positive parenting in several outcome evaluations (Bogenschneider & Stone, 1997; Dworkin, Gonzalez, Gengler, & Olson, 2011; Riley, Meinhardt, Nelson, Salisbury & Winnett, 1991). In addition, FCS educators have developed and evaluated evidence-based marriage curricula (Goddard & Olsen, 2004), and recent efforts of FCS faculty have established evidence-based programs to support military families (Carroll, Robinson, Orthner, Matthews, & Smith-Rotabi, 2008).

Purpose of Current Study

Prior research provides some preliminary evidence that differences exist between FCS and 4-H educators in their comfort with and/or use of evidence-based and/or prevention programs. As outlined above, FCS educators may have had more experience developing and implementing evidence-based programming than 4-H educators. Furthermore, 4-H educators may experience specific barriers that make them less likely than their FCS colleagues to adopt evidence-based and/or prevention programs. To date, however, there have been few studies that directly compare FCS and 4-H educators.

The study reported here builds on a report of Extension system readiness for EBP implementation and dissemination (Ralston et al., 2011; Spoth, Ralston et al., 2013) that was part of a series of projects aimed at developing strategies for scaling-up the PROSPER partnership-based delivery system (Spoth & Greenberg, 2011). The purpose of the preliminary research was to assess readiness of all state Extension systems for potential implementation of the PROSPER model, which encompasses a system that supports the implementation of evidence-based prevention programs within local community settings. The survey addressed five readiness areas, including understanding the degree to which state Extension system staff in the youth and family program areas were ready for, knowledgeable about, engaged in, and committed to implementing evidence-based and other prevention programs.

In the study, we analyzed data from a national sample of FCS and 4-H educators. The primary purpose of the study was to discover the unique knowledge, attitudes, and perceptions of barriers to implementing EBPs and other prevention programs among county-level Extension educators within each program area. Our primary hypothesis was that compared to their 4-H counterparts, FCS educators would be more likely to support using evidence-based and/or prevention programs because FCS educators would have more knowledge regarding these practices, would report fewer barriers to employing them, and would be better prepared to collaborate with community agencies, schools, and groups on implementing EBPs. This hypothesis was based in part on the formal training that FCS educators receive and use in their practice.

Method

Participants

During fall, 2009, Extension administrators, faculty, coordinators, specialists, educators, and assistants involved in the youth and family programming areas across the United States were invited to participate in a Web-based survey to assess their attitudes and knowledge regarding prevention, evidence-based programs, and collaboration and partnership activities. Out of a possible 4,181 participants, 958 (23%) completed the Web surveys, a rate consistent with similar Web-based surveys (Hamilton, 2009).

In the current study, our analyses were limited to the 225 educators in the sample whose title identified them as a FCS or 4-H/Youth Development Educator. These educators (97 Family; 128 4-H/Youth) were distributed among 30 states, for an average of eight participants in each state (Min = 1; Max = 34). On average, participants had been in their current position for 9.5 years (SD = 8.6), and their tenure with their state's Extension system averaged 11.0 years (SD = 9.4). Ninety-six percent of the sample had full-time positions. The educational status of the sample was high: 30.3% had a minimum of a college degree, 63.8% had a master's degree, and 3.6% had a terminal degree.

Procedures

The potential respondent pool was created through compiling a full list of FCS and 4-H personnel from each state's (and Washington DC's) Web-based employee rosters. Once rosters were collected (N=5,072), they were examined for completeness and size. To balance the sample, employee names were randomly selected from state systems that had over 100 names on their roster so that there was a maximum of 100 potential participants from each state. This process resulted in the sampling frame of 4,181 participants.

Once the sampling frame was finalized, participants were recruited through a series of letters to State Extension Directors and regional administrators of Extension within each state. A $2,000 incentive was offered to three state Extension Systems that each had the highest response rate in one of three different size categories: relatively small (< 100 employees), medium (between 100- 250 employees), and large (approximately greater than 250 employees). In addition, one $500 award for professional development was offered to one randomly selected respondent within each state's Extension System. State Extension Directors were asked to send a notification letter to their staff about the survey, before the survey invite emails were sent.

Participants were officially invited to take the survey via an email that came directly from data collection staff. This email included a consent letter, a survey link, and an individual access code. Surveys and reminders were sent to all potential participants over a 12-week period in fall 2009.

Measures

Eight measures that assessed participant attitudes toward and knowledge of evidence-based programming, prevention programming (which may or may not be evidence-based), and collaboration and partnership activities were included in this article. Information on each measure is included in Table 1.

Table 1.
Self-Report Measures of Attitudes and Knowledge Towards Prevention Programming

Scale name	Description	# of items	Sample item	Range	Reliability
Focus on prevention	Perceptions of importance of prevention	5	How important are each of the following areas of prevention for the communities in your state? (Delinquency, obesity, etc.)	1 = not important 5 = very important	α = .85
Support for prevention	Perceptions of Extension's support for prevention programming	3	Your state Extension system is committed to planning and conducting prevention programming.	1 = strongly disagree 5 = strongly agree	α = .75
Obstacles to prevention programming	Perceptions of the barriers to implementing prevention programs	3	There are no financial resources to support prevention programming.	1 = strongly disagree 5 = strongly agree	α = .65
Individual knowledge of Evidence-based programs (EBPs)	Knowledge or attitudes of evidence-based programs	2	I know where to go to find information on evidence-based programs.	1 = strongly disagree 5 = strongly agree	r = .69
Requirements and appeal of EBPs	Importance of factors when deciding on new EBPs	3	It was being successfully implemented by colleagues.	1 = not important 5 = very important	α = .57
Extension commitment to EBPs	Perceptions of Extension's commitment to using EBPs	3	Most of the children, youth and families programs offered by Extension use evidence-based models.	1 = strongly disagree 5 = strongly agree	α = .58
Resources and support for collaboration and partnerships	Perceptions of support available to engage in collaborative efforts	2	Extension program staff is provided with the monetary support necessary to engage in collaborative efforts that serve children, youth, and families.	1 = strongly disagree 5 = strongly agree	r = .48
Collaboration and partnership activity	Perceptions of collaborative activities of Extension program staff	5	Extension program staff has successfully collaborated on prevention programming with public schools.	1 = strongly disagree 5 = strongly agree	α = .77

Analysis Plan

The data had a 2-level hierarchical structure; individuals (Level 1) were nested within states (Level 2). Moreover, the number of respondents within each state varied, creating an unbalanced sample. As a result, multilevel mixed models utilizing proc mixed in SAS Version 9.2 were used to conduct analyses.

Preliminary analyses were conducted to test for significant between (Level 2) variance (i.e., significant agreement among responses from different individuals within the same state) in order to decide if a random Level 2 intercept should be estimated in the analyses. The variance due to state (i.e., Level 2 between variance) for all scales ranged from 0.02 to 0.09 with one exception—the between variance of the requirements and appeal of EBPs scale was zero. As a result, all models included estimating a random Level 2 intercept, with the exception of the requirements and appeal of EBPs scale. Estimating a Level 2 random intercept accounts for the shared variance among participants within the same state; therefore, it is a more appropriate and conservative analysis (Singer & Willett, 2003). Final statistical models tested for the fixed effect of educator type (4-H/Youth vs. Family) while estimating a random intercept for state. Models were assessed with the Restricted Maximum Likelihood (Singer & Willett, 2003) using the Satterthwaite degrees of freedom method.

Results

Descriptive statistics and participation numbers are in Table 2. Family and youth educators perceive a relatively strong level of the importance of prevention and support for prevention programs for the communities in their state. Endorsements from their colleagues and supervisors and strong outcomes were also important considerations when selecting a new EBP. Individuals seemed to perceive a moderate level of obstacles to implementing prevention programs and felt they had a moderate level of knowledge of EBP. Levels of collaboration and partnership activities and Extension's commitment to EBP were moderate to high. However, participants seemed to perceive a lack of available resources for collaboration and partnerships.

Table 2.
Descriptive Statistics of Attitude and Knowledge Scales

Scale	N	Mean	SD	Min	Max
Focus on Prevention	225	4.22	0.69	1.40	5.00
Support of Prevention	225	3.88	0.69	2.33	5.00
Obstacles to Prevention Programming	225	2.78	0.74	1.00	5.00
Knowledge of EBPs	225	3.49	0.98	1.00	5.00
Requirements and Appeal of EBPs	222	3.89	0.70	1.00	5.00
Extension Commitment to EBPs	225	3.63	0.69	1.67	5.00
Resources and Support for Collaboration and Partnerships	225	2.58	0.86	1.00	5.00
Collaboration and Partnership Activities	224	3.54	0.62	1.80	5.00

Results of the multi-level mixed models, including the least squares means, the 95% Upper Confidence Level (UCL) and Lower Confidence Level (LCL), and significance values are presented in Table 3. Significant differences in the expected direction were found in five out of the eight scales. Specifically, FCS educators compared to 4-H educators had higher levels on focus on prevention, perceived agency support of prevention, knowledge of EBPs, Extension's commitment to EBPs, and collaboration and partnership activities. One additional scale, resources and support for collaboration and partnerships, was in the expected direction and approached significance levels. Means of the two groups were not significantly different on the last two scales, obstacles to prevention programming and requirements and appeal of EBPs.

Table 3.
Results From Multi-Level Mixed Models Assessing Differences in Attitudes and Knowledge of Family Versus Youth Educators

Scale	Least Squares Mean	LCL	UCL	Significance
Focus on Prevention				p = .02
Family	4.33	4.15	4.52
Youth	4.11	3.95	3.95
Support of Prevention				p = .0002
Family	4.07	3.91	4.23
Youth	3.71	3.57	3.85
Obstacles to Prevention Programming				p = .27
Family	2.84	2.65	3.02
Youth	2.71	2.57	2.88
Knowledge of EBPs				p < .0001
Family	3.80	3.56	4.04
Youth	3.25	3.04	3.45
Requirements and Appeal of EBPs				p = .66
Family	3.86	3.72	4.00
Youth	3.90	3.78	4.03
Extension Commitment to EBPs				p = .002
Family	3.78	3.62	3.94
Youth	3.48	3.34	3.61
Resources and Support for Collaboration and Partnerships				p = .06
Family	2.71	2.52	2.90
Youth	2.49	2.33	2.65
Collaboration and Partnership Activities				p = .02
Family	3.64	3.50	3.78
Youth	3.43	3.31	3.55

Discussion

The results of the study reported here complement those of previous investigations in that a majority of Extension educators reported that they value EBPs and prevention programs. Descriptive data in the form of mean scores reported in Table 2 suggest that educators generally support using evidence-based and/or prevention programs within their own communities. 82.8% felt that prevention of substance abuse among youth was important or very important, and 70.3% felt the same way about delinquent behaviors. However, only 55% of participants agreed or strongly agreed that their Extension leaders are committed to evidence-based programming.

Furthermore, the unique contribution of our findings is that we were able to use a national sample to directly compare the attitudes and knowledge about evidence-based and/or prevention programs between FCS educators to 4-H educators. The results of these comparisons were consistent with the New York statewide findings of Hamilton and colleagues (2013) reviewed earlier. Specifically, we found that compared to 4-H educators, FCS educators were significantly more likely to focus on prevention, to support prevention efforts, to report knowledge of and commitment to evidence-based programs, and to engage in collaborative programming efforts. These findings support our hypothesis that FCS educators are more familiar with and more willing to implement evidence-based programs and/or prevention strategies.

The reservation of 4-H educators to implement evidence-based programs, and particularly pre-packaged prevention programs, may be related to their commitment to positive youth development (PYD) programming. Historically, tension has existed among professionals advocating a prevention approach to youth programming and those in favor of PYD approaches. In recent years, such tensions have diminished as youth development professionals have found common ground that stresses risk reduction as well as promotion of strengths and competencies (Catalano, Hawkins, Berglund, Pollard, & Arthur, 2002; Lerner et al., 2011; Perkins & Caldwell, 2005). However, remaining biases toward traditional prevention practices may, at least partially, explain our current results. Another potential explanation for this difference may have to do with the nature of Extension programming; 4-H programs often are a collection of distinct activities and events. Finally, as noted earlier, the education and training of FCS educators involves emphases on evidence-informed decision-making and programming.

We were surprised to find no differences in reports of barriers to using prevention programs. Given prior research (Bikos et al., 2011; Hill & Parker, 2005), we had expected to find more barriers reported by 4-H educators relative to their FCS counterparts. On the contrary, the findings indicate that experiences with and attitudes toward prevention programming may be more important predictors of willingness to use them than obstacles.

Our findings signal that FCS educators might be more ready and/or willing than their 4-H colleagues to integrate evidence-based programs and prevention programs into their programming dossiers. As noted earlier, this integration comes with benefits such as an increased chance of program effectiveness, accountability to funders, and simplicity of ready-made curricula. Nevertheless, limitations exist, depending on the specific program selected: the possibility of less flexibility in program implementation, less control over programming details, and less ability to tailor content. Also, costs may be associated with acquiring an established evidence-based or other prevention curriculum, although such financial investments could be recovered by saving time ordinarily dedicated to program development. The evidence base of many of these programs also may result in more program effectiveness, which could offset costs by reducing the need for future programs.

Limitations

The study has several limitations. First, while the response rate to our survey was consistent with previous Web-based surveys, less than 25% of eligible respondents participated. Thus, the sample may be biased in unknown ways concerning the perceptions of Extension educators. Similarly, the sample was drawn from only 30 states, and 13 states had three or fewer respondents. Perceptions and attitudes may have been different in states not represented in the study, and they may not be entirely representative of the perceptions and attitudes of those that did not respond to the survey. Finally, several of our constructs were assessed with two-item measures, and others had relatively low internal consistency. Therefore, these results should be considered preliminary, pending replication with stronger instruments. Nevertheless, the strengths of the study, in terms of reach and depth, outweigh the limitations.

Implications for Practice

Given the realities of the movement towards evidence-based practice and using pre-packaged prevention programs, increased demands for accountability by stakeholders, and shrinking numbers of Extension professionals who can be dedicated to program development, the use of established evidence-based and other prevention programs will likely increase in the coming years. Based on the results of the study reported here, Extension administrators and stakeholders may be able to help facilitate this process within Extension by supporting efforts to remove barriers and provide additional support and resources in order to create a culture that encourages collaborative efforts to use evidence-based and other pre-packaged prevention programs.

Although the differences were not large, our results indicate that 4-H educators seem less likely to be prepared to implement evidence-based and/or prevention programming as compared to their FCS counterparts. Given this finding, Extension program leaders and administrators may wish to identify ways to incorporate the best aspects of existing 4-H culture (e.g., parent involvement and project-based learning) into evidence-based and/or prevention programming. Then, Extension administrators can convey these ideas and values when communicating about evidence-based and/or prevention programs with 4-H educators. For example, previous authors have noted the benefits of 4-H educators' flexibility, experience, and connections with local communities (Dunifon et al., 2004; Fetsch et al., 2012).

Rather than eliminating these strengths when implementing a pre-packaged evidence-based and/or prevention program, we recommend that 4-H educators integrate such characteristics into the selected programming. While in some ways they might lose a degree of flexibility in their programming, they can and should engage community members in the selection, implementation, and evaluation of these strategies, and considerable flexibility exists regarding how these processes will work best in different communities. There are examples where 4-H educators are leading community-based implementations of such initiatives (see Spoth, 2007; Spoth, Greenberg, Bierman, & Redmond, 2004). Because of their skills and experiences, 4-H educators are well positioned to be successful at engaging community members in this process, which may improve resiliency of youth and families in their communities.

We may also see more acceptance of packaged evidence-based and other prevention programs if educators understand that many contemporary prevention programs incorporate strengths-based principles and are developed for all youth; few are strictly deficit-based targeting specific populations of youth. By limiting existing biases and removing some of the above-mentioned barriers, we may create an environment in which FCS and 4-H educators maintain many of the valued traditions of existing Extension programming, while maximizing the likelihood of success in addressing stakeholder needs.

Conclusions

As Extension budgets continue to tighten, serving wider geographical areas and securing external funding are new standards for program administration and support. Thus, with funding becoming increasingly reliant on using evidence-based and other pre-packaged programming, implementing these programs is one way that Extension professionals can be successful and use limited resources wisely. Although some see drawbacks to using established evidence-based and other prevention programs, most professionals agree that these initiatives can be efficient and effective (Catalano et al., 2004). By helping to remove current barriers and encouraging educators from FCS and 4-H program areas to incorporate evidence-based strategies into their current efforts, we can help Extension effectively meet the needs of youth and families, while being able to document positive outcomes.

Acknowledgements

The authors would like to acknowledge and thank the following individuals for their contributions in survey design, data collection, and/or input on reliability analyses that led up to enabling this article to be completed: Katerina Ralston, Lisa Schainker, Eugenia Hanlon, Jane Todey, Cleve Redmond, Mark Feinberg, and Mark Greenberg. We also sincerely thank the State Cooperative Extension Systems that supported this research by encouraging participation of their employees of all levels to participate in the study. We also thank all of our participants for giving their time to complete the survey on which this study was based.

References

Bikos, L. H., Kocheleva, J. A., Campbell, T., Daryani, R., Chahil, S., Brown, R., . . . Pavese, L. (2011). Investigating the utilization of research evidence in the 4-H Youth Development program. Journal of Youth Development, 6. Retrieved from: http://nae4a.memberclicks.net/assets/documents/JYDfinal0602.pdf

Bogenschneider, K., & Stone, M. (1997). Delivering parent education to low and high risk parents of adolescents via age-paced newsletters. Family Relations, 46, 123-134.

Carroll, E. B., Robinson, L. C., Orthner, D., Matthews, W., & Smith-Rotabi, K. (2008). Essential life skills for military families: Mobilizing the Cooperative Extension Service in NC. Journal of Family and Consumer Sciences, 100(1), 52.

Catalano, R. F., Berglund, M. L., Ryan, J. A. M., Lonczak, H. S., & Hawkins, J. D. (2004). Positive youth development in the United States: Research findings on evaluations of positive youth development programs. The Annals of the American Academy, 591, 98-124.

Catalano, R. F., Hawkins, J. D., Berglund, M. L., Pollard, J. A., & Arthur, M. W. (2002). Prevention science and positive youth development: competitive or cooperative frameworks? Journal of Adolescent Health, 31(6), 230-239.

Dunifon, R., Duttweiler, M., Pillemer, K., Tobias, D., & Trochim, W. M. K. (2004). Evidence-based Extension. Journal of Extension [On-line], 42(2) Article 2FEA2. Available at: http://www.joe.org/joe/2004april/a2.php

Dworkin, J., Gonzalez, C., Gengler, C., & Olson, K. (2011). Using newsletters to improve parents' communication with their early adolescents. Journal of Family and Consumer Sciences, 103, 47-54.

Fetsch, R. J., MacPhee, D., & Boyer, L. K. (2012). Evidence-based programming: What is a process an Extension agent can use to evaluate a program's effectiveness? Journal of Extension [On-line], 50(5) Article 5FEA2. Available at: http://www.joe.org/joe/2012october/a2.php

Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S.,…Ji, P. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6, 151-175.

Goddard, H. W., & Olsen, C. S. (2004). Cooperative Extension initiatives in marriage and couples education. Family Relations, 53, 433-439.

Hamilton, M. B. (2009). Online survey response rates and times: Background and guidance for industry. Ipathia, Inc. / SuperSurvey. Retrieved from: http://www.supersurvey.com/papers/supersurvey_white_paper_response_rates.pdf 

Hamilton, S. F., Chen, E. K., Pillemer, K., & Meador, R. H. (2013). Research use by Cooperative Extension Educators in New York State. Journal of Extension [On-line], 51(3) Article 3FEA2. Available at: http://www.joe.org/joe/2013june/a2.php

Hill, L. G., & Parker, L. A. (2005). Extension as a delivery system for prevention programming: Capacity, barriers, and opportunities. Journal of Extension [On-line], 43(1) Article 1FEA1. Available at http://www.joe.org/joe/2005february/a1.php

Lerner, R. M., Lerner, J. V., Lewin-Bizan, S., Bowers, E. P., Boyd, M. J., Mueller, M. K., ... & Napolitano, C. M. (2011). Positive youth development: Processes, programs, and problematics. Journal of Youth Development, 6, 40-64.

Molgaard, V. K. (1997). The Extension Service as key mechanism for research and services delivery for prevention of mental health disorders in rural areas. American Journal of Community Psychology, 25(4), 515544.

Mincemoyer C., Perkins D., Ang P., Greenberg M. T., Spoth R. L., Redmond C., & Feinberg M. (2008). Improving the reputation of cooperative extension as a source of prevention education for youth and families: The effects of the PROSPER model. Journal of Extension [On-line], 46(1) Article 1FEA6. Available at: http://www.joe.org/joe/2008february/a6.php

National Research Council and Institute of Medicine (2009). Preventing mental, emotional, and behavioral disorders among young people: progress and possibilities. Committee on the Prevention of Mental Disorders and Substance Abuse Among Children, Youth, and Young Adults: Research advances and promising interventions. Mary Ellen O'Connell, Thomas Boat, and Kenneth E. Warner, Editors. Washington DC: The National Academies Press.

Olson, J. R. (2010). Choosing effective youth-focused prevention strategies: A practical guide for applied family professionals. Family Relations, 59, 207-220.

Perkins, D. F., & Caldwell, L. (2005). Resiliency, protective processes, promotion, and community youth development. In Witt, P., & Caldwell, L. (Eds.), Recreation and youth development (149-167). State College, PA: Venture Publishing.

Perkins, D. F., Mincemoyer, C. C., & Lillehoj, C. J. (2006). Extension educators' perceptions of community readiness, knowledge of prevention science, and experience with collaboration. Journal of Family and Consumer Sciences, 98, 20-26.

Riley, D., Meinhardt, G., Nelson, C., Salisbury, M. J., & Winnett, T. (1991). How effective are age-paced newsletters for new parents? A replication and extension of earlier studies. Family Relations, 247-253.

Scholl, J., & Paster, A. (2011). Locating, analyzing and making available a century of 4-H research studies, 1911-2011. Journal of Youth Development, 6, 66-81.

Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis. New York: Oxford University Press.

Small, S. A., Cooney, S. M., & O'Connor, C. (2009). Evidence-informed program improvement: Using principles of effectiveness to enhance the quality and impact of family-based prevention programs. Family Relations, 58, 1-13.

Ralston, E., Spoth, R., Shin, C., Schainker, L., Perkins, D, Redmond, C. & Greenberg, M. (2011, June). Evaluating state readiness and capacity: Framework and findings. Paper presented as part of the symposium, Developing a National EBI Delivery System Based on the PROSPER Partnership model. Society for Prevention Research Conference, Washington, D.C.

Spoth, R. (2007). Opportunities to meet challenges in rural prevention research: findings from an evolving community-university partnership model. Journal of Rural Health, 23(supplemental), 42-54.

Spoth, R. L., & Greenberg, M. T. (2005). Toward a comprehensive strategy for effective practitioner-scientist partnerships and larger-scale community health and well-being. American Journal of Community Psychology, 35, 107-126.

Spoth, R. L., & Greenberg, M. T. (2011). Impact challenges in community science-with-practice: Lessons from PROSPER on transformative practitioner-scientist partnerships and prevention infrastructure development. American Journal of Community Psychology, 40, 1178-1191.

Spoth, R., Greenberg, M., Bierman, K., & Redmond, C. (2004). PROSPER Community-university partnership model for public education systems: Capacity-building for evidence-based, competence-building prevention. Prevention Science (Invited article for Special issue), 5(1), 31-39.

Spoth, R., Ralston, E., Schainker, L., Shin, C., & Redmond, C. (2013). Summary Results of the National Extension Survey. Manuscript in preparation.