Steps Toward Creating and Validating an Evidence-Based Couples Curriculum

Introduction

A movement toward evidence-based practice has been evident across a variety of disciplines that involve Extension specialists, educators, and other professionals. This approach emphasizes the importance of basing programs, policies, and other forms of outreach on high-quality scientific research (Dunifon, Duttweiler, Pillemer, Tobias, & Trochim, 2004). Although the approach has its roots in medicine (historically referred to as "evidence-based medicine"), it has spread to other fields of practice that commonly intersect with the mission of Extension. For example, professionals in the field of prevention science have developed a variety of youth- and family-focused prevention strategies that are based on a strong empirical and theoretical foundation (Small, Cooney, & O'Connor, 2009; Spoth, 2008; Spoth, Kavanagh, & Dishion, 2002). As a result, youth development and family life specialists and educators now have access to a variety of evidence-based programs, policies, and other strategies that they can help implement within their local communities.

Purpose

The purpose of the study described here was to push forward the process of evidence-based practice in the area of marriage and couples curricula development. As with other content areas of interest to Extension professionals, there has been an increased interest in basing marriage and couples programming on high-quality research and theory. Indeed, in recent years, numerous empirically validated marriage curricula have evolved (Jakubowski, Milne, Brunner, & Miller, 2004).

Despite the trends toward evidence-based practice, however, some scholars have suggested that there is still room for improvement in many current marriage and couples programs. Gottman, for one, (1999) faulted many such curricula for making recommendations that are not consistent with empirical findings. For example, he challenged the active listening model, anger as a dangerous emotion, the quid pro quo error, noncontingent positivity, and the harmony model. In other words, he disputes the validity of many of the marriage movement's sacred cows. Others have agreed with this assessment, arguing that many marriage programs are only loosely connected to research and are rarely evaluated (Adler-Baeder, Higginbotham, & Lamke, 2004). Even some popular couples programs that claim to have strong evaluation data actually get only small changes on a few of the multitude of measured variables (Jakubowski et al., 2004).

More recent meta-analytic studies have suggested that the availability of effective theory-based marriage programs has increased in recent years. However, enthusiasm about the effectiveness of such efforts is tempered somewhat by the fact that effect sizes remain small to moderate, and few studies have demonstrated long-term effectiveness beyond 3 to 6 months post-program (Blanchard, Hawkins, Baldwin, & Fawcett, 2009; Hawkins, Blanchard, Baldwin, & Fawcett, 2008). In sum, it appears that while a variety of notable examples of theoretically based and empirically validated couples curricula currently exist, the field as a whole has room to form tighter connections among theory, research, and programming efforts.

Project Description

The enduring challenge for curriculum developers is translating the vast and disparate findings on marriage into practical and useful lessons and activities that actually change couple dynamics in positive ways. To begin to address this challenge, we propose a systematic process for building and validating marriage curricula that is similar to that proposed by Adler, Higginbotham, and Lamke (2004), except that we suggest that empirical findings be summarized in a model rather than simply inventoried. Specifically, we suggest the following steps:

1. Empirical findings should be summarized in a model.
2. The model should be tested to demonstrate its connection to the desired outcomes.
3. A couples curriculum should be based on the confirmed model.
4. The curriculum should be carefully evaluated to be sure that the essential elements of the model were effectively conveyed to participants and that they led to the expected outcomes.

The first step in this systematic process is reviewed in Goddard and Olsen (2004) in which a six-dimension model, the Marriage and Couples Education Model (M/CEM), and its development are described. This is an attempt by scholars in university Extension to summarize and organize research on marriage. M/CEM was intended to provide a summary for marriage research, much as the National Extension Parent Education Model (Smith, Cudaback, Goddard, & Myers-Walls, 1994) provided a summary of parenting research. Table 1 shows M/CEM. This model was a precursor to the enlarged model, the National Extension Relationship and Marriage Education Model (NERMEM), which has not yet been published (Adler-Baeder et al., Manuscript in preparation).

Table 1.
A Proposed Marriage and Couples Education Model (M/CEM)

Dimension	Practices
Commitment	Make the relationship primary. Make couple time a priority. Set limits on intrusions. Build in rituals of connection.
Growth	Continue development of personal strengths. Support partner's use of signature strengths. Support partner's growth. Show respect for fundamental rights as a human.
Understanding	Understand partner through his or her world view. Make allowances for continuing differences. Accept and value differences. Understand and appreciate partner's pressures and needs.
Nurturance	Find and cultivate common interests and activities. Develop affectional synchrony with partner (languages of love). Affect balance: Five positives for each negative. Supplement and balance rather than compete and criticize.
Problem Solving	Stay calm in the face of differences. Being open to other views. Consider multiple courses of action. Accept some differences as a part of relationship. Allow time for changes.
Service	Develop a couple mission Involvement in common purposes. Build relationship on values as well as feelings.

This article focuses on the second step in the above-mentioned process. Specifically, the purpose of the study described here was to provide a preliminary assessment of the validity and reliability of a new measure that was designed to assess each of the six dimensions of the M/CEM model. In the study, we gathered data from a statewide random sample of married persons using the Marriage and Couples Functioning Measure (MCFM), conducted an exploratory factor analysis on the data, assessed the reliability of the emerging factors, and examined the degree to which each factor relates to measures of marital satisfaction and relationship optimism. More specifically, we tested the following hypotheses:

1. The items that comprise the MCFM will load onto six distinct factors that include marital commitment, growth, understanding, nurturance, problem solving, and service.
2. Each of the six factors will demonstrate a high level of internal consistency.
3. Each of the factors will be significantly related to independent measures of marital satisfaction and relationship optimism.

Method

Participants

A random sample of 1,204 married individuals living in Arkansas was surveyed through a structured telephone interview during the fall of 2004. The metropolitan central Arkansas area was oversampled in order to better capture the views of Arkansas' African-American population. The sample only included married persons because processes may be different for people in other forms of relationships. The analytical sample included primarily White (81%) and Black (16%) respondents, with approximately 3% coming from other racial/ethnic backgrounds. Study participants had been married for an average of 20 years at the time of the interview, with length of marriage ranging from newly married through 69 years.

Measures

Healthy marriage was assessed in two ways. First we used the Kansas Marital Satisfaction Scale (Schumm et al., 1986; alpha = .96 in the study described here), perhaps the most well-established measure of marital satisfaction. Second, because of concerns that satisfaction scales only assess the affective bases of marital well-being (Fowers, 2000), we developed the Relationship Optimism Scale, which includes the following two items: "How certain are you that the two of you [still] will be married five years from now?" and "How stable do you feel your marriage is?" Both items included four response categories ranging from one to four (alpha = .91 in the study described here). Responses on both of the above-mentioned measures were coded such that higher scores reflected more positive reports of marital satisfaction and relationship optimism.

The Marriage and Couples Functioning Measure, which was the primary focus of the investigation, was comprised of 23 items. The index was designed to include the following six subscales: commitment, growth, understanding, nurturance, problem solving, and service. Each hypothesized subscale included four items, with the exception of the growth subscale, which included three. Response categories for each item ranged from one to four, and all items were coded such that higher scores indicated a more positive view of the marriage.

Procedure

In order to determine the degree to which the Marriage and Couples Functioning Measure assesses each of the hypothesized subscales, we conducted an exploratory factor analysis (maximum likelihood extraction, varimax rotation). Next, we assessed the internal consistency of the emerging factors by calculating Cronbach's alpha for each factor. Finally, we assessed the concurrent validity of the measure by running a series of correlations and regression analyses in which we examined the strength of relations among each of the factors and measures of marital satisfaction and relationship optimism.

Results

Factor Analysis

The factor analysis revealed four, rather than six, distinct factors within the Marriage and Couples Functioning Measure. Both the commitment and service constructs emerged as independent factors as expected. However, the remaining two factors included a combination of the hypothesized subscales. Specifically, both the growth and understanding items loaded on a single factor, and the nurturance and problem solving items loaded on a single factor. In light of these findings, it appears as if the Marriage and Couples Functioning Measure includes four distinct subscales as follows: 1) commitment, 2) growth/understanding, 3) nurturance/problem solving, and 4) service. Individual items and their factor loadings are included in Table 2.

Reliability of the Measure

Next, we calculated Cronbach's alphas for each of the four constructs that emerged from the factor analysis. Results from these analyses revealed that each of the constructs had a high degree of internal consistency. Table 2 includes the alpha values for each of the subscales.

Table 2.
Marriage and Couples Functioning Measure

Dimension	Items	Factor loadings	Alpha
Commitment	I want this relationship to last.	.256	.753
	I make sure that other people don't come between my partner and me.	.385
	I have friendships that encourage my commitment to my marriage.	.628
	I have family members who encourage my commitment to my marriage.	.695
Growth/Understanding	I feel that I am making progress in my life.	.542	.828
	I believe that my spouse has important talents.	.500
	I encourage my spouse to develop his/her friendships.	.622
	I try to understand how my spouse sees things.	.588
	I notice what pressures my spouse feels.	.484
	I am comfortable with the fact that my spouse and I see some things differently.	.511
	I feel understood when my spouse listens to me.	.356
Nurturance/ Problem Solving	My spouse and I do many enjoyable things together.	.623	.879
	I show love to my spouse in the way she/he prefers.	.588
	We compliment each other more than we criticize each other.	.740
	I regularly remind my spouse of the good I see in our marriage.	.648
	I learn from my spouse's point of view.	.522
	My spouse and I work out our differences without attacking each other.	.654
	I am willing to live with some irritations in our relationship.	.256
	I can make creative and positive use of problems that come our way.	.435
Service	My spouse and I share many of the same values.	.653	.876
	Our relationship is stronger because of our common values.	.745
	Our relationship is stronger because of our joint projects.	.407
	My spouse and I feel like partners in a common cause.	.531

Validity of the Measure

We conducted two sets of analyses to measure the concurrent validity of the instrument. First, we calculated simple bivariate correlations between each subscale and both the Kansas Marital Satisfaction Scale and the Relationship Optimism Scale. Results indicated significant positive correlations between each subscale and each of the outcome measures, suggesting that the subscales are related to outcomes as expected (see Table 3).

Table 3.
Correlation Coefficients for Subscales and Outcome Measures

	Commitment	Growth/Understanding	Nurturance/Problem Solving	Service
Marital Satisfaction	.434**	.422**	.557**	.551**
Relationship Optimism	.486**	.366**	.482**	.555**
** p<.001

In order to assess the degree to which each of the subscales contributes uniquely to the variance in the outcome measures, we conducted two regression analyses, one with the Kansas Marital Satisfaction Scale as the outcome and one with the Relationship Optimism Scale as the outcome. In both equations, the four subscales were entered together on the first step of the equation. Results indicated that in both equations, each of the four subscales explained a unique and statistically significant component of the variance in the outcome variable. However, as shown in Tables 4 and 5, the growth/understanding construct was negatively related to each of the outcomes in these analyses, despite having a positive relation in the bivariate analyses. Such contradictory results are likely due to close relations among the subscales and became apparent when all of the subscales were entered simultaneously in the regression equation.

Table 4.
Regression Coefficients with Marital Satisfaction as Outcome

Variables	Beta	Zero-order correlation	Partial Correlation	t
Commitment	.168	.434	.160	5.432**
Growth/Understanding	-.105	.408	-.085	-2.859*
Nurturance/Problem Solving	.360	.561	.250	8.638**
Service	.246	.551	.186	6.345**
* p<.01 ** p<.001

Table 5.
Regression Coefficients with Relationship Optimism as Outcome

Variables	Beta	Zero-order correlation	Partial Correlation	t
Commitment	.292	.492	.275	9.542**
Growth/Understanding	-.177	.356	-.144	-4.880**
Nurturance/Problem Solving	.183	.488	.131	4.427**
Service	.368	.561	.275	9.563**
** p<.001

Discussion

Reliability and Validity of the Measure

The results from the study described in this article provide mixed evidence regarding the reliability and validity of the Marriage and Couples Functioning Measure. Data reported in this article suggest that the overall measure is comprised of four, rather than the hypothesized six, distinct subscales. In the study, these subscales included couples' commitment, growth/understanding, nurturance/problem solving, and service. Such results suggest that the hypothesized constructs of growth and understanding were more functionally similar than we anticipated, as were the constructs of nurturance and problem solving. Perhaps, for example, problem solving is only effective when accompanied by nurturance.

There are two other possible explanations for such an outcome. First, it is possible that there are really only four factors predicting marital satisfaction and that the M/CEM model needs to be revised to reflect four rather than six dimensions. Perhaps both growth and understanding, as well as nurturance and problem solving, are so closely related that they actually reflect two rather than four distinct constructs. A second explanation for the results of the factor analysis is that the current measure may not be sensitive enough to pick up the differences between growth and understanding and between nurturance and problem solving.

If the former explanation is accurate, our results suggest that the four-construct measure is both a reliable and valid measure. Indeed, each of these four subscales demonstrated high levels of internal consistency, as evidenced by high Cronbach's alpha scores; and the instrument demonstrated concurrent validity because, as expected, each of the subscales was positively correlated with a measure of marital satisfaction and a measure of relationship optimism at the bivariate level.

Some Unexpected Mixed Results

While findings at the bivariate level were promising, the results of our regression analyses provide some evidence that the measure and the M/CEM model may need more refinement. The results of the regression analyses did support part of the new measure. Specifically, they indicated that three of the four subscales provided a unique contribution to explaining the variance in both marital satisfaction and relationship optimism, as expected. Such findings suggest that the commitment, nurturance/support, and service subscales are conceptually distinct, and each provides unique insights into predicting both marital satisfaction and relationship optimism. The results related to the growth/understanding subscale, however, were inconsistent. As noted above, the subscale was significantly and positively related to the outcome measures at the bivariate level. When entered with the other subscales into the multiple regression equations, however, the growth/understanding measure demonstrated a significant negative relation with each of the outcomes.

The contradictory findings are most likely due to overlap among the growth/understanding measure and the other subscales (Cohen & Cohen, 1983). However, they may indicate a suppression effect in which growth and understanding are detrimental when in the presence of the other three constructs. Further research would be necessary to fully explore this possibility. In either case, however, these regression analyses suggest that this subscale may need further refinement. If further refinement does not address the problem, then we may need to reconceptualize the M/CEM model itself.

Future Directions

As we move forward with refinements of the Marriage and Couples Functioning Measure and the M/CEM model, we can move towards the third step of the program development process as outlined in the introduction to this article. Specifically, we can begin to build a coherent, research-based couples curriculum designed to improve relationship outcomes among program participants. A curriculum based on a model that has been established as related to favorable marital outcomes is more defensible and, we hope, more likely to be effective than one that represents an idiosyncratic summary of recommendations loosely connected to research. We hope that these efforts can help move this field forward towards "evidence-based" programming. This process can mirror some of the efforts that have been made in the field of youth- and family-oriented prevention, as suggested earlier in this article (Greenberg, 2004; Hawkins, Catalano, & Arthur, 2002).

It may also be informative to operationalize and test the NERMEM model that is an expanded version of the M/CEM model. This would facilitate the comparison of key factors in predicting healthy marital functioning.

As this work progresses and a coherent curriculum is developed, we will be able to enter the fourth step of the process, which is program evaluation. Indeed, perhaps the most meaningful test for any curriculum based on the M/CEM model will be an evaluation study using an experimental design. We hypothesize that an engaging, practical, instructionally effective curriculum built on the foundation of M/CEM will produce significant improvements in reported relationship satisfaction and optimism. As we look forward, we plan to use a combination of process evaluations designed to untangle which instructional elements have the greatest impacts, and theory-driven outcome evaluations to examine overall program effectiveness (Adler-Baeder, Kerpelman, Griffin, & Schramm, 2011; Chen, 1987; Patton, 2008). We firmly believe that such a systematic approach to program design and evaluation will help move the entire field of marriage and couple programming forward.

Limitations

As with any empirical investigation, there are several notable limitations associated with the study described here. First, our sample is not necessarily representative of married couples living in all parts of the country. Indeed, we would hesitate before generalizing our findings beyond similar married populations living in the southeastern United States. Second, as mentioned above, the sample only includes married couples. As such, we do not know the degree to which this measure will work for other types of relationships, such as cohabitating but unmarried couples. Third, the study only measures one type of validity. As we continue to refine the Marriage and Couples Functioning Measure, we hope to conduct future validation studies that assess other forms of validity such as predictive, convergent, and discriminant validity.

A final limitation is that the study is only a single incremental step toward more effective marriage education. Several challenging tasks remain, including refining the measure and/or theory, developing the curriculum, and testing it. As we move through these steps, we also will need to consider if marital satisfaction and relationship optimism are the best target outcomes for couple education. Given how our society strives to balance agentic with sociocentric drives, there may be more appropriate outcomes for us to consider.

Conclusions

Despite the above-mentioned limitations, we believe that through the study described here we have made two important contributions to this area of research and practice. First, we have outlined a process that Extension professionals from diverse programming areas can follow to make their own programming more evidence-based. Indeed, by identifying an area of programming need, developing instruments to help assess such need, and finally taking the steps to validate such instruments, we can improve our chances of facilitating positive changes in the lives of our stakeholders. Furthermore, by taking such a systematic approach to the early stages of curriculum development, we set the stage for later theory-driven evaluations, which can help us understand not only if the curriculum worked, but why it succeeded or failed (Chen, 1987).

A second positive outcome of the study is specifically related to the field of couples curricula. Specifically, through the study we describe, we have completed an important step in the process of developing a strong, evidence-based curriculum. Indeed, we have demonstrated that three dimensions of the Marriage and Couples Functioning Measure are both valid and reliable indicators of optimal functioning among married couples. In addition to these findings, the study also provides preliminary data to confirm that the M/CEM model is a defensible foundation for a couples' curriculum. Indeed, despite the inconsistencies outlined earlier, these data suggest that the basic underlying dimensions of the M/CEM outlined in Table 1 are related as expected to both marital satisfaction and relationship optimism at the bivariate level.

As we continue to move this work forward, we will need to examine the inconsistencies that we observed in our analyses. As we make progress on these issues, we will be in a strong position to develop a true evidence-based curriculum that Extension professionals can use to help prevent marital problems and improve functioning among married couples.

References

Adler-Baeder, F., Behnke, A., Brotherson, S., Futris, T. G., Goddard, H. W., Higginbotham, B., et al. (Manuscript in preparation). The national Extension relationship and marriage education model. Athens, GA: University of Georgia Cooperative Extension.

Adler-Baeder, F., Higginbotham, B., & Lamke, L. (2004). Putting empirical knowledge to work: Linking research and programming on marital quality. Family Relations, 53, 537-546.

Adler-Baeder, F., Kerpelman, J., Griffin, M. M., & Schramm, D. G. (2010). Evaluating multiple prevention programs: Methods, results, and lessons learned. Journal of Extension [On-line], 48(6) Article 6FEA1. Available at: http://www.joe.org/joe/2010december/a1.php

Blanchard, V. L., Hawkins, A. J., Baldwin, S. A., & Fawcett, E. B. (2009). Investigating the effects of marriage and relationship education on couples' communication skills: A meta-analytic study. Journal of Family Psychology, 23, 203-214.

Chen, H. T. (1987). The theory-driven approach to validity. Evaluation and Program Planning, 10, 95-103.

Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: L. Erlbaum Associates.

Dunifon, R., Duttweiler, M., Pillemer, K., Tobias, D., & Trochim, W. M. K. (2004). Evidence-based Extension. Journal of Extension [On-line], 42(2) Article 2FEA2. Available at: http://www.joe.org/joe/2004april/a2.php

Fowers, B. J. (2000). Beyond the myth of marital happiness: How embracing the virtues of loyalty, generosity, justice, and courage can strengthen your relationship. San Francisco, CA: Jossey-Bass.

Goddard, H. W., & Olsen, C. S. (2004). Cooperative Extension initiatives in marriage and couples education. Family Relations, 53, 433-439.

Gottman, J. M. (1999). The marriage clinic: A scientifically-based marital therapy. New York: W. W. Norton and Co.

Greenberg, M. T. (2004). Current and future challenges in school-based prevention: The researcher perspective. Prevention Science, 5, 5-13.

Hawkins, A. J., Blanchard, V. L., Baldwin, S. A., & Fawcett, E. B. (2008). Does marriage and relationship education work? A meta-analytic study. Journal of Consulting and Clinical Psychology, 76, 723-734.

Hawkins, J. D., Catalano, R. F., & Arthur, M. W. (2002). Promoting science-based prevention in communities. Addictive Behaviors, 27, 951-976.

Jakubowski, S. F., Milne, E. P., Brunner, H., & Miller, R. B. (2004). A review of empirically supported marital enrichment programs. Family Relations, 53, 528-536.

Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Thousand Oaks, CA: Sage Publications, Inc.

Schumm, W. R., Paff-Bergen, L. A., Hatch, R. C., Obiorah, F. C., Copeland, J. M., Meens, L. D., et al. (1986). Concurrent and discriminant validity of the Kansas Marital Satisfaction Scale. Journal of Marriage and the Family, 48, 381-387.

Small, S. A., Cooney, S. M., & O'Connor, C. (2009). Evidence-informed program improvement: Using principles of effectiveness to enhance the quality and impact of family-based prevention programs. Family Relations, 58, 1-13.

Spoth, R. L. (2008). Translating family-focused prevention science into effective practice: Toward a translational impact paradigm. Current Directions in Psychological Science, 17, 415-421.

Spoth, R.L ., Kavanagh, K. A., & Dishion, T. J. (2002). Family-centered preventive intervention science: Toward benefits to larger populations of children, youth, and families. Prevention Science, 3, 145-152.