Assessing Face Validity of a Physical Activity Questionnaire for Spanish-Speaking Women in California

Introduction

Latinos are increasing in number in the U.S., and the proportion of Latinos in California is expected to nearly equal the number of non-Latino whites in 2020 (Kinsey, 1990). Overall, minority and low-income populations suffer a disproportionate burden of death and disability from cardiovascular disease (United States Department of Health and Human Services [HHS], 2010a). Heart disease is the leading cause of death for women of most racial/ethnic groups in the U.S., including Latina women (HHS, 2010b). Given the high risk of chronic disease in this population and the expression of interest in prevention from the community (Farner, Rhoads, Cutz, & Farner, 2005), interventions focused on diet and physical activity with a preventative approach are needed (Dart, Frable, & Bradley, 2008; Robinson, Anding, Garza, & Hinojosa, 2003).

Many studies have demonstrated a decrease in risk of heart disease and other chronic diseases with an increase in physical activity. A recent literature review indicates a decrease in risk of cardiovascular and heart diseases of 49% with increased physical activity/exercise (Kruk, 2007). To reflect the U.S. Dietary Guidelines 2005 (United States Department of Agriculture [USDA], 2010), the USDA recommended that physical activity promotion be an additional emphasis of its education programs (HHS, 2008). USDA programs include the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) (USDA Food and Nutrition Service, 2009), Head Start (HHS Administration for Children and Families, 2010), and two Extension programs—the Expanded Food and Nutrition Education Program (EFNEP) (USDA, 2009a) and the Supplemental Nutrition Assistance Education Program (SNAP-Ed) (USDA, 2009b) (previously known as the Food Stamp Program).

Development of appropriate survey and evaluation assessment tools for the diverse segments of the target audience is warranted to ensure that language and content is appropriate for each group (Radhakrishna, 2007; Townsend, 2006; Townsend, Sylva, Martin, Metz, & Wooten-Swanson, 2008). For the USDA education programs, the target audience is both English- and Spanish-speaking low-income individuals in rural and urban settings. In the study reported here, we targeted low-income Spanish-speaking women in an urban setting in California.

Currently, no rigorously validated Spanish-language physical activity survey or evaluation tools exist to assess frequency, intensity, and duration of physical activity among low-literacy clients participating in Extension programs. Activity questionnaires should be specific to the cultural group with which they will be used, and wording must be clear to respondents in that particular group (Banna, Vera Becerra, & Townsend, 2008; Townsend, 2006; Townsend et al., 2008). Often, however, these questionnaires are simply translated from English to Spanish for use without further testing. Existing Spanish-language questionnaires that have undergone validation procedures are few. Aside from tools validated in populations in Spain (Elosua et al., 2000; Martınez-Gonzalez, Lopez-Fontana, Varo, Sanchez-Villegas, & Martinez, 2005), virtually no self-administered validated questionnaires for low-income Spanish speakers in the U.S. were identified. Thus, there is a need for properly validated survey and evaluation tools designed for the Latino community in the U.S. (Geisinger, 1994; Warix, Nieto, & Nicolay, 2006). Evaluation tools should have a low respondent burden and should be able to be administered in a group setting (Townsend, 2006; Townsend et al., 2008).

The International Physical Activity Questionnaire (IPAQ) (Craig et al., 2003) was developed to facilitate international physical activity surveillance and gathers information about the frequency, duration, and intensity of activity in a given time period. This article describes the process of adapting the IPAQ for a limited-resource Spanish-speaking audience in California and estimates the tool's resulting face validity, an assessment by clients of how well an item or group of items function for them (Litwin, 1995; Townsend, 2006; Townsend et al., 2008). A future study will further evaluate additional psychometric properties of the tool, including criterion validity and reliability.

Methods

The research reported here received approval from the Institutional Review Board at the [University blinded]. A Flesch-Kincaid analysis of the original self-administered short version of the IPAQ indicated a reading level of grade 8.6, with a reading ease of 64.8 (scores closer to 100 indicate greater ease). To adapt this tool for a low-literacy Spanish-speaking audience, nine steps were taken.

Step 1: Modification of the English Version of the IPAQ to Lower the Reading Level of the Text

Three, four, and five-syllable words were replaced by words with fewer syllables where appropriate. For example, the phrase "vigorous activity" was replaced by "breathe very hard," lowering the reading difficulty and increasing understanding by Spanish-speaking clients.

Step 2: Elimination of Extraneous Words

The original version of the IPAQ, for example, contained examples of the activities in question in the text (i.e., "heavy lifting, digging, aerobics, or fast bicycling"); we eliminated these examples and instead inserted photographs demonstrating examples commonly practiced by this population and using Latino models (described subsequently).

Step 3: Shortening of Sentences

All questions inquiring about number of days in the original IPAQ began with, "During the last 7 days," followed with a description of the activities in question. To shorten sentence length for an improved readability score, we split these items into two sentences. The first in each case is now, "Think about the last 7 days."

Step 4: Translation of Text

Several methods were considered for translating the instrument. Back translation is often the method of choice in adapting measures for culturally diverse audiences. This technique requires translation of items from the original version to the second language and subsequent rendering of items back to the original language by a second translator unfamiliar with the tool (Geisinger, 1994). Several potential problems may arise with this method, however. First, translators who are aware that their work will be subject to back-translation often use wording that unquestionably reproduces the original translation instead of the most appropriate word in the target language. In addition, back translation does not allow for the substitution of item content for cultural reasons. To avoid these problems, another technique involving translation by one party and subsequent review by a panel of bilingual experts familiar with both cultures was used (Geisinger, 1994).

In light of the potential problems associated with back translation, we chose a three-stage method: 1) Translation of the instrument by a visiting Mexican scholar with an advanced degree in nutrition; 2) Cognitive interviewing sessions to determine the preferences and understanding of the target audience (Willis, 1994); 3) Final approval by a group of experts. The original Mexican translator first performed the initial translation, and the questionnaire was then taken to members of the target population and tested for clarity and appropriateness of translation using cognitive interviewing (Willis, 1994). Clarity of text was determined using several strategies, including asking respondents to verbalize their thought process in answering the question, paraphrasing, and probing questions. We selected our target audience, low-income Spanish-speaking women, based on the demographics of the audience in the USDA education programs in California. After the selection of preferred translations by members of the target audience, a panel of experts that included the University's official translator and a professor specializing in questionnaire design reviewed item wording and response options, and further modifications were made. The original translator then reviewed the final version and discrepancies were resolved.

Step 5: Cognitive Testing of Text

Cognitive testing is a form of structured interviewing designed to improve face validity of survey and evaluation tools (Willis, 1994). Willis (1994) developed three strategies that may be used to uncover the cognitive processes that occur as respondents think about and develop answers to survey questions. The first strategy, the concurrent think-aloud technique, requires respondents to verbalize their thought process as they respond to a question. Paraphrasing is the second strategy, which requires respondents to restate the item using their own words in response to this question: "What does this question mean to you in your own words?" The third strategy is the use of probes, a set of questions the interviewer uses to prompt respondents to further elucidate their responses. Examples of probes include: "Is there a better word we could use?" or "Can you think of a better way to ask this question so that it would be clearer to other clients in this EFNEP class?" Our sample for cognitive testing was drawn from sites frequented by low-income Spanish-speaking women, as subsequently described.

Cognitive testing of other aspects of the questionnaire was also conducted. The aforementioned procedures were applied to the title, instructions, and response options. Respondents were asked to verbalize their thoughts with regards to these aspects of the questionnaire, paraphrase when necessary, and respond to probing questions to determine acceptability.

Step 6: Addition of Photographs

For the sixth step, color photographs were taken to reflect physical activity behaviors of Spanish speakers in California. Initial photographs were taken using results of a review of the literature regarding physical activities typical of the population. Subjects used for the photographs were of Hispanic heritage. Color photograph presentation was selected over black and white photographs, black and white drawings, or text only questions based on previous research indicating the preferences of SNAP-Ed and EFNEP clients (Townsend et al., 2008). Client reasons included improved understanding of the behaviors in the question (i.e., cognitive function) (Levie & Lentz, 1982), enhanced readability by reducing the word count (Kincaid, Fishburne, Rogers, & Chissom, 1975), as well as more cues to understanding the text (i.e., compensatory function) (Levie & Lentz, 1982). The cognitive and compensatory functions of the color photographs were particularly important for clients whose primary language was not English (Townsend et al., 2008).

Step 7: Cognitive Testing of Photographs

For the seventh step, photographs were modified or replaced based on feedback from low-income Spanish-speaking respondents. Cognitive testing procedures were also applied to the photographs using the procedures reported by Townsend et al. (Banna, Vera Becerra, Kaiser, & Townsend, 2010; Townsend et al., 2008). Specific probes for photographs included, "What do you see in this photo?" and "Looking at this photo, is there a better way we might illustrate this question for other clients at this SNAP-Ed site?" Given that our goal was an improved readability score compared to a text only version, we also asked, "Are there any words in the question we might remove and illustrate in a photo?" Testing of the photographs revealed the degree to which the clients felt that the photographs were relevant to them and reflected the activities mentioned in the questions (Banna et al., 2010; Townsend et al., 2008).

Cognitive testing sessions took place at the agencies where low-income subjects were recruited, which are frequented by members of our target population of low-literate Spanish speakers. Subjects were recruited only for cognitive testing procedures, and did not participate in any subsequent testing. There were two rounds of interviews, the first in English, and the second in Spanish with a separate set of respondents after translation and with photographs (n=20). The interviewer referred to a pre-prepared interview protocol to provide consistency among interviews (available from first author). All issues were resolved through an iterative interview process (Townsend et al., 2008; Willis, 1994).

After repeated interviews rendered a version of the questionnaire that respondents collectively deemed satisfactory, we then administered the questionnaire in its entirety, without discussion following each question. Respondents then provided feedback about the order of the questions, content, and overall appearance (Townsend et al., 2008).

Subjects for cognitive testing procedures (n=20) were recruited from Head Start sites, WIC clinics, a food bank, and a YMCA day care center. Subjects in the first round of interviews spoke English only, and most subjects in the second round spoke Spanish only. All subjects in the second round spoke Spanish as a first language. Subjects in all interviews were female, and all Spanish-speaking subjects were of Mexican origin.

Step 8: Development of Instruction Guide for Field Staff

To address any questions that might arise during the administration of the checklist in the field and provide consistency in administration of the tool, a comprehensive instruction guide was developed (Banna, Townsend, Davidson, & Leavens, 2008). This was based on a previously tested model (Banna, Townsend, Vera Becerra et al., 2008; Banna et al., 2010; Townsend, Davidson, Leaven, Metz, & Martin, 2006). The guide included a list of questionnaire items, a description of photographs, potential client inquiries regarding items, and answers to inquiries (seventh step). Issues that clients may have with the Spanish translation, photographs, and intended meaning of items, as well as issues that arose during the cognitive testing sessions are addressed within the guide. After initial development of the guide with the help of the Mexican scholar, an expert panel of bilingual individuals familiar with the target population, including Public Health Institute personnel, a registered dietitian, and the university's official translator, reviewed the guide and discrepancies were resolved.

Step 9: Readability Assessment

For the ninth step, the Flesch-Kincaid (Kincaid et al., 1975) and the Fernández-Huerta (Fernández-Huerta, 1959) scores, the equivalent of the Flesch Reading Ease for English text, were used to estimate the reading level of questionnaire text and its reading ease. In the development of these algorithms, elements that were specific to each language were considered. Both formulas consider word length and count the number of syllables per word (Fernández-Huerta, 1959; Kincaid et al., 1975). In addition, the Flesch formula considers number of words per sentence, while the Fernández-Huerta uses number of sentences per 100 words as an indicator of difficulty. Neither formula accounts for the effect of realistic visuals on readability, producing readability scores that overestimate the difficulty of the tool (Townsend et al., 2008). A readability index calculation was performed on the final translation. Both English and Spanish text were included in this version of the questionnaire.

Results

Several broad issues with the text and photographs were discovered through cognitive interviewing procedures. Issues included, 1) variable interpretation of specific terms, 2) ambiguous response options, 3) misunderstanding the instructions, and 4) inadequate photographic accompaniment with text. Examples of each issue are presented in Table 1. A section of the final version of the physical activity behaviors questionnaire for Spanish speakers in California is presented in Figure 1 (Banna & Townsend, 2008).

Table 1.
Cognitive Testing Sessions: Four Issues Identified with One Example Each

Problems Identified by Clients with Translation/ Photo During Cognitive Testing Interviews	Solution Proposed by Clients and/or Experts and Confirmed by Clients	Final Item or Content1
1) Variable interpretation of specific terms
Several items required participants to think about time spent in "moderate" activity. Participants interpreted the word "moderate" variably, with some considering light activities such as watering plants falling in this category.	Instead of using a subjective term, we decided to use physical signs such as breathing a little bit harder than normal accompanying moderate activity to convey the intended meaning.	Think about the last 7 days. On how many days did you breathe a little harder than normal at work? Piense en los últimos 7 dìas. ¿Cuántos dìas hizo actividades que le hicieron respirar algo más fuerte de lo normal en el trabajo?
2) Inappropriate scale for response options
The items addressing sitting provided response options from "0" to "more than 60 minutes." Respondents indicated that it would be very unusual for someone to sit for less than one hour per day. In addition, social desirability was an issue, as respondents felt self-conscious selecting the highest option, "more than 60 minutes."	Changing the scale of the options to reflect typical sitting habits of respondents made the question more reflective of their behaviors and seemed to eliminate the social desirability issue. The format of the response options was also modified to include a continuum instead of bubbles.	Think about the last 7 days at home, at work, and in your spare time. How many hours did you spend sitting on a weekday? [Scale changed to 0 to more than 6 hours] Piense en el tiempo que pasó en casa, en el trabajo y en su tiempo libre en los últimos 7 dìas. ¿Cuántas horas pasó sentado en un dìa en la semana? [Scale changed to 0 to más de 6 horas]
3) Ambiguous instructions
Instructions at the start of the questionnaire required participants to "mark the circle that best answers each question." Thinking about the "best" answer to each question led respondents to think about the ideal physical activity behaviors and how they should respond to each question.	We eliminated the instructions altogether based on the belief that the response procedure would be self-explanatory and that instructions would add confusion.	No instructions included.
4) Inadequate photographic accompaniment with text
One item addressing activity during spare time was accompanied by a woman swimming. Most respondents indicated that they very rarely go swimming, either due to lack of access to facilities or lack of interest.	To include a photograph that would more accurately reflect respondents' leisure activity habits, we gathered information about common activities performed. We found that dancing, using a gym, and playing sports with children were among the most common moderate activities reported.	Current photos contain a couple dancing, a man playing baseball with a child, and a woman bicycling at a gym.
1For items provided, both English and Spanish versions of the same item are listed.

Figure 1.
Two Examples of Items From Physical Activity Behaviors Tool On the Go!/¡De Prisa!

The final version included both English text and the Spanish translation. A Flesch Reading Ease analysis of the English text indicated a reading ease score of 96.3 (with a score of 100 indicating very low difficulty), a considerable improvement from the score of the original English IPAQ of 64.8. The grade level of the final revised English version was 2.0, also improved from the original English IPAQ grade level of 8.6. A readability index calculation of questionnaire text in Spanish using the Fernández-Huerta formula revealed a Huerta Reading Ease score of 98, indicating low reading difficulty. When the Flesch Reading Ease formula was applied to the Spanish text, a difficulty level of 62.1 resulted; however, this formula was designed to analyze English text.

Discussion

The first phase of questionnaire development reported here involved evaluation of face validity of the questionnaire (Nunnally, 1994). In this phase, members of the target audience determined how practical, pertinent, and relevant the questionnaire was for the group (Nunnally, 1994). Cognitive testing has been used in a few studies in nutrition education as a means to assess how respondents react and respond to English text in questionnaires, surveys (Carbone, Campbell, & Honess-Morreale, 2002) and text with visuals in a food behavior checklist (Townsend et al., 2008). We have only seen the method reported for assessment of translations and visuals by Spanish speakers for one tool, a food behavior checklist (Banna et al., 2010).

Cognitive testing procedures resulted in a tool with increased readability and suitability for a low-literacy population (Townsend et al., 2008). The readability index calculation of 98 using the Huerta Reading Ease (HRE) (Fernández-Huerta, 1959) was an improvement from the original Spanish version of the IPAQ, with a HRE score of 81. A considerably different score of 62.1 resulted using the Flesch Reading Ease on the Spanish-language text, indicating that this formula is not appropriate for use with non-English text. Generally, translation into Spanish yields both longer words and sentences, necessitating use of a formula specifically designed for the Spanish language.

The content of the questionnaire reflects the need for evaluation tools for low-literate groups and physical activity goals of USDA's education programs. As inactivity has been identified as an independent risk factor for chronic disease (Heath, 2009), health education programs seek not only to promote engagement in moderate and vigorous tasks, but also to reduce time spent performing sedentary activities (HHS, 2008). In order to adequately assess risk for chronic disease, physical activity questionnaires should encompass the full range of activities, from sedentary to vigorous. Sections addressing time spent sitting, as well as time spent performing activities at moderate and vigorous levels are included in the current questionnaire. Items reflect the aim of the USDA's education programs both to promote physical activity and to reduce inactivity.

Limitations to the research reported here include:

• These findings may not be generalized beyond the sample of low-income Spanish-speaking women from Mexico.


• Further testing is recommended to assess reliability and criterion validity.

Implications based on these results include:

• Phase 1, evaluation of face validity, has been completed. At the completion of Phase 2, which will involve further testing of the questionnaire to assess reliability and criterion validity, the final versions of the physical activity behaviors questionnaire (Banna & Townsend, 2008) and instruction guide (Banna, Townsend, Davidson et al., 2008) will be available for use to evaluate the impact of an education intervention in low-income Spanish speakers in community settings in California.


• The questionnaire may be self-administered in a group setting in a short time period (approximately 15 minutes) with the supervision of one community health worker familiar with the On the Go!/¡De Prisa! instruction guide, allowing EFNEP and SNAP-Ed instructors to focus on the education session.


• Researchers involved in the development of evaluation tools for nutrition education interventions for Spanish speakers may use this process as a model for tool development.

Conclusion

Nutrition educators working in USDA education programs such as EFNEP, SNAP-Ed, WIC, and Head Start may find the description of our process and our observations useful for developing instruments that are culturally appropriate. When further validation of the current questionnaire is completed, we expect that EFNEP, SNAP-Ed, and other nutrition assistance programs will find this tool useful with Spanish-speaking Mexican-American clients in California.

Acknowledgements

The authors thank Luz Elvia Vera Becerra, Joan Gonen, and Myriam Grajales-Hall for their work on questionnaire translation, as well as Lynn-Kai Chao for his assistance with the photographs and graphic design. The authors also thank Larissa Leavens for her work on the instruction guide.

References

Banna, J. C., & Townsend, M. S. (2008). University of California On the Go! ¡De Prisa! [Physical activity (PA) assessment for low-income communities. English and Spanish, 20-item 11-page evaluation tool for Spanish-speaking low-income clients. Contains 5 PA constructs: transport; at home; spare time; sitting at home; and at work.]. Retrieved from: http://townsendlab.ucdavis.edu

Banna, J. C., Townsend, M. S., Davidson, C., & Leavens, L. (2008). Administering the University of California On the Go! ¡De Prisa! [Physical activity (PA) assessment for low-income communities]. Instruction Guide (English & Spanish, 19 pages). Retrieved from: http://townsendlab.ucdavis.edu

Banna, J. C., Townsend, M. S., Vera Becerra, L. E., Fourney, A., Perez, M., Heredia, C., et al. (2008). Administering the Food Stamp Program Lista de hábitos alimenticios (Food Behavior Checklist) for Spanish speakers: Instruction Guide. English and Spanish. Retrieved from: http://townsendlab.ucdavis.edu

Banna, J. C., Vera Becerra, L. E., Kaiser, L. L., & Townsend, M. S. (2010). Using qualitative methods to improve questionnaires for Spanish speakers: Assessing face validity of a food behavior checklist J Am Diet Assoc. 110: 80-90.

Banna, J. C., Vera Becerra, L. E., & Townsend, M. S. (2008). Assessing face validity of a physical activity questionnaire for Spanish speakers in California. Paper presented at the Seventh Annual Conference of the International Society of Behavioral Nutrition & Physical Activity Program and Abstracts; B21:137. Banff, Alberta, Canada.

Carbone, E., Campbell, M., & Honess-Morreale, L. (2002). Use of cognitive interview techniques in the development of nutrition surveys and interactive nutrition messages for low-income populations. J Am Diet Assoc. 102: 690-696.

Craig, C., Marshall, A., Sjostrom, M., Bauman, A., Booth, M., Ainsworth, B., et al. (2003). The International Physical Activity Questionnaire (IPAQ): A comprehensive reliability and validity study in twelve countries. Med Sci Sports Exerc. 35(8): 1381-1395.

Dart, L., Frable, P. J., & Bradley, P. J. (2008). Families and community partners learning together to prevent obesity. Journal of Extension [On-line], 46(1) Article 1IAW2. Available at: http://www.joe.org/joe/2008february/iw2.php

Elosua, R., Garcia, M., Aguilar, A., Molina, L., Covas, M., & Marrugat, J. (2000). Validation of the Minnesota Leisure Time Physical Activity Questionnaire in Spanish women. Med Sci Sports Exerc. 32(8): 1431-1437.

Farner, S., Rhoads, M. E., Cutz, G. & Farner, B. (2005). Assessing the educational needs and interests of the Hispanic population: The role of Extension. Journal of Extension [On-line], 43(4) Article 4RIB2. Available at: http://www.joe.org/joe/2005august/rb2.php

Fernández-Huerta, J. (1959). Medidas sencillas de lecturabilidad (Simple readability measures). Consigna. 214: 29-32.

Geisinger, K. (1994). Cross-cultural normative assessment: Translation and adaptation issues influencing the normative interpretation of assessment instruments. Psychological Assessment. 6(4): 304-312.

Heath, G. (2009). Physical activity transitions and chronic disease. Am J Lifestyle Med. 3(1): 27S-31S.

Kincaid, J., Fishburne, R., Rogers, R., & Chissom, B. (1975). Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. Millington, TN: Naval Technical Training, U. S. Naval Air Station, Memphis, TN.

Kinsey, J. (1990). U.S. demographic trends and their relationship to food markets. St. Paul, Minnesota: University of Minnesota, Institute of Agriculture, Forestry and Home Economics.

Kruk, J. (2007). Physical activity in the prevention of the most frequent chronic diseases: an analysis of the recent evidence. Asian Pac J Cancer Prev. 8(3): 325-338.

Levie, W., & Lentz, R. (1982). Effects of text illustrations: A review of research. Educ Comm Technol J. 30: 195-232.

Litwin, M. S. (1995). How to measure survey reliability and validity. Thousand Oaks, California: Sage Publications.

Martınez-Gonzalez, M., Lopez-Fontana, C., Varo, J., Sanchez-Villegas, A., & Martinez, J. (2005). Validation of the Spanish version of the physical activity questionnaire used in the Nurses' Health Study and the Health Professionals' Follow-up Study. Public Health Nutrition. 8(7): 920-927.

Nunnally, J. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.

Radhakrishna, R. (2007). Tips for developing and testing questionnaires/instruments. Journal of Extension [On-line], 45(1) Article 1TOT2. Available at: http://www.joe.org/joe/2007february/tt2.php

Robinson, S. F., Anding, J., Garza, B., & Hinojosa, I. (2003). Designing nutrition education programs to reach Mexican American populations. Journal of Extension [On-line], 41(1) Article 1IAW2. Available at: http://www.joe.org/joe/2003february/iw2.php

Townsend, M. S. (2006). Evaluating food stamp nutrition education: Process for development and validation of evaluation measures. J Nutr Educ & Behav. 38: 18-24.

Townsend, M. S., Davidson, C., Leaven, L., Metz, D., & Martin, A. (2006). Administering the Food Stamp Program Food Behavior Checklist: Instruction Guide. Retrieved from: http://townsendlab.ucdavis.edu

Townsend, M. S., Kaiser, L. L., Allen, L. H., Joy, A. B., & Murphy, S. P. (2003). Selecting items for a food behavior checklist for a limited resource audience. J Nutr Educ & Behav. 35: 69-82.

Townsend, M. S., Sylva, K., Martin, A., Metz, D., & Wooten-Swanson, P. (2008). Improving readability for an evaluation tool for low-income clients using visual information processing theories. J Nutr Educ Behav. 40: 181-186.

United States Department of Agriculture. (2009a). Expanded Food and Nutrition Education Program (EFNEP). Retrieved from: http://www.csrees.usda.gov/nea/food/efnep/efnep.html

United States Department of Agriculture. (2009b). Supplemental Nutrition Assistance Program (SNAP). Retrieved from: http://www.fns.usda.gov/fsp/snap.htm

United States Department of Agriculture. (2010). Dietary Guidelines for Americans. Retrieved from: http://www.cnpp.usda.gov/dietaryguidelines.htm

United States Department of Agriculture Food and Nutrition Service. (2009). Special Supplemental Nutrition Program for Women, Infants, and Children (WIC). Retrieved from: http://www.fns.usda.gov/wic/

United States Department of Health & Human Services. (2008). HHS Announces Physical Activity Guidelines for Americans. Retrieved from: http://www.health.gov/paguidelines

United States Department of Health & Human Services Administration for Children and Families. (2010). Office of Head Start. Retrieved from: http://www.acf.hhs.gov/programs/ohs/

United States Department of Health & Human Services Centers for Disease Control and Prevention. (2010a). Eliminate Disparities in Cardiovascular Disease. Retrieved from: http://www.cdc.gov/omhd/AMH/factsheets/cardio.htm

United States Department of Health & Human Services Centers for Disease Control and Prevention. (2010b). Women and Heart Disease Fact Sheet. Retrieved from: http://www.cdc.gov/DHDSP/library/pdfs/fs_women_heart.pdf

Warrix, M., Nieto, R., & Nicolay, M. (2006). Developing culturally appropriate evaluation instruments for Hispanics with diabetes. Journal of Extension [On-line], 44(6) Article 6TOT1. Available at: http://www.joe.org/joe/2006december/tt1.php

Willis, G. (1994). Cognitive interviewing and questionnaire design: A training manual (Working Paper Series No. 7). Hyattsville, MD: Centers for Disease Control and Prevention, National Center for Health Statistics.