THE AGING of the US population has led to an extraordinary increase in the prevalence of Alzheimer disease and related dementias (ADRD).1,2 Many individuals with ADRD receive extensive informal care from unpaid family members and friends who themselves can experience poor physical and mental health outcomes associated with their caregiving role. Although findings are mixed, evidence suggests that caregivers may be at greater risk for chronic stress, sleep deprivation (ie, particularly when there is nocturnal wandering by patients or bowel or bladder incontinence), anxiety, depression, physical health conditions, and early mortality.3–11
Studies about caregiving in rural contexts suggest that rural life adds increased complexity to care. Although the circumstances of rural caregivers in certain regions may be gradually improving (eg, where Internet availability and connectivity have been improved), in general, caregivers in rural settings face problems such as fewer available services (or lack of knowledge about available services), fewer health care providers and health education services, difficulties with transportation to formal (eg, memory clinics) and informal services (eg, peer support groups), weather problems in winter, and isolation.12–15 Moreover, rural caregivers from diverse backgrounds are frequently underserved by health and social service systems due to identified barriers such as language, limited financial resources, and lack of culturally effective approaches.16–19
Rural Latinx caregivers tend to provide more hours of care and experience higher stress and poorer physical and mental health than white counterparts.19 They also are less likely to seek formal mental health support.18,20 In addition, Latinx caregivers are more likely than other groups to attribute symptoms of dementia to the normal aging process, which leads to longer delays in diagnosis and a tendency not to seek formal treatment.21,22 Although less is known about the experiences of American Indian/Alaska Native (AI/AN) ADRD family caregivers, studies suggest that extended family is frequently the sole provider of care to dependent older adults in rural and reservation settings.23 Compared with noncaregivers, AI/AN caregivers report experiencing poorer mental and physical health.24,25
Family caregiving presents challenges to diverse rural caregivers. However, research also suggests that Latinx and AI/AN ADRD family caregivers possess culturally–based strengths that may serve as protective factors against poor health outcomes.26,27 For example, Latinx and AI/AN caregivers report more positive aspects of caregiving than other groups, which has been attributed to collective cultures’ respect for elders, shared caregiving tasks, and community support.22,28 Certain tribal groups, such as those living in the United States’ rural Southwest, also value traditional healing methods for both mental and physical health issues, which serve as a complement to biomedical and mental health services.29 Rural Southwestern Latinx populations, particularly older adults who may be less identified with Eurocentric culture, widely use home remedies (estimated at 75%), which few discuss with health care providers.30
Community-based assessments of rural ADRD family caregivers have been undertaken.12,13 However, few have focused on diverse rural ADRD caregivers in a comprehensive way, and even fewer have explicitly assessed culturally–based strengths that may impact their experiences with caregiving.31 To develop targeted supports for rural Latinx and AI/AN caregivers, it is critical that we understand both the needs and strengths of these populations and that we develop this understanding through a systematic, community-informed, partnered approach.32,33 The purpose of this study is to extend the limited body of research focusing on rural white, Latinx, and AI/AN ADRD family caregivers in order to work toward improved health outcomes for these populations. Specifically, we detail the systematic Delphi process our research team used to develop and content-validate a community assessment survey for rural ADRD family caregivers focused on needs, strengths, assets, and resources. We also describe key benefits and challenges associated with developing and validating these tools in diverse rural communities. This information may serve as guidance to other research teams that wish to undertake similar, and often much-needed, community-based projects.
Development of the rural caregiver community assessment survey
The Figure depicts the survey development process. All study activities were approved by the institutional review board (IRB) at Northern Arizona University. The community assessment survey was developed for use with populations in rural Northern Arizona, where 5 of the 6 counties of the region report the poorest health outcomes in Arizona.34 Northern Arizona encompasses almost 66 000 square miles of mostly rural areas that are sparsely populated. Latinx communities comprise approximately 14% of the population, and Native American communities account for 24%.35
The first and second authors maintained detailed research logs documenting the steps in the process of survey development, including benefits and challenges. The foundational survey items were based upon existing instruments that are validated or frequently used in research with diverse and/or rural caregiver populations (Table 1), as well as literature addressing the health needs and priorities of rural older adults in our region of the United States.46 In some cases, items and response options were used as originally written and in other cases, they were tailored to be used with diverse family caregivers in our region. For example, we added questions to the demographic block about place of birth (“Where were you born? American Indian Nation; United States outside of American Indian Nation; Other”) and language (“What language(s) do you speak at home? English; Spanish; Native language such as Navajo, Mojave, Havasupai, or Hopi; Other”). We also added response options to several items that reflected rural life for diverse caregivers. For example, to the standard list of Activities of Daily Living/Instrumental Activities of Daily Living, we included the response options, “Sitting, listening, and talking with them,” and “Attending religious, spiritual or cultural activities.” To questions about transportation to and from health care providers, we included the response option, “I hitchhike,” and to questions about transportation problems that caregivers may experience, we included response options such as, “The car broke down,” “Someone else needed to use the car,” “I don’t have a driver’s license,” and “I couldn’t afford gas.” Survey constructs were grouped together into sections according to guidelines described by Dillman et al.47
Foundational Survey Instruments
|Caregiver burden||Zarit Burden Interview—Short Version (ZBI-12)||Bédard et al (2001)36||12||5-point scale ranging from 0 (never) to 4 (nearly always)||Designed to assess the impact caregiving has had on caregivers’ lives.|
|Rewards||Positive Aspects of Caregiving Scale (PAC)||Tarlow et al (2004)37||9||5-point scale ranging from 1 (disagree a lot) to 5 (agree a lot)||Designed to assess perception of benefits within the caregiving context. Two domains Self-Affirmation and Outlook on Life.|
|Social support||Interpersonal Support Evaluation List—12 (ISEL-12)||Cohen et al (1985)38||12||4-point scale ranging from 0 (definitely false) to 3 (definitely true)||Designed to assess the perceived availability of social support. Three subscales: Appraisal, Belonging, and Tangible.|
|Anxiety||Generalized Anxiety Disorder—7 Scale (GAD-7)||Spitzer et al (2006)39||7||4-point scale ranging from 0 (not at all) to 3 (nearly every day)||Designed to assess anxiety according to the criterion as identified in the DSM.|
|Depression||Patient Health Questionnaire (PHQ-9; PHQ-2)||Kroenke et al (2001)40||9||4-point scale ranging from 0 (not at all) to 3 (nearly every day)||Designed to assess depression according to the core criterion as identified in the DSM.|
|Kroenke et al (2003)41||2|
|Health status||Self-Rated Health (SRH)||Ware et al (1996)42||12||5-point scale ranging from 1 (excellent) to 5 (poor).||Designed to assess general health status.|
|Sleep||Pittsburgh Sleep Quality Index (PSQI)||Buysse et al (1989)43||9||Open-ended format or a 5-point scale (varying anchors dependent on question)||Designed to assess quality and patterns of sleep in older adults.|
|Stress||Perceived Stress Scale (PSS)||Cohen et al (1983)44||10||5-point scale ranging from 0 (never) to 4 (very often)||Designed to measure overall perceived stress experienced in the past 30 d.|
|Cultural perceptions||Cultural Justification for Caregiving Scale (CJCS)||Dilworth-Anderson et al (2005)45||10||4-point scale ranging from 1 (strongly disagree) to 4 (strongly agree)||Designed to assess a caregiver’s cultural reasons and expectations for providing care.|
Abbreviation: DSM, Diagnostic and Statistical Manual of Mental Disorders.
Expert panel participants and Delphi process
There is no consensus on sample size for a Delphi process, but some have suggested that at least 10 well-informed respondents are sufficient.48 The goal for the expert panel in our study was to include 10 rural White, rural Latinx, and rural AI/AN policy makers (n = 2), health care providers (n = 4), and ADRD family caregivers (n =4). We anticipated that this would provide a sufficient diversity of opinions about the survey and, at the same time, allow the quantitative and qualitative feedback about needed revisions to be manageable and actionable. Fifteen individuals were nominated to serve on the panel, 11 were invited, and 9 ultimately chose to participate in the modified Delphi process (3 policy makers, 2 health care providers, and 4 family caregivers).49,50 The selection of panelists was based upon the research team’s knowledge of their expertise, their responsiveness and availability, their willingness to provide critical and thorough feedback, and their representation of 3 levels of community impact on caregiving (ie, policy, health care, and caregivers themselves).
Of the 9 final panelists, 8 were female and 1 was male. Four indicated that they were White, 1 Hispanic/Latinx, and 4 AI/AN. Policy makers and providers ranged from 37 to 65 years of age and each held at least a bachelor’s degree. Policy makers were trained in fields such as sociology or psychology. The 2 providers were educated as advanced nurse practitioners. Family caregivers ranged in age from 47 to 74 years and had been providing care for their loved one for between 21 months and 11 years. Three of the family caregivers were adult children of an individual with ADRD and 1 was a spouse. Panelists received a $50 gift card in appreciation for their time, although some panelists declined the gift card.
Round 1 of the Delphi process took place between September and October of 2019. All communication with the expert panel took place by telephone, text, electronic mail, or hard copy mail, without face-to-face meetings. Our 6-person multidisciplinary, multicultural research team, through reflection and group discussions, developed a paper/PDF format rating form for Delphi members to complete. The rating form included a 5-point Likert scale from strongly disagree (1), disagree (2), neutral (3), agree (4), to strongly agree (5), and panelists rated the degree to which each section of the survey was (1) inclusive of different cultural groups, (2) respectful of cultural values and norms, (3) comprehensive with respect to needs, assets, and resources, and (4) relevant to the lived experiences of diverse rural caregivers. Panelists also provided demographic information and rich qualitative feedback about modifying phrases to make the assessment more relevant to diverse rural respondents, removing some sections altogether, and adding items and response options to better reflect rural life. Six of the 9 rating forms were delivered and received as PDFs by electronic mail, with the other forms being delivered and received as hard copies.
After completed rating forms were received, comments were organized by theme into a Microsoft Word document. Sections in which fewer than 75% of experts indicated agree or strongly agree on any of the 4 criteria were scrutinized and improved on the basis of panelist feedback, within the broader goal of shortening the overall survey to decrease burden on community respondents and enhance its overall utility.
Round 2 of the Delphi process took place in November 2019 and included 6 panelists. Three of the original 9 panelists did not respond to repeated requests for feedback. To minimize the burden on panelists, this round consisted of an abbreviated survey containing only the 2 sections (Cultural Customs, Beliefs, and Traditions; Provider Interactions) that fell below the 75% agreement cutoff, averaged across the 4 evaluation criteria. We also provided panelists with a narrative description of revisions that were made to the other sections, based upon the first round of feedback, to inform their review of the revised survey. We discontinued the Delphi process after all sections received at least 75% agreement, averaged across the 4 criteria. Panelist feedback in round 2 was used to make additional improvements to the sections under consideration, as well as to the overall survey.
Finalizing the rural caregiver community assessment survey
Two research team members with personal and professional expertise with rural white, rural Latinx, and rural AI/AN ADRD family caregivers in our region reviewed the survey once more for language and flow. The survey was then translated into Spanish by a professional translation service. The translated version was reviewed by team members with conversational Spanish skills and familiarity with regional culture, idioms, and colloquialisms, as well as verified by 2 additional reviewers who were native Spanish speakers in order to ensure that the items on the Spanish version were accurate and comparable in meaning to the original (English) version and that they reflected regional dialects. After a small pilot test of the survey, final adjustments were made to the English and Spanish versions before submitting for IRB approval for the implementation phase of the study. Institutional review board–approved content was used to create paper booklet format and Qualtrics e-format versions of the surveys for implementation in the 6-county region of Arizona.
The Delphi process resulted in significant changes to the survey, including complete elimination of 2 sections (Social Support, Neighborhood Characteristics). Social Support items were trimmed and incorporated into other sections of the survey. Neighborhood Characteristic items were eliminated and additional open-ended questions about neighborhood factors were added to a focus group guide used in an associated qualitative study, which served as a complement to the community assessment survey.
Based upon panelist feedback, we focused more on transportation issues in rural areas including distance to health care providers (eg, “About how long does it usually take you to travel to the nearest medical provider? Answer hours/minutes for each type of transportation you use: Car ride; Bus ride; Bike ride; Walking; Hitchhiking; Other”), asked about the living situation of the person with ADRD and, if applicable, the distance from that person’s home to the caregiver’s home (eg, “Does the person with dementia live in the same home as you? If yes, are there other people in your household who provide regular unpaid care for the person with Alzheimer’s or dementia? If no, about how far do they live from you (in miles)?”), and included questions about long-term care (including the ability to “age in place”) in rural areas (eg, “On a scale of 1 [I feel like I can’t get any help at all] to 5 [I feel like I can get all the help I need], please circle how much you feel that you can get specific types of help for yourself and the person with Alzheimer’s or dementia: Emotional help (e.g., someone to listen to you, someone who knows what you are going through; Physical help (e.g., lifting and transferring), financial help, help with transportation, temporary relief from caregiving, etc.”). We added questions about challenges of obtaining and managing medications for the person with ADRD, which has been reported in other studies as a concern among rural older adults,46 as well as questions about caregiver health behaviors (eg, nutrition, exercise, substance use) that may be more problematic in rural areas.51–55 We also included more questions about the roles that immediate and extended family members have in providing care (“How many people do you have near that you can readily count on for help in times of trouble or difficulty?”), as well as respondents’ perceptions about culture and caregiving (eg, “Do you consider yourself a member of a particular religious, spiritual, cultural, or traditional group?”; “Do you feel that your spiritual, cultural, or traditional practices help you deal with your caregiving responsibilities?”).
Whenever possible, we incorporated panelist feedback. However, because of the study aims and/or regulatory issues, it was not feasible to honor some suggestions. For example, we were not able to survey ADRD caregivers who were children, structure the survey so that it could accommodate caregivers with multiple care recipients, or ascertain specific AI/AN tribal affiliation due to time and IRB constraints. Panelists also had several excellent suggestions about open-ended questions for the survey (eg, “Is talking about dementia OK? Small towns come with challenges of word getting out.”; “How do their elders perceive dementia?”; “In the rural-most regions, how did they overcome challenges to receive supports?”). However, due to space limitations, we opted to include these in the focus group guide instead.
The final survey consists of 8 sections (1. Caregiver Characteristics [14 items]; 2. Care Receiver Characteristics [13 items]; 3. Type of Care Provided [7 items]; 4. Information and Access to Support [9 items]; 5. Caregiving Experience [18 items]; 6. Caregiver Health [28 items]; 7. Cultural Customs, Beliefs, and Traditions [26 items]; and 8. Provider Interactions [16 items]) with 131 total items. Data from paper and e-surveys are entered into a Research Electronic Data Capture (REDCap) database and monitored according to race/ethnicity and respondent zip code to ensure that we are achieving a representative geographic view of rural caregiver needs and strengths. Metadata (eg, average time to complete the survey, geolocation) available through Qualtrics for completed e-surveys indicate that, on average, the survey requires approximately 57 minutes to complete. Readers are encouraged to reach out to the study authors with questions or requests for a copy of the survey items.
Final expert panel ratings of the survey content are presented in Table 2. Of the 4 evaluation criteria, on average, panelists had the highest level of agreement about the extent to which the survey was inclusive of different cultural groups (92.9%) and respectful of cultural values and norms (89.9%). There was less agreement about the extent to which the survey was comprehensive with respect to needs, assets, and resources (85.2%), and relevant to the lived experiences of diverse rural caregivers (75.4%). The sections on type of care provided and information and access to supports were rated as less comprehensive (66.7% and 71.4%, respectively) and the sections on caregiver characteristics and care receiver characteristics were rated as less relevant (both at 66.7%). The sections on cultural customs, beliefs, and traditions and provider interactions received the highest average ratings from panelists across all 4 criteria (both at 100%). However, considering the revisions that were made to the 6 sections that were not included in the second formal review, more sections may have approached 100% agree/strongly disagree had they been included in the round 2 review.
Expert Panel Ratings of the Rural Caregiver Community Assessment Survey (N = 9)
|Criteria||Inclusive||Respectful||Comprehensive||Relevant||Average % A/SA Across Criteria|
|Modal Response||% A/SA||Modal Response||% A/SA||Modal Response||% A/SA||Modal Response||% A/SA|
|1. Caregiver Characteristics||A||88.9||A||88.9||A||88.9||A||66.7||83.4|
|2. Care Receiver Characteristics||A||88.9||A||77.8||A||77.8||A/SA||66.7||77.8|
|3. Type of Care Provided||A||88.9||A||77.8||A||66.7||A||75.0||77.1|
|4. Information and Access to Supports||A||87.5||A||85.7||SA||71.4||A||85.7||82.6|
|5. Caregiving Experiences||A||88.9||A||88.9||A||88.9||A||88.9||88.9|
|6. Caregiver Health||A||100||A||100||A||87.5||A||100||96.9|
|7. Cultural Customs, Beliefs, and Traditions||A/SA||100||A/SA||100||A/SA||100||SA||100||100|
|8. Provider Interactions||A/SA||100||SA||100||A/SA||100||A/SA||100||100|
|Average % A/SA across sections||92.9||89.9||85.2||75.4||88.3|
Abbreviations: A, agree; SA, strongly agree.
Benefits and challenges
The research team identified key benefits and challenges of the community assessment survey development process. Benefits included raising awareness in the community about the need for such an assessment, as well as about opportunities to participate in the community assessment itself. This awareness facilitated conversations and partnerships with community agencies and subsequent participant recruitment for the survey study. The development process also served to maximize data quality by ensuring that the survey was as inclusive, respectful, comprehensive, and relevant to diverse rural populations as possible. Challenges included the time and communication-intensive nature of the methods (eg, for panelists who were not comfortable using electronic mail) and difficulties associated with determining the scope of the survey so that it could include panelist input but still be manageable in length and rigorous from a research standpoint. These results are discussed in greater detail later.
Studies have described the needs of rural ADRD caregivers,12,13 although little research has been conducted with diverse rural caregivers, including the strengths of these populations that may be leveraged to improve health outcomes.31 Furthermore, few published studies provide clear guidance about how to develop and validate strengths-based community survey assessments for diverse rural populations. This article provides details about the systematic Delphi process our research team used to develop and content-validate a community assessment survey for rural white, rural Latinx, and rural AI/AN ADRD family caregivers that focuses on needs, as well as culturally–based strengths, assets, and resources.
The survey development and validation methods our team used were time intensive. Key activities included identifying suitable panelists (ie, those with a rural and/or diverse perspective, with the time and capacity to review survey items and provide critical feedback), developing those relationships, creating the Delphi protocol and forms, securing IRB approval, communicating with panelists throughout the process, and incorporating panelist feedback in an iterative way. This is consistent with other community-based health research projects conducted with diverse rural populations.32 Compared with developing the survey strictly within the research team, however, we think that our methods conferred a variety of benefits to the community assessment. For example, the process of identifying and recruiting expert panelists raised awareness among the panelists themselves, their community organizations, and their community constituents more broadly about the need for a systematic investigation into community needs and strengths. It also raised awareness about the community assessment itself, which facilitated recruitment for the survey, as well as for the complementary focus group study. Perhaps most importantly, we believe that the development process we used increased the overall quality of the survey. In response to panelists’ ratings and qualitative feedback, we made a number of improvements to the survey to ensure that it would be optimally inclusive, respectful, comprehensive, and relevant to diverse rural populations. We anticipate that these improvements will promote diversity within our sample, as well as encourage respondents to be invested in the study and forthcoming about their experiences as caregivers.
There also were challenges associated with the development and validation process. First, some panelists did not have access to or were not comfortable with using e-mail. For those panelists, delivery of study materials and study-related communications took place by hard copy and telephone. Similar challenges have been reported in community engagement projects with other rural caregiver populations.56,57 Second, it was difficult to narrow the scope of the assessment and of the survey itself to a manageable number of items that would allow us to achieve our aims without overburdening community respondents. Related to this problem, we found it challenging to balance the occasionally competing goals of developing an assessment that would be easy for respondents to complete, useful for understanding community needs and strengths, and also rigorous from a traditional research standpoint.
Others have reflected upon this issue in the context of evaluating clinical interventions. Some have suggested that the social validity (ie, “The acceptability of and satisfaction with [measures], usually assessed by soliciting opinions from the people who receive and implement them.”58(p1406)) of measures be given equal weight to traditional research validity when conducting assessments, particularly with diverse populations.59–61 In the end, our team prioritized ease of use and utility over traditional research rigor. However, the final survey retained a number of established scales/subscales that will provide data with which to move empirical knowledge forward in the field of rural family caregiving.
This study has limitations that should be acknowledged. First, although we had a diversity of expert panelists in terms of race/ethnicity and role (policy makers, providers, family caregivers), all but one of the panelists were female. It would have been valuable to include more male panelists, as studies suggest that gender may influence perceptions about health62,63 and, thus, it may influence perceptions of community health assessments. Second, for the expert panelists who were family caregivers, we did not collect data about the severity of their loved one’s ADRD including the type and intensity of the care they provided. These factors may have impacted these panelists’ perceptions of the survey on criteria such as comprehensiveness and relevance to family caregiving. Finally, we have not yet conducted a psychometric analysis of the revised survey and, thus, cannot be sure of the extent to which the new items compare with the original items from the standardized measures in terms of reliability and validity. As we continue to collect and work with these data, conducting this analysis will be a high priority.
Examining the development and validation of community assessment tools is an important step in the creation of meaningful, targeted supports to promote the health and well-being of diverse rural ADRD family caregivers. This article described the process that was used to develop and validate the content of one such tool that focuses on community needs, as well as culturally based strengths, assets, and resources. We also provided a discussion of the benefits and challenges associated with conducting this work in rural communities. Through continued efforts such as this, health outcomes among diverse rural ADRD family caregivers may eventually be improved.