RAC Part One

Description

Please read the article and answer the questions

Don't use plagiarized sources. Get Your Custom Assignment on
RAC Part One
From as Little as $13/Page

Unformatted Attachment Preview

Nembhard et al. BMC Health Services Research
https://doi.org/10.1186/s12913-020-4986-0
(2020) 20:137
RESEARCH ARTICLE
Open Access
A quasi-experiment assessing the sixmonths effects of a nurse care coordination
program on patient care experiences and
clinician teamwork in community health
centers
Ingrid M. Nembhard1* , Eugenia Buta2, Yuna S. H. Lee3, Daren Anderson4, Ianita Zlateva4 and Paul D. Cleary5
Abstract
Background: Recognition that coordination among healthcare providers is associated with better quality of care
and lower costs has increased interest in interventions designed to improve care coordination. One intervention is
to add care coordination to nurses’ role in a formal way. Little is known about effects of this approach, which tends
to be pursued by small organizations and those in lower-resource settings. We assessed effects of this approach on
care experiences of high-risk patients (those most in need of care coordination) and clinician teamwork during the
first 6 months of use.
Methods: We conducted a quasi-experimental study using a clustered, controlled pre-post design. Changes in staff
and patient experiences at six community health center practice locations that introduced the added-role approach
for high-risk patients were compared to changes in six locations without the program in the same health system. In
the pre-period (6 months before intervention training) and post-period (about 6 months after intervention launch,
following 3 months of training), we surveyed clinical staff (N = 171) and program-qualifying patients (3007 preperiod; 2101 post-period, including 113 who were enrolled during the program’s first 6 months). Difference-indifferences models examined study outcomes: patient reports about care experiences and clinician-reported
teamwork. We assessed frequency of patient office visits to validate access and implementation, and contextual
factors (training, resources, and compatibility with other work) that might explain results.
Results: Patient care experiences across all high-risk patients did not improve significantly (p > 0.05). They improved
somewhat for program enrollees, 5% above baseline reports (p = 0.07). Staff-perceived teamwork did not change
significantly (p = 0.12). Office visits increased significantly for enrolled patients (p < 0.001), affirming program implementation (greater accessing of care). Contextual factors were not reported as problematic, except that 41% of nurses reported incompatibility between care coordination and other job demands. Over 75% of nurses reported adequate training and resources. Conclusions: There were some positive effects of adding care coordination to nurses’ role within 6 months of implementation, suggesting value in this improvement strategy. Addressing compatibility between coordination and other job demands is important when implementing this approach to coordination. Keywords: Nurse care coordination, Patient care experience, Office visit frequency, Teamwork * Correspondence: [email protected] 1 The Wharton School, University of Pennsylvania, Health Care Management Department, 3641 Locust Walk, 207 Colonial Penn Center, Philadelphia, PA 19104, USA Full list of author information is available at the end of the article © The Author(s). 2020 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Nembhard et al. BMC Health Services Research (2020) 20:137 Background Efforts to improve care coordination have increased in recent years due to the recognition that coordination is a central part of high quality care yet is often less than optimal in healthcare [1–4]. Care coordination refers to “the deliberate organization of patient care activities between two or more participants (including the patient) involved in a patient’s care to facilitate the appropriate delivery of healthcare services” ([5], p., 5]). In the United States (U.S.), 35% of patients with serious illnesses or chronic conditions report having experienced a coordination failure [6]. Such failures have resulted in medical complications, preventable hospitalizations, duplicative testing, and morbidity increases [4] estimated to cost the U.S. healthcare system $25 to 45 billion in 2011 [7, 8]. Other countries (e.g. Canada, France, Norway, Sweden, and Switzerland) struggle with care coordination as well, where 30% or more of patients report experiencing coordination failures [6]. To improve care coordination in several countries [4, 9–11], many health systems and organizations have implemented or are implementing nurse care coordination initiatives in which nurses provide additional care and support to patients with coordination needs such as those with a chronic illness, transitioning from hospital to home, or with multiple medical and behavioral health issues [12–14]. In this approach, nurses work closely with designated patients and providers to coordinate multi-specialty care and help patients manage their illnesses. Core responsibilities in this role include monitoring patient health and facilitating development, communication, and delivery of care plans with other care team members [15, 16]. Nurses in many organizations perform these activities, which are within their scope of practice [17, 18]. New programs have structured these activities, clarifying authority, tasks, options, and responsibility, to enhance nurses’ visibility, effectiveness, and efficiency as coordinators. These programs address calls from professional and scientific groups for nurse coordinator roles to be more explicit, developed, and designed deliberately into training and delivery organizations [15, 18–20]. These programs should lead to better experiences for patients and clinicians because assignment of responsibility to one person and coordination improve the logic, continuity, and efficiency of care [5, 12, 18, 21–23]. Currently, two approaches to nurse care coordination are common. In the first, adopted mainly in large health systems and medical groups, a nurse serves exclusively as care coordinator for a panel of patients. This “exclusive-role approach,” has been used, for example, by participants in the U.S. Medicare Care Coordination Demonstration program [24]. In the second “added-role approach”, a nurse performs care coordination in Page 2 of 14 addition to existing responsibilities. Although reviews of research on the first approach indicate mixed results [25–27], increasing evidence from controlled trials indicates that patients with these coordinators experience better technical quality of care, lower hospital readmissions, lower costs, and better care experiences (e.g., provider-patient communication) than patients who do not have a coordinator [12, 17, 28–37]. Little is known about the experiences of patients served by nurses in the added-role approach, which may be pursued more by smaller organizations or those in lower-resource settings, which are many of the settings across the world. There is also limited information, particularly in primary care settings about the effectiveness of this approach, even though these settings are increasingly expected to coordinate care with patients and other providers [38]. In primary care settings such as federally qualified health centers (FQHCs) in the U.S., a type of community health center that serves disproportionately more complex patients with multiple co-morbidities and socioeconomic disadvantages than do private practices and health systems [39], the imperative for coordination is especially great, but there is little evidence about the effects of adding care coordination to the nursing role. Nurses dedicating any increased attention to these tasks may be positive for patients in need and clinicians. On the other hand, the potential positive effects of the added-role approach may not be realized because of the inability to focus exclusively on coordination tasks. In this manuscript, we examine the early (six-month) effects of a nurse care coordination program in FQHC practices that use the added-role approach for high-risk patients using two measures: care experiences of these patients and clinician-reported teamwork. High-risk patients have complex and/or multiple medical and psychosocial problems, which may require them to see as many as 16 physicians per year, making them most in need of care coordination, most at risk for coordination failures, and most likely to benefit from care coordination, [1, 40] although recent studies suggest that benefits may extend beyond this group [41]. We also examine an indicator of implementation effectiveness, the frequency of patient office visits, and contextual factors because they can influence implementation, and thus program outcomes [42]. We focus on effects in the program’s first 6 months because early experiences with a program are often consequential for long-term success [43–46]. Also, departure from past patterns is often salient to participants early, before they become accustomed to new patterns and adjust expectations, [47] making early assessments a window into program functioning. Currently, there is limited investigation of the early effects of nurse care coordination programs, leaving organizations with little Nembhard et al. BMC Health Services Research (2020) 20:137 Page 3 of 14 knowledge about what to expect. Research on other patient-nurse and coordination interventions in other settings (e.g., skilled nursing visits in home health care [48–50]) suggests that positive effects can materialize in 6 months. Methods Study setting and design This study was conducted in a statewide, multi-site FQHC with 12 sites that provide comprehensive primary medical, dental, and behavioral healthcare services to over 140,000 patients a year. The center serves patients with all types of primary care needs and emphasizes serving the uninsured, underinsured, and special populations such as patients with HIV/AIDS, diabetes, and chronic mental health issues. The FQHC has been recognized as a Primary Care Medical Home by the Joint Commission [51] and a level 3 Patient-Centered Medical Home by the National Commission on Quality Assurance [52]. Thus, each site has demonstrated commitment to patient-centered care, comprehensive care, coordinated care, access to care, and a systems-based approach to quality and safety. We conducted a cluster quasi-experiment in which pre-post intervention changes in clinician and patient experiences in six sites (clusters) that introduced a nurse care coordination program for high-risk patients using the added-role approach (“intervention group”) were compared to changes in experiences in six sites without the program at the time of our study (“comparison group”). Sites in the comparison group implemented the program after our data collection. The FQHC used a sequential roll-out plan (all locations (3) in one county every 3 months) as it does for certain large-scale initiatives for operational reasons (e.g., maintaining crosscoverage between providers in county and having sufficient resources for implementation). When deciding about comparison sites, the FQHC's leadership selected pairs of sites that were relatively similar based on number of patients, patient population profile, and the organization of sites. Sites were allocated to the intervention group if the intervention could begin sooner there than at pair site, given staff work and training schedules, etc. The selected intervention and comparison sites were similar at baseline and follow-up on all but two characteristics for which we could obtain data Table 1 Comparison of Intervention and Comparison Groups’ Characteristics at Baseline and Follow-up Baseline (Median[range]) Characteristics Intervention Centers (N = 6) Follow-up (Median[range]) Comparison Total PIntervention value* Centers (N = 6) Comparison Centers (N = 6) Total Centers (N = 6) Centers (N = 12) 501 [333; 511] 443 [301; 528] 450 [301; 528] 0.20 918 [631; 1114] 841 [611; 1000] 841 [611; 1114] 0.42 Medicaid patients (%) 71 [59; 74] 64 [55; 81] 68 [55; 81] 0.75 72 [60; 74] 65 [56; 81] 68 [56; 81] 0.94 Medicare patients (%) 8 [4; 10] 13 [9; 17] 10 [4; 17] 0.01 8 [5; 10] 12 [8; 15] 9 [5; 15] 0.02 Private insurance patients (%) 10 [9; 12] 12 [7; 20] 11 [7; 20] 0.42 11 [8; 12] 12 [7; 18] 11 [7; 18] 0.26 Uninsured patients (%) 9 [3; 23] 6 [2; 18] 6 [2; 23] 0.26 9 [3; 22] 6 [2; 16] 7 [2; 22] 0.29 White patients (%) 28 [13; 66] 41 [27; 65] 32 [13; 66] 0.34 27 [13; 64] 38 [25; 63] 31 [13; 64] 0.38 Black patients (%) 8 [7; 20] 15 [2; 22] 12 [2; 22] 0.52 8 [5; 19] 12 [2; 18] 10 [2; 19] 0.87 Hispanic patients (%) 49 [12; 64] 34 [18; 58] 43 [12; 64] 0.26 50 [13; 64] 34 [18; 59] 43 [13; 64] 0.30 Other race patients (%) 4 [2; 5] 5 [4; 12] 4 [2; 12] 0.05 4 [2; 7] 6 [3; 22] 4 [2; 22] 0.20 Race unknown patients (%) Number of patient visits per full-time employee in 6-month period Centers (N = 12) Pvalue* Patient insurance status Patient race 4 [2; 10] 2 [1; 11] 4 [1; 11] 0.19 5 [3; 10] 4 [2; 12] 4 [2; 12] 0.51 Patients eligible for care coordination^ 330 [114; 1396] 745 [162; 1538] 410 [114; 1538] 0.63 . . . . Productivity indicator 1.21 [1.04; 1.37] 1.00 [0.96; 1.19] 1.12 [0.96; 0.06 1.37] 1.23 [1.02; 1.28] 1.05 [0.81; 1.16] 1.06 [0.81; 0.07 1.28] Supervisor support for staff, indicative of work climate (staff reported, 1–4 scale)~ 3.60 [2.96; 3.80] 3.67 [3.49; 3.84] 3.66 [2.96; 0.57 3.84] 3.37 [3.23; 3.88] 3.57 [3.23; 3.93] 3.56 [3.23; 0.57 3.93] ^ Baseline values apply to follow-up period as well because the starting sample of eligible patients remained the focus throughout the study. *p-value from Wilcoxon rank-sum tests comparing intervention to comparison centers. ~Supervisor support measured by 5 items from the FQHC’s staff survey Nembhard et al. BMC Health Services Research (2020) 20:137 (Table 1). Wilcoxon rank-sum tests indicated that the groups differed significantly with respect to percent of patients with Medicare as their health insurer (p = 0.02 and p = 0.01 at baseline and follow-up, respectively) and percent of patients with “other race” (p = 0.05 at baseline). We adjust for these differences in our analyses. Our primary study outcomes were two indicators of program effectiveness: patient reports about their care experiences and clinician reports of teamwork in their centers. If care coordination programs function as intended, patient experiences, as reflected in responses to questions about care coordination, timeliness of care, and support for self-management should improve, as should clinician teamwork. Because degree and fidelity of program implementation are critical determinants of program effectiveness, we collected the implementation information that we could, given resource limitations and concerns about staff burden. We obtained information about numbers of telephone calls to patients, but those data turned out to be inconsistent and of poor quality and so are not presented. The other measure of program implementation that we have is the number of patient office visits, which is a proxy measure of accessibility of care, engagement with patients, monitoring, and follow-up to achieve care plan goals (e.g., condition controlled, no preventable hospitalization). If the care coordination program was implemented as intended, there should be an increase in patient office visits in the early months of the program to address outstanding patients’ care needs and self-management training. Research on programs that incorporate the exclusiverole approach has found that primary care office visits increase with coordination programs in the first 2 years, while emergency department visits decline for high utilizers [53]. Over a longer period, not covered by this study, office visits should decline due to better patient health and self-management. Because implementation and effectiveness are often influenced by resources, training, and compatibility with current work, [42, 54, 55] we also assessed these contextual factors via nurse surveys, because these factors may help explain our results. Other non-program specific contextual factors (e.g., employee workload, patient population profile, and supervisor support for workers, which shapes work climate) were examined as well (Table 1). Intervention In intervention sites, every nurse’s role was expanded to include care coordination for adult patients who were expected to benefit most from this effort. These were defined by the organization as patients who were 18 years of age or older, had two or more visits with a primary care provider (PCP) in the past 12 months, and had been identified as “high risk.” Patients were Page 4 of 14 classified as high risk if they had: 1) two or more emergency room visits in the past 12 months; 2) one or more hospitalizations in the past 12 months; 3) a Type 2 diabetes diagnosis on their problem list and a hemoglobin A1C test in the past 12 months greater than 9%; 4) a diagnosis of persistent asthma diagnosis on problem list and two or more asthma control test scores < 19 in the past 12 months; or 5) four or more of specified chronic illnesses on their active problem list, including Type 2 diabetes, chronic obstructive pulmonary disease, hypertension, asthma, coronary artery disease, or behavioral health diagnosis. A subset of the eligible patients (those with greatest immediate need as perceived by staff) was enrolled in the program at the outset due to time and resource constraints. Other patients were also enrolled when a PCP or nurse identified the patient as needing care coordination (e.g., newly discharged from a hospital). As part of the new program, nurses were expected to work with enrolled patients to help them navigate their healthcare and lead a weekly panel management meeting with enrollees’ PCP and behavioral health provider. The sessions were to be used to review patient progress, identify additional patients who needed coordination, and plan coordinated care. To implement the program, the organization introduced the nurse care coordinator role to all staff via meetings and other communications (e.g., newsletters). It also provided three resources to nurses to support their effectiveness as coordinators: training, a “playbook”, and an electronic dashboard. All nurses in the intervention sites received 23 h of training over a period of 2 to 3 months from experts within the organization and outside consultants. The training covered care plan development, panel management, documentation, transition care support, motivational interviewing, self-management goal setting, chronic disease management, and behavioral health disorders — evidence-based components of nurse care coordination [17]. The playbook provided instructions for each task within the new nurse role, information on additional resources, and measures to evaluate performance. The electronic dashboard leveraged information in the organization’s electronic health record system, which aided nurse tracking of patients and activities. No other group was assigned care coordination responsibilities. The organization reinforced its commitment to the role change by monitoring nurse performance and providing feedback reports to nurses. It was expected that the program would lead to more coordinated and timely care, greater patient support for self-management, and care for mental health. Study outcomes Patient care experiences We collected patient surveys that asked about care experiences during two periods at each center. The first Nembhard et al. BMC Health Services Research (2020) 20:137 (baseline) period covered the 6 months prior to start of nurse training in the intervention centers, and was before nurses were told about the intervention and patients who would be in the program were known. In intervention and comparison centers, we invited a random sample of the high-risk (i.e., program-eligible) patients described earlier that had visited the center in the preceding 6 months (N = 5525) to complete the Consumer Assessment of Healthcare Providers and Systems Clinician & Group (CG-CAHPS) survey [56, 57] and PatientCentered Medical Home (PCMH) Supplemental Item Set [57, 58]. These surveys assess multiple aspects of patient care experiences, [57, 59] and have been used in other studies of care coordination [60, 61]. The sites already administered these surveys for performance monitoring. With funding provided by the CAHPS Program, we supplemented sites’ surveying to capture the patients in this study. We assessed the program’s impact using patients’ responses to questions about four aspects of care targeted by the program and therefore expected to be affected by experiencing the program: timeliness of care, coordination of care, support for patient self-management, and care for mental health. Timeliness of care was hypothesized to increase because patients in the program would have priority access to care; their nurse care coordinators would try to be highly responsive. Coordination of care for program enrollees was to improve because nurses would focus on ensuring that enrollees’ needs were met as seamlessly as possible. Support for selfmanagement and care for mental health were additional program foci and areas of training for nurses; therefore, we expected that nurse efforts in these areas would be reflected in patient reports of their experiences. We focused on these four standard measures of patient care experience, rather than care coordination alone, recognizing that nurse care coordination efforts should manifest in multiple ways [19, 20]. Table 2, Part A lists the items used from the CG-CAHPS survey to measure these aspects of care, response options, and the reliability of the scales in our sample. Patients indicated whether they experienced the action described in each question using a four-point scale (1 = never to 4 = always) or No (=1)/Yes (=4) response. We averaged responses for the items in each composite to arrive at a score for each aspect of their experience. The four composite scores are highly correlated (p-values < 0.001), so to simplify analyses and presentation, we averaged them to arrive at an overall patient care experience score for each person. After the program had been in effect for 6 months following nurse training, we again invited a random sample of program-eligible patients that had visited the center in the preceding 6 months (N = 4661) to complete the Page 5 of 14 Table 2 Study Measures A. Patient-reported care experience (4 components) Timeliness of care (Cronbach’s alpha (α) = 0.89) ▪ When you phoned this provider’s office to get an appointment for care you needed right away, how often did you get an appointment as soon as you needed? ▪ When you made an appointment for a check-up or routine care with this provider, how often did you get an appointment as soon as you needed? ▪ How often were you able to get the care you needed from this provider’s office during evenings, weekends, or holidays? ▪ When you phoned this provider’s office during regular office hours, how often did you get an answer to your medical question that same day? ▪ When you phoned this provider’s office after regular office hours, how often did you get an answer to your medical question as soon as you needed? ▪ How often did you see this provider within 15 min of your appointment time? Care coordination (Cronbach’s alpha (α) = 0.73) ▪ How often did this provider seem to know the important information about your medical history? ▪ When this provider ordered a blood test, x-ray, or other test for you, how often did someone from this provider’s office follow up to give you those results? ▪ Did you get the help you needed from this provider’s office to manage these different providers and services? ▪ How often did the provider named seem informed and up-to-date about the care you got from specialists? ▪ How often did you and anyone in this provider’s office talk about all the prescription medicines you were taking? Support for patient self-management (Cronbach’s alpha (α) = 0.65) ▪ In the last 6 months, did anyone in this provider’s office talk with you about specific goals for your health? ▪ In the last 6 months, did anyone in this provider’s office ask you if there are things that make it hard for you to take care of your health? Care for mental health (Cronbach’s alpha (α) = 0.78) ▪ In the last 6 months, did anyone in this provider’s office ask you if there was a period of time when you felt sad, empty or depressed? ▪ In the last 6 months, did you and anyone in this provider’s office talk about things in your life that worry you or cause you stress? ▪ In the last 6 months, did you and anyone in this provider’s office talk about a personal problem, family problem, alcohol use, drug use, or a mental or emotional illness? B. Clinician-reported teamwork (2 components) Interprofessional Collaboration (Cronbach’s alpha (α) = 0.77) ▪ Nurses and physicians plan together to make decisions about care for complex patients. ▪ Open communication between care providers takes place as decisions are made for complex patients. ▪ Decision-making about patient care for complex patients is well-coordinated. ▪ The input of ancillary staff is regularly sought when developing care plans. Relational coordination (Cronbach’s alpha (α) = 0.75) ▪ The people on this team share my goals for the care of patients. ▪ The people on this team know about the work I do with patients. ▪ The people on this team respect me and the work I do with patients. ▪ The people on this team communicate with me in a timely way about the status of patients. Note: Cronbach’s alpha (α) above 0.70 indicates satisfactory reliability of a measure and between 0.50 and 0.70 indicates moderate reliability. The reported alphas are based on baseline data. For the first two aspects of care, patients indicated whether they experienced the action described in each question using a four-point scale (1 = never to 4 = always), except for the third item in the care coordination scale for which they replied No (=1) or Yes (=4). For the third and fourth aspects of care, they replied No (=1) or Yes (=4). For staffreported teamwork, staff responded using a four-point response scale (1 = strongly disagree to 4 = strongly agree) CG-CAHPS survey with additional items. All 145 program enrollees received an invitation by design. Followup at 6 months allowed us to avoid contamination of the comparison group: per the organization’s fixed roll-out plan, the program (training) was scheduled to begin in Nembhard et al. BMC Health Services Research (2020) 20:137 the first set of comparison centers at this time. This planned endpoint also aligned with our study objective to assess early effects of the added-role approach. In both the baseline and follow-up periods, we mailed a copy of the survey in English and Spanish to each patient in the sample. Approximately 2 weeks after the first mailing, members of the sample were sent a thank you/ reminder postcard. Approximately 2 weeks after that, another survey package was mailed to those who had not responded. If no response was received after two to three more weeks, we called the patients. A minimum of six calls per person were made on different days and at different times of the week. In the baseline period, 3209 patients of the 5525 contacted (58%) answered the survey; of those, 3007 (94%) confirmed having visited the center in the prior 6 months (intervention group = 934; comparison group = 2073). In the follow-up period, 2306 patients of the 4661 contacted (49%) answered the survey; of those, 2101 (91%) confirmed having visited the center in the prior 6 months (intervention group sample size = 774; comparison group sample size = 1327). In total, 943 patients answered the survey in both periods (643 in control group; 300 in intervention group), and 113 program enrollees responded (78% of the 145 enrolled). Teamwork During the month in which we began both the baseline and follow-up patient surveys, we administered an “organizational assessment survey” via the internet or paper to all primary care team members (PCP, nurses, medical assistants, and behavioral health providers). We recruited team members to participate via informational presentations during lunchtime staff meetings and email, and confirmed willingness to participate via signed consent forms. The survey consisted of validated survey scales for assessing core aspects of teamwork i.e., relational coordination and interprofessional collaboration [62–64]. Interprofessional collaboration refers to the degree of cooperation among individuals with different disciplinary backgrounds [65], while relational coordination refers to the presence of high-quality communication and relationships characterized by shared goals, shared knowledge, and mutual respect needed for task integration [66]. Each scale included four items (Table 2, Part B). Team members indicated their level of agreement with each item using a four-point response scale (1 = strongly disagree to 4 = strongly agree). Because scores for the two scales were highly correlated (p < 0.001), we averaged them to arrive at a summary teamwork score reported by each respondent. At baseline, 96 of 190 (51%) team members completed the survey (intervention group = 43; control group = 53). At follow-up, 135 of 188 (72%) members completed the Page 6 of 14 survey (intervention group = 57; control group = 78). Sixty members participated at both baseline and followup. We used their responses in our analyses to assess program effect based on the experiences of a stable population and minimize the possible confounding effect of respondents new to the centers. This longitudinal sample was 39% PCPs, 22% nurses, 24% medical assistants, and 15% behavioral health providers. The majority were female (71%), full-time staff (89%), and with the organization more than 2 years (82%). Except for the percentage with more than 2 years with the organization (63%), this sample was demographically like the full sample consisting of 33% PCPs, 23% nurses, 28% medical assistants, 18% behavioral health providers, 83% female, and 88% full-time staff. Implementation measures Office visit frequency We obtained information about patients’ number of office visits via response to a question in the CG-CAHPS survey: “In the last 6 months, how many times did you visit this provider to get care for yourself?” Seven response options were offered: none, 1 time (coded as 1), 2 (coded as 2), 3 (coded as 3), 4 (coded as 4), 5 to 9 (coded as 7, the midpoint), and 10 or more times (coded as 10). Patients who did not recall any visits were excluded from study (N = 202 (7%) at baseline and 205 (9%) at follow-up). Contextual factors: training, resources, and compatibility with other job demands The organizational survey administered to primary care team members during the follow-up period included additional questions for nurses about program training, resources, and their new role’s compatibility with other job demands, which we used to assess whether these factors posed a challenge to implementation and effectiveness. Four items were adapted from Venkatesh et al.’s [67] facilitating attributes scale: “I have the resources necessary to coordinate care for complex patients,” “I have the knowledge necessary to coordinate care for complex patients,” “Coordinating care for complex patients is not compatible with other tasks that I’m required to perform,” and “It is easy for me to coordinate care for complex patients.” A fifth resource-related item drew from the FHQC’s employee survey: “I have adequate authority to carry out my work.” We asked nurses at intervention centers to report their level of agreement with each statement (1 = strongly disagree to 4 = strongly agree). Other non-program specific contextual factors that can affect implementation (e.g., supervisor support for workers and workload) and could be assessed for intervention and comparison groups at baseline and followup were evaluated for potential inclusion as covariates. Nembhard et al. BMC Health Services Research (2020) 20:137 Covariates In models assessing patient care experience