This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Cancer, is properly cited. The complete bibliographic information, a link to the original publication on http://cancer.jmir.org/, as well as this copyright and license information must be included.
Practice-based research is essential to generate the data necessary to understand outcomes in ambulatory oncology care. Although there is an increased interest in studying ambulatory oncology care, given the rising patient volumes and complexity in those settings, little guidance is available on how best to recruit ambulatory oncology practices for research.
This paper aimed to describe the facilitators and barriers to recruiting ambulatory oncology practices into a large multisite study.
Using a mixed methods design, we sought to recruit 52 ambulatory oncology practices that have participated in a state-wide quality improvement collaborative for the quantitative phase. We used 4 domains of the Consolidated Framework for Implementation Research (CFIR) to describe facilitators and barriers to recruitment.
We successfully recruited 28 of the 52 collaborative-affiliated practices, collecting survey data from 2223 patients and 297 clinicians.
Our experience provides other researchers with challenges to anticipate and possible solutions for common issues. Using the CFIR as a guide, we identified numerous recruitment barriers and facilitators and devised strategies to enhance recruitment efforts. In conclusion, researchers and clinicians can partner effectively to design and implement research protocols that ultimately benefit patients who are increasingly seeking care in ambulatory practices.
A growing proportion of health care is delivered in ambulatory practice settings, yet little information about ambulatory care quality and safety is available [
We sought to recruit ambulatory oncology practices that delivered chemotherapy to patients with cancer in our study. The purpose of our study was to understand health care delivery by characterizing clinician communication processes, communication technologies, and adverse patient outcomes in ambulatory oncology practices and to examine how these practices and technologies influence safe chemotherapy administration. Once practices agreed to participate, we recruited clinicians who worked in those practices and patients who were cared for at those sites. During the recruitment of practices into our study, we faced challenges mirroring what has already been reported in the literature [
Thus, the purpose of this paper was to report on the facilitators and barriers to the recruitment of ambulatory oncology practices in Michigan, United States, and share the lessons we learned.
The overall study is using a mixed methods design. We began with a quantitative phase, by distributing questionnaires to all prescribers (ie, physicians, physician assistants, and nurse practitioners) and registered nurses who work in a sample of ambulatory oncology practices. In addition, for 6 weeks, site study coordinators completed a 1-page daily event log, and patients completed a 1-page self-reported symptom questionnaire. We then used the survey results to identify 8 practices for subsequent exploration via in-depth qualitative methods, and the analysis phase of the project is ongoing.
The study setting includes ambulatory oncology practices that belong to the Michigan Oncology Quality Consortium (MOQC). MOQC is an alliance of ambulatory oncology practices formed with the purpose of sharing and benchmarking their data to improve the quality of oncology care. As we were interested in targeting ambulatory oncology practices throughout the state of Michigan in the United States, we partnered with MOQC, which currently has 52 affiliated practices all over Michigan.
We invited all MOQC-affiliated ambulatory oncology practices to participate in our study. We sought to recruit as many MOQC-affiliated practices as possible because we were interested in understanding the variation in clinician communication processes, communication technologies, and adverse patient outcomes in ambulatory oncology practices. The practices identified employees to serve as study coordinators who were responsible for distributing clinician questionnaires once, completing daily event logs, and distributing self-report questionnaires to patients daily for 6 weeks. Clinician questionnaires were about the usability of and satisfaction with the electronic medical record, communication among clinicians, perceptions of a safety climate, and perceptions of the work environment. The daily event logs summarized clinic activities and events related to chemotherapy (eg, the number of patients prescheduled and the number of patients who called the clinic for toxicity management). In the patient questionnaires, patients were asked to report symptoms related to their chemotherapy treatment. The survey procedures have been described elsewhere [
By collaborating with MOQC, we had access to the latest information about the various practices to use for recruitment purposes (eg, name and contact information of the practice manager). Practice recruitment occurred on a rolling basis from April 2017 to November 2017. During that time, we held weekly meetings, where we reviewed practice enrollment and survey response data. KV maintained a tracking sheet of all MOQC sites that had tabulated information of when sites were initially contacted, the dates of follow-up, and the identified reasons for nonparticipation. Overall, 2 research team members (MM and KV) reviewed the notes taken during these meetings to identify barriers and facilitators to recruitment. The entire research team met regularly to discuss and confirm emerging barriers and facilitators.
We used the Consolidated Framework for Implementation Research (CFIR) to help identify facilitators and adapt barriers to successful recruitment and implementation of data collection. We followed the example set by Coronado et al [
Using the CFIR framework, we successfully recruited 28 of the 52 MOQC-affiliated practices to participate in our study (a recruitment rate of almost 54%). From those participating practices, survey data were collected from 297 clinicians (a 68% response rate) and 2223 patients (a 58.7% response rate).
Intervention attributes refer to features of the intervention that can influence its execution, in our case, the characteristics surrounding data collection that could be customized to each practice without compromising the quality of data collected. Facilitators included involving physicians and ambulatory practice staff in the early stages of the project to get feedback on study materials and protocols. For example, 1 site developed a comprehensive, 1-page
Implementation processes are the strategies that affect the implementation of interventions. Although we are not conducting an intervention as a part of our study, the implementation process refers to the training and use of staff for data collection. Each facility designated a
As we were asking sites to collect data from multiple sources, practice champions were required to attend a Web-based training session to learn about the study and assure that data would be collected uniformly at each site. Participating in the training was a mandatory requirement for engaging with us in the study, and training had to be completed before data collection could begin. We used PowerPoint slides (Microsoft Corporation, Redmond, Washington) [
To further facilitate data collection, we scheduled 15- to 20-min telephone conversations with practice administrators and physician leaders to describe the study in greater detail. Before each scheduled conversation, the study information consisting of a single-page overview of the study and a
The external context refers to environmental factors outside of each ambulatory practice, including MOQC practices throughout the state of Michigan and our efforts to recruit as many MOQC-affiliated practices as possible. Our recruitment efforts required several facilitative strategies, beginning with our presence at a MOQC biannual meeting that was attended by practice managers, physician leaders, and other MOQC stakeholders. The director of MOQC, who is a coinvestigator in the study, introduced the study to meeting attendees, highlighting how participation in the study would be relevant to the quality of care in individual practice settings. We provided attendees with a 1-page overview and study FAQ sheet. After the meeting, the MOQC director personally spoke with most physician leaders to remind them of the study and let them know that the study team would be contacting practices as a part of the recruitment process. In this way, the close professional relationships established by the director of MOQC with affiliated practices meant that we did not approach practices
One of our earliest strategies to facilitate recruitment was to have a MOQC representative on the study team who was known to the practice staff. The MOQC representative served as a liaison to the MOQC director and as a point of contact for queries coming from MOQC-affiliated practices about the study. It was important that the study not fracture the practices’ pre-existing relationships with the MOQC office and the director; therefore, having a MOQC representative on the study team was essential.
The internal context refers to the characteristics of the implementing organization, specifically the unique characteristics and culture of each practice. Each practice differed both in terms of size and ownership as well as variation in patient populations served. The study team tailored procedures to the unique characteristics of each practice to facilitate recruitment. For example, some practices consisted of two or more physical locations. To develop a comprehensive understanding of the overall practice, our goal was to recruit all locations of a single practice into the study, although some smaller locations saw patients only 2 or 3 days a week. We treated each physical location as an individual unit in recognition of the unique culture of each location.
We also acknowledged that the practice staff were in the best position to advise us on data collection procedures. We had conversations with the practice staff to clarify the inclusion criteria for patients in our study because in some practices, noncancer patients were treated with antineoplastic agents. Clarifying that the drug had to be delivered to a patient with cancer made it easier for practices to determine which patients to include.
The practice administrators were often gatekeepers who allowed access to stakeholders within each site so as to facilitate access, and the MOQC representative on our team sent periodic study updates to the practice administrators about the number of participating practices without identifying specific sites. Although the institutional review board (IRB) of our institution deemed our study to be exempt from an ongoing IRB review, we learned that this was insufficient for many practices that required separate IRB determination from their own home institutions. Upon request, we shared with practices the study protocol developed for our IRB.
To our knowledge, we are the first to report on ambulatory oncology practice recruitment, as the research to date has focused on recruiting primary care practices. Recruiting practices for research requires multiple strategies to succeed, both at the practice and individual levels, no matter the specialty. By framing our recruitment strategies according to CFIR domains of intervention attributes, implementation process, external context, and internal context [
Coronado et al [
In recruiting ambulatory practices, it is important to enlist the support of more than the medical director of the practice, as he or she may have limited time to devote to research or not necessarily be a
A central facilitator that affected the external context focused on our collaboration with MOQC. There were many advantages to the collaboration that facilitated recruitment, including access to an established infrastructure and the latest information about each of the practices in the consortium to facilitate recruitment. Johnston et al [
During the MOQC biannual meeting, and in individual phone calls, the MOQC director was careful to highlight how participation in the study would be relevant to the quality of care in individual practice settings to address barriers in the internal context. The director of MOQC, as well as the MOQC representative mentioned earlier, had a positive influence on site participation because of the professional networks built with physicians over the years. Such alliances are the foundation of practice-based research networks, which operate as loose coalitions of primary care practices to improve clinical practice and patient outcomes [
We also used many strategies to build a rapport with individual practices. Considering that practices function through the work of many people, it was important to build a rapport with a variety of providers and staff. Our early strategies for building a rapport focused on communication that frequently flowed through the practice administrator, receptionist, and support staff. This, along with having the MOQC representative on the study team, counteracted the inability to build a rapport with a practice’s receptionist, which has been identified as a barrier to recruitment [
Another cited barrier to research participation is the lack of monetary incentives [
We learned 2 important lessons related to distributing monetary incentives, which have not previously been reported. First, university policy required that a current W-9 be on file for each practice before incentive disbursement. A W-9 is a form used in the US income tax system to confirm information for income-generating purposes. Completing this paperwork, even though required by the Internal Revenue Service in the US, added to the overall burden, especially for smaller practices. As we did not have the information to complete a W-9 on their behalf, we used email and telephone prompts to encourage practices to complete the W-9 paperwork. The second lesson was related to communication about the incentives. As some practices were spread across multiple physical locations, each participating location was eligible for the incentive. However, as the university sent out incentives addressed to the practice and not each location, confusion arose when one practice called to ask why they had received an honorarium check. As a result, we intercepted the outgoing mail so that we could insert a thank you note and an explanation for the enclosed check, before returning the letter to the mail.
We learned another lesson through our challenges with getting an IRB approval from multiple sites and navigating a complex IRB system. Our experience may no longer be relevant in the near future, at least for research conducted in the United States, as US researchers will face new challenges because of the changes in the IRB process. Specifically, the Federal Policy for the Protection of Human Subjects (also known as the Common Rule) in the United States changed in January 2019 [
Recruiting practices took longer than anticipated, providing us with a final lesson learned. Recruitment took 9 months, a time frame reported in other studies [
Practice-based research in ambulatory care is essential to generate the data necessary to understand patterns of health care delivery, correlates, and outcomes in these diverse and understudied settings. Generating robust research data in ambulatory practice settings requires novel partnerships among researchers, coalitions, and a broad array of clinicians and practice administrators. Our experience provides other researchers with challenges to anticipate and possible solutions to common issues. Using CFIR as a guide, we devised strategies to facilitate recruitment efforts and minimize barriers. In conclusion, researchers and clinicians can partner effectively to design and implement research protocols that benefit not only researchers and providers but also the patients who are increasingly seeking care in ambulatory practices.
Consolidated Framework for Implementation Research
frequently asked questions
institutional review board
Michigan Oncology Quality Consortium
National Institutes of Health
This project was supported by grant number R01HS024914 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.
MM conceived the paper. LB, JG, MB, KV, and CF provided critical intellectual input.
None declared.