The study was carried out as a systematic mapping study to structure, understand, and organize existing research work on HCPs’ experience with ePRO systems [17,18]. Systematic reviews provide a synthesis of valuable studies in a particular field of research that is not possible for a practitioner to read on their own [19], while the systematic mapping study aims to structure the research area [18]. A mapping study is a practical method for a researcher who needs to understand and organize the existing research work in an individual domain [17].

Keywords for this study included patient-reported outcomes, barriers, and cancer. We also used the MeSH terms neoplasms, patient-reported outcome measures, telemedicine, assessment, and patient outcomes as search terms. These keywords were combined with Boolean, and search results were narrowed by the publication date of the year 2010 onward to identify appropriate studies, as shown in Multimedia Appendix 1.

Papers were included if (1) they were written in English, (2) the studies included the use of ePROs, (3) they were published between or during 2010 and 2021, (4) the target population included patients with cancer, and (5) they mentioned HCPs’ perspectives (barriers and facilitators) of using PROMs.

Papers were excluded if they were (1) focused on PROM use in clinical trials, (2) review studies, and (3) focused on the implementation of PROM for clinical practice.

For the purposes of this study, we considered the PROMs used in the past 3 months after the implementation period.

Figure 1 illustrates the study selection process and shows the number of included and excluded papers. The papers were imported from 3 academic research databases to Covidence (Covidence.org). Covidence, a systematic review management tool, was used to remove duplicates and manage all the references included in the title and abstract screening, full-text review, and extraction. The searches from the databases were done from August 22, 2021, to September 29, 2021. A total of 152 papers were imported using agreed-upon search terms, and after careful screening, 17 papers remained.

A considerable share of studies excluded during the title and abstract screening stage were studies that used PROs as data sources but did not assess the barriers and facilitators of using the system. Some of the papers dealt with patient-reported data, but the information was obtained through, for example, a survey on paper or an interview. At that stage, studies related to ePRO implementation rather than barriers and facilitators in using ePRO after the implementation period remained for further full-text review. There was a need to decide when the implementation turned into routine usage, and 3-month implementation period criteria were established.

The main author did the data selection independently. A random selection of 23% (n=25) of full-text papers was reviewed by a different researcher to determine interrater reliability. Interrater reliability was determined using Cohen κ and found to be acceptable at 0.63 (SE 0.25; 95% CI 0.14-1.11). The reviewers had divided opinions on the eligibility of one of the jointly evaluated papers. Therefore, there was a need to clarify when implementation became the routine use of PRO. Multimedia Appendix 1 shows the papers selected according to the criteria of this study. Later in this study, the papers are referenced based on the sequence numbers in the References section.

As shown in Table 1, papers addressing the perspectives of HCPs on the barriers to and facilitators of using ePRO data in the treatment of patients with cancer have been published in recent years, while there was no publishing in the early 2010s. The table shows that there is a clear minority of papers describing only ePRO systems in 2020-2021 papers compared to papers focusing on both ePROs and nondigital PROMs.

The data on the included papers were extracted as follows: authors, title, year of publication, study objective, cancer type, study design, duration of data collection, study population, administered PROM, ePRO name, sole use of ePRO, barriers, and facilitators. Thematic meta-synthesis for the extracted data was performed by adapting the methods for thematic synthesis described by Thomas and Harden [20]. The verbatim findings of these 17 studies were entered into the Excel (Microsoft Corp) file. Each line of text was coded according to its meaning and content. The authors of the paper independently reviewed the data and grouped them into themes. Themes were then discussed and merged. After several rounds of reviewing the coding, the final 7 descriptive themes for the barriers and facilitators were determined. The 7 themes were summarized into 3 categories that united the themes: work environment, value to users, and suggested features.

Of the total 17 papers, 8 introduced a study in which study participants had used only ePROs [21-28]. In the other 9 papers [29-37], only some of the participants had used ePROs, while others had used nondigital PROMs. The ePRO systems used were AmbuFlex [21,26], OncoQuest [22], Noona [23], the KLIK method [25], and PatientViewpoint [28]. No type of cancer was particularly emphasized, but PROMs were used in different target populations, such as patients with breast, lung, head and neck, prostate, melanoma cancer, and pediatric patients with cancer. The most frequently used PROM was the Edmonton Symptom Assessment Scale (ESAS) [32-34]. Studies were carried out in multiple countries: the Netherlands [22,25,31], Germany [27,29], the United Kingdom [24], the United States [23,28,35,37], Denmark [21,26], Canada [32,34], Australia [33], and 41 other countries [30]. There was no mention of the country where the PROM was implemented in 1 paper [36].

The following themes were identified from the barriers and facilitators presented in the selected papers: clinical workflow, organization, and infrastructure, value to patients, value to HCP, digital health literacy, data visualization and perceived features, and usability.

This mapping study identified multiple barriers and facilitators to using ePROs for cancer care. The highlights of these are presented in condensed form as takeaway points for easy reading in categories such as work environment, value to users, and suggested features. Our work exposes the need for future studies on the use of ePROs compared to studies on the use of paper PROs.

The findings of this study strongly support the active integration of ePRO into the surrounding work environment. Earlier knowledge emphasizes the importance of functional workflow [12]. This leads to the notion that ePRO’s operations should be integrated into the hospital workflow to allow users to experience ePROs’ seamless use in the hospital setting. Based on this, it could be valuable to optimize the use of ePRO together with hospital operations at the time of the implementation of the ePRO. This study also revealed the need for ePRO integration in different hospital units. If the same ePRO is in use in different specialties, the system data could be better used in the multidisciplinary care of patients. Further, ePRO integration into the EHR in the hospital was highlighted in 9 of the 17 papers analyzed, which is well-aligned with prior works [9,12,14].

There are different lines of thought regarding the moment in which ePROs are best deployed for patients. The use of web-based ePRO at home may be advantageous, as the memory of the symptoms may be fresher, whereas use during the clinical visit may increase overall use. If patients complete the ePRO at the beginning of the clinical visit, it may help reduce the fear of losing personal contact with professionals and lower digital literacy needs [38]. Allowing patients to choose when to complete the ePRO could be a good compromise. Finally, when the patient is at the doctor’s office, patient-clinician face-to-face communication is needed instead of clinicians looking at the computer screen to look at the ePRO information [11]. This is also supported by Gilligan et al [39] in their consensus guideline of patient-clinician communication, where they recommend considering using PROs to prepare the patient visit.

This study emphasizes multiple benefits for patients and the importance of patients’ opinions for HCPs regarding the use of PRO systems [25]. These findings are consistent with Roberts et al [14]. Their study demonstrates the desire of HCP to be active in implementing PROMs into routine oncology care if patients benefit from the use and if the use of PROM improves health care. According to these studies, it is important to inform HCPs about patients’ views and what facilitates patients’ use of ePRO. It may strengthen HCPs’ experience of the relevance of ePROs and improve the user experience if patients have positive experiences and feel that they will benefit from using ePRO.

This study highlighted the potential of ePRO systems to help develop the treatment of patients by using the data generated by the system, for example, about the symptom groups of the patient population [33]. A recently published study has also shown the important role of ePRO data in examining the benefits and efficacy of new innovative treatments [40]. Based on this, one can assume that the benefits of ePRO are wider than monitoring the treatment of an individual patient. ePRO data also play an important role in the development of treatments for patients with cancer.

The number of comments on the different themes shows how strongly the ePRO value for patients and HCPs facilitates the use of ePROs. Of 128 comments that presented barriers or facilitators, 60 described the value to patients or HCPs. In other words, for “value to users,” 25 of the comments concerned “usability” and “data visualization and perceived features” of the systems. Although this proportion is not as prominent compared to the “value to users” theme, there are some explicit features mentioned that should be considered while developing the system to become more practical for the users.

According to this study (Table 1), only a few studies are available on the barriers and facilitators of ePRO use, including both ePRO and nondigital PROMs. In total, 8 papers described studies that used ePROs alone; the other 9 papers had both electronic and nondigital PRO data. Interestingly, studies on ePRO only have declined in recent years. The smaller number of ePRO studies could suggest that using ePRO has not supplanted the use of nondigital PROs. Thus, there is still a need for the knowledge and development of ePROs.

This study has several limitations. As systematic mapping studies rely on the selection process to identify relevant studies, there is a risk of bias that can affect the results and conclusions of the study. Search results are only as current as the date of the last search performed. The quality of the studies included varies greatly, and the limitations of individual studies can affect the overall results and conclusions of the mapping study.

Further, this kind of study does not allow for in-depth analysis of individual studies or a detailed synthesis of the findings. However, systematic mapping studies are helpful to provide an overview of the existing literature, which was the goal of this study.

We decided to focus on the barriers and facilitators experienced by professionals. Patients’ experiences were excluded, except when reported by professionals. This resulted in a one-sided perspective on the use of ePRO systems. Other stakeholders’ opinions were not discussed. We grouped the barriers and facilitators compiled from the analyzed papers into themes and themes into categories. These themes and categories are the researchers’ views on the issue. Some papers included comments from respondents who had never used ePRO systems. We could not ascertain that those responses were different from responses by users who had experience using ePROs.

In the data screening phase, we made efforts to remove the barriers and facilitators identified in the paper version of PROs. Thus, it is possible that the comments from the paper version of PROMs and ePROs were partially mixed.

Studies that demonstrated the barriers and facilitators of the implementation phase of PROM were excluded from the study. The decision to limit the implementation phase to 3 months may have indirectly affected our findings. Although this decision helped in classifying and screening the results, relevant papers might have been excluded.

In this study, we provided a broad overview of the barriers and facilitators affecting the use of ePROs. Our work focused on how the working culture and service integration affects the success of ePRO. A greater understanding of barriers and facilitators is useful to software developers and clinical research organizations to create smoother implementations. We found that there are multiple ways to develop ePROs and their working environments to meet the needs of HCPs. They can be summarized into 3 categories: work environment, value to users, and suggested features. The takeaway points detail the findings of this study.

Based on this study, there is still a lack of information on the national and international knowledge of ePROs. Since there are only a few studies on fully electronically completed PRO data, future research should explore the barriers and facilitators of using ePROs, specifically in organizations where users have sufficient experience using ePROs. As this study is limited by the literature currently available in the selected databases, further work may expand on the knowledge by including additional sources and terms. Future work could also focus on exploring how the implementation of ePROs may affect the patient’s journey through the health care system. It would also be interesting to understand whether more usability and features are required of an ePRO than of a paper PROM, given that it is possible to implement features beyond those of paper versions.

It would also be interesting to explore in more detail how common it is that patients rate their symptoms differently than physicians treating them and how patients’ personal experiences are considered in treatment. Can ePRO be further developed to identify rating differences? Would it help if ePRO were to add more detailed parameters to the symptoms? Concerning challenging symptom descriptions, research may be conducted to determine which symptoms or symptom descriptions differ most in terms of patient and physician perceptions and, based on this, develop an ePRO to highlight a potential bias.