This paper is in the following e-collection/theme issue:

Innovations and Technology in Cancer Care (400) Research Instruments, Questionnaires, and Tools (540) New Methods (902) Design, Usability, and Evaluation of Research Instruments, Scales, and Measures (46) User Needs and Competencies (270) Methods in Cancer Research (24)

Published on in Vol 3, No 2 (2017): Jul-Dec

Electronic-Based Patient-Reported Outcomes: Willingness, Needs, and Barriers in Adjuvant and Metastatic Breast Cancer Patients

Electronic-Based Patient-Reported Outcomes: Willingness, Needs, and Barriers in Adjuvant and Metastatic Breast Cancer Patients

Electronic-Based Patient-Reported Outcomes: Willingness, Needs, and Barriers in Adjuvant and Metastatic Breast Cancer Patients

Authors of this article:

Andreas D Hartkopf1 Author Orcid Image ;   Joachim Graf2 Author Orcid Image ;   Elisabeth Simoes2 Author Orcid Image ;   Lucia Keilmann3 Author Orcid Image ;   Nina Sickenberger3 Author Orcid Image ;   Paul Gass4 Author Orcid Image ;   Diethelm Wallwiener1 Author Orcid Image ;   Lina Matthies3 Author Orcid Image ;   Florin-Andrei Taran1 Author Orcid Image ;   Michael P Lux4 Author Orcid Image ;   Stephanie Wallwiener3 Author Orcid Image ;   Eric Belleville5 Author Orcid Image ;   Christof Sohn3 Author Orcid Image ;   Peter A Fasching4 Author Orcid Image ;   Andreas Schneeweiss3 Author Orcid Image ;   Sara Y Brucker2 Author Orcid Image ;   Markus Wallwiener3 Author Orcid Image
Article Authors Cited by (27) Tweetations (1) Metrics

    Original Paper

    1Department of Women’s Health, University Hospital Tuebingen, Tübingen, Germany

    2Research Institute for Women's Health, Department of Women’s Health, University Hospital Tuebingen, Tübingen, Germany

    3Hospital for General Obstetrics and Gynecology, National Cancer Center, University Hospital Heidelberg, Heidelberg, Germany

    4Department of Gynecology and Obstetrics, University Hospital Erlangen, University Breast Center Franconia, Comprehensive Cancer Center Erlangen, Erlangen, Germany

    5Clin-Sol Ltd., Würzburg, Germany

    *these authors contributed equally

    Corresponding Author:

    Markus Wallwiener

    Hospital for General Obstetrics and Gynecology

    National Cancer Center

    University Hospital Heidelberg

    Im Neuenheimer Feld 440

    Heidelberg, 69120

    Germany

    Phone: 49 6221 56 36956

    Email: markus.wallwiener@gmail.com


    Background: Patient-reported outcomes (PROs) play an increasingly important role as an adjunct to clinical outcome parameters in measuring health-related quality of life (HRQoL). In fact, PROs are already the accepted gold standard for collecting data about patients’ subjective perception of their own state of health. Currently, paper-based surveys of PRO still predominate; however, knowledge regarding the feasibility of and barriers to electronic-based PRO (ePRO) acceptance remains limited.

    Objective: The objective of this trial was to analyze the willingness, specific needs, and barriers of adjuvant breast cancer (aBC) and metastatic breast cancer (mBC) patients in nonexposed (no exposure to electronic assessment) and exposed (after exposure to electronic assessment decision, whether a tablet-based questionnaire is favored) settings before implementing digital ePRO assessment in relation to health status. We also investigated whether providing support can increase the patients’ willingness to participate in such programs.

    Methods: The nonexposed patients only answered a paper-based questionnaire, whereas the exposed patients filled out both paper- and tablet-based questionnaires. The assessment comprised socioeconomic variables, HRQoL, preexisting technical skills, general attitude toward electronic-based surveys, and potential barriers in relation to health status. Furthermore, nonexposed patients were asked about the existing need for technological support structures. In the course of data evaluation, we performed a frequency analysis as well as chi-square tests and Wilcoxon signed-rank tests. Subsequently, relative risks analysis, univariate categorical regression (CATREG), and mediation analyses (Hayes’ bias-corrected bootstrap) were performed.

    Results: A total of 202 female breast cancer patients completed the PRO assessment (nonexposed group: n=96 patients; exposed group: n=106 patients). Self-reported technical skills were higher in exposed patients (2.79 vs 2.33, P ≤.001). Significant differences were found in relation to willingness to use ePRO (92.3% in the exposed group vs 59% in the nonexposed group; P=.001). Multiple barriers were identified, and most of them showed statistically significant differences in favor of the exposed patients (ie, data security [13% in the exposed patients vs 30% in the nonexposed patients; P=.003] and no prior technology usage [5% in the exposed group vs 15% in the nonexposed group; P=.02]), whereas the differences in disease burden (somatic dimension: 4% in the exposed group vs 9% in the nonexposed group; P=.13) showed no significance. In nonexposed patients, requests for support services were identified, which could increase their ePRO willingness.

    Conclusions: Significant barriers in relation to HRQoL, cancer-related restrictions, and especially the setting of the survey were identified in this trial. Thus, it is necessary to address and eliminate these barriers to ensure data accuracy and reliability for future ePRO assessments. Exposure seems to be a potential option to increase willingness to use ePRO and to reduce barriers.

    JMIR Cancer 2017;3(2):e11

    doi:10.2196/cancer.6996

    Keywords

    breast cancer; patient-reported outcome measures; electronic patient- reported outcome; technical skills; willingness to use; needs and barriers 


    Patient-Reported Outcomes in Breast Cancer Patients

    Current advances in immuno-oncology and various target treatment combinations provide promising results such as long-term survival in cancer patients. However, treatment environments are challenging with regard to balancing clinical outcome and monitoring of quality of life, for example, in breast cancer patients [1,2]. Hence, patient-reported outcomes (PROs) play an increasingly important role as an adjunct to clinical outcomes in clinical practice [3]. A PRO is defined as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else” [4]. PROs comprise various aspects of the subjectively perceived state of health from the patient’s point of view, such as the health-related quality of life (HRQoL) [3-9].

    Novel Assessment of HRQoL and Adverse Events in Clinical Routine

    PROs are assumed to be versatile and heterogeneous because they implicate many health conditions such as HRQoL, symptom severity (eg, using a pain scale), physical mobility, degree of psychological stress, disease-related impairment in daily routine, patient satisfaction, and drug adherence [3-13]. HRQoL is an important tool in clinical routine that comprises physical, emotional, mental, social, and behavioral components in terms of the patient’s well-being and functioning from the patient’s subjective perspective [12-15]. Furthermore, PROs reflect treatment success in a patient-centered manner [3-5,15-18]. Thus, PROs should be used to measure the effectiveness of new interventions to complement the results of efficacy studies, which only evaluate the success of therapeutic interventions in a clinical trial [4,18]. In the case of oncology patients, the patients’ subjective perception of their own state of health is considered an important indication of the efficacy and safety of a specific therapy [19-25]. For example, in patients with metastatic breast cancer, PROs are a relevant source of information indicating whether the primary treatment aim of prolonging life with a more reasonable HRQoL is achieved [24,26-29]. The relevance of validated PRO questionnaires (eg, the European Organization for Research and Treatment of Cancer Quality of Life questionnaire-Core 30 item [EORTC QLQ-C30]) has been confirmed in several studies, in which patients with chronic diseases assessed their quality of life as being significantly worse, as compared with the clinical assessment [27-31]. Due to its high practicability and validity, EORTC QLQ-C30 is one of the most commonly used questionnaires for measuring PRO in patients with breast cancer.

    Electronic Monitoring of PRO on the Rise

    Validated PROs for measuring cancer-specific HRQoL (eg, EORTC QLQ-C30 and Functional Assessment of Cancer Therapy-General [FACT-G]) are already the accepted gold standard for data collection for closely related variables such as HRQoL, satisfaction with care, and drug adherence [32-34]. Currently, paper-based surveys of PRO still predominate in clinical routine, especially because there is a lack of validated electronic-based PRO (ePRO) measurement instruments pertaining to various oncological conditions [35]. There is growing demand for information and communication on behalf of patients and increasing integration of information technology in health care, which is why data on patient-relevant end points has increasingly been collected electronically in recent years. Thus, the potential of electronic health (eHealth) solutions in health care research is becoming increasingly apparent [36,37]. The benefits of digital data capture include real-time data capture, screening for deterioration of adverse events (AEs), potential cost-effectiveness for health centers, and therefore, a potential for longitudinal symptom assessment [38,39]. Long-term digital AE monitoring also seems feasible. Nevertheless, knowledge regarding patient acceptance, feasibility, and barriers remains limited, especially in relation to health status and socioeconomic aspects [40-45]. Previously collected data regarding barriers showed that older metastatic breast cancer patients with a higher disease burden may be less inclined to complete ePRO questionnaires (eg, by using tablet devices) [46]. The technical experience and skills of the patient population also have a significant impact on the adoption and adherence rates. Patients who participated in Web-based symptom monitoring showed both a 16% higher improvement in HRQoL and a 6% higher 12-month overall survival, were 7% less frequently admitted to the emergency room [38], and were more willing to use ePRO [46]. To date, little research has been conducted on whether the willingness to use eHealth applications increases when patients are exposed to it. No studies could be identified which focalize on whether the use of ePRO can be increased or potential barriers alleviated by exposure. However, studies from geriatrics indicate that reservations of elderly patients can be deferred to eHealth applications when faced directly with them [47-49]. It is also unclear to what extent sociodemographic variables influence exposure. For reliable and valid measurement of ePRO, it is relevant to identify all the variables that influence patients’ response behavior.

    Aims

    The main aim of this study was to analyze the willingness, specific needs, and barriers of adjuvant breast cancer (aBC) and metastatic breast cancer (mBC) patients before implementing digital ePRO assessment in relation to health status (HRQoL and therapy setting [aBC vs mBC]). We also investigated whether providing support can increase their willingness to participate in such programs. We analyzed potential differences in the willingness of aBC and mBC patients in relation to the survey setting (nonexposed vs exposed survey). The main aim of the study was to analyze the influence of an ePRO tool on the patients’ willingness to participate. Second, possible hurdles for ePRO that determine nonresponse rates should be identified in breast cancer patients. With the long-term goal being to use ePRO exclusively, appropriate barriers must be identified. This trial evaluated the patients’ general acceptance and practicability of ePRO in aBC and mBC subgroups. The goal was to analyze whether there was coherence between the health status (aBC vs mBC) and the willingness/frequency of barriers and between the survey setting (nonexposed vs exposed survey) and the willingness/frequency of barriers. To achieve the aims, aBC and mBC patients with and without ePRO exposure were asked to fill out questionnaires about their sociodemographic indications, technical skills, HRQoL, willingness to use, and potential barriers.


    Sample and Study Design

    From July 2015 to May 2016, paper-based PRO questionnaires were completed by female aBC and mBC patients treated consecutively at the Department of Women’s Health in Tuebingen, Germany, and the National Cancer Center in Heidelberg, Germany. To analyze the dependency of identified barriers regarding health status in aBC and mBC patients, we compared nonexposed and exposed patients. The patients were recruited from two different studies: 106 exposed patients were recruited from electronic-based Patient-Reported Outcomes and Compliance Analysis (ePROCOM) and 96 nonexposed patients from another study [46]. All female breast cancer patients aged more than 18 years who either had metastasis or were undergoing adjuvant treatment, who additionally had sufficient knowledge of German to answer the questionnaire, and who declared their consent to fill out the questionnaires during an outpatient visit to the hospital under the supervision of an attending physician were included in the study. All patients were recruited from the PRAEGNANT network [50]. Patients had no prior exposure to any electronic assessment tools in the study in which they were currently included. If patients had prior contact with ePRO in other studies, they were not asked to participate (exclusion criteria).

    After filling out their paper-based PRO questionnaires, the nonexposed patients were asked whether they would be interested and confident in using electronic assessments prospectively and whether there were any preexisting barriers. Exposed patients were provided with the actual electronic assessment application (ePROCOM). They were requested to fill out both the paper- and tablet-based PRO questionnaires so that the reliability of an ePRO tool could be analyzed. After filling out both questionnaires, they were also asked about their preferences toward future usage of either paper-based assessment or ePRO. The aim of ePROCOM was to evaluate the general patient acceptance and practicability of a Web-based application for a PRO-questionnaire for patients with aBC or mBC. The ePROCOM patients were asked to participate to compare the response behavior of patients in paper-based and Web-based questionnaires (publication in preparation). Inclusion criteria of ePROCOM were female gender, full legal age, aBC or mBC diagnosis, sufficient language skills in German, and signed declaration of consent. The ePROCOM patients were also asked to complete the questionnaire during an outpatient visit to the hospital under the supervision of an attending physician. We have previously reported on the influence of age, educational status, HRQoL, and technical skills of mBC patients [46].

    The patients of both arms of the study (exposed and nonexposed) were informed about the aims of the study and were asked for their consent ex ante. The ethics committee gave prior consent for the study (project number 196/2015B02 and 089/2015B02). Randomization in this setting was not feasible, as patients were recruited from different studies. However, the trial design enabled identification of the main barriers in breast cancer patients for participating in ePRO in relation to the exposed versus nonexposed setting with regard to sociodemographic factors, therapy setting (aBC vs mBC), and HRQoL.

    Assessments

    The assessment comprised 3 parts. The first part focused on the patientsʼ socioeconomic variables. The second part focused on HRQoL according to the EORTC QLQ‑C30, comprising 30 questions in 5 subscales, various symptom scales, and individual items related to the patientsʼ health status on a multidimensional level. We used only those 2 questions from the EORTC QLQ‑C30 that focused on the patient’s health status and HRQoL on a 7-point Likert scale (from 1=very poor to 7=excellent). The acceptance level and identification of barriers and acceptance, but not HRQoL, constituted the main focus of the analyses. Patients also completed the entire EORTC QLQ‑C30 questionnaire; data on every single function and symptom scale are available upon request. Mean values were calculated in accordance with the official EORTC guidelines, which require a separate score to be calculated for each scale. The scores ranged from 0 to 100 [51,52]. In the third part of the questionnaire, the patients were asked about preexisting technical skills such as use of electronic technology at home, routine usage of digital devices such as computers, Internet use, their general attitude toward electronic-based surveys, and potential barriers in relation to their health status. Furthermore, the patients in the nonexposed survey were also asked about existing technological support structures because they only completed the paper-based questionnaire, whereas the exposed group filled out both paper- and tablet-based questionnaires. The trial design was based on the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. This guidance plan was developed specifically for assessing the effectiveness of interventions and included aspects of reach, effectiveness, adoption, implementation, and maintenance [53,54].

    Statistical Analysis

    A frequency analysis was first performed using the Statistical Package for the Social Sciences (SPSS) version 21 (IBM) to determine the descriptive characteristics of the collected data. The goal was to demonstrate how the barriers of technology-based surveys are distributed over the entire population. The influence of the barriers on the rejection of electronic-based surveys was also identified, and the barriers among patients with preferences for paper-based questionnaires and ePRO were compared in relation to socioeconomic variables, health status, and technical skills in self-perception. Differences between nonexposed and exposed patients were identified using chi-square tests (if the variables were dichotomous and binary coded) and Wilcoxon signed-rank tests in ordinal- and metric-scaled data, because the paired samples were not normally distributed in the Shapiro-Wilks test and in quantile-quantile plots. Furthermore, a relative risks analysis was calculated to identify the influence of ePRO exposure on usability and barriers. Subsequently, we performed univariate categorical regression (CATREG) analysis to ascertain regression context between ePRO exposure and willingness to use the identified barriers [55,56]. Mediation analyses (Hayes’ bias-corrected bootstrap) were then performed to expose the potential interferences of the regression model [56]. Finally, demand for technical support was measured through frequency analysis in the nonexposed group. Beforehand, we performed chi-square tests and Shapiro-Wilks test between mBC and aBC patients in both groups to identify possible statistically significant differences in relation to HRQoL and willingness to use. A bilateral P value of <.05 was considered statistically significant in all analyses (alpha=.05). The survey was conceived as an explorative study, in which all P values were to be understood purely descriptively and had no confirmatory value. Figure 1 was created in Microsoft Excel 2010.

    ‎
    Figure 1. Barriers for using electronic-based patient-reported outcome.
    View this figure

    Sociodemographic Variables and Technical Skills

    A total of 202 female breast cancer patients completed the PRO assessment (nonexposed group: n=96 patients; exposed group: n=106 patients). We did not find significant intragroup differences between aBC and mBC patients. Table 1 shows the sociodemographic characteristics of the study group. Nonexposed patients were significantly older compared with the exposed group, and their self-rated HRQoL was reported to be worse in the EORTC QLQ-C30 survey. However, the differences in HRQoL between both groups were not statistically significant. The level of education was significantly higher in the exposed group.

    The technical skills are shown in Table 2. In all dimensions and at all levels, the self-reported technical skills were higher in exposed patients, including considerable time of computer and Internet use and higher frequency of tablet usage.

    Willingness to Use Technology-Based Surveys (ePRO)

    The results for both treatment groups suggest that the introduction of electronic surveys will indeed improve clinical care and completion of ePRO questionnaires; however, there were significant differences between exposed and nonexposed patients. Exposed patients more often suggested that hospital care could be improved by using ePRO questionnaires and more frequently rated ePRO assessments as being more suitable, less tiring, and less difficult (Table 2). Before exposure to the ePRO application, both groups were asked about their potential ePRO assessment usage. Overall, the disposition for potential ePRO usage was high, with 77% of all patients indicating willingness. However, there were significant differences with regard to the HRQoL (Table 2). As the percentage of adjuvant patients was obviously higher in the exposed group, adjuvant patients showed higher usage willingness, whereas the nonexposed group (with a higher percentage of metastatic patients) showed less willingness. The ePRO willingness was 92% in exposed versus 59% in the nonexposed group.

    Identifying existing barriers is crucial for future implementation of ePROs in routine clinical practice. The patients were asked whether there are any existing barriers related to privacy, technology, or disease that would negatively influence their willingness to use technology-based surveys. Multiple barriers in seven dimensions were identified, and most of them showed statistically significant differences between both groups in favor of the exposed patients (Figure 1). The most evident item was concern about data security, followed by two technological barriers (lack of technical knowledge; experience and discomfort when using technology). All barriers with statistically significant differences were reported more often in nonexposed patients. In contrast, differences in the burden of the disease as a reason for nonusage were not significant between both groups.

    Table 1. Sociodemographic characteristics of exposed and nonexposed treatment groups. Statistically significant values presented in italics.
    Sociodemographic variablesExposed
    (n=106)    		95% CINonexposed
    (n=96)    		95% CIP value
    (alpha=.05)
    Age in years





    		Mean (median)51.0 (52)
    		56.68 (54)
    		.001

    		Standard deviation [range (minimum-maximum)]11.31 [54 (30-84)]
    		12.38 [60 (20-85)]

    Level of education (1=lowest; 6=highest)





    		Median3.0
    		3.0
    		.03

    		Interquartile range (25%-quartile-75%-quartile)2.0 (3.0-5.0)
    		2.0 (2.0-4.0)
    		.03

    		No qualification, n (%)1 (.9)(0.00-0.06)1 (1)(0.00-0.07).94

    		Main/secondary school leaving certificate, n (%)43 (40.6)(0.32-0.50)59 (61)(0.53-0.69).003

    		Advanced technical certificate, n (%)19 (17.9)(0.10-0.26)15 (16)(0.08-0.23).67

    		High school diploma (“Abitur”), n (%)33 (31.1)(0.22-0.40)13 (14)(0.07-0.22).003

    		Not specified, n (%)10 (9.4)(0.02-0.15)8 (8)(0.01-0.13).78
    Therapy setting





    		Metastatic, n (%)30 (28.3)(0.19-0.35)65 (68)(0.62-0.76).001

    		Adjuvant treatment, n (%)76 (71.7)(0.61-0.83)31 (32)(0.26-0.37).001
    Health-related quality of life (EORTC QLQ C-30)a





    		Mean (median)60.8 (66.67)(0.55-0.66)58.1 (58.3)(0.52-0.63).45

    		Standard deviation [range (minimum-maximum)]23.75 [100 (0-100)]
    		21.0 [91.7 (0-91.7)]
    		.45

    aEORTC QLQ‑C30: European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-Core 30 item.

    Table 2. Self-reported technical skills for metastatic and adjuvant patients. Statistically significant values presented in italics.
    Technical skills and ePRO evaluationExposed95% CINonexposed95% CIP value
    (alpha=.05)
    Computer skills (self-perception by the patients, 1=lowest; 4=highest)

    		Median3.0
    		2.0
    		<.001

    		Interquartile range (25%-quartile-75%-quartile)0.0 (3.0-3.0)
    		1.0 (2.0-3.0)
    		<.001


    		n=99
    		n=81


    		Beginner/no skills (=1), n (%)4 (4)(0.01-0.08)10 (12)(0.06-0.18).04

    		Basic (=2), n (%)20 (20)(0.12-0.28)37 (46)(0.37-0.58)<.001

    		Advanced (=3), n (%)68 (69)(0.58-0.77)30 (37)(0.27-0.47)<.001

    		Professional (=4), n (%)7 (7)(0.03-0.13)4 (5)(0.01-0.11).55
    Computer use, in years





    		Mean (standard deviation)17.49 (7.12)
    		16.73 (8.25)
    		.52
    Internet use, in years





    		Mean (standard deviation)13.57 (5.60)
    		11.84 (6.53)
    		.07
    Tablet PC use (1=lowest; 4=highest)





    		Median3.0
    		1.5
    		<.001

    		Interquartile range (25%-quartile-75%-quartile)3.0 (1.0-4.0)
    		2.0 (1.0-30)
    		<.001


    		n=94
    		n=66


    		Not at all (=1), n (%)33 (35)(0.26-0.45)33 (50)(0.35-0.63).06

    		A little (=2), n (%)6 (6)(0.02-0.12)10 (15)(0.08-0.23).07

    		Moderate (=3), n (%)13 (14)(0.07-0.21)19 (29)(0.20-0.42).02

    		Very much (=4), n (%)42 (45)(0.34-0.54)4 (6)(0.02-0.14)<.001


    		n=104
    		n=86

    Willingness to use technology-based surveys (ePRO), n (%)96 (92.3)(0.90-0.99)51 (59)(0.49-0.70).001


    		n=92
    		n=56

    Do you think that the introduction of electronic surveys will improve clinical care?, n (%)87 (95)(0.89-0.99)45 (80)(0.70-0.89).007
    Comparison of e-based and paper-based questionnaires

    		ePRO is less suitable (=1), more suitable (=5)






    		Median4.0
    		3.0
    		<.001


    		Interquartile range (25%-quartile-75%-quartile)2.0 (3.0-5.0)
    		1.25 (3.0-4.25)
    		<.001

    		ePRO is more tiring (=1), less tiring (=5)






    		Median4.0
    		3.0
    		<.001


    		Interquartile range (25%-quartile-75%-quartile)2.0 (3.0-5.0)
    		1.0 (3.0-4.0)
    		<.001

    		ePRO is more difficult (=1), less difficult (=5)






    		Median4.0
    		3.0
    		<.001


    		Interquartile range (25%-quartile-75%-quartile)2.0 (3.0-5.0)
    		2.0 (2.0-4.0)
    		<.001

    aePRO: electronic-based patient-reported outcome.

    Table 3. Relative risks of willingness to use and different barriers in exposed patients in relation to the nonexposed group. Statistically significant values presented in italics.
    Willingness to use and barriersRelative risk in exposed patients (95% CI)
    Willingness to use ePROa11.834 (4.405-31.794)
    Data privacy issues0.371 (0.179-0.769)
    Lack of technical knowledge/experience0.372 (0.138-1.006)
    Discomfort when using technology0.243 (0.77-0.761)
    I am afraid of damaging the device-
    No Internet access0.120 (0.15-0.976)
    Burden of disease preventing ePRO usage (mental dimension)0.363 (0.093-1.411)
    Burden of disease preventing ePRO usage (somatic dimension)2.089 (0.373-11.687)

    aePRO: electronic-based patient-reported outcome.

    Relative Risks, Regression, and Mediation Analyses

    Table 3 shows the results of the probability analyses. It is apparent that the probability of willingness to use is almost 11 times higher after exposure in this collective, whereas the relative risks of existing barriers are obviously lower (especially data privacy issues and discomfort when using technology).

    The CATREG analysis substantiates a statistically significant regression context between ePRO exposition and willingness, whereas the influence of the identified barriers was only low and partly not significant (Table 4) because the respective sample sizes of patients with existing barriers were too small for a valid calculation. Overall, 16.6% of the cases with expressed willingness to use can be attributed to exposure. Mediation effects of age and computer skills against the influence of exposure on willingness to use were only low (Table 5), whereas the mediation influence of education, HRQoL, and therapy setting were not statistically significant because the differences between exposed and nonexposed patients were too small. Including the variables of age and computer skills toward influence of exposure increased the explainability of the willingness to use aspect to 31.9%.

    Needs and Possible Technological Support Structures

    After finding strongly distinct barriers for ePRO among nonexposed patients, we asked them how they would rate the importance of 5 possible support services to help them complete a Web-based questionnaire about medical treatment, side effects, health status, and HRQoL (Table 6). On-site support services were rated as being moderately or highly important by 38%. A total of 32% patients expressed desire for a technical briefing for relatives who would support them while using the ePRO tool. Technical telephone support was rated as moderately important or very important by 52% of the nonexposed patients. The most relevant topic was data security, and 71% of the patients wanted to have full information regarding data protection measures (moderate and high importance). At least 61% would appreciate receiving direct feedback after using the ePRO application.

    Table 4. Categorical regression analyses. Statistically significant values presented in italics.
    Influence of exposureRR2BetaP value
    (alpha=.05)
    Willingness to use ePROa.407.166.407<.001
    Data privacy issues.207.043−.207.004
    Lack of technical knowledge/experience.129.017−.129<.001
    Discomfort when using technology.166.028−.166.01
    I am afraid of damaging the device.052.003−.052.43
    No Internet access.138.019−.138.02
    Burden of disease preventing ePRO usage (mental dimension).106.011−.106.12
    Burden of disease preventing ePRO usage (somatic dimension).045.002−.045.52

    aePRO: electronic-based patient-reported outcome.

    Table 5. Willingness to use: mediation effect of sociodemographics, skills, and health-related quality of life.
    Willingness to use: mediation effect of variablesRModR2ModP value
    (alpha=.05)    		Indirect effect of Xa on Yb95% CI
    Age.246.062<.001.363(0.073-0.867)
    Level of education.152.023.04.235(0.017-0.608)
    Computer skills.302.091<.001.536(0.196-1.976)
    Health-related quality of life.085.007.28.057(−0.035 to 0.353)
    Therapy setting.092.008.18.63(−0.042 to 0.157)
    R2ges = R2 + R2Mod / Age + R2Mod / Skills = .166 + .062 + .091 = .319

    aX=exposure/no exposure.

    bY=willingness to use.

    Table 6. Electronic-based patient-reported outcome preferences regarding technical support structures: How important would you rate the following support services to complete an electronic-based patient-reported outcome questionnaire during the hospital visit about your medical (after) treatment, your side effects, your health status, and your quality of life?
    Support variablesNonexposed setting


    		n (%)95% CI
    Technical briefing and onboarding completed on site, (N=69)


    		Not at all26 (38)(0.28-0.53)

    		A little16 (23)(0.15-0.37)

    		Moderate13 (19)(0.07-0.23)

    		Very much14 (20)(0.10-0.32)
    Technical briefing should include relatives, (N=64)


    		Not at all35 (55)(0.42-0.68)

    		A little9 (14)(0.07-0.25)

    		Moderate8 (13)(0.05-0.22)

    		Very much12 (19)(0.08-0.27)
    Telephone support, (N=64)


    		Not at all17 (27)(0.18-0.40)

    		A little14 (22)(0.12-0.32)

    		Moderate12 (22)(0.12-0.32)

    		Very much19 (30)(0.18-0.42)
    Transparency of data privacy, (N=70)


    		Not at all11 (16)(0.10-0.28)

    		A little9 (13)(0.06-0.24)

    		Moderate12 (17)(0.08-0.27)

    		Very much38 (54)(0.38-0.63)
    I get a direct feedback (from a doctor or the hospital), (N=68)


    		Not at all17 (25)(0.18-0.40)

    		A little10 (15)(0.07-0.27)

    		Moderate14 (21)(0.10-0.28)

    		Very much27 (40)(0.25-0.50)

    Principal Findings

    The majority of breast cancer patients expressed interest in adopting ePRO based on the impression that ePRO would positively impact hospital care and based on enhanced usability (more suitable, less tiring, and less difficult to read than paper-based PRO). Differences in relation to the setting of the survey and the patient’s self-reported health status were significant because the HRQoL was higher and the number of metastatic patients was lower in the exposed group. Patients in the nonexposed group more often had reservations and were critical toward ePRO, and their willingness to use corresponding tools was because of the following barriers: Patients were often afraid of using technical devices such as tablet PCs, (especially those with metastatic diseases in the nonexposed group), and they were concerned about data privacy issues and disease-related barriers (Figure 1). Thus, the willingness to participate in ePRO assessments can be increased by offering an ePRO tool, and the influence of barriers can also be reduced in metastatic patients. Our data demonstrated that patients generally had prevalent reservations regarding electronic assessment before exposure; however, they showed willingness to use electronic assessments after exposure. Whereas 16.6% of the cases expressing willingness to use were attributed to exposure (Table 4), mediation effects of age and computer skills against exposure’s influence were only low (Table 5). We found higher barriers in the nonexposed group characterized by lower HRQoL and a higher number of metastatic patients (Table 1), which suggests that health status influences the acceptance of ePRO and the emergence of barriers. The dimensions of reach and effectiveness of the RE-AIM framework could be analyzed for future improvements. The development of ideal ePRO tools has to consider the identified barriers (technical skills, HRQoL, and sociodemographic aspects) for utilization of ePRO, preferably in the general patient population and independent of their multidimensional characteristics.

    Comparison With Prior Work

    The results of this study contrast with those of a previous study, which identified no differences in the feasibility assessment of ePRO in relation to HRQoL [38]. The number of ePRO systems has increased in recent years, especially in oncology clinical practice, but other studies did not focus on the possible barriers to usability [57,58]. We have not found any studies in which cancer patients were asked about their barriers. Our group previously showed that older mBC patients (>62 years) with higher burden of disease may be less willing to complete ePRO questionnaires [46]. In this study, some significant barriers in relation to HRQoL, survey setting, and cancer-related restrictions were identified, whereas other reports only described the acceptance of ePRO without ascertaining barriers [40-42,57,58]. Our results agree with Basch et al [38], who reported higher self-reported computer experience (and thus potentially higher acceptance for ePRO) in patients with higher HRQoL. No other studies identified specific barriers related to technical skills, HRQoL, and sociodemographic issues as predictive factors for nonresponse in ePRO.

    Limitations and Relevance

    Our study was developed as a bicentric trial, and the patients were surveyed while they were receiving chemotherapy intervention. We did not enquire about the tumor stage, extent of metastasis, and the administered therapy. Furthermore, psycho-oncological information was not gathered, although psycho-oncological distress is a commonly associated burden that could potentially influence the willingness to use ePRO. There was no significant mediation effect of the therapy setting (aBC and mBC), although the number of metastatic patients was significantly higher in the nonexposed group. Also, HRQoL seemed not to be an influencing factor for willingness to use, as there were no significant differences in relation to HRQoL between exposed and nonexposed patients and no significant mediation effect. As it is known that low HRQoL and metastatic situation influence the willingness to use [46], willingness was assumed to be poor in mBC patients at the beginning, because metastasis was associated with poorer HRQoL. Probably, there were no differences in HRQoL (both in comparison of exposed and nonexposed patients as well as in the intragroup analyses), but this hypothesis could not be confirmed in this study. Hence, it can be postulated that a metastasis situation has a negative effect on usability compared with patients in adjuvant therapy especially if it results in a poorer HRQoL. An indirect effect was shown by the fact that for the exposed group significantly less metastasized patients could be recruited. The aspects of age and computer skills appeared as significant limitations, as exposed patients were significantly younger and had significantly better skills, which indicates that especially younger patients with previous experience in technology could be motivated to use ePRO.

    The most important result of the study was the fact that the survey setting (nonexposed vs exposed setting) could influence the willingness to use ePRO and the probability of barriers in all mBC and aBC breast cancer patients. The willingness among exposed patients was higher, as only the patients who could envisage answering HRQoL questions with a tablet could be included in this study. In total, 130 patients declined to participate in this group, so the total impact might be negligible; however, this limitation generally occurs in other ePRO trials. Because patients with barriers were rather unwilling to take part in the study, it is unclear how exposed patients are influenced by the approach of the study personnel to participate. Therefore, the barriers in the nonexposed group must be taken seriously because they could also represent patients with potential reservations about ePRO. The comparison between nonexposed and exposed patients shows that the willingness among women with breast cancer can be increased, and barriers can be reduced by educating the patients. To prevent statistical bias in future surveys and to increase the reliability of ePRO questionnaires, the identified barriers must be eliminated. Patients with cancer, who are often limited by their disease, should be thoroughly informed about privacy security issues and the universal handling of such confidential information to address their concerns and increase their potential willingness to use ePRO applications.

    Conclusions

    Although general patient acceptance of ePRO was high, we identified technical and disease-related barriers. These findings underscore the need to be aware of such barriers and to eliminate them to enhance the practicability of ePRO and ensure data accuracy, reliability, and validity for future ePRO assessments to measure HRQoL. Whereas fewer preexisting barriers were found in younger breast cancer patients, older patients with poorer HRQoL and less preexisting technical skills more frequently reported barriers for ePRO. Our study showed that barriers can be overcome after exposure and the willingness to participate in ePRO assessments significantly increased. Hence, the dimensions of reach and effectiveness of the RE-AIM framework, in particular, were analyzed in this paper. The development of ideal ePRO tools has to consider the identified barriers (technical skills, HRQoL, and sociodemographic aspects) for the utilization of ePRO, preferably in the general patient population and independent of their multidimensional characteristics. Tailored educational and support services need to be implemented and evaluated in future research to relieve reservations and increase ePRO compliance. Willingness to use ePRO is dependent on sociodemographic aspects, technical skills, HRQoL, and therapy setting, but patients’ acceptance of the tool can be increased when they experience it firsthand.

    Conflicts of Interest

    PG has received non-declared honoraria from Novartis.

    1. Reilly CM, Bruner DW, Mitchell SA, Minasian LM, Basch E, Dueck AC, et al. A literature synthesis of symptom prevalence and severity in persons receiving active cancer treatment. Support Care Cancer 2013 Jun;21(6):1525-1550 [FREE Full text] [CrossRef] [Medline]
    2. Fromme EK, Eilers KM, Mori M, Hsieh Y, Beer TM. How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30. J Clin Oncol 2004 Sep 1;22(17):3485-3490. [CrossRef] [Medline]
    3. Deshpande PR, Rajan S, Sudeepthi BL, Abdul NC. Patient-reported outcomes: a new era in clinical research. Perspect Clin Res 2011 Oct;2(4):137-144 [FREE Full text] [CrossRef] [Medline]
    4. FDA. 2009. Guidance for industry: patient-reported outcome measures; use in medical product development to support labeling claims, Stand   URL: https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf [accessed 2017-07-14] [WebCite Cache]
    5. Snyder CF, Jensen RE, Segal JB, Wu AW. Patient-reported outcomes (PROs): putting the patient perspective in patient-centered outcomes research. Med Care 2013 Aug;51(8 Suppl 3):S73-S79 [FREE Full text] [CrossRef] [Medline]
    6. Bitton A, Onega T, Tosteson AN, Haas JS. Toward a better understanding of patient-reported outcomes in clinical practice. Am J Manag Care 2014 Apr;20(4):281-283 [FREE Full text] [Medline]
    7. Brettschneider C, Lühmann D, Raspe H. Informative value of patient reported outcomes (PRO) in health technology assessment (HTA). GMS Health Technol Assess 2011 Feb 02;7:Doc01 [FREE Full text] [CrossRef] [Medline]
    8. Fung CH, Hays RD. Prospects and challenges in using patient-reported outcomes in clinical practice. Qual Life Res 2008 Dec;17(10):1297-1302 [FREE Full text] [CrossRef] [Medline]
    9. Valderas JM, Alonso J. Patient reported outcome measures: a model-based classification system for research and clinical practice. Qual Life Res 2008 Nov;17(9):1125-1135. [CrossRef] [Medline]
    10. Sears ED, Chung KC. A guide to interpreting a study of patient-reported outcomes. Plast Reconstr Surg 2012 May;129(5):1200-1207 [FREE Full text] [CrossRef] [Medline]
    11. Weldring T, Smith SM. Patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs). Health Serv Insights 2013 Aug;6:61-68 [FREE Full text] [CrossRef] [Medline]
    12. Baumhauer JF, Bozic KJ. Value-based healthcare: patient-reported outcomes in clinical decision making. Clin Orthop Relat Res 2016 Jun;474(6):1375-1378. [CrossRef] [Medline]
    13. Hays R, Reeve B. Measurement and modeling of health-related quality of life. In: International Encyclopedia of Public Health. New York, NY: Elsevier Inc; 2008:241-252.
    14. Ghosh RK, Ghosh SM, Ganguly G. Health-related quality of life and its growing importance in clinical practice. N Z Med J 2010 Apr 30;123(1313):99-101. [Medline]
    15. Wiklund I. Assessment of patient-reported outcomes in clinical trials: the example of health-related quality of life. Fundam Clin Pharmacol 2004 Jun;18(3):351-363. [CrossRef] [Medline]
    16. Patrick DL, Burke LB, Powers JH, Scott JA, Rock EP, Dawisha S, et al. Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value Health 2007;10 Suppl 2:S125-S137 [FREE Full text] [CrossRef] [Medline]
    17. Willke RJ, Burke LB, Erickson P. Measuring treatment impact: a review of patient-reported outcomes and other efficacy endpoints in approved product labels. Control Clin Trials 2004 Dec;25(6):535-552. [CrossRef] [Medline]
    18. EMA (European Medicines Agency). EMA. Reflection paper on the regulatory guidance for the use of health related quality of life (HRQL) measures in the evaluation of medicinal products, stand   URL: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003637.pdf [accessed 2017-07-14] [WebCite Cache]
    19. Atkinson TM, Ryan SJ, Bennett AV, Stover AM, Saracino RM, Rogak LJ, et al. The association between clinician-based common terminology criteria for adverse events (CTCAE) and patient-reported outcomes (PRO): a systematic review. Support Care Cancer 2016 Aug;24(8):3669-3676. [CrossRef] [Medline]
    20. Eton DT, Elraiyah TA, Yost KJ, Ridgeway JL, Johnson A, Egginton JS, et al. A systematic review of patient-reported measures of burden of treatment in three chronic diseases. Patient Relat Outcome Meas 2013;4:7-20 [FREE Full text] [CrossRef] [Medline]
    21. Coon CD. The use of patient-reported outcomes in demonstrating safety and efficacy in oncology. Clin Ther 2016 Apr;38(4):756-758. [CrossRef] [Medline]
    22. Efficace F, Fayers P, Pusic A, Cemal Y, Yanagawa J, Jacobs M, European Organization for Research and Treatment of Cancer Quality-of-Life Group (Patient-Reported Outcome Measurements Over Time in Oncology Registry). Quality of patient-reported outcome reporting across cancer randomized controlled trials according to the CONSORT patient-reported outcome extension: A pooled analysis of 557 trials. Cancer 2015 Sep 15;121(18):3335-3342 [FREE Full text] [CrossRef] [Medline]
    23. Velikova G, Keding A, Harley C, Cocks K, Booth L, Smith AB, et al. Patients report improvements in continuity of care when quality of life assessments are used routinely in oncology practice: secondary outcomes of a randomised controlled trial. Eur J Cancer 2010 Sep;46(13):2381-2388. [CrossRef] [Medline]
    24. Basch E. New frontiers in patient-reported outcomes: adverse event reporting, comparative effectiveness, and quality assessment. Annu Rev Med 2014;65:307-317. [CrossRef] [Medline]
    25. Brim RL, Pearson SD. The use and reporting of patient-reported outcomes in phase III breast cancer trials. Clin Trials 2013 Apr;10(2):243-249. [CrossRef] [Medline]
    26. Hurvitz SA, Lalla D, Crosby RD, Mathias SD. Use of the metastatic breast cancer progression (MBC-P) questionnaire to assess the value of progression-free survival for women with metastatic breast cancer. Breast Cancer Res Treat 2013 Dec;142(3):603-609 [FREE Full text] [CrossRef] [Medline]
    27. Kanatas A, Velikova G, Roe B, Horgan K, Ghazali N, Shaw RJ, et al. Patient-reported outcomes in breast oncology: a review of validated outcome instruments. Tumori 2012 Nov;98(6):678-688 [FREE Full text] [CrossRef] [Medline]
    28. Welslau M, Diéras V, Sohn J, Hurvitz SA, Lalla D, Fang L, et al. Patient-reported outcomes from EMILIA, a randomized phase 3 study of trastuzumab emtansine (T-DM1) versus capecitabine and lapatinib in human epidermal growth factor receptor 2-positive locally advanced or metastatic breast cancer. Cancer 2014 Mar 1;120(5):642-651 [FREE Full text] [CrossRef] [Medline]
    29. Di MM, Basch E, Bryce J, Perrone F. Patient-reported outcomes in the evaluation of toxicity of anticancer treatments. Nat Rev Clin Oncol 2016 May;13(5):319-325. [CrossRef] [Medline]
    30. Ligibel JA, Giobbie-Hurder A, Shockro L, Campbell N, Partridge AH, Tolaney SM, et al. Randomized trial of a physical activity intervention in women with metastatic breast cancer. Cancer 2016 Apr 15;122(8):1169-1177. [CrossRef] [Medline]
    31. Hansen DG, Larsen PV, Holm LV, Rottmann N, Bergholdt SH, Søndergaard J. Association between unmet needs and quality of life of cancer patients: a population-based study. Acta Oncol 2013 Feb;52(2):391-399. [CrossRef] [Medline]
    32. Basch E. Patient-reported outcomes in drug safety evaluation. Ann Oncol 2009 Dec;20(12):1905-1906 [FREE Full text] [CrossRef] [Medline]
    33. Basch E, Jia X, Heller G, Barz A, Sit L, Fruscione M, et al. Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes. J Natl Cancer Inst 2009 Dec 2;101(23):1624-1632 [FREE Full text] [CrossRef] [Medline]
    34. Basch E, Goldfarb S. Electronic patient-reported outcomes for collecting sensitive information from patients. J Support Oncol 2009;7(3):98-99. [Medline]
    35. Campbell N, Ali F, Finlay AY, Salek SS. Equivalence of electronic and paper-based patient-reported outcome measures. Qual Life Res 2015 Aug;24(8):1949-1961. [CrossRef] [Medline]
    36. Li J, Talaei-Khoei A, Seale H, Ray P, Macintyre CR. Health care provider adoption of ehealth: systematic literature review. Interact J Med Res 2013;2(1):e7 [FREE Full text] [CrossRef] [Medline]
    37. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011 Jan;8(1):e1000387 [FREE Full text] [CrossRef] [Medline]
    38. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 2016 Feb 20;34(6):557-565. [CrossRef] [Medline]
    39. Judson TJ, Bennett AV, Rogak LJ, Sit L, Barz A, Kris MG, et al. Feasibility of long-term patient self-reporting of toxicities from home via the Internet during routine chemotherapy. J Clin Oncol 2013 Jul 10;31(20):2580-2585 [FREE Full text] [CrossRef] [Medline]
    40. Schwartzberg L. Electronic patient-reported outcomes: the time is ripe for integration into patient care and clinical research. Am Soc Clin Oncol Educ Book 2016;35:e89-e96 [FREE Full text] [CrossRef] [Medline]
    41. Coons SJ, Gwaltney CJ, Hays RD, Lundy JJ, Sloan JA, Revicki DA, ISPOR ePRO Task Force. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value Health 2009 Jun;12(4):419-429. [CrossRef] [Medline]
    42. Barentsz MW, Wessels H, van Diest PJ, Pijnappel RM, Haaring C, van der Pol CC, et al. Tablet, web-based, or paper questionnaires for measuring anxiety in patients suspected of breast cancer: patients' preferences and quality of collected data. J Med Internet Res 2014;16(10):e239 [FREE Full text] [CrossRef] [Medline]
    43. Richter JG, Becker A, Koch T, Nixdorf M, Willers R, Monser R, et al. Self-assessments of patients via Tablet PC in routine patient care: comparison with standardised paper questionnaires. Ann Rheum Dis 2008 Dec;67(12):1739-1741. [CrossRef] [Medline]
    44. Steele GC, Gill A, Khan AI, Hans PK, Kuluski K, Cott C. The electronic patient reported outcome tool: testing usability and feasibility of a mobile app and portal to support care for patients with complex chronic disease and disability in primary care settings. JMIR Mhealth Uhealth 2016 Jun 02;4(2):e58 [FREE Full text] [CrossRef] [Medline]
    45. Schäffeler N, Pfeiffer K, Grischke EM, Wallwiener D, Garbe C, Zipfel S, et al. [Acceptance and reliability of an electronic psychooncological screening of patients with breast cancer: a randomized controlled study]. Psychother Psychosom Med Psychol 2013 Sep;63(9-10):374-380. [CrossRef] [Medline]
    46. Graf J, Simoes E, Wißlicen K, Rava L, Walter CB, Hartkopf A, et al. Willingness of patients with breast cancer in the adjuvant and metastatic setting to use electronic surveys (ePRO) depends on sociodemographic factors, health-related quality of life, disease status and computer skills. Geburtshilfe Frauenheilkd 2016 May;76(5):535-541 [FREE Full text] [CrossRef] [Medline]
    47. Levine M, Richardson JE, Granieri E, Reid MC. Novel telemedicine technologies in geriatric chronic non-cancer pain: primary care providers' perspectives. Pain Med 2014 Feb;15(2):206-213 [FREE Full text] [CrossRef] [Medline]
    48. Narasimha S, Madathil KC, Agnisarman S, Rogers H, Welch B, Ashok A, et al. Designing telemedicine systems for geriatric patients: a review of the usability studies. Telemed J E Health 2017 Jun;23(6):459-472. [CrossRef] [Medline]
    49. Srinivasan S, Glover J, Bouknight J. Geriatric Telepsychiatry: A Clinician's Guide. Cham, Switzerland: Springer International Publishing AG; 2017.
    50. Fasching PA, Brucker SY, Fehm TN, Overkamp F, Janni W, Wallwiener M, et al. Biomarkers in patients with metastatic breast cancer and the PRAEGNANT study network. Geburtshilfe Frauenheilkd 2015 Jan;75(1):41-50 [FREE Full text] [CrossRef] [Medline]
    51. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, et al. The European organization for research and treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 1993 Mar 3;85(5):365-376. [Medline]
    52. EORTC data center. EORTC. 2001. EORTC QLQ-C30 scoring manual   URL: http://www.eortc.be/qol/files/scmanualqlq-c30.pdf [accessed 2017-07-24] [WebCite Cache]
    53. Glasgow RE, Klesges LM, Dzewaltowski DA, Estabrooks PA, Vogt TM. Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues. Health Educ Res 2006 Oct;21(5):688-694 [FREE Full text] [CrossRef] [Medline]
    54. Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a systematic review of use over time. Am J Public Health 2013 Jun;103(6):e38-e46. [CrossRef] [Medline]
    55. Dusseldorp E, Meulman JJ. Prediction in medicine by integrating regression trees into regression analysis with optimal scaling. Methods Inf Med 2001;40(5):403-409. [Medline]
    56. Hayes AF, Scharkow M. The relative trustworthiness of inferential tests of the indirect effect in statistical mediation analysis: does method really matter? Psychol Sci 2013 Oct;24(10):1918-1927. [CrossRef] [Medline]
    57. Jensen RE, Snyder CF, Abernethy AP, Basch E, Potosky AL, Roberts AC, et al. Review of electronic patient-reported outcomes systems used in cancer clinical care. J Oncol Pract 2014 Jul;10(4):e215-e222 [FREE Full text] [CrossRef] [Medline]
    58. Bennett AV, Jensen RE, Basch E. Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin 2012;62(5):337-347 [FREE Full text] [CrossRef] [Medline]

    ‎
    aBC: adjuvant breast cancer
    AEs: adverse events
    CATREG: categorical regression
    eHealth: electronic health
    EORTC QLQ-C30: European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-Core 30 item
    ePRO: electronic-based patient-reported outcome
    FACT-G: Functional Assessment of Cancer Therapy-General
    HRQoL: health-related quality of life
    mBC: metastatic breast cancer
    PRO: patient-reported outcome
    PROCOM: Patient-Reported Outcomes and Compliance Analysis
    RE-AIM: Reach, Effectiveness, Adoption, Implementation, and Maintenance
    SPSS: Statistical Package for the Social Sciences

    Edited by M Focsa; submitted 15.11.16; peer-reviewed by R Jensen, M Teufel, S Bolboaca; comments to author 11.12.16; revised version received 12.01.17; accepted 29.05.17; published 07.08.17

    Copyright

    ©Andreas D Hartkopf, Joachim Graf, Elisabeth Simoes, Lucia Keilmann, Nina Sickenberger, Paul Gass, Diethelm Wallwiener, Lina Matthies, Florin-Andrei Taran, Michael P Lux, Stephanie Wallwiener, Eric Belleville, Christof Sohn, Peter A Fasching, Andreas Schneeweiss, Sara Y Brucker, Markus Wallwiener. Originally published in JMIR Cancer (http://cancer.jmir.org), 07.08.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Cancer, is properly cited. The complete bibliographic information, a link to the original publication on http://cancer.jmir.org, as well as this copyright and license information must be included.