This paper is in the following e-collection/theme issue:

Innovations and Technology in Cancer Care (422) Breast Cancer (206) Patient-Reported Outcome Measures (PROMs) (288)

Published on in Vol 4, No 2 (2018): Jul-Dec

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/11373, first published .
Web-Based Patient-Reported Outcomes Using the International Consortium for Health Outcome Measurement Dataset in a Major German University Hospital: Observational Study

Web-Based Patient-Reported Outcomes Using the International Consortium for Health Outcome Measurement Dataset in a Major German University Hospital: Observational Study

Web-Based Patient-Reported Outcomes Using the International Consortium for Health Outcome Measurement Dataset in a Major German University Hospital: Observational Study

Authors of this article:

Maria M Karsten1 Author Orcid Image ;   Dorothee Speiser1 Author Orcid Image ;   Claudia Hartmann2 Author Orcid Image ;   Nele Zeuschner1 Author Orcid Image ;   Kai Lippold3 Author Orcid Image ;   Verena Kiver1 Author Orcid Image ;   Peter Gocke4 Author Orcid Image ;   Valerie Kirchberger5 Author Orcid Image ;   Jens-Uwe Blohmer1 Author Orcid Image
Article Authors Cited by (18) Tweetations (2) Metrics

    Original Paper

    1Klinik für Gynäkologie mit Brustzentrum, Charité Universitätsmedizin Berlin, Berlin, Germany

    2Ärztliches Direktorat, Charité Universitätsmedizin Berlin, Berlin, Germany

    3Geschäftsstelle Vorstand, Charité Universitätsmedizin Berlin, Berlin, Germany

    4Stabstelle Digitale Transformation, Charité Universitätsmedizin Berlin, Berlin, Germany

    5Value-Based Healthcare, Geschäftsbereich Strategische Unternehmensentwicklung, Charité Universitätsmedizin Berlin, Berlin, Germany

    *these authors contributed equally

    Corresponding Author:

    Maria M Karsten, MD

    Klinik für Gynäkologie mit Brustzentrum

    Charité Universitätsmedizin Berlin

    Charitéplatz 1

    Berlin, 10117

    Germany

    Phone: 49 30450664279

    Email: maria-margarete.karsten@charite.de


    Background: Collecting patient-reported outcome (PRO) data systematically enables objective evaluation of treatment and its related outcomes. Using disease-specific questionnaires developed by the International Consortium for Health Outcome Measurement (ICHOM) allows for comparison between physicians, hospitals, and even different countries.

    Objective: This pilot project aimed to establish a digital system to measure PROs for new patients with breast cancer who attended the Charité Breast Center This approach should serve as a blueprint to further expand the PRO measurement to other disease entities and departments.

    Methods: In November 2016, we implemented a Web-based system to collect PRO data at Charité Breast Center using the ICHOM dataset. All new patients at the Breast Center were enrolled and answered a predefined set of questions using a tablet computer. Once they started their treatment at Charité, automated emails were sent to the patients at predefined treatment points. Those emails contained a Web-based link through which they could access and answer questionnaires.

    Results: By now, 541 patients have been enrolled and 2470 questionnaires initiated. Overall, 9.4% (51/541) of the patients were under the age of 40 years, 49.7% (269/541) between 40 and 60 years, 39.6% (214/541) between 60 and 80 years, and 1.3% (7/541) over the age of 80 years. The average return rate of questionnaires was 67.0%. When asked about the preference regarding paper versus Web-based questionnaires, 6.0% (8/134) of the patients between 50 and 60 years, 6.0% (9/150) between 60 and 70 years, and 12.7% (9/71) over the age of 70 years preferred paper versions.

    Conclusions: Measuring PRO in patients with breast cancer in an automated electronic version is possible across all age ranges while simultaneously achieving a high return rate.

    JMIR Cancer 2018;4(2):e11373

    doi:10.2196/11373

    Keywords

    breast cancer; International Council Health Outcome Measurement; mobile phone; patient-reported outcomes 


    In Germany, every year, 70,000 women receive the diagnosis of breast cancer. Almost 30% are under the age of 55 years. Due to improved screening and treatment modalities, there has been a significant improvement in overall survival in the last decade [1-3]. Breast cancer-specific mortality in Europe reduced from 17.90 women in 2002 to 15.20 per 100,000 women in 2012 [4]. However, survival gains are often associated with the loss of physical functioning; increased morbidity; and new challenges regarding the emotional, social, and financial aspects of life [5-8]. Therefore, this increase in life expectancy of patients with cancer must lead to increased scrutiny regarding the long-term side effects of new and existing cancer treatments [9,10]. An important aspect in evaluating effects of any therapies is patients’ voice and perception. This applies even more to patients with cancer. The best way to address this aspect is to use patient-reported outcome (PRO) measures. The US Food and Drug Agency describes PRO as “any report coming directly from patients about a health condition and its treatment” [11]. Nowadays, using electronic media, like smartphones and tablet computers, measuring PRO data is much easier and less time- and cost-consuming than it was in the past. The use of PRO data allows for the real-time evaluation of therapy concepts and monitoring of new treatments. At the same time, it is an easy way for a long-term follow-up. Increasingly complex therapies in medicine simultaneously require an increase in documentation—time missing in direct patient communication [12,13]. However, this time is essentially needed to adequately assess a patient’s situation and symptoms. PRO has been shown in multiple studies to help clinicians to adequately assess patient symptoms, save time for patient communication, and therefore improve patient care and even survival [14,15]. The aim of this study was to establish a PRO system in a major German university hospital.


    After obtaining ethics approval from the Charité Ethics Commission (EA 4/127/16), we implemented a Web-based system to collect PRO data at the Charité Breast Center. Data capturing started in November 2016. The PRO data collection was based on an international standard set for breast cancer outcome measures, which was developed by the International Consortium for Health Outcome Measurement (ICHOM). All new patients, who attended the breast clinic, were included in the PRO measurement. Afterward, those who had a diagnosis of breast cancer and received their treatment at Charité Breast Center were stratified into follow-up. After patients registered at the clinic, they were asked by the receptionist if they would be willing to participate and received a personal log-in after they signed consent. The waiting time until their appointment was used to answer the ICHOM questions as well as questions regarding their medical history on a tablet computer. The decision to use a tablet computer in the clinic setting was one of the comforts because it allowed the patients to continue sitting where they felt the most comfortable in the waiting area. The follow-up emails patients received were designed in a way that they could also be answered on a mobile device.

    After the successful completion of the questionnaire, answers and calculated PRO scores were immediately available to treating physicians for the upcoming consultation. During that, treating physicians had the option to add missing clinical data. If they decide not to record the clinical data necessary, it was later added by support staff. Once patients entered specific care pathways, like chemotherapy or surgery, an automated process was started through which they received follow-up emails containing the access code to their individual PRO measurement questionnaires. The follow-up emails were sent 6 weeks after their first treatment, then every 3 months thereafter for 2 years. After the first 2 years are finished, they will receive a follow-up questionnaire every 6 months for another 3 years and after this only on a yearly basis.

    From a technical standpoint, the system for PRO collection was installed on campus as an on-premise installation. It was therefore only available within the Charité network. The core system was supported by an additional patient portal, which acted as an outward facing tool, to interact with patients. The patient portal was hosted in a different environment to allow for access to the Web. It enables patients to complete questionnaires from home using a secure connection.


    Monthly Increase in the Number of Patients Who Participate in the Patient-Reported Outcome Measurement Since the Program Implementation

    Figure 1 shows monthly increase in the number of patients who were entered into the PRO evaluation at the Breast Center and agreed to participate after its implementation in November 2016. After an initial increase in January 2017 with 40 patients, there was a decline in participation, with the lowest rate in March 2017, with only 4 patients included. From July 2017, there was a marked increase in patient numbers—a trend that continued from there on.

    Increase in Patients Numbers for Patient-Reported Outcome Measurement Compared With the Total Number of New Patients

    Table 1 compares the number of new patients who entered into the PRO system with the total number of new patients seen at the Charité Breast Center. With the exemption of January 2017, where 34.4% (40/116) of the new patients were added to the PRO system, the percentage of all new patients stayed below 20% until July 2017, with the lowest percentage (4/166, 2.4%) in March 2017. After July 2017, there was a marked increase in adding new patients, almost continuously increasing and reaching its highest percentage in December 2017, with 73.5% (86/117) of all new patients included in the PRO system.

    Table 2 shows patients’ characteristics regarding educational levels.

    ‎
    Figure 1. Monthly increase in patient numbers for patient-reported outcome measurement since implementation.
    View this figure
    Table 1. Increase in patient numbers for patient-reported outcome measurement over time.
    PeriodNew patients seen, nPatients who participated in patient-reported outcome measurement, n (%)
    November 201610513 (12.3)
    December 2016907 (7.8)
    January 201711640 (34.4)
    February 201712410 (8.1)
    March 20171664 (2.4)
    April 201710420 (19.2)
    May 201713925 (17.9)
    June 20171129 (7.6)
    July 201711655 (47.4)
    August 201710854 (50.0)
    September 201716367 (41.1)
    October 20179359 (63.4)
    November 2017169115 (68.0)
    December 201711786 (73.5)
    Table 2. Level of academic training in all patients included in patient-reported outcome measurement (N=541).
    Education levelValue, n (%)
    Minimum German schooling requirement201 (37.1)
    Mid-level education153 (28.2)
    Higher level education187 (34.5)

    Decline in Patient Numbers to Participate in the Long-Term Follow-Up

    Figure 2 shows the percentage of breast cancer patients who agreed to participate in the electronic follow-up PRO measurement compared with those who did not want to participate. In the age group of 20-30 years, 66.67% (2/3) agreed to follow up; in the age group of 30-40 years, 75% (6/8) agreed to follow up. The highest number was seen in the age group of 40-50 years, with 90.91% (37/40) followed up; 75.76% (25/33) agreed to follow up in the age group 60-70 years. Participation levels were the same, with 75% in the age groups of 70-80 (12/16) and >80 (3/4) years.

    Return Rates of Questionnaires From Patients Participating in the Follow-Up Patient-Reported Outcome Measurement

    Figure 3 shows the return rates of the online questionnaires completed by those patients who participated in the follow-up PRO measurement. After an initial drop to 28.6% (12/42) in February 2017, there was a continuous increase in the return rate from 54.2% (13/24) in March of 2017 to 82.9% (68/82) in September 2017, with then another slow decline in October 2017 to 75.3 (70/93). November and December 2017 showed steady return rates above 81.3% (122/150) and 82.6% (114/138), respectively.

    Preference for Using a Digital Way to Measure Patient-Reported Outcome Compared With a Paper-Based Version Across Age Groups

    Figure 4 shows the percentage of patients who preferred a paper-based version of the questionnaire instead of a digital one. Notably, 100% (51/51) of the patients in the age groups of 20-40 years preferred a digital version, while 2.9% (7/244) of the patients in the age group of 40-50 years preferred a paper-based version. This increases to 5.9% (13/122) in the age group of 50-70 years and then further increases to 13.0% (3/23) in those aged 70-80 years. Above 80 years of age, there was a 100% (2/2) preference for paper-based questionnaires compared with digital questionnaires.

    ‎
    Figure 2. Patients who agreed to follow up versus who declined to follow up patient reported outcome measurement.
    View this figure
    ‎
    Figure 3. Return rates of the digital questionnaires completed by patients who participated in the long-term patient-reported outcome measurement.
    View this figure
    ‎
    Figure 4. Preference for paper versus digital questionnaire according to age groups.
    View this figure

    Principal Findings

    This study documents the successful implementation of measuring PROs using the ICHOM dataset for breast cancer in a German university hospital for the first time. During the implementation period, we made numerous observations. First, it takes at least 6 months to implement and establish a working system, get all key stakeholders to adopt it, while simultaneously solve those technical problems that arise during the implementation phase. Second, most patients, across all age groups, are willing to participate in the initial measurement as well as in the long-term follow-up. Third, contrary assumption, most patients, even those aged >60 years, prefer a digital survey over a paper-based way to answer the PRO questionnaires.

    Our observation regarding the required timeframe to establish a successful electronic PRO system is matched by previous publications [16-18]. A crucial point in establishing a successful digital program is to explain and educate all health care providers who will be involved in the collection of PROs about the purpose and benefit of PRO. While there is increasing interest and knowledge about PRO measurement, there are concerns regarding workflow, increase in workload due to the additional measurements, as well as data overload and creation of additional needs [19-21]. These findings are mirrored in our low patient accrual data in the first 6 months after the implementation. It took this time to train, educate, and convince all involved staff members from front desk staff to treating physicians. Once this barrier was broken, there was a steady increase in patients with breast cancer who agreed to participate in the follow-up PRO measurement. Only a small percentage of patients declined to participate in the PRO follow-up. The highest percentage of patients who agreed to follow up was in the age group of 40-50 years, at an astonishing 90.91% (37/40). The lowest rate was found in the age group of 20-30 years, with only 66.67% (2/3) agreeing to follow up. This is in part because of the low numbers of patients we have in this age group. Previous work has shown that especially technology literacy plays a role in the extent to which patients are participating in electronic-based PRO measurement [22]. We did not gather information regarding technology literacy but asked a question regarding the level of education patients had because there have been data showing a correlation between the level of education and the willingness to participate in Web-based data collection as well as the effects regarding PRO-based interventions [15,23]. As shown in Table 2, there was a fairly even distribution of educational levels in our patient population. Also, the access and ability to use a computer, smartphone, or smartwatch is a prerequisite, and therefore, patients who had neither of these and thereby often did not have an email address were not able to participate in the study. This could be a potential bias, especially regarding the return rates.

    In addition to the finding that it is possible to establish a successful PRO measurement program at a German breast center integrated into the routine clinical workflow, this work showed for the first time that the majority of patients with breast cancer treated in a German university hospital preferred a digital survey. Patients did not want to fill out questionnaires in a paper and pencil-based version—despite contrary believes. In addition to this interesting finding, we were able to show that we can simultaneously achieve a high adherence rate even in the long-term follow-up. Similar observations have been made previously in the United States, for example, at Memorial Sloan Kettering Cancer Center [24], the University of North Carolina [25], or Group Health Cooperative in Seattle [26].

    Limitations

    Since this is a retrospective analysis of an implementation trial, it is not without limitations. In the beginning, reasons for decline in the number of patients participating was not systematically collected. Since this systematic approach was started only at a later point, we currently do not have enough data on this matter but are collecting them now. Also, we did not have enough resources to be able to contact those patients who decided not to continue in the follow-up. This point and the first point are important aspects, and we are currently addressing both and plan to publish the results in a following publication.

    Comparison With Prior Work

    While there is an increasing interest in the potential of PRO measurements in almost all disease entities [27-30], there is still a lack of standards regarding what to measure and how to measure it. The ICHOM initiative has therefore created a working group for a wide range of diseases with the goal to establish standard sets to compare outcomes between different providers, hospitals, and even countries [31]. This is the first published work to show that the implementation of one of their standard sets—in our case for breast cancer—is feasible and lays the foundation for further improvements in the complex care of patients with breast cancer.

    Conclusions

    The goal of this pilot trial was to create a template on how to establish a successful Web-based PRO measurement system at a German university hospital, setting the stage for what to expect and showing that it is possible to measure PRO in a digital manner in patients with breast cancer of all age groups.

    Conflicts of Interest

    None declared.

    1. Carioli G, Malvezzi M, Rodriguez T, Bertuccio P, Negri E, La Vecchia C. Trends and predictions to 2020 in breast cancer mortality: Americas and Australasia. Breast 2018 Feb;37:163-169. [CrossRef] [Medline]
    2. Bergh J, Jönsson PE, Glimelius B, Nygren P, SBU-group. Swedish Council of Technology Assessment in Health Care. A systematic overview of chemotherapy effects in breast cancer. Acta Oncol 2001;40(2-3):253-281. [Medline]
    3. Coleman MP, Quaresma M, Berrino F, Lutz J, De Angelis R, Capocaccia R, CONCORD Working Group. Cancer survival in five continents: a worldwide population-based study (CONCORD). Lancet Oncol 2008 Aug;9(8):730-756 [FREE Full text] [CrossRef] [Medline]
    4. Carioli G, Malvezzi M, Rodriguez T, Bertuccio P, Negri E, La Vecchia C. Trends and predictions to 2020 in breast cancer mortality in Europe. Breast 2017 Dec;36:89-95. [CrossRef] [Medline]
    5. Seabury SA, Frasco MA, van Eijndhoven E, Sison S, Zacker C. The impact of self- and physician-administered cancer treatment on work productivity and healthcare utilization. Res Social Adm Pharm 2018 Dec;14(5):434-440. [CrossRef] [Medline]
    6. Spathis A, Booth S, Grove S, Hatcher H, Kuhn I, Barclay S. Teenage and Young Adult Cancer-Related Fatigue Is Prevalent, Distressing, and Neglected: It Is Time to Intervene. A Systematic Literature Review and Narrative Synthesis. J Adolesc Young Adult Oncol 2015 Mar;4(1):3-17 [FREE Full text] [CrossRef] [Medline]
    7. Capelan M, Battisti NML, McLoughlin A, Maidens V, Snuggs N, Slyk P, et al. The prevalence of unmet needs in 625 women living beyond a diagnosis of early breast cancer. Br J Cancer 2017 Oct 10;117(8):1113-1120 [FREE Full text] [CrossRef] [Medline]
    8. Lambert SD, Harrison JD, Smith E, Bonevski B, Carey M, Lawsin C, et al. The unmet needs of partners and caregivers of adults diagnosed with cancer: a systematic review. BMJ Support Palliat Care 2012 Sep;2(3):224-230. [CrossRef] [Medline]
    9. Jain V, Bahia J, Mohebtash M, Barac A. Cardiovascular Complications Associated With Novel Cancer Immunotherapies. Curr Treat Options Cardiovasc Med 2017 May;19(5):36. [CrossRef] [Medline]
    10. Byun DJ, Wolchok JD, Rosenberg LM, Girotra M. Cancer immunotherapy - immune checkpoint blockade and associated endocrinopathies. Nat Rev Endocrinol 2017 Dec;13(4):195-207 [FREE Full text] [CrossRef] [Medline]
    11. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res 2013 Jun 11;13:211 [FREE Full text] [CrossRef] [Medline]
    12. Tai-Seale M, Olson CW, Li J, Chan AS, Morikawa C, Durbin M, et al. Electronic Health Record Logs Indicate That Physicians Split Time Evenly Between Seeing Patients And Desktop Medicine. Health Aff (Millwood) 2017 Dec 01;36(4):655-662 [FREE Full text] [CrossRef] [Medline]
    13. Joukes E, Abu-Hanna A, Cornet R, de Keizer NF. Time Spent on Dedicated Patient Care and Documentation Tasks Before and After the Introduction of a Structured and Standardized Electronic Health Record. Appl Clin Inform 2018 Dec;9(1):46-53. [CrossRef] [Medline]
    14. Basch E, Jia X, Heller G, Barz A, Sit L, Fruscione M, et al. Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes. J Natl Cancer Inst 2009 Dec 02;101(23):1624-1632 [FREE Full text] [CrossRef] [Medline]
    15. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol 2016 Feb 20;34(6):557-565 [FREE Full text] [CrossRef] [Medline]
    16. Rotenstein LS, Huckman RS, Wagle NW. Making Patients and Doctors Happier - The Potential of Patient-Reported Outcomes. N Engl J Med 2017 Oct 05;377(14):1309-1312. [CrossRef] [Medline]
    17. Trautmann F, Hentschel L, Hornemann B, Rentsch A, Baumann M, Ehninger G, et al. Electronic real-time assessment of patient-reported outcomes in routine care-first findings and experiences from the implementation in a comprehensive cancer center. Support Care Cancer 2016 Dec;24(7):3047-3056. [CrossRef] [Medline]
    18. Antunes B, Harding R, Higginson IJ, EUROIMPACT. Implementing patient-reported outcome measures in palliative care clinical practice: a systematic review of facilitators and barriers. Palliat Med 2014 Feb;28(2):158-175. [CrossRef] [Medline]
    19. Donaldson MS. Taking PROs and patient-centered care seriously: incremental and disruptive ideas for incorporating PROs in oncology practice. Qual Life Res 2008 Dec;17(10):1323-1330. [CrossRef] [Medline]
    20. Jagsi R, Chiang A, Polite BN, Medeiros BC, McNiff K, Abernethy AP, et al. Qualitative analysis of practicing oncologists' attitudes and experiences regarding collection of patient-reported outcomes. J Oncol Pract 2013 Nov;9(6):e290-e297. [CrossRef] [Medline]
    21. Cowan RA, Suidan RS, Andikyan V, Rezk YA, Einstein MH, Chang K, et al. Electronic patient-reported outcomes from home in patients recovering from major gynecologic cancer surgery: A prospective study measuring symptoms and health-related quality of life. Gynecol Oncol 2016 Nov;143(2):362-366 [FREE Full text] [CrossRef] [Medline]
    22. Hartkopf AD, Graf J, Simoes E, Keilmann L, Sickenberger N, Gass P, et al. Electronic-Based Patient-Reported Outcomes: Willingness, Needs, and Barriers in Adjuvant and Metastatic Breast Cancer Patients. JMIR Cancer 2017 Aug 07;3(2):e11 [FREE Full text] [CrossRef] [Medline]
    23. Barentsz MW, Wessels H, van Diest PJ, Pijnappel RM, Haaring C, van der Pol CC, et al. Tablet, web-based, or paper questionnaires for measuring anxiety in patients suspected of breast cancer: patients' preferences and quality of collected data. J Med Internet Res 2014 Oct 31;16(10):e239 [FREE Full text] [CrossRef] [Medline]
    24. Judson TJ, Bennett AV, Rogak LJ, Sit L, Barz A, Kris MG, et al. Feasibility of long-term patient self-reporting of toxicities from home via the Internet during routine chemotherapy. J Clin Oncol 2013 Jul 10;31(20):2580-2585 [FREE Full text] [CrossRef] [Medline]
    25. Wood WA, Deal AM, Abernethy A, Basch E, Battaglini C, Kim YH, et al. Feasibility of frequent patient-reported outcome surveillance in patients undergoing hematopoietic cell transplantation. Biol Blood Marrow Transplant 2013 Mar;19(3):450-459 [FREE Full text] [CrossRef] [Medline]
    26. Aiello EJ, Taplin S, Reid R, Hobbs M, Seger D, Kamel H, et al. In a randomized controlled trial, patients preferred electronic data collection of breast cancer risk-factor information in a mammography setting. J Clin Epidemiol 2006 Jan;59(1):77-81. [CrossRef] [Medline]
    27. Mahmud I, Kelley T, Stowell C, Haripriya A, Boman A, Kossler I, et al. A Proposed Minimum Standard Set of Outcome Measures for Cataract Surgery. JAMA Ophthalmol 2015 Nov;133(11):1247-1252. [CrossRef] [Medline]
    28. Martin NE, Massey L, Stowell C, Bangma C, Briganti A, Bill-Axelson A, et al. Defining a standard set of patient-centered outcomes for men with localized prostate cancer. Eur Urol 2015 Mar;67(3):460-467 [FREE Full text] [CrossRef] [Medline]
    29. Haws BE, Khechen B, Guntin JA, Cardinal KL, Bohl DD, Singh K. Validity of PROMIS in minimally invasive transforaminal lumbar interbody fusion: a preliminary evaluation. J Neurosurg Spine 2018 Jul;29(1):28-33. [CrossRef] [Medline]
    30. Kotecha D, Ahmed A, Calvert M, Lencioni M, Terwee CB, Lane DA. Patient-Reported Outcomes for Quality of Life Assessment in Atrial Fibrillation: A Systematic Review of Measurement Properties. PLoS One 2016;11(11):e0165790 [FREE Full text] [CrossRef] [Medline]
    31. Ong WL, Schouwenburg MG, van Bommel ACM, Stowell C, Allison KH, Benn KE, et al. A Standard Set of Value-Based Patient-Centered Outcomes for Breast Cancer: The International Consortium for Health Outcomes Measurement (ICHOM) Initiative. JAMA Oncol 2017 May 01;3(5):677-685. [CrossRef] [Medline]

    ‎
    ICHOM: International Consortium for Health Outcome Measurement
    PRO: patient-reported outcomes

    Edited by G Eysenbach; submitted 22.06.18; peer-reviewed by J Heil, A Davoudi, P Jank; comments to author 01.08.18; accepted 10.09.18; published 20.12.18

    Copyright

    ©Maria M Karsten, Dorothee Speiser, Claudia Hartmann, Nele Zeuschner, Kai Lippold, Verena Kiver, Peter Gocke, Valerie Kirchberger, Jens-Uwe Blohmer. Originally published in JMIR Cancer (http://cancer.jmir.org), 20.12.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Cancer, is properly cited. The complete bibliographic information, a link to the original publication on http://cancer.jmir.org/, as well as this copyright and license information must be included.