This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Cancer, is properly cited. The complete bibliographic information, a link to the original publication on http://cancer.jmir.org/, as well as this copyright and license information must be included.
Collecting patient-reported outcome (PRO) data systematically enables objective evaluation of treatment and its related outcomes. Using disease-specific questionnaires developed by the International Consortium for Health Outcome Measurement (ICHOM) allows for comparison between physicians, hospitals, and even different countries.
This pilot project aimed to establish a digital system to measure PROs for new patients with breast cancer who attended the Charité Breast Center This approach should serve as a blueprint to further expand the PRO measurement to other disease entities and departments.
In November 2016, we implemented a Web-based system to collect PRO data at Charité Breast Center using the ICHOM dataset. All new patients at the Breast Center were enrolled and answered a predefined set of questions using a tablet computer. Once they started their treatment at Charité, automated emails were sent to the patients at predefined treatment points. Those emails contained a Web-based link through which they could access and answer questionnaires.
By now, 541 patients have been enrolled and 2470 questionnaires initiated. Overall, 9.4% (51/541) of the patients were under the age of 40 years, 49.7% (269/541) between 40 and 60 years, 39.6% (214/541) between 60 and 80 years, and 1.3% (7/541) over the age of 80 years. The average return rate of questionnaires was 67.0%. When asked about the preference regarding paper versus Web-based questionnaires, 6.0% (8/134) of the patients between 50 and 60 years, 6.0% (9/150) between 60 and 70 years, and 12.7% (9/71) over the age of 70 years preferred paper versions.
Measuring PRO in patients with breast cancer in an automated electronic version is possible across all age ranges while simultaneously achieving a high return rate.
In Germany, every year, 70,000 women receive the diagnosis of breast cancer. Almost 30% are under the age of 55 years. Due to improved screening and treatment modalities, there has been a significant improvement in overall survival in the last decade [
After obtaining ethics approval from the Charité Ethics Commission (EA 4/127/16), we implemented a Web-based system to collect PRO data at the Charité Breast Center. Data capturing started in November 2016. The PRO data collection was based on an international standard set for breast cancer outcome measures, which was developed by the International Consortium for Health Outcome Measurement (ICHOM). All new patients, who attended the breast clinic, were included in the PRO measurement. Afterward, those who had a diagnosis of breast cancer and received their treatment at Charité Breast Center were stratified into follow-up. After patients registered at the clinic, they were asked by the receptionist if they would be willing to participate and received a personal log-in after they signed consent. The waiting time until their appointment was used to answer the ICHOM questions as well as questions regarding their medical history on a tablet computer. The decision to use a tablet computer in the clinic setting was one of the comforts because it allowed the patients to continue sitting where they felt the most comfortable in the waiting area. The follow-up emails patients received were designed in a way that they could also be answered on a mobile device.
After the successful completion of the questionnaire, answers and calculated PRO scores were immediately available to treating physicians for the upcoming consultation. During that, treating physicians had the option to add missing clinical data. If they decide not to record the clinical data necessary, it was later added by support staff. Once patients entered specific care pathways, like chemotherapy or surgery, an automated process was started through which they received follow-up emails containing the access code to their individual PRO measurement questionnaires. The follow-up emails were sent 6 weeks after their first treatment, then every 3 months thereafter for 2 years. After the first 2 years are finished, they will receive a follow-up questionnaire every 6 months for another 3 years and after this only on a yearly basis.
From a technical standpoint, the system for PRO collection was installed on campus as an on-premise installation. It was therefore only available within the Charité network. The core system was supported by an additional patient portal, which acted as an outward facing tool, to interact with patients. The patient portal was hosted in a different environment to allow for access to the Web. It enables patients to complete questionnaires from home using a secure connection.
Monthly increase in patient numbers for patient-reported outcome measurement since implementation.
Increase in patient numbers for patient-reported outcome measurement over time.
Period | New patients seen, n | Patients who participated in patient-reported outcome measurement, n (%) |
November 2016 | 105 | 13 (12.3) |
December 2016 | 90 | 7 (7.8) |
January 2017 | 116 | 40 (34.4) |
February 2017 | 124 | 10 (8.1) |
March 2017 | 166 | 4 (2.4) |
April 2017 | 104 | 20 (19.2) |
May 2017 | 139 | 25 (17.9) |
June 2017 | 112 | 9 (7.6) |
July 2017 | 116 | 55 (47.4) |
August 2017 | 108 | 54 (50.0) |
September 2017 | 163 | 67 (41.1) |
October 2017 | 93 | 59 (63.4) |
November 2017 | 169 | 115 (68.0) |
December 2017 | 117 | 86 (73.5) |
Level of academic training in all patients included in patient-reported outcome measurement (N=541).
Education level | Value, n (%) |
Minimum German schooling requirement | 201 (37.1) |
Mid-level education | 153 (28.2) |
Higher level education | 187 (34.5) |
Patients who agreed to follow up versus who declined to follow up patient reported outcome measurement.
Return rates of the digital questionnaires completed by patients who participated in the long-term patient-reported outcome measurement.
Preference for paper versus digital questionnaire according to age groups.
This study documents the successful implementation of measuring PROs using the ICHOM dataset for breast cancer in a German university hospital for the first time. During the implementation period, we made numerous observations. First, it takes at least 6 months to implement and establish a working system, get all key stakeholders to adopt it, while simultaneously solve those technical problems that arise during the implementation phase. Second, most patients, across all age groups, are willing to participate in the initial measurement as well as in the long-term follow-up. Third, contrary assumption, most patients, even those aged >60 years, prefer a digital survey over a paper-based way to answer the PRO questionnaires.
Our observation regarding the required timeframe to establish a successful electronic PRO system is matched by previous publications [
In addition to the finding that it is possible to establish a successful PRO measurement program at a German breast center integrated into the routine clinical workflow, this work showed for the first time that the majority of patients with breast cancer treated in a German university hospital preferred a digital survey. Patients did not want to fill out questionnaires in a paper and pencil-based version—despite contrary believes. In addition to this interesting finding, we were able to show that we can simultaneously achieve a high adherence rate even in the long-term follow-up. Similar observations have been made previously in the United States, for example, at Memorial Sloan Kettering Cancer Center [
Since this is a retrospective analysis of an implementation trial, it is not without limitations. In the beginning, reasons for decline in the number of patients participating was not systematically collected. Since this systematic approach was started only at a later point, we currently do not have enough data on this matter but are collecting them now. Also, we did not have enough resources to be able to contact those patients who decided not to continue in the follow-up. This point and the first point are important aspects, and we are currently addressing both and plan to publish the results in a following publication.
While there is an increasing interest in the potential of PRO measurements in almost all disease entities [
The goal of this pilot trial was to create a template on how to establish a successful Web-based PRO measurement system at a German university hospital, setting the stage for what to expect and showing that it is possible to measure PRO in a digital manner in patients with breast cancer of all age groups.
International Consortium for Health Outcome Measurement
patient-reported outcomes
None declared.