Adaptation and implementation of a multinational eHealth intervention for people with cancer: reflections from the field

Corresponding Author: Eileen Furlong, UCD School of Nursing, Midwifery and Health Systems, Health Sciences Centre, Belfield, Dublin 4, Ireland Tel: +353 1 716 6478 Andrew Darley, UCD School of Nursing, Midwifery and Health Systems, Health Sciences Centre, Belfield, Dublin 4, Ireland Alison Buick, UCD School of Nursing, Midwifery and Health Systems, Health Sciences Centre, Belfield, Dublin 4, Ireland Patricia Fox, UCD School of Nursing, Midwifery and Health Systems, Health Sciences Centre, Belfield, Dublin 4, Ireland Grigorios Kotronoulas, Department of Computing and Information Sciences, University of Strathclyde, Glasgow, United Kingdom Morven Miller, Department of Computing and Information Sciences, University of Strathclyde, Glasgow, United Kingdom Robert Smith, Docobo Ltd, Surrey, United Kingdom Christine Miaskowski, Department of Physiological Nursing, University of California, United States Elisabeth Patiraki, National and Kapodistrian University of Athens, Papadiamantopoulou, Goudi, Athens, Greece


Background
The expanding field of electronic health (eHealth) and the global integration of technology into healthcare systems [1][2][3][4] have become more apparent over the past two decades. This leap in technology capabilities has led to many promising eHealth advancements in the cancer setting. For instance, an increasing numbers of healthcare initiatives in cancer care have utilised patients' self-reports to facilitate remote symptom monitoring [5][6][7][8][9][10][11][12][13].
Despite the research conducted so far, there is a notable lack of empiric evidence describing the preparation and implementation of an eHealth intervention across multiple countries and clinical sites [14,15]. This omission may be in part due to the fact that eHealth remains a relatively new area of research, that is characterized by exploratory studies implementing novel technology in healthcare practice and assessing their feasibility in a single-country [16][17][18][19][20][21].
Conducting cross-cultural, multinational research requires collaboration and multiple considerations to ensure an intervention's validity, fidelity, and appropriateness within different cultural and clinical settings [22][23][24]. The current paper seeks to address this important gap in the knowledge within the cancer eHealth literature by describing the process that was used to adapt and implement an evidence-based, remote symptom monitoring system, ASyMS (Advanced Symptom Management System Remote Technology) prior to use in a multicentre European randomised controlled trial (RCT) within thirteen clinical centres across five countries (i.e. Norway, Austria, Greece, Ireland, United Kingdom). The title of the RCT is electronic Symptom Management using Advanced Symptom Management System (ASyMS) Remote Technology for patients with cancer, for which the acronym is eSMART [25].

The ASyMS intervention
This international project involves a large RCT that aims to evaluate the short and longterm impact of a mobile-based, remote monitoring intervention (ASyMS) on symptoms in patients receiving first-line chemotherapy for breast cancer, colorectal cancer, or haematological malignancies. The ASyMS intervention utilises mobile device technology to enable real-time, 24-hour monitoring and management of patients' self-reported chemotherapy-related toxicities (CRTs).
Patients use a dedicated mobile device to complete a symptom questionnaire (Chemotherapy Toxicity Self-Assessment Questionnaire (CTAQ)) which assesses ten chemotherapy (CTX) related symptoms (i.e., nausea, vomiting, diarrhoea, constipation, hand-foot syndrome, mucositis, paraesthesia, flu-like symptoms/infection, fatigue, pain and up to six additional symptoms) once a day and at any time they feel unwell. This information is analysed by an integrated clinical risk algorithm which results in the automated generation of evidence-based, self-care advice to patients on the mobile device based on their symptom reports.
For any reports that require clinical intervention, the algorithm generates 'real time' alerts to the clinical sites via a dedicated clinician handset. This clinician handset is a specialised mobile device used to receive alerts and a clinician who is the 'alert handler' carries this device at all times. Alerts can be either amber (i.e. related to moderate symptoms that should be addressed within 8 hours) or red alerts (i.e. severe symptoms that should be responded to within a 30-minute timeframe).

Figure 1. Patient Device and Clinician Handset
Once the alert is received, the alert handler views the patient's 'real-time' symptom reports on a secure stand-alone ASyMS wesbite, before contacting the patient to initiate the appropriate care intervention. Alert handlers can access information stored on the secure ASyMS website, including all patients' symptom reports, demographic and clinical information, contact telephone numbers and addresses to facilitate a clinical assessment with the patient. Clinical algorithms that are based on international, national, and local guidelines as well as feedback from clinicians and patients determine the appropriate, standardised interventions for the type of alert generated. The alert handler documents on the patients' clinical case notes the actions/interventions performed, and closes the alert on the ASyMS website.

Methods
In preparation for the use of ASyMS within an international, multi-centre randomised controlled trial the following steps were undertaken:  Review of the literature on the assessment and management of chemotherapy-related toxicity to ensure ASyMS reflected national guidelines and international best practice.
 Translate and linguistically validate the assessment questionnaires and study materials into the required languages for the participating sites.
 Preparation of the participating clinical sites for the use of ASyMS through assessment of infrastructure and human and material resource requirements.
 Testing and assessment of the technological readiness of the ASyMS system at the participating sites prior to commencing the randomised controlled trial.
Logged data were used to assess the technological readiness of ASyMS across multiple sites, prior to its use in the main RCT. These assessments included: clinician initial response times to alerts, clinician handling times of alerts, and technical issues reported in the support platform for the intervention. All data were extracted from the study's secure researcher database hosted by the study technological partner, Docobo. Figure 2 illustrates the Alert Generation and Handling System. Technological readiness was evaluated and confirmed using two Technological Feasibility Evaluation forms developed by the study investigators -one for clinicians (Appendix A) and one for the study's technological partner (Appendix B) who was responsible for maintaining the ASyMS system. On completion of the testing phase at each clinical site, a representative from the clinical site and technological partner were each required to complete their respective assessment.
The three key parameters, derived from the technological requirements set out in the Study Protocol were included in both assessments, namely: 1. System Set-Up: to assess whether clinicians and researchers involved in the study felt they had received sufficient training on the ASyMS system; were able to register participants to use the ASyMS system (using handset, tablet, and PC) and were confident to educate and register a new patient on a handset. Each research nurse, clinician, and research assistant involved in the study was provided with training on the nature and use of the ASyMS system. This training included education regarding how the ASyMS system works, patient registration, and alert handling. Once trained, they were setup and registered on the ASyMS system with the appropriate functions of patient registration and alert handling. Each clinical site was given the option of individual logins for their users or a generic login to use the system.

2.
Data transfer: to assess whether data were successfully transferred among: electronic clinical and demographic patient data; patient devices; tablets collecting PROM data and electronic clinical case note reviews and the study server. It was essential that all devices involved in the study (i.e., patient devices, clinician handsets, tablets) had the required mobile or Wi-Fi connectivity for the intervention to be safe and effective. Prior to the testing period, the study technology partner conducted technological connectivity assessments of all devices at each clinical site. This testing required that clinicians carried the device over the course of a week and rotated the device amongst staff members taking part in the study, to determine any technological issues and potential areas in the hospital where the phone did not work or connect.
3. Usability issues: to assess whether clinicians could access and log onto the ASyMS website to ensure timely and problem-free access, and their ability to log on and use the ASyMS clinician handset for the receipt of alerts as well as handling a patient generated alert using the ASyMS website. Clinicians were evaluated on their ability to use the clinician handset, log onto the ASyMS clinician website, handle patient alerts and complete medical reviews at the end of the patients' chemotherapy cycle. Clinicians were evaluated on their ability to use the ASyMS technical support system if there were any technological issues encountered, from both clinician and patient perspectives.

Scoping Review
To standardise the ASyMS intervention across Europe, a scoping review was undertaken to ensure that the intervention was evidence-based and consistent with international, national, and relevant local guidelines for assessing and managing the most common chemotherapy-related symptoms [26].
Following the completion of the scoping review, a consultation exercise was undertaken with clinicians (clinical advisory group) and patients (patient advisory group) at the participating clinical sites. This approach aligns with evidence that found the inclusion of clinician and patient consultation is more likely to lead to research that will translate into clinical practice [27,28]. Patient and clinician advisory groups informed the content of the symptom questionnaires, symptom protocols, clinical algorithms, and self-care advice to ensure consensus across the multiple European clinical sites. The review of literature combined with feedback from clinician and patient advisory groups provided valuable information which enabled agreement amongst study partners on the format and content of the intervention, as well as making it current, evidence-based, and culturally sensitive.
Additionally, to facilitate the development and refinement of the intervention, monthly videoconferences were held with all ASyMS study partners and investigators to provide an opportunity to update, assess progress, and identify any issues within partner countries. These teleconferences facilitated open discussions and actions around issues including ethics and governance, data protection, study instruments, technology development and language translation processes. Likewise, clinicians and researchers committed to monthly teleconferences at this early stage and throughout the feasibility trial to discuss practical and clinical issues.

Translation of study tools and related documentation
A substantial methodological challenge for cross-cultural research is the standardization of the research instruments, particularly the translation of instruments without losing the underlying context or cultural connotations of the wording [15,29,30]. This process is often time consuming, but a crucial investment in order to have confidence in the outcomes of the study [30].
In order for ASyMS to be adapted and implemented within the various European countries, it was essential for all relevant documents to be translated into the appropriate languages (German, Greek, Norwegian). The two key components of the process were: (a) translation and linguistic validation of questionnaires (where appropriate) into the required languages for the participating sites; (b) translation of all additional study components and supporting documentation into the required languages (e.g., patient information letters, consent forms).
We evaluated four companies, which fulfilled the following criteria for consideration:  Table 1   hour coverage as required.

Adherence and acceptability
Data regarding the usability of ASyMS were collected and consolidated by the study technology partner and analysed by the relevant study work package Leads (EF, PF, AD).
Across all sites, 86% of the patients who were eligible agreed to participate (see Figure 1 for flow of patients through the study). Reasons for declining to participate included being too busy, fears that study involvement would increase worry and stress levels due to thinking more about their diagnosis, and concerns about using technology. At an organisational level, it was notable that of the thirteen sites that completed the feasibility study, two reported the intervention was not feasible to integrate into their clinical practice, (i.e., one site in the United Kingdom and one in Ireland). The reason for their decision was due to the management and related alert handling responsibilities of the system outside the normal working day.
Of those who enrolled, adherence to the protocol was high with 96% of the patients completing the testing phase. Two patients withdrew from the feasibility study: one because of technical difficulties and the other because their chemotherapy treatment was discontinued.  Figure 3). Similarly, no differences were found in the adherences rates for completing the daily questionnaire by cancer type (breast cancer = 87.5%, colorectal cancer = 90.3% and haematological cancers =80.6% (p = .477) (See Figure 4). Table 2. Compliance of daily questionnaire by country.
Data are plotted as means ± 1 standard error  During the testing period, it became apparent that clinicians were receiving numerous alerts from patients about mucositis. These alerts occurred because, even with prompt and appropriate interventions, mucositis takes time to improve. Consequently, patients reported this symptom over multiple days which triggered an alert to the clinician based on our clinical algorithm. This alert continued to occur even though patients had triggered this alert and been contacted and given appropriate information and interventions. The subsequent telephone contact between clinicians and patients was reported as both anxiety-inducing for the patient (e.g., some were worried that they were triggering alerts despite performing the recommended self-care interventions) and time consuming for clinicians (who had already given the patient appropriate symptom advice). This algorithm was modified that while clinicians were alerted to a patient's initial report of mucositis, depending on the severity of the mucositis subsequent alerts would be 'silenced' for one or two days, allowing time for the intervention time to improve the mucositis. These modifications required technical changes in the ASyMS system and subsequent simultaneous ethical submissions at all clinical sites in order to implement the change.

Technical Issues
ASyMS has a dedicated online support platform for clinicians and researchers to report technical problems and have them solve these problems. This technical support platform allowed users to log, solve, and track issues that arose during the feasibility study. This approach facilitated rapid and tailored responses, as well as acting as a record of correspondence on the technological issues. During the testing phase, a total 112 issues were logged, the most common being difficulties using the clinical server (32.25%), which is the online platform for clinicians and researchers to enrol patients, handle alerts and monitor trial progress. In addition, 25% of the issues were related to the clinician handset and 18.8% were related to the patient device. These issues were rectified at each site, through investigation by the technology partner and additional training on using the system, before progression to the main trial.

Process to confirm and check readiness to participate in RCT
When each site completed the testing period, they were assessed for their readiness to move onto the RCT. This evaluation was done using the Technological Feasibility Evaluation Checklists (Appendix A and B). Across the thirteen sites, eleven sites successfully passed the technological feasibility evaluation to participate in the RCT. Of the two sites who did not proceed to the RCT, one was not assessed with the checklist as they did not achieve the required recruitment number of patients for the feasibility study (n=6). Their decision to withdraw from research was due to lack of staffing resources to undertake 24 hour clinician alert handling. Despite several attempts, it was decided that this clinical site did not have the capacity to recruit more participants and therefore could not implement ASyMS effectively. A second site did not pass the readiness screening for a variety of reasons including: ongoing connectivity issues to the mobile network, user issues with the registration of new patients to ASyMS system, and clinicians' inability to log onto the ASyMS system in a timely manner to handle alerts. Following discussions with the clinical site and the ASyMS research team, the decision was made not to include the site as part of the RCT.
Of the eleven sites that moved on to participate in the RCT, a number initially had discrepancies between reports by the technology company and reports by the clinical sites. Discrepancies included issues with Wi-Fi/mobile connectivity issues, local firewall regulations, clinicians' log-ins, patient enrolment, and completion of patient case note reviews. These issues were investigated and resolved by the researchers at University College Dublin (AB, AD). Each unique issue required individualised attention and partner involvement to resolve. For example, a local firewall regulation that obstructed the technology from using a local WiFi/mobile network connection required that clinicians seek help from their local IT department. At another site the technology company need to assist with the installation of software on the mobile devices. When issues arose with logging on to the system and patient enrolment and alert handling, clinicians were provided with follow-up training sessions to allow them to ask questions and ensure they had the knowledge and skills to use the system. Once each site had satisfactorily completed all of the required components and were verified by the technology company, the Chief Investigator signed off on the site as being ready to start the RCT.

Discussion
The aim of this study was to describe the processes undertaken in the preparation of an international multi-centre randomised controlled trial and to provide the results of the feasibility study. The successful adaptation of the ASyMS intervention is evident via implementation in thirteen cancer clinical practices across five European countries (i.e., Austria, Greece, Ireland, Norway, United Kingdom). Implementation was achieved through collaborative work with study partners and the implementation of an iterative process to resolve problems in each clinical site. Use of the Technological Feasibility Evaluation Checklists (Appendices A and B) provided valuable quality assurance across all clinical sites. The undertaking of cross-cultural and multi-centre research requires several considerations to address the complexities involved in capturing electronic data [32], and researchers in this study faced diverse and unique challenges. The time needed to ensure the European integration in preparation for the feasibility study was significant.
While the intervention was based on preliminary in the United Kingdom [11], the revision of the intervention to make it applicable across our European sites involved significant input. A systematic review [26] of the international evidence on the management of chemotherapy induced toxicity was done. The ASyMS patient survey, risk algorithm, and alert management design were refined based on this scoping review. Moreover, the content of the ASyMS intervention had to reflect not only current international standards identified in the scoping review; it had to be delivered in the appropriate language. All study documentation, including the electronic assessments, had to be translated from English into Greek, Norwegian, and German. This rigorous process involved multiple iterations and tests of linguistic validity, which was time consuming and costly, a finding which is supported by McIntosh et al. 2016 [24]. Nevertheless, effective and culturally sensitive translation was vital to ensure the intervention was appropriate for its intended users in each European country.
The testing phase was a crucial step in the transition from adaptation to implementation.
This highlighted additional areas where the technology needed to be adapted in order to meet the diverse needs of the users (i.e., clinicians, patients).  [24]. The establishment of relationships between teams facilitated the researchers' ability to identify and recruit patients, as well as establish rapport with clinicians who would assist with recruitment.
In addition, clinician became familiar with the study protocol and procedures. Based on previous research that showed clinicians' concern and apprehension about new eHealth technologies [33][34][35], the testing of ASyMS helped the research team identify clinicians' concerns and provide additional training sessions that afforded clinicians the opportunity to learn about the ASyMS protocol, express their concerns, and ask questions about the technology.
A number of changes were made to the patient questionnaire, the risk algorithm, and the alert management system. The clinical risk algorithm for ASyMS was amended to reflect current clinical guidelines and feedback from the clinician advisory group regarding the fever threshold for a red alert. In addition, the self-care library available on the ASyMS patient device was updated based on feedback from clinicians and patients [26].
The trial management group meetings, which were held monthly during the feasibility trial, identified that a 4 hour response requirement to manage was not feasible in busy oncology units and that the algorithm for mucositis warranted modification. These issues with the ASyMS algorithms were only apparent when it was implemented in multiple clinical sites. In addition, our feasibility testing allowed us to identify sites that were not appropriate for the RCT because of heavy workload and views that the intervention was too complex. These findings echo the importance of testing an intervention in different contexts, as well as the establishment of communication pathways that clinicians and researchers can use to gain first-hand experience about the intervention [36].
The findings from the testing phase showed that the intervention was applicable and acceptable to both patients and clinicians. Patients' with cancer were able to use the ASyMS intervention to complete the daily questionnaire and access self-care information.
Prior studies have explored mechanisms by which patient reporting of symptoms may confer clinical benefits and enhance symptom management [5]. The 64 patients in five European countries, from three different diagnostic groups, consented to use the eHealth device over one cycle of chemotherapy. Patient adherence was high and alert activation and handling results were similar to previous ASyMS studies conducted among patients with a variety of cancers [11,17,37,38]. Our testing phase should be interpreted in the context of two key limitations in that patients were not recruited prior to the initiation of chemotherapy. Therefore, some patients were chemotherapy naïve and others had received previous chemotherapy treatments which may have affected the results of the daily questionnaire data. However, despite these limitations, our work provided significant data around feasibility, changes needed for future use, and the perceived benefits of such a system in oncology units.

Conclusion
Patients with cancer undergoing chemotherapy require prompt identification of symptoms and interventions are needed to decrease the symptom burden and enhance quality of life. eHealth interventions can assist with the promotion of self-care skills to manage the side effects of chemotherapy and provide an immediate electronic connection with the clinicians. However, the development and deployment of such a system demands significant and substantial collaborative preparatory work across multinational settings. The issues and findings discussed in this paper outline the importance of effective collaborative project management, diligent use of checklists, clear division of responsibilities with each partner, country, and associated clinical sites, along with addressing cultural and language requisites so that the scientific integrity and reproducibility of the study is assured.

Clinical Site
Parameters of Effectiveness Feasibility Parameters (Part 1) Please complete this form and send it to glasgow-esmart@strath.ac.uk and andrew.darley@ucd.ie Have each of the following been completed satisfactorily? Y/N

Setup
Training of research nurse / assistant to use ASyMS

Registration of patients on ASyMS
Transfer of data from patient handset to study server (successful connectivity indicated by a green segment in the connectivity history bar)

Connectivity
Technological connectivity of ASyMS (mobile connectivity/Wi-Fi/other)clinician handsets Technological connectivity of ASyMS (mobile connectivity/Wi-Fi/other)tablets Technological connectivity of ASyMS (used wireless/wired network connectivity) -system

Clinician related
Patients registered on the server have become available on the PROM terminal Completion of electronic PROM data by patients and successful transfer to study server Completion of electronic Case Note Review data and successful transfer of to study server

Support system
Have been able to login to the eSMART support system If "no" was answered to any of the above questions, please provide details: Please sign and date form on page 2 All clinical sites that meet the afore-mentioned requirements will proceed to Part 2.

Clinician related
Ability of clinicians to log on and use ASyMS clinician handset for the receipt of alerts Ability of clinicians to access and log onto the ASyMS web-portal Ability of clinicians to deal with an alert using the ASyMS web portal Completion of electronic PROMs (pre-and post-CTx assessments) by patients and successful transfer to study server Completion of Case Note Review and successful transfer to study server

Support system
Ability to access/use the eSMART support system If "no" was answered to any of the above questions, please provide details on page 2: Please sign and date form on page 2 If "no" was answered to any of the above questions, please provide details: All clinical sites that meet the afore-mentioned requirements will proceed to Part 2.
PI name and signature: Date: University College Dublin Researcher name and signature: Date: CI name and signature: Date: