ListeningTime, a Web-based preparatory communication tool for elderly patients with cancer, was developed to help patients (better) prepare their encounters with oncological HCPs. In addition, the tool was designed to support HCPs in preparing their encounters with elderly patients. An overarching aim of the project was to develop ListeningTime in a participatory way to increase its uptake and use. The participatory development process of ListeningTime, including the content and techniques used, was extensively described in a previous publication [14]. In short, ListeningTime is a website, containing 2 video diaries of simulated patient-HCP encounters in which different communication strategies are demonstrated. Patients are asked to watch a selection of personally relevant video fragments, based on an algorithm. HCPs are asked to watch one entire diary. Furthermore, the website contains an audio-facility. Patients and HCPs can relisten their audiorecorded encounter through the facility. For this study, ListeningTime was evaluated in real-life clinical practice among both patients and providers.

A cross-sectional design was used to evaluate the usability, perceived usefulness, and actual use of ListeningTime, according to and by elderly cancer survivors and patients and their oncological HCPs. First, members of a patient panel were approached to evaluate ListeningTime through a Web-based questionnaire. Second, ListeningTime was evaluated in real-life clinical practice through a pilot study in Dutch hospitals, using a Web-based questionnaire and examining user statistics and log files.

This study was conducted according to the Dutch privacy legislation. According to the Dutch legislation, approval by a medical ethics committee was not required. Participation was voluntary, and participants gave their informed consent at the start of their participation.

Elderly patients with cancer were approached through the Dutch patient panel “kanker.nl” (translated as “cancer.nl”); this patient panel consists of 169 cancer survivor or patients aged ≥65 years, of which 88 were invited to evaluate ListeningTime. In March 2016, they were invited to fill in a Web-based questionnaire to evaluate ListeningTime. They were asked to navigate through the website while answering the questions. Oncological HCPs (ie, oncologists and oncology nurses) from 3 Dutch hospitals were invited to partake in the pilot study to evaluate ListeningTime.

At the start, HCPs were asked to visit the website ListeningTime, create a personal log-in account, sign the digital informed consent, fill in a baseline questionnaire, and watch one of the 2 video diaries of simulated patient-HCP encounters containing 12 short video fragments (Textbox 1). After HCPs had watched the video diary (or diaries), they were asked to include patients for the pilot study.

From April to December 2016, HCPs approached eligible patients during their medical visits, and handed out a leaflet to patients asking to visit the website ListeningTime before their next visit. Patients were eligible if they were aged ≥ 65 years, diagnosed with cancer, had internet access, spoke and read Dutch, and were not in the palliative or terminal phase of the disease. Interested patients who visited the website were informed about the study, instructed on the website to create a personal log-in and sign a digital informed consent form and fill in a baseline questionnaire to get access to the selection of 6 personally relevant video fragments. The selection of 6 personal relevant video fragments and the order of the fragments varied per patient, based on the algorithm. The algorithm was based on the level of patients’ confidence in communication with the oncological HCP (through the Perceived Efficacy in Patient-Physician Interactions Questionnaire [20]), the importance of discussing several subjects (eg, quality of life, intimate issues as based on patients’ needs assessment [3]) and their sex (male or female).

Fragments 1 and 2 were always offered as first 2 fragments (Textbox 1). The stories of patients in the 2 diaries differed (ie, one diary tells the story of a female patient with lymphoma, the other diary that of a male patient with prostate cancer) and also their participation level during the stimulated encounters differed (ie, one diary represents a more “active or assertive” patient, the other diary a more “passive” patient). Figure 1 shows a screenshot of the video fragments.

On the informed consent form, patients could opt for audiorecording of their next encounter with their oncological HCP. In case of consent, their HCP audiorecorded this next visit and uploaded the recording on ListeningTime; this enabled patients, their spouses, and HCPs to relisten their audiorecorded encounter, using their personal log-in.

Within 1 week after the (audiorecorded) visit to their HCP, patients were asked to evaluate ListeningTime through a Web-based questionnaire. At the end of the study, HCPs were also asked to evaluate ListeningTime through a Web-based questionnaire.

The Web-based questionnaire of the patient panel was used to assess the usability and perceived usefulness of ListeningTime; this questionnaire inquired about patients’ sociodemographic characteristics, their first impression of the website, textual parts of the website, log-in procedure, video fragments, audio-facility, and other remarks. The Web-based questionnaire in the pilot study was used to evaluate the perceived usefulness of ListeningTime, according to patients and oncological HCPs in hospital-based care. Furthermore, this questionnaire assessed the textual parts of the website, log-in procedure, video fragments, audio-facility, and other remarks.

The usability of ListeningTime was measured with the System Usability Scale (SUS) [21]. The SUS includes 10 items about several facets of usability, for example, the complexity of the website and the ease of using it, scored on a 5-point Likert scale, ranging from 0 (strongly disagree) to 4 (strongly agree). SUS scores were calculated following the guidelines from the original publication [21]. As individual items of the SUS are not meaningful on their own, a total SUS score will be calculated. SUS scores range from 0 to 100; higher scores indicate higher usability. A previous study, evaluating nearly 10 years of SUS data collected, indicated that the SUS is a highly robust and versatile tool and also provides details on what constitutes an acceptable SUS score [22].

The perceived usefulness of ListeningTime, that is, “the degree to which a person believes that using a particular system would enhance his or her job performance” [23], was measured using questions and statements. Similar questions and statements were used in previous studies [24,25]. Multimedia Appendix 1 describes the questions that were asked through the patient panel (18 questions), and questions and statements included in the pilot study (14 questions or statements for patients; 13 questions or statements for HCPs).

The actual use of ListeningTime by patients and HCPs in the pilot study was examined using user statistics and log files, that is, automatically generated files mapping the interactions between program and users; this allowed us to get insight into what extent patients and HCPs actually used the website, including the log-in frequency, playing video fragments, and using the audio-facility.

We considered the rating of ListeningTime “high” in case ≥70% of cancer survivors or patients perceived ListeningTime as useful (in both the patient panel and pilot study), the usability was rated as “good” or higher [22], and 70% of the included patients actually used ListeningTime (ie, logged on, watched the video fragments) in the pilot study [26]. As the use of the audio-facility was optional, we considered the rating of the audio-facility “high” in case all patients who made use of it found it useful.

The ultimate goal was to implement ListeningTime, in case of high rating, as a publicly available, standalone intervention, that is, without the research context and without support of professionals. Therefore, we collaborated from the start of the project with several partners. This participatory development method was pursued to create awareness of the potential of ListeningTime and to prepare for a successful implementation (see for more details about the participatory development process of ListeningTime [14]). These partners included representatives from hospitals, the Nederlandse Federatie van Kankerpatiënten organisaties, the “Quality institute for oncological and palliative research and practice” (IKNL: Integraal Kankercentrum Nederland) and “kanker.nl” During the project, implementation of ListeningTime by one or several of these partners was discussed.

Descriptive statistics were used to analyze the results. Data analyses were performed in Stata version 14.

ListeningTime is a useful and user-friendly communication tool for elderly patients with cancer. It helps patients to (better) prepare the clinical encounter with their oncological HCP and overcome communication barriers. Patients most valued the video fragments and the audio-facility to relisten their recorded consultations. They mentioned that ListeningTime supported their informational needs (eg, know what you can ask), emotional needs (eg, how to deal with emotions and ask for support), and their cognitive needs (eg, better remember what the doctor has told).

Patients often feel emotionally overwhelmed after diagnosis or during cancer treatment and have a need for emotional support. In addition, most patients with cancer report difficulties in understanding and fully processing the HCPs’ information [27,28]. ListeningTime seems to offer an opportunity to fulfill these needs.

Previous research found that combining audiovisual information with conversational style is the best way to present eHealth information about cancer treatment to (younger and older) adults [29]; this can explain patients’ high rating of ListeningTime as we used a combination of audiovisual information with conversational style in the video fragments. However, for more experienced patients, the video fragments seem less useful. Future research is necessary to get insight into which moment is or are (most) appropriate to use ListeningTime (eg, at the start of a disease trajectory).

Likewise, oncological HCPs were positive about ListeningTime as a supportive tool for patients. They valued the video fragments and the possibility to relisten the audiorecorded consultation. However, they also mentioned that the video fragments were too simple for patients. It is possible that HCPs overestimate their patients’ communication skills or that they included mainly experienced patients during the pilot study. Analyzing real-life, video- or audiorecordings of patient- provider encounters in this setting can provide insights into the communication process and role of both patients and providers. As only 8 patients audiorecorded their encounter during this study, it is not possible to draw conclusions. As mentioned before, HCPs were positive about the possibility to relisten the audiorecordings. However, they did not relisten their own recorded encounters; this could be attributed to several reasons, for example, owing to the lack of time or not feeling the need to relisten. For this study, the main aim was to support elderly patients with cancer in overcoming their communication barriers; ListeningTime seems to offer this opportunity. Although oncological HCPs participated in this study to support patients in their communication skills and, therefore, used ListeningTime, we did not offer a specific communication training for HCPs. The high use of ListeningTime by patients, however, can also be attributed to the involvement of HCPs in including patients and asking them to visit the website. Over recent years, many eHealth interventions have been developed. However, numerus eHealth interventions have not been evaluated, have reported attrition (like dropout and nonusage) and adoption problems (ie, poor uptake after implementation) [26,30,31]. By actively involving elderly patients with cancer and their providers in developing ListeningTime, the use and uptake of this intervention was expected to increase [32,33]. The evaluation of ListeningTime, indeed, showed that patients valued ListeningTime and, as a result, the video fragments became publicly available for all elderly patients with cancer. A previous study found similar results and concluded that actively involving patients with cancer in designing and evaluating a Web-based tool is feasible and appreciated [34]. For the design of the website, guidelines for targeting elderly patients online were followed, that is, avoiding large amounts of text by using “pull out” menus for more detailed information and larger font size [16]; this could have supported the use of ListeningTime.

Although ListeningTime was developed to support elderly patients with cancer especially, the tool might be very useful for younger patients as well. A recent study found no differences in website satisfaction between younger and older patients with cancer using a mode-tailored website [35]. Nevertheless, it should be tested if ListeningTime is also useful for younger patients with cancer. To this extent, it would be interesting to know how many (elderly and younger) patients (and their significant others) use the educational video fragments of ListeningTime since the implementation on the website “kanker.nl.”

In this study, we evaluated a Dutch Web-based communication tool. However, the results might be useful and relevant at the international level as well. As our results indicate, a tool as ListeningTime can be highly valuable to offer to elderly patients with cancer. It consists of multiple useful techniques, that is, a tailoring algorithm, modeling videos (including simulation questions), and an audio-facility [14], which can be useful for other countries and settings as well.

To the best of our knowledge, this is one of the first studies using a participatory process to develop a Web-based intervention, that is, with the help of elderly patients with cancer and their providers [30,34]. In addition, the educational video fragments of ListeningTime were implemented through the website “www.kanker.nl.” This success can be attributed to the participatory nature of the development process and the inclusion of partners from the start of the project. Unfortunately, it was not technically possible to include the tailoring algorithm and the audio-facility of ListeningTime on the website of “kanker.nl.” For further implementation of ListeningTime, the involvement of HCPs (or hospitals) might be necessary. Unfortunately, this was beyond the scope of this project.

For future research, it might be interesting to investigate the (combined) effect of the video fragments and audiorecordings on real-life communication between patients and HCPs; examine the effect of the simulation question at the end of each video fragment (eg, how do patients use or reflect on these questions, is it a crucial part of the video fragment, what is a good simulation question); explore other ways to provide patients with educational videos and audiorecordings of their clinical encounters; and how to implement interventions like ListeningTime in close collaboration with HCPs. A necessary first step before developing eHealth interventions is to investigate if the targeted patient population feels the need for the proposed eHealth intervention.

This study has some limitations. First, the results may be influenced by the relatively small study sample. However, this is an exploratory pilot study. Larger, controlled studies are necessary to replicate (or contradict) our findings. Second, it is possible that only interested patients participated. However, this is also the targeted group that will watch the video fragments on “kanker.nl.” Third, we aimed to include a wide range of elderly patients with cancer, with different (stages of) disease and different levels of participation (eg, active and passive). As patients volunteered to partake, it is possible that the results of this study represent the more “active” patients—those who feel confident in participating during medical encounters. In addition, we are not aware of the number of patients approached by providers in the hospital and the number nonresponders.

ListeningTime was highly rated, both by elderly patients with cancer and their oncological HCPs. As a result, the video fragments of ListeningTime are publicly available for all (elderly) patients with cancer through the Dutch website “kanker.nl,” without the research context and the involvement of professionals.