This study was embedded within a pilot study and used a convergent parallel mixed methods approach [15], combining (1) qualitative exploration of patients’ experiences of app use and (2) quantitative analysis of user behavior. The underlying phenomenon of interest was patient experience with both components of the SOFIA app, conceptualized as the interplay between patients’ subjective experiences and their observable usage behavior. The mixed methods design was chosen to capture both dimensions and to better understand the mechanisms shaping engagement.

The study was a prospective single-center, 2-arm randomized controlled external pilot trial. The study protocol and details of the study methods are described elsewhere [11,16].

To estimate the key parameters for the main trial, a total of 70 participants (35 per group) were required, based on the simulation-based approach for determining sample size in external pilot trials with continuous outcome variables [17].

Participants in the IG received access to the SOFIA app (for a detailed description, [11]). SOFIA was integrated into the Mika app, a digital therapeutic providing individualized supportive care and aiming to reduce psychological distress in people with cancer. The app entails different components, including symptom monitoring and coaching modules (for a detailed description, [11]).

Monitoring: SOFIA Monitoring comprised 11 physical and up to 9 mental ePROs, assessed twice weekly with email reminders. Physical symptoms were selected by experienced medical oncologists based on European Society for Medical Oncology guidelines [18] and included weakness, diarrhea, melena, dry cough, dyspnea, reduced urinary output, joint and muscle pain, skin toxicity, fever, and jaundice.

Coaching: The coaching component included modules such as Discover and Journeys. The module “Discover” is the content library of the Mika app, consisting of articles and videos with easily understandable and evidence-based information on several cancer-related topics (eg, cancer and treatment types, nutrition in cancer; Table S1 in Multimedia Appendix 1), written and peer-reviewed by medical experts. An integrated statistical content recommendation engine delivers individual content recommendations to the user based on age, gender, cancer type, therapy type, and reported symptoms. By considering the symptoms the patient has reported in the symptom monitoring module, as well as patterns in their reading behavior, the algorithm predicts the likelihood of the user engaging with specific content and, based on the estimated read probability, presents the respective content to the user.

“Journeys” are the resource-activating psycho-oncological training courses of the Mika app that aim to help patients cope with cancer-associated distress in different areas (eg, living with immunotherapy; Table S1 in Multimedia Appendix 1). There are 13 journeys, each of which entails 3 subsequent stages, which in turn include 3 exercises. Exercises can be allocated to 1 of 4 categories: knowledge, mindfulness, relaxation, or physical exercise.

The Ethics Committee of Heidelberg University Hospital (reference S-581/2018) approved the trial protocol, and the study was registered in the German Clinical Trial Register (reference DRKS00021064). All participants were informed about the study procedures and signed informed consent before being included in the study.

Of the 34 patients in the IG, 2 patients left the study directly after randomization. For the remaining 32 participants, user data were available. Regarding the availability of interview data, 29 patients participated in the midway assessment (T1), while 23 patients participated in the postintervention assessment (T2). One patient discontinued participation due to dissatisfaction with the application, 2 patients died during the study period, 1 patient was physically unable to participate in the interview, and 2 patients could not be reached by phone.

Patient characteristics are presented in Table 1. The study flow chart can be found in Multimedia Appendix 3. For a detailed explanation of the study flow [11].

In our IG sample, 6 patients received steroids due to adverse events after the start of immunotherapy. Five patients had brain metastases, resulting in 1 patient undergoing surgery, 3 receiving radiotherapy, and 1 death. No patients had meningeal metastases. Five patients had thyroid dysfunction: 3 alone and 2 combined with other side effects.

For both SOFIA Monitoring and SOFIA Coaching, we identified two main categories: (1) perceived benefits and (2) criticism or recommendations. In the following, we describe the corresponding generic categories and subcategories for each and present illustrative quotations from the interview notes. The quotations were translated into English by CS. The numbering is based on the order in which the statements (eg, #1) and their documentation appear in the Excel spreadsheet (eg, a=first data spreadsheet), as well as the measurement time (T1 vs T2). Figures 1 and 2 summarize the results of the qualitative analysis by presenting the coding structure with main categories and subcategories.

Patients reported that they had the feeling that somebody was interested in their symptoms and that they were not alone with their fears. The patients felt reassured knowing what the typical symptoms were, which gave them a sense of security.

The patients appreciated the clarity of the presentation, the scaling of symptom intensity, and the relevant questions tailored to immunotherapy. They found the application easy and quick to use. See Table 2 for subcategories and examples.

Some patients reported that the ePRO helped them remember symptoms for the physician-patient consultation and to report them during the consultation (subcategory: reminder regarding symptom course). Patients also used it to prepare for consultations with their physicians (subcategory: preparation of questions for the medical consultation). They also saw it as a time saver, as it meant that physicians were already able to obtain a detailed overview, facilitating communication and the identification of relevant symptoms (subcategory: time savings). Examples are depicted in Table 3.

At T1, 62% (18/29) of patients reported that physicians did not use their input during the consultation; 17% (5/29) indicated that physicians used it partially, and 7% (2/29) reported that their input was used; 14% (4/29) were unable to judge that. At T2 (n=21 answers were available for this question), 14 (67%) patients reported no use of their input, 3 (14%) reported partial use, and 4 (19%) reported use. Patients expressed a desire for more feedback and a better integration of the ePRO assessment into their treatment.

Based on the log data of the 32 active users of the coaching features of the SOFIA app, we found that 16 (50%) of them had read at least 1 article from the “Discover” module. The median number of articles read per user in this group was 7.5. The corresponding mean value of 14.8 (SD 19.9) was distorted by 2 outliers (52 and 72 articles). As seen from the more detailed analysis in Table 6, more articles were read in the intervention period of the study than in the follow-up period.

The app organizes articles in different categories. The most frequently read category was “cancer therapy” (k=47), closely followed by “COVID and cancer” (k=38), “relieving symptoms” (k=37), “healthy lifestyle” (k=33), and “nutrition and cancer” (k=28). All data are depicted in figures in Multimedia Appendix 4 (pie chart, bar chart, and box plot).

The log data of the “Journeys” module of the SOFIA app revealed that 87.5% (28/32) users started at least 1 journey; 25% (8/32) of them completed at least 1 journey. Table 7 shows that most of the engagement in journeys took place in the intervention period.

Table 8 shows the popularity of journeys among users. By far, the most popular journey was “Living with immuno-therapy.” Figures in Multimedia Appendix 5 show the titles and frequencies of journeys completed.

In the SOFIA app, journeys include exercises. Fifteen out of 32 (46.9%) users were engaged in at least 1 such exercise. In this group, an average of 9.1 (SD 8.8) exercises were started (median 9, IQR 6.5). Figures in Multimedia Appendix 6 show the most frequently started exercises. Multimedia Appendix 7 includes a joint display table linking the main qualitative themes of SOFIA Coaching to their respective quantitative usage patterns.

DHIs are an integral part of oncological care. The evidence for this is very clear. Many efficacy studies do not provide accurate user information. The results of our convergent parallel mixed methods approach showed that, qualitatively, patients reported feelings of empowerment and support, appreciated the user-friendly design, and reported perceived improvement of patient-physician communication (ePRO assessment) as well as access to helpful and validated information (coaching component). Simultaneously, they wished for more flexibility in the ePRO assessments and more integration of the ePRO assessment into the clinical routine, as well as more specific data. Regarding the coaching component, some participants wished for more updates or did not prefer digital use. Quantitative user data indicated that over 50% (16/32) of patients read articles, engaged with journeys, and performed exercises, with immunotherapy-relevant articles, the management of side effects, and living with immunotherapy being of particular interest.

The results of our main study revealed high acceptability and feasibility of the ePRO assessment in patients undergoing ICI therapy, with an adherence rate of 91% [11]. We conducted qualitative interviews to gain deeper insights into user perspectives and needs. The findings of our accompanying qualitative study corroborate those of a previous investigation involving 186 patients with various cancer entities undergoing diverse cancer therapies, as well as 55 clinicians [22]. That study examined patient and clinician perspectives on the use of a DHI for collecting ePROs. Three overarching themes were identified for both stakeholder groups: (1) acceptability and functionality (relevance of symptom items, usability, reminders, perceived health status); (2) impact on clinical care (information and resource provision, contribution to research, self-monitoring, reassurance, connection to the hospital, and support for guided decision-making); (3) personal value of using eRAPID—the tool used for ePRO assessment (clinician engagement with eRAPID, facilitation of consultations, and support for medication or treatment adjustments) [22]. Furthermore, the results are in line with the recently published systematic review of qualitative data [12] identifying “improved self-management,” “safe and accessible support,” and “better access to health information” as key benefits, and “inadequate content design,” “technical difficulties,” and “negative user perceptions and behaviors” as key barriers. Consistent with our generic categories—user-friendly application (acceptability and functionality), facilitation of patient-physician communication (impact on clinical care), and empowerment, support, or availability of helpful and interesting information (personal value)—these findings provide several implications for the development of future DHIs including ePRO assessments in oncology. First, ePRO systems should prioritize high acceptability and usability by ensuring intuitive design, relevant symptom items, and reliable reminder functions (eg, directly via the app) [2]. Second, features that translate PROs into clinically actionable information—such as automated alerts or structured summaries—are essential to support timely decision-making. Third, future tools should actively facilitate patient-physician communication through clear symptom visualizations and seamless integration into electronic health records. In this context, ePROs could be particularly useful for gathering information about how patients tolerate therapies such as tyrosine kinase inhibitors. These therapies are often used in combination with ICIs and are frequently associated with chronic side effects that are difficult to assess during routine consultations, but which can have a significant impact on HRQoL [23].

Moreover, ePRO systems should promote patient empowerment by enabling self-monitoring, offering feedback on symptom trajectories, and providing access to supportive resources. Successful implementation further requires strong clinician engagement and incorporation into routine workflows. The results of our previously published study [11] revealed that 65% (n=28) of physicians (who treated patients in the IG) never or seldom used the graph for monitoring patients’ care—even though over 50% of physicians evaluated ePROs as useful and saw significant benefits in managing treatment with ICI (eg, in preparing for consultations or detecting side effects). These reports are in line with the results of the answers of our patients, as over 60% of patients reported no usage of their ePRO assessments in the clinical routine, but patients wished that their reports would be better integrated into the patient-physician consultation. It is important to note that these implications are derived from qualitative perceptions and descriptive usage patterns and should not be interpreted as evidence of causal mechanisms at an individual level.

In future studies, the integration of ePRO data into clinical workflows should be systematically evaluated, including the role of physician training. In line with studies showing that the assessment of ePROs and screenings should have consequences and aftercare [24], this is important for patients’ adherence to reporting ePROs regularly. It would then be particularly interesting to gain a deeper understanding of patients’ and physicians’ perspectives when ePROs are more integrated into clinical encounters.

Finally, offering flexible assessments while simultaneously enabling standardized data capture for research and quality improvement will be crucial for maximizing the clinical utility of next-generation ePRO solutions.

Furthermore, the factor of personal value (empowerment, support, availability of helpful information) might have led to improved HRQoL and reduced psychological burden in patients in the IG compared with the control group in our sample [11]. However, this interpretation should be viewed with caution, as our small sample size did not allow us to demonstrate the mediating effects of these factors.

Patients reported benefiting from the availability of relevant and helpful information, as well as from the user-friendly and clear design. This is consistent with previous studies showing that access to helpful information and safe, accessible support is a key factor in eHealth interventions for patients with cancer [12,22]. These findings underline the importance of providing reliable and accessible information. In particular, patients benefited from the journeys and topics addressing side effects and preparation for physician consultations. These findings are further supported by our user-data analyses, which show that patients actively engaged with the app’s content. During the observation period, 81% (26/32) started a Journey module, and 50% (16/32) accessed written articles. In the follow-up period, engagement declined, but 22% (7/32) read articles, and 12% (4/32) started journeys. The frequent use of Journey modules reflects the Journey format’s strong appeal. Content analyses indicated that users were particularly interested in topics related to immunotherapy, current developments, and the management of treatment-related side effects, which was underlined by our qualitative data (ie, availability of helpful information; dealing with issues). These patterns are consistent with recent studies demonstrating that patients with cancer seek timely, therapy-specific information and guidance [25]. Future studies should take this into account when developing digital contents for patients with cancer.

However, 50% (16/32) of users did not engage with the SOFIA app’s articles. Criticisms identified in the qualitative evaluation of the SOFIA Coaching program may explain the limited use (eg, low preference for digital formats, lack of updates, or articles perceived as too general or insufficiently relevant). Future studies should provide cancer-entity-specific information, ensure regular content updates, and consider that DHIs alone may not reach all patients. To avoid disadvantaging individuals who are less digitally inclined, alternative, nondigital information formats, which are often provided by cancer-specific nongovernmental funding and nonprofit-making organizations, should continue to be offered in parallel.

Our study provides important insights into the assessment of ePROs and the use of DHIs in patients with cancer and represents an important addition to our feasibility study. In particular, the mixed methods approach, combining qualitative and quantitative data, represents a major strength of this study. However, several limitations have to be discussed. We only assessed the patients’ perspective in our qualitative approach. Although we quantitatively assessed physicians’ views on the benefits, usability, and use of the ePRO assessment in a sample of 33 physicians involved in treating patients in the IG [11], a complementary qualitative study would likely have provided deeper insights into perceived benefits and barriers. Due to anonymization processes, it was not possible to match the user data with other PROs and the qualitative data. Furthermore, we were unable to provide a detailed analysis of user behavior, such as engagement over time, weekly trajectories, dropout curves, and patterns of sustained vs early disengagement. Nevertheless, the data provide important insights in the usage of DHIs.

The sample for the qualitative analysis was constrained by the number of participants in the intervention, as determined by the sample size calculation for the primary outcomes of the main trial. Therefore, data saturation was not explicitly addressed in the qualitative part. However, recurring themes were identified across interviews, and our sample is larger than in comparable previous studies [12]. The concept of saturation has also been critiqued, as it assumes that no additional insights can emerge beyond a certain point [26]. Further, since the interviewers had prior contact with the participants during the recruitment phase of the study, a potential influence of social desirability has to be taken into account. However, we consider this influence to be minor, as the first contact was brief and participants also raised critical points during the phone interviews.

This mixed methods study revealed the benefits and criticisms of ePRO assessments and digital coaching tools for patients with cancer under ICI-therapy. Results indicated that a user-friendly app, perceived facilitation of patient-physician communication, and several dimensions of personal value (eg, empowerment, support, or availability of helpful and interesting information) (personal values) may represent pivotal aspects for patients with cancer using DHIs. Quantitative data underline the importance of therapy-specific contents. In sum, we derived several indications for future DHIs in patients with cancer, which may contribute to the improvement of acceptance and usage and thereby further support the clinical benefit.