Our research team has undertaken a 3-year mixed method study to co-design, develop, and test LotusLab, a digital hub for information after childhood cancer. LotusLab is designed for caregivers of childhood cancer survivors and for young people (aged 12 years and older) at EOT. LotusLab has been co-designed with young people, caregivers, and HCPs in alignment with participatory design principles [31]. Figure 1 shows an overview of the co-design activities across the app development and testing process.

As a result of this iterative development process, the LotusLab prototype, available for β-testing, provided tailored information and videos, surveys, PROMs, notifications and reminders, and storage of personal survivorship documentation (manuscript under review). Figure 2 shows screenshots of the prototype.

Following planning, design, α-testing (ie, early-phase internal testing), and content development phases, the current study was conducted during the β-testing phase, which focused on testing performance and experience in “real-world” environments. The aim was to evaluate parents’ and HCPs’ perceptions of LotusLab and the use of a proxy-reported outcome measure embedded within the app. Usability, acceptability, usefulness, and feasibility (refer to Table 1 for descriptions) were evaluated through quantitative surveys and qualitative semistructured interviews. Descriptions of these 4 domains were informed by digital health evaluation and implementation science literature [32-34] to ensure that we captured technical outcomes related to the app itself, as well as implementation factors.

Parents or caregivers (hereafter referred to as parents) were eligible to participate if their child (aged 0‐18 y) had finished cancer treatment for any cancer diagnosis within the previous 14 months and was receiving medical cancer surveillance at The RCH’s Children’s Cancer Center (RCH CCC). Eligible clinicians included RCH CCC medical or specialist nursing staff who provided direct clinical care and coordination to families recruited into the study.

Parent participants were identified by RCH CCC clinical staff, who introduced the study and provided an information letter. A member of the research team contacted the parent within 2 weeks to ascertain their willingness to participate. Medical and specialist nursing staff who were directly involved in the care of the recruited families were approached via email to participate in the study. All participants provided written consent. Following parent and HCP consent, the study team used the hospital appointment booking system to identify and record the date of the patient’s next medical surveillance appointment.

Parents and HCPs were sent instructional emails detailing how to download and explore LotusLab, including how to enroll in the “Lotus Off Treatment” program. HCPs are integral to the successful implementation of digital health interventions. Therefore, as potential deliverers and trusted advocates, HCPs were asked to explore the parent (end user) interface and to evaluate the potential value and use of LotusLab to support their patients and families.

Once enrolled, participants had access to 15 topics, comprising videos and written information and resources tailored to parents and related to the EOT period. If available, the patient’s personalized treatment summary and surveillance roadmap were uploaded to the app. All participants were encouraged to explore the LotusLab layout and content. In addition, parents were asked to complete an in-app survey called “About You and Your Child” to provide basic demographic data and to self-rate their digital literacy. Within the survey, parents provided details about their child’s age, which triggered the in-app release of the corresponding parent proxy-reported Pediatric Quality of Life Inventory (PedsQL; version 4.0), a widely used, validated 23-item measure assessing child health-related quality of life across 4 dimensions: physical, emotional, social, and school functioning [35]. Participants received email and mobile notifications alerting them to the availability of the PedsQL for in-app completion.

Parents were asked to complete the PedsQL within 2 weeks prior to their child’s appointment. Although the PedsQL is validated for self-reporting by children aged 5 to 18 years [35], this early-phase testing was designed to capture parents’ experiences only and therefore included proxy reports for children aged 2 to 4, 5 to 7, 8 to 12, and 13 years and older. Further development and testing of the adolescent and young adult interface are ongoing.

PedsQL results were exported from the app by the study team, and subscale and total scores were manually calculated as per standardized scoring instructions. This procedure was undertaken due to app capability limitations at this phase of testing and the ongoing development of the clinician user interface. Preliminary testing focused on initial perceptions of parents and clinicians regarding the use of and discussion of the PROM in the real-world clinical setting. To support HCP understanding and interpretation of PedsQL domains and scores, and to aid consultation discussions, a custom PedsQL report was shared via email with the relevant HCP prior to the patient’s upcoming appointment (Multimedia Appendix 1). Parents and HCPs were encouraged to discuss the results of the PedsQL during the medical consultation. A retrospective note audit was undertaken to capture any documentation of the PedsQL results and discussion.

Following the consultation, participants were invited to complete usability and acceptability surveys administered via REDCap (REDCap Consortium) software [36] and to take part in an online semistructured interview to discuss their experience with LotusLab.

Brief demographic data related to patient and family factors (eg, diagnosis, age of the child, etc) were captured via an inbuilt survey titled “About You and Your Child.” Additional treatment information or relevant dates were accessed via the electronic medical record (EMR) and stored securely. Information related to HCPs (eg, discipline, role, primary institution, and years of oncology experience) was gathered during the consent process, either verbally or via email.

Parents’ digital literacy level was captured to enhance the accuracy of usability testing and to interpret differences between user abilities and issues with the LotusLab design and functionality. The validated 3-item Digital Health Care Literacy Scale was selected to assess participants’ comfort and confidence with mobile technology, specifically the ability to install and use apps and solve technical issues on their own [37].

Usability was measured and benchmarked with the System Usability Scale (SUS), a validated and widely used measure for assessing the usability of digital products [38,39]. The short 10-item questionnaire shows strong internal consistency and construct validity [40], and the benchmark mean usability score of 68 (SD 12.5) is considered appropriate for assessing the usability of digital health apps across different contexts [39]. Scores greater than 51 can be interpreted as “okay,” scores of 75 and above can be interpreted as “good,” and scores of 86 and above as “excellent” [41].

Acceptability was measured using a study-specific scale that was adapted from the 8-item abbreviated acceptability rating profile (AARP) [42]. Wording of the items was adapted to assess the app (eg, “I would be willing to recommend this app to other parents”), and items were scored on a 5-point Likert scale (strongly disagree, disagree, neutral, agree, and strongly agree; Multimedia Appendix 2).

Semistructured interviews were conducted with parents and HCPs to elicit in-depth perceptions of the app. Interviews were recorded, transcribed, and deidentified for analysis in NVivo (version 15), a software program for data analysis [43] (Multimedia Appendix 3).

Figure 3 presents digital literacy results. Six of the 8 parents (75%) “agreed” or “strongly agreed” that they could use or install an app on their device without asking for help and indicated that they could solve basic technical problems on their device. Two (25%) parents “strongly disagreed” that they could do this.

Parents responded with a high mean SUS usability score of 86.88 (SD 8.74; range 75‐100) compared with the benchmark usability score of 68 [39]. All parents “agreed” or “strongly agreed” that the app was “quick to learn” and that they felt confident using the app. Parents individually rated usability as either “excellent” (5/8) or “good” (3/8).

All parents “agreed” or “strongly agreed” that the app was fit for purpose and that a digital solution such as LotusLab was needed for young people who had recently completed treatment for cancer. Figure 4 shows parent ratings of LotusLab’s acceptability.

Semistructured individual interviews were conducted online (using Microsoft Teams) with both parents and HCPs. Interviews ranged from 19 to 41 minutes and were conducted between October 2023 and April 2024. Refer to Table 1 for a priori theme definitions and Table 3 for themes, subthemes, and exemplar quotes from parents and HCPs.

Parents felt that LotusLab was easy to navigate and appreciated the “clean” and “clear” look and feel. They valued the ability to choose topics according to personal interest and importance. Practical suggestions for improvement centered on the ability to view completed topics and PROM results once submitted, the ability to share content with others (eg, friends, medical team, and community health), as well as a search facility.

Overall, HCPs also found the app easy to navigate; however, they had more suggestions about the app layout, suggesting topic groupings such as “emotions” and a “more colorful and tiles layout” could help make it easier to navigate. HCPs echoed parents’ comments that the ability to share content and reaccess information through a search function and “Quick Links” sections would enhance the user experience.

Parents commented positively on the relevant and relatable content. Two parents stated they wished they had access to the app earlier. There were many suggestions for additional content (eg, development of a frequently asked questions section, information on how parents can support adolescents, and the need for additional videos from the perspectives of children and young people). Specifically, 2 parents reflected on the need for additional content for young people diagnosed with brain tumors, addressing the impacts of cognitive impairment on returning to school and maintaining friendship groups. Parents provided positive feedback about the video content and its ability to enhance accessibility for users with low general and health literacy levels.

HCPs provided positive feedback on the app content, particularly in relation to the videos and psychosocial and emotional needs information at the EOT. There was feedback that additional app content would be needed to enhance its relevance and suitability for patients who had undergone bone marrow transplant and those with brain tumors, due to specific medical and psychosocial needs. HCPs felt that while an app is an appropriate mechanism for young people to access information, LotusLab required enhancements to make it more engaging for young people, such as reducing the amount of text, reordering content to ensure videos always appear at the top, and making it look less like a “learning module.” HCPs suggested that information relating to the LotusLab sign-up process should also be available in languages other than English to ensure equity of access.

Parents felt the app provided additional value by collating trusted information in one place (taking out the “legwork”), providing education, alleviating concerns, and allowing people to digest information in their own time.

All parents reported that they found the technical process of completing the PedsQL within the app easy. In addition, most found the process meaningful as a mechanism for “checking in.” Two participants did not discuss the survey results with their oncologists, as they felt that these medical practitioners were not best placed to support their identified needs and that it would have been more beneficial to discuss the results with a different member of the health care team, for example, an occupational therapist. One parent found some of the specific survey questions challenging to answer. They felt some questions were not appropriate to their child’s current situation due to the long-term and evolving cognitive impacts of their diagnosis and treatment. This participant cautioned about the importance of delivering surveys like the PedsQL on a case-by-case basis to ensure relevance to a family’s current situation.

Some participants had difficulty accessing their treatment summaries and roadmaps uploaded to the “My Files” section of the app. Despite this challenge, all parents felt that access to these documents would be useful, especially when visiting other health professionals or when they were unable to access the existing EMR patient portal.

HCPs felt the value of LotusLab centered on its ability to address identified gaps by providing connections to information at the EOT, and they regarded the app as “a good bridge.” In addition, HCPs considered that the app could help normalize experiences. Multiple HCPs felt that LotusLab could potentially reduce the need for parents to consult Google and save HCPs time in answering common questions.

HCPs responded positively to the concept of using the app for PROMs, streamlining the data collection process, and using the results as a mechanism for meaningful discussion of broader quality of life issues and concerns. However, HCPs flagged potential concerns about their capacity to review results captured in the app prior to a consultation, and about the risk of identifying needs but feeling they were “not in the right position to solve a lot of those issues.” They acknowledged that identifying a need that is not addressed introduces risk and could potentially result in a negative experience for patients and caregivers.

HCPs noted that access to personalized survivorship documents, such as a treatment summary, could be empowering, especially for young people who do not always have ownership of these documents. One clinician cited specific examples from their career when access to a treatment summary would have been beneficial for both the treating clinician and the patient, for example, when a childhood cancer survivor presented as an adult to an emergency department but had “no idea” which cancer they had or what treatment they had received for their cancer.

The interviews explored in detail the potential barriers to engagement and sustainability of the app. Some parents felt they would be less likely to continue engaging with the app the further off treatment they were, or when feeling “ready to get back to normal” and “not wishing to revisit the past.” Two participants noted that there was a risk the app could feel like “homework,” which could reduce engagement among young people.

Parent-identified enablers to engagement and sustainability included promotion of the app by staff, such as specialist nurses, who know the families well and are best placed to discuss what to expect during the EOT period. Parents noted that embedding the app within existing discussions, such as “EOT reviews,” would be helpful and suggested the app could be advertised through posters in waiting areas.

There were mixed perspectives on the timing of the introduction to LotusLab. Although LotusLab was designed to provide EOT information, many parents felt that access to the app would be beneficial in the months leading up to completion of treatment. One parent felt that introduction to the app should be on a case-by-case basis, as the EOT period can be complex and a specific time point can be hard to determine, but most agreed on the value of knowing what to expect. It was also suggested by several participants that the app could have value during active treatment, lending itself well to “different starts.”

HCPs identified barriers to engagement and sustainability, including the risk that only a self-selecting cohort of people would download and access the app and that those with the greatest unmet information needs might require additional support to engage with the app. One HCP noted that many people turn off their notifications; therefore, ongoing engagement could be challenging. In relation to the collection of PROMs, multiple HCPs raised concerns about the capacity to review survey results prior to consultations due to time constraints, as well as safety concerns centered on ensuring that all results are reviewed and acted upon by an appropriate professional in a timely manner.

HCP-suggested enablers also centered on embedding the implementation of LotusLab within existing roles or services, for example, integrating it into a survivorship clinical nurse coordinator role and including it in “new patient packs.” One HCP suggested that the app could be shared through family networks, such as social media groups. One HCP noted that, in order to streamline LotusLab implementation, it could be helpful to undertake robust change management processes with staff. As per parent feedback, HCPs also suggested creating “a version of this at diagnosis” so that the app could become a longer-term companion across the cancer pathway.