Published on in Vol 7, No 1 (2021): Jan-Mar

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/26911, first published .
Development and Early Feasibility of Chatbots for Educating Patients With Lung Cancer and Their Caregivers in Japan: Mixed Methods Study

Development and Early Feasibility of Chatbots for Educating Patients With Lung Cancer and Their Caregivers in Japan: Mixed Methods Study

Development and Early Feasibility of Chatbots for Educating Patients With Lung Cancer and Their Caregivers in Japan: Mixed Methods Study

Original Paper

1Department of Respiratory Medicine, Hyogo Prefectural Amagasaki General Medical Center, Amagasaki, Japan

2Department of Internal Medicine, Kyoto Min-Iren Chuo Hospital, Kyoto, Japan

3School of Social Information Science, University of Hyogo, Kobe, Japan

Corresponding Author:

Yuki Kataoka, MD, MPH, DrPH

Department of Respiratory Medicine

Hyogo Prefectural Amagasaki General Medical Center

Higashi-Naniwa-Cho 2-17-77

Amagasaki, 660-8550

Japan

Phone: 81 6480 7000

Email: youkiti@gmail.com


Background: Chatbots are artificial intelligence–driven programs that interact with people. The applications of this technology include the collection and delivery of information, generation of and responding to inquiries, collection of end user feedback, and the delivery of personalized health and medical information to patients through cellphone- and web-based platforms. However, no chatbots have been developed for patients with lung cancer and their caregivers.

Objective: This study aimed to develop and evaluate the early feasibility of a chatbot designed to improve the knowledge of symptom management among patients with lung cancer in Japan and their caregivers.

Methods: We conducted a sequential mixed methods study that included a web-based anonymized questionnaire survey administered to physicians and paramedics from June to July 2019 (phase 1). Two physicians conducted a content analysis of the questionnaire to curate frequently asked questions (FAQs; phase 2). Based on these FAQs, we developed and integrated a chatbot into a social network service (phase 3). The physicians and paramedics involved in phase I then tested this chatbot (α test; phase 4). Thereafter, patients with lung cancer and their caregivers tested this chatbot (β test; phase 5).

Results: We obtained 246 questions from 15 health care providers in phase 1. We curated 91 FAQs and their corresponding responses in phase 2. In total, 11 patients and 1 caregiver participated in the β test in phase 5. The participants were asked 60 questions, 8 (13%) of which did not match the appropriate categories. After the β test, 7 (64%) participants responded to the postexperimental questionnaire. The mean satisfaction score was 2.7 (SD 0.5) points out of 5.

Conclusions: Medical staff providing care to patients with lung cancer can use the categories specified in this chatbot to educate patients on how they can manage their symptoms. Further studies are required to improve chatbots in terms of interaction with patients.

JMIR Cancer 2021;7(1):e26911

doi:10.2196/26911

Keywords



Distress among patients with cancer and their caregivers has received increasing attention from medical staff. Early palliative care (EPC) from the time of diagnosis has been shown to have a positive impact on quality of life [1]. Through EPC, patients and their caregivers gain experience in several areas, including prompt and personalized symptom management [2].

Nonetheless, patient education remains a problem for health care providers [3]. Advancements in health care increase the amount of information that needs to be provided during patient education. Consequently, burnout among health care providers involved in EPC has become a problem [4].

Chatbots are artificial intelligence (AI)–driven programs that interact with people [5] through text messages and outputs on cellphone- or web-based platforms [6]. A simple rendition of this system involves a user entering text and AI predicting the corresponding predefined category and then sending the response corresponding to that category to the user. Considering the potential of this system to substitute the conversational mode of education that humans use and to save on labor, this technology has attracted increasing attention. Several chatbots are currently available for patients with breast cancer [7,8]; however, no chatbots are available for those with lung cancer and their caregivers. In this study, we aimed to develop and evaluate the early feasibility of a chatbot designed to answer questions of patients with lung cancer and their caregivers on a web-based platform whenever they experience unfamiliar symptoms; this would help them improve their knowledge of symptom management.


Study Design

To develop a chatbot for patients with lung cancer, we used a sequential mixed methods approach (Table 1) [9]. We adopted this approach because of the lack of sufficient nationwide data on the categories of frequently asked questions (FAQs) asked by patients with lung cancer and their caregivers. We also decided to adopt an initial qualitative approach to generate hypotheses. We adopted the lens of health care providers to provide palliative care. This study was conducted at a tertiary care hospital in Japan.

Table 1. Study overview.
Phase #ProceduresProductEvaluation method
1Conducted a questionnaire-based survey with physicians and paramedicsCategories of FAQsaQualitative
2Formulated responsesFAQ-response pairsN/Ab
3Developed the chatbotChatbot version 1N/A
4Conducted an α test with the physicians and paramedics involved in phase 1Chatbot version 2Quantitative
5Conducted a β test with patients with lung cancer and their caregiversChatbot version 3Quantitative

aFAQs: frequently asked questions.

bN/A: not applicable.

Phase 1: Qualitative Survey

We conducted a web-based anonymized qualitative survey from June to July 2019. We included physicians in the Department of Respiratory Medicine at our institution, along with paramedics including nurses, pharmacists, physical therapists, occupational therapists, and clerks working with patients with lung cancer in the clinic, emergency department, or ward. We asked these health care workers to respond to the FAQs obtained from patients with lung cancer or their caregivers in an open-ended manner. We also referred to a previous study [10] and website [11] that categorized data obtained from telephone-based consultations with patients with lung cancer in Japan or their caregivers.

Two board-certified respiratory physicians (TT and YK) conducted a content analysis on the questions from the perspective of improving self-management of symptoms. TT generated these categories, and YK confirmed them. Disagreements were resolved through discussion between TT and YK.

Phase 2: Generation of Responses

Based on the categories generated in phase 1, TT generated the appropriate responses, and YK proofread them. Wherever necessary, staff members from the relevant departments were asked to review the responses. We then edited the responses on the basis of their comments.

Phase 3: Development of the Chatbot

We decided to use Bot Designer (LINE Corp)—a bot designing service—owing to its ease of access, and the LINE social media platform to implement the chatbot. In Japan, LINE is the most popular social network service, being used by approximately 70% of the general population [12]. To generate automated responses, we used Google Cloud’s Dialogflow [13]. The architecture of this design is illustrated in Figure 1.

We adopted two natural language processing systems to match the responses. The following workflow process was followed: after receiving a question in Japanese from a user, Dialogflow determined whether the keywords included in the question matched the predetermined keywords for each category. If a match was identified, an appropriate response developed in phase 2 was sent to the patients via LINE. Otherwise, the text was translated into English and parsed to identify an appropriate category; thereafter, a response was sent to the patients.

Figure 1. Chatbot architecture.
View this figure
Phase 4: α Test (Quantitative)

From May to June 2020, we invited the participants in phase 1 to an α test that involved the addition of keywords corresponding to the categories specified on the basis of the errors.

Phase 5: β Test (Quantitative)

From July to December 2020, we invited patients with lung cancer and their caregivers to the β test through paper flyers. Individual consent was obtained before registering the chatbot. We asked the participants to indicate their satisfaction levels with a 5-point Likert scale after the experiments (0=highly dissatisfied, 5=highly satisfied).

Statistical Analysis

Descriptive statistics were used to summarize the results. We performed an unpaired t test for univariate analysis. We used Stata (version 16.1, Stata Corp) for statistical analysis.

Ethical Consideration

The study protocol was approved by the institutional review board of the Hyogo Prefectural Amagasaki General Medical Center (number 1-31). We obtained individual consent from the participants through a web form.


Phase 1: Qualitative Survey

We obtained 246 questions from 15 health care providers. We identified 5 major categories (home care: n=50, 20%, changes in health condition: n=54, 22%, prognosis: n=20, 8%, lifestyle: n=82, 33%, and medications: n=39, 16%), and 75 supplementary categories (Multimedia Appendix 1). We identified 7 additional categories by screening resources (Multimedia Appendix 2).

Phase 2: Formulation of Responses

We formulated responses for 82 categories identified in phase 1. The responses contained brief information intended to teach patients and their caregivers to deal with symptoms and direct them to various weblinks to information resources that are maintained by the public budget.

Phase 3: Development of a Chatbot

Based on the FAQs identified in phase 2, we developed the first version of the chatbot and integrated it in the LINE social media network. Figure 2 shows a sample interaction shared with the chatbot.

Figure 2. A sample conversation with the chatbot.
View this figure

Phase 4: α Test (Quantitative)

A total of 14 medical staff participated in the α test. They asked 71 questions to the chatbot during this period. Of these, 11 (15%) questions did not match the appropriate categories. Therefore, we added 3 new categories. Some of the categories had corresponding responses because the questions included the names of products associated with drinking and smoking. Thus, we added keywords for the corresponding categories to match responses.

Phase 5: β Test (Quantitative)

From among 309 patients who received either oral or intravenous chemotherapy at our hospital, 11 patients and 1 caregiver participated in the β test. Of them, 9 patients were aged <60 years, 1 was aged between 60 and 70 years, and 2 were aged >70 years. All of them used computers or smartphones daily. The participants were asked 60 questions, of which 8 (13%) did not match with the appropriate categories. Furthermore, 2 (3%) questions had corresponding responses, and 6 (10%) questions did not. Additionally, 3 (5%) questions based on daily conversation did not have corresponding responses (eg, “I played golf today in the morning”).

After the test, 7 participants responded to the postexperimental questionnaire. The mean satisfaction score was 2.7 (SD 0.5) points out of 5. We used two free feedback comments: “those who have a connection with a patient group, the answers were not too much to hope for” and “there were many times when I did not get a proper answer.” We added 3 categories based on the unmatched questions. Finally, 91 categories and responses were curated [14].


Principal Findings

This is the first study to develop a chatbot and evaluate its early feasibility to improve the knowledge of symptom management among patients with lung cancer and their caregivers. Based on the experience of medical professionals, we developed an autoresponsive chatbot, which could respond to most of the FAQs. However, it was not adequately used, and user satisfaction was low.

Comparison with Previous Studies

The categories identified in this study could be used as a reference for patient education in future. Currently, only one chatbot is available to empower patients with cancer and their caregivers. This chatbot was developed only for patients with breast cancer [6,15,16]. Given that patients with breast cancer and those with lung cancer differ in their age of onset, chemotherapy regimens, and cancer-related symptoms [17], it is important to increase the number of categories to improve their knowledge of symptom management based on the FAQs identified in this study.

However, since the generation of the chatbot alone did not yield adequate questions from participants, the low response rate potentially indicates that our chatbot may not be acceptable in its current form. Further studies are required to activate the interaction between patients and chatbots. A previous systematic review reported that web-based interactions between physicians and patients or those among patients may improve the patients’ quality of life [18]. Thus, chatbots can be incorporated into web-based networks for increased interaction. Indeed, one of our patients involved in the β test (phase 5) preferred interactions with other patients rather than the chatbot. Therefore, further studies are required to generate a platform that initially forms a text messaging–based online support group involving medical professionals and patients and then gradually reduces the need for these professionals to respond to patient queries by involving chatbots instead.

Limitations

This study has two major limitations of note. First, 8 (13%) questions did not match suitably with responses in phase 5, and a patient complained of not receiving appropriate responses. Although the ability of the chatbot to match questions with categories was not a major problem, questions for nonexistent categories remained unmatched. Therefore, it is necessary to add educational categories and responses through further discussion. Furthermore, the integration of existing chatbots for daily conversations may improve the proportion of matches [19]. Second, this was a single-center study, and further studies are needed to evaluate the applicability of our findings in other hospitals.

Conclusions

Medical staff who provide care to patients with lung cancer may be able to educate them using a chatbot containing the categories of FAQs curated in this study. Nonetheless, further studies are required to improve this chatbot in terms of interaction, especially considering its low usage in this study.

Acknowledgments

The authors acknowledge Sho Kawahara and Satoru Usami for their assistance in constructing the system in this study. The authors would like to acknowledge Editage for English editing. This study was funded by the Yasuda Memorial Medical Foundation. The funder played no role in the design of the study; collection, analysis, and interpretation of the data; or in writing the manuscript.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Categories from the questionnaire.

XLSX File (Microsoft Excel File), 30 KB

Multimedia Appendix 2

Categories from the references.

XLSX File (Microsoft Excel File), 10 KB

References

  1. Hui D, Hannon B, Zimmermann C, Bruera E. Improving patient and caregiver outcomes in oncology: Team-based, timely, and targeted palliative care. CA Cancer J Clin 2018 Sep;68(5):356-376 [FREE Full text] [CrossRef] [Medline]
  2. Hannon B, Swami N, Rodin G, Pope A, Zimmermann C. Experiences of patients and caregivers with early palliative care: A qualitative study. Palliat Med 2017 Jan;31(1):72-81. [CrossRef] [Medline]
  3. Fisher R, Croxson C, Ashdown H, Hobbs F. GP views on strategies to cope with increasing workload: a qualitative interview study. Br J Gen Pract 2017 Feb;67(655):e148-e156 [FREE Full text] [CrossRef] [Medline]
  4. Parola V, Coelho A, Cardoso D, Sandgren A, Apóstolo J. Prevalence of burnout in health professionals working in palliative care: a systematic review. JBI Database System Rev Implement Rep 2017 Jul;15(7):1905-1933. [CrossRef] [Medline]
  5. Bibault J, Chaix B, Nectoux P, Pienkowsky A, Guillemasse A, Brouard B. Healthcare ex Machina: Are conversational agents ready for prime time in oncology? Clin Transl Radiat Oncol 2019 May;16:55-59 [FREE Full text] [CrossRef] [Medline]
  6. Milne-Ives M, de Cock C, Lim E, Shehadeh M, de Pennington N, Mole G, et al. The Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review. J Med Internet Res 2020 Oct 22;22(10):e20346 [FREE Full text] [CrossRef] [Medline]
  7. Chaix B, Bibault J, Pienkowski A, Delamon G, Guillemassé A, Nectoux P, et al. When Chatbots Meet Patients: One-Year Prospective Study of Conversations Between Patients With Breast Cancer and a Chatbot. JMIR Cancer 2019 May 02;5(1):e12856 [FREE Full text] [CrossRef] [Medline]
  8. Bibault J, Chaix B, Guillemassé A, Cousin S, Escande A, Perrin M, et al. A Chatbot Versus Physicians to Provide Information for Patients With Breast Cancer: Blind, Randomized Controlled Noninferiority Trial. J Med Internet Res 2019 Nov 27;21(11):e15787 [FREE Full text] [CrossRef] [Medline]
  9. Östlund U, Kidd L, Wengström Y, Rowa-Dewar N. Combining qualitative and quantitative research within mixed method research designs: a methodological review. Int J Nurs Stud 2011 Mar;48(3):369-383 [FREE Full text] [CrossRef] [Medline]
  10. Miyazawa N, Kakizoe T. Anxiety Among Cancer Patients is Changing in Accordance with Advances in Cancer Treatment and the Development of Social Cancer Management: Ten Years' Experience as a Consultant for Cancer Patients and Their Families and Opinions Based on an Analysis of the Present Status. JJLC 2019;59(3):238-243. [CrossRef]
  11. Japan Cancer Society.   URL: https:/​/www.​jcancer.jp/​consultion_and_support/​%E3%81%8C%E3%82%93% E7%9B%B8%E8%AB%87%E3%83%9B%E3%83%83%E3%83%88%E3%83%A9%E3%82%A4%E3%83%B3 [accessed 2020-11-27]
  12. Supplementary financial information for the third quarter of the fiscal year ending December 31, 2020. LINE.   URL: https://d.line-scdn.net/stf/linecorp/ja/ir/all/FY20Q3_earning_releases_JP.pdf [accessed 2020-11-30]
  13. Dialogflow. Google Cloud.   URL: https://cloud.google.com/dialogflow/docs [accessed 2020-11-29]
  14. Kataoka Y. Development and early feasibility of a chatbot for educating patients with lung cancer in Japan: a mixed-method study. OSF. 2020 Dec 29.   URL: https://osf.io/y72mv/ [accessed 2021-01-04]
  15. Chaix B, Bibault J, Pienkowski A, Delamon G, Guillemassé A, Nectoux P, et al. When Chatbots Meet Patients: One-Year Prospective Study of Conversations Between Patients With Breast Cancer and a Chatbot. JMIR Cancer 2019 May 02;5(1):e12856 [FREE Full text] [CrossRef] [Medline]
  16. Bibault J, Chaix B, Guillemassé A, Cousin S, Escande A, Perrin M, et al. A Chatbot Versus Physicians to Provide Information for Patients With Breast Cancer: Blind, Randomized Controlled Noninferiority Trial. J Med Internet Res 2019 Nov 27;21(11):e15787 [FREE Full text] [CrossRef] [Medline]
  17. Guest JF, Ruiz FJ, Greener MJ, Trotman IF. Palliative care treatment patterns and associated costs of healthcare resource use for specific advanced cancer patients in the UK. Eur J Cancer Care (Engl) 2006 Mar;15(1):65-73. [CrossRef] [Medline]
  18. McAlpine H, Joubert L, Martin-Sanchez F, Merolli M, Drummond KJ. A systematic review of types and efficacy of online interventions for cancer patients. Patient Educ Couns 2015 Mar;98(3):283-295. [CrossRef] [Medline]
  19. Singh S, Thakur H. Survey of Various AI Chatbots Based on Technology Used. 2020 Presented at: 2020 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO); 4-5 June 2020; Noida, India p. 1074-1079. [CrossRef]


AI: artificial intelligence
EPC: early palliative care
FAQ: frequently asked question


Edited by D Vollmer Dahlke; submitted 03.01.21; peer-reviewed by M Katapodi, P Zarogoulidis; comments to author 04.02.21; revised version received 15.02.21; accepted 23.02.21; published 10.03.21

Copyright

©Yuki Kataoka, Tomoyasu Takemura, Munehiko Sasajima, Naoki Katoh. Originally published in JMIR Cancer (http://cancer.jmir.org), 10.03.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Cancer, is properly cited. The complete bibliographic information, a link to the original publication on http://cancer.jmir.org/, as well as this copyright and license information must be included.