This was a nationwide, retrospective observational cohort study conducted with secondary use of the Japan claims database over a 19-year period between 2003 and 2022. The study used longitudinal patient data from the MDV (MDV Co, Ltd) claims database, which is built over 450 Japanese hospital administrative data (covers approximately 26% of “Diagnosis Procedure Combination hospitals,” ie, the hospitals that perform higher than the lowest in the university hospital group), containing over 38 million patient data since the year 2003. The MDV database is a national, patient-level, anonymized, and longitudinal claims database including hospital claims, discharge summaries, hospitalization data, and outpatient and prescription data. The ICD-10 (International Statistical Classification of Diseases, Tenth Revision) was used for disease diagnosis, and the medical procedures were coded using Japanese Procedure Codes. Hospitalized death and other clinical evaluations at discharge were available from the “discharge summary” data.

The study period was divided into a look-back period, treatment (time-at-risk) period, and follow-up period. The look-back period for covariates was the period from the first record of MM in the MDV database till the index date. Time-at-risk was the period from the start of the treatment to the end of the treatment for MM, while follow-up time was the period from the end of the treatment for MM to the last record registered in the MDV database. The index date was the record at the start of the treatment line in patients with transplant-ineligible NDMM or RRMM between 2003 and June 2021.

Male and female patients aged 18 years and older at index date with a confirmed diagnosis of MM (transplant-ineligible NDMM or RRMM), between 2003 and 2022, were included. Patients were required to have confirmed administration of any of the treatment regimens for first line for the transplant-ineligible NDMM or second line and third line for the RRMM. The unit of analysis (sample) in this study was an individual treatment of first line used for the patients who were transplant-ineligible NDMM or RRMM, second line, and third line. Patients whose DoT cannot be calculated were excluded from the ML analysis.

This study was conducted on 4848 samples for MM using 647 variables extracted from the MDV database. Variables were selected based on the collected opinions of local clinical experts and publications, and the results of a feasibility analysis. Final variables were determined after consulting local clinical experts on the results of the feasibility analysis and were broadly classified as patient, MM treatment-related, and MM treatment-related comorbidity management variables (Table 1). The primary end point was to identify the factors that correlate with the short DoT at the threshold of 3, 6, and 12 months. The DoT was defined as the time from the initiation of a line of treatment to the last day of the line of treatment.

Data analysis included 5 steps as discussed in Figure 1.

ML models were constructed using 647 variables extracted from the MDV database to identify important predictors associated with DoT. The deep learning used in this study was applied to the exploratory research of biomarkers that are associated with the response of the immune checkpoint inhibitors [19,20]. The binary variables that represented DoT within 3, 6, or 12 months were categorized as target variables, while variables used to develop ML models and understand the characteristics of clusters listed in Table 1 were categorized as explanatory variables.

All the feature variables were classified into continuous variables, binary variables, and nominal categorical variables. Continuous variables are normalized to have a mean of zero and an SD of 1 to eliminate bias due to differences in scale between variables. Binary variables were encoded to −1 or 1, and one-hot encoding (1 and −1) was used for nominal categorical variables [21]. The variables with missing values were imputed with 0 because the 0-value input would not change the output in the neural network models [22]. The target variables were set as a binary class and presented in 3 threshold patterns (0 for DoT <3 months, <6 months, or <12 months; and 1 for DoT ≥3 months, ≥6 months, or ≥12 months).

Unlike the randomized controlled trials used in clinical trials, the patients enrolled in the MDV database used in this study are a heterogeneous patient population (Figure 2A). In other words, the impacts of each variable on DoT differ across patients. We used point-wise linear (PWL) models to predict DoT. The PWL model was used as the explainable deep learning model, which generated a custom-made logistic regression (LR) model for each sample by a meta-learning approach (Figure 2B) [23]. Especially, although the LR model has a weight vector w for the original variable vector x⁽ⁿ⁾ ([n] is the sample index), the PWL model has a point-wise weight vector w⁽ⁿ⁾ for the original variable vector x⁽ⁿ⁾ (Figure 2B). On the other hand, extreme gradient boosting (XGBoost) [24,25] is a decision-trees–based machine learning model capable of addressing patients’ heterogeneity. Although XGBoost achieves as high a performance as deep learning models do, in general, XGBoost cannot calculate the weight, that is, the impact on the DoT of each variable. In the finalization of the DoT prediction model, the area under the receiver operating characteristic curve (AUROC), recall, precision, and F₁-score were calculated. The prediction performance of the PWL model was compared with the XGBoost (gradient-boosted decision trees) [24,25] and the LR model with a combination of L1 (Lasso) and L2 (Ridge) regularization terms (commonly known as elastic-net). The LR model is similar to PWL in that it explains feature importance by referencing the magnitude of weights (model parameters). The key difference between LR and PWL is that LR assigns a single weight coefficient to each feature across all samples, whereas PWL can assign different weights to the same feature for different samples. Consequently, PWL retains the capability to perform individualized factor analysis for each sample. The prediction performances of these models were calculated by the area under the curve (AUC) evaluated by 10-fold double cross-validation [26]. A “10-fold cross-validation” procedure was used, including 10 repeated attempts after classifying the data into 10 quadrants, with 1 quadrant for evaluation and the remaining 9 for learning. To avoid overlearning in hyperparameter tuning, the learning data were further divided into 10 pieces, 1 of which was used for tuning evaluation. The generalizability of the DoT prediction model was evaluated using the test set of the outer loop of 10-fold double cross-validation, and the datasets were reconstructed or hyperparameters were fine-tuned in case any significant difference (>0.1) was confirmed between the mean AUC in the test set and the training set.

A clustering analysis of 4848 individual samples was performed based on the similarity of the coefficients of custom-made LR models calculated from the PWL model to understand the number of patterns of the relationship between each sample and DoT (3, 6, and 12 months). All the samples were classified into subgroups using the regression coefficient vector tailored to each sample in the PWL model. Cluster analysis was conducted using the k-means method [27,28], and the number of clusters (k) was determined by the Elbow method [29]. The characteristics of each cluster were analyzed by using all the variables listed in Table 1. The P values for continuous variables were assessed by t test and 1-way ANOVA, while the P values for categorical variables were assessed by chi-square test. To understand the characteristics within a particular cluster, we categorized all 4848 samples into 2 groups—those predicted not to continue treatment after the respective DoT threshold and those predicted to continue treatment after the respective DoT threshold. Continuous variables and categorical variables significantly different between the groups were assessed by 1-way ANOVA and chi-square test, respectively. The P values were adjusted by the Bonferroni correction method.

All data used in this study were deidentified and anonymized patient records obtained from the MDV database for secondary research purposes. In accordance with the Ethical Guidelines for Epidemiological Research issued by the Japanese Ministry of Health, Labor, and Welfare, the study did not require informed consent or approval from an Institutional Review Board [30].

This analysis was conducted in 2762 patients (4848 individual samples) with MM. Of these, 1450 (52.5%) were male; 2079 (75.3%) patients were included during the index year 2016-2021. At the first MM diagnosis, the mean age, Charlson Comorbidity Index (CCI) score, and ADL score were 69.6 (SD 11.8) years, 0.5 (SD 1.1), and 14.6 (SD 7.2), respectively (Table 2).

The AUROC score of the PWL model to predict DoT at 3, 6, and 12 months was 0.61, 0.64, and 0.66, respectively. The corresponding values for the XGBoost model were 0.62, 0.64, and 0.66, respectively. The PWL model with the target variable DoT at 12 months performed the best; however, no model reached an AUC>0.7 (Figure 3). Moreover, the slightly smaller SD in PWL suggested more consistent performance across different folds of cross-validation, making it the model of choice for this analysis. The recall, precision, and F₁-score of all models are presented in the supplementary material (Multimedia Appendix 1). The precision and F₁-score were decreased when the DoT length increased, and the thresholds shortened.

All 4848 samples were categorized before and after each DoT threshold (3, 6, and 12 months) to determine the factors responsible for shortened DoT. As cluster B at DoT of 3 months and cluster C at DoT of 6 and 12 months were predicted to continue the treatment, patients from these clusters were further divided into those predicted to continue treatment after the DoT threshold, and those predicted to discontinue treatment within the DoT threshold, and the incidence of events during and after treatment was compared. There were a lesser number of treatments included in DoT <3 months than DoT ≥3 months (n=1859, 38.3% vs n=2989, 61.7%), while DoT <6 (n=2753, 56.8%) and <12 months (n=3672, 75.7%) had a higher number of treatments compared with DoT ≥6 (n=2095, 43.2%) and ≥12 months (n=1176, 24.3%), respectively. In cluster B, a significantly (P<.01) higher percentage of patients used immunomodulatory drugs (IMiDs) and used or maintained aspirin during MM therapy in DoT ≥3 than in DoT <3 months group. However, the rate of subsequent transplantation in the next treatment was significantly (P<.01) higher in DoT <3 than DoT ≥3 months group. Similarly, in cluster C, a significantly (P<.01) higher percentage of patients used IMiDs or aspirin during MM therapy in the DoT ≥6 months than in the DoT <6 months group. In cluster C, a significantly (P<.01) higher percentage of patients used IMiDs during MM therapy in DoT ≥12 months than in the DoT <12 months group (Table 3). A summary of the key clinical traits for each cluster after the DoT threshold analysis is presented in Table 4.

This retrospective real-world observational study using longitudinal patients’ data from the nationwide MDV claims database analyzed the predictive factors that shorten DoT in Japanese patients with MM over a 19-year period (2003-2022). Additionally, this analysis used ML models to determine the factors responsible for shorter DoT using information, such as type of treatment and patient characteristics from the MDV claims database, categorizing all the data in different clusters at 3, 6, and 12 months. Finally, a clustering analysis was conducted to investigate the characteristics of the sample in each cluster and the relationship between predicted DoT and the comorbidity management before and after the treatment thresholds (3, 6, and 12 months).

Patients in cluster B than cluster A (at 3 months), and patients in cluster C than cluster A and B (at 6 and 12 months) were supposed to be in poor condition as the number of treatment lines and pretreatment CCI were high; patients had more comorbidities, and elevated laboratory parameters. Cluster B (3-month DoT) and cluster C (6- and 12-month DoT) consistently captured subgroups with high comorbidity burden and elevated pretreatment CCI, yet demonstrated longer treatment durations. Additionally, the use of IMiDs for MM treatment was significantly higher in patients who met predicted DoT at each threshold versus those who did not. In contrast to the clinical findings, the data from this study indicate that the choice of drug, irrespective of whether it is used for induction or maintenance therapy, affects the expected DoT.

From our recent real-world study in Japan, it was found that the DoT decreased continuously with increasing the line of treatment in patients with MM, especially after 2016 when several newer therapy options were in force [12]. In clinical practice, treatment may be discontinued for various reasons; however, for the desired prognosis, it is important to continue the medications for an appropriate duration. The DoT has a strong association with the clinical outcomes and survival in patients with MM [31,32]; therefore, it was inevitable to determine the factors affecting the shortened DoT. Although it was evident that the availability of new drugs might increase the treatment options, making it easier to change treatment regimens, the huge pattern of treatment makes it difficult to identify the cause of shortened DoT. Therefore, this analysis was conducted using the national MDV claims database to determine the predictive factors that could help clinicians in the decision-making process while considering the management of MM. Additionally, this study found the characteristics of patients (age, CCI, comorbidity, treatment lines, laboratory parameters, and care requirement) who had shorter DoT as well as the characteristics of patients with frequent treatment change, which reflects the actual picture of clinical practice. This information is critical to continually improving MM management, achieving optimal patient care, and understanding the burden of disease and health services available to Japanese patients with MM in the real world.

In line with the published evidence [33,34], our study compared the prediction performance of the PWL model with the XGBoost and the LR models. The AUROC values of constructed models were all between 0.6 and 0.7 (Figure 2). Values between 0.6 and 0.7 indicate satisfactory performance [35-37]. The ranges observed in this study are consistent with those reported in previous real-world data–based modeling studies [33,38,39]. Administrative claims data often face inherent data limitations, such as the absence of genomic or staging variables, which can constrain predictive performance. In real-world settings, such models can still provide clinically relevant insights for subgroup identification and hypothesis generation. Nonetheless, models built on such data can still yield clinically relevant insights—particularly for subgroup identification and hypothesis generation in real-world settings. In our study, we generated the models using the variables required in the claims database. For improvements in prediction performance, the use of MM-specific laboratory parameters might be required. Furthermore, in our study, the PWL model showed greater AUROC than the linear regression model at the DoT thresholds of 3, 6, and 12 months. However, these scores were the differences within the error bars between PWL and XGBoost models at each DoT threshold.

DoT predictive models in our study were specifically designed to forecast the DoT in patients with MM, rather than to evaluate clinical effectiveness between specific treatment groups. Due to this focus, these models were not intended for direct application in clinical decision-making or disease diagnosis. Consequently, we did not perform propensity score matching or sensitivity analysis, which are typically used in studies evaluating treatment efficacy. Instead, our efforts were concentrated on carefully developing and comparing the PWL model with the XGBoost and LR models to thoroughly assess the factors influencing DoT. The PWL model differs from LR by generating a separate set of regression coefficients for each individual sample, rather than using a single global set. This approach allows identification of factors that are most relevant for each patient, which may enhance interpretability and personalization in real-world datasets where treatment decisions and outcomes are highly individualized. Unlike randomized controlled trials, the MDV claim database includes a diverse and unselected patient population, without adjustments for specific demographics or conditions. This makes the data more representative of real-world variability. Multiple patterns can thus be anticipated based on the combination of variables that affect DoT. The LR model cannot recognize multiple patterns because the model has only a single regression coefficient to a variable for the whole sample. On the other hand, since the PWL model has a custom-made regression coefficient, it is possible to perform individual analysis for each sample. Therefore, we performed the cluster analysis using the regression coefficient vector tailored to each sample in the PWL model to understand the number of patterns. The PWL model enabled case-level insights, highlighting which patient features most influenced predicted treatment duration. Such interpretability is especially valuable in clinical settings, where understanding the rationale behind risk stratification supports shared decision-making. In our study, the k-means method was used for cluster analysis to classify all the samples into subgroups, and the Elbow method was used to determine the number of clusters. The k-means method is an unsupervised ML approach widely used in research for clustering samples based on similarities [28,40]. Additionally, the k-means method could substantiate heterogeneous longitudinal data into distinct, homogeneous clusters [27,28]. The Elbow method is an established method to determine the appropriate number of clusters that rides the clustering algorithm many times with different values of k (number of clusters) [41], and defines the optimal number of clusters to efficiently categorize the data [42,43].

From the baseline demographics and predictors of cluster assignment, it was found that patients in cluster B at DoT of 3 months and cluster C at DoT of 6 and 12 months, compared with other clusters, had advanced disease and continued the treatment beyond the threshold duration. Interestingly, we observed that in these clusters, more patients had CCI ≥4, ≥1 line of treatment, and comorbidities (pneumonia, depression, cancer, GERD, and other bowel dysfunction) than in other clusters at each DoT threshold. Additionally, the laboratory parameters such as estimated glomerular filtration rate (MDV specific), leukocytes, neutrophils, and lactate dehydrogenase were significantly higher in the cluster C than the cluster A and B at DoT 12 months. An all-inclusive assessment of comorbidities and overall status (with the help of laboratory parameters) may help to determine a patient’s suitability for appropriate therapy as well as the adverse effects that may occur. Altogether, the results from this study support the understanding that the patient’s condition and management-related factors may be associated with shortened DoT, consistent with clinical findings [12,44,45]. Evidence suggests that severe disease, older age, and comorbidities are the prognostic factors for poor survival in patients with MM [46,47]. Although it was not in the scope of this analysis to comment on the causal relationship between predictive factors and patient survival, and their effect on longitudinal outcomes and DoT, our findings validate a depiction of shortened DoT in those with reduced disease burden. This is consistent with the findings of the retrospective study conducted by Ailawadhi S et al [32], which reported that each additional month of therapy was associated with a reduced probability of disease progression and death at 2 years. Similar results were also observed in a pooled analysis of 3 phase III trials that evaluated the advantage of novel agent-based continuous therapy versus fixed DoT in patients with newly diagnosed myeloma [48].

To understand the characteristics of the clusters, we analyzed the relationship between the explanatory variables and the flag that indicated the samples belonged to each cluster. Additionally, we calculated the association between the variables during and the flag to investigate the tendency to the event during and after treatment. The significance was detected by the test for noncorrelation at a significance level false discovery rate of <0.05 using the Benjamini-Hochberg method. The Benjamini-Hochberg method is a validated tool used in multiple correlation analyses that could decrease the false discovery rate [19].

Significantly, more patients used IMiDs and aspirin beyond the treatment thresholds (both ≥3 [cluster B] and ≥6 months [cluster C]) compared with before treatment thresholds (<3 and <6 months), and in cluster C at DoT ≥12 months, a higher proportion of patients used IMiDs during MM therapy compared with DoT <12 months group. The use of IMiDs was significantly higher when comparing sample populations predicted as DoT of <3, <6, and <12 months with those who did not. However, this observational association may reflect confounding factors, such as patient eligibility, tolerability, or physician preference, and does not imply causation. In patients with MM, lenalidomide, one of the commonly used IMiDs, is sometimes used in maintenance therapy, and the results from this study suggest that IMiD therapies are associated with treatment continuation for MM in patients with high CCI, severe disease, and comorbidities. Furthermore, these results imply that the use of appropriate IMiD may impact DoT in patients with MM. Since the treatment choices for patients with MM depend on several factors, including the type of previous therapy and comorbidity, these data may assist the clinical decision-making on whether IMiDs should be used in these patients. Similar to IMiD use, the management of thromboprophylaxis was also associated with treatment continuation for MM in patients with high CCI.

Our analysis identified several key factors that might be associated with shortened DoT for patients with MM. The first factor is the patient’s disease status—patients with poorer health status (CCI ≥4, ≥1 previous line of treatment, and comorbidities including pneumonia, depression, cancer, GERD, or bowel dysfunction), along with elevated laboratory parameters, showed longer DoT. Second was the management-related factors, including use of IMiDs and management of thromboprophylaxis: patients who were treated with IMiDs for MM showed extended DoT.

The strengths of the study include (1) this study is a precedent of data-driven research to understand the characteristics of patients with MM whose DoT was shortened; (2) by using AI, based on ML techniques, we were able to build a prediction model using many factors that are difficult to build with a conventional statistical model; (3) in this study, subgrouping was automatically done by the ML-based DoT predictive models as subgrouping analysis is required when analyzing real-world data due to heterogeneity of the patient population; (4) this study used the latest data from nationwide MDV claims database which was based on Japanese general population, and represented real-world clinical practice.

This study also has certain limitations. First, the database is restricted to the Japanese population; hence, caution should be taken while generalizing the results. This database only included the data of the insured population, and it was impossible to follow patients outside of the hospital or those who switched hospitals. Not all laboratory tests were conducted in the entire patient population, which again restricts the generalizability of the findings. Furthermore, since the cause of death was unavailable, it could not be determined whether the shorter DoT was due to treatment failure, toxicity, patient preference, or unrelated mortality. Additionally, death was captured only for hospitalized patients, and important prognostic factors for MM, such as cytogenetic profile, R-International Staging System stage, and presence of all plasmacytomas, were not collected. This may affect the clinical granularity and predictive robustness of the models. Their inclusion may further enhance both robustness and interpretability of future predictive tools. We partially addressed this by incorporating surrogate measures, such as CCI and ADL scores. Furthermore, it is difficult to evaluate the causal relationship between predictive factors and DoT, even if the relationship between the factors and shortening of DoT is assumed, and the presence of unknown confounding that affects the continuation of treatment cannot be ruled out.

This study is the first to apply ML to assess predictive factors for shortened DoT in Japanese patients with MM. Using the PWL model, an explainable deep learning method, we identified key patient and management factors influencing DoT. These insights can contribute to improving patient management in hospitals, optimizing care, and enhancing understanding of the burden of disease and health services for Japanese patients with MM in real-world settings. Further statistical analysis is needed to refine patient segmentation and ensure the effective application of these insights in clinical environments.