Hemophagocytic lymphohistiocytosis (HLH) is a rare but often life-threatening syndrome represented by excessive inflammation resulting in organ dysfunction and potentially death [1]. Early recognition and treatment of HLH are critical to patient outcomes. Traditionally, HLH has been classified as primary, where there is a documented or presumed genetic etiology [2,3], or secondary, where excessive inflammation can be attributed to a trigger such as infection, malignancy, or a rheumatological condition in the absence of a genetic etiology [4]. Although there are an increasing number of genes identified responsible for primary HLH [5], there are patients with presumed primary HLH without a known mutation. Further, patients with both primary and secondary HLH may require HLH-directed therapy. Consequently, diagnostic criteria have been developed for HLH (Multimedia Appendix 1), which often guide decision-making about treatment initiation. Criteria were initially proposed in 2004 [2,3] and recently updated in 2024 [3].

In general, diagnosis of HLH can be made based upon molecular or functional cellular findings consistent with HLH in combination with meeting at least 5 clinical criteria (Textbox 1). While these criteria have been widely used for diagnosis and treatment decision-making, there have been questions raised about the specificity of these criteria, particularly as they relate to secondary HLH [4]. HLH frequently presents with nonspecific clinical features that overlap with severe infection, sepsis, malignancy, and other inflammatory syndromes, leading to substantial diagnostic uncertainty in real-world clinical practice. Most studies of HLH have focused on small cohorts of participants who have received a clinical diagnosis of HLH. To evaluate the utility of HLH criteria, it may be informative to determine the association of each individual HLH criterion against an overall clinical HLH diagnosis, whether HLH-specific treatments received or an HLH outcome such as mortality.

Most studies of patients with HLH use small, highly curated datasets. Such studies are therefore limited in their ability to evaluate how well diagnostic criteria perform in broader, unselected clinical populations. Few studies have leveraged electronic health record (EHR) data, which may represent broader populations and contain a wider range of clinical information. Leveraging EHR data to facilitate clinical research of HLH requires the construction of an accurate patient cohort. Given that EHR data are not entered with the goal of facilitating future downstream research, manually entered data such as diagnosis codes can often be incorrect or missing. Computable phenotypes are machine-evaluable definitions for a given condition developed using EHR data. Multisource computable phenotypes incorporating laboratory results, medications, procedures, and clinical events may improve case ascertainment and reduce bias compared with diagnosis codes alone [6]. To our knowledge, no attempt has been made to use diverse EHR data to develop a machine-evaluable approach to HLH identification [7-9]. It is unknown how alternative EHR-based definitions perform for a clinically heterogeneous and dynamically evolving syndrome such as HLH.

The primary objective of this study was to compare various approaches to using EHR data to build a computable phenotype of patients with HLH—specifically, we compared three approaches: (1) HLH-specific diagnosis code usage, (2) HLH-specific treatment plan application, and (3) HLH clinical criteria, among all patients at a pediatric hospital. Our hypothesis was that the use of diagnosis codes and treatment plans would undercapture patients and the use of clinical criteria would overcapture patients. Among patients who met at least five HLH clinical criteria, the secondary objective was to explore characteristics and outcomes among those who received HLH-directed therapy versus those who did not.

Figure 1 shows the intersection of the 388 patients identified as having HLH using our three approaches to HLH-cohort creation using three different computable phenotypes (diagnosis code, treatment plan, or HLH criteria). All patients with an HLH-specific treatment plan also had either an ICD-10 HLH diagnosis or met clinical HLH criteria. Only 36 patients were identified by all three methods as having HLH. Table 1 shows the number of patients with an HLH ICD-10 diagnosis code, an HLH treatment plan, or who met at least 5 HLH clinical criteria. An HLH ICD-10 diagnosis (n=220 patients) and meeting 5 or more clinical criteria (n=245) were much more common than an HLH treatment plan (n=42 patients). Among the 220 patients with an HLH ICD-10 code, only 35 patients received chemotherapy during the HLH-specific encounter. However, 85 patients received HLH-directed therapy during the HLH-specific encounter. Conversely, among the 245 patients who met at least 5 clinical criteria, 78 patients received chemotherapy and 193 patients received HLH-directed therapy during the HLH-specific encounter.

Table 2 shows the demographic features of unique patients who met 5 or more HLH criteria stratified by receipt of HLH-directed therapy versus not. The median age of the cohort was 7.7 years (IQR 1.4‐13.9). Of the 193 patients who received HLH-directed therapies, 67.9% (n=131) only received steroids and 29% (n=56) received steroids in combination with an additional HLH-directed therapy. Those with sepsis were less likely to receive HLH-directed therapy (13.5% vs 4.7%; P=.05). Those patients receiving HLH-directed therapy were more likely to have an ICD-10 HLH diagnosis code during that encounter (P=.001). Notably, there was no difference in any specific HLH criteria between those who did and did not receive HLH-directed therapy although high ferritin was almost universal in both groups. The absolute difference in in-hospital mortality and 30-day mortality rates between the groups was 1.5% (95% CI –1.1 to 1.1) and 1.2% (95% CI –1.2 to 1.1), respectively. The absolute difference in median length of stay between the two groups was 12.6 days (95% CI 4.1 to 22.4).

Table 3 shows that among the 1325 encounters (where individual patients can have more than one encounter) with high ferritin and fever, cytopenia, hypertriglyceridemia or hypofibrinogenemia, and splenomegaly were very common. Of the 1325 encounters, 19% (n=252) had >5 HLH criteria. Even among those with less than 5 clinical criteria, cytopenia was present in 44% (n=475) of encounters while hypertriglyceridemia or hypofibrinogenemia and splenomegaly were present in 24% (n=255) and 27% (n=290) of encounters, respectively.

Multimedia Appendix 1 shows the distribution of clinical criteria among those with an HLH ICD-10 code and among those with at least 5 clinical criteria. Some patients with their first HLH ICD-10 diagnosis code had no clinical criteria.

In this study, we compared three approaches to constructing an EHR-derived cohort of pediatric patients with HLH: (1) ICD-10 diagnosis codes, (2) HLH-specific treatment plans, and (3) fulfillment of HLH-2004 clinical criteria. Our findings demonstrate substantial variation in the populations identified by each approach, with diagnosis codes and clinical criteria identifying a far greater number of encounters than HLH-specific treatment plans. We also identified that among patients meeting at least five HLH-2004 clinical criteria, a large proportion received HLH-directed therapy, and that among a cohort of patients with fever and an elevated ferritin, a substantial portion have other findings of HLH such as cytopenia, hypertriglyceridemia or hypofibrinogenemia or splenomegaly.

Comparing our three approaches to EHR-based cohort construction yields numerous interesting insights. First, we observed that HLH ICD-10 codes were relatively common in our dataset. A large proportion of patients with an HLH ICD-10 code did not receive HLH-directed therapy, suggesting that coded diagnoses may be used for suspected but unconfirmed HLH. Conversely, while HLH treatment plans were highly specific, they were rarely used. This is likely because many pediatric patients with HLH do not require chemotherapy and receive HLH-directed medications that are ordered outside of a chemotherapy plan. Thus, the use of treatment plans, which are likely the most accurate way to identify HLH-directed medications, likely underestimates the true HLH population. Clinical criteria, while nonspecific, identified the broadest cohort, consistent with the recognition that HLH represents a clinical syndrome overlapping with other hyperinflammatory states. The exclusive use of these criteria can also be problematic in an EHR cohort, where certain laboratory tests may be done at a referring institution such as for a patient with HLH being referred for a hematopoietic stem cell transplant. This is supported by the fact that we had some patients who met zero HLH clinical criteria but had an HLH ICD-10 code applied. Future research leveraging EHR data to assemble an HLH cohort needs to consider the advantages and disadvantages of different techniques for cohort assembly to select the best approach for a given project. For example, a researcher interested in understanding the diagnostic odyssey of patients with HLH prior to treatment to improve time to diagnosis might use diagnostic criteria as this would more broadly capture patients compared to using a more narrow approach like diagnostic codes. Future work should also explore the usage of a combination of approaches such as having both HLH-criteria met and an HLH diagnosis code and using additional filters in the treatment approach such as dosages or duration for medications that might be used for indications other than HLH management such as dexamethasone. It will also be important to validate these approaches at other institutions to understand generalizability.

Using our newly developed EHR-based cohort of children with HLH, we were able to generate new insights into the management and outcomes of patients with HLH. Of the 245 patients who met the HLH-2004 criteria for HLH, 27% (n=52) received no HLH-directed therapy. No differences were identified in the occurrence of HLH-specific criteria such as ferritin or fever between the two cohorts. From this work, it remains unclear as to what drove the difference in clinical decision-making to start HLH-directed therapy for patients meeting 5 criteria. We did identify that patients who did not receive HLH-directed therapy were more likely to have sepsis than those who did [17]. This is not surprising given that we know that sepsis and HLH have an overlapping presentation. Patients who received HLH-directed therapy also had a significantly longer length of stay (33.4 vs 20.8 d). It is important to note that regardless of whether or not a patient who met 5 criteria received therapy, their 30-day mortality risk was approximately 1 in 6. This is similar to other contemporary studies of patients treated for HLH, where mortality rates have ranged from 23% to 55% [14,18-20]. Given the high mortality rate in those who did not receive therapy, it raises the question of whether or not we are failing to treat a subset of patients with HLH with appropriate HLH-directed therapies.

We also evaluated the distribution of HLH criteria within an EHR-derived cohort of patients with high ferritin and fever. An elevated ferritin and fever are often used as the entrance criteria into a “rule-out” HLH algorithm at many institutions that results in the ordering of additional HLH-specific labs or consultation of an HLH-specific team such as hematology/oncology and/or rheumatology [16,21]. In this cohort that mirrors a potential cohort of patients where an HLH work-up and diagnosis may be considered, we identified that a significant proportion of patients will meet additional HLH criteria. However, only 20% (n=252) of the patients in this cohort met >5 HLH criteria, suggesting that if fever and ferritin are used as entrance criteria into an HLH algorithm, 5 patients would need to be evaluated to identify 1 patient meeting the HLH clinical criteria.

The strengths of this study are access to all data in a curated and validated EHR data set, which allows for the inclusion of patients without an HLH diagnosis. Unlike prior studies restricted to curated clinical cohorts, our approach provides insight into the challenges of applying HLH diagnostic frameworks in real-world, unselected populations. However, limitations of this study include that it was done at a single pediatric institution, so results may not generalize to other institutions. Specifically, in the Canadian context, we may underutilize ICD codes compared to American peers given different implications on billing and remuneration [22]. Second, we were limited in validating the performance of our cohorts against a gold-standard cohort, as no such curated dataset exists that includes both primary and secondary patients with HLH. Third, EHR-derived data are prone to misclassification. Diagnosis codes may not reflect confirmed diagnoses, laboratory results may be incomplete (as SickKids is a large referral center so some labs may have been done at a referring institution and may not have been included in our dataset), and text-mining strategies (eg, for splenomegaly) have imperfect specificity. Fourth, we centered our analysis on the HLH-2004 criteria given these were the criteria being used by clinicians during our study period. The HLH-2024 criteria have since been published, which differ slightly from the HLH-2004 criteria by removal of the natural killer cell activity criterion. Given that only 5% (n=60) of patients had natural killer cell activity testing performed in our cohort, removal of this criteria is unlikely to impact results. Finally, treatment receipt was defined by medication administration records, which may not fully capture the intent of medication prescribing, and this analysis may be confounded by medication indication.

Constructing HLH cohorts from EHR data is challenging, with diagnosis codes, treatment plans, and clinical criteria each capturing distinct but overlapping populations. While clinical criteria identify the broadest group, they lack specificity, underscoring the limitations of applying rigid diagnostic frameworks to real-world EHR data. Developing accurate, machine-evaluable HLH phenotypes will be critical to advancing both clinical care and research in this rare but important condition.