Using the VHA electronic health record databases, we identified 20,532 patients treated with curative intent for nonmetastatic primary NSCLC who were diagnosed between January 1, 2008, and April 30, 2016. We used the International Classification of Diseases, 9th/10th Edition (ICD-9/10) diagnosis codes (Table S1 in Multimedia Appendix 1). We also retrieved demographic, clinical, and vital status information for these patients. For inclusion, patients must have had either ≥1 inpatient or ≥2 outpatient NSCLC diagnosis codes, and the date of diagnosis must be the date of the earliest code demonstrating connectedness to care [17]. Curative-intent treatment was classified based on the ICD-9/10 procedure codes and Current Procedural Terminology (CPT) codes (Table S2 in Multimedia Appendix 1) within 6 months of diagnosis. The NSCLC stage was classified according to the American Joint Committee on Cancer TNM (tumor, nodes, and metastasis) staging system (seventh edition in effect during the observation period). We excluded patients who did not receive curative-intent therapy, received chemotherapy only, died within 6 months of initiating therapy, had stage 4 cancer, had a non–skin cancer diagnosis (except melanoma and cervical cancer) within 5 years before the NSCLC diagnosis, or had any diagnosis of lung cancer before 2008. Participant follow-up ended on December 31, 2016.

The study cohort included a random subset of 1890 eligible patients with completely abstracted radiology reports and clinical notes from a 9238-sample cohort. An additional 369 patients from the sample cohort had no radiology reports directly relevant to NSCLC surveillance. To avoid overrepresentation of patients without abstractable radiology reports in our analysis, we weighted these patients so that their proportion in the study cohort was the same as their proportion among all eligible patients in the sample cohort—bringing the final analysis sample to 1888.

We abstracted image indications (surveillance, symptoms, abnormality follow-up, other, and unknown) and modalities (positron emission tomography [PET]; chest X-ray [CXR]; chest CT; and other [magnetic resonance imaging, non–chest CT, and bone scan]) from radiology reports. Imaging indications were identified in the report text and recorded as documented.

We used the previously published semiautomated chart abstraction protocol by our group for obtaining detailed radiology data [18]. A searchable radiology index was created, which included full-text radiology reports that were then queried using SQL (structured query language) to identify all radiology-related text documents to be indexed. VISA, an open-source full-text search engine written in Java, was used to search the radiology reports for text relevant to the user query. Upon searching, VISA returned the user-defined full-text radiology reports with highlighted relevant text snippets, which were then manually reviewed, annotated, and coded by an abstractor.

Clinical notes from patient charts were used to abstract any missing data on disease stage, smoking status, NSCLC recurrence or an SPLC, and tumor histology. Final pathologic staging took precedence during chart abstraction. Smoking at the time of diagnosis or initial presentation was considered “current.” The presence of recurrent NSCLC or SPLC was determined based on documentation by a treating clinician. Histologic subtypes were identified directly from pathologic reports when available or collected from other clinical note types (eg, progress notes). Patients who had small cell histology, had recurrence or SPLC outside the lungs within 120 days after treatment, or did not undergo curative-intent treatment were also excluded.

The primary outcome was receipt of guideline-concordant surveillance imaging. The National Comprehensive Cancer Network (NCCN) recommendations for posttreatment surveillance imaging at the time of diagnosis were used as a base set of guidelines [19]. Although the published guidelines are updated, they have consistently recommended routine posttreatment surveillance with chest CT imaging approximately every 6 months during this study period. We defined guideline-concordant surveillance as performing chest CT imaging for surveillance purposes (as documented by the image indications) within 120 to 270 days following the first day of treatment or first day of last treatment with multimodality therapy. Imaging performed explicitly for surveillance purposes via other imaging modalities (eg, PET) was considered nonconcordant surveillance. The 120- to 270-day (4‐9 mo) posttreatment interval was adopted to represent the first surveillance window, with the intent of creating a more liberal window with a 2- to 3-month buffer on either end of the published 6-month guideline (Multimedia Appendix 1).

This study was approved by the institutional review boards of Stanford University and the Department of Veterans Affairs (IRB-46841). The study data used were deidentified. Informed consent was waived as per institutional review board review.

To estimate the cumulative probability of patients receiving guideline-concordant surveillance imaging within the initial 120- to 270-day surveillance window after curative-intent therapy, we used a competing risks framework to account for competing events. This can help provide unbiased estimates of patients experiencing the event of interest, given that competing events materially alter their probability or preclude them entirely. For example, receiving a diagnosis of recurrent NSCLC before scheduled surveillance precludes guideline-concordant surveillance, as imaging after recurrence can no longer be considered surveillance. Recurrences discovered during surveillance imaging were coded as surveillance events. An example schematic of the time-to-event analysis helps visualize a few of these relatively common clinical scenarios (Figure S1 in Multimedia Appendix 1).

We used the Aalen-Johansen estimator and corresponding CIs to analyze the primary outcome, that is, the receipt of guideline-concordant surveillance imaging within the 120 to 270 days after curative-intent treatment. Death, recurrence, SPLC, and nonconcordant surveillance were treated as the competing events in this analysis.

To evaluate the factors associated with the primary outcome, we used multivariable cause-specific Cox regression to estimate hazard ratios (HRs) and CIs. We considered various demographic factors (eg, sex, age, race, marital status, and year of diagnosis) and clinical factors (eg, treatment, histology, and disease stage) in this model. To adjust for multiple testing, we applied the Bonferroni correction method for 11 independent variables in each model. We reported unadjusted P values and interpreted any value <.004 (<0.05/11) as significant. For consistency, we calculated 99.6% CIs for all point estimates. Missing data were treated as a separate category in models. To avoid very small groups, we combined stage 0 and stage 1 (due to the very small proportion of patients in stage 0). In addition, recurrences and SPLC malignancies were combined as a single competing event referred to as “recurrence” for brevity. A univariable model of the primary outcome is shown in Table S4 in Multimedia Appendix 1. Statistical analysis was performed using R (version 4.2.2) [20].

Of 135,513 patients diagnosed with primary NSCLC between January 1, 2008, and April 30, 2016, a total of 20,532 met the study eligibility requirements (Figure 1). The study cohort comprised a random sample of 1890 patients with abstracted clinical notes and radiology reports, while 369 of all eligible patients had no relevant radiology reports available; these 369 patients were weighted to prevent overestimation. The effective study sample size was 1888 patients after weighting (see Cohort and Definitions section; Table 1). The mean age of the analysis cohort was 66.4 (SD 7.9) years, of which 95.9% (1811/1888) were male, 71.1% (1342/1888) were White, and 43.1% (814/1888) were married. Of the 1888 patients, 57% (n=1076) presented with stage 1 disease, and the most common treatment modality was surgery alone (n=1068, 56.6%).

Overall, the most common imaging performed during the initial 120- to 270-day window was chest CT (1460/3278, 44.5%; Table 2). CXR accounted for 36.3% (1190/3278) of all imaging performed, while the remaining 11.8% (386/3278) and 7.4% (242/3278) were PET scans or other imaging modalities, respectively. Chest CT was found to be the most common modality when evaluating imaging performed for surveillance. The proportion of CXR imaging performed for surveillance was 12.2% (176/1448), while the proportion of chest CT and PET imaging performed for surveillance was 70.9% (1026/1448) and 14.4% (209/1448), respectively. Chest CT remained the most common imaging procedure for surveillance across all types of primary treatment groups (Table 3). However, PET imaging comprised 24% (46/192) of surveillance imaging in patients receiving radiation therapy as a part of their primary treatment (38/129, 29.5% in radiation alone), compared to 14.4% (209/1448) across all other treatment groups combined.

To estimate the cumulative probability of patients receiving guideline-concordant surveillance imaging after curative-intent therapy, we used a competing risks framework to correctly estimate the cumulative probability by accounting for competing events (eg, death or recurrence). This analysis shows that the cumulative probability of receiving guideline-concordant imaging for surveillance was 43.8% (827/1888; 99.6% CI 41.6%-46.1%; Figure 2). The cumulative probability of receiving nonconcordant surveillance imaging within this window (120-270 days) was 15.2% (287/1888; 99.6% CI 13.0%-17.8%), whereas 33.4% (630/1888; 99.6% CI 30.4%-36.6%) of patients received no imaging at all. A total of 5% (95/1888; 99.6% CI 3.8%-6.7%) of patients experienced recurrence, while 2.6% (49/1888; 99.6% CI 1.8%-3.7%) of patients died during the posttreatment surveillance period, that is, roughly 7.6% (144/1888) of patients were ineligible for surveillance during this period.

We evaluated the association between the receipt of guideline-concordant surveillance imaging and various demographic and clinical variables using multivariable cause-specific Cox regression (Table 4). Compared to patients treated for NSCLC between 2008 and 2010, patients receiving treatment from 2014 to 2016 were significantly more likely to receive guideline-concordant surveillance imaging (HR 1.42, 99.6% CI 1.09-1.87, P<.001). There was a similar trend toward an increased likelihood of guideline-concordant surveillance in years 2011 to 2013 compared to 2008 to 2010, although this did not reach statistical significance. No other variables were significantly associated with receipt of guideline-concordant surveillance imaging. Although radiation alone was not significantly associated with a decreased likelihood of receiving guideline-concordant surveillance imaging (HR 0.68, 99.6% CI 0.45-1.03, P=.007), radiation alone was associated with an increased likelihood of nonconcordant surveillance imaging in the same model (HR 1.93, 99.6% CI 1.10-3.38, P<.001) consistent with the increased use of PET for surveillance in this group (Table S3 in Multimedia Appendix 1).

There were no significant factors associated with developing recurrence or SPLC in the multivariable cause-specific Cox proportional hazards model for competing outcomes (Table S3 in Multimedia Appendix 1). Compared to surgery alone, treatment with surgery and radiation was associated with an increased likelihood of death in the same model (HR 12.3, 99.6% CI 2.52-60.19, P<.001).

Despite widespread recommendations for posttreatment surveillance, we found that roughly one-third of patients undergoing treatment for NSCLC before 2016 did not receive any form of surveillance. Moreover, the true fraction of patients receiving surveillance imaging consistent with recommended guidelines in this study was less than half (43.8%). This is an improvement over results from our previous study of patients’ first imaging procedures within 4 to 8 months after resection (representing the first surveillance window) of early-stage NSCLC in the Medicare population, where 24.6% of patients received a chest CT scan and 59.6% received a CXR [7]. Similar to other studies, our work demonstrates evidence of the increasing use of posttreatment chest CT imaging for surveillance over time in line with routine surveillance guidelines [9,14]. Sustained improvement in posttreatment surveillance adherence necessitates dedicated efforts both at the individual level, through education and interventions, and at a systems level, through multidisciplinary collaboration and advocacy.

We observed that the rate of guideline-concordant surveillance CT imaging is substantially lower than that previously reported by Malhotra et al [10] and Erb et al [9]. In their 2018 study using SEER-Medicare data, Malhotra et al [10] defined adherent imaging as a patient receiving at least a CT scan or PET and CT scan during the 7- to 18-month, 19- to 30-month, 31- to 42-month, and 43- to 60-month intervals after treatment. The rate of adherent surveillance imaging at 18 months was 70.6%. The variation in reported rates across these and other studies is likely reflected in the fact that very few large studies concerning posttreatment surveillance contain information on the indication for posttreatment imaging. Therefore, many of these studies likely reflect imaging done for various indications, including the evaluation of symptoms (eg, cough and pneumonia), or are unknown and do not truly reflect imaging done for the explicit purpose of surveillance. In addition, the study by Malhotra et al [10] defines the receipt of adherent imaging as receiving CT or PET and CT despite guideline recommendations against the routine use of PET imaging for posttreatment surveillance of NSCLC, which was excluded in our definition. Our study, as well as other studies, has demonstrated that nonconcordant surveillance imaging makes up a significant fraction of radiology evaluations in patients treated for NSCLC. This may be due in part to clinical practice variations in using distinct treatment modalities to monitor treatment response or disease progression (ie, the use of PET or CT imaging in patients undergoing radiation or neoadjuvant therapy). Differences in patient population, comorbidities, and systems of care delivery also likely play some role in varying the reported rates of surveillance.

An important strength of our study is the inclusion of detailed VHA data that provided insights into the indications for radiology procedures. Indeed, true rates of posttreatment surveillance were previously obscured by the fact that imaging reported from other studies may have been done for purposes other than surveillance, as it is nearly impossible to reliably categorize imaging indications using administrative and claims data alone. Another strength of our study was the ability of the competing risks model to evaluate true surveillance rates in the presence of competing events that alter estimates put forth in other studies. Indeed, we found 7.6% (144/1888) of patients were not eligible to receive surveillance in our sample cohort because of death or recurrence. Accounting for competing events addresses the potential for overestimating the incidence of the event of interest, a potential weakness of using the Kaplan-Meier method or claims-based data in this setting. Each of these features present in this study likely describes our report of relatively lower rates of guideline-concordant surveillance compared with the literature. Thus, using the most liberal interpretation of our data, excluding those patients with competing events of early recurrence or death, 59% (1114/1888) of our cohort received imaging with the explicit intent of surveillance (44%, 827/1888 guideline concordant and 15%, 287 discordant). This is consistent with other reports lacking indication data, yet 33% (630/1888) received no imaging at all during this initial surveillance period. This allows us to highlight the latter group of patients with no surveillance imaging as the most important group to target when considering interventions in the future.

There are limitations to be acknowledged in this study. First is the use of administrative ICD-9/10 and CPT codes to identify patients with relevant treatment procedures. We did not have structured diagnosis dates and staging information from a cancer registry for cohort construction. Thus, we used a conservative approach to ensure patients included in the cohort were newly diagnosed with NSCLC. Our approach may have missed some patients with a previous NSCLC diagnosis and the 20,532-patient cohort such that the data may not reflect the total incidence of early-stage NSCLC in VHA data during this period. Second, we abstracted only radiology report information on patients receiving imaging studies at VHA facilities. Veterans who received relevant imaging studies at external facilities do not have reports that are easily viewable in the VHA database except through an archival process requiring a review request of any scanned reports that were provided by the community imaging clinic. In our sensitivity analyses, we examined the rates of non-VA care including imaging paid for by the VHA (fee-based care) and care that veterans received through Medicare or Medicaid. Data are not available for any community care services patients pay for out of pocket or those covered by other insurance providers such as a veteran’s private insurance provider, although dual use of care from private insurance providers is rare among patients who otherwise rely on the VHA for the majority of their care and, in particular, cancer care [21]. This sensitivity analysis identified that among the entire cohort, fewer than 16% of veterans had any non-VA imaging scans performed in the available administrative data, with 8% of these scans corresponding to CT scans. Thus, the inclusion of these studies would still result in the underuse of surveillance overall. Third, our dataset reflects patterns from 2008 to 2016 as well as care within the VHA, which is a more homogenous population than the general population. Given the changing guidelines, practice patterns, and differences in the general population, our observations may differ from the reality of the non-VA population in the current state. Yet, cancer outcomes within the VHA and quality of care (including lung cancer in particular) have been reported in the literature as being noninferior and at times superior to lung cancer care provided in the non-VA population [22]. Thus, if anything, our main limitation might be in underestimating the true adherence rate in the non-VA setting.

In conclusion, the use of this detailed clinical database, along with the novel application of a competing risks model, has provided additional clarity on the current state of adherence to posttreatment surveillance. Our analysis shows that the rate of guideline-concordant surveillance in this national cohort was lower than that reported in many studies. There is a need to develop efficient strategies to monitor the rate of guideline-concordant surveillance as well as patient education and health care professional advocacy to ensure these patients do not fall through the cracks of the health system. In addition, future studies are needed to determine how guideline adherence and other aspects of current surveillance practices influence clinical outcomes, especially given the recent literature casting doubts on the survival benefits of routine surveillance imaging [22,23].