Secondary data studies use data that were originally collected for purposes other than research. For instance, epidemiologists may conduct analyses using administrative health data generated from operating hospitals and compensating physicians for health care services rendered to patients. In many high-income countries, administrative health data are collected across large geographic areas, encompassing a heterogeneous set of patients with diverse medical needs [1]. When researchers explore real-world clinical outcomes using these data, they must adjust for variation in the burden of disease to attain unbiased results. One way to quantify differences in patient health is to measure the number and severity of concurrent medical conditions, or ’comorbidities’. The comorbidity indices introduced by Charlson and Elixhauser are often used in risk adjustment models [2, 3]; however, many other options exist. Furthermore, a careful review of methodologies reveals inconsistency in application across studies. These differences may have major implications for the utility and effectiveness of comorbidity indices when attempting to adjust for patient complexity across populations and healthcare settings. Furthermore, they may invalidate comparisons across studies.

The aim of this work is twofold. First, we provide an overview of frequently used comorbidity measures available within the medical literature that rely on coded secondary data. A wide range of options is described, emphasising the Charlson, Elixhauser, and medication-based comorbidity indices. Then, we examine implementation-related considerations, including linking data, applying a look-back period, representing disease as either individual variables or a weighted sum, ensuring the diagnosis occurred prior to the study index date, and interpreting “paradoxical” diagnoses. Our objective was to offer comprehensive guidance to researchers in selecting and applying a comorbidity index to use in a risk adjustment model based on conceptual merit, performance metrics, and the context of their study.

Comorbidity measures are tools that capture variability in the burden of coexisting diseases. They may be incorporated into a model as the exposure or used to stratify patients. Although they are commonly used and consistently demonstrate value in healthcare management and research, there is no gold standard [4–6]. The focus of this paper is restricted to comorbidity indices that use coded data. These can be divided into two broad categories: 1) diagnosis- and 2) medication-based measures (Figure 1). The comorbidity indices introduced by Charlson and Elixhauser use case definitions to identify pre-established comorbidities. In contrast, others have aimed to comprehensively capture disease using the groupings present in the International Classification of Diseases (ICD) coding structure. Finally, medication-based comorbidity measures, including the Chronic Disease Score (CDS) and RxRisk, map dispensed medications to chronic disease. Although recording of dispensed medications is generally viewed as more reliable and complete than reporting of diagnoses, the disease mapping process implies medical conditions with no available treatment, medications that treat multiple diseases, and instances where patients fail to fill prescriptions are not captured [7–11]. Given these limitations, it is perhaps not surprising that diagnosis-based indices often perform better than prescription-based measures in comparative studies [4, 7, 12, 13]. However, using prescription data may be preferred due to data availability or the study setting.

A research team may select a comorbidity index after using existing literature to identify the most relevant medical conditions associated with the exposure-outcome relationship. However, the choice of comorbidity index should not be restricted to the comorbidities included in the study. Any additional diseases relevant to the research question should be coded as indicator variables and carefully reported.

The Charlson Comorbidity Index (CCI) was initially developed to predict mortality using 19 conditions identified from a chart review of a sample of 559 medical patients admitted to a single hospital over one month in 1984 [2]. Each patient’s comorbidity score was computed by summing the empirically-derived weights associated with the conditions. The CCI was validated on a sample of 685 patients treated for primary breast cancer between 1962 and 1969 and used to predict non-cancer death within ten years of the commencement of anti-neoplastic therapy. Limitations of this study include the small sample size, the date of data collection, and the challenge in attributing cause of death in the validation study. Nonetheless, the CCI has been cited over 45,000 times and has displayed excellent discrimination across diverse cohorts [12, 14–18].

Three adaptations of the CCI to the International Classification of Diseases, version 9 (ICD-9) were created to facilitate its application to administrative health data [19–21]. Although disease prevalence differed based on the definition used, comorbidity index performance was determined to be essentially equivalent [4], [22]. Similar findings were observed when three ICD-10 adaptations were compared, suggesting robustness to disease definition [18]. The most commonly used disease definitions are those introduced by Deyo and Quan for ICD-9 and ICD-10, respectively [4]. In 2011, updated weights were proposed to reflect advancements in standards of care, evolution in understanding of disease burden, and novel therapies; however, the original CCI weights continue to be widely used [16].

Elixhauser’s comorbidity measure was constructed using ICD-9-CM coded administrative data from nonmaternal adult admissions across 439 acute care hospitals in 1992 [3]. The outcomes of interest included hospital charges, length of stay, and in-hospital death. Subgroup analyses were conducted on ten disease-specific cohorts to test model performance across acute, chronic, surgical, and nonsurgical conditions. Thirty comorbidities were identified as being associated with the outcomes of interest during the variable selection process. These disease groups did not have pre-assigned weights but were intended to be included as individual binary covariates in risk adjustment models, and researchers were encouraged to exclude comorbidities deemed irrelevant to their research question. Compared to the CCI, Elixhauser’s approach is more comprehensive in terms of the medical conditions, study populations, and outcomes considered. However, adding 30 variables to a model may cause collinearity and instability. van Walraven and colleagues have introduced a weighted scoring system that uses the Elixhauser co-morbidities and was derived with in-hospital death as the outcome [23].

ICD is used internationally to record and track diseases, health conditions, and other health problems. Some researchers have attempted to comprehensively capture patient complexity by utilising the ontology of ICD codes. Comorbidity measures such as the Multi-Morbidity (MM) Index consider all ICD-9 diagnoses, whereas the Multipurpose Australian Comorbidity Scoring System (MACSS) consists of the 100 most frequently used codes [24–27]. The Multidimensional Multiple Morbidity (MMM) Index groups codes by body system and standardises severity scores to range from zero to one within each group [24] The highest relevant score is selected from within each body system and used in the model for each patient [24]. Compared to ICD-9, ICD-10 offers a more complete and precise list of diagnoses, making it more feasible to consider blocks of codes [24], [28, 29].

There are important considerations that must be recognised with regard to the utility and performance of indices that rely on coded diagnoses. An ICD structure-based approach supports a more nuanced capture of complexity by including more codes. However, it may lead to the inclusion of broad, nonhomogeneous disease groups wherein the differential impacts of the conditions are obscured [24], [26], [30]. In contrast, itemising too many comorbidities may deter uptake and can yield unstable disease weights due to collinearity and overfitting [14], [17], [31]. The inclusion of rare diseases may also result in model instability [16]. Thus, the challenge is in selecting and implementing a variable reduction approach that will identify comorbidities that are prevalent or have a significant impact on health outcomes. Unfortunately, there is no consistent technique that is used in the literature. For instance, prior studies have used minimum prevalence thresholds ranging from 0.02 to 2%, but the choice has been arbitrary [26, 27, 29, 30, 32, 33]. Leveraging machine learning techniques with clinical expertise may advance research in this domain.

There are many other ways to adjust for comorbidity. Some researchers have combined the Charlson and Elixhauser indices into a single comorbidity index [17, 34–37]. Others have supplemented the existing indices with clinical knowledge and created disease-specific measures for medical conditions such as cerebral palsy [38], cardiovascular disease [31, 39–42], and cancer [43]. Comorbidity measures such as the Global Risk-Adjustment Model (GRAM), Adjusted Clinical Groups (ACG) System, Diagnostic Cost Groups (DCG), and the Centers for Medicare and Medicaid Services Hierarchical Condition Categories (CMS-HCC) were developed for remuneration purposes and combine demographic and diagnostic information to create clinically homogenous groups [44, 45].

Another way to characterise the burden of comorbid disease is by using the medications an individual receives. The Chronic Disease Score (CDS), RxRisk, and others include medications dispensed from outpatient settings. The CDS was constructed using a consensus-based approach to select 17 chronic conditions, identify relevant medication classes, and develop a weighting scheme [46]. An adult patient is considered to have one of the chronic diseases if dispensed any drug in the corresponding medication class(es) over a one-year period. This approach was evaluated on the basis of year-to-year stability and the strength of association with physician rating of disease severity, patient-reported health status, mortality, and hospitalisation [46]. The revised CDS, or CDS-2, includes 28 chronic and mental health conditions, offers a more inclusive list of medications, and assigns empirically derived weights to medication classes based on healthcare utilisation and costs [47].

The RxRisk is an expansion of the CDS-2 designed for both pediatric and adult populations [9]. The component medical conditions and associated drug classes differ based on the patient’s age [9]. In contrast, the RxRisk-V was derived to predict cost using data from the Veterans Health Administration (VHA) and includes 45 disease categories of research or clinical priority [11]. Pratt and team mapped the RxRisk-V to the Anatomical Therapeutic Chemical (ATC) Classification System and demonstrated moderate to high accuracy in predicting one-year mortality among older adults in Australia [48]. The RxRisk-V was expanded by Johnson and colleagues to include 26 additional disease categories and used to predict mortality [49].

Two additional pharmacy-based comorbidity measures are worth noting. The Medication-based Disease Burden Index (MDBI) comprises 20 chronic conditions that are leading contributors to global death and managed pharmacologically [50]. In contrast, the medication-based Chronic Disease Score (medCDS) captures the most prevalent chronic conditions among elderly patients in Germany and predicts death using medication classes derived from treatment guidelines [51]. Some researchers may feel the international scope of the MDBI makes it an appealing option, whereas others may appreciate that the medCDS uses treatment guidelines to select medications.

Some contemporary measures of comorbidity incorporate both diagnostic and pharmaceutical data. Examples include the Nordic Multimorbidity Index [52]and the Cambridge Multimorbidity Score [53, 54]. The Nordic Multimorbidity Index was constructed using 50 covariates to predict 5-year mortality [52]. In contrast, the Cambridge Multimorbidity Score was derived using 37 long-term conditions to predict primary care consultations, unplanned hospital admissions, and death [53]. Data regarding diagnosis and dispensed medications have also been used to assess associations between broad categories of mental disorders and medical conditions [55].

Comorbidity indices are widely used despite variation in discrimination based on the cohort and the area of application (e.g., length of stay, mortality, readmission, etc.) [12, 25]. Differences in performance within a specific setting are often modest, and we would not suggest there is a particular comorbidity measure that uniformly outperforms other measures. For example, poor performance is consistently noted when predicting readmission [33, 56]. Likewise, there is no index that is consistently preferred when considering either short- or long-term health events [57, 58]. Variation in discrimination based on outcome may result from discordant effects of comorbidity or signify the absence of medical conditions that are relevant to the outcome [59]. Failure to adjust for the competing risk of death in the analysis may also introduce bias. For these reasons, the recommendations below are considered setting agnostic. Furthermore, rather than evaluating the magnitude of the metric reported, the focus is on statistically significant differences based on variation in methodology.

Figure 2 summarises the considerations for implementation of comorbidity indices that are discussed herein: 1) the look-back window used for disease ascertainment; 2) whether to link additional data sources; 3) how to represent comorbid conditions; 4) temporality of diagnoses; and 5) inclusion of paradoxical conditions. Although we present these as decisions to be made after selecting a comorbidity index, careful thought may ultimately impact the choice of index, making this process somewhat iterative.

The look-back period is the interval measured retrospectively from the index hospital visit used for disease ascertainment. Due to limited data availability, early comorbidity indices did not use a look-back period but were developed using a single hospital record. Contemporary researchers can readily follow patients across time; however, there is debate concerning the necessity of implementing a look-back period and the optimal time frame. Intuitively, looking retrospectively should allow researchers to correct for incomplete recording of comorbidities across visits and improve model performance [37]. However, there is also an increased risk of capturing conditions that may be misdiagnosed, in remission, or resolved [19, 37]. Furthermore, when considering the index hospital stay exclusively, patients are risk-stratified based on medical conditions that necessitate active monitoring and impact their treatment [60]. Focusing on these diagnoses may be more valuable than a comprehensive patient’s disease profile assessment.

Ultimately, the decision of whether or not to use a look-back period may be moot. Studies that used a look-back period noted increased disease prevalence but no meaningful improvement in discrimination for short-term outcomes such as in-hospital mortality, inpatient costs, length of stay, 30-day mortality, and 30-day readmission [14, 56, 58, 61, 62]. Some studies have observed a slight improvement from using a one-year look-back period when predicting one-year mortality; however, this finding was inconsistent [14, 56, 58, 61]. Increasing the look-back period beyond one year in studies that assessed mortality and resource use demonstrated minimal benefit [61, 63].

Negligible or inconsistent improvement in performance from an increase in data may seem surprising. However, the index visit is expected to capture the diagnoses that impact a patient’s immediate health status and, ergo, short-term outcomes [14, 19]. Holman argues that a one-year look-back can act as a compromise in identifying active comorbidities; however, the current literature suggests only studies involving long-term outcomes (i.e., ≥one year) will benefit [25, 28] In summary, the absence of strong evidence favoring a look-back period means the index hospital visit will likely provide sufficient information for outcomes within one year. A one-year look-back window is recommended when considering outcomes beyond one year.

Further work is recommended to examine whether the look-back period should vary based on the disease/comorbidity of interest. For example, Tonelli and colleagues suggest disease-specific look-back windows and algorithms that change based on the database used [64]. Another important area for future research is exploring how disease severity and the competing risk of death change as the look-back period increases; this is expected to vary based on the comorbidity, the context, and the setting of the study.

Linking diagnostic data from hospital billing records with practitioner claims from primary care, specialist, and ambulatory encounters is expected to increase disease ascertainment and yield a more complete and accurate description of a patient’s comorbidities. However, compared to inpatient settings, disease is expected to be captured earlier and at a less severe stage in an outpatient environment, suggesting a weaker strength of association with health outcomes of interest [65]. Furthermore, there is also an increased risk of misclassifying rule-out diagnoses as comorbidities [12, 19].

Prior research has shown an increase in the prevalence of both acute and chronic conditions when using linked inpatient and outpatient data [28, 66]. Discrimination improved when physician claims were used to augment hospital records in a study that examined one-year hospital costs; however, results were mixed when one-year mortality was considered [28, 66, 67]. Linkage between hospital and pharmaceutical data has also been studied. Dispensed medications have been demonstrated to identify unrecorded chronic conditions with no statistically significant benefit in discrimination [7, 11].

It is important to consider why a diagnosis may be missing from a hospital record and to recognise that this absence may reflect reduced disease severity. Likewise, the environment where the initial diagnosis is made may have important implications. The certainty and severity of a diagnosis made in a hospital or primary care setting are expected to vary due to accessibility, technological infrastructure, laboratory facilities, available resources, and disparate workflows. Unfortunately, quantifying severity using current coding systems is incredibly challenging, and additional research is needed to confirm this hypothesis. There is also an absence of work that explores the impact of data linkage on outcomes within one year, and further investigation in this area is warranted. Nonetheless, when exploring long-term events, there may be a benefit in supplementing hospital records with one year of data from other settings to allow researchers to identify diagnoses associated with health deterioration over time.

There are two core strategies for capturing comorbidity in a risk adjustment model. A single score can be used to summarise the impact of concurrent conditions, or an independent variable can be assigned to each condition. The CCI is an example of the single score method, wherein pre-established weights assigned to 19 conditions are summed to estimate the cumulative effect of comorbidities. This approach is easy to apply and interpret: a higher comorbidity score implies increased complexity. As a weighted score, a comorbidity index allows us to consider the role of comorbidity as an exposure or risk stratify patients into groups based on disease severity. Furthermore, since a single variable is used to represent comorbidity, nonlinear associations and interactions with sociodemographic factors, such as age and sex, are easy to model [34, 39, 68]. However, patients with the same score are not necessarily medically similar and heterogeneity in the disease profile of patients may mask the impact of specific comorbidities. The weighted score method is expected to perform best when applied to samples wherein the distribution of comorbidities and their prognostic impact are comparable to the sample from which the disease weights were derived. Yet, application is observed across various age groups, disease subpopulations, outcome measures, health care settings, and time [15, 43, 45, 63]. Differences in discrimination have been noted based on the cohort (e.g., patients with congestive heart failure, diabetes, or chronic renal failure [69]), outcome (e.g., 1-year mortality or hospitalisation [63]), and country of dataset origin (e.g., Canada, Japan, or Switzerland [18]). On a more granular level, Gagne and colleagues also noted differences in discrimination based on the measure of mortality (i.e., 30-day, 90-day, 180-day and 1-year mortality [34]). It is important to routinely revise disease weights to reflect changes in the standards of care and the availability of novel pharmaceutical therapies [17, 34]. Thus, from a recency standpoint, using the revised Charlson weights proposed by Quan and colleagues in 2011 is preferable to using the original weights proposed in 1987 [16].

An alternative approach to the above is to build an additive model that includes one covariate for each comorbidity of interest. A common choice is to model the medical conditions identified by Elixhauser and colleagues [3]. When applying this method, the coefficient associated with each disease will fluctuate based on the data. This approach is expected to improve model performance [12, 18, 69]. However, the increased number of covariates may lead to overfitting and collinearity, particularly when considering small samples, rare diseases, and infrequent outcomes [13, 14, 17, 18, 35]. For instance, although HIV/AIDS is among the comorbidities proposed by Elixhauser, it may be reasonable to omit this disease and ensure the stability of the model, particularly if the condition is rarely observed in the sample and is not directly relevant to the research question.

Comorbidity indices aim to quantify patients’ overall health and group individuals with similar levels of risk. For this purpose, using a single covariate appropriate to the health care setting and study timeframe is recommended over using many disease-specific indicators [17]. A scoring system obviates the need to identify and exclude rare diseases that jeopardise the stability of the model. Furthermore, only a single regression coefficient needs to be considered, not several disease-specific associations. The most compelling argument against using a single score is the increased flexibility and, therefore, improved performance that is expected from using one variable for each respective comorbidity. However, a literature review revealed that models using individual variables do not consistently outperform those using a weighted sum [16–18, 23, 31, 39, 45, 62, 68, 70]. Suppose a research team is interested in specific risk factors or concordant comorbidities. In that case, these can be excluded from the comorbidity index and modelled as independent covariates alongside the modified weighted sum [71, 72]. Nonetheless, using binary variables may be preferred if it is better aligned with the study aims and design and if there is sufficient power based on sample size [73].

When a comorbidity index is used in a risk adjustment model, it is essential to consider the study index date. When forecasting outcomes at the time of admission, incidents that arise after the patient is admitted must be excluded since they represent future events relative to cohort time zero. In contrast, events that transpired during the hospital stay would be expected to influence the patient’s baseline state at the time of discharge, although the research question may still require differentiation between comorbidities and complications of care [30, 56]. Likewise, it is important to consider immortal time bias especially when study index date and exposure initiation are not simultaneous. The artificial creation of a time period where health events cannot occur (i.e., immortal time) would result in biased results due to an overestimation of time at risk [74]. Further, in the context of a comorbidity index, it is also important to consider how changes in disease status during the follow-up period may bias results [75]. Establishing the temporality of a diagnosis can be challenging when using administrative data [62]. Therefore, it is vital to consider the implications of misclassification and proposed preventative measures.

Misclassifying in-hospital complications as comorbidities when using time of admission as the index date is expected to overestimate model performance and disease prevalence [33, 76–78]. Southern et al. compared models that used the diagnosis type indicator to identify and exclude complications to those that did not [78]. Consistent overlap in the 95% confidence intervals of the odds ratios, deviance, and discrimination was observed [78]. Likewise, Roos et al. considered three surgical procedures and implemented two variations of the CCI but only observed one instance where failure to exclude complications resulted in “spuriously high discrimination”, evidenced by a relative change of 7% [79]. However, the proportion of patients with at least one complication and the proportion of diagnoses which denote complications is known to vary based on the cohort and the condition studied [76, 78, 79]. Thus, the repercussions may be more significant if this error is common or if outcomes differ substantially among patients when the diagnosis is acquired during the hospital stay [79].

It is difficult to distinguish between complications and acute conditions, especially when the cohort includes a diverse group of patients [4]. As Jean and colleagues explain, even though it is common for patients to experience adverse events in the hospital, specific complications are rare, and instances that result in significant disability and impairment occur with an even lower frequency [80]. Various solutions have been proposed. Clinical knowledge can be used to identify and exclude common complications of care [62]. Alternatively, evidence of acute events may be extracted from prior visits but discredited if they are only captured during the index admission [19, 21]. In Canada, a diagnosis-type indicator is used to differentiate between diagnoses made pre- and post-admission [78]. More widespread recording of this data element is the most foolproof approach to establish temporality relative to the index date.

A chronic condition is expected to increase the risk of poor health outcomes. However, a statistical model may indicate that certain comorbidities are protective factors. Some weighting schemes exclude these “paradoxical conditions” (i.e., assign a weight of 0) while others assign a negative weight [17]. For instance, the algorithms used to derive disease weights for both the original and updated CCI exclude negative values [2, 16]. These authors may have felt that content validity dictates that a disease should increase patient complexity and the risk of poor outcomes [42, 59, 81, 82]. In contrast, van Walraven and colleagues allow the Elixhauser comorbidities to be assigned a value less than zero [23]. Some authors postulate that paradoxical conditions may represent real clinical phenomena that are not yet understood or indicate unmeasured differences in patients [35, 62, 83]. For example, patients with paradoxical conditions may be monitored more closely, receive treatment at an earlier stage of disease, have longer hospital stays, seek follow-up care more consistently, or adhere to drug therapies that yield the perceived protective association [13, 25, 33, 40, 66, 83]. An alternative explanation is bias from incomplete coding. More precisely, the documentation of paradoxical conditions may signify a healthier individual who lacks serious comorbidities or complications [23, 34, 35, 62, 72, 81, 84, 85]. Vaughan-Sarrazin and colleagues studied the prevalence of paradoxical conditions and their impact on risk adjustment [85]. The group concluded that contributing mechanisms may vary after observing inconsistency in paradoxical conditions among patients with acute myocardial infarction (AMI) and those who underwent coronary artery bypass graft (CABG) surgery [85]. Within each cohort, reporting of paradoxical conditions increased over time, but the direction of the association remained consistent [85]. Some studies have shown that model performance is enhanced by including paradoxical conditions [83, 85], while others have identified no benefit [63].

It is essential to acknowledge that a comorbidity score of zero indicates the patient may not have the conditions included in the comorbidity index, but does not imply they are immune to adverse outcomes. As such, an individual with a negative comorbidity score should be interpreted as one with a comparatively lower but still non-zero risk. Since the concept that specific comorbidities may be “protective” is founded in data and accepted in statistical modelling, excluding negative weights in comorbidity scoring systems may not be appropriate. However, studies that explore paradoxical conditions have focused on short-term outcomes. Further research examining whether these protective relationships persist for different outcomes and time periods beyond 30 days is warranted. Based on current evidence, retention of paradoxical conditions is recommended but only for short-term (<30 days) outcomes.

Comorbidity indices allow researchers to adjust for differences in the baseline complexity of patients in a risk adjustment model. Comparative studies suggest that diagnosis-based indices often perform better than prescription-based measures [4, 7, 12, 13]. However, within these two groups, no single choice consistently excels above the rest. The use of a comorbidity index that combines diagnostic and prescription fill data, such as the Nordic Multimorbidity Index, is also a compelling option [52].

Implementing a comorbidity measure as a weighted sum makes it easier to interpret complexity with little to no decline in discrimination. If specific comorbidities are central to the study, it may be appropriate to exclude these from the weighted sum and code them as covariates alongside the modified risk score. However, the increased flexibility from coding a large set of binary variables is worth considering in studies with large samples or when the cohort differs from the sample from which the weights were derived. Health events that arise after the study index date should be excluded from the model to respect temporality. For studies focusing on outcomes beyond one year, there may be some benefit from using data from a one-year look-back window or linking with additional data sources to adjust for incomplete capture of secondary diagnoses. For short-term outcomes, it is important to acknowledge that certain diagnoses may be associated with a “paradoxical” reduced risk. Choosing the most appropriate comorbidity index for the research question while keeping these considerations in mind will improve interpretability, consistency, and overall quality in future epidemiologic studies.

It is important to consider the consequences of not using comorbidity indices appropriately. Application of these indices beyond their intended purpose may reduce analytic precision and fail to accurately adjust for the underlying health status of patients. This may introduce bias from misclassification and inaccurate risk stratification. If differences between healthier patients and those with more complex health needs exist but are distorted, there may be an over- or underestimation of the effect size, which could jeopardise generalisability. Another potential concern is residual confounding, wherein biased estimates of the association between the exposure and outcome are obtained. Incorrect attribution of differences to the exposure rather than patient complexity may lead to inaccurate conclusions and compromise the study’s validity. This is particularly concerning for observational studies wherein patients are not randomised to the exposure of interest. Finally, inconsistent use of comorbidity indices, including variation in coding practices, may reduce comparability across studies and increase between-study heterogeneity in subsequent meta-analyses.