Considerations for selecting and implementing comorbidity indices when using secondary data sources: a guide for health researchers

Abstract

Comorbidity measures, such as the Charlson Comorbidity Index, are commonly used in risk adjustment models to account for variability in disease burden. This narrative synthesis describes and critiques available comorbidity indices and offers implementation guidance to researchers based on a critical review of existing literature. First, common comorbidity measures are described. Instruments derived using case definitions, grouping of International Classification of Diseases (ICD) codes, and mapping of dispensed medications to chronic conditions are presented. Comorbidity indices that combine diagnostic and medication data are also introduced. No single option consistently outperforms the rest. Next, important considerations when applying a comorbidity index are described. It is crucial to respect temporality and exclude health events that arise after the study index date. Researchers must also weigh the interpretability of using a weighted sum against the flexibility of using a large set of binary variables. When modelling long-term outcomes, there are benefits to applying a one-year look-back window and augmenting data via linkage. For short-term outcomes, certain chronic conditions may exhibit a protective association; however, not all indices capture these relationships. Implementation of these findings will improve the interpretability of comorbidity measures and the quality of future studies.

Highlights

• This narrative synthesis serves to address the challenge of quantifying differences in patient complexity when using administrative health data.
• Available comorbidity measures are described, and guidance on implementation is provided following a critical review of existing literature.
• Differences in the performance of comorbidity measures are often modest, and no single option consistently outperforms the others within a particular setting.
• When implementing a comorbidity measure, health events that arise after the study index date must be excluded. For long-term outcomes, applying a one-year look-back window and augmenting data via linkage is recommended. For short-term outcomes, certain chronic conditions may exhibit a protective association; however, not all indices capture these relationships.
• It is easier to interpret the strength of association between complexity and the outcome when comorbidity is coded as a single weighted sum. However, the flexibility associated with using a large set of binary variables merits consideration in studies with large sample sizes.

Introduction

Secondary data studies use data that were originally collected for purposes other than research. For instance, epidemiologists may conduct analyses using administrative health data generated from operating hospitals and compensating physicians for health care services rendered to patients. In many high-income countries, administrative health data are collected across large geographic areas, encompassing a heterogeneous set of patients with diverse medical needs [1]. When researchers explore real-world clinical outcomes using these data, they must adjust for variation in the burden of disease to attain unbiased results. One way to quantify differences in patient health is to measure the number and severity of concurrent medical conditions, or ’comorbidities’. The comorbidity indices introduced by Charlson and Elixhauser are often used in risk adjustment models [2, 3]; however, many other options exist. Furthermore, a careful review of methodologies reveals inconsistency in application across studies. These differences may have major implications for the utility and effectiveness of comorbidity indices when attempting to adjust for patient complexity across populations and healthcare settings. Furthermore, they may invalidate comparisons across studies.

The aim of this work is twofold. First, we provide an overview of frequently used comorbidity measures available within the medical literature that rely on coded secondary data. A wide range of options is described, emphasising the Charlson, Elixhauser, and medication-based comorbidity indices. Then, we examine implementation-related considerations, including linking data, applying a look-back period, representing disease as either individual variables or a weighted sum, ensuring the diagnosis occurred prior to the study index date, and interpreting “paradoxical” diagnoses. Our objective was to offer comprehensive guidance to researchers in selecting and applying a comorbidity index to use in a risk adjustment model based on conceptual merit, performance metrics, and the context of their study.

Available comorbidity measures

Comorbidity measures are tools that capture variability in the burden of coexisting diseases. They may be incorporated into a model as the exposure or used to stratify patients. Although they are commonly used and consistently demonstrate value in healthcare management and research, there is no gold standard [4–6]. The focus of this paper is restricted to comorbidity indices that use coded data. These can be divided into two broad categories: 1) diagnosis- and 2) medication-based measures (Figure 1). The comorbidity indices introduced by Charlson and Elixhauser use case definitions to identify pre-established comorbidities. In contrast, others have aimed to comprehensively capture disease using the groupings present in the International Classification of Diseases (ICD) coding structure. Finally, medication-based comorbidity measures, including the Chronic Disease Score (CDS) and RxRisk, map dispensed medications to chronic disease. Although recording of dispensed medications is generally viewed as more reliable and complete than reporting of diagnoses, the disease mapping process implies medical conditions with no available treatment, medications that treat multiple diseases, and instances where patients fail to fill prescriptions are not captured [7–11]. Given these limitations, it is perhaps not surprising that diagnosis-based indices often perform better than prescription-based measures in comparative studies [4, 7, 12, 13]. However, using prescription data may be preferred due to data availability or the study setting.

Figure 1: Comorbidity measures that use coded data. Comorbidity measures that use coded data can be broadly categorised based on their reliance on diagnostic codes, medications, or both. Measures that use diagnostic codes may consist of case definitions or utilise groupings inherent to the International Classification of Diseases (ICD) coding structure.

A research team may select a comorbidity index after using existing literature to identify the most relevant medical conditions associated with the exposure-outcome relationship. However, the choice of comorbidity index should not be restricted to the comorbidities included in the study. Any additional diseases relevant to the research question should be coded as indicator variables and carefully reported.

Charlson comorbidity index

The Charlson Comorbidity Index (CCI) was initially developed to predict mortality using 19 conditions identified from a chart review of a sample of 559 medical patients admitted to a single hospital over one month in 1984 [2]. Each patient’s comorbidity score was computed by summing the empirically-derived weights associated with the conditions. The CCI was validated on a sample of 685 patients treated for primary breast cancer between 1962 and 1969 and used to predict non-cancer death within ten years of the commencement of anti-neoplastic therapy. Limitations of this study include the small sample size, the date of data collection, and the challenge in attributing cause of death in the validation study. Nonetheless, the CCI has been cited over 45,000 times and has displayed excellent discrimination across diverse cohorts [12, 14–18].

Three adaptations of the CCI to the International Classification of Diseases, version 9 (ICD-9) were created to facilitate its application to administrative health data [19–21]. Although disease prevalence differed based on the definition used, comorbidity index performance was determined to be essentially equivalent [4], [22]. Similar findings were observed when three ICD-10 adaptations were compared, suggesting robustness to disease definition [18]. The most commonly used disease definitions are those introduced by Deyo and Quan for ICD-9 and ICD-10, respectively [4]. In 2011, updated weights were proposed to reflect advancements in standards of care, evolution in understanding of disease burden, and novel therapies; however, the original CCI weights continue to be widely used [16].

Elixhauser comorbidity measure

Elixhauser’s comorbidity measure was constructed using ICD-9-CM coded administrative data from nonmaternal adult admissions across 439 acute care hospitals in 1992 [3]. The outcomes of interest included hospital charges, length of stay, and in-hospital death. Subgroup analyses were conducted on ten disease-specific cohorts to test model performance across acute, chronic, surgical, and nonsurgical conditions. Thirty comorbidities were identified as being associated with the outcomes of interest during the variable selection process. These disease groups did not have pre-assigned weights but were intended to be included as individual binary covariates in risk adjustment models, and researchers were encouraged to exclude comorbidities deemed irrelevant to their research question. Compared to the CCI, Elixhauser’s approach is more comprehensive in terms of the medical conditions, study populations, and outcomes considered. However, adding 30 variables to a model may cause collinearity and instability. van Walraven and colleagues have introduced a weighted scoring system that uses the Elixhauser co-morbidities and was derived with in-hospital death as the outcome [23].

ICD-based comorbidity measures

ICD is used internationally to record and track diseases, health conditions, and other health problems. Some researchers have attempted to comprehensively capture patient complexity by utilising the ontology of ICD codes. Comorbidity measures such as the Multi-Morbidity (MM) Index consider all ICD-9 diagnoses, whereas the Multipurpose Australian Comorbidity Scoring System (MACSS) consists of the 100 most frequently used codes [24–27]. The Multidimensional Multiple Morbidity (MMM) Index groups codes by body system and standardises severity scores to range from zero to one within each group [24] The highest relevant score is selected from within each body system and used in the model for each patient [24]. Compared to ICD-9, ICD-10 offers a more complete and precise list of diagnoses, making it more feasible to consider blocks of codes [24], [28, 29].

There are important considerations that must be recognised with regard to the utility and performance of indices that rely on coded diagnoses. An ICD structure-based approach supports a more nuanced capture of complexity by including more codes. However, it may lead to the inclusion of broad, nonhomogeneous disease groups wherein the differential impacts of the conditions are obscured [24], [26], [30]. In contrast, itemising too many comorbidities may deter uptake and can yield unstable disease weights due to collinearity and overfitting [14], [17], [31]. The inclusion of rare diseases may also result in model instability [16]. Thus, the challenge is in selecting and implementing a variable reduction approach that will identify comorbidities that are prevalent or have a significant impact on health outcomes. Unfortunately, there is no consistent technique that is used in the literature. For instance, prior studies have used minimum prevalence thresholds ranging from 0.02 to 2%, but the choice has been arbitrary [26, 27, 29, 30, 32, 33]. Leveraging machine learning techniques with clinical expertise may advance research in this domain.

Other diagnosis-based measures

There are many other ways to adjust for comorbidity. Some researchers have combined the Charlson and Elixhauser indices into a single comorbidity index [17, 34–37]. Others have supplemented the existing indices with clinical knowledge and created disease-specific measures for medical conditions such as cerebral palsy [38], cardiovascular disease [31, 39–42], and cancer [43]. Comorbidity measures such as the Global Risk-Adjustment Model (GRAM), Adjusted Clinical Groups (ACG) System, Diagnostic Cost Groups (DCG), and the Centers for Medicare and Medicaid Services Hierarchical Condition Categories (CMS-HCC) were developed for remuneration purposes and combine demographic and diagnostic information to create clinically homogenous groups [44, 45].

The chronic disease score and RxRisk

Another way to characterise the burden of comorbid disease is by using the medications an individual receives. The Chronic Disease Score (CDS), RxRisk, and others include medications dispensed from outpatient settings. The CDS was constructed using a consensus-based approach to select 17 chronic conditions, identify relevant medication classes, and develop a weighting scheme [46]. An adult patient is considered to have one of the chronic diseases if dispensed any drug in the corresponding medication class(es) over a one-year period. This approach was evaluated on the basis of year-to-year stability and the strength of association with physician rating of disease severity, patient-reported health status, mortality, and hospitalisation [46]. The revised CDS, or CDS-2, includes 28 chronic and mental health conditions, offers a more inclusive list of medications, and assigns empirically derived weights to medication classes based on healthcare utilisation and costs [47].

The RxRisk is an expansion of the CDS-2 designed for both pediatric and adult populations [9]. The component medical conditions and associated drug classes differ based on the patient’s age [9]. In contrast, the RxRisk-V was derived to predict cost using data from the Veterans Health Administration (VHA) and includes 45 disease categories of research or clinical priority [11]. Pratt and team mapped the RxRisk-V to the Anatomical Therapeutic Chemical (ATC) Classification System and demonstrated moderate to high accuracy in predicting one-year mortality among older adults in Australia [48]. The RxRisk-V was expanded by Johnson and colleagues to include 26 additional disease categories and used to predict mortality [49].

Other prescription-based measures

Two additional pharmacy-based comorbidity measures are worth noting. The Medication-based Disease Burden Index (MDBI) comprises 20 chronic conditions that are leading contributors to global death and managed pharmacologically [50]. In contrast, the medication-based Chronic Disease Score (medCDS) captures the most prevalent chronic conditions among elderly patients in Germany and predicts death using medication classes derived from treatment guidelines [51]. Some researchers may feel the international scope of the MDBI makes it an appealing option, whereas others may appreciate that the medCDS uses treatment guidelines to select medications.

Combining prescription and diagnosis data

Some contemporary measures of comorbidity incorporate both diagnostic and pharmaceutical data. Examples include the Nordic Multimorbidity Index [52]and the Cambridge Multimorbidity Score [53, 54]. The Nordic Multimorbidity Index was constructed using 50 covariates to predict 5-year mortality [52]. In contrast, the Cambridge Multimorbidity Score was derived using 37 long-term conditions to predict primary care consultations, unplanned hospital admissions, and death [53]. Data regarding diagnosis and dispensed medications have also been used to assess associations between broad categories of mental disorders and medical conditions [55].

Considerations when using comorbidity indices in research

Comorbidity indices are widely used despite variation in discrimination based on the cohort and the area of application (e.g., length of stay, mortality, readmission, etc.) [12, 25]. Differences in performance within a specific setting are often modest, and we would not suggest there is a particular comorbidity measure that uniformly outperforms other measures. For example, poor performance is consistently noted when predicting readmission [33, 56]. Likewise, there is no index that is consistently preferred when considering either short- or long-term health events [57, 58]. Variation in discrimination based on outcome may result from discordant effects of comorbidity or signify the absence of medical conditions that are relevant to the outcome [59]. Failure to adjust for the competing risk of death in the analysis may also introduce bias. For these reasons, the recommendations below are considered setting agnostic. Furthermore, rather than evaluating the magnitude of the metric reported, the focus is on statistically significant differences based on variation in methodology.

Figure 2 summarises the considerations for implementation of comorbidity indices that are discussed herein: 1) the look-back window used for disease ascertainment; 2) whether to link additional data sources; 3) how to represent comorbid conditions; 4) temporality of diagnoses; and 5) inclusion of paradoxical conditions. Although we present these as decisions to be made after selecting a comorbidity index, careful thought may ultimately impact the choice of index, making this process somewhat iterative.

Figure 2: Considerations when implementing a comorbidity index. When implementing a comorbidity index, it is important to reflect on which data sources to use, the length of the look-back period, the format in which comorbidities are coded, the timing of diagnoses, and the handling of protective conditions.

Look-back period

The look-back period is the interval measured retrospectively from the index hospital visit used for disease ascertainment. Due to limited data availability, early comorbidity indices did not use a look-back period but were developed using a single hospital record. Contemporary researchers can readily follow patients across time; however, there is debate concerning the necessity of implementing a look-back period and the optimal time frame. Intuitively, looking retrospectively should allow researchers to correct for incomplete recording of comorbidities across visits and improve model performance [37]. However, there is also an increased risk of capturing conditions that may be misdiagnosed, in remission, or resolved [19, 37]. Furthermore, when considering the index hospital stay exclusively, patients are risk-stratified based on medical conditions that necessitate active monitoring and impact their treatment [60]. Focusing on these diagnoses may be more valuable than a comprehensive patient’s disease profile assessment.

Ultimately, the decision of whether or not to use a look-back period may be moot. Studies that used a look-back period noted increased disease prevalence but no meaningful improvement in discrimination for short-term outcomes such as in-hospital mortality, inpatient costs, length of stay, 30-day mortality, and 30-day readmission [14, 56, 58, 61, 62]. Some studies have observed a slight improvement from using a one-year look-back period when predicting one-year mortality; however, this finding was inconsistent [14, 56, 58, 61]. Increasing the look-back period beyond one year in studies that assessed mortality and resource use demonstrated minimal benefit [61, 63].

Negligible or inconsistent improvement in performance from an increase in data may seem surprising. However, the index visit is expected to capture the diagnoses that impact a patient’s immediate health status and, ergo, short-term outcomes [14, 19]. Holman argues that a one-year look-back can act as a compromise in identifying active comorbidities; however, the current literature suggests only studies involving long-term outcomes (i.e., ≥one year) will benefit [25, 28] In summary, the absence of strong evidence favoring a look-back period means the index hospital visit will likely provide sufficient information for outcomes within one year. A one-year look-back window is recommended when considering outcomes beyond one year.

Further work is recommended to examine whether the look-back period should vary based on the disease/comorbidity of interest. For example, Tonelli and colleagues suggest disease-specific look-back windows and algorithms that change based on the database used [64]. Another important area for future research is exploring how disease severity and the competing risk of death change as the look-back period increases; this is expected to vary based on the comorbidity, the context, and the setting of the study.

Data linkage

Linking diagnostic data from hospital billing records with practitioner claims from primary care, specialist, and ambulatory encounters is expected to increase disease ascertainment and yield a more complete and accurate description of a patient’s comorbidities. However, compared to inpatient settings, disease is expected to be captured earlier and at a less severe stage in an outpatient environment, suggesting a weaker strength of association with health outcomes of interest [65]. Furthermore, there is also an increased risk of misclassifying rule-out diagnoses as comorbidities [12, 19].

Prior research has shown an increase in the prevalence of both acute and chronic conditions when using linked inpatient and outpatient data [28, 66]. Discrimination improved when physician claims were used to augment hospital records in a study that examined one-year hospital costs; however, results were mixed when one-year mortality was considered [28, 66, 67]. Linkage between hospital and pharmaceutical data has also been studied. Dispensed medications have been demonstrated to identify unrecorded chronic conditions with no statistically significant benefit in discrimination [7, 11].

It is important to consider why a diagnosis may be missing from a hospital record and to recognise that this absence may reflect reduced disease severity. Likewise, the environment where the initial diagnosis is made may have important implications. The certainty and severity of a diagnosis made in a hospital or primary care setting are expected to vary due to accessibility, technological infrastructure, laboratory facilities, available resources, and disparate workflows. Unfortunately, quantifying severity using current coding systems is incredibly challenging, and additional research is needed to confirm this hypothesis. There is also an absence of work that explores the impact of data linkage on outcomes within one year, and further investigation in this area is warranted. Nonetheless, when exploring long-term events, there may be a benefit in supplementing hospital records with one year of data from other settings to allow researchers to identify diagnoses associated with health deterioration over time.

Using a single disease score or a set of conditions

There are two core strategies for capturing comorbidity in a risk adjustment model. A single score can be used to summarise the impact of concurrent conditions, or an independent variable can be assigned to each condition. The CCI is an example of the single score method, wherein pre-established weights assigned to 19 conditions are summed to estimate the cumulative effect of comorbidities. This approach is easy to apply and interpret: a higher comorbidity score implies increased complexity. As a weighted score, a comorbidity index allows us to consider the role of comorbidity as an exposure or risk stratify patients into groups based on disease severity. Furthermore, since a single variable is used to represent comorbidity, nonlinear associations and interactions with sociodemographic factors, such as age and sex, are easy to model [34, 39, 68]. However, patients with the same score are not necessarily medically similar and heterogeneity in the disease profile of patients may mask the impact of specific comorbidities. The weighted score method is expected to perform best when applied to samples wherein the distribution of comorbidities and their prognostic impact are comparable to the sample from which the disease weights were derived. Yet, application is observed across various age groups, disease subpopulations, outcome measures, health care settings, and time [15, 43, 45, 63]. Differences in discrimination have been noted based on the cohort (e.g., patients with congestive heart failure, diabetes, or chronic renal failure [69]), outcome (e.g., 1-year mortality or hospitalisation [63]), and country of dataset origin (e.g., Canada, Japan, or Switzerland [18]). On a more granular level, Gagne and colleagues also noted differences in discrimination based on the measure of mortality (i.e., 30-day, 90-day, 180-day and 1-year mortality [34]). It is important to routinely revise disease weights to reflect changes in the standards of care and the availability of novel pharmaceutical therapies [17, 34]. Thus, from a recency standpoint, using the revised Charlson weights proposed by Quan and colleagues in 2011 is preferable to using the original weights proposed in 1987 [16].

An alternative approach to the above is to build an additive model that includes one covariate for each comorbidity of interest. A common choice is to model the medical conditions identified by Elixhauser and colleagues [3]. When applying this method, the coefficient associated with each disease will fluctuate based on the data. This approach is expected to improve model performance [12, 18, 69]. However, the increased number of covariates may lead to overfitting and collinearity, particularly when considering small samples, rare diseases, and infrequent outcomes [13, 14, 17, 18, 35]. For instance, although HIV/AIDS is among the comorbidities proposed by Elixhauser, it may be reasonable to omit this disease and ensure the stability of the model, particularly if the condition is rarely observed in the sample and is not directly relevant to the research question.

Comorbidity indices aim to quantify patients’ overall health and group individuals with similar levels of risk. For this purpose, using a single covariate appropriate to the health care setting and study timeframe is recommended over using many disease-specific indicators [17]. A scoring system obviates the need to identify and exclude rare diseases that jeopardise the stability of the model. Furthermore, only a single regression coefficient needs to be considered, not several disease-specific associations. The most compelling argument against using a single score is the increased flexibility and, therefore, improved performance that is expected from using one variable for each respective comorbidity. However, a literature review revealed that models using individual variables do not consistently outperform those using a weighted sum [16–18, 23, 31, 39, 45, 62, 68, 70]. Suppose a research team is interested in specific risk factors or concordant comorbidities. In that case, these can be excluded from the comorbidity index and modelled as independent covariates alongside the modified weighted sum [71, 72]. Nonetheless, using binary variables may be preferred if it is better aligned with the study aims and design and if there is sufficient power based on sample size [73].

Temporality

When a comorbidity index is used in a risk adjustment model, it is essential to consider the study index date. When forecasting outcomes at the time of admission, incidents that arise after the patient is admitted must be excluded since they represent future events relative to cohort time zero. In contrast, events that transpired during the hospital stay would be expected to influence the patient’s baseline state at the time of discharge, although the research question may still require differentiation between comorbidities and complications of care [30, 56]. Likewise, it is important to consider immortal time bias especially when study index date and exposure initiation are not simultaneous. The artificial creation of a time period where health events cannot occur (i.e., immortal time) would result in biased results due to an overestimation of time at risk [74]. Further, in the context of a comorbidity index, it is also important to consider how changes in disease status during the follow-up period may bias results [75]. Establishing the temporality of a diagnosis can be challenging when using administrative data [62]. Therefore, it is vital to consider the implications of misclassification and proposed preventative measures.

Misclassifying in-hospital complications as comorbidities when using time of admission as the index date is expected to overestimate model performance and disease prevalence [33, 76–78]. Southern et al. compared models that used the diagnosis type indicator to identify and exclude complications to those that did not [78]. Consistent overlap in the 95% confidence intervals of the odds ratios, deviance, and discrimination was observed [78]. Likewise, Roos et al. considered three surgical procedures and implemented two variations of the CCI but only observed one instance where failure to exclude complications resulted in “spuriously high discrimination”, evidenced by a relative change of 7% [79]. However, the proportion of patients with at least one complication and the proportion of diagnoses which denote complications is known to vary based on the cohort and the condition studied [76, 78, 79]. Thus, the repercussions may be more significant if this error is common or if outcomes differ substantially among patients when the diagnosis is acquired during the hospital stay [79].

It is difficult to distinguish between complications and acute conditions, especially when the cohort includes a diverse group of patients [4]. As Jean and colleagues explain, even though it is common for patients to experience adverse events in the hospital, specific complications are rare, and instances that result in significant disability and impairment occur with an even lower frequency [80]. Various solutions have been proposed. Clinical knowledge can be used to identify and exclude common complications of care [62]. Alternatively, evidence of acute events may be extracted from prior visits but discredited if they are only captured during the index admission [19, 21]. In Canada, a diagnosis-type indicator is used to differentiate between diagnoses made pre- and post-admission [78]. More widespread recording of this data element is the most foolproof approach to establish temporality relative to the index date.

Paradoxical conditions

A chronic condition is expected to increase the risk of poor health outcomes. However, a statistical model may indicate that certain comorbidities are protective factors. Some weighting schemes exclude these “paradoxical conditions” (i.e., assign a weight of 0) while others assign a negative weight [17]. For instance, the algorithms used to derive disease weights for both the original and updated CCI exclude negative values [2, 16]. These authors may have felt that content validity dictates that a disease should increase patient complexity and the risk of poor outcomes [42, 59, 81, 82]. In contrast, van Walraven and colleagues allow the Elixhauser comorbidities to be assigned a value less than zero [23]. Some authors postulate that paradoxical conditions may represent real clinical phenomena that are not yet understood or indicate unmeasured differences in patients [35, 62, 83]. For example, patients with paradoxical conditions may be monitored more closely, receive treatment at an earlier stage of disease, have longer hospital stays, seek follow-up care more consistently, or adhere to drug therapies that yield the perceived protective association [13, 25, 33, 40, 66, 83]. An alternative explanation is bias from incomplete coding. More precisely, the documentation of paradoxical conditions may signify a healthier individual who lacks serious comorbidities or complications [23, 34, 35, 62, 72, 81, 84, 85]. Vaughan-Sarrazin and colleagues studied the prevalence of paradoxical conditions and their impact on risk adjustment [85]. The group concluded that contributing mechanisms may vary after observing inconsistency in paradoxical conditions among patients with acute myocardial infarction (AMI) and those who underwent coronary artery bypass graft (CABG) surgery [85]. Within each cohort, reporting of paradoxical conditions increased over time, but the direction of the association remained consistent [85]. Some studies have shown that model performance is enhanced by including paradoxical conditions [83, 85], while others have identified no benefit [63].

It is essential to acknowledge that a comorbidity score of zero indicates the patient may not have the conditions included in the comorbidity index, but does not imply they are immune to adverse outcomes. As such, an individual with a negative comorbidity score should be interpreted as one with a comparatively lower but still non-zero risk. Since the concept that specific comorbidities may be “protective” is founded in data and accepted in statistical modelling, excluding negative weights in comorbidity scoring systems may not be appropriate. However, studies that explore paradoxical conditions have focused on short-term outcomes. Further research examining whether these protective relationships persist for different outcomes and time periods beyond 30 days is warranted. Based on current evidence, retention of paradoxical conditions is recommended but only for short-term (<30 days) outcomes.

Conclusions

Comorbidity indices allow researchers to adjust for differences in the baseline complexity of patients in a risk adjustment model. Comparative studies suggest that diagnosis-based indices often perform better than prescription-based measures [4, 7, 12, 13]. However, within these two groups, no single choice consistently excels above the rest. The use of a comorbidity index that combines diagnostic and prescription fill data, such as the Nordic Multimorbidity Index, is also a compelling option [52].

Implementing a comorbidity measure as a weighted sum makes it easier to interpret complexity with little to no decline in discrimination. If specific comorbidities are central to the study, it may be appropriate to exclude these from the weighted sum and code them as covariates alongside the modified risk score. However, the increased flexibility from coding a large set of binary variables is worth considering in studies with large samples or when the cohort differs from the sample from which the weights were derived. Health events that arise after the study index date should be excluded from the model to respect temporality. For studies focusing on outcomes beyond one year, there may be some benefit from using data from a one-year look-back window or linking with additional data sources to adjust for incomplete capture of secondary diagnoses. For short-term outcomes, it is important to acknowledge that certain diagnoses may be associated with a “paradoxical” reduced risk. Choosing the most appropriate comorbidity index for the research question while keeping these considerations in mind will improve interpretability, consistency, and overall quality in future epidemiologic studies.

It is important to consider the consequences of not using comorbidity indices appropriately. Application of these indices beyond their intended purpose may reduce analytic precision and fail to accurately adjust for the underlying health status of patients. This may introduce bias from misclassification and inaccurate risk stratification. If differences between healthier patients and those with more complex health needs exist but are distorted, there may be an over- or underestimation of the effect size, which could jeopardise generalisability. Another potential concern is residual confounding, wherein biased estimates of the association between the exposure and outcome are obtained. Incorrect attribution of differences to the exposure rather than patient complexity may lead to inaccurate conclusions and compromise the study’s validity. This is particularly concerning for observational studies wherein patients are not randomised to the exposure of interest. Finally, inconsistent use of comorbidity indices, including variation in coding practices, may reduce comparability across studies and increase between-study heterogeneity in subsequent meta-analyses.

Acknowledgements

This study was supported by funding from the Izaak Walton Killam Doctoral Scholarship, Alberta Innovates Graduate Student Scholarship, Alberta Graduate Excellence (Doctoral) Scholarship, and Queen Elizabeth II Graduate (Doctoral) Scholarship.

Statement on conflicts of interest

The authors declare no conflicts of interest.

Ethics statement

Ethics approval was not sought for this study, which relied exclusively on publicly available information.

Data availability statement

This paper uses information that has been collected and analysed in prior research studies, rather than individual patient data. Therefore, there are no data to be made available.

Abbreviations

ACG	Adjusted Clinical Group
AMI	Acute Myocardial Infarction
ATC	Anatomical Therapeutic Chemical Classification System
CABG	Coronary Artery Bypass Graft
CCI	Charlson Comorbidity Index
CDS	Chronic Disease Score
CMS-HCC	Centers for Medicare and Medicaid Services Hierarchical Condition Categories
DCG	Diagnostic Cost Groups
GRAM	Global Risk-Adjustment Model
HIV/AIDS	Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome
ICD	International Classification of Disease
MACSS	Multipurpose Australian Comorbidity Scoring System
MDBI	Medication-based Disease Burden Index
medCDS	Medication-based Chronic Disease Score
MM	Multi-Morbidity Index
MMM	Multidimensional Multiple Morbidity Index
VHA	Veterans Health Administration

References

1. Lucyk K, Tang K, Quan H. Barriers to data quality resulting from the process of coding health information to administrative data: A qualitative study. BMC Health Serv Res. 2017;17(1). 10.1186/s12913-017-2697-y

   10.1186/s12913-017-2697-y

2. Charlson ME, Pompei P, ALes KL, MacKenzie CR. A New Method of Classifying Prognostic Comorbidity in Longitudinal Studies : Development and Validation. J Chronic Dis. 1987;40(5):373–83. 10.1016/0021-9681(87)90171-8

   10.1016/0021-9681(87)90171-8

3. Elixhauser A, Steiner C, Harris DR. Comorbidity Measures for Use with Administrative Data. Med Care. 1998;36(1):8–27. 10.1097/00005650-199801000-00004

   10.1097/00005650-199801000-00004

4. Yurkovich M, Avina-Zubieta JA, Thomas J, Gorenchtein M, Lacaille D. A systematic review identifies valid comorbidity indices derived from administrative health data. J Clin Epidemiol. 2015;68(1):3–14. 10.1016/j.jclinepi.2014.09.010

   10.1016/j.jclinepi.2014.09.010

5. Tang J, Wan JY, Bailey JE. Performance of comorbidity measures to predict stroke and death in a community-dwelling, hypertensive medicaid population. Stroke. 2008;39(7):1938–44. 10.1161/STROKEAHA.107.504688

   10.1161/STROKEAHA.107.504688

6. Sharabiani M, Aylin P, Bottle A. Systematic review of comorbidity indices for administrative data. Med Care. 2012;50(12):1109–18. 10.1097/MLR.0b013e31825f64d0

   10.1097/MLR.0b013e31825f64d0

7. Bang JH, Hwang SH, Lee EJ, Kim Y. The predictability of claim-data-based comorbidity-adjusted models could be improved by using medication data. BMC Med Inform Decis Mak. 2013;13(1). 10.1186/1472-6947-13-128

   10.1186/1472-6947-13-128

8. Dominick KL, Dudley TK, Coffman CJ, Bosworth HB. Comparison of three comorbidity measures for predicting health service use in patients with osteoarthritis. Arthritis Care Res. 2005;53(5):666–72. 10.1002/art.21440

   10.1002/art.21440

9. Fishman PA, Goodman MJ, Hornbrook MC, Meenan RT, Bachman DJ, O’Keeffe Rosetti MC. Risk adjustment using automated ambulatory pharmacy data the RxRisk model. Med Care. 2003;41(1):84–99. 10.1097/00005650-200301000-00011

   10.1097/00005650-200301000-00011

10. Johnson RE, Hornbrook MC, Nichols GA. Replicating the chronic disease score (CDS) from automated pharmacy data. J Clin Epidemiol. 1994;47(10):1191–9. 10.1016/0895-4356(94)90106-6

    10.1016/0895-4356(94)90106-6

11. Sloan KL, Sales AE, Liu CF, Fishman P, Nichol P, Suzuki NT, et al. Construction and characteristics of the RxRisk-V A VA-adapted pharmacy-based case-mix instrument. Med Care. 2003;41(6):761–74. 10.1097/00005650-200306000-00009

    10.1097/00005650-200306000-00009

12. Lix LM, Quail J, Teare G, Acan B. Performance of comorbidity measures for predicting outcomes in population-based osteoporosis cohorts. Osteoporos Int. 2011;22(10):2633–43. 10.1007/s00198-010-1516-7

    10.1007/s00198-010-1516-7

13. Schneeweiss S, Wang PS, Avorn J, Glynn RJ. Improved comorbidity adjustment for predicting mortality in Medicare populations. Health Serv Res. 2003;38(4):1103–20. 10.1111/1475-6773.00165

    10.1111/1475-6773.00165

14. Dobbins TA, Creighton N, Currow DC, Young JM. Look back for the Charlson index did not improve risk adjustment of cancer surgical outcomes. J Clin Epidemiol. 2015;68(4):379–86. 10.1016/j.jclinepi.2014.12.002

    10.1016/j.jclinepi.2014.12.002

15. Lix LM, Quail J, Fadahunsi O, Teare GF. Predictive performance of comorbidity measures in administrative databases for diabetes cohorts. BMC Health Serv Res. 2013;13(1):1. 10.1186/1472-6963-13-340

    10.1186/1472-6963-13-340

16. Quan H, Li B, Couris CM, Fushimi K, Graham P, Hider P, et al. Updating and validating the charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011;173(6):676–82. 10.1093/aje/kwq433

    10.1093/aje/kwq433

17. Simard M, Sirois C, Candas B. Validation of the Combined Comorbidity Index of Charlson and Elixhauser to Predict 30-Day Mortality Across ICD-9 and ICD-10. Med Care. 2018;56(5):441–7. 10.1097/MLR.0000000000000905

    10.1097/MLR.0000000000000905

18. Sundararajan V, Quan H, Halfan P, Fushimi K, Luthi JC, Burnand B, et al. Cross-national comparative performance of three versions of the ICD-10 charlson index. Med Care. 2007;45(12):1210–5. 10.1097/MLR.0b013e3181484347

    10.1097/MLR.0b013e3181484347

19. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45(6):613–9. 10.1016/0895-4356(92)90133-8

    10.1016/0895-4356(92)90133-8

20. D’Hoore W, Bouckaert A, Tilquin C. Practical considerations on the use of the Charlson comorbidity index with administrative data bases. J Clin Epidemiol. 1996;49(12):1429–33. 10.1016/S0895-4356(96)00271-5

    10.1016/S0895-4356(96)00271-5

21. Romano PS, Roos LL, Jollis JG. Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives. J Clin Epidemiol. 1993;46(10):1075–9. 10.1016/0895-4356(93)90103-8

    10.1016/0895-4356(93)90103-8

22. Needham DM, Scales DC, Laupacis A, Pronovost PJ. A systematic review of the Charlson comorbidity index using Canadian administrative databases: A perspective on risk adjustment in critical care research. J Crit Care. 2005;20(1):12–9. 10.1016/j.jcrc.2004.09.007

    10.1016/j.jcrc.2004.09.007

23. Van Walraven C, Austin PC, Jennings A, Quan H, Forster AJ. A modification of the elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care. 2009;47(6):626–33. 10.1097/MLR.0b013e31819432e5

    10.1097/MLR.0b013e31819432e5

24. Alemi F, Levy CR, Kheirbek RE. The Multi-Morbidity Index: A Tool for Assessing the Prognosis of Patients from History of Illness. eGEMs. 2016;4(1):19. 10.13063/2327-9214.1235

    10.13063/2327-9214.1235

25. Holman CDAJ, Preen DB, Baynham NJ, Finn JC, Semmens JB. A multipurpose comorbidity scoring system performed better than the Charlson index. J Clin Epidemiol. 2005;58(10):1006–14. 10.1016/j.jclinepi.2005.01.020

    10.1016/j.jclinepi.2005.01.020

26. Kheirbek RE, Alemi F, Fletcher R. Heart Failure Prognosis: Comorbidities Matter. J Palliat Med. 2015;18(5):447–52. 10.1089/jpm.2014.0365

    10.1089/jpm.2014.0365

27. Riviere P, Tokeshi C, Hou J, Nalawade V, Sarkar R, Paravati AJ, et al. Claims-Based Approach to Predict Cause-Specific Survival in Men With Prostate Cancer. JCO Clin Cancer Informatics. 2019;(3):1–7. 10.1200/cci.18.00111

    10.1200/cci.18.00111

28. Crooks CJ, Card TR, West J. The use of a Bayesian hierarchy to develop and validate a co-morbidity score to predict mortality for linked primary and secondary care data from the NHS in England. PLoS One. 2016;11(10):1–16. 10.1371/journal.pone.0165507

    10.1371/journal.pone.0165507

29. Stausberg J, Hagn S. New morbidity and comorbidity scores based on the structure of the ICD-10. PLoS One. 2015;10(12):1–11. 10.1371/journal.pone.0143365

    10.1371/journal.pone.0143365

30. Robbins GT, Goldstein R, Siddiqui S, Huang DS, Zafonte R, Schneider JC. Capture rates of comorbidity measures at inpatient rehabilitation facilities after a stroke or brain injury. PM R. 2022;14(4):462–71. 10.1002/pmrj.12589

    10.1002/pmrj.12589

31. Fernando DT, Berecki-Gisolf J, Newstead S, Ansari Z. The Australian Injury Comorbidity Index to Predict Mortality. Ann Emerg Med. 2020;75(3):339–53. 10.1016/j.annemergmed.2019.10.003

    10.1016/j.annemergmed.2019.10.003

32. Hyer JM, Ejaz A, Tsilimigras DI, Paredes AZ, Mehta R, Pawlik TM. Novel Machine Learning Approach to Identify Preoperative Risk Factors Associated with Super-Utilization of Medicare Expenditure Following Surgery. JAMA Surg. 2019;154(11):1014–21. 10.1001/jamasurg.2019.2979

    10.1001/jamasurg.2019.2979

33. Toson B, Harvey LA, Close JCT. New ICD-10 version of the Multipurpose Australian Comorbidity Scoring System outperformed Charlson and Elixhauser comorbidities in an older population. J Clin Epidemiol. 2016;79:62–9. 10.1016/j.jclinepi.2016.04.004

    10.1016/j.jclinepi.2016.04.004

34. Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64(7):749–59. 10.1016/j.jclinepi.2010.10.004

    10.1016/j.jclinepi.2010.10.004

35. Menendez ME, Neuhaus V, Ring D. Inpatient mortality after orthopaedic surgery. Int Orthop. 2015;39(7):1307–14. 10.1007/s00264-015-2702-1

    10.1007/s00264-015-2702-1

36. Pritchard E, Fawcett N, Quan TP, Crook D, Peto TE, Walker AS. Combining Charlson and Elixhauser scores with varying lookback predicated mortality better than using individual scores. J Clin Epidemiol. 2021;130:32–41. 10.1016/j.jclinepi.2020.09.020

    10.1016/j.jclinepi.2020.09.020

37. Sun JW, Rogers JR, Her Q, Welch EC, Panozzo CA, Toh S, et al. Adaptation and Validation of the Combined Comorbidity Score for ICD-10-CM. Med Care. 2017;55(12):1046–51. 10.1097/MLR.0000000000000824

    10.1097/MLR.0000000000000824

38. Whitney DG, Kamdar NS. Development of a new comorbidity index for adults with cerebral palsy and comparative assessment with common comorbidity indices. Dev Med Child Neurol. 2021;63(3):313–9. 10.1111/dmcn.14759

    10.1111/dmcn.14759

39. Fernando DT, Berecki-Gisolf J, Newstead S, Ansari Z. Australian Injury Comorbidity Indices (AICIs) to predict burden and readmission among hospital-admitted injury patients. BMC Health Serv Res. 2021;21(1):1–21. 10.1186/s12913-021-06149-1

    10.1186/s12913-021-06149-1

40. Newman WC, Neal DW, Hoh BL. A new comorbidities index for risk stratification for treatment of unruptured cerebral aneurysms. J Neurosurg. 2016;125(3):713–9. 10.3171/2015.8.JNS14553

    10.3171/2015.8.JNS14553

41. Newman WC, Kubilis PS, Hoh BL. Validation of a neurovascular comorbidities index for retrospective database analysis. J Neurosurg. 2019;130(1):273–7. 10.3171/2017.8.JNS171413

    10.3171/2017.8.JNS171413

42. Polanczyk CA, Rohde LE, Philbin EA, Di Salvo TG. A New Casemix Adjustment Index for Hospital Mortality among Patients with Congestive Heart Failure. Med Care. 1998;36(10):1489–99. 10.1097/00005650-199810000-00007

    10.1097/00005650-199810000-00007

43. Pule L, Buckley E, Niyonsenga T, Roder D. Optimizing the measurement of comorbidity for a South Australian colorectal cancer population using administrative data. J Eval Clin Pract. 2020;26(4):1250–8. 10.1111/jep.13305

    10.1111/jep.13305

44. Meenan RT, Goodman MJ, Fishman PA, Hornbrook MC, O’Keeffe-Rosetti MC, Bachman DJ. Using risk-adjustment models to identify high-cost risks. Med Care. 2003;41(11):1301–12. 10.1097/01.MLR.0000094480.13057.75

    10.1097/01.MLR.0000094480.13057.75

45. Mehta HB, Yong S, Sura SD, Hughes BD, Kuo YF, Williams SB, et al. Development of comorbidity score for patients undergoing major surgery. Health Serv Res. 2019;54(6):1223–32. 10.1111/1475-6773.13209

    10.1111/1475-6773.13209

46. Von Korff M, Wagner EH, Saunders K. A chronic disease score from automated pharmacy data. J Clin Epidemiol. 1992;45(2):197–203. 10.1016/0895-4356(92)90016-G

    10.1016/0895-4356(92)90016-G

47. Clark DO, Von Korff M, Saunders K, Baluch WM, Simon GE. A chronic disease score with empirically derived weights. Medical Care. 1995;33(8):783-95. 10.1097/00005650-199508000-00004

    10.1097/00005650-199508000-00004

48. Pratt NL, Kerr M, Barratt JD, Kemp-Casey A, Kalisch Ellett LM, Ramsay E, et al. The validity of the Rx-Risk Comorbidity Index using medicines mapped to the Anatomical Therapeutic Chemical (ATC) Classification System. BMJ Open. 2018;8(4):1–8. 10.1136/bmjopen-2017-021122

    10.1136/bmjopen-2017-021122

49. Johnson ML, El-Serag HB, Tran TT, Hartman C, Richardson P, Abraham NS. Adapting the Rx-Risk-V for mortality prediction in outpatient populations. Med Care. 2006;44(8):793–7. 10.1097/01.mlr.0000218804.41758.ef

    10.1097/01.mlr.0000218804.41758.ef

50. George J, Vuong T, Bailey MJ, Kong DC, Marriott JL, Steward K. Development and validation of the medication-based disease burden index. AnnPharmacother. 2006;40(4):645–50. 10.1345/aph.1G204

    10.1345/aph.1G204

51. Quinzler R, Freitag MH, Wiese B, Beyer M, Brenner H, Dahlhaus A, et al. A novel superior medication-based chronic disease score predicted all-cause mortality in independent geriatric cohorts. J Clin Epidemiol. 2019;105:112–24. 10.1016/j.jclinepi.2018.09.004

    10.1016/j.jclinepi.2018.09.004

52. Kristensen KB, Lund LC, Jensen PB, Broe A, Rotbain E, Damkier P, et al. Development and Validation of a Nordic Multimorbidity Index Based on Hospital Diagnoses and Filled Prescriptions. Clin Epidemiol. 2022;14:567–79. 10.2147/CLEP.S353398

    10.2147/CLEP.S353398

53. Payne RA, Mendonca SC, Elliott MN, Saunders CL, Edwards DA, Marshall M, et al. Development and validation of the Cambridge Multimorbidity Score. CMAJ. 2020;192(5):E107–14. 10.1503/cmaj.190757

    10.1503/cmaj.190757

54. Harrison H, Ip S, Renzi C, Li Y, Barclay M, Usher-Smith J, et al. Implementation and external validation of the Cambridge Multimorbidity Score in the UK Biobank cohort. BMC Med Res Methodol. 2024;24(1):1–11. 10.1186/s12874-024-02175-9

    10.1186/s12874-024-02175-9

55. Momen NC, Plana-Ripoll O, Agerbo E, Benros ME, Børglum AD, Christensen MK, et al. Association between Mental Disorders and Subsequent Medical Conditions. N Engl J Med. 2020;382(18):1721–31. 10.1056/nejmoa1915784

    10.1056/nejmoa1915784

56. Toson B, Harvey LA, Close JCT. The ICD-10 Charlson Comorbidity Index predicted mortality but not resource utilization following hip fracture. J Clin Epidemiol. 2015;68(1):44–51. 10.1016/j.jclinepi.2014.09.017

    10.1016/j.jclinepi.2014.09.017

57. Ladha KS, Zhao K, Quraishi SA, Kurth T, Eikermann M, Kaafarani HMA, et al. The Deyo-Charlson and Elixhauser-van Walraven Comorbidity Indices as predictors of mortality in critically ill patients. BMJ Open. 2015;5(9):1–8. 10.1136/bmjopen-2015-008990

    10.1136/bmjopen-2015-008990

58. Chu YT, Ng YY, Wu SC. Comparison of different comorbidity measures for use with administrative data in predicting short- and long-term mortality. BMC Health Serv Res. 2010;10. 10.1186/1472-6963-10-140

    10.1186/1472-6963-10-140

59. Waite K, Oddone E, Weinberger M, Samsa G, Foy M, Henderson W. Lack of association between patients’ measured burden of disease and risk for hospital readmission. J Clin Epidemiol. 1994;47(11):1229–36. 10.1016/0895-4356(94)90127-9

    10.1016/0895-4356(94)90127-9

60. Canadian Institute for Health Information. Canadian Coding Standards for Version 2022 ICD-10-CA and CCI. 2022. [Available at: https://secure.cihi.ca/free_products/COVID-19_Addendum_Pandemics_Epidemics-en.pdf, last accessed January 22, 2025]

61. Chaudhry S, Jin L, Meltzer D. Use of a self-report-generated Charlson comorbidity index for predicting mortality. Med Care. 2005;43(6):607–15. 10.1097/01.mlr.0000163658.65008.ec

    10.1097/01.mlr.0000163658.65008.ec

62. Johnston JA, Wagner DP, Timmons S, Welsh D, Tsevat J, Render ML. Impact of different measures of comorbid disease on predicted mortality of intensive care unit patients. Med Care. 2002;40(10):929–40. 10.1097/00005650-200210000-00010

    10.1097/00005650-200210000-00010

63. Stanley J, Sarfati D. The new measuring multimorbidity index predicted mortality better than Charlson and Elixhauser indices among the general population. J Clin Epidemiol. 2017;92:99–110. 10.1016/j.jclinepi.2017.08.005

    10.1016/j.jclinepi.2017.08.005

64. Tonelli M, Wiebe N, Fortin M, Guthrie B, Hemmelgarn BR, James MT, et al. Methods for identifying 30 chronic conditions: Application to administrative data. BMC Med Inform Decis Mak. 2015;15(31). 10.1186/s12911-015-0155-5

    10.1186/s12911-015-0155-5

65. Stukenborg GJ, Connors DPW and AF, Jr., Wagner DP, Connors AF. Comparison of the Performance of Two Comorbidity Measures, with and without Information from Prior Hospitalizations. Med Care. 2017;39(7):727–39. 10.1097/00005650-200107000-00009

    10.1097/00005650-200107000-00009

66. Kurichi JE, Stineman MG, Kwong PL, Bates BE, Reker DM. Assessing and using comorbidity measures in elderly veterans with lower extremity amputations. Gerontology. 2007;53(5):255–9. 10.1159/000101703

    10.1159/000101703

67. Farley JF, Harley CR, Devine JW. A comparison of comorbidity measurements to predict healthcare expenditures.pdf. Am J Manag Care. 2006;12:110–7. https://pubmed.ncbi.nlm.nih.gov/16464140/

68. Fernando DT, Berecki-Gisolf J, Newstead S, Ansari Z. The Australian Injury Comorbidity Indices (AICIs) to predict in-hospital complications: A population-based data linkage study. PLoS One. 2020;15(9): e0238182. 10.1371/journal.pone.0238182

    10.1371/journal.pone.0238182

69. Li B, Evans D, Faris P, Dean S, Quan H. Risk adjustment performance of Charlson and Elixhauser comorbidities in ICD-9 and ICD-10 administrative databases. BMC Health Serv Res. 2008;8:1–7. 10.1186/1472-6963-8-12

    10.1186/1472-6963-8-12

70. Morre BJ, White S, Washington R, Coenen N, Elixhauser A. Identifying Increased Risk of Readmission and In-hospital Mortality Using Hospital Administrative Data. Med Care. 2017;55:698–705. 10.1097/MLR.0000000000000735

    10.1097/MLR.0000000000000735

71. Piette JD, Kerr EA. The impact of comorbid chronic conditions on diabetes care. Diabetes Care. 2006;29(3):725–31. 10.2337/diacare.29.03.06.dc05-2078

    10.2337/diacare.29.03.06.dc05-2078

72. Weir RE, Lyttle CS, Meltzer DO, Dong TS, Ruhnke GW. The Relative Ability of Comorbidity Ascertainment Methodologies to Predict In-Hospital Mortality among Hospitalized Community-acquired Pneumonia Patients. Med Care. 2018;56(11):950–5. 10.1097/MLR.0000000000000989

    10.1097/MLR.0000000000000989

73. Möller S, Bliddal M, Rubin KH. Methodical considerations on adjusting for Charlson Comorbidity Index in epidemiological studies. Eur J Epidemiol. 2021;36(11):1123–8. 10.1007/s10654-021-00802-z

    10.1007/s10654-021-00802-z

74. Hernán MA, Sterne JAC, Higgins JPT, Shrier I, Hernández-Díaz S. A Structural Description of Biases That Generate Immortal Time. Epidemiology. 2025;36(1):107–14. 10.1097/EDE.0000000000001808

    10.1097/EDE.0000000000001808

75. Harding BN, Weiss NS. Point: Immortal Time Bias - What Are the Determinants of Its Magnitude? Am J Epidemiol. 2019;188(6):1013–5. 10.1093/aje/kwz067

    10.1093/aje/kwz067

76. Ghali WA, Quan H, Brant R. Risk adjustment using administrative data. J Gen Intern Med. 2001;16(8):519–24. 10.1046/j.1525-1497.2001.016008519.x

    10.1046/j.1525-1497.2001.016008519.x

77. Johnston TC, Coory MD, Scott I, Duckett S. Should we add clinical variables to administrative data?: The case of risk-adjusted case fatality rates after admission for acute myocardial infarction. Med Care. 2007;45(12):1180–5. 10.1097/MLR.0b013e318148477c

    10.1097/MLR.0b013e318148477c

78. Southern DA, Quan H, Ghali WA. Comparison of the elixhauser and charlson/deyo methods of comorbidity measurement in administrative data. Med Care. 2004;42(4):355–60. 10.1097/01.mlr.0000118861.56848.ee

    10.1097/01.mlr.0000118861.56848.ee

79. Roos LL, Stranc L, James RC, Li J. Complications, comorbidities, and mortality: improving classification and prediction. Health Serv Res. 1997;32(2):229–38; discussion 239-42. https://pubmed.ncbi.nlm.nih.gov/9180617/

80. Jean RA, Chiu AS, Boffa DJ, Detterbeck FC, Blasberg JD, Kim AW. When good operations go bad: The additive effect of comorbidity and postoperative complications on readmission after pulmonary lobectomy. Surg (United States). 2018;164(2):294–9. 10.1016/j.surg.2018.03.019

    10.1016/j.surg.2018.03.019

81. Albertsen LW, Heide-Jørgensen U, Grey SAJSC, Jackson R, Sørensen HT, Schmidt M. The danish comorbidity index for acute myocardial infarction (Dancami): Development, validation and comparison with existing comorbidity indices. Clin Epidemiol. 2020;12:1299–311. 10.2147/CLEP.S277325

    10.2147/CLEP.S277325

82. Constantinou P, Tuppin P, Fagot-Campagna A, Gastaldi-Ménager C, Schellevis FG, Pelletier-Fleury N. Two morbidity indices developed in a nationwide population permitted performant outcome-specific severity adjustment. J Clin Epidemiol. 2018;103:60–70. 10.1016/j.jclinepi.2018.07.003

    10.1016/j.jclinepi.2018.07.003

83. Whiteneck GG, Ketchum JM, Almeida EJ, Goldstein R, Brown AW, Corrigan JD, et al. Developing an Index of Medical Conditions Associated with Outcomes after Moderate-to-Severe Traumatic Brain Injury. J Neurotrauma. 2021;38(5):593–603. 10.1089/neu.2020.7366

    10.1089/neu.2020.7366

84. Mayo NE, Nadeau L, Levesque L, Miller S, Poissant L, Tamblyn R. Does the addition of functional status indicators to case-mix adjustment indices improve prediction of hospitalization, institutionalization, and death in the elderly? Med Care. 2005;43(12): 1194–202. 10.1097/01.mlr.0000185749.04875.cb

    10.1097/01.mlr.0000185749.04875.cb

85. Vaughan-Sarrazin MS, Lu X, Cram P. The impact of paradoxical comorbidities on risk-adjusted mortality of Medicare beneficiaries with cardiovascular disease. Medicare Medicaid Res Rev. 2011;1(3):1–17. 10.5600/mmrr.001.03.a02

    10.5600/mmrr.001.03.a02