Predictors of missing Electronic Medical Record (EMR) discharge summaries: A retrospective study on data from a large Canadian cohort

Background: Healthcare systems worldwide have adopted the electronic medical record (EMR). EMRs are an ecient method of interprofessional communication, and can improve data availability for secondary research purposes. The discharge summary (DS) is a crucial document for both interprofessional communication, and coding of data for research purposes. We aimed to assess the completeness of our EMRs by assessing the presence of a DS in the EMR. Additionally, we evaluated the presence of predictors for a missing DS. Method: A retrospective chart review was conducted on 3,011 inpatient charts from 3 hospitals in Calgary, Alberta Canada. 893 charts were found to be missing an electronic DS. A 10% sample was drawn to assess for presence of a paper DS. A list of variables was compiled to assess for association between patient and hospital characteristics, patient comorbidities, and the absence of an electronic DS. A Chi-square test, Fisher’s test and logistic regression were conducted to assess for associations. Results: The univariate analyses showed that age, being a surgical patient, a Charlson Comorbidity Index (CCI) of </1, as well as patients with myocardial infarctions, congestive heart failure, cerebrovascular disease, dementia, chronic pulmonary disease, diabetes, and renal disease were associated with a missing DS. The multivariate logistic regression showed that those that were middle aged, surgical patients, or with fewer comorbidities were more likely to have a missing DS. Within the 10% sample, approximately 50% of all patients were from a surgical department, all of which were missing both electronic and paper discharge summaries. Conclusion: Our study rst describe associated discharge lack of discharge summaries in surgical units. heightened risk of readmissions, and poor usability of coded health data. Our study is the rst to describe and identify predictors of missing electronic DS. In an era of integrating and relying on the EMR for patient care and secondary data purposes, efforts should be focused on improving DS documentation among surgical wards, middle age patients, or those with fewer comorbidities. The ndings of our study will alert researchers to carefully conduct analyses using EMR data, taking into consideration the completeness of the EMR, and its inuencing factors.


Introduction
Due to recent health care modernisation, health care providers are becoming increasingly dependent on electronic medical records (EMR). Furthermore, the EMR has become a widely accepted source of data for epidemiological, health services, drug utilization, regulatory and safety surveillance research [1]. An EMR accumulates relevant information from a patient's visit to a health care facility, including course of observation, treatment and discharge plan. Clinical documentation in the EMR can therefore exist in a variety of formats, ranging from free-text documentation to imaging les and numerical bedside monitor trends. With the use of machine learning techniques such as natural language processing and data

Study Cohort
In this retrospective study, six registered nurses performed a chart review of 3,045 randomly selected inpatient charts (one chart for one patient) at three acute care hospitals in Calgary. In From January 2015 to January 2016, there were 111,148 inpatient admissions to Calgary hospitals. The nurses had varying clinical experience, both in years of practice and area of expertise. The intent of the chart review was to identify 50 of a derived set of Charlson and Elixhauser health conditions. The chart review took place between August 2016 and June 2017. Patients met inclusion criteria if they were adults (> = 18 years), had an Alberta personal health care number, and had an inpatient visit for any service outside of obstetrics between January 1 st , 2015 and June 30 th , 2015. A nal number of 3,011 patient charts were used for this study. From the initial 3,045, 30 were in the eld of obstetrics and thus were excluded. An additional 4 patient charts were missing information from the chart review and were excluded ( Figure 1).

Datasets Used
The provincial electronic health record system for Alberta Health Services began in 1997, and has since been growing in complexity, as healthcare services continue to expand in electronic availability. Since 2006, the system used by Calgary healthcare providers for documentation in the inpatient electronic medical record is Sunrise Clinical Manager TM (SCM)-EMR. However, only data collected since 2011 has been validated for research use [9]. SCM-EMR contains the electronic medical record, as well as laboratory and diagnostic imaging reporting. It includes free text (e.g., DS), structured data (laboratory data), and numerical bedside monitor trend data.
In this study, SCM-EMR was used to draw information on the 3,011 patient charts used in the chart review. The nurses were given access to both SCM-EMR and the paper copies of the chart to complete the chart review. For every patient visit, the chart reviewer rstly reviewed the patient's electronic chart, and then reviewed the paper chart if additional information was needed for data abstraction. For the purpose of a larger study with the same research group at the University of Calgary, the data produced by SCM-EMR on these 3,011 patients was also analyzed. During the analysis, it was found that 893 of the 3,011 inpatient charts were missing a DS.

Extended Chart Review
A random 10% sample (85 charts) was drawn from the 893 missing DS charts to assess for the presence of a DS in paper format, which would not have been detected in the SCM-EMR data. Additionally, the 85 charts were assessed for information on the unit and department from which the patient was discharged, as well as the attending physician's specialty.

Study Variables
For those with a missing electronic DS, a list of possible associated variables was collected. These variables were compiled using expertise from physicians and nurses. Age was categorized according to the parameters used by the Centre for Disease Control and Prevention. Length of stay parameters were drawn from the Discharge Abstract Database within the Canadian Institute for Health Informatics (CIHI). According to CIHI, the average length of stay for Albertan patients in 2018 was 7.8 days [10]. Therefore, the four categories used to classify length of stay were deemed appropriate by study authors. The day of the week was used to assess whether the presence of a DS was in uenced by the patient's discharge falling on a weekday or weekend. Due to decreased hospital staff during weekends, e ciency and documentation can be affected. Study authors included the day of the week variable as a measure to observe whether decreased e ciency on weekends would result in the absence of a DS. The DS is crucial for interprofessional communication, particularly when a patient is being transferred from one facility to another. Therefore, disposition was included as a variable to assess whether an interfacility transfer affected the presence of a DS in the EMR. Finally, the 17 Charlson comorbidities are a common set of secondary diseases that can accompany the primary disease of a patient [11]. The Charlson Comorbidity Index (CCI) is used to weigh the comorbidities, and to perform risk adjustment in outcomes research from administration databases (such as inpatient mortality). Both the CCI and the individual Charlson comorbidities were included in the analysis for associated variables.

Statistical Analysis
Study variables were reported as numbers and percentages for categorical variables. Chi-Square test or Fisher's exact test, when appropriate, were performed to assess the differences of associated variables between missing DS and non-missing DS charts. Logistic regression was used to identify the variables that were independently associated with missing DS. Logistic regression results were reported using odds ratios (ORs) and 95% con dence intervals (95% CIs). A p-value of <0.05 was used for statistical signi cance. Analyses were performed using Stat 14.0 (Stat Corp, College Station, TX, USA).

Results
By comparing the 893 missing DS patients with the remaining 2,118 patients with a DS, the univariate analyses showed that age, being a surgical patient, a Charlson Comorbidity Index (CCI) of </1, as well as patients with myocardial infarctions, congestive heart failure, cerebrovascular disease, dementia, chronic pulmonary disease, diabetes with complications, and renal disease were associated with missing DS (Table 1 and Table 2). Table 1. Frequencies of patient and hospital characteristics assessed for association with missing discharge summaries in the EMR Table 2. Frequencies of comorbidities assessed for association with missing discharge summaries in the EMR The multivariate logistic regression suggested that patient age, surgical patient, and CCI score were all found to be independently associated with missing discharge summaries (Table 3). Within the 10% sample of missing electronic DS charts, 51 of the 85 charts (60%) were also missing paper discharge summaries. Forty of the 51 charts missing an electronic and paper DS contained "Discharge Instructions", an instructional carbon-copy paper document for the patient most often given post-surgery which outlines the precautionary measures to avoid complications to the surgical site. All 40 of the patients with discharge instructions were discharged from a surgical unit. The remaining 11 patients with no electronic or paper DS or discharge instructions were discharged from non-surgical units (Table 4). In an effort to assess whether residents and medical students producing the DS contributed to the missing summaries, DS author and level of training was included as a variable of analysis. However, for those with a paper DS, the signature and level of training were often illegible. For the remaining cohort with no DS, author was not identi ed since no DS was authored. Therefore, this variable was not used in the analysis.

Discussion
The DS is a crucial document within the patient chart, and its absence can have detriments on the level of completion of the EHR. Consequently, patient outcomes, interprofessional collaboration, and administrative data quality are all impacted by an incomplete EMR. Our objective was to assess the completeness of EMR data through analysis of the presence of a DS, and to nd variables associated with missing discharge summaries. Alberta Health Services' current EMR system has been in use for over a decade. It is one of the most validated electronic systems in Canada, and has great utility for interprofessional collaboration and secondary research purposes [9]. Despite this, and the evidence supporting a fully electronic EMR system, our ndings demonstrated that EMR incompleteness was associated with patient baseline and setting characteristics. Interestingly, physicians were relying on paper charting or simply not creating discharge summaries when patients are middle-aged, surgical and have fewer comorbidities. The ndings of our study will caution researchers when using EMR data pertaining to these 3 patient characteristics.
Upon further assessment of the 85-chart sample, 70% were from outpatient surgical procedures, speci cally urological and orthopedic procedures. The one urology clinic and two orthopedic clinics in Calgary were contacted for further information on DS practices. Across all clinics, staff reported limited need for a DS post-procedure, as the surgeon performing the procedure is often the surgeon that will see the patient in follow-up at the clinic. Additionally, given the simple nature of many of these procedures, there is no perceived need for a copy to be made to the family physician. A DS paper template is provided for surgeons at one orthopedic surgery site, albeit not used. Discharge instructions are thus the only document created, with the sole purpose of informing the patient with post-procedure care. However, both orthopedic and urological clinics reported creating a DS in the context of a complex case, wherein a nurse practitioner creates the DS, not the surgeon performing the procedure.
Our nding of missing EMR discharge summaries in surgical departments aligns with existing literature, though our study is the rst to report on complete absence of the electronic DS [12]. Most studies report on the illegibility or scarcity of information within the surgical DS, while we found the document to be missing all together [13]. The lack of standardization in the creation of a DS by the surgeon performing the procedure calls for further investigation into departmental-or unit-culture norms. Possibly, there have been no voiced concerns within the healthcare system to incentivize surgeons to create a DS. However, this is dubious, given the extensive national and international literature on both general practitioner and patient dissatisfaction with post-procedure information and outcomes [12,13,14]. There is a dearth in literature on surgeon perspectives toward the necessity of a DS. When broadened from surgical to physician perspectives from other specialties, one study found physicians often do not feel su ciently informed on the patient's health history to create a DS. Additionally, avoidance of redundancy of patient information within the EMR is a primary deterrent to creation of a DS [15]. More investigation is needed on the surgeon's perceived necessity, or lack thereof, for a DS, as a means to understand why surgical cases are often missing a DS within the EMR and paper record.
The reported incentive in surgical departments to create a DS for complex cases can explain the association found in this study's larger cohort between lower CCI scores and a missing DS. A lower CCI score indicates fewer or less severe comorbidities, which can simplify the procedure. A simple procedure would thus not qualify as "complex" and would not be considered for a DS. Outside of the surgical realm, one study focusing on general medicine found that patients being discharged with a lower CCI score had nearly double the time to dictation from point of discharge (measured in days), compared to those with a higher CCI score, despite low CCI patients having signi cantly shorter lengths of stay. This apparent lack of urgency to create a DS could be re ected throughout healthcare provider practices for patients with fewer or less severe comorbidities, leading to the absence of a DS in this cohort of patients. Interestingly, the length of the DS (words used) in patients with lower CCI scores did not differ from that of patients with higher CCI scores, which could represent the intrinsic need for communication between inpatient and outpatient physicians [16], regardless of the complexity of the case. Additionally, the associated risks for readmission increase for every 3 days gone without a DS once a patient is discharged [17]. Therefore, for patients where no DS is ever created, the posed health risks could be detrimental.
With regards to the statistically signi cant nding between those aged 45-64 and the absence of a DS, there is a lack of literature to support this discovery. However, it could be related to the decreased complexity of these patients, and consequent non-urgency for creation of a DS. Compared to those aged 65 and greater, disease prevalence for many of the Charlson comorbidities, as well as the most commonly reported diseases (e.g. hypertension, type 2 diabetes, coronary artery disease), is far less frequent. Therefore, the perceived urgency in reporting on a patient undergoing a simple procedure with few comorbidities could be less compared to the urgency for reporting on a complex patient. This could result in an absence of a DS.

Limitations
Since some of the documents for charts within Calgary medical facilities are still in paper format, this study cannot fully report on the quality of the EMR; rather, it reports on the completeness of the patient medical record. Nonetheless, regardless of paper or electronic documentation, there was still a signi cant number of records missing a DS entirely.
Within the 893 charts, there is a possibility that charts were captured in the missing DS cohort, when there was actually a paper DS present. Nonetheless, the lack of an electronic DS within the electronic medical record is still relevant. Given the predominantly electronic form of communication between hospitals and general practitioners in Alberta, the inconsistency in availability of documentation in one single location can delay processes for practitioners searching for important health information. Additionally, the delay in receiving a paper document versus quickly accessing the electronic document from a general practitioners o ce can hinder health delivery services.

Implications of ndings
In addition to having negative consequences on patient health outcomes, the absence of a DS can also affect the use of EMR data for secondary purposes [18]. In the process of transforming patient data into a data repository, a coder's data abstraction and the resulting data quality can be hindered with an absent DS. Within coding practices, the DS is treated as a gold-standard document for coders to abstract, verify, and validate their coding. When the DS is missing, coders must use additional physician documentation (e.g. the History and Physical) to abstract their necessary codes. However, without the DS, if there is inconsistent information across different documents, coders cannot verify their codes against their goldstandard and are forced to omit codes. Therefore, the rigor with which the they can con dently code is consequently. This can lead to missing data within the database the codes are entered in, which decreases the quality of data. This data quality is crucial given its use for national statistics on population diseases and morbidity. Advances in medicine and research will be hindered if the missing DS phenomenon and subsequent poor data quality continues.

Conclusion
In conclusion, our chart review on a large Canadian cohort found a large percentage of incomplete EMR data, as indicated by a missing DS. The absence of a DS within the EMR can present several issues for healthcare processes, including hindered communication between hospitals and general practitioners, heightened risk of readmissions, and poor usability of coded health data. Our study is the rst to describe and identify predictors of missing electronic DS. In an era of integrating and relying on the EMR for patient care and secondary data purposes, efforts should be focused on improving DS documentation among surgical wards, middle age patients, or those with fewer comorbidities. The ndings of our study will alert researchers to carefully conduct analyses using EMR data, taking into consideration the completeness of the EMR, and its in uencing factors.  Table 2. Frequencies of comorbidities assessed for association with missing discharge summaries in the EMR Figure 1 Flow diagram of nal cohort used for chart review