IJPDS International Journal of Population Data Science IJPDS 2399-4908 Swansea University 10.23889/ijpds.v11i5.3729 11:5:3729 S2399490821037290 Population Data Science Enhanced Depression Phenotyping Using Machine Learning and Electronic Medical Records for Disease Surveillance: A Validation Study in Inpatient Settings LinNa1 A. MartinElliot2 LeeSeungwon2 GrothmanAllison3 LiNa1 AdhikariKamala4 RiaziKiarash1 A. EastwoodCathy1 A. SouthernDanielle5 QuanHude1 Centre for Health Informatics, University of Calgary, Calgary, Canada; Department of Community Health Sciences, Cumming School of Medicine, Community Health Sciences, University of Calgary, Calgary, Canada; Libin Cardiovascular Institute of Alberta, Calgary, Canada Centre for Health Informatics, University of Calgary, Calgary, Canada; Provincial Research Data Services, Health Shared Services Provincial Health Corporation, Calgary, Canada Faculty of Graduate Studies, University of Calgary, Calgary, Canada Department of Community Health Sciences, Cumming School of Medicine, Community Health Sciences, University of Calgary, Calgary, Canada; Cancer Prevention and Screening Innovation, Public Health Evidence and Innovation, Primary Care Alberta, Calgary, Canada Centre for Health Informatics, University of Calgary, Calgary, Canada; Department of Community Health Sciences, Cumming School of Medicine, Community Health Sciences, University of Calgary, Calgary, Canada 11 5 3729 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. This article is available from the IJPDS website at: https://ijpds.org/article/view/3729 Background

Accurate detection of inpatient depression is essential for surveillance and resource allocation. Traditional International Classification of Diseases (ICD) codes suffer from low sensitivity and delayed availability.

 Objective

To develop and validate machine learning (ML) phenotyping algorithms for depression using unstructured inpatient clinical notes from electronic medical records (EMRs), compared with ICD-10-CA codes and manual chart review.

 Methods

This retrospective cohort study included a derivation cohort (N = 3032 adults, Jan–June 2015) and a temporally independent external validation cohort (N = 10659 adults, 2017–2022) from Calgary acute care hospitals. Both cohorts were deterministically linked to DAD and EMR databases. Manual chart reviews defined the reference standard for depression status. We compared ICD-10-CA codes against three EMR-based algorithms: Keyword Search, Concept Model and Document-Concept Model. The latter two employed a supervised ML classifier (XGBoost) utilizing clinical concepts extracted via the cTAKES natural language processing (NLP) pipeline. Primary measures were prevalence, sensitivity, positive predictive value (PPV), specificity, negative predictive value (NPV), and F1 score.

 Results

In derivation cohort (prevalence: 18.1%), the Concept Model achieved 76.4% sensitivity and 76.7% F1 score, significantly outperforming Keyword Search (F1: 66.7%), Document-Concept Model (F1: 73.7%), and ICD-10-CA codes (sensitivity: 20.0%; F1: 32.6%). In external validation (prevalence: 13.1%), the Concept Model retained robust performance (sensitivity: 92.3%; F1: 80.1%), compared with ICD-10-CA (sensitivity: 6.1%; F1: 11.4%).

 Conclusion

An EMR-based Concept Model using NLP and ML markedly improves depression case detection over ICD coding in inpatient settings. The methods could potentially facilitate psychiatric surveillance and epidemiological research using real-world clinical data.