The production, collection and use of population data for research is becoming more prevalent across multiple sectors, but particularly in health and healthcare [1–3]. For example, the use of electronic health records has seen a significant increase among researchers and clinicians [4, 5]. However, population datasets often contain Personal Identifiable Information (PII) and/or Personal Health Identifying Information (PHII), which researchers have the responsibility to keep confidential. In Canada, the use of population data containing PII and PHII in research is governed by the Canadian Tri-Council Policy Statement on Ethical Conduct for Research Involving Humans, which includes three core principles: respect for persons, concern for welfare, and justice [6]. One effective way to preserve privacy and abide by this ethical framework is to de-identify the data before it is used in research. De-identification refers to the removal or masking of PII/PHII in a dataset and for research purposes, it may be preferable to anonymisation, a process that eliminates all identifying details in a data record with no way of back-tracking to link related data records together [7]. When a record number, file number or other encrypted linkage tool is retained in the original data, the data are not referred to as ‘anonymised’ but are instead ‘de-identified’ and can be used in data linkage applications.

The federally mandated Freedom of Information and Protection of Privacy Act (FIPPA) and the provincially mandated Personal Health Information Act (PHIA) provide definitions of PII and PHII and set out guidelines to inform the process of de-identifying structured data [7]. Structured data are organised into specific value sets and are typically stored in a database [8]. Meanwhile, unstructured or free text data do not have pre-defined values; for example, reports created by physicians may contain free text data that vary widely in structure and content [8]. Currently, there is very little formal guidance available on how to de-identify free text data, and none that we could find that differentiates PII and PHII in the de-identification process. In this matter, the distinction between PII (recorded information that could identify an individual or groups of individuals) and PHII (specific health information about an individual or groups of individuals) is important, because specific approaches for de-identification are needed if health information is present in the data [8] (see Table 1 for examples [9]).

The natural language processing (NLP) research community has made great strides in developing methods for automatically de-identifying data. There are currently two primary approaches in use:

These two methods can be used for de-identification of both structured and free text data. De-identifying structured data is a relatively straight-forward process compared to de-identifying free text data, because structured data typically have a limited number of clearly identified fields; in addition, there is some literature to inform and guide the process [7]. De-identifying free text data, however, may necessitate a more sophisticated approach, since identifying information may occur anywhere in the free text and may include either PII or PHII or both. Some researchers have developed hybrid approaches in an attempt to combine the advantages of rule-based and ML methods for de-identification of PHII [11]. The hybrid approaches take advantage of the fact that certain types of PHII exhibit predictable lexical patterns and thus lend themselves well to de-identification via rule-based methods, whereas other frequently encountered PHII types, particularly those with unpredictable lexical variations, are more amenable to machine learning approaches [13]. More recently, the use of deep learning methods has been explored to de-identify electronic health records [14]. These methods have the ability to learn the most relevant features from the raw data, minimising the need for human input and making the pre-processing and feature engineering steps less time consuming [15].

Data de-identification techniques are advancing quickly and have a growing number of applications in research settings. In this scoping review, we provide an overview of what is known about NLP methods used to de-identify free text data.

We based our scoping review approach on Arksey and O’Malley’s (2005) well-established scoping review framework, which comprises five stages [16]: identifying the research question; identifying relevant studies; study selection; charting the data; and collating, summarising, and reporting the results. Our research question was: What methods are used to de-identify free text data?

The metrics mentioned to measure performance by the articles are presented in Table 10. Six out of the 18 articles mentioned evaluation metrics for assessing the performance of NLP de-identification approaches. Some articles used terms commonly used in the computer science literature such as recall and precision while others used terms that have the same meaning from epidemiology such as sensitivity and specificity. Additionally, while the articles discuss the same metrics, some of them use different formulas in varying contexts. For instance, in Kushida et al. (2012), the term precision is employed to evaluate the performance of Stat De-id, a statistical learning-based system originally introduced in Uzuner et al. (2008) [20, 65]. However, in Meystre et al. (2010), the precision formula is not provided, but instead, reference is made to how HMS Scrubber was evaluated by Beckwith et al. (2006) [19, 36].

Free text data contain a wealth of information that is valuable in research. To take full advantage of this information, de-identification approaches for free text data must ensure the privacy and confidentiality of individuals described in the data. The discussion of de-identification of data in health research previously focused on structured data. The growth and importance of free text data in health records and health research has resulted in the need for advances in de-identification approaches. This scoping review of reviews identifies published de-identification methods for free text data. We have categorized the methods as rule-based methods, machine learning, deep learning and a combination of these and other approaches. Most of the articles we found in our search refer to de-identification methods (primarily rule-based and machine learning methods) that target some or all categories of PHII defined by HIPAA.

In general, experts in the field are using rule-based methods with anonymisation models to de-identify data; in particular, they use K-anonymity, I-diversity and t-closeness. Sakpere et al. (2014) assert that K-anonymity methods are best suited for data stream anonymity, such as phone numbers [31]. However, Senosi et al. (2017) found that researchers only give anonymisation strategies an average rating for protecting privacy [32]. Additionally, Stubbs et al. (2015) observes that even if automated rule-based solutions are beneficial, some PHII is still included in the data since the success of the de-identification process depends on the dictionaries used [25]. Yogarajan et al. (2020) argues that machine learning methods for de-identification need to improve in areas such as maintaining correctness and usability of data [26]. Meystre et al. (2010) states that machine learning methods combined with rule-based approaches such as HMS Scrubber perform better than a single method at de-identification of free text data [19].

Recently published articles reviewed a number of approaches, including systems based on machine learning and hybrid systems that use a combination of different de-identification methods, including deep learning methods (e.g., NeuroNER and Bidirectional Encoder Representations from Transformers (BERT) [22, 26]. Shickel et al. (2018) found systems based on deep learning performed better than other methods on lexical features [15]. However, deep learning techniques require large datasets to perform effectively [15]. Deep learning methods also make validating accuracy challenging due to the nature of the method. While they do represent significant progress in de-identification, the size of the required datasets for acceptable performance is an important limitation.

This scoping review provides an overview of de-identification methods for free text data. As computation power and the availability of free text from electronic health records have increased, the importance of de-identification methods in advancing the use of text data for research has also grown. While this review sought to classify de-identification techniques, no single approach or rule-based method was found to meet the high standards required to address the needs of research privacy regulators in protecting the privacy of patients since no single approach could reliably de-identify all PHII in population data records [20]. The combination of multiple tools in a hybrid format appears to be the most promising future direction.

The University of Manitoba Health Research Ethics Board does not require review of review articles.