Real-World Evidence (RWE) refers to the analysis of data collected outside of a clinical trial setting [1]. Such data collected outside of conventional randomized controlled trials is aptly referred to as Real-World Data (RWD), an umbrella term that includes healthcare data collected from patients, clinicians, hospitals, and payers [2]. Using varied data sources to generate RWE enables assessments of the effectiveness, safety and cost-benefit of drug therapies and other clinical interventions. Real world evidence is also used to inform decisions that aim to drive improvements in policy, system delivery, and ultimately health outcomes, with mutual benefit to health ecosystem stakeholders (clinicians, partners, patients, and citizens).

RWD-based research can be invaluable in interrogating common complex diseases. This approach is more efficient, less expensive, and has better generalizability than traditional clinical trials [3]. In addition, populations that are typically excluded from traditional research studies—such as children, pregnant women, and the elderly—have the opportunity to be included in RWE studies. Also, richer information on patients and their health can be captured using RWE studies, ultimately helping to determine factors that affect the disease course and outcomes in these populations.

This case study provides an in-depth look at the foundational elements used for a RWE generation in NL, including a strong public-private partnership, robust data and data infrastructure, and prioritization of privacy and ethics.

The rich data landscape in Canada stands in stark contrast to its low levels of data access and use relative to the rest of the world. Canada has thirteen provinces and territories, and each health ministry governs each system of publicly-funded healthcare. This individual governing system structure creates barriers between data sources and limits the ability of both public and private sector researchers to connect with RWD. While these challenges exist, there has been increasing interest from Canadian healthcare stakeholders, health ministries, and public and private payers to leverage RWE to inform decision-making.

Recognizing their role as an industry leader in driving discussions of RWE generation, Janssen Canada developed partnerships across different regions of the country to identify key shared areas of interest to improve population health. These long-term, public-private RWE partnerships are grounded in strong governance and reciprocal trust – two ingredients necessary for effective collaboration towards meaningful evidence generation. This model can provide deeper insights into burden of illness, quality and efficiency of healthcare delivery, drug effectiveness, and long-term safety. In terms of provincial research capabilities, RWE partnerships can apply scientific best practices and advance methodologies, generate evidence about disease pathways, healthcare delivery and the effects of clinical interventions, and ultimately support decision making and scientific exchange.

In October 2017, Janssen Canada announced the Janssen and Newfoundland and Labrador Health Innovation Partnership (JANL-HIP). This partnership involves the Government of Newfoundland and Labrador, Eastern Health, the Newfoundland and Labrador Centre for Health Information (NLCHI), Janssen Inc., and Memorial University of Newfoundland (MUN). JANL-HIP’s work focuses on RWE projects that will bring mutually beneficial solutions to Janssen, the health system, and to patients of Newfoundland and Labrador.

Newfoundland and Labrador (NL) has a population of 520 998 living in 11 census divisions in 281 municipalities. Nearly two-thirds (59.6%) of the population is reported to live in a large urban population centre, 9.0% in medium population centre, 12.7% in small population centres and 18.7% in rural communities [4]. There are four regional health authorities in NL, namely Eastern Health, Central Health, Western Health and Labrador-Grenfell Health. These health authorities are responsible for the healthcare needs and services of residents within their catchment area.

The NL Centre for Health Information (NLCHI) is responsible for developing, operating, and managing a comprehensive and aligned information system that fully integrates and uses data and health information from all health and community services system components [5]. Through collaboration with the healthcare system, the NLCHI supports the development of data and technical standards, maintains provincial health data and information assets, carries out analytics, evaluation, and decision support services, and supports health research. The NLCHI is also responsible for implementing digital health systems in the province to enable improved patient access and healthcare delivery and create improved accountability, stability, and efficiency in the provincial healthcare system. Digital health systems include virtual care, the electronic health record (HEALTHeNL), the electronic medical record (eDOCS NL), telehealth, and telepathology [6].

The NLCHI is a custodian of many key health databases in the province that capture healthcare interactions for all individuals accessing healthcare services in NL. Data captured include vital statistics, fee-for-service physician billings, hospitalizations, prescriptions filled at community pharmacies, laboratory data, and emergency department visits [7]. In addition, the NLCHI has an internal de-identification process that allows the consolidation of individuals with multiple healthcare numbers into one unique ID, enabling data linkage across multiple datasets.

NLCHI has been leading efforts to advance the availability, quality, and use of health system data for decision making, research, and innovation purposes through its Provincial Data Lab and Data Governance Framework [8]. The Provincial Data Lab includes a data repository and secure, virtual environments that allow users to interact with data and information in a privacy protective manner.

The Provincial Data Lab is where internal NLCHI users currently work with data; however, there is also a Research and Evaluation Environment where users external to NLCHI can access and analyze data regardless of geography. Once all required approvals and data sharing agreements are in place, authorized users can remotely access the Provincial Data Lab from any computer via a virtual private network and two-factor authentication. Individuals who access record-level data are provided with access to various analytical tools and software and the ability for project teams to access data, code, and output in common project spaces. In addition to providing access to data and information, the Provincial Data Lab also provides data management, audit, storage, retention, and secure disposal services to users.

The success of a unique RWE partnership model is rooted in a common purpose, strong governance, and effective operations. The JANL-HIP is a collaboration to fund and lead the generation of RWE, the scope of which is limited to projects focused on topics of interest to the members. These topics are agreed upon and documented by all parties. Project criteria include: (i) a focus on improved health outcomes; (ii) identification of analytic findings that can inform provincial health policy; (iii) timely conduct of the project (for example, data output within twelve months and completed analysis within two years); and (iv) pan-provincial considerations.

The JANL-HIP governance model is comprised primarily of a Steering Committee and a RWE Working Group (Figure 1).

A Steering Committee was established with representation from all funding partners. These partners may also include mutually agreed upon third-party collaborators on the Steering Committee. In general, decisions of the Steering Committee are expected to be made by consensus, with each party having one vote. The Steering Committee members established terms of reference and nominated a chairperson from MUN. In addition, MUN was responsible for the management, administration, and distribution of the Partnership Fund, including managing the budget.

As the main governing body for JANL-HIP, the role of the Steering Committee involved

The RWE Working Group was established to undertake the RWE projects, ideally with equal representation from the partners. There can also be representation from other collaborators, as agreed upon by the Steering Committee. In general, decisions of the Working Group are expected to be by consensus.

The role of the Working Group includes the following:

In essence, the JANL-HIP governance and operations model guides the utilization of RWD to undertake relevant and important RWE projects to support patient care and health outcomes.

In addition to strong partnership governance, privacy and security are also essential considerations given the sensitive nature of the data needed for such projects. Real-world studies constitute one example of secondary use of data in that data are collected and gathered for other purposes, such as in the context of healthcare provision. [9]. Secondary use of data in RWE studies is often considered an economical approach to answering research questions as it relies on the reanalysis of previously gathered data. Such types of studies have become increasingly common in an era of big data and as researchers discover the value of re-interrogating information [10].

The primary issue faced by users of secondary data are privacy and confidentiality [11]. The risk level depends on the type of information gathered and whether there are any identifying features included in the dataset. Because current access and secondary use of data pertain to studies for which consent may not have initially been sought, oversight must safeguard the secondary use of RWD.

Studies using personal health information in the province of NL are governed by complementary legislation and ethical guidance. In NL, personal health information and its safeguarding fall under the purview of the Personal Health Information Act (PHIA), which was proclaimed in 2011 [12]. PHIA governs the use of personal health information in NL, designates custodians of personal health information, and establishes the rules that they must follow for the collection, use, and disclosure of this information. Because studies utilizing these data do not necessarily obtain direct consent from patients, the role of the custodian is to ensure that identifying features are removed and to consider potential harm to patients. Unlike clinical trials or other more traditional research studies, the use of secondary data does not typically pose physical risks to patients. Instead, potential risks pertain largely to privacy stemming from the use of personal data [13].

The ethics requirements of studies undertaken in NL are regulated first and foremost by provincial legislation. Health research taking place in NL is governed by the Health Research Ethics Board (HREB), a mechanism of the Health Research Ethics Authority (HREA), which the provincial government established under the Health Research Ethics Authority Act in 2011 [14]. The HREB reviews and renders an approval decision on all health-related research in NL. While some countries have laws in place that govern Research Ethics Board (REB) operations, Canada lacks a uniform research oversight regulation at the federal level, relying instead on guidelines established by the Panel on Research Ethics for research involving humans as outlined in the Tri-Council Policy Statement version 2 (TCPS2) [15].

Ethics approval is of particular importance in NL, given the local geographic context. Data that may not otherwise lead to identifiable participants (such as postal code) can be identifying in communities with only a few dozen inhabitants and a handful of sufferers of specific diseases. Complementary legislation, ethical guidance, and NLCHI’s governance framework for the Provincial Data Lab all helped guide the team and ensure the security and proper usage of real-world health data. This combined oversight is a crucial component to any research study carried in NL, including real-world studies.

There are several other real-world evidence initiatives in Canada, providing access to large-scale RWD for analysis to generate evidence beneficial to the healthcare system. RWE can range from broad population-based academic initiatives to disease-specific collaborations to commercial-led partnerships, as illustrated in these examples.

Newfoundland and Labrador have a high prevalence of psoriatic disease than anywhere else in Canada; therefore, its study is critical to the provincial healthcare system and the patients it serves. However, results derived from traditional research methods might not wholly represent most people living with the disease. This is because only a subgroup of the psoriatic population can meet the requirement to enroll in clinical trials or other psoriasis registries. For example, despite high-quality Phase III randomized trials in psoriasis and psoriatic disease, the efficacy of advanced targeted immune therapies in biologically experienced patients is not as robust. This is because the response rate for biologically experienced patients is lower than biologically naïve psoriatic patients, and their drug persistence is much shorter [24]. Also, the generalizability of these cost-effectiveness studies is a challenge as only a subset of the entire biologically eligible PsA spectrum patients is entered into a clinical trial. Additionally, psoriatic patients with concomitant medical illness are not included in clinical trials, likely resulting in greater tolerability and increased drug safety profile because medically complicated patients cannot tolerate prolonged immunosuppression [25].

The opposite scenario also occurs, as noted in a Canadian biologic registry, where patients with such milder disease PsA are over-represented in clinic patients receiving biologic agents compared to the clinical trials [26]. This has economic implications as patients with milder disease may be managed with palliative care, and the therapeutic effect size between DMARDs and biologics appears to be smaller [26]. So, including real-life data from registries will be important in determining the actual value of biologic agents and, overall, the prevalence and effect of psoriatic disease in the province.

Recognizing that RWE studies have the potential to provide further insights into disease associations, the partnership intends to use RWD available through the NLCHI to explore the association between psoriasis and mental illness and other metabolic diseases such as obesity, diabetes, and hypertension associated with active psoriatic disease [27, 28].

Our first study investigates the association between psoriasis and mental health [29]. Data on 15,100 patients with psoriasis and 75,500 controls (1:5) was collected from the NLCHI Electronic Health Records, and the cases and controls were matched for age, sex, and geography. Patients with psoriasis were identified through ICD-9 code 696 from dermatologists in the province. Diagnosis of mental illnesses was obtained from physician’s visits and hospitalization records using ICD-9 and ICD-10 codes.

Following the study analysis, the percentage of people coded for anxiety in the psoriatic cohort was 36.5% compared to 28.9%, p < 0.0001 in the comparison group; and depression was 37.0% compared to 30.1%, p < 0.0001. The most significant increased risk for anxiety [OR 1.4 (1.20–1.67)] and depression [OR 1.65 (1.36–2.00)] among psoriasis was found for patients in the 0 to 20 age group.

Through this research, the JANL-HIP database provides relevant information regarding the burden of mental health disease in NL. Upcoming analyses will focus on attempted and completed suicides and the impact of health care resources of psoriasis patients compared to the control cohort. The partnership also plans to interrogate the association between psoriatic disease and cardiovascular diseases as psoriatic patients are at increased risk for cardiovascular disease compared to the general population, and cardiovascular disease is among the leading causes of death for psoriasis and PsA patients [30, 31]. We have also completed linkage of the JANL-HIP data with the NL- APPROACH database, an ongoing prospective data collection that captures the population of patients undergoing cardiac catheterization in Newfoundland.

RWE studies, like this example for psoriatic studies in NL, can generate evidence about disease pathways, healthcare delivery, clinical pathways, and support and influence clinical decision-making. RWE studies can also provide necessary data on the comparative effectiveness of traditional disease-modifying drugs and biologic agents and information on drug persistence and safety.

Conducting RWE studies does have limitations. These limitations include lack of randomization and internal validity [32]; risk of using compromised data which can be due to high drop rates or incomplete reporting by study participants [32–34]; likelihood of performing a biased study as a result of using data from a very homogenous population and therefore deriving results that cannot be extrapolated easily to other populations; possibility of determining association but not causation [34] and lack of a study design and standardization for reporting [35]. These potential shortcomings have led to calls for caution and transparency when working with RWD [3, 34]. Despite these limitations, the insights gained from RWE studies are invaluable and will augment existing study results and address limitations found in randomized clinical trial studies [35].

There are multiple determinants to positive patient health outcomes. Traditional research identifies only a portion of these determinants, leaving a gap. RWE is complementary to traditional research, providing both an efficient and holistic examination of the patient population. JANL-HIP has brought together diverse system stakeholders with the common aim of closing this gap by interrogating RWD in NL, and will provide good value for the healthcare system as it can impart further data on long-term effectiveness, adverse events, and drug persistence of psoriatic disease.

With NL’s harmonized medical care system infrastructure, NLCHI’s Data Governance Framework and Provincial Data Lab containing multiple sources of high-quality data, and provincial review processes for ethics, privacy, and security and unique complimentary legislative framework, JANL-HIP has a winning formula to realize its vision. This public-private partnership demonstrates a successful model of RWE generation to improve the healthcare system and provide a better quality of care for patients.

Funding for the JANL-HIP partnership is through Janssen Inc. and in-kind support from Memorial University. It should be noted that no Janssen products are discussed within this manuscript.

Proton Rahman has received grants from Janssen, Novartis and Speaker’s Bureau/Consultancy fromAbbVie, Amgen, Bristol Myers Squibb, Celgene, Eli Lilly, Janssen, Merck, Novartis, Pfizer, UCB.

Wayne Gulliver has received Grants/research support from AbbVie, Amgen, Eli Lilly, Novartis, Pfizer and Honoraria for Ad Boards/Invited Talks/Consultation from AbbVie, Actelion, Amgen, Arylide, Bausch Health, Boehringer, Celgene, Cipher, Eli Lilly, Galderma, Janssen, LEO Pharma, Merck, Novartis, PeerVoice, Pfizer, Sanofi-Genzyme, Tribute, UCB, Valeant. Other: Clinical trials (study fees): AbbVie, Asana Biosciences, Astellas, Boerhinger-lngleheim, Celgene, Corrona/National Psoriasis Foundation, Devonian, Eli Lilly, Galapagos, Galderma, Janssen, LEO Pharma, Novartis, Pfizer, Regeneron, UCB.

Evelyn Pyper, Janey Shin, Houng Hew and Brenda Fisher are employees of Janssen Inc. The other authors have no conflict of interest to declare.