J Epidemiol Community Health: first published as 10.1136/jech-2018-211654 on 24 April 2020. Downloaded from http://jech.bmj.com/ on February 28, 2024 by guest.

Protected by copyright.
Glossary

Glossary for public health surveillance in the age of

data science
Arnaud Chiolero ,1,2,3,4 David Buckeridge4

1
Population Health Laboratory ABSTRACT therefore, the right time for a glossary for public
(#PopHealthLab), Department Public health surveillance is the ongoing systematic health surveillance and monitoring, with an
of Community Health, University
of Fribourg, Fribourg, collection, analysis and interpretation of data, closely emphasis on new data-science developments.7
Switzerland integrated with the timely dissemination of the resulting We do not aim to cover the whole field of sur-
2
Institute of Primary Health Care information to those responsible for preventing and veillance but rather focus on how data science is
(BIHAM), University of Bern, controlling disease and injury. With the rapid changing methods and concepts, going from data
Bern, Switzerland
3
Observatoire valaisan de la
development of data science, encompassing big data and generation and collection to information dissemi-
santé (OVS), Sion, Switzerland artificial intelligence, and with the exponential growth of nation for decision-making (figure 2).
4
Department of Epidemiology, accessible and highly heterogeneous health-related data,
Biostatistics and Occupational from healthcare providers to user-generated online ABERRATION DETECTION
Health, McGill University, content, the field of surveillance and health monitoring is In public health, aberration detection is the identifi-
Montreal, Canada
changing rapidly. It is, therefore, the right time for a short cation of anomalous events or patterns in data, with
glossary of key terms in public health surveillance, with an a clinical or public health potential relevance, that is,
Correspondence to
Prof Arnaud Chiolero, emphasis on new data-science developments in the field. statistical signals in surveillance data that may be of
Population Health Laboratory epidemiological importance.8 A major challenge, of
(#PopHealthLab), Department growing importance with the use of highly hetero-
of Community Health,
genous types of surveillance data, is to account for
University of Fribourg, 1700
Fribourg, Switzerland; PURPOSE OF THIS GLOSSARY random variability and measurement error, which
[email protected] ‘Only describe, don’t explain’, attributed to Ludwig makes it difficult to tease out the ‘signal’ upon which
Wittgenstein the decision to intervene is based from the ‘back-
Received 16 November 2019 Public health surveillance is the ongoing systema- ground’ noise.9 Traditionally, outbreak detection
Revised 15 January 2020
tic collection, analysis and interpretation of data, and infectious disease surveillance have relied on
Accepted 29 February 2020
closely integrated with the timely dissemination of reports from clinicians and laboratories. At the
the resulting information to those responsible for turn of the century, surveillance expanded to con-
preventing and controlling disease and injury.1 It is sider prediagnostic or syndromic data, such as the
a core element of public health practice, through
routine monitoring and reporting systems, and of
population health science—the science that informs
public health and prevention strategies—through
observational evidence.2 More specifically, surveil-
lance aims to provide health decision-makers with
timely and useful information to set priorities, to
identify the need for interventions and to evaluate
the effects of interventions.3 It is related to public
health research but differs in its purposes (figure 1):
research aims to increase general knowledge while
surveillance aims to provide information for deci-
sion and action in public health.1
With, on the one hand, the rapid development
of data science, encompassing big data and arti- Figure 1 Health data and related information are
ficial intelligence (AI), and, on the other hand, used, on one hand, to increase general knowledge,
the exponential growth of accessible and highly which corresponds traditionally to a public health
heterogeneous health-related data, from electro- research activity. On the other hand, they are also key
nic medical records used by healthcare providers for guiding decisions and actions by stakeholders in
to user-generated online content,4-6 the field of public health, which corresponds to public health sur-
surveillance and health monitoring is changing veillance activities. The knowledge produced by
© Author(s) (or their
employer(s)) 2020. Re-use
rapidly with a widening scope of application, an research is eventually used to improve public health
permitted under CC BY-NC. increasing depth and new methods. It is, surveillance.
No commercial re-use. See
rights and permissions.
Published by BMJ.

To cite: Chiolero A,
Buckeridge D. J Epidemiol
Community Health Figure 2 Steps in the data processing of public health surveillance, from data generation and collection to information
2020;74:612–616. dissemination for decision-making.
612 Chiolero A, Buckeridge D. J Epidemiol Community Health 2020;74:612–616. doi:10.1136/jech-2018-211654
 J Epidemiol Community Health: first published as 10.1136/jech-2018-211654 on 24 April 2020. Downloaded from http://jech.bmj.com/ on February 28, 2024 by guest. Protected by copyright.
Glossary
count of patients visiting an emergency room5 (see also detection). Mining EHRs aims to gather information from unstruc-
Syndromic surveillance). With the growth in volume and variety tured narrative data21 (see also Electronic medical record).
of accessible surveillance data, aberration detection methods
have evolved from the analysis of time series of case counts to DATA VISUALISATION
the complex modelling of individual-level surveillance cases with Data visualisation has always been an important tool of public health
covariates drawn from multiple sources5,8; it is also applied surveillance. However, with the growth in available data and the
beyond the field of human infectious diseases. improvement in statistical tools, data exploration through visualisa-
tion has gained importance for surveillance and monitoring activities.
BIG DATA AND DATA SCIENCE The field has evolved with contributions of computer science merging
Big data refers to the massive amount of data that is more and more scientific visualisation, information visualisation and visual analytics,
easily accessible through the digitalisation of all aspects of health, making visualisation an important part of surveillance data
healthcare and related areas.10 It is characterised by its variety, analyses22; it is a powerful tool to understand complex multilayer
volume and velocity—the ‘3Vs’.11 Multiple sources of data have data, which are not easily captured by simple indicators. It has a major
become usable for public health surveillance, for example, mobile impact on how temporal and spatial analyses are conducted and
phones, online searches, social media, credit card transactions, reported. The production of continuously updated maps and atlas
wearable and ambient sensors, electronic health records (EHRs), of diseases and risk factors has become possible by leveraging big data,
medico-administrative records and pharmacy sales. While public thereby strengthening the surveillance of numerous conditions, nota-
health monitoring relies traditionally on well-defined and high- bly of infectious diseases.23 Visualisation of healthcare outputs
quality data, effective use of big data for surveillance requires new through maps has also become a standard tool for health services
analytical methods such as data mining and data visualisation; data research aiming to address unwarranted variation in healthcare.24
science is becoming mainstream in public health, integrating knowl- Data visualisation is also gaining importance for displaying complex
edge and skills from informatics and biostatistics. One major chal- longitudinal data from EHRs25 (see also Electronic medical record).
lenge in the analysis of big data is to account for the low quality, the One major change is the possibility of tailoring visualisation surveil-
poor data consistency across setting and time and the lack of meta- lance output to users’ needs through interactive data visualisation.22
data (see also Source population and selectivity bias). The question-
able ‘veracity’ (the fourth ‘V’) of big data refers actually to its poor
ETHICS OF PUBLIC HEALTH SURVEILLANCE AND PRIVACY
quality and high noise. Of critical importance is to go from big to
‘smart’ data, that is, data that can be transformed into information. PROTECTION
While the development of big data and related data-science meth- In 2017, the WHO issued international ethics guidelines on public
ods opens the way to data-informed or data-driven healthcare and health surveillance.26,27 Surveillance activities raise ethical issues due
public health,12 it also raises major concerns about privacy protec- to data collection methods, notably when the identity of individuals
tion (see also Ethics of public health surveillance and privacy protec- is recorded. More broadly, it is necessary to account for the balance
tion). At the policy level, the use of big data for surveillance raises between the protection of privacy and the benefits at a population
issues of access and benefit sharing, accountability and transparency level. With the development of surveillance based on the analyses of
and quality and safety.13,14 medicoadministrative,6 social media or geospatial mobile phone
data, and with growing linkage possibilities, individual privacy pro-
DATA, INFORMATION, KNOWLEDGE AND WISDOM PYRAMID tection has become a major concern. The increasing sophistication
The data, information, knowledge and wisdom (DIKW) pyramid is and broadening possibilities for data linkage put at risk data manage-
a framework to help understand the hierarchal relationships from ment transparency and accountability.13,14 The new European Union
data to wisdom.15 It has gained importance in public health mon- General Data Protection Regulation (GDPR) is the current legal
itoring, with the growing use of all types of data for surveillance framework for the collection of personal data in European
activities, notably to highlight that data do not speak by itself and countries18; it aims notably to give citizens more control over their
need to be transformed to become information, for example, in the own data and to harmonise data protection across Europe. The
form of health indicators,16,17 with the latter having to be contex- broad principles of GDPR include having a legitimate basis for data
tualised to become knowledge and eventually wisdom, for example, collection, purpose limitation, transparency, as much privacy and
to inform health policy decisions18 (see also figure 2). The DIKW data minimisation as possible and accountability for all data use.18
pyramid also highlights that surveillance is not the mere collection
and analysis of data, but a complex multilayer activity at the core of ELECTRONIC MEDICAL (EMR) OR HEALTH RECORD (EHR) AND
public health decision-making process, allowing evidence-informed PERSONAL HEALTH RECORD (PHR)
policy-making19 (see also Evidence based and data-informed public The increasing adoption of electronic records to manage medical
health). Recently, it has been proposed to review this pyramid, by and health data creates new opportunities for public health
deemphasising the notion of wisdom and by adding ‘evidence’ monitoring.28 An electronic medical record (EMR) is used to
between information and knowledge (DIEK)20; evidence emerges integrate, manage and analyse patient data collected in a clinical
through the comparison of information and is used to build action- context, often within one clinic or institution. An EHR is
able knowledge for public health. intended to have a broader scope, encompassing all health-
related data over the life course. A related concept is a personal
DATA MINING health record (PHR), which is an EHR controlled by a patient. In
The discovery of patterns in large data sets by drawing on a range of all cases, these records are useful for population monitoring to
methods from engineering, computer science and statistics is called the extent that they record concepts and health events in
data mining (see also Big data and data science). These methods are a consistent and unambiguous manner (eg, through the use of
applied in an automated or semiautomated manner, usually with no data standards and ontology29), which enables different systems
a priori specification of the pattern to be detected. In a health mon- to exchange data, or interoperate30 (See also Interoperability).
itoring context, some methods used for detecting aberrations or out- Major challenges remain such as how to define the denominators
breaks can be considered data mining methods5 (see also Aberration for events extracted from EHR.31
Chiolero A, Buckeridge D. J Epidemiol Community Health 2020;74:612–616. doi:10.1136/jech-2018-211654 613
 J Epidemiol Community Health: first published as 10.1136/jech-2018-211654 on 24 April 2020. Downloaded from http://jech.bmj.com/ on February 28, 2024 by guest. Protected by copyright.
Glossary
EVIDENCE-BASED AND DATA-INFORMED PUBLIC HEALTH standards such as Health Level Seven and Fast healthcare
At the crossroad between population health science2 and applied Interoperability Resource allow public health surveillance systems
public health research, public health surveillance is a core ele- to interoperate with laboratory systems and information exchange
ment of evidence-based public health (figure 3).32 Indeed, popu- standards such as Statistical Data and Metadata Exchange allow
lation assessment, production of indicators and reports and public health systems to interoperate with web-based systems to
evaluations are typical activities and outcomes of public health automate the dissemination of population-based indicators.
surveillance. Monitoring the literature is also an integral part of
surveillance, for example, to allow comparison and benchmark- MACHINE LEARNING, ARTIFICIAL INTELLIGENCE
ing or to challenge measurement and definition of indicators. In AI can be defined in terms of human intelligence, such that any
the age of data science, the management of surveillance data and machine that can act like a human is displaying AI.36 The ability of
information has gained importance in the evidence-based public a machine to perform any intellectual task is called Artificial
heath cycle, with the policy-making process becoming not only General Intelligence or Strong AI and is thought to require
evidence based but also data informed if not data driven. a range of skills, such as natural language processing, knowledge
Evidence-based public health should also guide how surveillance representation, automated reasoning, machine learning, computer
system is designed33 (see also Population health record). vision and robotics. Each of these skills is the subject of considerable
research in AI, employing different connectionist (ie, data driven) or
FORECASTING symbolic (ie, using logic and symbols) approaches. Recent algorith-
Data collected through surveillance are often analysed to identify mic advances have enabled profound gains in the performance of
important changes in population health. Inference about change neural networks for machine learning.37 In epidemiology and pub-
requires an estimate of the expected state of population health, lic health surveillance, machine learning is used as one tool to
which is obtained through forecasting, or predicting future popu- execute causal inference analysis, diagnosis and prognosis studies,
lation health status using data collected in the past. Many meth- genome-wide association studies, geospatial applications or
ods are available for forecasting, from a simple average of forecasting.38 Such machine learning methods also have the poten-
historical values to multivariate time-series methods.34 tial to advance aberration detection.5
Forecasting of expected values is a critical step in routine surveil-
lance for outbreaks and is also used to estimate the future burden POPULATION HEALTH RECORD
from chronic diseases and other prevalent conditions. The accu- The International Organization for Standardization (ISO) has
racy of a forecast usually decreases as the length of the horizon defined a population health record (PopHR) as a system analogous
increases and is usually evaluated by comparing forecasts to to an EHR but containing aggregated and usually deidentified data
actual values once data become available. Because the perfor- for public health and other epidemiological purposes.39 The con-
mance of predictive models depends on the quality and stability cept of the PopHR was subsequently developed further, noting that
(across eg, time and space) of data, forecasting methods must its primary purpose is to support efficient and effective public health
adapt to the relatively low quality and selectivity of big data (see practice, that it should be based on an explicit population health
also Source population and selectivity bias). framework and that it should make available indicators that docu-
ment the current status and influences of the health of a defined
INTEROPERABILITY population.40 While PopHR systems have yet to be adopted widely
Increasingly, public health surveillance draws data from a wide in public health practice, researchers have developed and imple-
range of sources and makes information available to many stake- mented demonstration systems,33 along with formal ontologies to
holders. This acquisition of data and dissemination of information support information integration in a PopHR.41
has traditionally been a manual process, but as volumes continue
to grow, automation of data and information exchange becomes
necessary. Such automation requires the definition and adoption
PRECISION PUBLIC HEALTH
Precision public health is inspired by precision medicine with the
of standards that indicate clearly how information systems should
idea that a better use of all types of data, encompassing geogra-
interact with one another or interoperate. The term semantic
phy, physical and sociodemographic characteristics, as well as
interoperability is used to define the ability for one information
health behaviours and biomarkers, at a local or community
system to receive data from another system and to reliably process
scale, would help design specific public health policy for a given
this data to produce information.35 For example, messaging
population, and be more effective than general policy.42,43 Some
have argued that the term is problematic, causing confusion with
the precision medicine movement and focusing attention on
individual diagnosis and treatment.44,45 Others have suggested
that precision public health merely rebrands modern public
health surveillance activities and adds little value.45

SECONDARY USE OF DATA

Surveillance activities are relying increasingly on the use of data
not specifically collected for that purpose, including data a priori
not related to health.46,47 The secondary use of data is not new in
surveillance, but it has grown in importance and depth, leading to
a paradigm shift in surveillance. Indeed, the classical approach is
(1) to define or choose the health problem for which surveillance
is necessary, (2) to define and collect the data needed and (3) to
Figure 3 Public health surveillance is a central element of evidence- analyse data to address your problem. Along this approach,
based public health. Inspired by Brownson et al 2009.32 ‘designed data’ specifically tailored to address surveillance goals

614 Chiolero A, Buckeridge D. J Epidemiol Community Health 2020;74:612–616. doi:10.1136/jech-2018-211654

 J Epidemiol Community Health: first published as 10.1136/jech-2018-211654 on 24 April 2020. Downloaded from http://jech.bmj.com/ on February 28, 2024 by guest. Protected by copyright.
Glossary
are used. The more contemporary data-driven approach is (1) to SYNDROMIC SURVEILLANCE
collect data from multiple source without knowing a priori what Case definitions based on syndromes can enhance the sensitivity
will be done with this data and (2) to analyse data to see if they and timeliness of surveillance. Around the turn of the millen-
could help solve surveillance problems. With this approach, nium, surveillance of syndromes was implemented on a large
‘organic data’ not specifically tailored for surveillance are used scale by applying automated algorithms to clinical data.56 The
(see also Big data).48 Designed and organic data have specific automated detection of syndromes in clinical data and by auto-
advantages and disadvantages. On the one hand, validity and mated statistical analysis to detect aberrations in the frequency of
reliability of designed data are often documented. Further, syndromes are defining characteristics of syndromic
designed data collection processes are defined and the ethical surveillance57 (see also Aberration detection). Although an early
and legal frameworks for collection are explicit; the lack of motivation for syndromic surveillance was rapid outbreak detec-
such clear frameworks for organic data is a major current issue tion, the use of non-specific, prediagnostic data can make it
(see also Ethics of public health surveillance and privacy protec- challenging to detect a signal quickly with an acceptable rate of
tion). On the other hand, resources needed to collect designed false alerts.58 Nonetheless, due to their potential to provide real-
data are larger than for organic data. Also, the reporting delay can time information about population health, syndromic surveil-
be shorter with organic data compared with designed data. lance systems routinely contribute to situational awareness in
However, the source population of organic data can be tricky to many public health systems and are often deployed for mass
identify (see also Source population and selectivity bias).31 gathering events.

SOURCE POPULATION AND SELECTIVITY BIAS

Public health surveillance aims to gather information on the CONCLUSION
health-related characteristics of a specific population, which Data-science and newly accessible data are driving innovation in
most often is a group of people living in a given location. methods for public health surveillance and monitoring, offering
More broadly, a population is a group of people sharing new opportunities. However, disappointment is also to be
a characteristic, such as a medical condition or treated in expected due to the challenge in extracting value from healthcare
specific healthcare facilities.2,49 With some types of big data, data which often lack consistent structure and clear meaning.59
one difficulty is to define the source population from which Fostering the ability of primary data providers to improve the
this data have emerged; completeness or representativeness structure and semantics of the data they collect can make it easier
of the supposedly source population cannot be ensured due to obtain meaningful information and, eventually, knowledge
to the non-probabilistic character of this data, resulting from these data. Stronger semantic interoperability between
from the selectivity of people from which data are health information systems35 and more consistent data structure
recorded.50,51 Routinely collected data are often event will be essential to help moving from big to smart data, that is,
based rather than population based, with no information data that can be used to produce information, and to transform
on the individuals who did not experience the event,46 and health systems which are currently data rich but information
the link between the event and the individual can be diffi- poor into systems which are data and information rich.60
cult to establish. Further, the source population can change Finally, while many resources are directed towards data collec-
very rapidly, for example, for sales, online and any other tion and processing, the resources and expertise needed to make
user-generated data, and in an unpredictable manner. As these data truly useful for surveillance, namely background
a result, denominators cannot be easily computed, and infer- knowledge on public health and on the processes generating the
ence beyond the study population is problematic, due to data,6 are critical more than ever in an age of data science;
a selectivity bias (see also Secondary use of data). knowledge brokers are needed to bridge data science, health
Selectivity bias is a term used to highlight the challenge of monitoring and public health.
identifying and defining the source population per se of big Contributors AC and BD both drafted the paper and reviewed it before submission.
data; it differs from selection bias which refers usually to
Funding The authors have not declared a specific grant for this research from any
a sampling issue, making the data used for the analysis funding agency in the public, commercial or not-for-profit sectors.
problematic for inference to the source or target population.
Competing interests None declared.
SURVEILLANCE BIAS Patient consent for publication Not applicable.
Many conditions and health-related events under surveillance are Provenance and peer review Commissioned; externally peer reviewed.
sensitive to the modality and intensity of detection activities, for Open access This is an open access article distributed in accordance with the
example, several types of cancer, thromboembolism or postopera- Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which
tive infections.52,53 Surveillance bias occurs when such conditions permits others to distribute, remix, adapt, build upon this work non-commercially, and
license their derivative works on different terms, provided the original work is properly
are sought with differential intensity across populations or over
cited, appropriate credit is given, any changes made indicated, and the use is non-
time, or according to care setting and patient characteristics.54,55 commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
As a result, the difference in the frequency (incidence, prevalent) of
the condition may not reflect a change in the risk of this condition, ORCID iD
Arnaud Chiolero http://orcid.org/0000-0002-5544-8510
but instead a difference in the frequency of detection. For instance,
between-hospital differences in the frequency of thromboembolism
REFERENCES
following hip surgery can reflect between-hospital differences in 1 Thacker SB, Berkelman RL. Public health surveillance in the United States. Epidemiol
postsurgery screening activities (large number of cases identified in Rev 1988;10:164–90.
hospitals with intense screening activities vs low number in other 2 Keyes KM, Galea S. Population health science. Oxford University Press, New York, 2016.
hospitals), rather than any difference in the quality of care.55 3 Nsubuga P, White ME, Thacker SB, et al. Public health surveillance: a tool for targeting
and monitoring interventions. In: Jamison DT, Breman JG, Measham AR, et al. eds. 2nd
A related concept is the ‘streetlight effect’ which occurs when ed. Disease control priorities in developing countries. Washington (DC): The
surveillance activities are not concentrated on what matters, but International Bank for Reconstruction and Development / The World Bank; New York:
on what is measurable, even if it is not relevant. Oxford University Press;, 2006.

Chiolero A, Buckeridge D. J Epidemiol Community Health 2020;74:612–616. doi:10.1136/jech-2018-211654 615

 J Epidemiol Community Health: first published as 10.1136/jech-2018-211654 on 24 April 2020. Downloaded from http://jech.bmj.com/ on February 28, 2024 by guest. Protected by copyright.
Glossary
4 Lee LM, Thacker SB. Public health surveillance and knowing about health in the context 33 Shaban-Nejad A, Lavigne M, Okhmatovskaia A, et al. PopHR: a knowledge-based
of growing sources of health data. Am J Prev Med 2011;41:636–40. platform to support integration, analysis, and visualization of population health data.
5 Yuan M, Boston-Fisher N, Luo Y, et al. A systematic review of aberration detection Ann N Y Acad Sci 2017;1387:44–53.
algorithms used in public health surveillance. J Biomed Inform 2019;94:103181. 34 Burkom HS, Murphy SP, Shmueli G. Automated time series forecasting for
6 Sarrazin MS, Rosenthal GE. Finding pure and simple truths with administrative data. biosurveillance. Stat Med 2007;26:4202–18.
JAMA 2012;307:1433–5. 35 Dixon BE, Vreeman DJ, Grannis SJ. The long road to semantic interoperability
7 Groseclose SL, Buckeridge DL. Public health surveillance systems: recent advances in in support of public health: experiences from two states. J Biomed Inform
their use and evaluation. Annu Rev Public Health 2017;38:57–79. 2014;49:3–8.
8 Faverjon C, Berezowski J. Choosing the best algorithm for event detection based on the 36 Panch T, Mattie H, Celi LA. The “inconvenient truth” about AI in healthcare. NPJ Digit
intended application: a conceptual framework for syndromic surveillance. J Biomed Med 2019;2:77.
Inform 2018;85:126–35. 37 Ardila D, Kiraly AP, Bharadwaj S, et al. End-to-end lung cancer screening with
9 Chiolero A, Anker D. Screening interval: a public health blind spot. Lancet Pub Health three-dimensional deep learning on low-dose chest computed tomography. Nat Med
2019;4:e171–2. 2019;25:954–61.
10 Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA 38 Bi Q, Goodman KE, Kaminsky J, et al. What is machine learning? A primer for the
2013;309:1351–2. epidemiologist. Am J Epidemiol 2019;188:2222–39.
11 Mooney SJ, Westreich DJ, El-Sayed AMJE. Epidemiology in the era of big data. 39 ISO/TR 20514. Health informatics - electronic health record - definition, scope, and
Epidemiology 2015;26:390. context. 2005.
12 Raghupathi W, Raghupathi V. Big data analytics in healthcare: promise and potential. 40 Friedman DJ, Parrish RG 2nd. The population health record: concepts, definition,
Health Inf Sci Syst 2014;2:3. design, and implementation. JAMA 2010;17:359–66.
13 Vayena E, Dzenowagis J, Brownstein JS, et al. Policy implications of big data in the 41 Shaban-Nejad A, Okhmatovskaia A, Izadi MT, et al. PHIO: a knowledge base for
health sector. Bull World Health Organ 2018;96:66–8. interpretation and calculation of public health indicators. Stud Health Technol Inform
14 Vayena E, Haeusermann T, Adjekum A, et al. Digital health: meeting the ethical and 2013;192:1207.
policy challenges. Swiss Med Wkly 2018;148:w14571. 42 Desmond-Hellmann S. Progress lies in precision. Science 2016;353:731.
15 Rowley J. The wisdom hierarchy: representations of the DIKW hierarchy. J Inf Sci 43 Dowell SF, Blazes D, Desmond-Hellmann S. Four steps to precision public health.
2007;33:163–80. Nature 2016;540:189–91.
16 Etches V, Frank J, Di Ruggiero E, et al. Measuring population health: a review of 44 Seeking precision in public health. Nat Med 2019;25:1177.
indicators. Annu Rev Public Health 2006;27:29–55. 45 Chowkwanyun M, Bayer R, Galea S. “Precision” public health - between novelty and
17 Chiolero A, Paccaud F, Fornerod L. [How to conduct public health surveillance? The hype. New Engl J Med 2018;379:1398–400.
example of the Observatoire Valaisan de la Sante in Switzerland]. Sante Publique 46 Jorm L. Routinely collected data as a strategic resource for research: priorities for
2014;26:75–84. methods and workforce. Public Health Res Pr 2015;25:e2541540.
18 Verschuuren M, Van Oers H. Population health monitoring: climbing the information 47 Benchimol EI, Smeeth L, Guttmann A, et al. The reporting of studies conducted using
pyramid. Switzerland, Springer, 2018. observational routinely-collected health data (RECORD) statement. PLoS Med
19 Oxman AD, Lavis JN, Lewin S, et al. SUPPORT tools for evidence-informed health 2015;12:e1001885.
policymaking (STP) 1: what is evidence-informed policymaking? Health Res Policy Syst 48 Ann Keller S, Koonin SE, Shipp SJS. Big data and city living: what can it do for us?
2009;7:S1. Significance 2012;9:4–7.
20 Dammann O. Data, information, evidence, and knowledge:: a proposal for health 49 Keyes KM, Galea S. Setting the agenda for a new discipline: population health science.
informatics and data science. Online J Public Health Inform 2018;10:e224. Am J Public Health 2016;106:633–4.
21 Jensen PB, Jensen LJ, Brunak S. Mining electronic health records: towards better 50 Buelens B, Daas P, Burger J, et al. Selectivity of big data: statistics Netherlands. The
research applications and clinical care. Nat Rev Genet 2012;13:395–405. Netherlands: The Hague/Heerlen, 2014.
22 O’Donoghue SI, Baldi BF, Clark SJ, et al. Visualization of biomedical data. JAMIA Open 51 Beresewicz M, Lehtonen RT, Reis F, et al. An overview of methods for treating
2018;1:275–304. selectivity in big data sources. Eurostat. Luxembourg: Publications Office of the
23 Hay SI, George DB, Moyes CL, et al. Big data opportunities for global infectious disease European Union, 2018.
surveillance. PLoS Med 2013;10:e1001413. 52 Welch HG, Brawley OW. Scrutiny-dependent cancer and self-fulfilling risk factors. Ann
24 Birkmeyer JD, Reames BN, McCulloch P, et al. Understanding of regional variation in Intern Med. 2018;168:143–5.
the use of surgery. Lancet 2013;382:1121–9. 53 Welch HG, Kramer BS, Black WC. Epidemiologic signatures in cancer. New Engl J Med
25 West VL, Borland D, Hammond WE. Innovative information visualization of electronic 2019;381:1378–86.
health record data: a systematic review. JAMA 2015;22:330–9. 54 Chiolero A, Santschi V, Paccaud F. Public health surveillance with electronic medical
26 World Health Organization. WHO guidelines on ethical issues in public health surveil- records: at risk of surveillance bias and overdiagnosis. Eur J Public Health
lance. Geneva: World Health Organization 2017. 2013;23:350–1.
27 Fairchild AL, Haghdoost AA, Bayer R, et al. Ethics of public health surveillance: new 55 Haut ER, Pronovost PJ. Surveillance bias in outcomes reporting. JAMA
guidelines. Lancet Pub Health 2017;2:e348–9-. 2011;305:2462–3.
28 Klompas M, McVetta J, Lazarus R, et al. Integrating clinical practice and public health 56 Mandl KD, Overhage JM, Wagner MM, et al. Implementing syndromic surveillance:
surveillance using electronic medical record systems. Am J Public Health 2012;102: a practical guide informed by the early experience. J Am Med Inform Assn
S325–32. 2004;11:141–50.
29 Gonzalez C, Blobel BG, Lopez DM. Ontology-based framework for electronic health 57 Soler MS, Fouillet A, Viso AC, et al. Assessment of syndromic surveillance in Europe.
records interoperability. Stud Health Technol Inform 2011;169:694–8. Lancet 2011;378:1833–4.
30 Moreno Conde A. Quality framework for semantic interoperability in health informatics: 58 Buckeridge DL. Outbreak detection through automated surveillance: a review of the
definition and implementation. London, UK: UCL (University College London), 2016. determinants of detection. J Biomed Inform 2007;40:370–9.
31 Cocoros NM, Ochoa A, Eberhardt K, et al. Denominators matter: understanding 59 Greene JA, Lea AS. Digital futures past - The long arc of big data in medicine. New Engl
medical encounter frequency and its impact on surveillance estimates using EHR data. J Med 2019;381:480–5.
EGEMS 2019;7:31. 60 OCDE (2019), Health in the 21st Century : Putting Data to Work for Stronger Health
32 Brownson RC, Fielding JE, Maylahn CM. Evidence-based public health: a fundamental Systems, OECD Health Policy Studies, Éditions OCDE, Paris, https://doi.org/10.1787/
concept for public health practice. Annu Rev Public Health 2009;30:175–201. e3b23f8e-en.

616 Chiolero A, Buckeridge D. J Epidemiol Community Health 2020;74:612–616. doi:10.1136/jech-2018-211654