Advancing the Framework for
Assessing Causality of Health
and Welfare Effects to Inform
National Ambient Air Quality
Standard Reviews
_____
Committee on Assessing Causality from a
Multidisciplinary Evidence Base for National
Ambient Air Quality Standards
Board on Environmental Studies and Toxicology
Division on Earth and Life Studies
Board on Mathematical Sciences and Analytics
Division on Engineering and Physical Sciences
Consensus Study Report
NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This activity was supported by a contract between the National Academy of Sciences and the U.S. Environmental Protection Agency. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.
International Standard Book Number-13: 978-0-309-69011-9
International Standard Book Number-10: 0-309-69011-0
Digital Object Identifier: https://doi.org/10.17226/26612
This publication is available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2022 by the National Academy of Sciences. National Academies of Sciences, Engineering, and Medicine and National Academies Press and the graphical logos for each are all trademarks of the National Academy of Sciences. All rights reserved.
Printed in the United States of America.
Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2022. Advancing the Framework for Assessing Causality of Health and Welfare Effects to Inform National Ambient Air Quality Standard Reviews. Washington, DC: The National Academies Press. https://doi.org/10.17226/26612.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org.
Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task.
Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies.
Rapid Expert Consultations published by the National Academies of Sciences, Engineering, and Medicine are authored by subject-matter experts on narrowly focused topics that can be supported by a body of evidence. The discussions contained in rapid expert consultations are considered those of the authors and do not contain policy recommendations. Rapid expert consultations are reviewed by the institution before release.
For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo.
COMMITTEE ON ASSESSING CAUSALITY FROM A MULTIDISCIPLINARY EVIDENCE BASE FOR NATIONAL AMBIENT AIR QUALITY STANDARDS
Members
ARMISTEAD G. RUSSELL (Co-Chair), Georgia Institute of Technology
ELIZABETH A. STUART (Co-Chair), Johns Hopkins Bloomberg School of Public Health
JILL S. BARON, U.S. Geological Survey and Colorado State University
JOAN WENNSTROM BENNETT, Rutgers University (until November 2021)
KIROS T. BERHANE, Columbia University Mailman School of Public Health
EDMUND A. C. CROUCH, Green Toxicology, LLC
IVÁN DÍAZ, New York University Langone Health
FRANCESCA DOMINICI, Harvard T.H. Chan School of Public Health
CHARLES T. DRISCOLL, Syracuse University
JOEL KAUFMAN, University of Washington
JANA B. MILFORD, University of Colorado Boulder
RICHARD L. POIROT, Private Air Quality/Climate Consultant
RICHARD L. SMITH, University of North Carolina
JUSTIN G. TEEGUARDEN, Pacific Northwest National Laboratory
Staff
SAMMANTHA MAGSINO, Project Director
LYLY LUHACHECK, Associate Program Officer
LESLIE BEAUCHAMP, Senior Program Assistant
KALEY BEINS, Program Officer (until July 2021)
TYLER KLOEFKORN, Senior Program Officer (until September 2021)
CARL-GUSTAV ANDERSON, Program Officer (until February 2022)
CHRISTOPHER REA, Program Officer (until February 2021)
TAMARA DAWSON, Program Coordinator (until August 2021)
MELISSA FRANKS, Senior Program Assistant (until May 2021)
Sponsor
U.S. ENVIRONMENTAL PROTECTION AGENCY
BOARD ON ENVIRONMENTAL STUDIES AND TOXICOLOGY
Members
FRANK W. DAVIS (Chair), University of California, Santa Barbara
ANN M. BARTUSKA, U.S. Department of Agriculture
DANA BOYD BARR, Emory University
GERMAINE M. BUCK LOUIS, George Mason University
FRANCESCA DOMINICI (NAM), Harvard University
GEORGE M. GRAY, The George Washington University
R. J. LEWIS, ExxonMobil Biomedical Sciences, Inc.
LINSEY C. MARR, Virginia Polytechnic Institute and State
MARIE L. MIRANDA, Children’s Environmental Health Initiative
R. CRAIG POSTLEWAITE, U.S. Department of Defense
REZA J. RASOULPOUR, Corteva Agriscience
JOSHUA TEWKSBURY, Smithsonian Tropical Research Institute
SACOBY M. WILSON, University of Maryland, College Park
TRACEY J. WOODRUFF, University of California, San Francisco
Staff
CLIFFORD DUKE, Director
RAYMOND A. WASSEL, Scholar and Director of Environmental Studies
KATHRYN GUYTON, Senior Program Officer
NATALIE ARMSTRONG, Associate Program Officer
LAURA LLANOS, Finance Business Partner
LESLIE BEAUCHAMP, Senior Program Assistant
THOMASINA LYLES, Senior Program Assistant
BOARD ON MATHEMATICAL SCIENCES AND ANALYTICS
Members
MARK L. GREEN (Chair), University of California, Los Angeles
HELENE BARCELO, Mathematical Sciences Research Institute
BONNIE BERGER, Massachusetts Institute of Technology
RUSSEL E. CAFLISCH, New York University
DAVID S. CHU, Institute for Defense Analyses
DUANE COOPER, Morehouse College
JAMES H. CURRY, University of Colorado Boulder
RONALD D. FRICKER, JR., Virginia Polytechnic Institute and State University
LYDIA E. KAVRAKI, Rice University
TAMARA G. KOLDA, MathSci.ai
PETROS KOUMOUTSAKOS, Harvard University
RACHEL KUSKE, Georgia Institute of Technology
YANN A. LECUN, Facebook and New York University
JILL C. PIPHER, Brown University
YORAM SINGER, Princeton University
TATIANA TORO, University of Washington, Seattle
LANCE A. WALLER, Emory University
AMIE WILKINSON, University of Chicago
KAREN WILLCOX, Oden Institute for Computational Engineering and Sciences and The University of Texas at Austin
Staff
MICHELLE K. SCHWALBE, Director
BRITTANY SEGUNDO, Program Officer
SAMANTHA KORETSKY, Research Assistant
HEATHER LOZOWSKI, Senior Finance Business Partner
JOSEPH PALMER, Senior Project Assistant
This page intentionally left blank.
Reviewers
This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
We thank the following individuals for their review of this report:
MARIE-ABÈLE BIND, Harvard University and Massachusetts General Hospital
LOUIS ANTHONY COX, Cox Associates
NEIL DONAHUE, Carnegie Mellon University
RODERICK LITTLE, University of Michigan
ROGER McCLELLAN, Independent Advisor
JON SAMET, University of Colorado
KATHLEEN WEATHERS, Cary Institute of Ecosystem Studies
Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report nor did they see the final draft before its release. The review of this report was overseen by DAVID EATON, University of Washington, and MARK UTELL, University of Rochester School of Medicine and Dentistry. They were responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring committee and the National Academies.
This page intentionally left blank.
Contents
Interpretation of Task and Study Boundaries
Committee Information Gathering
2 HISTORICAL AND LEGAL PERSPECTIVES ON THE NAAQS FRAMEWORK FOR ASSESSING CAUSALITY
Legal Background for the NAAQS
Historical Perspective on the 2015 Preamble Framework
Defining Causal Effects: Potential Outcomes
Definition of “Cause” from the Preamble
Evaluating Causality Through Mechanistic Understanding
4 INFLUENCE OF THE INDIVIDUAL STUDY ON THE BODY OF EVIDENCE
Study Selection and Evaluation in EPA’s Causal Determination Framework
Types of Evidence Used in Integrated Science Assessments to Support Causal Determinations
Transparency, Reproducibility, and Replicability in Individual Studies
5 EVIDENCE SYNTHESIS IN INTEGRATED SCIENCE ASSESSMENTS
Current Approach to Evidence Synthesis and Integration for Health Effects
Integration and Synthesis Examples for Health Effects from the 2020 Ozone ISA
Current Approach to Evidence Synthesis and Integration for Welfare Effects
Integration and Synthesis Examples for Welfare from the 2020 Ozone ISA
Integration and Synthesis Examples from the 2020 NOx-SOx-PM ISA
Evolution of the ISA Process Since Publication of the Preamble
6 EXAMPLE CRITIQUES OF THE ISA PROCESS AND CAUSAL DETERMINATION FRAMEWORK
CASAC Comments from Recent ISA Reviews
Critiques Related to the Weight of Evidence Approach
Critiques Related to Systematic Review
Critiques Related to the Causal Categories
Critiques Related to Populations at Increased Risk and Selection of Endpoints
Critiques Related to External Review
Synthesis of Critiques and Reviews of the ISA Framework
7 ILLUSTRATIVE FRAMEWORKS FOR CAUSAL DETERMINATIONS
Preamble to the Integrated Science Assessments (ISAs) (2015a)
Toxic Substances Control Act (TSCA)
World Health Organization (WHO)
Integrated Risk Information System (IRIS)
Office of Health Assessment and Translation (OHAT)
International Agency for Research on Cancer (IARC)
United Nations Economic Commission for Europe (UNECE)
Canadian Smog Science Assessment (CSSA)
Synthesis: What Might Be Useful in the ISA Causal Determination Framework
8 EMERGING METHODS FOR OBSERVATIONAL STUDIES CONTRIBUTING TO CAUSAL DETERMINATIONS
Advances in Methods for Exposure Assessment
Clarifying and Assessing the Validity of Causal Assumptions in Observational Studies
Assessing the Quality of Individual Studies with Respect to Handling Measured Confounders
Methods for Handling Unmeasured Confounders and Post-Treatment Variables
This page intentionally left blank.