Implementing Machine Learning at State Departments of Transportation: A Guide (2024)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

References AASHTO. (n.d.). AASHTO: Transportation Systems Management and Operations Guidance. http://aashtotsmo guidance.org/. Brownlee, J. (2019). How Much Training Data Is Required for Machine Learning? Machine Learning Mastery. MachineLearningMastery.Com. https://machinelearningmastery.com/much-training-data-required-machine -learning/. Cetin, M., S. Ishak, H. Townsend, M. Samach, K. Ozbay, et al. (2024). NCHRP Web-Only Document 404: Imple- menting and Leveraging Machine Learning at State Departments of Transportation. Transportation Research Board, Washington, DC. Defense Innovation Unit (DIU). (2022a). Phase 1: Planning Worksheet for DIU RAI Guidelines. Department of Defense (DOD). https://assets.ctfassets.net/3nanhbfkr0pc/1vJvimVkijueLbJzqcaOMr/691bcab582aafeb6de 3f18b9266a6294/Planning_Worksheet_DIU-AI_Guidelines.pdf. Defense Innovation Unit (DIU). (2022b). Phase 3: Deployment Worksheet for DIU RAI Guidelines. Depart- ment of Defense (DOD). https://assets.ctfassets.net/3nanhbfkr0pc/1qm3pz4U9KkyPAuz1RQsfh/f24d8d 8746413242de04a7d999ea0b96/Deploy_Worksheet_DIU-AI_Guidelines.pdf. Defense Innovation Unit (DIU). (n.d.). Responsible AI Guidelines. https://www.diu.mil/responsible-ai-guidelines. Federal Highway Administration (FHWA). (2023). TPM Toolbox. U.S. DOT. https://www.tpmtools.org/. Federal Highway Administration (FHWA) Office of Operations. (2020). Operations Benefit/Cost Analysis Desk Reference, Chapter 1: Introduction. U.S. DOT. https://ops.fhwa.dot.gov/publications/fhwahop12028/sec1.htm. Federal Highway Administration (FHWA) Office of Operations. (2016). Factsheet: Capability Maturity Frame- works for Transportation Systems Management and Operations Program Areas. U.S. DOT. https://ops.fhwa .dot.gov/publications/fhwahop16031/index.htm. General Services Administration (GSA) IT Modernization Centers of Excellence. (n.d.). Chapter 6: AI Capability Maturity. https://coe.gsa.gov/coe/ai-guide-for-government/ai-capability-maturity/index.html. Gettman, D. (2019). Raising Awareness of Artificial Intelligence for Transportation Systems Management and Operations. FHWA-HOP-19-052. Federal Highway Administration, Office of Operations. https://rosap.ntl .bts.gov/view/dot/44201. Government Accountability Office (GAO). (2021). Artificial Intelligence: An Accountability Framework for Federal Agencies and Other Entities. GAO-21-519SP. https://www.gao.gov/products/gao-21-519sp. Gutta, S. (2020). The 5 V’s of Big Data. Medium. Analytics Vidhya. https://medium.com/analytics-vidhya/the -5-vs-of-big-data-2758bfcc51d. INFORMS. (2023). Analytics Maturity Model. https://analyticsmaturity.informs.org/. INFORMS. (n.d.). Certified Analytics Professional (CAP) and Associate Certified Analytics Professional (aCAP) Examination Study Guide. INFORMS Organization Support Resources Subcommittee. (2019). How Organizations Can Get Started With Analytics: Including Data Science, Machine Learning, Operations Research, and Artificial Intelligence. Catonsville, MD. https://www.informs.org/Explore/Building-Successful-O.R.-and-Analytics-Teams. ITS Deployment Evaluation. (2022a). Artificial Intelligence based Roadway Safety and Work Zone Detection Technology in Nevada Uncovered 20 Percent More Crashes Than Previously Reported and Reduced Crash Response Times by Nine to Ten Minutes on Average. U.S. DOT ITS Joint Program Office. https://www.itskrs .its.dot.gov/2022-b01642. ITS Deployment Evaluation. (2022b). Leveraging Existing Infrastructure and Computer Vision for Pedestrian Detection. U.S. DOT ITS Joint Program Office. https://www.itskrs.its.dot.gov/decision-support/case-study /leveraging-existing-infrastructure-and-computer-vision-pedestrian. 73 

74   Implementing Machine Learning at State Departments of Transportation: A Guide ITS Deployment Evaluation. (2022c). Video-Based Advanced Analytics is Detailed Enough to Identify Near- Crashes, Classify Road User Types, and Detect Speeding Infractions and Lane Violations. U.S. DOT ITS Joint Program Office. https://www.itskrs.its.dot.gov/2022-b01617. ITS Deployment Evaluation. (2021). The Cost To Implement Video Data Storage and Networking Services to Support 68 Traffic Cameras in New York City Was Estimated at $1,663 per Year Annualized Over Three Years. U.S. DOT ITS Joint Program Office. https://www.itskrs.its.dot.gov/node/209597. ITS Deployment Evaluation. (n.d.). ROI Use Cases: Adaptive Signal Control. U.S. DOT ITS Joint Program Office. https://www.itskrs.its.dot.gov/sites/default/files/doc/Adaptive_Traffic_Signals_BCA_Use_Case_20220708 _Final_508.pdf. ITS Joint Program Office. (n.d.). A Guide for Leveraging ITS Evaluation Tools for Benefit-Cost Analysis (BCA) and Return-on-Investment (ROI). ITS Deployment Evaluation. U.S. DOT. https://www.itskrs.its.dot.gov /decision-support/roi. Lane, I. F., R. Ghani, R. S. Jarmin, F. Kreuter, and J. Lane. (2021). Big Data and Social Science: Data Science Methods and Tools for Research and Practice. Second Edition. Boca Raton, FL: CRC Press. https://textbook .coleridgeinitiative.org/. Lee, T. B. (2020). Machine Vision Systems That Enable Self-Driving Taxis Cost Approximately $10,000 to $15,000 per Vehicle. U.S. DOT Joint Program Office ITS Deployment Evaluation. https://www.itskrs.its.dot.gov/node /209190. Liu, C. (2022). More Performance Evaluation Metrics for Classification Problems You Should Know. KDnuggets. https://www.kdnuggets.com/more-performance-evaluation-metrics-for-classification-problems-you-should -know. Mitsa, T. (2019). How Do You Know You Have Enough Training Data? Medium. https://towardsdatascience.com /how-do-you-know-you-have-enough-training-data-ad9b1fd679ee. Molnar, C. (2023). Interpretable Machine Learning: A Guide For Making Black Box Models Explainable. Second Edition. https://christophm.github.io/interpretable-ml-book/. Newberry, J. (2024). Innovative Artificial Intelligence and Machine Learning Strategies in State Departments of Transportation. Lectern Session 2006: State of the Art and Future Vision on Artificial Intelligence Research and Applications in Transportation. Transportation Research Board (TRB) Annual Meeting 2024, Washington, DC,. Office of the Secretary (2022). Benefit Cost Analysis Guidance for Discretionary Grant Programs. U.S. DOT. https://www.transportation.gov/sites/dot.gov/files/2022-03/Benefit%20Cost%20Analysis%20Guidance %202022%20%28Revised%29.pdf. Ozbay, K., J. Gao, F. Zuo, H. Yang, and A. Kurkcu. (2021). Reference-Free Video-to-Real Distance Approximation- Based Pedestrian Detection System Amid COVID-19 Pandemic. C2 SMART. https://c2smart.engineering .nyu.edu/wp-content/uploads/C2SMART-Final-Report-Social-Distancing-Pedestrian-Detection-2021.pdf. Payghode, V., A. Goyal, A. Bhan, S. S. Iyer, and A. K. Dubey. (2023). Object Detection and Activity Recognition in Video Surveillance Using Neural Networks. International Journal of Web Information Systems, Vol. 19, No. 3/4: 123–138. https://doi.org/10.1108/IJWIS-01-2023-0006. Pecheux, K. K., B. B. Pecheux, G. Ledbetter, and C. Lambert. (2020). NCHRP Research Report 952: Guide- book for Managing Data from Emerging Technologies for Transportation. Transportation Research Board, Washington, DC. https://nap.nationalacademies.org/catalog/25844/guidebook-for-managing-data-from -emerging-technologies-for-transportation. Pitropakis, N., E. Panaousis, T. Giannetsos, E. Anastasiadis, and G. Loukas. (2019). A Taxonomy and Survey of Attacks Against Machine Learning. Computer Science Review 34: 100199. https://doi.org/10.1016/j.cosrev .2019.100199. Purves, D. (2019). What Does AI’s Success Playing Complex Board Games Tell Brain Scientists? Proceedings of the National Academy of Sciences Vol. 116, No. 30: 14785–14787. https://doi.org/10.1073/pnas.1909565116. Rigaki, M., and S. Garcia. (2023). A Survey of Privacy Attacks in Machine Learning. ACM Computing Surveys Vol. 56, No. 4: 101:1–101:34. https://doi.org/10.1145/3624010. SAS Insights staff. (n.d.). 8 Ways an Enterprise Data Strategy Enables Big Data Analytics. https://www.sas.com /en_us/insights/articles/data-management/8-ways-an-enterprise-data-strategy-enables-big-data-analytics .html. Silver, D., T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, et al. (2017). Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. arXiv preprint arXiv: 1712.01815. Spy Pond Partners, LLC, and Atkins North America Inc. (2021). NCHRP Research Report 956: Guidebook for Data and Information Systems for Transportation Asset Management. Transportation Research Board, Washington, DC. Spy Pond Partners, LLC, and Iteris Inc. (2015). NCHRP Research Report 814: Data to Support Transportation Agency Business Needs: A Self-Assessment Guide. Transportation Research Board, Washington, DC. https:// www.trb.org/Main/Blurbs/173470.aspx. 

References  75 Spy Pond Partners, LLC, MLP LLC, and High Street Consulting Group. (2019). NCHRP Project 08-108: Devel- oping National Performance Management Data Strategies to Address Data Gaps, Standards, and Quality. Final Report. Transportation Research Board, Washington, DC. https://onlinepubs.trb.org/onlinepubs/nchrp /nchrp_rpt_920NPM.pdf. Steier, D. (2021). Developing and Implementing an Enterprise AI Strategy. Managing AI in Transportation, Carnegie Mellon University Executive Education Course. TensorFlow. (2023). Introduction to Fairness Indicators. Google’s TensorFlow Python package. https://www .tensorflow.org/responsible_ai/fairness_indicators/tutorials/Fairness_Indicators_Example_Colab. U.S. Department of State. (2023). Enterprise Artificial Intelligence Strategy FY2024–FY2025: Empowering Diplomacy through Responsible AI. https://www.state.gov/wp-content/uploads/2023/11/Department-of -State-Enterprise-Artificial-Intelligence-Strategy.pdf. U.S. Office of Management and Budget. (2020). Federal Data Strategy Framework: Mission, Principles, Practices, and Actions. https://strategy.data.gov/assets/docs/2020-federal-data-strategy-framework.pdf. Vasudevan, M., J. O’Hara, H. Townsend, S. Asare, S. Muhammad, K. Ozbay, D. Yang, J. Gao, A. Kukcu, and F. Zuo. (2022a). NCHRP Research Report 997: Algorithms to Convert Basic Safety Messages into Traffic Measures. Transportation Research Board, Washington, DC. https://doi.org/10.17226/26840. Vasudevan, M., H. Townsend, M. Samach, A. Ali, P. Walsh, P. Wang, A. Seshadri, and I. McManus. (2022b). Artificial Intelligence (AI) for Intelligent Transportation Systems (ITS): Challenges and Potential Solutions, Insights, and Lessons Learned. FHWA-JPO-22-968. U.S. DOT Intelligent Transportation Systems Joint Program Office. https://rosap.ntl.bts.gov/view/dot/66971. Vasudevan, M., H. Townsend, and E. Schweikert. (2020). Identifying Real-World Transportation Applications Using Artificial Intelligence (AI): Plan for Artificial Intelligence for Intelligent Transportation Systems. FHWA-JPO-20-813. U.S. DOT Intelligent Transportation Systems Joint Program Office. https://rosap.ntl.bts .gov/view/dot/53932. Wyatt, B. (2024). Building a Community of Practice Around Roadway Data. Texas DOT Presentation, Webinar on Data in Roadway Digital Infrastructure, Digital Infrastructure Webinar Series. ITS America. https://itsa .org/event/di-webinar-series-data-in-roadway-digital-infrastructure/.