-
Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving
Authors:
Bernard Lange,
Masha Itkina,
Jiachen Li,
Mykel J. Kochenderfer
Abstract:
Environment prediction frameworks are critical for the safe navigation of autonomous vehicles (AVs) in dynamic settings. LiDAR-generated occupancy grid maps (L-OGMs) offer a robust bird's-eye view for the scene representation, enabling self-supervised joint scene predictions while exhibiting resilience to partial observability and perception detection failures. Prior approaches have focused on det…
▽ More
Environment prediction frameworks are critical for the safe navigation of autonomous vehicles (AVs) in dynamic settings. LiDAR-generated occupancy grid maps (L-OGMs) offer a robust bird's-eye view for the scene representation, enabling self-supervised joint scene predictions while exhibiting resilience to partial observability and perception detection failures. Prior approaches have focused on deterministic L-OGM prediction architectures within the grid cell space. While these methods have seen some success, they frequently produce unrealistic predictions and fail to capture the stochastic nature of the environment. Additionally, they do not effectively integrate additional sensor modalities present in AVs. Our proposed framework performs stochastic L-OGM prediction in the latent space of a generative architecture and allows for conditioning on RGB cameras, maps, and planned trajectories. We decode predictions using either a single-step decoder, which provides high-quality predictions in real-time, or a diffusion-based batch decoder, which can further refine the decoded frames to address temporal consistency issues and reduce compression losses. Our experiments on the nuScenes and Waymo Open datasets show that all variants of our approach qualitatively and quantitatively outperform prior approaches.
△ Less
Submitted 30 July, 2024;
originally announced July 2024.
-
ASTPrompter: Weakly Supervised Automated Language Model Red-Teaming to Identify Likely Toxic Prompts
Authors:
Amelia F. Hardy,
Houjun Liu,
Bernard Lange,
Mykel J. Kochenderfer
Abstract:
Typical schemes for automated red-teaming large language models (LLMs) focus on discovering prompts that trigger a frozen language model (the defender) to generate toxic text. This often results in the prompting model (the adversary) producing text that is unintelligible and unlikely to arise. Here, we propose a reinforcement learning formulation of the LLM red-teaming task which allows us to disc…
▽ More
Typical schemes for automated red-teaming large language models (LLMs) focus on discovering prompts that trigger a frozen language model (the defender) to generate toxic text. This often results in the prompting model (the adversary) producing text that is unintelligible and unlikely to arise. Here, we propose a reinforcement learning formulation of the LLM red-teaming task which allows us to discover prompts that both (1) trigger toxic outputs from a frozen defender and (2) have low perplexity as scored by the defender. We argue these cases are most pertinent in a red-teaming setting because of their likelihood to arise during normal use of the defender model. We solve this formulation through a novel online and weakly supervised variant of Identity Preference Optimization (IPO) on GPT-2 and GPT-2 XL defenders. We demonstrate that our policy is capable of generating likely prompts that also trigger toxicity. Finally, we qualitatively analyze learned strategies, trade-offs of likelihood and toxicity, and discuss implications. Source code is available for this project at: https://github.com/sisl/ASTPrompter/.
△ Less
Submitted 12 July, 2024;
originally announced July 2024.
-
The Ethics of Advanced AI Assistants
Authors:
Iason Gabriel,
Arianna Manzini,
Geoff Keeling,
Lisa Anne Hendricks,
Verena Rieser,
Hasan Iqbal,
Nenad Tomašev,
Ira Ktena,
Zachary Kenton,
Mikel Rodriguez,
Seliem El-Sayed,
Sasha Brown,
Canfer Akbulut,
Andrew Trask,
Edward Hughes,
A. Stevie Bergman,
Renee Shelby,
Nahema Marchal,
Conor Griffin,
Juan Mateos-Garcia,
Laura Weidinger,
Winnie Street,
Benjamin Lange,
Alex Ingerman,
Alison Lentz
, et al. (32 additional authors not shown)
Abstract:
This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user, across one or more domains, in line with the user's expectations. The paper starts by considering the technology itself, pro…
▽ More
This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user, across one or more domains, in line with the user's expectations. The paper starts by considering the technology itself, providing an overview of AI assistants, their technical foundations and potential range of applications. It then explores questions around AI value alignment, well-being, safety and malicious uses. Extending the circle of inquiry further, we next consider the relationship between advanced AI assistants and individual users in more detail, exploring topics such as manipulation and persuasion, anthropomorphism, appropriate relationships, trust and privacy. With this analysis in place, we consider the deployment of advanced assistants at a societal scale, focusing on cooperation, equity and access, misinformation, economic impact, the environment and how best to evaluate advanced AI assistants. Finally, we conclude by providing a range of recommendations for researchers, developers, policymakers and public stakeholders.
△ Less
Submitted 28 April, 2024; v1 submitted 24 April, 2024;
originally announced April 2024.
-
A Framework for Assurance Audits of Algorithmic Systems
Authors:
Khoa Lam,
Benjamin Lange,
Borhane Blili-Hamelin,
Jovana Davidovic,
Shea Brown,
Ali Hasan
Abstract:
An increasing number of regulations propose AI audits as a mechanism for achieving transparency and accountability for artificial intelligence (AI) systems. Despite some converging norms around various forms of AI auditing, auditing for the purpose of compliance and assurance currently lacks agreed-upon practices, procedures, taxonomies, and standards. We propose the criterion audit as an operatio…
▽ More
An increasing number of regulations propose AI audits as a mechanism for achieving transparency and accountability for artificial intelligence (AI) systems. Despite some converging norms around various forms of AI auditing, auditing for the purpose of compliance and assurance currently lacks agreed-upon practices, procedures, taxonomies, and standards. We propose the criterion audit as an operationalizable compliance and assurance external audit framework. We model elements of this approach after financial auditing practices, and argue that AI audits should similarly provide assurance to their stakeholders about AI organizations' ability to govern their algorithms in ways that mitigate harms and uphold human values. We discuss the necessary conditions for the criterion audit and provide a procedural blueprint for performing an audit engagement in practice. We illustrate how this framework can be adapted to current regulations by deriving the criteria on which bias audits can be performed for in-scope hiring algorithms, as required by the recently effective New York City Local Law 144 of 2021. We conclude by offering a critical discussion on the benefits, inherent limitations, and implementation challenges of applying practices of the more mature financial auditing industry to AI auditing where robust guardrails against quality assurance issues are only starting to emerge. Our discussion -- informed by experiences in performing these audits in practice -- highlights the critical role that an audit ecosystem plays in ensuring the effectiveness of audits.
△ Less
Submitted 28 May, 2024; v1 submitted 26 January, 2024;
originally announced January 2024.
-
Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments
Authors:
Bernard Lange,
Jiachen Li,
Mykel J. Kochenderfer
Abstract:
Navigating complex and dynamic environments requires autonomous vehicles (AVs) to reason about both visible and occluded regions. This involves predicting the future motion of observed agents, inferring occluded ones, and modeling their interactions based on vectorized scene representations of the partially observable environment. However, prior work on occlusion inference and trajectory predictio…
▽ More
Navigating complex and dynamic environments requires autonomous vehicles (AVs) to reason about both visible and occluded regions. This involves predicting the future motion of observed agents, inferring occluded ones, and modeling their interactions based on vectorized scene representations of the partially observable environment. However, prior work on occlusion inference and trajectory prediction have developed in isolation, with the former based on simplified rasterized methods and the latter assuming full environment observability. We introduce the Scene Informer, a unified approach for predicting both observed agent trajectories and inferring occlusions in a partially observable setting. It uses a transformer to aggregate various input modalities and facilitate selective queries on occlusions that might intersect with the AV's planned path. The framework estimates occupancy probabilities and likely trajectories for occlusions, as well as forecast motion for observed agents. We explore common observability assumptions in both domains and their performance impact. Our approach outperforms existing methods in both occupancy prediction and trajectory prediction in partially observable setting on the Waymo Open Motion Dataset.
△ Less
Submitted 8 March, 2024; v1 submitted 25 September, 2023;
originally announced September 2023.
-
Engaging Engineering Teams Through Moral Imagination: A Bottom-Up Approach for Responsible Innovation and Ethical Culture Change in Technology Companies
Authors:
Benjamin Lange,
Geoff Keeling,
Amanda McCroskery,
Ben Zevenbergen,
Sandra Blascovich,
Kyle Pedersen,
Alison Lentz,
Blaise Aguera y Arcas
Abstract:
We propose a "Moral Imagination" methodology to facilitate a culture of responsible innovation for engineering and product teams in technology companies. Our approach has been operationalized over the past two years at Google, where we have conducted over 50 workshops with teams across the organization. We argue that our approach is a crucial complement to existing formal and informal initiatives…
▽ More
We propose a "Moral Imagination" methodology to facilitate a culture of responsible innovation for engineering and product teams in technology companies. Our approach has been operationalized over the past two years at Google, where we have conducted over 50 workshops with teams across the organization. We argue that our approach is a crucial complement to existing formal and informal initiatives for fostering a culture of ethical awareness, deliberation, and decision-making in technology design such as company principles, ethics and privacy review procedures, and compliance controls. We characterize some of the distinctive benefits of our methodology for the technology sector in particular.
△ Less
Submitted 28 October, 2023; v1 submitted 12 June, 2023;
originally announced June 2023.
-
LOPR: Latent Occupancy PRediction using Generative Models
Authors:
Bernard Lange,
Masha Itkina,
Mykel J. Kochenderfer
Abstract:
Environment prediction frameworks are integral for autonomous vehicles, enabling safe navigation in dynamic environments. LiDAR generated occupancy grid maps (L-OGMs) offer a robust bird's eye-view scene representation that facilitates joint scene predictions without relying on manual labeling unlike commonly used trajectory prediction frameworks. Prior approaches have optimized deterministic L-OG…
▽ More
Environment prediction frameworks are integral for autonomous vehicles, enabling safe navigation in dynamic environments. LiDAR generated occupancy grid maps (L-OGMs) offer a robust bird's eye-view scene representation that facilitates joint scene predictions without relying on manual labeling unlike commonly used trajectory prediction frameworks. Prior approaches have optimized deterministic L-OGM prediction architectures directly in grid cell space. While these methods have achieved some degree of success in prediction, they occasionally grapple with unrealistic and incorrect predictions. We claim that the quality and realism of the forecasted occupancy grids can be enhanced with the use of generative models. We propose a framework that decouples occupancy prediction into: representation learning and stochastic prediction within the learned latent space. Our approach allows for conditioning the model on other available sensor modalities such as RGB-cameras and high definition maps. We demonstrate that our approach achieves state-of-the-art performance and is readily transferable between different robotic platforms on the real-world NuScenes, Waymo Open, and a custom dataset we collected on an experimental vehicle platform.
△ Less
Submitted 24 August, 2023; v1 submitted 3 October, 2022;
originally announced October 2022.
-
How Do We Fail? Stress Testing Perception in Autonomous Vehicles
Authors:
Harrison Delecki,
Masha Itkina,
Bernard Lange,
Ransalu Senanayake,
Mykel J. Kochenderfer
Abstract:
Autonomous vehicles (AVs) rely on environment perception and behavior prediction to reason about agents in their surroundings. These perception systems must be robust to adverse weather such as rain, fog, and snow. However, validation of these systems is challenging due to their complexity and dependence on observation histories. This paper presents a method for characterizing failures of LiDAR-ba…
▽ More
Autonomous vehicles (AVs) rely on environment perception and behavior prediction to reason about agents in their surroundings. These perception systems must be robust to adverse weather such as rain, fog, and snow. However, validation of these systems is challenging due to their complexity and dependence on observation histories. This paper presents a method for characterizing failures of LiDAR-based perception systems for AVs in adverse weather conditions. We develop a methodology based in reinforcement learning to find likely failures in object tracking and trajectory prediction due to sequences of disturbances. We apply disturbances using a physics-based data augmentation technique for simulating LiDAR point clouds in adverse weather conditions. Experiments performed across a wide range of driving scenarios from a real-world driving dataset show that our proposed approach finds high likelihood failures with smaller input disturbances compared to baselines while remaining computationally tractable. Identified failures can inform future development of robust perception systems for AVs.
△ Less
Submitted 26 March, 2022;
originally announced March 2022.
-
Attention Augmented ConvLSTM for Environment Prediction
Authors:
Bernard Lange,
Masha Itkina,
Mykel J. Kochenderfer
Abstract:
Safe and proactive planning in robotic systems generally requires accurate predictions of the environment. Prior work on environment prediction applied video frame prediction techniques to bird's-eye view environment representations, such as occupancy grids. ConvLSTM-based frameworks used previously often result in significant blurring and vanishing of moving objects, thus hindering their applicab…
▽ More
Safe and proactive planning in robotic systems generally requires accurate predictions of the environment. Prior work on environment prediction applied video frame prediction techniques to bird's-eye view environment representations, such as occupancy grids. ConvLSTM-based frameworks used previously often result in significant blurring and vanishing of moving objects, thus hindering their applicability for use in safety-critical applications. In this work, we propose two extensions to the ConvLSTM to address these issues. We present the Temporal Attention Augmented ConvLSTM (TAAConvLSTM) and Self-Attention Augmented ConvLSTM (SAAConvLSTM) frameworks for spatiotemporal occupancy prediction, and demonstrate improved performance over baseline architectures on the real-world KITTI and Waymo datasets.
△ Less
Submitted 10 September, 2021; v1 submitted 19 October, 2020;
originally announced October 2020.
-
Mixing Real and Synthetic Data to Enhance Neural Network Training -- A Review of Current Approaches
Authors:
Viktor Seib,
Benjamin Lange,
Stefan Wirtz
Abstract:
Deep neural networks have gained tremendous importance in many computer vision tasks. However, their power comes at the cost of large amounts of annotated data required for supervised training. In this work we review and compare different techniques available in the literature to improve training results without acquiring additional annotated real-world data. This goal is mostly achieved by applyi…
▽ More
Deep neural networks have gained tremendous importance in many computer vision tasks. However, their power comes at the cost of large amounts of annotated data required for supervised training. In this work we review and compare different techniques available in the literature to improve training results without acquiring additional annotated real-world data. This goal is mostly achieved by applying annotation-preserving transformations to existing data or by synthetically creating more data.
△ Less
Submitted 17 July, 2020;
originally announced July 2020.
-
Implementing High-Order FIR Filters in FPGAs
Authors:
Philipp Födisch,
Artsiom Bryksa,
Bert Lange,
Wolfgang Enghardt,
Peter Kaever
Abstract:
Contemporary field-programmable gate arrays (FPGAs) are predestined for the application of finite impulse response (FIR) filters. Their embedded digital signal processing (DSP) blocks for multiply-accumulate operations enable efficient fixed-point computations, in cases where the filter structure is accurately mapped to the dedicated hardware architecture. This brief presents a generic systolic st…
▽ More
Contemporary field-programmable gate arrays (FPGAs) are predestined for the application of finite impulse response (FIR) filters. Their embedded digital signal processing (DSP) blocks for multiply-accumulate operations enable efficient fixed-point computations, in cases where the filter structure is accurately mapped to the dedicated hardware architecture. This brief presents a generic systolic structure for high-order FIR filters, efficiently exploiting the hardware resources of an FPGA in terms of routability and timing. Although this seems to be an easily implementable task, the synthesizing tools require an adaptation of the straightforward digital filter implementation for an optimal mapping. Using the example of a symmetric FIR filter with 90 taps, we demonstrate the performance of the proposed structure with FPGAs from Xilinx and Altera. The implementation utilizes less than 1% of slice logic and runs at clock frequencies up to 526 MHz. Moreover, an enhancement of the structure ultimately provides an extended dynamic range for the quantized coefficients without the costs of additional slice logic.
△ Less
Submitted 12 October, 2016; v1 submitted 11 October, 2016;
originally announced October 2016.
-
Towards Automatic Migration of ROS Components from Software to Hardware
Authors:
Anders Blaabjerg Lange,
Ulrik Pagh Schultz,
Anders Stengaard Soerensen
Abstract:
The use of the ROS middleware is a growing trend in robotics in general, ROS and hard real-time embedded systems have however not been easily uniteable while retaining the same overall communication and processing methodology at all levels. In this paper we present an approach aimed at tackling the schism between high-level, flexible software and low-level, real-time software. The key idea of our…
▽ More
The use of the ROS middleware is a growing trend in robotics in general, ROS and hard real-time embedded systems have however not been easily uniteable while retaining the same overall communication and processing methodology at all levels. In this paper we present an approach aimed at tackling the schism between high-level, flexible software and low-level, real-time software. The key idea of our approach is to enable software components written for a high-level publish-subscribe software architecture to be automatically migrated to a dedicated hardware architecture implemented using programmable logic. Our approach is based on the Unity framework, a unified software/hardware framework based on FPGAs for quickly interfacing high-level software to low-level robotics hardware.
△ Less
Submitted 28 July, 2014;
originally announced July 2014.