Showing 1–49 of 49 results for author: Cooper, M

Predicting Long-Term Allograft Survival in Liver Transplant Recipients

Authors: Xiang Gao, Michael Cooper, Maryam Naghibzadeh, Amirhossein Azhie, Mamatha Bhat, Rahul G. Krishnan

Abstract: Liver allograft failure occurs in approximately 20% of liver transplant recipients within five years post-transplant, leading to mortality or the need for retransplantation. Providing an accurate and interpretable model for individualized risk estimation of graft failure is essential for improving post-transplant care. To this end, we introduce the Model for Allograft Survival (MAS), a simple line… ▽ More Liver allograft failure occurs in approximately 20% of liver transplant recipients within five years post-transplant, leading to mortality or the need for retransplantation. Providing an accurate and interpretable model for individualized risk estimation of graft failure is essential for improving post-transplant care. To this end, we introduce the Model for Allograft Survival (MAS), a simple linear risk score that outperforms other advanced survival models. Using longitudinal patient follow-up data from the United States (U.S.), we develop our models on 82,959 liver transplant recipients and conduct multi-site evaluations on 11 regions. Additionally, by testing on a separate non-U.S. cohort, we explore the out-of-distribution generalization performance of various models without additional fine-tuning, a crucial property for clinical deployment. We find that the most complex models are also the ones most vulnerable to distribution shifts despite achieving the best in-distribution performance. Our findings not only provide a strong risk score for predicting long-term graft failure but also suggest that the routine machine learning pipeline with only in-distribution held-out validation could create harmful consequences for patients at deployment. △ Less

Submitted 10 August, 2024; originally announced August 2024.

Comments: Accepted at MLHC 2024

Axiomatic Characterisations of Sample-based Explainers

Authors: Leila Amgoud, Martin C. Cooper, Salim Debbaoui

Abstract: Explaining decisions of black-box classifiers is both important and computationally challenging. In this paper, we scrutinize explainers that generate feature-based explanations from samples or datasets. We start by presenting a set of desirable properties that explainers would ideally satisfy, delve into their relationships, and highlight incompatibilities of some of them. We identify the entire… ▽ More Explaining decisions of black-box classifiers is both important and computationally challenging. In this paper, we scrutinize explainers that generate feature-based explanations from samples or datasets. We start by presenting a set of desirable properties that explainers would ideally satisfy, delve into their relationships, and highlight incompatibilities of some of them. We identify the entire family of explainers that satisfy two key properties which are compatible with all the others. Its instances provide sufficient reasons, called weak abductive explanations.We then unravel its various subfamilies that satisfy subsets of compatible properties. Indeed, we fully characterize all the explainers that satisfy any subset of compatible properties. In particular, we introduce the first (broad family of) explainers that guarantee the existence of explanations and their global consistency.We discuss some of its instances including the irrefutable explainer and the surrogate explainer whose explanations can be found in polynomial time. △ Less

Submitted 12 August, 2024; v1 submitted 9 August, 2024; originally announced August 2024.

ACM Class: I.2.6

Backward explanations via redefinition of predicates

Authors: Léo Saulières, Martin C. Cooper, Florence Dupin de Saint Cyr

Abstract: History eXplanation based on Predicates (HXP), studies the behavior of a Reinforcement Learning (RL) agent in a sequence of agent's interactions with the environment (a history), through the prism of an arbitrary predicate. To this end, an action importance score is computed for each action in the history. The explanation consists in displaying the most important actions to the user. As the calcul… ▽ More History eXplanation based on Predicates (HXP), studies the behavior of a Reinforcement Learning (RL) agent in a sequence of agent's interactions with the environment (a history), through the prism of an arbitrary predicate. To this end, an action importance score is computed for each action in the history. The explanation consists in displaying the most important actions to the user. As the calculation of an action's importance is #W[1]-hard, it is necessary for long histories to approximate the scores, at the expense of their quality. We therefore propose a new HXP method, called Backward-HXP, to provide explanations for these histories without having to approximate scores. Experiments show the ability of B-HXP to summarise long histories. △ Less

Submitted 5 August, 2024; originally announced August 2024.

ACM Class: I.2.6

Human-Centered Design Recommendations for LLM-as-a-Judge

Authors: Qian Pan, Zahra Ashktorab, Michael Desmond, Martin Santillan Cooper, James Johnson, Rahul Nair, Elizabeth Daly, Werner Geyer

Abstract: Traditional reference-based metrics, such as BLEU and ROUGE, are less effective for assessing outputs from Large Language Models (LLMs) that produce highly creative or superior-quality text, or in situations where reference outputs are unavailable. While human evaluation remains an option, it is costly and difficult to scale. Recent work using LLMs as evaluators (LLM-as-a-judge) is promising, but… ▽ More Traditional reference-based metrics, such as BLEU and ROUGE, are less effective for assessing outputs from Large Language Models (LLMs) that produce highly creative or superior-quality text, or in situations where reference outputs are unavailable. While human evaluation remains an option, it is costly and difficult to scale. Recent work using LLMs as evaluators (LLM-as-a-judge) is promising, but trust and reliability remain a significant concern. Integrating human input is crucial to ensure criteria used to evaluate are aligned with the human's intent, and evaluations are robust and consistent. This paper presents a user study of a design exploration called EvaluLLM, that enables users to leverage LLMs as customizable judges, promoting human involvement to balance trust and cost-saving potential with caution. Through interviews with eight domain experts, we identified the need for assistance in developing effective evaluation criteria aligning the LLM-as-a-judge with practitioners' preferences and expectations. We offer findings and design recommendations to optimize human-assisted LLM-as-judge systems. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Comments: 14 pages, 6 figures, Accepted for publication in ACL 2024 Workshop HuCLLM

Homomorphisms and Embeddings of STRIPS Planning Models

Authors: Arnaud Lequen, Martin C. Cooper, Frédéric Maris

Abstract: Determining whether two STRIPS planning instances are isomorphic is the simplest form of comparison between planning instances. It is also a particular case of the problem concerned with finding an isomorphism between a planning instance $P$ and a sub-instance of another instance $P_0$ . One application of such a mapping is to efficiently produce a compiled form containing all solutions to P from… ▽ More Determining whether two STRIPS planning instances are isomorphic is the simplest form of comparison between planning instances. It is also a particular case of the problem concerned with finding an isomorphism between a planning instance $P$ and a sub-instance of another instance $P_0$ . One application of such a mapping is to efficiently produce a compiled form containing all solutions to P from a compiled form containing all solutions to $P_0$. We also introduce the notion of embedding from an instance $P$ to another instance $P_0$, which allows us to deduce that $P_0$ has no solution-plan if $P$ is unsolvable. In this paper, we study the complexity of these problems. We show that the first is GI-complete, and can thus be solved, in theory, in quasi-polynomial time. While we prove the remaining problems to be NP-complete, we propose an algorithm to build an isomorphism, when possible. We report extensive experimental trials on benchmark problems which demonstrate conclusively that applying constraint propagation in preprocessing can greatly improve the efficiency of a SAT solver. △ Less

Submitted 24 June, 2024; originally announced June 2024.

The end of multiple choice tests: using AI to enhance assessment

Authors: Michael Klymkowsky, Melanie M. Cooper

Abstract: Effective teaching relies on knowing what students know-or think they know. Revealing student thinking is challenging. Often used because of their ease of grading, even the best multiple choice (MC) tests, those using research based distractors (wrong answers) are intrinsically limited in the insights they provide due to two factors. When distractors do not reflect student beliefs they can be igno… ▽ More Effective teaching relies on knowing what students know-or think they know. Revealing student thinking is challenging. Often used because of their ease of grading, even the best multiple choice (MC) tests, those using research based distractors (wrong answers) are intrinsically limited in the insights they provide due to two factors. When distractors do not reflect student beliefs they can be ignored, increasing the likelihood that the correct answer will be chosen by chance. Moreover, making the correct choice does not guarantee that the student understands why it is correct. To address these limitations, we recommend asking students to explain why they chose their answer, and why "wrong" choices are wrong. Using a discipline-trained artificial intelligence-based bot it is possible to analyze their explanations, identifying the concepts and scientific principles that maybe missing or misapplied. The bot also makes suggestions for how instructors can use these data to better guide student thinking. In a small "proof of concept" study, we tested this approach using questions from the Biology Concepts Instrument (BCI). The result was rapid, informative, and provided actionable feedback on student thinking. It appears that the use of AI addresses the weaknesses of conventional MC test. It seems likely that incorporating AI-analyzed formative assessments will lead to improved overall learning outcomes. △ Less

Submitted 11 June, 2024; originally announced June 2024.

InterpreTabNet: Distilling Predictive Signals from Tabular Data by Salient Feature Interpretation

Authors: Jacob Si, Wendy Yusi Cheng, Michael Cooper, Rahul G. Krishnan

Abstract: Tabular data are omnipresent in various sectors of industries. Neural networks for tabular data such as TabNet have been proposed to make predictions while leveraging the attention mechanism for interpretability. However, the inferred attention masks are often dense, making it challenging to come up with rationales about the predictive signal. To remedy this, we propose InterpreTabNet, a variant o… ▽ More Tabular data are omnipresent in various sectors of industries. Neural networks for tabular data such as TabNet have been proposed to make predictions while leveraging the attention mechanism for interpretability. However, the inferred attention masks are often dense, making it challenging to come up with rationales about the predictive signal. To remedy this, we propose InterpreTabNet, a variant of the TabNet model that models the attention mechanism as a latent variable sampled from a Gumbel-Softmax distribution. This enables us to regularize the model to learn distinct concepts in the attention masks via a KL Divergence regularizer. It prevents overlapping feature selection by promoting sparsity which maximizes the model's efficacy and improves interpretability to determine the important features when predicting the outcome. To assist in the interpretation of feature interdependencies from our model, we employ a large language model (GPT-4) and use prompt engineering to map from the learned feature mask onto natural language text describing the learned signal. Through comprehensive experiments on real-world datasets, we demonstrate that InterpreTabNet outperforms previous methods for interpreting tabular data while attaining competitive accuracy. △ Less

Submitted 11 June, 2024; v1 submitted 1 June, 2024; originally announced June 2024.

Comments: ICML 2024 Spotlight

OtterROS: Picking and Programming an Uncrewed Surface Vessel for Experimental Field Robotics Research with ROS 2

Authors: Thomas M. C. Sears, M. Riley Cooper, Sabrina R. Button, Joshua A. Marshall

Abstract: There exist a wide range of options for field robotics research using ground and aerial mobile robots, but there are comparatively few robust and research-ready uncrewed surface vessels (USVs). This workshop paper starts with a snapshot of USVs currently available to the research community and then describes "OtterROS", an open source ROS 2 solution for the Otter USV. Field experiments using Otter… ▽ More There exist a wide range of options for field robotics research using ground and aerial mobile robots, but there are comparatively few robust and research-ready uncrewed surface vessels (USVs). This workshop paper starts with a snapshot of USVs currently available to the research community and then describes "OtterROS", an open source ROS 2 solution for the Otter USV. Field experiments using OtterROS are described, which highlight the utility of the Otter USV and the benefits of using ROS 2 in aquatic robotics research. For those interested in USV research, the paper details recommended hardware to run OtterROS and includes an example ROS 2 package using OtterROS, removing unnecessary non-recurring engineering from field robotics research activities. △ Less

Submitted 22 April, 2024; v1 submitted 8 April, 2024; originally announced April 2024.

Comments: 8 pages, 6 figures. Accepted to the IEEE ICRA Workshop on Field Robotics 2024

CIS-UNet: Multi-Class Segmentation of the Aorta in Computed Tomography Angiography via Context-Aware Shifted Window Self-Attention

Authors: Muhammad Imran, Jonathan R Krebs, Veera Rajasekhar Reddy Gopu, Brian Fazzone, Vishal Balaji Sivaraman, Amarjeet Kumar, Chelsea Viscardi, Robert Evans Heithaus, Benjamin Shickel, Yuyin Zhou, Michol A Cooper, Wei Shao

Abstract: Advancements in medical imaging and endovascular grafting have facilitated minimally invasive treatments for aortic diseases. Accurate 3D segmentation of the aorta and its branches is crucial for interventions, as inaccurate segmentation can lead to erroneous surgical planning and endograft construction. Previous methods simplified aortic segmentation as a binary image segmentation problem, overlo… ▽ More Advancements in medical imaging and endovascular grafting have facilitated minimally invasive treatments for aortic diseases. Accurate 3D segmentation of the aorta and its branches is crucial for interventions, as inaccurate segmentation can lead to erroneous surgical planning and endograft construction. Previous methods simplified aortic segmentation as a binary image segmentation problem, overlooking the necessity of distinguishing between individual aortic branches. In this paper, we introduce Context Infused Swin-UNet (CIS-UNet), a deep learning model designed for multi-class segmentation of the aorta and thirteen aortic branches. Combining the strengths of Convolutional Neural Networks (CNNs) and Swin transformers, CIS-UNet adopts a hierarchical encoder-decoder structure comprising a CNN encoder, symmetric decoder, skip connections, and a novel Context-aware Shifted Window Self-Attention (CSW-SA) as the bottleneck block. Notably, CSW-SA introduces a unique utilization of the patch merging layer, distinct from conventional Swin transformers. It efficiently condenses the feature map, providing a global spatial context and enhancing performance when applied at the bottleneck layer, offering superior computational efficiency and segmentation accuracy compared to the Swin transformers. We trained our model on computed tomography (CT) scans from 44 patients and tested it on 15 patients. CIS-UNet outperformed the state-of-the-art SwinUNetR segmentation model, which is solely based on Swin transformers, by achieving a superior mean Dice coefficient of 0.713 compared to 0.697, and a mean surface distance of 2.78 mm compared to 3.39 mm. CIS-UNet's superior 3D aortic segmentation offers improved precision and optimization for planning endovascular treatments. Our dataset and code will be publicly available. △ Less

Submitted 23 January, 2024; originally announced January 2024.

A Holistic Assessment of the Reliability of Machine Learning Systems

Authors: Anthony Corso, David Karamadian, Romeo Valentin, Mary Cooper, Mykel J. Kochenderfer

Abstract: As machine learning (ML) systems increasingly permeate high-stakes settings such as healthcare, transportation, military, and national security, concerns regarding their reliability have emerged. Despite notable progress, the performance of these systems can significantly diminish due to adversarial attacks or environmental changes, leading to overconfident predictions, failures to detect input fa… ▽ More As machine learning (ML) systems increasingly permeate high-stakes settings such as healthcare, transportation, military, and national security, concerns regarding their reliability have emerged. Despite notable progress, the performance of these systems can significantly diminish due to adversarial attacks or environmental changes, leading to overconfident predictions, failures to detect input faults, and an inability to generalize in unexpected scenarios. This paper proposes a holistic assessment methodology for the reliability of ML systems. Our framework evaluates five key properties: in-distribution accuracy, distribution-shift robustness, adversarial robustness, calibration, and out-of-distribution detection. A reliability score is also introduced and used to assess the overall system reliability. To provide insights into the performance of different algorithmic approaches, we identify and categorize state-of-the-art techniques, then evaluate a selection on real-world tasks using our proposed reliability metrics and reliability score. Our analysis of over 500 models reveals that designing for one metric does not necessarily constrain others but certain algorithmic techniques can improve reliability across multiple metrics simultaneously. This study contributes to a more comprehensive understanding of ML reliability and provides a roadmap for future research and development. △ Less

Submitted 29 July, 2023; v1 submitted 20 July, 2023; originally announced July 2023.

Needs, Passions and Loot Boxes -- Exploring Reasons for Problem Behaviour in Relation to Loot Box Engagement

Authors: Dylan Mercury Cooper

Abstract: Research on the convergence of gaming and gambling has been around since the 1990s. The emergence of loot boxes in video games in the mid 2010s, a game mechanic with a chance-based outcome that shares structural and psychological similarities to gambling, caused public controversy and lead to the inception of a new field of study, loot box research. Since then, various studies have found a relatio… ▽ More Research on the convergence of gaming and gambling has been around since the 1990s. The emergence of loot boxes in video games in the mid 2010s, a game mechanic with a chance-based outcome that shares structural and psychological similarities to gambling, caused public controversy and lead to the inception of a new field of study, loot box research. Since then, various studies have found a relationship between loot box engagement and problem gambling as well as problem gaming. Due to the cross-sectional nature of this data, however, inferences about causality are limited. While loot box research has extensively investigated the relationship between loot box engagement and problem behaviour, little research has been done to explain the underlying motivations of players that drive them to interact with loot boxes. The goal of this thesis is to provide possible explanations for the relationship between loot box engagement and problem gamblers or problem gamers. In doing so, it draws upon two prominent psychological theories. Self-Determination Theory and the Dualistic Model of Passion. Self-Determination Theory's concept of psychological needs and their satisfaction or frustration is hereby used to explain the development of harmonious or obsessive passions, which are introduced in the Dualistic Model of Passion. These obsessive passions have been shown to be possible antecedents of behavioural addictions, such as problem gambling or problem gaming. Thus, the interplay between needs, passions and loot box opening could elucidate the aforementioned correlations between loot box engagement and problem behaviour. However, further research, especially utilising longitudinal data, is needed to better understand these processes. △ Less

Submitted 10 July, 2023; originally announced July 2023.

Copula-Based Deep Survival Models for Dependent Censoring

Authors: Ali Hossein Gharari Foomani, Michael Cooper, Russell Greiner, Rahul G. Krishnan

Abstract: A survival dataset describes a set of instances (e.g. patients) and provides, for each, either the time until an event (e.g. death), or the censoring time (e.g. when lost to follow-up - which is a lower bound on the time until the event). We consider the challenge of survival prediction: learning, from such data, a predictive model that can produce an individual survival distribution for a novel i… ▽ More A survival dataset describes a set of instances (e.g. patients) and provides, for each, either the time until an event (e.g. death), or the censoring time (e.g. when lost to follow-up - which is a lower bound on the time until the event). We consider the challenge of survival prediction: learning, from such data, a predictive model that can produce an individual survival distribution for a novel instance. Many contemporary methods of survival prediction implicitly assume that the event and censoring distributions are independent conditional on the instance's covariates - a strong assumption that is difficult to verify (as we observe only one outcome for each instance) and which can induce significant bias when it does not hold. This paper presents a parametric model of survival that extends modern non-linear survival analysis by relaxing the assumption of conditional independence. On synthetic and semi-synthetic data, our approach significantly improves estimates of survival distributions compared to the standard that assumes conditional independence in the data. △ Less

Submitted 20 June, 2023; originally announced June 2023.

Comments: 23 pages, 7 figures

Widespread Increases in Future Wildfire Risk to Global Forest Carbon Offset Projects Revealed by Explainable AI

Authors: Tristan Ballard, Matthew Cooper, Chris Lowrie, Gopal Erinjippurath

Abstract: Carbon offset programs are critical in the fight against climate change. One emerging threat to the long-term stability and viability of forest carbon offset projects is wildfires, which can release large amounts of carbon and limit the efficacy of associated offsetting credits. However, analysis of wildfire risk to forest carbon projects is challenging because existing models for forecasting long… ▽ More Carbon offset programs are critical in the fight against climate change. One emerging threat to the long-term stability and viability of forest carbon offset projects is wildfires, which can release large amounts of carbon and limit the efficacy of associated offsetting credits. However, analysis of wildfire risk to forest carbon projects is challenging because existing models for forecasting long-term fire risk are limited in predictive accuracy. Therefore, we propose an explainable artificial intelligence (XAI) model trained on 7 million global satellite wildfire observations. Validation results suggest substantial potential for high resolution, enhanced accuracy projections of global wildfire risk, and the model outperforms the U.S. National Center for Atmospheric Research's leading fire model. Applied to a collection of 190 global forest carbon projects, we find that fire exposure is projected to increase 55% [37-76%] by 2080 under a mid-range scenario (SSP2-4.5). Our results indicate the large wildfire carbon project damages seen in the past decade are likely to become more frequent as forests become hotter and drier. In response, we hope the model can support wildfire managers, policymakers, and carbon market analysts to preemptively quantify and mitigate long-term permanence risks to forest carbon projects. △ Less

Submitted 3 May, 2023; originally announced May 2023.

Comments: 6 pages, 5 figures. Published in ICLR 2023 Workshop: Tackling Climate Change with Machine Learning

On Computing Probabilistic Abductive Explanations

Authors: Yacine Izza, Xuanxiang Huang, Alexey Ignatiev, Nina Narodytska, Martin C. Cooper, Joao Marques-Silva

Abstract: The most widely studied explainable AI (XAI) approaches are unsound. This is the case with well-known model-agnostic explanation approaches, and it is also the case with approaches based on saliency maps. One solution is to consider intrinsic interpretability, which does not exhibit the drawback of unsoundness. Unfortunately, intrinsic interpretability can display unwieldy explanation redundancy.… ▽ More The most widely studied explainable AI (XAI) approaches are unsound. This is the case with well-known model-agnostic explanation approaches, and it is also the case with approaches based on saliency maps. One solution is to consider intrinsic interpretability, which does not exhibit the drawback of unsoundness. Unfortunately, intrinsic interpretability can display unwieldy explanation redundancy. Formal explainability represents the alternative to these non-rigorous approaches, with one example being PI-explanations. Unfortunately, PI-explanations also exhibit important drawbacks, the most visible of which is arguably their size. Recently, it has been observed that the (absolute) rigor of PI-explanations can be traded off for a smaller explanation size, by computing the so-called relevant sets. Given some positive δ, a set S of features is δ-relevant if, when the features in S are fixed, the probability of getting the target class exceeds δ. However, even for very simple classifiers, the complexity of computing relevant sets of features is prohibitive, with the decision problem being NPPP-complete for circuit-based classifiers. In contrast with earlier negative results, this paper investigates practical approaches for computing relevant sets for a number of widely used classifiers that include Decision Trees (DTs), Naive Bayes Classifiers (NBCs), and several families of classifiers obtained from propositional languages. Moreover, the paper shows that, in practice, and for these families of classifiers, relevant sets are easy to compute. Furthermore, the experiments confirm that succinct sets of relevant features can be obtained for the families of classifiers considered. △ Less

Submitted 12 December, 2022; originally announced December 2022.

Comments: arXiv admin note: text overlap with arXiv:2207.04748, arXiv:2205.09569

Feature Necessity & Relevancy in ML Classifier Explanations

Authors: Xuanxiang Huang, Martin C. Cooper, Antonio Morgado, Jordi Planes, Joao Marques-Silva

Abstract: Given a machine learning (ML) model and a prediction, explanations can be defined as sets of features which are sufficient for the prediction. In some applications, and besides asking for an explanation, it is also critical to understand whether sensitive features can occur in some explanation, or whether a non-interesting feature must occur in all explanations. This paper starts by relating such… ▽ More Given a machine learning (ML) model and a prediction, explanations can be defined as sets of features which are sufficient for the prediction. In some applications, and besides asking for an explanation, it is also critical to understand whether sensitive features can occur in some explanation, or whether a non-interesting feature must occur in all explanations. This paper starts by relating such queries respectively with the problems of relevancy and necessity in logic-based abduction. The paper then proves membership and hardness results for several families of ML classifiers. Afterwards the paper proposes concrete algorithms for two classes of classifiers. The experimental results confirm the scalability of the proposed algorithms. △ Less

Submitted 7 February, 2023; v1 submitted 27 October, 2022; originally announced October 2022.

Label Sleuth: From Unlabeled Text to a Classifier in a Few Hours

Authors: Eyal Shnarch, Alon Halfon, Ariel Gera, Marina Danilevsky, Yannis Katsis, Leshem Choshen, Martin Santillan Cooper, Dina Epelboim, Zheng Zhang, Dakuo Wang, Lucy Yip, Liat Ein-Dor, Lena Dankin, Ilya Shnayderman, Ranit Aharonov, Yunyao Li, Naftali Liberman, Philip Levin Slesarev, Gwilym Newton, Shila Ofek-Koifman, Noam Slonim, Yoav Katz

Abstract: Text classification can be useful in many real-world scenarios, saving a lot of time for end users. However, building a custom classifier typically requires coding skills and ML knowledge, which poses a significant barrier for many potential users. To lift this barrier, we introduce Label Sleuth, a free open source system for labeling and creating text classifiers. This system is unique for (a) be… ▽ More Text classification can be useful in many real-world scenarios, saving a lot of time for end users. However, building a custom classifier typically requires coding skills and ML knowledge, which poses a significant barrier for many potential users. To lift this barrier, we introduce Label Sleuth, a free open source system for labeling and creating text classifiers. This system is unique for (a) being a no-code system, making NLP accessible to non-experts, (b) guiding users through the entire labeling process until they obtain a custom classifier, making the process efficient -- from cold start to classifier in a few hours, and (c) being open for configuration and extension by developers. By open sourcing Label Sleuth we hope to build a community of users and developers that will broaden the utilization of NLP models. △ Less

Submitted 31 October, 2022; v1 submitted 2 August, 2022; originally announced August 2022.

Comments: 7 pages, 2 figures To be published at EMNLP 2022

Provably Precise, Succinct and Efficient Explanations for Decision Trees

Authors: Yacine Izza, Alexey Ignatiev, Nina Narodytska, Martin C. Cooper, Joao Marques-Silva

Abstract: Decision trees (DTs) embody interpretable classifiers. DTs have been advocated for deployment in high-risk applications, but also for explaining other complex classifiers. Nevertheless, recent work has demonstrated that predictions in DTs ought to be explained with rigorous approaches. Although rigorous explanations can be computed in polynomial time for DTs, their size may be beyond the cognitive… ▽ More Decision trees (DTs) embody interpretable classifiers. DTs have been advocated for deployment in high-risk applications, but also for explaining other complex classifiers. Nevertheless, recent work has demonstrated that predictions in DTs ought to be explained with rigorous approaches. Although rigorous explanations can be computed in polynomial time for DTs, their size may be beyond the cognitive limits of human decision makers. This paper investigates the computation of δ-relevant sets for DTs. δ-relevant sets denote explanations that are succinct and provably precise. These sets represent generalizations of rigorous explanations, which are precise with probability one, and so they enable trading off explanation size for precision. The paper proposes two logic encodings for computing smallest δ-relevant sets for DTs. The paper further devises a polynomial-time algorithm for computing δ-relevant sets which are not guaranteed to be subset-minimal, but for which the experiments show to be most often subset-minimal in practice. The experimental results also demonstrate the practical efficiency of computing smallest δ-relevant sets. △ Less

Submitted 19 May, 2022; originally announced May 2022.

Efficient Explanations for Knowledge Compilation Languages

Authors: Xuanxiang Huang, Yacine Izza, Alexey Ignatiev, Martin C. Cooper, Nicholas Asher, Joao Marques-Silva

Abstract: Knowledge compilation (KC) languages find a growing number of practical uses, including in Constraint Programming (CP) and in Machine Learning (ML). In most applications, one natural question is how to explain the decisions made by models represented by a KC language. This paper shows that for many of the best known KC languages, well-known classes of explanations can be computed in polynomial tim… ▽ More Knowledge compilation (KC) languages find a growing number of practical uses, including in Constraint Programming (CP) and in Machine Learning (ML). In most applications, one natural question is how to explain the decisions made by models represented by a KC language. This paper shows that for many of the best known KC languages, well-known classes of explanations can be computed in polynomial time. These classes include deterministic decomposable negation normal form (d-DNNF), and so any KC language that is strictly less succinct than d-DNNF. Furthermore, the paper also investigates the conditions under which polynomial time computation of explanations can be extended to KC languages more succinct than d-DNNF. △ Less

Submitted 8 July, 2021; v1 submitted 4 July, 2021; originally announced July 2021.

Nonequilibrium Thermodynamics in Measuring Carbon Footprints: Disentangling Structure and Artifact in Input-Output Accounting

Authors: Samuel P. Loomis, Mark Cooper, James P. Crutchfield

Abstract: Multiregional input-output (MRIO) tables, in conjunction with Leontief analysis, are widely-used to assess the geographical distribution of carbon emissions and the economic activities that cause them. Majorization, a tool originating in economics that has found utility in statistical mechanics, can provide insight into how Leontief analysis links disparities in emissions with global income inequa… ▽ More Multiregional input-output (MRIO) tables, in conjunction with Leontief analysis, are widely-used to assess the geographical distribution of carbon emissions and the economic activities that cause them. Majorization, a tool originating in economics that has found utility in statistical mechanics, can provide insight into how Leontief analysis links disparities in emissions with global income inequality. We examine Leontief analysis as a model, drawing out similarities with modern nonequilibrium statistical mechanics. Paralleling the physical concept of thermo-majorization, we define the concept of eco-majorization and show it is a sufficient condition to determine the directionality of embodied emission flows. Surprisingly, relatively small trade deficits and a geographically heterogeneous emissions-per-dollar ratio greatly increases the appearance of eco-majorization, regardless of any further content in the MRIO tables used. Our results are bolstered by a statistical analysis of null models of MRIO tables, based on data provided by the Global Trade Aggregation Project9 △ Less

Submitted 12 November, 2021; v1 submitted 7 June, 2021; originally announced June 2021.

Comments: 14 pages, 5 figures; 1 appendix; http://csc.ucdavis.edu/~cmg/compmech/pubs/netacam.htm

Efficient Explanations With Relevant Sets

Authors: Yacine Izza, Alexey Ignatiev, Nina Narodytska, Martin C. Cooper, Joao Marques-Silva

Abstract: Recent work proposed $δ$-relevant inputs (or sets) as a probabilistic explanation for the predictions made by a classifier on a given input. $δ$-relevant sets are significant because they serve to relate (model-agnostic) Anchors with (model-accurate) PI- explanations, among other explanation approaches. Unfortunately, the computation of smallest size $δ$-relevant sets is complete for ${NP}^{PP}$,… ▽ More Recent work proposed $δ$-relevant inputs (or sets) as a probabilistic explanation for the predictions made by a classifier on a given input. $δ$-relevant sets are significant because they serve to relate (model-agnostic) Anchors with (model-accurate) PI- explanations, among other explanation approaches. Unfortunately, the computation of smallest size $δ$-relevant sets is complete for ${NP}^{PP}$, rendering their computation largely infeasible in practice. This paper investigates solutions for tackling the practical limitations of $δ$-relevant sets. First, the paper alternatively considers the computation of subset-minimal sets. Second, the paper studies concrete families of classifiers, including decision trees among others. For these cases, the paper shows that the computation of subset-minimal $δ$-relevant sets is in NP, and can be solved with a polynomial number of calls to an NP oracle. The experimental evaluation compares the proposed approach with heuristic explainers for the concrete case of the classifiers studied in the paper, and confirms the advantage of the proposed solution over the state of the art. △ Less

Submitted 1 June, 2021; originally announced June 2021.

Explanations for Monotonic Classifiers

Authors: Joao Marques-Silva, Thomas Gerspacher, Martin Cooper, Alexey Ignatiev, Nina Narodytska

Abstract: In many classification tasks there is a requirement of monotonicity. Concretely, if all else remains constant, increasing (resp. decreasing) the value of one or more features must not decrease (resp. increase) the value of the prediction. Despite comprehensive efforts on learning monotonic classifiers, dedicated approaches for explaining monotonic classifiers are scarce and classifier-specific. Th… ▽ More In many classification tasks there is a requirement of monotonicity. Concretely, if all else remains constant, increasing (resp. decreasing) the value of one or more features must not decrease (resp. increase) the value of the prediction. Despite comprehensive efforts on learning monotonic classifiers, dedicated approaches for explaining monotonic classifiers are scarce and classifier-specific. This paper describes novel algorithms for the computation of one formal explanation of a (black-box) monotonic classifier. These novel algorithms are polynomial in the run time complexity of the classifier and the number of features. Furthermore, the paper presents a practically efficient model-agnostic algorithm for enumerating formal explanations. △ Less

Submitted 31 May, 2021; originally announced June 2021.

Tokenising behaviour change: optimising blockchain technology for sustainable transport interventions

Authors: Iain Barclay, Michael Cooper, Alun Preece, Omer Rana, Ian Taylor

Abstract: Transport makes an impact across SDGs, encompassing climate change, health, inequality and sustainability. It is also an area in which individuals are able to make decisions which have potential to collectively contribute to significant and wide-ranging benefits. Governments and authorities need citizens to make changes towards adopting sustainable transport behaviours and behaviour change interve… ▽ More Transport makes an impact across SDGs, encompassing climate change, health, inequality and sustainability. It is also an area in which individuals are able to make decisions which have potential to collectively contribute to significant and wide-ranging benefits. Governments and authorities need citizens to make changes towards adopting sustainable transport behaviours and behaviour change interventions are being used as tools to foster changes in travel choices, towards more sustainable modes. Blockchain technology has the potential to bring new levels of scale to transport behaviour change interventions, but a rigorous approach to token design is required. This paper uses a survey of research projects and use cases to analyse current applications of blockchain technology in transport behaviour change interventions, and identifies barriers and limitations to achieving targeted change at scale. The paper draws upon these findings to outline a research agenda that brings a focus on correlating specific Behaviour Change Techniques (BCTs) to token design, and defines processes for standardising token designs in behaviour change tools. The paper further outlines architecture and operational considerations for blockchain-based platforms in behaviour change interventions, such that design choices do not compromise opportunities or wider environmental goals. △ Less

Submitted 5 April, 2021; originally announced April 2021.

Resonance: Replacing Software Constants with Context-Aware Models in Real-time Communication

Authors: Jayant Gupchup, Ashkan Aazami, Yaran Fan, Senja Filipi, Tom Finley, Scott Inglis, Marcus Asteborg, Luke Caroll, Rajan Chari, Markus Cozowicz, Vishak Gopal, Vinod Prakash, Sasikanth Bendapudi, Jack Gerrits, Eric Lau, Huazhou Liu, Marco Rossi, Dima Slobodianyk, Dmitri Birjukov, Matty Cooper, Nilesh Javar, Dmitriy Perednya, Sriram Srinivasan, John Langford, Ross Cutler , et al. (1 additional authors not shown)

Abstract: Large software systems tune hundreds of 'constants' to optimize their runtime performance. These values are commonly derived through intuition, lab tests, or A/B tests. A 'one-size-fits-all' approach is often sub-optimal as the best value depends on runtime context. In this paper, we provide an experimental approach to replace constants with learned contextual functions for Skype - a widely used r… ▽ More Large software systems tune hundreds of 'constants' to optimize their runtime performance. These values are commonly derived through intuition, lab tests, or A/B tests. A 'one-size-fits-all' approach is often sub-optimal as the best value depends on runtime context. In this paper, we provide an experimental approach to replace constants with learned contextual functions for Skype - a widely used real-time communication (RTC) application. We present Resonance, a system based on contextual bandits (CB). We describe experiences from three real-world experiments: applying it to the audio, video, and transport components in Skype. We surface a unique and practical challenge of performing machine learning (ML) inference in large software systems written using encapsulation principles. Finally, we open-source FeatureBroker, a library to reduce the friction in adopting ML models in such development environments △ Less

Submitted 22 November, 2020; originally announced November 2020.

Comments: Workshop on ML for Systems at NeurIPS 2020, Accepted

Journal ref: ML for Systems, NeurIPS 2020

Explaining Naive Bayes and Other Linear Classifiers with Polynomial Time and Delay

Authors: Joao Marques-Silva, Thomas Gerspacher, Martin C. Cooper, Alexey Ignatiev, Nina Narodytska

Abstract: Recent work proposed the computation of so-called PI-explanations of Naive Bayes Classifiers (NBCs). PI-explanations are subset-minimal sets of feature-value pairs that are sufficient for the prediction, and have been computed with state-of-the-art exact algorithms that are worst-case exponential in time and space. In contrast, we show that the computation of one PI-explanation for an NBC can be a… ▽ More Recent work proposed the computation of so-called PI-explanations of Naive Bayes Classifiers (NBCs). PI-explanations are subset-minimal sets of feature-value pairs that are sufficient for the prediction, and have been computed with state-of-the-art exact algorithms that are worst-case exponential in time and space. In contrast, we show that the computation of one PI-explanation for an NBC can be achieved in log-linear time, and that the same result also applies to the more general class of linear classifiers. Furthermore, we show that the enumeration of PI-explanations can be obtained with polynomial delay. Experimental results demonstrate the performance gains of the new algorithms when compared with earlier work. The experimental results also investigate ways to measure the quality of heuristic explanations △ Less

Submitted 4 November, 2020; v1 submitted 13 August, 2020; originally announced August 2020.

Strengthening neighbourhood substitution

Authors: Martin C. Cooper

Abstract: Domain reduction is an essential tool for solving the constraint satisfaction problem (CSP). In the binary CSP, neighbourhood substitution consists in eliminating a value if there exists another value which can be substituted for it in each constraint. We show that the notion of neighbourhood substitution can be strengthened in two distinct ways without increasing time complexity. We also show the… ▽ More Domain reduction is an essential tool for solving the constraint satisfaction problem (CSP). In the binary CSP, neighbourhood substitution consists in eliminating a value if there exists another value which can be substituted for it in each constraint. We show that the notion of neighbourhood substitution can be strengthened in two distinct ways without increasing time complexity. We also show the theoretical result that, unlike neighbourhood substitution, finding an optimal sequence of these new operations is NP-hard. △ Less

Submitted 13 July, 2020; originally announced July 2020.

ACM Class: I.2.8; I.2.10

An Evaluation of Visualization Methods for Population Statistics Based on Choropleth Maps

Authors: Lonni Besançon, Matthew Cooper, Anders Ynnerman, Frédéric Vernier

Abstract: We evaluate several augmentations to the choropleth map to convey additional information, including glyphs, 3D, cartograms, juxtaposed maps, and shading methods. While choropleth maps are a common method used to represent societal data, with multivariate data they can impede as much as improve understanding. In particular large, low population density regions often dominate the map and can mislead… ▽ More We evaluate several augmentations to the choropleth map to convey additional information, including glyphs, 3D, cartograms, juxtaposed maps, and shading methods. While choropleth maps are a common method used to represent societal data, with multivariate data they can impede as much as improve understanding. In particular large, low population density regions often dominate the map and can mislead the viewer as to the message conveyed. Our results highlight the potential of 3D choropleth maps as well as the low accuracy of choropleth map tasks with multivariate data. We also introduce and evaluate popcharts, four techniques designed to show the density of population at a very fine scale on top of choropleth maps. All the data, results, and scripts are available from https://osf.io/8rxwg/. △ Less

Submitted 1 May, 2020; originally announced May 2020.

MSC Class: 00A66 ACM Class: H.5

CompLex: A New Corpus for Lexical Complexity Prediction from Likert Scale Data

Authors: Matthew Shardlow, Michael Cooper, Marcos Zampieri

Abstract: Predicting which words are considered hard to understand for a given target population is a vital step in many NLP applications such as text simplification. This task is commonly referred to as Complex Word Identification (CWI). With a few exceptions, previous studies have approached the task as a binary classification task in which systems predict a complexity value (complex vs. non-complex) for… ▽ More Predicting which words are considered hard to understand for a given target population is a vital step in many NLP applications such as text simplification. This task is commonly referred to as Complex Word Identification (CWI). With a few exceptions, previous studies have approached the task as a binary classification task in which systems predict a complexity value (complex vs. non-complex) for a set of target words in a text. This choice is motivated by the fact that all CWI datasets compiled so far have been annotated using a binary annotation scheme. Our paper addresses this limitation by presenting the first English dataset for continuous lexical complexity prediction. We use a 5-point Likert scale scheme to annotate complex words in texts from three sources/domains: the Bible, Europarl, and biomedical texts. This resulted in a corpus of 9,476 sentences each annotated by around 7 annotators. △ Less

Submitted 11 June, 2020; v1 submitted 15 March, 2020; originally announced March 2020.

Comments: Proceedings of the 1st Workshop on Tools and Resources to Empower People with REAding DIfficulties (READI). pp. 57-62

Can visualization alleviate dichotomous thinking? Effects of visual representations on the cliff effect

Authors: Jouni Helske, Satu Helske, Matthew Cooper, Anders Ynnerman, Lonni Besançon

Abstract: Common reporting styles for statistical results in scientific articles, such as p-values and confidence intervals (CI), have been reported to be prone to dichotomous interpretations, especially with respect to the null hypothesis significance testing framework. For example when the p-value is small enough or the CIs of the mean effects of a studied drug and a placebo are not overlapping, scientist… ▽ More Common reporting styles for statistical results in scientific articles, such as p-values and confidence intervals (CI), have been reported to be prone to dichotomous interpretations, especially with respect to the null hypothesis significance testing framework. For example when the p-value is small enough or the CIs of the mean effects of a studied drug and a placebo are not overlapping, scientists tend to claim significant differences while often disregarding the magnitudes and absolute differences in the effect sizes. This type of reasoning has been shown to be potentially harmful to science. Techniques relying on the visual estimation of the strength of evidence have been recommended to reduce such dichotomous interpretations but their effectiveness has also been challenged. We ran two experiments on researchers with expertise in statistical analysis to compare several alternative representations of confidence intervals and used Bayesian multilevel models to estimate the effects of the representation styles on differences in researchers' subjective confidence in the results. We also asked the respondents' opinions and preferences in representation styles. Our results suggest that adding visual information to classic CI representation can decrease the tendency towards dichotomous interpretations - measured as the `cliff effect': the sudden drop in confidence around p-value 0.05 - compared with classic CI visualization and textual representation of the CI with p-values. All data and analyses are publicly available at https://github.com/helske/statvis. △ Less

Submitted 28 May, 2021; v1 submitted 17 February, 2020; originally announced February 2020.

Journal ref: IEEE Transactions on Visualization and Computer Graphics. 2021; 27(8)

Steepest ascent can be exponential in bounded treewidth problems

Authors: David A. Cohen, Martin C. Cooper, Artem Kaznatcheev, Mark Wallace

Abstract: We investigate the complexity of local search based on steepest ascent. We show that even when all variables have domains of size two and the underlying constraint graph of variable interactions has bounded treewidth (in our construction, treewidth 7), there are fitness landscapes for which an exponential number of steps may be required to reach a local optimum. This is an improvement on prior rec… ▽ More We investigate the complexity of local search based on steepest ascent. We show that even when all variables have domains of size two and the underlying constraint graph of variable interactions has bounded treewidth (in our construction, treewidth 7), there are fitness landscapes for which an exponential number of steps may be required to reach a local optimum. This is an improvement on prior recursive constructions of long steepest ascents, which we prove to need constraint graphs of unbounded treewidth. △ Less

Submitted 2 December, 2019; v1 submitted 19 November, 2019; originally announced November 2019.

Comments: 8 pages main text, 4 pages appendix, 1 page references; fixed error in f(a,b) to match code

MSC Class: F.2.2; G.2.0; J.3 ACM Class: F.2.2; G.2.0; J.3

Journal ref: Operations Research Letters 48 (2020) 217-224

Multilevel Visualisation of Topic Dependency Models for Assessment Design and Delivery: A Hypergraph Based Approach

Authors: Kendra M. L. Cooper, Hassan Khosravi

Abstract: The effective design and delivery of assessments in a wide variety of evolving educational environments remains a challenging problem. Proposals have included the use of learning dashboards, peer learning environments, and grading support systems; these embrace visualisations to summarise and communicate results. In an on-going project, the investigation of graph based visualisation models for ass… ▽ More The effective design and delivery of assessments in a wide variety of evolving educational environments remains a challenging problem. Proposals have included the use of learning dashboards, peer learning environments, and grading support systems; these embrace visualisations to summarise and communicate results. In an on-going project, the investigation of graph based visualisation models for assessment design and delivery has yielded promising results. Here, an alternative graph foundation, a two-weighted hypergraph, is considered to represent the assessment material (e.g., questions) and their explicit mapping to one or more learning objective topics. The visualisation approach considers the hypergraph as a collection of levels; the content of these levels can be customized and presented according to user preferences. A case study on generating hypergraph models using commonly available assessment data and a flexible visualisation approach using historical data from an introductory programming course is presented △ Less

Submitted 12 October, 2019; originally announced October 2019.

Comments: Published in the proceedings of the 25th International DMS Conference on Visualization and Visual Languages

Stackelberg Punishment and Bully-Proofing Autonomous Vehicles

Authors: Matt Cooper, Jun Ki Lee, Jacob Beck, Joshua D. Fishman, Michael Gillett, Zoë Papakipos, Aaron Zhang, Jerome Ramos, Aansh Shah, Michael L. Littman

Abstract: Mutually beneficial behavior in repeated games can be enforced via the threat of punishment, as enshrined in game theory's well-known "folk theorem." There is a cost, however, to a player for generating these disincentives. In this work, we seek to minimize this cost by computing a "Stackelberg punishment," in which the player selects a behavior that sufficiently punishes the other player while ma… ▽ More Mutually beneficial behavior in repeated games can be enforced via the threat of punishment, as enshrined in game theory's well-known "folk theorem." There is a cost, however, to a player for generating these disincentives. In this work, we seek to minimize this cost by computing a "Stackelberg punishment," in which the player selects a behavior that sufficiently punishes the other player while maximizing its own score under the assumption that the other player will adopt a best response. This idea generalizes the concept of a Stackelberg equilibrium. Known efficient algorithms for computing a Stackelberg equilibrium can be adapted to efficiently produce a Stackelberg punishment. We demonstrate an application of this idea in an experiment involving a virtual autonomous vehicle and human participants. We find that a self-driving car with a Stackelberg punishment policy discourages human drivers from bullying in a driving scenario requiring social negotiation. △ Less

Submitted 22 August, 2019; originally announced August 2019.

Comments: 10 pages, The 11th International Conference on Social Robotics

Variable elimination in binary CSPs

Authors: Martin C. Cooper, Achref El Mouelhi, Cyril Terrioux

Abstract: We investigate rules which allow variable elimination in binary CSP (constraint satisfaction problem) instances while conserving satisfiability. We study variable-elimination rules based on the language of forbidden patterns enriched with counting and quantification over variables and values. We propose new rules and compare them, both theoretically and experimentally. We give optimised algorithms… ▽ More We investigate rules which allow variable elimination in binary CSP (constraint satisfaction problem) instances while conserving satisfiability. We study variable-elimination rules based on the language of forbidden patterns enriched with counting and quantification over variables and values. We propose new rules and compare them, both theoretically and experimentally. We give optimised algorithms to apply these rules and show that each define a novel tractable class. Using our variable-elimination rules in preprocessing allowed us to solve more benchmark problems than without. △ Less

Submitted 10 May, 2019; originally announced May 2019.

Comments: 38 pages, 15 figures

Generalized minimum distance functions and algebraic invariants of Geramita ideals

Authors: Susan M. Cooper, Alexandra Seceleanu, Stefan O. Tohaneanu, Maria Vaz Pinto, Rafael H. Villarreal

Abstract: Motivated by notions from coding theory, we study the generalized minimum distance (GMD) function $δ_I(d,r)$ of a graded ideal $I$ in a polynomial ring over an arbitrary field using commutative algebraic methods. It is shown that $δ_I$ is non-decreasing as a function of $r$ and non-increasing as a function of $d$. For vanishing ideals over finite fields, we show that $δ_I$ is strictly decreasing a… ▽ More Motivated by notions from coding theory, we study the generalized minimum distance (GMD) function $δ_I(d,r)$ of a graded ideal $I$ in a polynomial ring over an arbitrary field using commutative algebraic methods. It is shown that $δ_I$ is non-decreasing as a function of $r$ and non-increasing as a function of $d$. For vanishing ideals over finite fields, we show that $δ_I$ is strictly decreasing as a function of $d$ until it stabilizes. We also study algebraic invariants of Geramita ideals. Those ideals are graded, unmixed, $1$-dimensional and their associated primes are generated by linear forms. We also examine GMD functions of complete intersections and show some special cases of two conjectures of Tohăneanu--Van Tuyl and Eisenbud-Green-Harris. △ Less

Submitted 11 September, 2019; v1 submitted 16 December, 2018; originally announced December 2018.

MSC Class: 13P25; 14G50; 94B27; 11T71

Journal ref: Adv. in Appl. Math. 112 (2020), 101940

Tools for online tutorials: comparing capture devices, tutorial representations, and access devices

Authors: Scott Carter, Pernilla Qvarfordt, Matthew Cooper, Aki Komori, Ville Makela

Abstract: Tutorials are one of the most fundamental means of conveying knowledge. Ideally when the task involves physical or digital objects, tutorials not only describe each step with text or via audio narration but show it as well using photos or animation. In most cases, online tutorial authors capture media from handheld mobile devices to compose these documents, but increasingly they use wearable devic… ▽ More Tutorials are one of the most fundamental means of conveying knowledge. Ideally when the task involves physical or digital objects, tutorials not only describe each step with text or via audio narration but show it as well using photos or animation. In most cases, online tutorial authors capture media from handheld mobile devices to compose these documents, but increasingly they use wearable devices as well. In this work, we explore the full life-cycle of online tutorial creation and viewing using head-mounted capture and displays. We developed a media-capture tool for Google Glass that requires minimal attention to the capture device and instead allows the author to focus on creating the tutorial's content rather than its capture. The capture tool is coupled with web-based authoring tools for creating annotatable videos and multimedia documents. In a study comparing standalone (camera on tripod) versus wearable capture (Google Glass) as well as two types of multimedia representation for authoring tutorials, we show that tutorial authors have a preference for wearable capture devices, especially when recording activities involving larger objects in non-desktop environments. Authors preferred document-based multimedia tutorials because they are more straightforward to compose and the step-based structure translates more directly to explaining a procedure. In addition, we explored using head-mounted displays for accessing tutorials in comparison to lightweight computing devices such as tablets. Our study included tutorials recorded with the same capture methods as in our access study. We found that although authors preferred head-mounted capture, tutorial consumers preferred video recorded by a camera on tripod that provides a more stable image of the workspace. △ Less

Submitted 26 January, 2018; originally announced January 2018.

On Singleton Arc Consistency for CSPs Defined by Monotone Patterns

Authors: Clement Carbonnel, David A. Cohen, Martin C. Cooper, Stanislav Zivny

Abstract: Singleton arc consistency is an important type of local consistency which has been recently shown to solve all constraint satisfaction problems (CSPs) over constraint languages of bounded width. We aim to characterise all classes of CSPs defined by a forbidden pattern that are solved by singleton arc consistency and closed under removing constraints. We identify five new patterns whose absence ens… ▽ More Singleton arc consistency is an important type of local consistency which has been recently shown to solve all constraint satisfaction problems (CSPs) over constraint languages of bounded width. We aim to characterise all classes of CSPs defined by a forbidden pattern that are solved by singleton arc consistency and closed under removing constraints. We identify five new patterns whose absence ensures solvability by singleton arc consistency, four of which are provably maximal and three of which generalise 2-SAT. Combined with simple counter-examples for other patterns, we make significant progress towards a complete classification. △ Less

Submitted 10 August, 2018; v1 submitted 20 April, 2017; originally announced April 2017.

Comments: v4: Full version of a STACS'18 paper; improved presentation

Journal ref: Algorithmica 81(4) 1699-1727 (2019)

The ACRV Picking Benchmark (APB): A Robotic Shelf Picking Benchmark to Foster Reproducible Research

Authors: Jürgen Leitner, Adam W. Tow, Jake E. Dean, Niko Suenderhauf, Joseph W. Durham, Matthew Cooper, Markus Eich, Christopher Lehnert, Ruben Mangels, Christopher McCool, Peter Kujala, Lachlan Nicholson, Trung Pham, James Sergeant, Liao Wu, Fangyi Zhang, Ben Upcroft, Peter Corke

Abstract: Robotic challenges like the Amazon Picking Challenge (APC) or the DARPA Challenges are an established and important way to drive scientific progress. They make research comparable on a well-defined benchmark with equal test conditions for all participants. However, such challenge events occur only occasionally, are limited to a small number of contestants, and the test conditions are very difficul… ▽ More Robotic challenges like the Amazon Picking Challenge (APC) or the DARPA Challenges are an established and important way to drive scientific progress. They make research comparable on a well-defined benchmark with equal test conditions for all participants. However, such challenge events occur only occasionally, are limited to a small number of contestants, and the test conditions are very difficult to replicate after the main event. We present a new physical benchmark challenge for robotic picking: the ACRV Picking Benchmark (APB). Designed to be reproducible, it consists of a set of 42 common objects, a widely available shelf, and exact guidelines for object arrangement using stencils. A well-defined evaluation protocol enables the comparison of \emph{complete} robotic systems -- including perception and manipulation -- instead of sub-systems only. Our paper also describes and reports results achieved by an open baseline system based on a Baxter robot. △ Less

Submitted 14 December, 2016; v1 submitted 16 September, 2016; originally announced September 2016.

Comments: 8 pages, submitted to RA:Letters

Binary Constraint Satisfaction Problems Defined by Excluded Topological Minors

Authors: David A. Cohen, Martin C. Cooper, Peter G. Jeavons, Stanislav Zivny

Abstract: The binary Constraint Satisfaction Problem (CSP) is to decide whether there exists an assignment to a set of variables which satisfies specified constraints between pairs of variables. A binary CSP instance can be presented as a labelled graph encoding both the forms of the constraints and where they are imposed. We consider subproblems defined by restricting the allowed form of this graph. One ty… ▽ More The binary Constraint Satisfaction Problem (CSP) is to decide whether there exists an assignment to a set of variables which satisfies specified constraints between pairs of variables. A binary CSP instance can be presented as a labelled graph encoding both the forms of the constraints and where they are imposed. We consider subproblems defined by restricting the allowed form of this graph. One type of restriction that has previously been considered is to forbid certain specified substructures (patterns). This captures some tractable classes of the CSP, but does not capture classes defined by language restrictions, or the well-known structural property of acyclicity. In this paper we extend the notion of pattern and introduce the notion of a topological minor of a binary CSP instance. By forbidding a finite set of patterns from occurring as topological minors we obtain a compact mechanism for expressing novel tractable subproblems of the binary CSP, including new generalisations of the class of acyclic instances. Forbidding a finite set of patterns as topological minors also captures all other tractable structural restrictions of the binary CSP. Moreover, we show that several patterns give rise to tractable subproblems if forbidden as topological minors but not if forbidden as sub-patterns. Finally, we introduce the idea of augmented patterns that allows for the identification of more tractable classes, including all language restrictions of the binary CSP. △ Less

Submitted 22 September, 2018; v1 submitted 18 August, 2016; originally announced August 2016.

Comments: Full version of an IJCAI'15 paper

Journal ref: Information and Computation 264 12-31 (2019)

Binarisation for Valued Constraint Satisfaction Problems

Authors: David A. Cohen, Martin C. Cooper, Peter G. Jeavons, Andrei Krokhin, Robert Powell, Stanislav Zivny

Abstract: We study methods for transforming valued constraint satisfaction problems (VCSPs) to binary VCSPs. First, we show that the standard dual encoding preserves many aspects of the algebraic properties that capture the computational complexity of VCSPs. Second, we extend the reduction of CSPs to binary CSPs described by Bulin et al. [LMCS'15] to VCSPs. This reduction establishes that VCSPs over a fixed… ▽ More We study methods for transforming valued constraint satisfaction problems (VCSPs) to binary VCSPs. First, we show that the standard dual encoding preserves many aspects of the algebraic properties that capture the computational complexity of VCSPs. Second, we extend the reduction of CSPs to binary CSPs described by Bulin et al. [LMCS'15] to VCSPs. This reduction establishes that VCSPs over a fixed valued constraint language are polynomial-time equivalent to Minimum-Cost Homomorphism Problems over a fixed digraph. △ Less

Submitted 27 July, 2017; v1 submitted 4 August, 2016; originally announced August 2016.

Comments: Subsumes 1507.01776

MSC Class: 08A70; 68Q25

Journal ref: SIAM Journal on Discrete Mathematics 31(4) (2017) 2279-2300

Simple epistemic planning: generalised gossiping

Authors: Martin C. Cooper, Andreas Herzig, Faustine Maffre, Frédéric Maris, Pierre Régnier

Abstract: The gossip problem, in which information (known as secrets) must be shared among a certain number of agents using the minimum number of calls, is of interest in the conception of communication networks and protocols. We extend the gossip problem to arbitrary epistemic depths. For example, we may require not only that all agents know all secrets but also that all agents know that all agents know al… ▽ More The gossip problem, in which information (known as secrets) must be shared among a certain number of agents using the minimum number of calls, is of interest in the conception of communication networks and protocols. We extend the gossip problem to arbitrary epistemic depths. For example, we may require not only that all agents know all secrets but also that all agents know that all agents know all secrets. We give optimal protocols for various versions of the generalised gossip problem, depending on the graph of communication links, in the case of two-way communications, one-way communications and parallel communication. We also study different variants which allow us to impose negative goals such as that certain agents must not know certain secrets. We show that in the presence of negative goals testing the existence of a successful protocol is NP-complete whereas this is always polynomial-time in the case of purely positive goals. △ Less

Submitted 14 June, 2016; v1 submitted 10 June, 2016; originally announced June 2016.

ACM Class: I.2.8; C.2.2; F.2.2; I.2.4

The Power of Arc Consistency for CSPs Defined by Partially-Ordered Forbidden Patterns

Authors: Martin C. Cooper, Stanislav Živný

Abstract: Characterising tractable fragments of the constraint satisfaction problem (CSP) is an important challenge in theoretical computer science and artificial intelligence. Forbidding patterns (generic sub-instances) provides a means of defining CSP fragments which are neither exclusively language-based nor exclusively structure-based. It is known that the class of binary CSP instances in which the brok… ▽ More Characterising tractable fragments of the constraint satisfaction problem (CSP) is an important challenge in theoretical computer science and artificial intelligence. Forbidding patterns (generic sub-instances) provides a means of defining CSP fragments which are neither exclusively language-based nor exclusively structure-based. It is known that the class of binary CSP instances in which the broken-triangle pattern (BTP) does not occur, a class which includes all tree-structured instances, are decided by arc consistency (AC), a ubiquitous reduction operation in constraint solvers. We provide a characterisation of simple partially-ordered forbidden patterns which have this AC-solvability property. It turns out that BTP is just one of five such AC-solvable patterns. The four other patterns allow us to exhibit new tractable classes. △ Less

Submitted 25 December, 2017; v1 submitted 27 April, 2016; originally announced April 2016.

MSC Class: 68Q15 ACM Class: F.4.1

Journal ref: Logical Methods in Computer Science, Volume 13, Issue 4 (December 27, 2017) lmcs:4049

Variable and value elimination in binary constraint satisfaction via forbidden patterns

Authors: David A. Cohen, Martin C. Cooper, Guillaume Escamocher, Stanislav Zivny

Abstract: Variable or value elimination in a constraint satisfaction problem (CSP) can be used in preprocessing or during search to reduce search space size. A variable elimination rule (value elimination rule) allows the polynomial-time identification of certain variables (domain elements) whose elimination, without the introduction of extra compensatory constraints, does not affect the satisfiability of a… ▽ More Variable or value elimination in a constraint satisfaction problem (CSP) can be used in preprocessing or during search to reduce search space size. A variable elimination rule (value elimination rule) allows the polynomial-time identification of certain variables (domain elements) whose elimination, without the introduction of extra compensatory constraints, does not affect the satisfiability of an instance. We show that there are essentially just four variable elimination rules and three value elimination rules defined by forbidding generic sub-instances, known as irreducible existential patterns, in arc-consistent CSP instances. One of the variable elimination rules is the already-known Broken Triangle Property, whereas the other three are novel. The three value elimination rules can all be seen as strict generalisations of neighbourhood substitution. △ Less

Submitted 12 February, 2015; originally announced February 2015.

Comments: A full version of an IJCAI'13 paper to appear in Journal of Computer and System Sciences (JCSS)

ACM Class: F.2.0

Journal ref: Journal of Computer and System Sciences 81(7) 1127-1143 (2015)

On Backdoors To Tractable Constraint Languages

Authors: Clement Carbonnel, Martin C. Cooper, Emmanuel Hebrard

Abstract: In the context of CSPs, a strong backdoor is a subset of variables such that every complete assignment yields a residual instance guaranteed to have a specified property. If the property allows efficient solving, then a small strong backdoor provides a reasonable decomposition of the original instance into easy instances. An important challenge is the design of algorithms that can find quickly a s… ▽ More In the context of CSPs, a strong backdoor is a subset of variables such that every complete assignment yields a residual instance guaranteed to have a specified property. If the property allows efficient solving, then a small strong backdoor provides a reasonable decomposition of the original instance into easy instances. An important challenge is the design of algorithms that can find quickly a small strong backdoor if one exists. We present a systematic study of the parameterized complexity of backdoor detection when the target property is a restricted type of constraint language defined by means of a family of polymorphisms. In particular, we show that under the weak assumption that the polymorphisms are idempotent, the problem is unlikely to be FPT when the parameter is either r (the constraint arity) or k (the size of the backdoor) unless P = NP or FPT = W[2]. When the parameter is k+r, however, we are able to identify large classes of languages for which the problem of finding a small backdoor is FPT. △ Less

Submitted 10 October, 2014; v1 submitted 14 April, 2014; originally announced April 2014.

Comments: 16 pages

Tractable Triangles and Cross-Free Convexity in Discrete Optimisation

Authors: Martin C. Cooper, Stanislav Živný

Abstract: The minimisation problem of a sum of unary and pairwise functions of discrete variables is a general NP-hard problem with wide applications such as computing MAP configurations in Markov Random Fields (MRF), minimising Gibbs energy, or solving binary Valued Constraint Satisfaction Problems (VCSPs). We study the computational complexity of classes of discrete optimisation problems given by allowi… ▽ More The minimisation problem of a sum of unary and pairwise functions of discrete variables is a general NP-hard problem with wide applications such as computing MAP configurations in Markov Random Fields (MRF), minimising Gibbs energy, or solving binary Valued Constraint Satisfaction Problems (VCSPs). We study the computational complexity of classes of discrete optimisation problems given by allowing only certain types of costs in every triangle of variable-value assignments to three distinct variables. We show that for several computational problems, the only non- trivial tractable classes are the well known maximum matching problem and the recently discovered joint-winner property. Our results, apart from giving complete classifications in the studied cases, provide guidance in the search for hybrid tractable classes; that is, classes of problems that are not captured by restrictions on the functions (such as submodularity) or the structure of the problem graph (such as bounded treewidth). Furthermore, we introduce a class of problems with convex cardinality functions on cross-free sets of assignments. We prove that while imposing only one of the two conditions renders the problem NP-hard, the conjunction of the two gives rise to a novel tractable class satisfying the cross-free convexity property, which generalises the joint-winner property to problems of unbounded arity. △ Less

Submitted 22 January, 2014; originally announced January 2014.

Comments: arXiv admin note: text overlap with arXiv:1008.4035 by other authors

Journal ref: Journal Of Artificial Intelligence Research, Volume 44, pages 455-490, 2012

An Algebraic Theory of Complexity for Discrete Optimisation

Authors: David A. Cohen, Martin C. Cooper, Paidi Creed, Peter G. Jeavons, Stanislav Zivny

Abstract: Discrete optimisation problems arise in many different areas and are studied under many different names. In many such problems the quantity to be optimised can be expressed as a sum of functions of a restricted form. Here we present a unifying theory of complexity for problems of this kind. We show that the complexity of a finite-domain discrete optimisation problem is determined by certain algebr… ▽ More Discrete optimisation problems arise in many different areas and are studied under many different names. In many such problems the quantity to be optimised can be expressed as a sum of functions of a restricted form. Here we present a unifying theory of complexity for problems of this kind. We show that the complexity of a finite-domain discrete optimisation problem is determined by certain algebraic properties of the objective function, which we call weighted polymorphisms. We define a Galois connection between sets of rational-valued functions and sets of weighted polymorphisms and show how the closed sets of this Galois connection can be characterised. These results provide a new approach to studying the complexity of discrete optimisation. We use this approach to identify certain maximal tractable subproblems of the general problem, and hence derive a complete classification of complexity for the Boolean case. △ Less

Submitted 28 July, 2012; originally announced July 2012.

Comments: 26 pages, full version of three conference papers: CP'06, MFCS'11, and CP'11

ACM Class: F.2.0

Journal ref: SIAM Journal on Computing 42(5) 1915-1939 (2013)

A Dichotomy for 2-Constraint Forbidden CSP Patterns

Authors: Martin C. Cooper, Guillaume Escamocher

Abstract: Although the CSP (constraint satisfaction problem) is NP-complete, even in the case when all constraints are binary, certain classes of instances are tractable. We study classes of instances defined by excluding subproblems. This approach has recently led to the discovery of novel tractable classes. The complete characterisation of all tractable classes defined by forbidding patterns (where a patt… ▽ More Although the CSP (constraint satisfaction problem) is NP-complete, even in the case when all constraints are binary, certain classes of instances are tractable. We study classes of instances defined by excluding subproblems. This approach has recently led to the discovery of novel tractable classes. The complete characterisation of all tractable classes defined by forbidding patterns (where a pattern is simply a compact representation of a set of subproblems) is a challenging problem. We demonstrate a dichotomy in the case of forbidden patterns consisting of either one or two constraints. This has allowed us to discover new tractable classes including, for example, a novel generalisation of 2SAT. △ Less

Submitted 18 January, 2012; originally announced January 2012.

Comments: 19 pages

A Maximal Tractable Class of Soft Constraints

Authors: D. Cohen, M. Cooper, P. Jeavons, A. Krokhin

Abstract: Many researchers in artificial intelligence are beginning to explore the use of soft constraints to express a set of (possibly conflicting) problem requirements. A soft constraint is a function defined on a collection of variables which associates some measure of desirability with each possible combination of values for those variables. However, the crucial question of the computational comple… ▽ More Many researchers in artificial intelligence are beginning to explore the use of soft constraints to express a set of (possibly conflicting) problem requirements. A soft constraint is a function defined on a collection of variables which associates some measure of desirability with each possible combination of values for those variables. However, the crucial question of the computational complexity of finding the optimal solution to a collection of soft constraints has so far received very little attention. In this paper we identify a class of soft binary constraints for which the problem of finding the optimal solution is tractable. In other words, we show that for any given set of such constraints, there exists a polynomial time algorithm to determine the assignment having the best overall combined measure of desirability. This tractable class includes many commonly-occurring soft constraints, such as 'as near as possible' or 'as soon as possible after', as well as crisp constraints such as 'greater than'. Finally, we show that this tractable class is maximal, in the sense that adding any other form of soft binary constraint which is not in the class gives rise to a class of problems which is NP-hard. △ Less

Submitted 30 June, 2011; originally announced July 2011.

Journal ref: Journal Of Artificial Intelligence Research, Volume 22, pages 1-22, 2004

The tractability of CSP classes defined by forbidden patterns

Authors: David A. Cohen, Martin C. Cooper, Páidí Creed, András Z. Salamon

Abstract: The constraint satisfaction problem (CSP) is a general problem central to computer science and artificial intelligence. Although the CSP is NP-hard in general, considerable effort has been spent on identifying tractable subclasses. The main two approaches consider structural properties (restrictions on the hypergraph of constraint scopes) and relational properties (restrictions on the language of… ▽ More The constraint satisfaction problem (CSP) is a general problem central to computer science and artificial intelligence. Although the CSP is NP-hard in general, considerable effort has been spent on identifying tractable subclasses. The main two approaches consider structural properties (restrictions on the hypergraph of constraint scopes) and relational properties (restrictions on the language of constraint relations). Recently, some authors have considered hybrid properties that restrict the constraint hypergraph and the relations simultaneously. Our key contribution is the novel concept of a CSP pattern and classes of problems defined by forbidden patterns (which can be viewed as forbidding generic subproblems). We describe the theoretical framework which can be used to reason about classes of problems defined by forbidden patterns. We show that this framework generalises relational properties and allows us to capture known hybrid tractable classes. Although we are not close to obtaining a dichotomy concerning the tractability of general forbidden patterns, we are able to make some progress in a special case: classes of problems that arise when we can only forbid binary negative patterns (generic subproblems in which only inconsistent tuples are specified). In this case we are able to characterise very large classes of tractable and NP-hard forbidden patterns. This leaves the complexity of just one case unresolved and we conjecture that this last case is tractable. △ Less

Submitted 8 July, 2014; v1 submitted 8 March, 2011; originally announced March 2011.

Journal ref: Journal Of Artificial Intelligence Research, Volume 45, pages 47-78, 2012

Hybrid tractability of soft constraint problems

Authors: Martin C. Cooper, Stanislav Zivny

Abstract: The constraint satisfaction problem (CSP) is a central generic problem in computer science and artificial intelligence: it provides a common framework for many theoretical problems as well as for many real-life applications. Soft constraint problems are a generalisation of the CSP which allow the user to model optimisation problems. Considerable effort has been made in identifying properties which… ▽ More The constraint satisfaction problem (CSP) is a central generic problem in computer science and artificial intelligence: it provides a common framework for many theoretical problems as well as for many real-life applications. Soft constraint problems are a generalisation of the CSP which allow the user to model optimisation problems. Considerable effort has been made in identifying properties which ensure tractability in such problems. In this work, we initiate the study of hybrid tractability of soft constraint problems; that is, properties which guarantee tractability of the given soft constraint problem, but which do not depend only on the underlying structure of the instance (such as being tree-structured) or only on the types of soft constraints in the instance (such as submodularity). We present several novel hybrid classes of soft constraint problems, which include a machine scheduling problem, constraint problems of arbitrary arities with no overlapping nogoods, and the SoftAllDiff constraint with arbitrary unary soft constraints. An important tool in our investigation will be the notion of forbidden substructures. △ Less

Submitted 24 August, 2010; originally announced August 2010.

Comments: A full version of a CP'10 paper, 26 pages

Journal ref: Artificial Intelligence 175(9-10) 1555-1569 (2011)

Arc consistency for soft constraints

Authors: Martin Cooper, Thomas Schiex

Abstract: The notion of arc consistency plays a central role in constraint satisfaction. It is known that the notion of local consistency can be extended to constraint optimisation problems defined by soft constraint frameworks based on an idempotent cost combination operator. This excludes non idempotent operators such as + which define problems which are very important in practical applications such as… ▽ More The notion of arc consistency plays a central role in constraint satisfaction. It is known that the notion of local consistency can be extended to constraint optimisation problems defined by soft constraint frameworks based on an idempotent cost combination operator. This excludes non idempotent operators such as + which define problems which are very important in practical applications such as Max-CSP, where the aim is to minimize the number of violated constraints. In this paper, we show that using a weak additional axiom satisfied by most existing soft constraints proposals, it is possible to define a notion of soft arc consistency that extends the classical notion of arc consistency and this even in the case of non idempotent cost combination operators. A polynomial time algorithm for enforcing this soft arc consistency exists and its space and time complexities are identical to that of enforcing arc consistency in CSPs when the cost combination operator is strictly monotonic (for example Max-CSP). A directional version of arc consistency is potentially even stronger than the non-directional version, since it allows non local propagation of penalties. We demonstrate the utility of directional arc consistency by showing that it not only solves soft constraint problems on trees, but that it also implies a form of local optimality, which we call arc irreducibility. △ Less

Submitted 30 November, 2001; v1 submitted 14 November, 2001; originally announced November 2001.

Comments: Figure 3 modified

ACM Class: I.2.8; F.4.1; D.3.3