Showing 1–40 of 40 results for author: Williams, P

The Language of Weather: Social Media Reactions to Weather Accounting for Climatic and Linguistic Baselines

Authors: James C. Young, Rudy Arthur, Hywel T. P. Williams

Abstract: This study explores how different weather conditions influence public sentiment on social media, focusing on Twitter data from the UK. By considering climate and linguistic baselines, we improve the accuracy of weather-related sentiment analysis. Our findings show that emotional responses to weather are complex, influenced by combinations of weather variables and regional language differences. The… ▽ More This study explores how different weather conditions influence public sentiment on social media, focusing on Twitter data from the UK. By considering climate and linguistic baselines, we improve the accuracy of weather-related sentiment analysis. Our findings show that emotional responses to weather are complex, influenced by combinations of weather variables and regional language differences. The results highlight the importance of context-sensitive methods for better understanding public mood in response to weather, which can enhance impact-based forecasting and risk communication in the context of climate change. △ Less

Submitted 10 July, 2024; originally announced July 2024.

Comments: 12 pages, 5 figures

The Unlikely Duel: Evaluating Creative Writing in LLMs through a Unique Scenario

Authors: Carlos Gómez-Rodríguez, Paul Williams

Abstract: This is a summary of the paper "A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing", which was published in Findings of EMNLP 2023. We evaluate a range of recent state-of-the-art, instruction-tuned large language models (LLMs) on an English creative writing task, and compare them to human writers. For this purpose, we use a specifically-tailored prompt (based on an epi… ▽ More This is a summary of the paper "A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing", which was published in Findings of EMNLP 2023. We evaluate a range of recent state-of-the-art, instruction-tuned large language models (LLMs) on an English creative writing task, and compare them to human writers. For this purpose, we use a specifically-tailored prompt (based on an epic combat between Ignatius J. Reilly, main character of John Kennedy Toole's "A Confederacy of Dunces", and a pterodactyl) to minimize the risk of training data leakage and force the models to be creative rather than reusing existing stories. The same prompt is presented to LLMs and human writers, and evaluation is performed by humans using a detailed rubric including various aspects like fluency, style, originality or humor. Results show that some state-of-the-art commercial LLMs match or slightly outperform our human writers in most of the evaluated dimensions. Open-source LLMs lag behind. Humans keep a close lead in originality, and only the top three LLMs can handle humor at human-like levels. △ Less

Submitted 22 June, 2024; originally announced June 2024.

Comments: Published in the XIX Conference of the Spanish Association for Artificial Intelligence (CAEPIA), 2024. Summary of our paper "A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing", published in Findings of EMNLP

MSC Class: 68T50 ACM Class: I.2.7

Journal ref: XX Conferencia de la Asociación Española para la Inteligencia Artificial (CAEPIA 2024), pages 225-226. ISBN 978-84-09-62724-0

Uncertainty quantification in automated valuation models with locally weighted conformal prediction

Authors: Anders Hjort, Gudmund Horn Hermansen, Johan Pensar, Jonathan P. Williams

Abstract: Non-parametric machine learning models, such as random forests and gradient boosted trees, are frequently used to estimate house prices due to their predictive accuracy, but such methods are often limited in their ability to quantify prediction uncertainty. Conformal Prediction (CP) is a model-agnostic framework for constructing confidence sets around machine learning prediction models with minima… ▽ More Non-parametric machine learning models, such as random forests and gradient boosted trees, are frequently used to estimate house prices due to their predictive accuracy, but such methods are often limited in their ability to quantify prediction uncertainty. Conformal Prediction (CP) is a model-agnostic framework for constructing confidence sets around machine learning prediction models with minimal assumptions. However, due to the spatial dependencies observed in house prices, direct application of CP leads to confidence sets that are not calibrated everywhere, i.e., too large of confidence sets in certain geographical regions and too small in others. We survey various approaches to adjust the CP confidence set to account for this and demonstrate their performance on a data set from the housing market in Oslo, Norway. Our findings indicate that calibrating the confidence sets on a \textit{locally weighted} version of the non-conformity scores makes the coverage more consistently calibrated in different geographical regions. We also perform a simulation study on synthetically generated sale prices to empirically explore the performance of CP on housing market data under idealized conditions with known data-generating mechanisms. △ Less

Submitted 11 December, 2023; originally announced December 2023.

A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing

Authors: Carlos Gómez-Rodríguez, Paul Williams

Abstract: We evaluate a range of recent LLMs on English creative writing, a challenging and complex task that requires imagination, coherence, and style. We use a difficult, open-ended scenario chosen to avoid training data reuse: an epic narration of a single combat between Ignatius J. Reilly, the protagonist of the Pulitzer Prize-winning novel A Confederacy of Dunces (1980), and a pterodactyl, a prehistor… ▽ More We evaluate a range of recent LLMs on English creative writing, a challenging and complex task that requires imagination, coherence, and style. We use a difficult, open-ended scenario chosen to avoid training data reuse: an epic narration of a single combat between Ignatius J. Reilly, the protagonist of the Pulitzer Prize-winning novel A Confederacy of Dunces (1980), and a pterodactyl, a prehistoric flying reptile. We ask several LLMs and humans to write such a story and conduct a human evalution involving various criteria such as fluency, coherence, originality, humor, and style. Our results show that some state-of-the-art commercial LLMs match or slightly outperform our writers in most dimensions; whereas open-source LLMs lag behind. Humans retain an edge in creativity, while humor shows a binary divide between LLMs that can handle it comparably to humans and those that fail at it. We discuss the implications and limitations of our study and suggest directions for future research. △ Less

Submitted 12 October, 2023; originally announced October 2023.

Comments: Accepted for publication in Findings of EMNLP 2023

MSC Class: 68T50; 00A64 ACM Class: I.2.7; J.5

Large Synthetic Data from the arXiv for OCR Post Correction of Historic Scientific Articles

Authors: Jill P. Naiman, Morgan G. Cosillo, Peter K. G. Williams, Alyssa Goodman

Abstract: Scientific articles published prior to the "age of digitization" (~1997) require Optical Character Recognition (OCR) to transform scanned documents into machine-readable text, a process that often produces errors. We develop a pipeline for the generation of a synthetic ground truth/OCR dataset to correct the OCR results of the astrophysics literature holdings of the NASA Astrophysics Data System (… ▽ More Scientific articles published prior to the "age of digitization" (~1997) require Optical Character Recognition (OCR) to transform scanned documents into machine-readable text, a process that often produces errors. We develop a pipeline for the generation of a synthetic ground truth/OCR dataset to correct the OCR results of the astrophysics literature holdings of the NASA Astrophysics Data System (ADS). By mining the arXiv we create, to the authors' knowledge, the largest scientific synthetic ground truth/OCR post correction dataset of 203,354,393 character pairs. We provide baseline models trained with this dataset and find the mean improvement in character and word error rates of 7.71% and 18.82% for historical OCR text, respectively. When used to classify parts of sentences as inline math, we find a classification F1 score of 77.82%. Interactive dashboards to explore the dataset are available online: https://readingtimemachine.github.io/projects/1-ocr-groundtruth-may2023, and data and code, within the limitations of our agreement with the arXiv, are hosted on GitHub: https://github.com/ReadingTimeMachine/ocr_post_correction. △ Less

Submitted 20 September, 2023; originally announced September 2023.

Comments: 6 pages, 1 figure, 1 table; training/validation/test datasets and all model weights to be linked on Zenodo on publication

A Generalized Bandsplit Neural Network for Cinematic Audio Source Separation

Authors: Karn N. Watcharasupat, Chih-Wei Wu, Yiwei Ding, Iroro Orife, Aaron J. Hipple, Phillip A. Williams, Scott Kramer, Alexander Lerch, William Wolcott

Abstract: Cinematic audio source separation is a relatively new subtask of audio source separation, with the aim of extracting the dialogue, music, and effects stems from their mixture. In this work, we developed a model generalizing the Bandsplit RNN for any complete or overcomplete partitions of the frequency axis. Psychoacoustically motivated frequency scales were used to inform the band definitions whic… ▽ More Cinematic audio source separation is a relatively new subtask of audio source separation, with the aim of extracting the dialogue, music, and effects stems from their mixture. In this work, we developed a model generalizing the Bandsplit RNN for any complete or overcomplete partitions of the frequency axis. Psychoacoustically motivated frequency scales were used to inform the band definitions which are now defined with redundancy for more reliable feature extraction. A loss function motivated by the signal-to-noise ratio and the sparsity-promoting property of the 1-norm was proposed. We additionally exploit the information-sharing property of a common-encoder setup to reduce computational complexity during both training and inference, improve separation performance for hard-to-generalize classes of sounds, and allow flexibility during inference time with detachable decoders. Our best model sets the state of the art on the Divide and Remaster dataset with performance above the ideal ratio mask for the dialogue stem. △ Less

Submitted 1 December, 2023; v1 submitted 5 September, 2023; originally announced September 2023.

Comments: Accepted to the IEEE Open Journal of Signal Processing (ICASSP 2024 Track)

Journal ref: IEEE Open Journal of Signal Processing, vol. 5, pp. 73-81, 2024

Model-free generalized fiducial inference

Authors: Jonathan P Williams

Abstract: Motivated by the need for the development of safe and reliable methods for uncertainty quantification in machine learning, I propose and develop ideas for a model-free statistical framework for imprecise probabilistic prediction inference. This framework facilitates uncertainty quantification in the form of prediction sets that offer finite sample control of type 1 errors, a property shared with c… ▽ More Motivated by the need for the development of safe and reliable methods for uncertainty quantification in machine learning, I propose and develop ideas for a model-free statistical framework for imprecise probabilistic prediction inference. This framework facilitates uncertainty quantification in the form of prediction sets that offer finite sample control of type 1 errors, a property shared with conformal prediction sets, but this new approach also offers more versatile tools for imprecise probabilistic reasoning. Furthermore, I propose and consider the theoretical and empirical properties of a precise probabilistic approximation to the model-free imprecise framework. Approximating a belief/plausibility measure pair by an [optimal in some sense] probability measure in the credal set is a critical resolution needed for the broader adoption of imprecise probabilistic approaches to inference in statistical and machine learning communities. It is largely undetermined in the statistical and machine learning literatures, more generally, how to properly quantify uncertainty in that there is no generally accepted standard of accountability of stated uncertainties. The research I present in this manuscript is aimed at motivating a framework for statistical inference with reliability and accountability as the guiding principles. △ Less

Submitted 23 July, 2023; originally announced July 2023.

Leveraging arbitrary mobile sensor trajectories with shallow recurrent decoder networks for full-state reconstruction

Authors: Megan R. Ebers, Jan P. Williams, Katherine M. Steele, J. Nathan Kutz

Abstract: Sensing is one of the most fundamental tasks for the monitoring, forecasting and control of complex, spatio-temporal systems. In many applications, a limited number of sensors are mobile and move with the dynamics, with examples including wearable technology, ocean monitoring buoys, and weather balloons. In these dynamic systems (without regions of statistical-independence), the measurement time h… ▽ More Sensing is one of the most fundamental tasks for the monitoring, forecasting and control of complex, spatio-temporal systems. In many applications, a limited number of sensors are mobile and move with the dynamics, with examples including wearable technology, ocean monitoring buoys, and weather balloons. In these dynamic systems (without regions of statistical-independence), the measurement time history encodes a significant amount of information that can be extracted for critical tasks. Most model-free sensing paradigms aim to map current sparse sensor measurements to the high-dimensional state space, ignoring the time-history all together. Using modern deep learning architectures, we show that a sequence-to-vector model, such as an LSTM (long, short-term memory) network, with a decoder network, dynamic trajectory information can be mapped to full state-space estimates. Indeed, we demonstrate that by leveraging mobile sensor trajectories with shallow recurrent decoder networks, we can train the network (i) to accurately reconstruct the full state space using arbitrary dynamical trajectories of the sensors, (ii) the architecture reduces the variance of the mean-square error of the reconstruction error in comparison with immobile sensors, and (iii) the architecture also allows for rapid generalization (parameterization of dynamics) for data outside the training set. Moreover, the path of the sensor can be chosen arbitrarily, provided training data for the spatial trajectory of the sensor is available. The exceptional performance of the network architecture is demonstrated on three applications: turbulent flows, global sea-surface temperature data, and human movement biomechanics. △ Less

Submitted 20 July, 2023; originally announced July 2023.

Comments: 11 pages, 5 figures, 2 tables

CIDER: Context sensitive sentiment analysis for short-form text

Authors: James C. Young, Rudy Arthur, Hywel T. P. Williams

Abstract: Researchers commonly perform sentiment analysis on large collections of short texts like tweets, Reddit posts or newspaper headlines that are all focused on a specific topic, theme or event. Usually, general-purpose sentiment analysis methods are used. These perform well on average but miss the variation in meaning that happens across different contexts, for example, the word "active" has a very d… ▽ More Researchers commonly perform sentiment analysis on large collections of short texts like tweets, Reddit posts or newspaper headlines that are all focused on a specific topic, theme or event. Usually, general-purpose sentiment analysis methods are used. These perform well on average but miss the variation in meaning that happens across different contexts, for example, the word "active" has a very different intention and valence in the phrase "active lifestyle" versus "active volcano". This work presents a new approach, CIDER (Context Informed Dictionary and sEmantic Reasoner), which performs context-sensitive linguistic analysis, where the valence of sentiment-laden terms is inferred from the whole corpus before being used to score the individual texts. In this paper, we detail the CIDER algorithm and demonstrate that it outperforms state-of-the-art generalist unsupervised sentiment analysis techniques on a large collection of tweets about the weather. CIDER is also applicable to alternative (non-sentiment) linguistic scales. A case study on gender in the UK is presented, with the identification of highly gendered and sentiment-laden days. We have made our implementation of CIDER available as a Python package: https://pypi.org/project/ciderpolarity/. △ Less

Submitted 10 July, 2024; v1 submitted 15 July, 2023; originally announced July 2023.

Comments: 20 pages, 6 figures, 3 tables

Real-World Performance of Autonomously Reporting Normal Chest Radiographs in NHS Trusts Using a Deep-Learning Algorithm on the GP Pathway

Authors: Jordan Smith, Tom Naunton Morgan, Paul Williams, Qaiser Malik, Simon Rasalingham

Abstract: AIM To analyse the performance of a deep-learning (DL) algorithm currently deployed as diagnostic decision support software in two NHS Trusts used to identify normal chest x-rays in active clinical pathways. MATERIALS AND METHODS A DL algorithm has been deployed in Somerset NHS Foundation Trust (SFT) since December 2022, and at Calderdale & Huddersfield NHS Foundation Trust (CHFT) since March 20… ▽ More AIM To analyse the performance of a deep-learning (DL) algorithm currently deployed as diagnostic decision support software in two NHS Trusts used to identify normal chest x-rays in active clinical pathways. MATERIALS AND METHODS A DL algorithm has been deployed in Somerset NHS Foundation Trust (SFT) since December 2022, and at Calderdale & Huddersfield NHS Foundation Trust (CHFT) since March 2023. The algorithm was developed and trained prior to deployment, and is used to assign abnormality scores to each GP-requested chest x-ray (CXR). The algorithm classifies a subset of examinations with the lowest abnormality scores as High Confidence Normal (HCN), and displays this result to the Trust. This two-site study includes 4,654 CXR continuous examinations processed by the algorithm over a six-week period. RESULTS When classifying 20.0% of assessed examinations (930) as HCN, the model classified exams with a negative predictive value (NPV) of 0.96. There were 0.77% of examinations (36) classified incorrectly as HCN, with none of the abnormalities considered clinically significant by auditing radiologists. The DL software maintained fast levels of service to clinicians, with results returned to Trusts in a mean time of 7.1 seconds. CONCLUSION The DL algorithm performs with a low rate of error and is highly effective as an automated diagnostic decision support tool, used to autonomously report a subset of CXRs as normal with high confidence. Removing 20% of all CXRs reduces workload for reporters and allows radiology departments to focus resources elsewhere. △ Less

Submitted 28 June, 2023; originally announced June 2023.

Comments: 7 pages, 5 figures, 2 tables. Submitted to Clinical Radiology

Deakin RF-Sensing: Experiments on Correlated Knowledge Distillation for Monitoring Human Postures with Radios

Authors: Shiva Raj Pokhrel, Jonathan Kua, Deol Satish, Philip Williams, Arkady Zaslavsky, Seng W. Loke, Jinho Choi

Abstract: In this work, we propose and develop a simple experimental testbed to study the feasibility of a novel idea by coupling radio frequency (RF) sensing technology with Correlated Knowledge Distillation (CKD) theory towards designing lightweight, near real-time and precise human pose monitoring systems. The proposed CKD framework transfers and fuses pose knowledge from a robust "Teacher" model to a pa… ▽ More In this work, we propose and develop a simple experimental testbed to study the feasibility of a novel idea by coupling radio frequency (RF) sensing technology with Correlated Knowledge Distillation (CKD) theory towards designing lightweight, near real-time and precise human pose monitoring systems. The proposed CKD framework transfers and fuses pose knowledge from a robust "Teacher" model to a parameterized "Student" model, which can be a promising technique for obtaining accurate yet lightweight pose estimates. To assure its efficacy, we implemented CKD for distilling logits in our integrated Software Defined Radio (SDR)-based experimental setup and investigated the RF-visual signal correlation. Our CKD-RF sensing technique is characterized by two modes - a camera-fed Teacher Class Network (e.g., images, videos) with an SDR-fed Student Class Network (e.g., RF signals). Specifically, our CKD model trains a dual multi-branch teacher and student network by distilling and fusing knowledge bases. The resulting CKD models are then subsequently used to identify the multimodal correlation and teach the student branch in reverse. Instead of simply aggregating their learnings, CKD training comprised multiple parallel transformations with the two domains, i.e., visual images and RF signals. Once trained, our CKD model can efficiently preserve privacy and utilize the multimodal correlated logits from the two different neural networks for estimating poses without using visual signals/video frames (by using only the RF signals). △ Less

Submitted 10 October, 2023; v1 submitted 24 May, 2023; originally announced May 2023.

Using Semantic Similarity and Text Embedding to Measure the Social Media Echo of Strategic Communications

Authors: Tristan J. B. Cann, Ben Dennes, Travis Coan, Saffron O'Neill, Hywel T. P. Williams

Abstract: Online discourse covers a wide range of topics and many actors tailor their content to impact online discussions through carefully crafted messages and targeted campaigns. Yet the scale and diversity of online media content make it difficult to evaluate the impact of a particular message. In this paper, we present a new technique that leverages semantic similarity to quantify the change in the dis… ▽ More Online discourse covers a wide range of topics and many actors tailor their content to impact online discussions through carefully crafted messages and targeted campaigns. Yet the scale and diversity of online media content make it difficult to evaluate the impact of a particular message. In this paper, we present a new technique that leverages semantic similarity to quantify the change in the discussion after a particular message has been published. We use a set of press releases from environmental organisations and tweets from the climate change debate to show that our novel approach reveals a heavy-tailed distribution of response in online discourse to strategic communications. △ Less

Submitted 29 March, 2023; originally announced March 2023.

Comments: 12 pages, 5 figures

The Digitization of Historical Astrophysical Literature with Highly-Localized Figures and Figure Captions

Authors: Jill P. Naiman, Peter K. G. Williams, Alyssa Goodman

Abstract: Scientific articles published prior to the "age of digitization" in the late 1990s contain figures which are "trapped" within their scanned pages. While progress to extract figures and their captions has been made, there is currently no robust method for this process. We present a YOLO-based method for use on scanned pages, after they have been processed with Optical Character Recognition (OCR), w… ▽ More Scientific articles published prior to the "age of digitization" in the late 1990s contain figures which are "trapped" within their scanned pages. While progress to extract figures and their captions has been made, there is currently no robust method for this process. We present a YOLO-based method for use on scanned pages, after they have been processed with Optical Character Recognition (OCR), which uses both grayscale and OCR-features. We focus our efforts on translating the intersection-over-union (IOU) metric from the field of object detection to document layout analysis and quantify "high localization" levels as an IOU of 0.9. When applied to the astrophysics literature holdings of the NASA Astrophysics Data System (ADS), we find F1 scores of 90.9% (92.2%) for figures (figure captions) with the IOU cut-off of 0.9 which is a significant improvement over other state-of-the-art methods. △ Less

Submitted 22 February, 2023; originally announced February 2023.

Comments: 29 pages, 10 figures, accepted for publication in the International Journal on Digital Libraries, special issue follow up to TPDL 2022 conference. arXiv admin note: substantial text overlap with arXiv:2209.04460

Valid Inference for Machine Learning Model Parameters

Authors: Neil Dey, Jonathan P. Williams

Abstract: The parameters of a machine learning model are typically learned by minimizing a loss function on a set of training data. However, this can come with the risk of overtraining; in order for the model to generalize well, it is of great importance that we are able to find the optimal parameter for the model on the entire population -- not only on the given training sample. In this paper, we construct… ▽ More The parameters of a machine learning model are typically learned by minimizing a loss function on a set of training data. However, this can come with the risk of overtraining; in order for the model to generalize well, it is of great importance that we are able to find the optimal parameter for the model on the entire population -- not only on the given training sample. In this paper, we construct valid confidence sets for this optimal parameter of a machine learning model, which can be generated using only the training data without any knowledge of the population. We then show that studying the distribution of this confidence set allows us to assign a notion of confidence to arbitrary regions of the parameter space, and we demonstrate that this distribution can be well-approximated using bootstrapping techniques. △ Less

Submitted 9 May, 2024; v1 submitted 21 February, 2023; originally announced February 2023.

Comments: 35 pages, 6 figures

Robustness of an Artificial Intelligence Solution for Diagnosis of Normal Chest X-Rays

Authors: Tom Dyer, Jordan Smith, Gaetan Dissez, Nicole Tay, Qaiser Malik, Tom Naunton Morgan, Paul Williams, Liliana Garcia-Mondragon, George Pearse, Simon Rasalingham

Abstract: Purpose: Artificial intelligence (AI) solutions for medical diagnosis require thorough evaluation to demonstrate that performance is maintained for all patient sub-groups and to ensure that proposed improvements in care will be delivered equitably. This study evaluates the robustness of an AI solution for the diagnosis of normal chest X-rays (CXRs) by comparing performance across multiple patient… ▽ More Purpose: Artificial intelligence (AI) solutions for medical diagnosis require thorough evaluation to demonstrate that performance is maintained for all patient sub-groups and to ensure that proposed improvements in care will be delivered equitably. This study evaluates the robustness of an AI solution for the diagnosis of normal chest X-rays (CXRs) by comparing performance across multiple patient and environmental subgroups, as well as comparing AI errors with those made by human experts. Methods: A total of 4,060 CXRs were sampled to represent a diverse dataset of NHS patients and care settings. Ground-truth labels were assigned by a 3-radiologist panel. AI performance was evaluated against assigned labels and sub-groups analysis was conducted against patient age and sex, as well as CXR view, modality, device manufacturer and hospital site. Results: The AI solution was able to remove 18.5% of the dataset by classification as High Confidence Normal (HCN). This was associated with a negative predictive value (NPV) of 96.0%, compared to 89.1% for diagnosis of normal scans by radiologists. In all AI false negative (FN) cases, a radiologist was found to have also made the same error when compared to final ground-truth labels. Subgroup analysis showed no statistically significant variations in AI performance, whilst reduced normal classification was observed in data from some hospital sites. Conclusion: We show the AI solution could provide meaningful workload savings by diagnosis of 18.5% of scans as HCN with a superior NPV to human readers. The AI solution is shown to perform well across patient subgroups and error cases were shown to be subjective or subtle in nature. △ Less

Submitted 31 August, 2022; originally announced September 2022.

Figure and Figure Caption Extraction for Mixed Raster and Vector PDFs: Digitization of Astronomical Literature with OCR Features

Authors: J. P. Naiman, Peter K. G. Williams, Alyssa Goodman

Abstract: Scientific articles published prior to the "age of digitization" in the late 1990s contain figures which are "trapped" within their scanned pages. While progress to extract figures and their captions has been made, there is currently no robust method for this process. We present a YOLO-based method for use on scanned pages, post-Optical Character Recognition (OCR), which uses both grayscale and OC… ▽ More Scientific articles published prior to the "age of digitization" in the late 1990s contain figures which are "trapped" within their scanned pages. While progress to extract figures and their captions has been made, there is currently no robust method for this process. We present a YOLO-based method for use on scanned pages, post-Optical Character Recognition (OCR), which uses both grayscale and OCR-features. When applied to the astrophysics literature holdings of the Astrophysics Data System (ADS), we find F1 scores of 90.9% (92.2%) for figures (figure captions) with the intersection-over-union (IOU) cut-off of 0.9 which is a significant improvement over other state-of-the-art methods. △ Less

Submitted 9 September, 2022; originally announced September 2022.

Comments: 16 pages, 3 figures, accepted to TPDL 2022

Enhancing Early Lung Cancer Detection on Chest Radiographs with AI-assistance: A Multi-Reader Study

Authors: Gaetan Dissez, Nicole Tay, Tom Dyer, Matthew Tam, Richard Dittrich, David Doyne, James Hoare, Jackson J. Pat, Stephanie Patterson, Amanda Stockham, Qaiser Malik, Tom Naunton Morgan, Paul Williams, Liliana Garcia-Mondragon, Jordan Smith, George Pearse, Simon Rasalingham

Abstract: Objectives: The present study evaluated the impact of a commercially available explainable AI algorithm in augmenting the ability of clinicians to identify lung cancer on chest X-rays (CXR). Design: This retrospective study evaluated the performance of 11 clinicians for detecting lung cancer from chest radiographs, with and without assistance from a commercially available AI algorithm (red dot,… ▽ More Objectives: The present study evaluated the impact of a commercially available explainable AI algorithm in augmenting the ability of clinicians to identify lung cancer on chest X-rays (CXR). Design: This retrospective study evaluated the performance of 11 clinicians for detecting lung cancer from chest radiographs, with and without assistance from a commercially available AI algorithm (red dot, Behold.ai) that predicts suspected lung cancer from CXRs. Clinician performance was evaluated against clinically confirmed diagnoses. Setting: The study analysed anonymised patient data from an NHS hospital; the dataset consisted of 400 chest radiographs from adult patients (18 years and above) who had a CXR performed in 2020, with corresponding clinical text reports. Participants: A panel of readers consisting of 11 clinicians (consultant radiologists, radiologist trainees and reporting radiographers) participated in this study. Main outcome measures: Overall accuracy, sensitivity, specificity and precision for detecting lung cancer on CXRs by clinicians, with and without AI input. Agreement rates between clinicians and performance standard deviation were also evaluated, with and without AI input. Results: The use of the AI algorithm by clinicians led to an improved overall performance for lung tumour detection, achieving an overall increase of 17.4% of lung cancers being identified on CXRs which would have otherwise been missed, an overall increase in detection of smaller tumours, a 24% and 13% increased detection of stage 1 and stage 2 lung cancers respectively, and standardisation of clinician performance. Conclusions: This study showed great promise in the clinical utility of AI algorithms in improving early lung cancer diagnosis and promoting health equity through overall improvement in reader performances, without impacting downstream imaging resources. △ Less

Submitted 31 August, 2022; originally announced August 2022.

The structure of segregation in co-authorship networks and its impact on scientific production

Authors: Ana Maria Jaramillo, Hywel T. P. Williams, Nicola Perra, Ronaldo Menezes

Abstract: Co-authorship networks, where nodes represent authors and edges represent co-authorship relations, are key to understanding the production and diffusion of knowledge in academia. Social constructs, biases (implicit and explicit), and constraints (e.g. spatial, temporal) affect who works with whom and cause co-authorship networks to organise into tight communities with different levels of segregati… ▽ More Co-authorship networks, where nodes represent authors and edges represent co-authorship relations, are key to understanding the production and diffusion of knowledge in academia. Social constructs, biases (implicit and explicit), and constraints (e.g. spatial, temporal) affect who works with whom and cause co-authorship networks to organise into tight communities with different levels of segregation. We aim to look at aspects of the co-authorship network structure that lead to segregation and its impact on scientific production. We measure segregation using the Spectral Segregation Index (SSI) and find 4 ordered segregation categories: completely segregated, highly segregated, moderately segregated and non-segregated communities. We direct our attention to the non-segregated and highly segregated communities, quantifying and comparing their structural topologies and k-core positions. When considering communities of both categories (controlling for size), our results show no differences in density and clustering but substantial variability in core position. Larger non-segregated communities are more likely to occupy cores near the network nucleus, while the highly segregated ones tend to be closer to the network periphery. Finally, we analyse differences in citations gained by researchers within communities showing different segregation categories. Researchers in highly segregated communities get more citations from their community members in middle cores and gain more citations per publication in middle/periphery cores. Those in non-segregated communities get more citations per publication in the nucleus. To our knowledge, this work is the first to characterise community segregation in co-authorship networks and investigate the relationship between community segregation and author citations. △ Less

Submitted 3 May, 2023; v1 submitted 20 July, 2022; originally announced July 2022.

Comments: 13 pages, 5 figures

Art-Attack: Black-Box Adversarial Attack via Evolutionary Art

Authors: Phoenix Williams, Ke Li

Abstract: Deep neural networks (DNNs) have achieved state-of-the-art performance in many tasks but have shown extreme vulnerabilities to attacks generated by adversarial examples. Many works go with a white-box attack that assumes total access to the targeted model including its architecture and gradients. A more realistic assumption is the black-box scenario where an attacker only has access to the targete… ▽ More Deep neural networks (DNNs) have achieved state-of-the-art performance in many tasks but have shown extreme vulnerabilities to attacks generated by adversarial examples. Many works go with a white-box attack that assumes total access to the targeted model including its architecture and gradients. A more realistic assumption is the black-box scenario where an attacker only has access to the targeted model by querying some input and observing its predicted class probabilities. Different from most prevalent black-box attacks that make use of substitute models or gradient estimation, this paper proposes a gradient-free attack by using a concept of evolutionary art to generate adversarial examples that iteratively evolves a set of overlapping transparent shapes. To evaluate the effectiveness of our proposed method, we attack three state-of-the-art image classification models trained on the CIFAR-10 dataset in a targeted manner. We conduct a parameter study outlining the impact the number and type of shapes have on the proposed attack's performance. In comparison to state-of-the-art black-box attacks, our attack is more effective at generating adversarial examples and achieves a higher attack success rate on all three baseline models. △ Less

Submitted 7 March, 2022; originally announced March 2022.

Is there an aesthetic component of language?

Authors: Harshit Parmar, Jeffrey P. Williams

Abstract: Speakers of all human languages make use of grammatical devices to express attributional qualities, feelings, and opinions as well as to provide meta-commentary on topics in discourse. In general, linguists refer to this category as 'expressives'in spite of the fact that defining exactly what 'expressives' are remains elusive. The elusiveness of expressives has given rise to considerable speculati… ▽ More Speakers of all human languages make use of grammatical devices to express attributional qualities, feelings, and opinions as well as to provide meta-commentary on topics in discourse. In general, linguists refer to this category as 'expressives'in spite of the fact that defining exactly what 'expressives' are remains elusive. The elusiveness of expressives has given rise to considerable speculation about the nature of expressivity as a linguistic principle. Specifically, several scholars have pointed out the 'special' or 'unusual' nature of expressives vis-a-vis 'normal' or 'natural' morpho-syntax. △ Less

Submitted 19 February, 2022; originally announced February 2022.

Conformal prediction for text infilling and part-of-speech prediction

Authors: Neil Dey, Jing Ding, Jack Ferrell, Carolina Kapper, Maxwell Lovig, Emiliano Planchon, Jonathan P Williams

Abstract: Modern machine learning algorithms are capable of providing remarkably accurate point-predictions; however, questions remain about their statistical reliability. Unlike conventional machine learning methods, conformal prediction algorithms return confidence sets (i.e., set-valued predictions) that correspond to a given significance level. Moreover, these confidence sets are valid in the sense that… ▽ More Modern machine learning algorithms are capable of providing remarkably accurate point-predictions; however, questions remain about their statistical reliability. Unlike conventional machine learning methods, conformal prediction algorithms return confidence sets (i.e., set-valued predictions) that correspond to a given significance level. Moreover, these confidence sets are valid in the sense that they guarantee finite sample control over type 1 error probabilities, allowing the practitioner to choose an acceptable error rate. In our paper, we propose inductive conformal prediction (ICP) algorithms for the tasks of text infilling and part-of-speech (POS) prediction for natural language data. We construct new conformal prediction-enhanced bidirectional encoder representations from transformers (BERT) and bidirectional long short-term memory (BiLSTM) algorithms for POS tagging and a new conformal prediction-enhanced BERT algorithm for text infilling. We analyze the performance of the algorithms in simulations using the Brown Corpus, which contains over 57,000 sentences. Our results demonstrate that the ICP algorithms are able to produce valid set-valued predictions that are small enough to be applicable in real-world applications. We also provide a real data example for how our proposed set-valued predictions can improve machine generated audio transcriptions. △ Less

Submitted 3 November, 2021; originally announced November 2021.

Solving the Families In the Wild Kinship Verification Challenge by Program Synthesis

Authors: Junyi Huang, Maxwell Benjamin Strome, Ian Jenkins, Parker Williams, Bo Feng, Yaning Wang, Roman Wang, Vaibhav Bagri, Newman Cheng, Iddo Drori

Abstract: Kinship verification is the task of determining whether a parent-child, sibling, or grandparent-grandchild relationship exists between two people and is important in social media applications, forensic investigations, finding missing children, and reuniting families. We demonstrate high quality kinship verification by participating in the 2021 Recognizing Families in the Wild challenge which provi… ▽ More Kinship verification is the task of determining whether a parent-child, sibling, or grandparent-grandchild relationship exists between two people and is important in social media applications, forensic investigations, finding missing children, and reuniting families. We demonstrate high quality kinship verification by participating in the 2021 Recognizing Families in the Wild challenge which provides the largest publicly available dataset in the field. Our approach is among the top 3 winning entries in the competition. We ensemble models written by both human experts and a foundation model, OpenAI Codex, trained on text and code. We use Codex to generate model variants, and also demonstrate its ability to generate entire running programs for kinship verification tasks of specific relationships. △ Less

Submitted 2 December, 2021; v1 submitted 13 October, 2021; originally announced October 2021.

Comments: 5 pages, 2 figures

Journal ref: IEEE International Conference on Automatic Face and Gesture Recognition, 2021

The ELITR ECA Corpus

Authors: Philip Williams, Barry Haddow

Abstract: We present the ELITR ECA corpus, a multilingual corpus derived from publications of the European Court of Auditors. We use automatic translation together with Bleualign to identify parallel sentence pairs in all 506 translation directions. The result is a corpus comprising 264k document pairs and 41.9M sentence pairs. We present the ELITR ECA corpus, a multilingual corpus derived from publications of the European Court of Auditors. We use automatic translation together with Bleualign to identify parallel sentence pairs in all 506 translation directions. The result is a corpus comprising 264k document pairs and 41.9M sentence pairs. △ Less

Submitted 15 September, 2021; originally announced September 2021.

Quantifying sources of uncertainty in drug discovery predictions with probabilistic models

Authors: Stanley E. Lazic, Dominic P. Williams

Abstract: Knowing the uncertainty in a prediction is critical when making expensive investment decisions and when patient safety is paramount, but machine learning (ML) models in drug discovery typically provide only a single best estimate and ignore all sources of uncertainty. Predictions from these models may therefore be over-confident, which can put patients at risk and waste resources when compounds th… ▽ More Knowing the uncertainty in a prediction is critical when making expensive investment decisions and when patient safety is paramount, but machine learning (ML) models in drug discovery typically provide only a single best estimate and ignore all sources of uncertainty. Predictions from these models may therefore be over-confident, which can put patients at risk and waste resources when compounds that are destined to fail are further developed. Probabilistic predictive models (PPMs) can incorporate uncertainty in both the data and model, and return a distribution of predicted values that represents the uncertainty in the prediction. PPMs not only let users know when predictions are uncertain, but the intuitive output from these models makes communicating risk easier and decision making better. Many popular machine learning methods have a PPM or Bayesian analogue, making PPMs easy to fit into current workflows. We use toxicity prediction as a running example, but the same principles apply for all prediction models used in drug discovery. The consequences of ignoring uncertainty and how PPMs account for uncertainty are also described. We aim to make the discussion accessible to a broad non-mathematical audience. Equations are provided to make ideas concrete for mathematical readers (but can be skipped without loss of understanding) and code is available for computational researchers (https://github.com/stanlazic/ML_uncertainty_quantification). △ Less

Submitted 18 May, 2021; originally announced May 2021.

Comments: 34 pages, 9 figures

Journal ref: Artificial Intelligence in the Life Sciences (2021)

ELITR Non-Native Speech Translation at IWSLT 2020

Authors: Dominik Macháček, Jonáš Kratochvíl, Sangeet Sagar, Matúš Žilinec, Ondřej Bojar, Thai-Son Nguyen, Felix Schneider, Philip Williams, Yuekun Yao

Abstract: This paper is an ELITR system submission for the non-native speech translation task at IWSLT 2020. We describe systems for offline ASR, real-time ASR, and our cascaded approach to offline SLT and real-time SLT. We select our primary candidates from a pool of pre-existing systems, develop a new end-to-end general ASR system, and a hybrid ASR trained on non-native speech. The provided small validati… ▽ More This paper is an ELITR system submission for the non-native speech translation task at IWSLT 2020. We describe systems for offline ASR, real-time ASR, and our cascaded approach to offline SLT and real-time SLT. We select our primary candidates from a pool of pre-existing systems, develop a new end-to-end general ASR system, and a hybrid ASR trained on non-native speech. The provided small validation set prevents us from carrying out a complex validation, but we submit all the unselected candidates for contrastive evaluation on the test set. △ Less

Submitted 5 June, 2020; originally announced June 2020.

Comments: IWSLT 2020

Complex networks for event detection in heterogeneous high volume news streams

Authors: Iraklis Moutidis, Hywel T. P. Williams

Abstract: Detecting important events in high volume news streams is an important task for a variety of purposes.The volume and rate of online news increases the need for automated event detection methods thatcan operate in real time. In this paper we develop a network-based approach that makes the workingassumption that important news events always involve named entities (such as persons, locationsand organ… ▽ More Detecting important events in high volume news streams is an important task for a variety of purposes.The volume and rate of online news increases the need for automated event detection methods thatcan operate in real time. In this paper we develop a network-based approach that makes the workingassumption that important news events always involve named entities (such as persons, locationsand organizations) that are linked in news articles. Our approach uses natural language processingtechniques to detect these entities in a stream of news articles and then creates a time-stamped seriesof networks in which the detected entities are linked by co-occurrence in articles and sentences. Inthis prototype, weighted node degree is tracked over time and change-point detection used to locateimportant events. Potential events are characterized and distinguished using community detectionon KeyGraphs that relate named entities and informative noun-phrases from related articles. Thismethodology already produces promising results and will be extended in future to include a widervariety of complex network analysis techniques. △ Less

Submitted 27 May, 2020; originally announced May 2020.

Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation

Authors: Biao Zhang, Philip Williams, Ivan Titov, Rico Sennrich

Abstract: Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations. In this paper, we explore ways to improve them. We argue that multilingual NMT requires stronger modeling capacity to support language pairs with varying typological characteristics, and overcome this bottleneck via langua… ▽ More Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations. In this paper, we explore ways to improve them. We argue that multilingual NMT requires stronger modeling capacity to support language pairs with varying typological characteristics, and overcome this bottleneck via language-specific components and deepening NMT architectures. We identify the off-target translation issue (i.e. translating into a wrong target language) as the major source of the inferior zero-shot performance, and propose random online backtranslation to enforce the translation of unseen training language pairs. Experiments on OPUS-100 (a novel multilingual dataset with 100 languages) show that our approach substantially narrows the performance gap with bilingual models in both one-to-many and many-to-many settings, and improves zero-shot performance by ~10 BLEU, approaching conventional pivot-based methods. △ Less

Submitted 24 April, 2020; originally announced April 2020.

Comments: ACL2020

Taming the Knight's Tour: Minimizing Turns and Crossings

Authors: Juan Jose Besa, Timothy Johnson, Nil Mamano, Martha C. Osegueda, Parker Williams

Abstract: We introduce two new metrics of "simplicity" for knight's tours: the number of turns and the number of crossings. We give a novel algorithm that produces tours with $9.25n+O(1)$ turns and $12n+O(1)$ crossings on an $n\times n$ board, and we show lower bounds of $(6-ε)n$ and $4n-O(1)$ on the respective problems of minimizing these metrics. Hence, our algorithm achieves approximation ratios of… ▽ More We introduce two new metrics of "simplicity" for knight's tours: the number of turns and the number of crossings. We give a novel algorithm that produces tours with $9.25n+O(1)$ turns and $12n+O(1)$ crossings on an $n\times n$ board, and we show lower bounds of $(6-ε)n$ and $4n-O(1)$ on the respective problems of minimizing these metrics. Hence, our algorithm achieves approximation ratios of $9.25/6+o(1)$ and $3+o(1)$. Our algorithm takes linear time and is fully parallelizable, i.e., the tour can be computed in $O(n^2/p)$ time using $p$ processors in the CREW PRAM model. We generalize our techniques to rectangular boards, high-dimensional boards, symmetric tours, odd boards with a missing corner, and tours for $(1,4)$-leapers. In doing so, we show that these extensions also admit a constant approximation ratio on the minimum number of turns, and on the number of crossings in most cases. △ Less

Submitted 17 January, 2022; v1 submitted 4 April, 2019; originally announced April 2019.

Comments: 43 pages, 29 figures. FUN 2020 (FUN with Algorithms); FUN 2020 special issue in TCS

The Human Geography of Twitter

Authors: Rudy Arthur, Hywel T. P. Williams

Abstract: Given the centrality of regions in social movements, politics and public administration we aim to quantitatively study inter- and intra-regional communication for the first time. This work uses social media posts to first identify contiguous geographical regions with a shared social identity and then investigate patterns of communication within and between them. Our case study uses over 150 days o… ▽ More Given the centrality of regions in social movements, politics and public administration we aim to quantitatively study inter- and intra-regional communication for the first time. This work uses social media posts to first identify contiguous geographical regions with a shared social identity and then investigate patterns of communication within and between them. Our case study uses over 150 days of located Twitter data from England and Wales. In contrast to other approaches, (e.g. phone call data records or online friendship networks) we have the message contents as well as the social connection. This allows us to investigate not only the volume of communication but also the sentiment and vocabulary. We find that the South-East and North-West regions are the most talked about; regions tend to be more positive about themselves than about others; people talk politics much more between regions than within. This methodology gives researchers a powerful tool to study identity and interaction within and between social-geographic regions. △ Less

Submitted 11 July, 2018; originally announced July 2018.

Comments: 11 pages, 5 figures, 4 tables, 6 pages of Supporting Information

Social Sensing of Floods in the UK

Authors: Rudy Arthur, Chris A. Boulton, Humphrey Shotton, Hywel T. P. Williams

Abstract: "Social sensing" is a form of crowd-sourcing that involves systematic analysis of digital communications to detect real-world events. Here we consider the use of social sensing for observing natural hazards. In particular, we present a case study that uses data from a popular social media platform (Twitter) to detect and locate flood events in the UK. In order to improve data quality we apply a nu… ▽ More "Social sensing" is a form of crowd-sourcing that involves systematic analysis of digital communications to detect real-world events. Here we consider the use of social sensing for observing natural hazards. In particular, we present a case study that uses data from a popular social media platform (Twitter) to detect and locate flood events in the UK. In order to improve data quality we apply a number of filters (timezone, simple text filters and a naive Bayes `relevance' filter) to the data. We then use place names in the user profile and message text to infer the location of the tweets. These two steps remove most of the irrelevant tweets and yield orders of magnitude more located tweets than we have by relying on geo-tagged data. We demonstrate that high resolution social sensing of floods is feasible and we can produce high-quality historical and real-time maps of floods using Twitter. △ Less

Submitted 13 November, 2017; originally announced November 2017.

Comments: 24 pages, 6 figures

The University of Edinburgh's Neural MT Systems for WMT17

Authors: Rico Sennrich, Alexandra Birch, Anna Currey, Ulrich Germann, Barry Haddow, Kenneth Heafield, Antonio Valerio Miceli Barone, Philip Williams

Abstract: This paper describes the University of Edinburgh's submissions to the WMT17 shared news translation and biomedical translation tasks. We participated in 12 translation directions for news, translating between English and Czech, German, Latvian, Russian, Turkish and Chinese. For the biomedical task we submitted systems for English to Czech, German, Polish and Romanian. Our systems are neural machin… ▽ More This paper describes the University of Edinburgh's submissions to the WMT17 shared news translation and biomedical translation tasks. We participated in 12 translation directions for news, translating between English and Czech, German, Latvian, Russian, Turkish and Chinese. For the biomedical task we submitted systems for English to Czech, German, Polish and Romanian. Our systems are neural machine translation systems trained with Nematus, an attentional encoder-decoder. We follow our setup from last year and build BPE-based models with parallel and back-translated monolingual training data. Novelties this year include the use of deep architectures, layer normalization, and more compact models due to weight tying and improvements in BPE segmentations. We perform extensive ablative experiments, reporting on the effectivenes of layer normalization, deep architectures, and different ensembling techniques. △ Less

Submitted 2 August, 2017; originally announced August 2017.

Comments: WMT 2017 shared task track; for Bibtex, see http://homepages.inf.ed.ac.uk/rsennric/bib.html#uedin-nmt:2017

Evaluating and Modelling Hanabi-Playing Agents

Authors: Joseph Walton-Rivers, Piers R. Williams, Richard Bartle, Diego Perez-Liebana, Simon M. Lucas

Abstract: Agent modelling involves considering how other agents will behave, in order to influence your own actions. In this paper, we explore the use of agent modelling in the hidden-information, collaborative card game Hanabi. We implement a number of rule-based agents, both from the literature and of our own devising, in addition to an Information Set Monte Carlo Tree Search (IS-MCTS) agent. We observe p… ▽ More Agent modelling involves considering how other agents will behave, in order to influence your own actions. In this paper, we explore the use of agent modelling in the hidden-information, collaborative card game Hanabi. We implement a number of rule-based agents, both from the literature and of our own devising, in addition to an Information Set Monte Carlo Tree Search (IS-MCTS) agent. We observe poor results from IS-MCTS, so construct a new, predictor version that uses a model of the agents with which it is paired. We observe a significant improvement in game-playing strength from this agent in comparison to IS-MCTS, resulting from its consideration of what the other agents in a game would do. In addition, we create a flawed rule-based agent to highlight the predictor's capabilities with such an agent. △ Less

Submitted 24 April, 2017; originally announced April 2017.

Comments: Proceedings of the IEEE Conference on Evolutionary Computation (2017)

Ms. Pac-Man Versus Ghost Team CIG 2016 Competition

Authors: Piers R. Williams, Diego Perez-Liebana, Simon M. Lucas

Abstract: This paper introduces the revival of the popular Ms. Pac-Man Versus Ghost Team competition. We present an updated game engine with Partial Observability constraints, a new Multi-Agent Systems approach to developing Ghost agents and several sample controllers to ease the development of entries. A restricted communication protocol is provided for the Ghosts, providing a more challenging environment… ▽ More This paper introduces the revival of the popular Ms. Pac-Man Versus Ghost Team competition. We present an updated game engine with Partial Observability constraints, a new Multi-Agent Systems approach to developing Ghost agents and several sample controllers to ease the development of entries. A restricted communication protocol is provided for the Ghosts, providing a more challenging environment than before. The competition will debut at the IEEE Computational Intelligence and Games Conference 2016. Some preliminary results showing the effects of Partial Observability and the benefits of simple communication are also presented. △ Less

Submitted 8 September, 2016; originally announced September 2016.

Information encryption in the expert management of strategic uncertainty

Authors: Seth Frey, Paul L. Williams, Dominic K. Albino

Abstract: Strategic agents in incomplete-information environments have a conflicted relationship with uncertainty: it can keep them unpredictable to their opponents, but it must also be overcome to predict the actions of those opponents. We use a multivariate generalization of information theory to characterize the information processing behavior of strategic reasoning experts. We compare expert and novice… ▽ More Strategic agents in incomplete-information environments have a conflicted relationship with uncertainty: it can keep them unpredictable to their opponents, but it must also be overcome to predict the actions of those opponents. We use a multivariate generalization of information theory to characterize the information processing behavior of strategic reasoning experts. We compare expert and novice poker players --- "sharks" and "fish" --- over 1.75 million hands of online two-player No-Limit Texas Hold'em (NLHE). Comparing the effects of privately known and publicly signaled information on wagering behavior, we find that the behavior of sharks coheres with information that emerges only from the interaction of public and private sources --- "synergistic" information that does not exist in either source alone. This implies that the effect of public information on shark behavior is better encrypted: it cannot be reconstructed without access to the hidden state of private cards. Integrative information processing affects not only one's own strategic behavior, but the ability of others to predict it. By characterizing the informational structure of complex strategic interactions, we offer a detailed account of how experts extract, process, and conceal valuable information in high-uncertainty, high-stakes competitive environments. △ Less

Submitted 13 May, 2016; originally announced May 2016.

Comments: 21 pages, 3 figures, 12 page SI

Programmable Multi-Node Quantum Network Design and Simulation

Authors: Venkat R. Dasari, Ronald J. Sadlier, Ryan Prout, Brian P. Williams, Travis S. Humble

Abstract: Software-defined networking offers a device-agnostic programmable framework to encode new network functions. Externally centralized control plane intelligence allows programmers to write network applications and to build functional network designs. OpenFlow is a key protocol widely adopted to build programmable networks because of its programmability, flexibility and ability to interconnect hetero… ▽ More Software-defined networking offers a device-agnostic programmable framework to encode new network functions. Externally centralized control plane intelligence allows programmers to write network applications and to build functional network designs. OpenFlow is a key protocol widely adopted to build programmable networks because of its programmability, flexibility and ability to interconnect heterogeneous network devices. We simulate the functional topology of a multi-node quantum network that uses programmable network principles to manage quantum metadata for protocols such as teleportation, superdense coding, and quantum key distribution. We first show how the OpenFlow protocol can manage the quantum metadata needed to control the quantum channel. We then use numerical simulation to demonstrate robust programmability of a quantum switch via the OpenFlow network controller while executing an application of superdense coding. We describe the software framework implemented to carry out these simulations and we discuss near-term efforts to realize these applications. △ Less

Submitted 5 April, 2016; originally announced April 2016.

Comments: 9 pages, scheduled for presentation at SPIE Quantum Information and Computation IX

Journal ref: Proc. SPIE 9873, Quantum Information and Computation IX, 98730B (2016)

Hybrid Social Networking Application for a University Community

Authors: O. Chigozie, P. Williams, N. E. Osegi

Abstract: A hybrid social network for building social communities for a university community is presented. The system employed the semantic ontology for an offline/online social network site (SNS). It captures the core features of an SNS including profile creation, friend invite/search, group formation, chatting/messaging, blogging and voting. Three core frameworks - the peer2me framework, SMSN semantic mob… ▽ More A hybrid social network for building social communities for a university community is presented. The system employed the semantic ontology for an offline/online social network site (SNS). It captures the core features of an SNS including profile creation, friend invite/search, group formation, chatting/messaging, blogging and voting. Three core frameworks - the peer2me framework, SMSN semantic mobile social network and Peoplepods framework were considered in the implementation phase. The results show remarkable matching performance for prosumers with similar interests with relevance close to unity. The social network was able to capture the needs of the university students by serving as a handy direction to popular locations within the campus. △ Less

Submitted 18 March, 2015; originally announced March 2015.

Comments: 7 pages, 6 figures

Information flow through a model of the C. elegans klinotaxis circuit

Authors: Eduardo J. Izquierdo, Paul L. Williams, Randall D. Beer

Abstract: Understanding how information about external stimuli is transformed into behavior is one of the central goals of neuroscience. Here we characterize the information flow through a complete sensorimotor circuit: from stimulus, to sensory neurons, to interneurons, to motor neurons, to muscles, to motion. Specifically, we apply a recently developed framework for quantifying information flow to a previ… ▽ More Understanding how information about external stimuli is transformed into behavior is one of the central goals of neuroscience. Here we characterize the information flow through a complete sensorimotor circuit: from stimulus, to sensory neurons, to interneurons, to motor neurons, to muscles, to motion. Specifically, we apply a recently developed framework for quantifying information flow to a previously published ensemble of models of salt klinotaxis in the nematode worm C. elegans. The models are grounded in the neuroanatomy and currently known neurophysiology of the worm. The unknown model parameters were optimized to reproduce the worm's behavior. Information flow analysis reveals several key principles underlying how the models operate: (1) Interneuron class AIY is responsible for integrating information about positive and negative changes in concentration, and exhibits a strong left/right information asymmetry. (2) Gap junctions play a crucial role in the transfer of information responsible for the information symmetry observed in interneuron class AIZ. (3) Neck motor neuron class SMB implements an information gating mechanism that underlies the circuit's state-dependent response. (4) The neck carries non-uniform distribution about changes in concentration. Thus, not all directions of movement are equally informative. Each of these findings corresponds to an experimental prediction that could be tested in the worm to greatly refine our understanding of the neural circuit underlying klinotaxis. Information flow analysis also allows us to explore how information flow relates to underlying electrophysiology. Despite large variations in the neural parameters of individual circuits, the overall information flow architecture circuit is remarkably consistent across the ensemble, suggesting that information flow analysis captures general principles of operation for the klinotaxis circuit. △ Less

Submitted 14 February, 2015; originally announced February 2015.

Journal ref: PLoS ONE 10(10): e0140397. (2015)

Towards a Synergy-based Approach to Measuring Information Modification

Authors: Joseph T. Lizier, Benjamin Flecker, Paul L. Williams

Abstract: Distributed computation in artificial life and complex systems is often described in terms of component operations on information: information storage, transfer and modification. Information modification remains poorly described however, with the popularly-understood examples of glider and particle collisions in cellular automata being only quantitatively identified to date using a heuristic (sepa… ▽ More Distributed computation in artificial life and complex systems is often described in terms of component operations on information: information storage, transfer and modification. Information modification remains poorly described however, with the popularly-understood examples of glider and particle collisions in cellular automata being only quantitatively identified to date using a heuristic (separable information) rather than a proper information-theoretic measure. We outline how a recently-introduced axiomatic framework for measuring information redundancy and synergy, called partial information decomposition, can be applied to a perspective of distributed computation in order to quantify component operations on information. Using this framework, we propose a new measure of information modification that captures the intuitive understanding of information modification events as those involving interactions between two or more information sources. We also consider how the local dynamics of information modification in space and time could be measured, and suggest a new axiom that redundancy measures would need to meet in order to make such local measurements. Finally, we evaluate the potential for existing redundancy measures to meet this localizability axiom. △ Less

Submitted 14 March, 2013; originally announced March 2013.

Comments: 9 pages, 4 figures

MSC Class: 94A15

Journal ref: Proceedings of the 2013 IEEE Symposium on Artificial Life (ALIFE), Singapore, p. 43-51, 2013

Generalized Measures of Information Transfer

Authors: Paul L. Williams, Randall D. Beer

Abstract: Transfer entropy provides a general tool for analyzing the magnitudes and directions---but not the \emph{kinds}---of information transfer in a system. We extend transfer entropy in two complementary ways. First, we distinguish state-dependent from state-independent transfer, based on whether a source's influence depends on the state of the target. Second, for multiple sources, we distinguish betwe… ▽ More Transfer entropy provides a general tool for analyzing the magnitudes and directions---but not the \emph{kinds}---of information transfer in a system. We extend transfer entropy in two complementary ways. First, we distinguish state-dependent from state-independent transfer, based on whether a source's influence depends on the state of the target. Second, for multiple sources, we distinguish between unique, redundant, and synergistic transfer. The new measures are demonstrated on several systems that extend examples from previous literature. △ Less

Submitted 7 February, 2011; originally announced February 2011.

Nonnegative Decomposition of Multivariate Information

Authors: Paul L. Williams, Randall D. Beer

Abstract: Of the various attempts to generalize information theory to multiple variables, the most widely utilized, interaction information, suffers from the problem that it is sometimes negative. Here we reconsider from first principles the general structure of the information that a set of sources provides about a given variable. We begin with a new definition of redundancy as the minimum information that… ▽ More Of the various attempts to generalize information theory to multiple variables, the most widely utilized, interaction information, suffers from the problem that it is sometimes negative. Here we reconsider from first principles the general structure of the information that a set of sources provides about a given variable. We begin with a new definition of redundancy as the minimum information that any source provides about each possible outcome of the variable, averaged over all possible outcomes. We then show how this measure of redundancy induces a lattice over sets of sources that clarifies the general structure of multivariate information. Finally, we use this redundancy lattice to propose a definition of partial information atoms that exhaustively decompose the Shannon information in a multivariate system in terms of the redundancy between synergies of subsets of the sources. Unlike interaction information, the atoms of our partial information decomposition are never negative and always support a clear interpretation as informational quantities. Our analysis also demonstrates how the negativity of interaction information can be explained by its confounding of redundancy and synergy. △ Less

Submitted 14 April, 2010; originally announced April 2010.

Comments: 14 pages, 9 figures

MSC Class: 94A15