Showing 1–31 of 31 results for author: Werner, M

Cross-View Geolocalization and Disaster Mapping with Street-View and VHR Satellite Imagery: A Case Study of Hurricane IAN

Authors: Hao Li, Fabian Deuser, Wenping Yina, Xuanshu Luo, Paul Walther, Gengchen Mai, Wei Huang, Martin Werner

Abstract: Nature disasters play a key role in shaping human-urban infrastructure interactions. Effective and efficient response to natural disasters is essential for building resilience and a sustainable urban environment. Two types of information are usually the most necessary and difficult to gather in disaster response. The first information is about disaster damage perception, which shows how badly peop… ▽ More Nature disasters play a key role in shaping human-urban infrastructure interactions. Effective and efficient response to natural disasters is essential for building resilience and a sustainable urban environment. Two types of information are usually the most necessary and difficult to gather in disaster response. The first information is about disaster damage perception, which shows how badly people think that urban infrastructure has been damaged. The second information is geolocation awareness, which means how people whereabouts are made available. In this paper, we proposed a novel disaster mapping framework, namely CVDisaster, aiming at simultaneously addressing geolocalization and damage perception estimation using cross-view Street-View Imagery (SVI) and Very High-Resolution satellite imagery. CVDisaster consists of two cross-view models, where CVDisaster-Geoloc refers to a cross-view geolocalization model based on a contrastive learning objective with a Siamese ConvNeXt image encoder, and CVDisaster-Est is a cross-view classification model based on a Couple Global Context Vision Transformer (CGCViT). Taking Hurricane IAN as a case study, we evaluate the CVDisaster framework by creating a novel cross-view dataset (CVIAN) and conducting extensive experiments. As a result, we show that CVDisaster can achieve highly competitive performance (over 80% for geolocalization and 75% for damage perception estimation) with even limited fine-tuning efforts, which largely motivates future cross-view models and applications within a broader GeoAI research community. The data and code are publicly available at: https://github.com/tum-bgd/CVDisaster. △ Less

Submitted 13 August, 2024; originally announced August 2024.

Defection-Free Collaboration between Competitors in a Learning System

Authors: Mariel Werner, Sai Praneeth Karimireddy, Michael I. Jordan

Abstract: We study collaborative learning systems in which the participants are competitors who will defect from the system if they lose revenue by collaborating. As such, we frame the system as a duopoly of competitive firms who are each engaged in training machine-learning models and selling their predictions to a market of consumers. We first examine a fully collaborative scheme in which both firms share… ▽ More We study collaborative learning systems in which the participants are competitors who will defect from the system if they lose revenue by collaborating. As such, we frame the system as a duopoly of competitive firms who are each engaged in training machine-learning models and selling their predictions to a market of consumers. We first examine a fully collaborative scheme in which both firms share their models with each other and show that this leads to a market collapse with the revenues of both firms going to zero. We next show that one-sided collaboration in which only the firm with the lower-quality model shares improves the revenue of both firms. Finally, we propose a more equitable, *defection-free* scheme in which both firms share with each other while losing no revenue, and we show that our algorithm converges to the Nash bargaining solution. △ Less

Submitted 22 June, 2024; originally announced June 2024.

Operational Framework for a Quantum Database

Authors: Carla Rieger, Michele Grossi, Gian Giacomo Guerreschi, Sofia Vallecorsa, Martin Werner

Abstract: Databases are an essential component of modern computing infrastructures and allow efficient access to data stored persistently. Their structure depends on the type and relationships of the stored data elements and on the access pattern. Extending the concept of databases to the quantum domain is expected to increase both the storage efficiency and the access parallelism through quantum superposit… ▽ More Databases are an essential component of modern computing infrastructures and allow efficient access to data stored persistently. Their structure depends on the type and relationships of the stored data elements and on the access pattern. Extending the concept of databases to the quantum domain is expected to increase both the storage efficiency and the access parallelism through quantum superposition. In addition, quantum databases may be seen as the result of a prior state preparation ready to be used by quantum algorithms when needed. On the other hand, limiting factors exist and include entanglement creation, the impossibility of perfect copying due to the no-cloning theorem, and the impossibility of coherently erasing a quantum state. In this work, we introduce quantum databases within the broader context of data structures using classical and quantum data and indexing. In particular, we are interested in quantum databases practical implementation and usability, focusing on the definition of the basic operations needed to create and manipulate data stored in a superposition state. Specifically, we address the case of quantum indexing in combination with classical data. For this scenario, we define the operations for database preparation, extension, removal of indices, writing, and read-out of data, as well as index permutation. We present their algorithmic implementation and highlight their advantages and limitations. Finally, we introduce steps toward defining the same operations in the more general context of quantum indexing and quantum data. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Comments: 26 pages, 4 figures

Privacy Can Arise Endogenously in an Economic System with Learning Agents

Authors: Nivasini Ananthakrishnan, Tiffany Ding, Mariel Werner, Sai Praneeth Karimireddy, Michael I. Jordan

Abstract: We study price-discrimination games between buyers and a seller where privacy arises endogenously--that is, utility maximization yields equilibrium strategies where privacy occurs naturally. In this game, buyers with a high valuation for a good have an incentive to keep their valuation private, lest the seller charge them a higher price. This yields an equilibrium where some buyers will send a sig… ▽ More We study price-discrimination games between buyers and a seller where privacy arises endogenously--that is, utility maximization yields equilibrium strategies where privacy occurs naturally. In this game, buyers with a high valuation for a good have an incentive to keep their valuation private, lest the seller charge them a higher price. This yields an equilibrium where some buyers will send a signal that misrepresents their type with some probability; we refer to this as buyer-induced privacy. When the seller is able to publicly commit to providing a certain privacy level, we find that their equilibrium response is to commit to ignore buyers' signals with some positive probability; we refer to this as seller-induced privacy. We then turn our attention to a repeated interaction setting where the game parameters are unknown and the seller cannot credibly commit to a level of seller-induced privacy. In this setting, players must learn strategies based on information revealed in past rounds. We find that, even without commitment ability, seller-induced privacy arises as a result of reputation building. We characterize the resulting seller-induced privacy and seller's utility under no-regret and no-policy-regret learning algorithms and verify these results through simulations. △ Less

Submitted 16 April, 2024; originally announced April 2024.

Comments: To appear in Symposium on Foundations of Responsible Computing (FORC 2024)

Detecting Anomalous Events in Object-centric Business Processes via Graph Neural Networks

Authors: Alessandro Niro, Michael Werner

Abstract: Detecting anomalies is important for identifying inefficiencies, errors, or fraud in business processes. Traditional process mining approaches focus on analyzing 'flattened', sequential, event logs based on a single case notion. However, many real-world process executions exhibit a graph-like structure, where events can be associated with multiple cases. Flattening event logs requires selecting a… ▽ More Detecting anomalies is important for identifying inefficiencies, errors, or fraud in business processes. Traditional process mining approaches focus on analyzing 'flattened', sequential, event logs based on a single case notion. However, many real-world process executions exhibit a graph-like structure, where events can be associated with multiple cases. Flattening event logs requires selecting a single case identifier which creates a gap with the real event data and artificially introduces anomalies in the event logs. Object-centric process mining avoids these limitations by allowing events to be related to different cases. This study proposes a novel framework for anomaly detection in business processes that exploits graph neural networks and the enhanced information offered by object-centric process mining. We first reconstruct and represent the process dependencies of the object-centric event logs as attributed graphs and then employ a graph convolutional autoencoder architecture to detect anomalous events. Our results show that our approach provides promising performance in detecting anomalies at the activity type and attributes level, although it struggles to detect anomalies in the temporal order of events. △ Less

Submitted 14 February, 2024; originally announced March 2024.

Comments: 12 pages, 2 figures, to appear in the ICPM 2023 Workshops Proceedings

Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks

Authors: Danfeng Hong, Bing Zhang, Hao Li, Yuxuan Li, Jing Yao, Chenyu Li, Martin Werner, Jocelyn Chanussot, Alexander Zipf, Xiao Xiang Zhu

Abstract: Artificial intelligence (AI) approaches nowadays have gained remarkable success in single-modality-dominated remote sensing (RS) applications, especially with an emphasis on individual urban environments (e.g., single cities or regions). Yet these AI models tend to meet the performance bottleneck in the case studies across cities or regions, due to the lack of diverse RS information and cutting-ed… ▽ More Artificial intelligence (AI) approaches nowadays have gained remarkable success in single-modality-dominated remote sensing (RS) applications, especially with an emphasis on individual urban environments (e.g., single cities or regions). Yet these AI models tend to meet the performance bottleneck in the case studies across cities or regions, due to the lack of diverse RS information and cutting-edge solutions with high generalization ability. To this end, we build a new set of multimodal remote sensing benchmark datasets (including hyperspectral, multispectral, SAR) for the study purpose of the cross-city semantic segmentation task (called C2Seg dataset), which consists of two cross-city scenes, i.e., Berlin-Augsburg (in Germany) and Beijing-Wuhan (in China). Beyond the single city, we propose a high-resolution domain adaptation network, HighDAN for short, to promote the AI model's generalization ability from the multi-city environments. HighDAN is capable of retaining the spatially topological structure of the studied urban scene well in a parallel high-to-low resolution fusion fashion but also closing the gap derived from enormous differences of RS image representations between different cities by means of adversarial learning. In addition, the Dice loss is considered in HighDAN to alleviate the class imbalance issue caused by factors across cities. Extensive experiments conducted on the C2Seg dataset show the superiority of our HighDAN in terms of segmentation performance and generalization ability, compared to state-of-the-art competitors. The C2Seg dataset and the semantic segmentation toolbox (involving the proposed HighDAN) will be available publicly at https://github.com/danfenghong. △ Less

Submitted 3 October, 2023; v1 submitted 26 September, 2023; originally announced September 2023.

Orientation-Guided Contrastive Learning for UAV-View Geo-Localisation

Authors: Fabian Deuser, Konrad Habel, Martin Werner, Norbert Oswald

Abstract: Retrieving relevant multimedia content is one of the main problems in a world that is increasingly data-driven. With the proliferation of drones, high quality aerial footage is now available to a wide audience for the first time. Integrating this footage into applications can enable GPS-less geo-localisation or location correction. In this paper, we present an orientation-guided training framewo… ▽ More Retrieving relevant multimedia content is one of the main problems in a world that is increasingly data-driven. With the proliferation of drones, high quality aerial footage is now available to a wide audience for the first time. Integrating this footage into applications can enable GPS-less geo-localisation or location correction. In this paper, we present an orientation-guided training framework for UAV-view geo-localisation. Through hierarchical localisation orientations of the UAV images are estimated in relation to the satellite imagery. We propose a lightweight prediction module for these pseudo labels which predicts the orientation between the different views based on the contrastive learned embeddings. We experimentally demonstrate that this prediction supports the training and outperforms previous approaches. The extracted pseudo-labels also enable aligned rotation of the satellite image as augmentation to further strengthen the generalisation. During inference, we no longer need this orientation module, which means that no additional computations are required. We achieve state-of-the-art results on both the University-1652 and University-160k datasets. △ Less

Submitted 2 August, 2023; originally announced August 2023.

Towards Mobility Data Science (Vision Paper)

Authors: Mohamed Mokbel, Mahmoud Sakr, Li Xiong, Andreas Züfle, Jussara Almeida, Taylor Anderson, Walid Aref, Gennady Andrienko, Natalia Andrienko, Yang Cao, Sanjay Chawla, Reynold Cheng, Panos Chrysanthis, Xiqi Fei, Gabriel Ghinita, Anita Graser, Dimitrios Gunopulos, Christian Jensen, Joon-Seok Kim, Kyoung-Sook Kim, Peer Kröger, John Krumm, Johannes Lauer, Amr Magdy, Mario Nascimento , et al. (23 additional authors not shown)

Abstract: Mobility data captures the locations of moving objects such as humans, animals, and cars. With the availability of GPS-equipped mobile devices and other inexpensive location-tracking technologies, mobility data is collected ubiquitously. In recent years, the use of mobility data has demonstrated significant impact in various domains including traffic management, urban planning, and health sciences… ▽ More Mobility data captures the locations of moving objects such as humans, animals, and cars. With the availability of GPS-equipped mobile devices and other inexpensive location-tracking technologies, mobility data is collected ubiquitously. In recent years, the use of mobility data has demonstrated significant impact in various domains including traffic management, urban planning, and health sciences. In this paper, we present the emerging domain of mobility data science. Towards a unified approach to mobility data science, we envision a pipeline having the following components: mobility data collection, cleaning, analysis, management, and privacy. For each of these components, we explain how mobility data science differs from general data science, we survey the current state of the art and describe open challenges for the research community in the coming years. △ Less

Submitted 7 March, 2024; v1 submitted 21 June, 2023; originally announced July 2023.

Comments: Updated to reflect the major revision for ACM Transactions on Spatial Algorithms and Systems (TSAS). This version reflects the final version accepted by ACM TSAS

Semi-supervised Learning from Street-View Images and OpenStreetMap for Automatic Building Height Estimation

Authors: Hao Li, Zhendong Yuan, Gabriel Dax, Gefei Kong, Hongchao Fan, Alexander Zipf, Martin Werner

Abstract: Accurate building height estimation is key to the automatic derivation of 3D city models from emerging big geospatial data, including Volunteered Geographical Information (VGI). However, an automatic solution for large-scale building height estimation based on low-cost VGI data is currently missing. The fast development of VGI data platforms, especially OpenStreetMap (OSM) and crowdsourced street-… ▽ More Accurate building height estimation is key to the automatic derivation of 3D city models from emerging big geospatial data, including Volunteered Geographical Information (VGI). However, an automatic solution for large-scale building height estimation based on low-cost VGI data is currently missing. The fast development of VGI data platforms, especially OpenStreetMap (OSM) and crowdsourced street-view images (SVI), offers a stimulating opportunity to fill this research gap. In this work, we propose a semi-supervised learning (SSL) method of automatically estimating building height from Mapillary SVI and OSM data to generate low-cost and open-source 3D city modeling in LoD1. The proposed method consists of three parts: first, we propose an SSL schema with the option of setting a different ratio of "pseudo label" during the supervised regression; second, we extract multi-level morphometric features from OSM data (i.e., buildings and streets) for the purposed of inferring building height; last, we design a building floor estimation workflow with a pre-trained facade object detection network to generate "pseudo label" from SVI and assign it to the corresponding OSM building footprint. In a case study, we validate the proposed SSL method in the city of Heidelberg, Germany and evaluate the model performance against the reference data of building heights. Based on three different regression models, namely Random Forest (RF), Support Vector Machine (SVM), and Convolutional Neural Network (CNN), the SSL method leads to a clear performance boosting in estimating building heights with a Mean Absolute Error (MAE) around 2.1 meters, which is competitive to state-of-the-art approaches. The preliminary result is promising and motivates our future work in scaling up the proposed method based on low-cost VGI data, with possibilities in even regions and areas with diverse data quality and availability. △ Less

Submitted 5 July, 2023; originally announced July 2023.

Comments: Accepted for GIScience 2023

Provably Personalized and Robust Federated Learning

Authors: Mariel Werner, Lie He, Michael Jordan, Martin Jaggi, Sai Praneeth Karimireddy

Abstract: Identifying clients with similar objectives and learning a model-per-cluster is an intuitive and interpretable approach to personalization in federated learning. However, doing so with provable and optimal guarantees has remained an open challenge. We formalize this problem as a stochastic optimization problem, achieving optimal convergence rates for a large class of loss functions. We propose sim… ▽ More Identifying clients with similar objectives and learning a model-per-cluster is an intuitive and interpretable approach to personalization in federated learning. However, doing so with provable and optimal guarantees has remained an open challenge. We formalize this problem as a stochastic optimization problem, achieving optimal convergence rates for a large class of loss functions. We propose simple iterative algorithms which identify clusters of similar clients and train a personalized model-per-cluster, using local client gradients and flexible constraints on the clusters. The convergence rates of our algorithms asymptotically match those obtained if we knew the true underlying clustering of the clients and are provably robust in the Byzantine setting where some fraction of the clients are malicious. △ Less

Submitted 18 December, 2023; v1 submitted 14 June, 2023; originally announced June 2023.

DAS-N2N: Machine learning Distributed Acoustic Sensing (DAS) signal denoising without clean data

Authors: Sacha Lapins, Antony Butcher, J. -Michael Kendall, Thomas S. Hudson, Anna L. Stork, Maximilian J. Werner, Jemma Gunning, Alex M. Brisbourne

Abstract: This article presents a weakly supervised machine learning method, which we call DAS-N2N, for suppressing strong random noise in distributed acoustic sensing (DAS) recordings. DAS-N2N requires no manually produced labels (i.e., pre-determined examples of clean event signals or sections of noise) for training and aims to map random noise processes to a chosen summary statistic, such as the distribu… ▽ More This article presents a weakly supervised machine learning method, which we call DAS-N2N, for suppressing strong random noise in distributed acoustic sensing (DAS) recordings. DAS-N2N requires no manually produced labels (i.e., pre-determined examples of clean event signals or sections of noise) for training and aims to map random noise processes to a chosen summary statistic, such as the distribution mean, median or mode, whilst retaining the true underlying signal. This is achieved by splicing (joining together) two fibres hosted within a single optical cable, recording two noisy copies of the same underlying signal corrupted by different independent realizations of random observational noise. A deep learning model can then be trained using only these two noisy copies of the data to produce a near fully-denoised copy. Once the model is trained, only noisy data from a single fibre is required. Using a dataset from a DAS array deployed on the surface of the Rutford Ice Stream in Antarctica, we demonstrate that DAS-N2N greatly suppresses incoherent noise and enhances the signal-to-noise ratios (SNR) of natural microseismic icequake events. We further show that this approach is inherently more efficient and effective than standard stop/pass band and white noise (e.g., Wiener) filtering routines, as well as a comparable self-supervised learning method based on masking individual DAS channels. Our preferred model for this task is lightweight, processing 30 seconds of data recorded at a sampling frequency of 1000 Hz over 985 channels (approx. 1 km of fiber) in $<$1 s. Due to the high noise levels in DAS recordings, efficient data-driven denoising methods, such as DAS-N2N, will prove essential to time-critical DAS earthquake detection, particularly in the case of microseismic monitoring. △ Less

Submitted 24 November, 2023; v1 submitted 17 April, 2023; originally announced April 2023.

Comments: Submitted for publication to Geophysical Journal International. For the purpose of open access, the author(s) has applied a Creative Commons Attribution (CC BY) licence to the Author Accepted Manuscript version arising from this submission

Geo-Information Harvesting from Social Media Data

Authors: Xiao Xiang Zhu, Yuanyuan Wang, Mrinalini Kochupillai, Martin Werner, Matthias Häberle, Eike Jens Hoffmann, Hannes Taubenböck, Devis Tuia, Alex Levering, Nathan Jacobs, Anna Kruspe, Karam Abdulahhad

Abstract: As unconventional sources of geo-information, massive imagery and text messages from open platforms and social media form a temporally quasi-seamless, spatially multi-perspective stream, but with unknown and diverse quality. Due to its complementarity to remote sensing data, geo-information from these sources offers promising perspectives, but harvesting is not trivial due to its data characterist… ▽ More As unconventional sources of geo-information, massive imagery and text messages from open platforms and social media form a temporally quasi-seamless, spatially multi-perspective stream, but with unknown and diverse quality. Due to its complementarity to remote sensing data, geo-information from these sources offers promising perspectives, but harvesting is not trivial due to its data characteristics. In this article, we address key aspects in the field, including data availability, analysis-ready data preparation and data management, geo-information extraction from social media text messages and images, and the fusion of social media and remote sensing data. We then showcase some exemplary geographic applications. In addition, we present the first extensive discussion of ethical considerations of social media data in the context of geo-information harvesting and geographic applications. With this effort, we wish to stimulate curiosity and lay the groundwork for researchers who intend to explore social media data for geo-applications. We encourage the community to join forces by sharing their code and data. △ Less

Submitted 1 November, 2022; originally announced November 2022.

Comments: Accepted for publication IEEE Geoscience and Remote Sensing Magazine

Optimal Data Selection: An Online Distributed View

Authors: Mariel Werner, Anastasios Angelopoulos, Stephen Bates, Michael I. Jordan

Abstract: The blessing of ubiquitous data also comes with a curse: the communication, storage, and labeling of massive, mostly redundant datasets. We seek to solve this problem at its core, collecting only valuable data and throwing out the rest via submodular maximization. Specifically, we develop algorithms for the online and distributed version of the problem, where data selection occurs in an uncoordina… ▽ More The blessing of ubiquitous data also comes with a curse: the communication, storage, and labeling of massive, mostly redundant datasets. We seek to solve this problem at its core, collecting only valuable data and throwing out the rest via submodular maximization. Specifically, we develop algorithms for the online and distributed version of the problem, where data selection occurs in an uncoordinated fashion across multiple data streams. We design a general and flexible core selection routine for our algorithms which, given any stream of data, any assessment of its value, and any formulation of its selection cost, extracts the most valuable subset of the stream up to a constant factor while using minimal memory. Notably, our methods have the same theoretical guarantees as their offline counterparts, and, as far as we know, provide the first guarantees for online distributed submodular optimization in the literature. Finally, in learning tasks on ImageNet and MNIST, we show that our selection methods outperform random selection by $5-20\%$. △ Less

Submitted 14 December, 2023; v1 submitted 25 January, 2022; originally announced January 2022.

Challenges and Opportunities on Using Games to Support IoT Systems Teaching

Authors: Bruno Pedraça de Souza, Claudia Maria Lima Werner

Abstract: Context: New systems have emerged within the Industry 4.0 paradigm. These systems incorporate characteristics such as autonomy in decision making and acting in the context of IoT systems, continuous connectivity between devices and applications in cyber-physical systems, omnipresence properties in ubiquitous systems, among others. Thus, the engineering of these systems has changed, drastically aff… ▽ More Context: New systems have emerged within the Industry 4.0 paradigm. These systems incorporate characteristics such as autonomy in decision making and acting in the context of IoT systems, continuous connectivity between devices and applications in cyber-physical systems, omnipresence properties in ubiquitous systems, among others. Thus, the engineering of these systems has changed, drastically affecting the manner of their construction process. In this context, to identify simple and playful alternatives to teach how to build them is a difficult task. Objective: To present how to teach IoT systems using games, to reveal challenges and opportunities obtained through a literature review. Method: A Structured Literature Review (StLR), supported by the Snowballing technique, was executed to find empirical studies related to teaching, games and IoT systems. Results: 12 papers were found about teaching IoT systems using games. As challenges and opportunities, many issues were identified related to IoT systems programming, modularity, hardware constraints, among others. Conclusion: In this work, research challenges and opportunities were found in the context of IoT systems teaching. Due to specific features of these systems, teaching their construction is a difficult activity to carry out. △ Less

Submitted 21 September, 2021; originally announced September 2021.

Comments: in portuguese

Informed Equation Learning

Authors: Matthias Werner, Andrej Junginger, Philipp Hennig, Georg Martius

Abstract: Distilling data into compact and interpretable analytic equations is one of the goals of science. Instead, contemporary supervised machine learning methods mostly produce unstructured and dense maps from input to output. Particularly in deep learning, this property is owed to the generic nature of simple standard link functions. To learn equations rather than maps, standard non-linearities can be… ▽ More Distilling data into compact and interpretable analytic equations is one of the goals of science. Instead, contemporary supervised machine learning methods mostly produce unstructured and dense maps from input to output. Particularly in deep learning, this property is owed to the generic nature of simple standard link functions. To learn equations rather than maps, standard non-linearities can be replaced with structured building blocks of atomic functions. However, without strong priors on sparsity and structure, representational complexity and numerical conditioning limit this direct approach. To scale to realistic settings in science and engineering, we propose an informed equation learning system. It provides a way to incorporate expert knowledge about what are permitted or prohibited equation components, as well as a domain-dependent structured sparsity prior. Our system then utilizes a robust method to learn equations with atomic functions exhibiting singularities, as e.g. logarithm and division. We demonstrate several artificial and real-world experiments from the engineering domain, in which our system learns interpretable models of high predictive power. △ Less

Submitted 13 May, 2021; originally announced May 2021.

SoK: Decentralized Finance (DeFi)

Authors: Sam M. Werner, Daniel Perez, Lewis Gudgeon, Ariah Klages-Mundt, Dominik Harz, William J. Knottenbelt

Abstract: Decentralized Finance (DeFi), a blockchain powered peer-to-peer financial system, is mushrooming. Two years ago the total value locked in DeFi systems was approximately 700m USD, now, as of April 2022, it stands at around 150bn USD. The frenetic evolution of the ecosystem has created challenges in understanding the basic principles of these systems and their security risks. In this Systematization… ▽ More Decentralized Finance (DeFi), a blockchain powered peer-to-peer financial system, is mushrooming. Two years ago the total value locked in DeFi systems was approximately 700m USD, now, as of April 2022, it stands at around 150bn USD. The frenetic evolution of the ecosystem has created challenges in understanding the basic principles of these systems and their security risks. In this Systematization of Knowledge (SoK) we delineate the DeFi ecosystem along the following axes: its primitives, its operational protocol types and its security. We provide a distinction between technical security, which has a healthy literature, and economic security, which is largely unexplored, connecting the latter with new models and thereby synthesizing insights from computer science, economics and finance. Finally, we outline the open research challenges in the ecosystem across these security types. △ Less

Submitted 15 September, 2022; v1 submitted 21 January, 2021; originally announced January 2021.

CrypTag: Thwarting Physical and Logical Memory Vulnerabilities using Cryptographically Colored Memory

Authors: Pascal Nasahl, Robert Schilling, Mario Werner, Jan Hoogerbrugge, Marcel Medwed, Stefan Mangard

Abstract: Memory vulnerabilities are a major threat to many computing systems. To effectively thwart spatial and temporal memory vulnerabilities, full logical memory safety is required. However, current mitigation techniques for memory safety are either too expensive or trade security against efficiency. One promising attempt to detect memory safety vulnerabilities in hardware is memory coloring, a security… ▽ More Memory vulnerabilities are a major threat to many computing systems. To effectively thwart spatial and temporal memory vulnerabilities, full logical memory safety is required. However, current mitigation techniques for memory safety are either too expensive or trade security against efficiency. One promising attempt to detect memory safety vulnerabilities in hardware is memory coloring, a security policy deployed on top of tagged memory architectures. However, due to the memory storage and bandwidth overhead of large tags, commodity tagged memory architectures usually only provide small tag sizes, thus limiting their use for security applications. Irrespective of logical memory safety, physical memory safety is a necessity in hostile environments prevalent for modern cloud computing and IoT devices. Architectures from Intel and AMD already implement transparent memory encryption to maintain confidentiality and integrity of all off-chip data. Surprisingly, the combination of both, logical and physical memory safety, has not yet been extensively studied in previous research, and a naive combination of both security strategies would accumulate both overheads. In this paper, we propose CrypTag, an efficient hardware/software co-design mitigating a large class of logical memory safety issues and providing full physical memory safety. At its core, CrypTag utilizes a transparent memory encryption engine not only for physical memory safety, but also for memory coloring at hardly any additional costs. The design avoids any overhead for tag storage by embedding memory colors in the upper bits of a pointer and using these bits as an additional input for the memory encryption. A custom compiler extension automatically leverages CrypTag to detect logical memory safety issues for commodity programs and is fully backward compatible. △ Less

Submitted 9 March, 2021; v1 submitted 12 December, 2020; originally announced December 2020.

Machine Learning Information Fusion in Earth Observation: A Comprehensive Review of Methods, Applications and Data Sources

Authors: S. Salcedo-Sanz, P. Ghamisi, M. Piles, M. Werner, L. Cuadra, A. Moreno-Martínez, E. Izquierdo-Verdiguier, J. Muñoz-Marí, Amirhosein Mosavi, G. Camps-Valls

Abstract: This paper reviews the most important information fusion data-driven algorithms based on Machine Learning (ML) techniques for problems in Earth observation. Nowadays we observe and model the Earth with a wealth of observations, from a plethora of different sensors, measuring states, fluxes, processes and variables, at unprecedented spatial and temporal resolutions. Earth observation is well equipp… ▽ More This paper reviews the most important information fusion data-driven algorithms based on Machine Learning (ML) techniques for problems in Earth observation. Nowadays we observe and model the Earth with a wealth of observations, from a plethora of different sensors, measuring states, fluxes, processes and variables, at unprecedented spatial and temporal resolutions. Earth observation is well equipped with remote sensing systems, mounted on satellites and airborne platforms, but it also involves in-situ observations, numerical models and social media data streams, among other data sources. Data-driven approaches, and ML techniques in particular, are the natural choice to extract significant information from this data deluge. This paper produces a thorough review of the latest work on information fusion for Earth observation, with a practical intention, not only focusing on describing the most relevant previous works in the field, but also the most important Earth observation applications where ML information fusion has obtained significant results. We also review some of the most currently used data sets, models and sources for Earth observation problems, describing their importance and how to obtain the data when needed. Finally, we illustrate the application of ML data fusion with a representative set of case studies, as well as we discuss and outlook the near future of the field. △ Less

Submitted 7 December, 2020; originally announced December 2020.

Journal ref: Information Fusion, Volume 63, November 2020, Pages 256-272

Automating Seccomp Filter Generation for Linux Applications

Authors: Claudio Canella, Mario Werner, Daniel Gruss, Michael Schwarz

Abstract: Software vulnerabilities in applications undermine the security of applications. By blocking unused functionality, the impact of potential exploits can be reduced. While seccomp provides a solution for filtering syscalls, it requires manual implementation of filter rules for each individual application. Recent work has investigated automated approaches for detecting and installing the necessary fi… ▽ More Software vulnerabilities in applications undermine the security of applications. By blocking unused functionality, the impact of potential exploits can be reduced. While seccomp provides a solution for filtering syscalls, it requires manual implementation of filter rules for each individual application. Recent work has investigated automated approaches for detecting and installing the necessary filter rules. However, as we show, these approaches make assumptions that are not necessary or require overly time-consuming analysis. In this paper, we propose Chestnut, an automated approach for generating strict syscall filters for Linux userspace applications with lower requirements and limitations. Chestnut comprises two phases, with the first phase consisting of two static components, i.e., a compiler and a binary analyzer, that extract the used syscalls during compilation or in an analysis of the binary. The compiler-based approach of Chestnut is up to factor 73 faster than previous approaches without affecting the accuracy adversely. On the binary analysis level, we demonstrate that the requirement of position-independent binaries of related work is not needed, enlarging the set of applications for which Chestnut is usable. In an optional second phase, Chestnut provides a dynamic refinement tool that allows restricting the set of allowed syscalls further. We demonstrate that Chestnut on average blocks 302 syscalls (86.5%) via the compiler and 288 (82.5%) using the binary-level analysis on a set of 18 widely used applications. We found that Chestnut blocks the dangerous exec syscall in 50% and 77.7% of the tested applications using the compiler- and binary-based approach, respectively. For the tested applications, Chestnut prevents exploitation of more than 62% of the 175 CVEs that target the kernel via syscalls. Finally, we perform a 6 month long-term study of a sandboxed Nginx server. △ Less

Submitted 4 December, 2020; originally announced December 2020.

HECTOR-V: A Heterogeneous CPU Architecture for a Secure RISC-V Execution Environment

Authors: Pascal Nasahl, Robert Schilling, Mario Werner, Stefan Mangard

Abstract: To ensure secure and trustworthy execution of applications, vendors frequently embed trusted execution environments into their systems. Here, applications are protected from adversaries, including a malicious operating system. TEEs are usually built by integrating protection mechanisms directly into the processor or by using dedicated external secure elements. However, both of these approaches onl… ▽ More To ensure secure and trustworthy execution of applications, vendors frequently embed trusted execution environments into their systems. Here, applications are protected from adversaries, including a malicious operating system. TEEs are usually built by integrating protection mechanisms directly into the processor or by using dedicated external secure elements. However, both of these approaches only cover a narrow threat model resulting in limited security guarantees. Enclaves in the application processor typically provide weak isolation between the secure and non-secure domain, especially when considering side-channel attacks. Although secure elements do provide strong isolation, the slow communication interface to the application processor is exposed to adversaries and restricts the use cases. Independently of the used implementation approach, TEEs often lack the possibility to establish secure communication to external peripherals, and most operating systems executed inside TEEs do not provide state-of-the-art defense strategies, making them vulnerable against various attacks. We argue that TEEs implemented on the main application processor are insecure, especially when considering side-channel attacks. We demonstrate how a heterogeneous architecture can be utilized to realize a secure TEE design. We directly embed a processor into our architecture to provide strong isolation between the secure and non-secure domain. The tight coupling of TEE and REE enables HECTOR-V to provide mechanisms for establishing secure communication channels. We further introduce RISC-V Secure Co-Processor, a security-hardened processor tailored for TEEs. To secure applications executed inside the TEE, RVSCP provides control-flow integrity, rigorously restricts I/O accesses to certain execution states, and provides operating system services directly in hardware. △ Less

Submitted 9 March, 2021; v1 submitted 11 September, 2020; originally announced September 2020.

DeFi Protocols for Loanable Funds: Interest Rates, Liquidity and Market Efficiency

Authors: Lewis Gudgeon, Sam M. Werner, Daniel Perez, William J. Knottenbelt

Abstract: We coin the term *Protocols for Loanable Funds (PLFs)* to refer to protocols which establish distributed ledger-based markets for loanable funds. PLFs are emerging as one of the main applications within Decentralized Finance (DeFi), and use smart contract code to facilitate the intermediation of loanable funds. In doing so, these protocols allow agents to borrow and save programmatically. Within t… ▽ More We coin the term *Protocols for Loanable Funds (PLFs)* to refer to protocols which establish distributed ledger-based markets for loanable funds. PLFs are emerging as one of the main applications within Decentralized Finance (DeFi), and use smart contract code to facilitate the intermediation of loanable funds. In doing so, these protocols allow agents to borrow and save programmatically. Within these protocols, interest rate mechanisms seek to equilibrate the supply and demand for funds. In this paper, we review the methodologies used to set interest rates on three prominent DeFi PLFs, namely Compound, Aave and dYdX. We provide an empirical examination of how these interest rate rules have behaved since their inception in response to differing degrees of liquidity. We then investigate the market efficiency and inter-connectedness between multiple protocols, examining first whether Uncovered Interest Parity holds within a particular protocol and second whether the interest rates for a particular token market show dependence across protocols, developing a Vector Error Correction Model for the dynamics. △ Less

Submitted 15 October, 2020; v1 submitted 11 June, 2020; originally announced June 2020.

Unstable Throughput: When the Difficulty Algorithm Breaks

Authors: Dragos I. Ilie, Sam M. Werner, Iain Stewart, William J. Knottenbelt

Abstract: In Proof-of-Work blockchains, difficulty algorithms serve the crucial purpose of maintaining a stable transaction throughput by dynamically adjusting the block difficulty in response to the miners' constantly changing computational power. Blockchains that may experience severe hash rate fluctuations need difficulty algorithms that quickly adapt the mining difficulty. However, without careful desig… ▽ More In Proof-of-Work blockchains, difficulty algorithms serve the crucial purpose of maintaining a stable transaction throughput by dynamically adjusting the block difficulty in response to the miners' constantly changing computational power. Blockchains that may experience severe hash rate fluctuations need difficulty algorithms that quickly adapt the mining difficulty. However, without careful design, the system could be gamed by miners using coin-hopping strategies to manipulate the block difficulty for profit. Such miner behavior results in an unreliable system due to the unstable processing of transactions. We provide an empirical analysis of how Bitcoin Cash's difficulty algorithm design leads to cyclicality in block solve times as a consequence of a positive feedback loop. In response, we mathematically derive a difficulty algorithm using a negative exponential filter which prohibits the formation of positive feedback and exhibits additional desirable properties, such as history agnosticism. We compare the described algorithm to that of Bitcoin Cash in a simulated mining environment and verify that the former would eliminate the severe oscillations in transaction throughput. △ Less

Submitted 30 April, 2021; v1 submitted 4 June, 2020; originally announced June 2020.

Step on the Gas? A Better Approach for Recommending the Ethereum Gas Price

Authors: Sam M. Werner, Paul J. Pritz, Daniel Perez

Abstract: In the Ethereum network, miners are incentivized to include transactions in a block depending on the gas price specified by the sender. The sender of a transaction therefore faces a trade-off between timely inclusion and cost of his transaction. Existing recommendation mechanisms aggregate recent gas price data on a per-block basis to suggest a gas price. We perform an empirical analysis of histor… ▽ More In the Ethereum network, miners are incentivized to include transactions in a block depending on the gas price specified by the sender. The sender of a transaction therefore faces a trade-off between timely inclusion and cost of his transaction. Existing recommendation mechanisms aggregate recent gas price data on a per-block basis to suggest a gas price. We perform an empirical analysis of historic block data to motivate the use of a predictive model for gas price recommendation. Subsequently, we propose a novel mechanism that combines a deep-learning based price forecasting model as well as an algorithm parameterized by a user-specific urgency value to recommend gas prices. In a comprehensive evaluation on real-world data, we show that our approach results on average in costs savings of more than 50% while only incurring an inclusion delay of 1.3 blocks, when compared to the gas price recommendation mechanism of the most widely used Ethereum client. △ Less

Submitted 20 January, 2021; v1 submitted 6 March, 2020; originally announced March 2020.

Journal ref: in proceedings of The 2nd International Conference on Mathematical Research for Blockchain Economy (MARBLE 2020)

Speeding up Word Mover's Distance and its variants via properties of distances between embeddings

Authors: Matheus Werner, Eduardo Laber

Abstract: The Word Mover's Distance (WMD) proposed by Kusner et al. is a distance between documents that takes advantage of semantic relations among words that are captured by their embeddings. This distance proved to be quite effective, obtaining state-of-art error rates for classification tasks, but is also impracticable for large collections/documents due to its computational complexity. For circumventin… ▽ More The Word Mover's Distance (WMD) proposed by Kusner et al. is a distance between documents that takes advantage of semantic relations among words that are captured by their embeddings. This distance proved to be quite effective, obtaining state-of-art error rates for classification tasks, but is also impracticable for large collections/documents due to its computational complexity. For circumventing this problem, variants of WMD have been proposed. Among them, Relaxed Word Mover's Distance (RWMD) is one of the most successful due to its simplicity, effectiveness, and also because of its fast implementations. Relying on assumptions that are supported by empirical properties of the distances between embeddings, we propose an approach to speed up both WMD and RWMD. Experiments over 10 datasets suggest that our approach leads to a significant speed-up in document classification tasks while maintaining the same error rates. △ Less

Submitted 8 May, 2020; v1 submitted 1 December, 2019; originally announced December 2019.

Comments: Author's final version, accepted for publication at ECAI2020

Pointing in the Right Direction - Securing Memory Accesses in a Faulty World

Authors: Robert Schilling, Mario Werner, Pascal Nasahl, Stefan Mangard

Abstract: Reading and writing memory are, besides computation, the most common operations a processor performs. The correctness of these operations is therefore essential for the proper execution of any program. However, as soon as fault attacks are considered, assuming that the hardware performs its memory operations as instructed is not valid anymore. In particular, attackers may induce faults with the go… ▽ More Reading and writing memory are, besides computation, the most common operations a processor performs. The correctness of these operations is therefore essential for the proper execution of any program. However, as soon as fault attacks are considered, assuming that the hardware performs its memory operations as instructed is not valid anymore. In particular, attackers may induce faults with the goal of reading or writing incorrectly addressed memory, which can have various critical safety and security implications. In this work, we present a solution to this problem and propose a new method for protecting every memory access inside a program against address tampering. The countermeasure comprises two building blocks. First, every pointer inside the program is redundantly encoded using a multi-residue error detection code. The redundancy information is stored in the unused upper bits of the pointer with zero overhead in terms of storage. Second, load and store instructions are extended to link data with the corresponding encoded address from the pointer. Wrong memory accesses subsequently infect the data value allowing the software to detect the error. For evaluation purposes, we implemented our countermeasure into a RISC-V processor, tested it on a FPGA development board, and evaluated the induced overhead. Furthermore, a LLVM-based C compiler has been modified to automatically encode all data pointers, to perform encoded pointer arithmetic, and to emit the extended load/store instructions with linking support. Our evaluations show that the countermeasure induces an average overhead of 10% in terms of code size and 7% regarding runtime, which makes it suitable for practical adoption. △ Less

Submitted 24 September, 2018; originally announced September 2018.

Comments: Accepted at ACSAC 2018

Securing Conditional Branches in the Presence of Fault Attacks

Authors: Robert Schilling, Mario Werner, Stefan Mangard

Abstract: In typical software, many comparisons and subsequent branch operations are highly critical in terms of security. Examples include password checks, signature checks, secure boot, and user privilege checks. For embedded devices, these security-critical branches are a preferred target of fault attacks as a single bit flip or skipping a single instruction can lead to complete access to a system. In th… ▽ More In typical software, many comparisons and subsequent branch operations are highly critical in terms of security. Examples include password checks, signature checks, secure boot, and user privilege checks. For embedded devices, these security-critical branches are a preferred target of fault attacks as a single bit flip or skipping a single instruction can lead to complete access to a system. In the past, numerous redundancy schemes have been proposed in order to provide control-flow-integrity (CFI) and to enable error detection on processed data. However, current countermeasures for general purpose software do not provide protection mechanisms for conditional branches. Hence, critical branches are in practice often simply duplicated. We present a generic approach to protect conditional branches, which links an encoding-based comparison result with the redundancy of CFI protection mechanisms. The presented approach can be used for all types of data encodings and CFI mechanisms and maintains their error-detection capabilities throughout all steps of a conditional branch. We demonstrate our approach by realizing an encoded comparison based on AN-codes, which is a frequently used encoding scheme to detect errors on data during arithmetic operations. We extended the LLVM compiler so that standard code and conditional branches can be protected automatically and analyze its security. Our design shows that the overhead in terms of size and runtime is lower than state-of-the-art duplication schemes. △ Less

Submitted 22 March, 2018; originally announced March 2018.

Comments: Accepted at DATE 2018

Sponge-Based Control-Flow Protection for IoT Devices

Authors: Mario Werner, Thomas Unterluggauer, David Schaffenrath, Stefan Mangard

Abstract: Embedded devices in the Internet of Things (IoT) face a wide variety of security challenges. For example, software attackers perform code injection and code-reuse attacks on their remote interfaces, and physical access to IoT devices allows to tamper with code in memory, steal confidential Intellectual Property (IP), or mount fault attacks to manipulate a CPU's control flow. In this work, we pre… ▽ More Embedded devices in the Internet of Things (IoT) face a wide variety of security challenges. For example, software attackers perform code injection and code-reuse attacks on their remote interfaces, and physical access to IoT devices allows to tamper with code in memory, steal confidential Intellectual Property (IP), or mount fault attacks to manipulate a CPU's control flow. In this work, we present Sponge-based Control Flow Protection (SCFP). SCFP is a stateful, sponge-based scheme to ensure the confidentiality of software IP and its authentic execution on IoT devices. At compile time, SCFP encrypts and authenticates software with instruction-level granularity. During execution, an SCFP hardware extension between the CPU's fetch and decode stage continuously decrypts and authenticates instructions. Sponge-based authenticated encryption in SCFP yields fine-grained control-flow integrity and thus prevents code-reuse, code-injection, and fault attacks on the code and the control flow. In addition, SCFP withstands any modification of software in memory. For evaluation, we extended a RISC-V core with SCFP and fabricated a real System on Chip (SoC). The average overhead in code size and execution time of SCFP on this design is 19.8% and 9.1%, respectively, and thus meets the requirements of embedded IoT devices. △ Less

Submitted 19 February, 2018; originally announced February 2018.

Comments: accepted at IEEE EuroS&P 2018

gearshifft - The FFT Benchmark Suite for Heterogeneous Platforms

Authors: Peter Steinbach, Matthias Werner

Abstract: Fast Fourier Transforms (FFTs) are exploited in a wide variety of fields ranging from computer science to natural sciences and engineering. With the rising data production bandwidths of modern FFT applications, judging best which algorithmic tool to apply, can be vital to any scientific endeavor. As tailored FFT implementations exist for an ever increasing variety of high performance computer hard… ▽ More Fast Fourier Transforms (FFTs) are exploited in a wide variety of fields ranging from computer science to natural sciences and engineering. With the rising data production bandwidths of modern FFT applications, judging best which algorithmic tool to apply, can be vital to any scientific endeavor. As tailored FFT implementations exist for an ever increasing variety of high performance computer hardware, choosing the best performing FFT implementation has strong implications for future hardware purchase decisions, for resources FFTs consume and for possibly decisive financial and time savings ahead of the competition. This paper therefor presents gearshifft, which is an open-source and vendor agnostic benchmark suite to process a wide variety of problem sizes and types with state-of-the-art FFT implementations (fftw, clfft and cufft). gearshifft provides a reproducible, unbiased and fair comparison on a wide variety of hardware to explore which FFT variant is best for a given problem size. △ Less

Submitted 11 July, 2017; v1 submitted 2 February, 2017; originally announced February 2017.

Journal ref: High Performance Computing, Theoretical Computer Science and General Issues, Vol. 10266, Springer International Publishing. (ISC High Performance 2017)

SGXIO: Generic Trusted I/O Path for Intel SGX

Authors: Samuel Weiser, Mario Werner

Abstract: Application security traditionally strongly relies upon security of the underlying operating system. However, operating systems often fall victim to software attacks, compromising security of applications as well. To overcome this dependency, Intel introduced SGX, which allows to protect application code against a subverted or malicious OS by running it in a hardware-protected enclave. However, SG… ▽ More Application security traditionally strongly relies upon security of the underlying operating system. However, operating systems often fall victim to software attacks, compromising security of applications as well. To overcome this dependency, Intel introduced SGX, which allows to protect application code against a subverted or malicious OS by running it in a hardware-protected enclave. However, SGX lacks support for generic trusted I/O paths to protect user input and output between enclaves and I/O devices. This work presents SGXIO, a generic trusted path architecture for SGX, allowing user applications to run securely on top of an untrusted OS, while at the same time supporting trusted paths to generic I/O devices. To achieve this, SGXIO combines the benefits of SGX's easy programming model with traditional hypervisor-based trusted path architectures. Moreover, SGXIO can tweak insecure debug enclaves to behave like secure production enclaves. SGXIO surpasses traditional use cases in cloud computing and makes SGX technology usable for protecting user-centric, local applications against kernel-level keyloggers and likewise. It is compatible to unmodified operating systems and works on a modern commodity notebook out of the box. Hence, SGXIO is particularly promising for the broad x86 community to which SGX is readily available. △ Less

Submitted 4 January, 2017; originally announced January 2017.

Comments: To appear in CODASPY'16

On the mixed $f$-divergence for multiple pairs of measures

Authors: Elisabeth M. Werner, Deping Ye

Abstract: In this paper, the concept of the classical $f$-divergence (for a pair of measures) is extended to the mixed $f$-divergence (for multiple pairs of measures). The mixed $f$-divergence provides a way to measure the difference between multiple pairs of (probability) measures. Properties for the mixed $f$-divergence are established, such as permutation invariance and symmetry in distributions. An Alex… ▽ More In this paper, the concept of the classical $f$-divergence (for a pair of measures) is extended to the mixed $f$-divergence (for multiple pairs of measures). The mixed $f$-divergence provides a way to measure the difference between multiple pairs of (probability) measures. Properties for the mixed $f$-divergence are established, such as permutation invariance and symmetry in distributions. An Alexandrov-Fenchel type inequality and an isoperimetric type inequality for the mixed $f$-divergence will be proved and applications in the theory of convex bodies are given. △ Less

Submitted 24 April, 2013; originally announced April 2013.

MSC Class: 94A15; 94A17

f-Divergence for convex bodies

Authors: Elisabeth M. Werner

Abstract: We introduce f-divergence, a concept from information theory and statistics, for convex bodies in R^n. We prove that f-divergences are SL(n) invariant valuations and we establish an affine isoperimetric inequality for these quantities. We show that generalized affine surface area and in particular the L_p affine surface area from the L_p Brunn Minkowski theory are special cases of f-divergences. We introduce f-divergence, a concept from information theory and statistics, for convex bodies in R^n. We prove that f-divergences are SL(n) invariant valuations and we establish an affine isoperimetric inequality for these quantities. We show that generalized affine surface area and in particular the L_p affine surface area from the L_p Brunn Minkowski theory are special cases of f-divergences. △ Less

Submitted 15 May, 2012; originally announced May 2012.

MSC Class: 52A20; 53A15