Showing 1–38 of 38 results for author: Blasch, E

Neuro-Symbolic Fusion of Wi-Fi Sensing Data for Passive Radar with Inter-Modal Knowledge Transfer

Authors: Marco Cominelli, Francesco Gringoli, Lance M. Kaplan, Mani B. Srivastava, Trevor Bihl, Erik P. Blasch, Nandini Iyer, Federico Cerutti

Abstract: Wi-Fi devices, akin to passive radars, can discern human activities within indoor settings due to the human body's interaction with electromagnetic signals. Current Wi-Fi sensing applications predominantly employ data-driven learning techniques to associate the fluctuations in the physical properties of the communication channel with the human activity causing them. However, these techniques often… ▽ More Wi-Fi devices, akin to passive radars, can discern human activities within indoor settings due to the human body's interaction with electromagnetic signals. Current Wi-Fi sensing applications predominantly employ data-driven learning techniques to associate the fluctuations in the physical properties of the communication channel with the human activity causing them. However, these techniques often lack the desired flexibility and transparency. This paper introduces DeepProbHAR, a neuro-symbolic architecture for Wi-Fi sensing, providing initial evidence that Wi-Fi signals can differentiate between simple movements, such as leg or arm movements, which are integral to human activities like running or walking. The neuro-symbolic approach affords gathering such evidence without needing additional specialised data collection or labelling. The training of DeepProbHAR is facilitated by declarative domain knowledge obtained from a camera feed and by fusing signals from various antennas of the Wi-Fi receivers. DeepProbHAR achieves results comparable to the state-of-the-art in human activity recognition. Moreover, as a by-product of the learning process, DeepProbHAR generates specialised classifiers for simple movements that match the accuracy of models trained on finely labelled datasets, which would be particularly costly. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Comments: 8 pages, 9 figures, accepted at 27th International Conference on Information Fusion (FUSION 2024)

Efficient Temporal Action Segmentation via Boundary-aware Query Voting

Authors: Peiyao Wang, Yuewei Lin, Erik Blasch, Jie Wei, Haibin Ling

Abstract: Although the performance of Temporal Action Segmentation (TAS) has improved in recent years, achieving promising results often comes with a high computational cost due to dense inputs, complex model structures, and resource-intensive post-processing requirements. To improve the efficiency while keeping the performance, we present a novel perspective centered on per-segment classification. By harne… ▽ More Although the performance of Temporal Action Segmentation (TAS) has improved in recent years, achieving promising results often comes with a high computational cost due to dense inputs, complex model structures, and resource-intensive post-processing requirements. To improve the efficiency while keeping the performance, we present a novel perspective centered on per-segment classification. By harnessing the capabilities of Transformers, we tokenize each video segment as an instance token, endowed with intrinsic instance segmentation. To realize efficient action segmentation, we introduce BaFormer, a boundary-aware Transformer network. It employs instance queries for instance segmentation and a global query for class-agnostic boundary prediction, yielding continuous segment proposals. During inference, BaFormer employs a simple yet effective voting strategy to classify boundary-wise segments based on instance segmentation. Remarkably, as a single-stage approach, BaFormer significantly reduces the computational costs, utilizing only 6% of the running time compared to state-of-the-art method DiffAct, while producing better or comparable accuracy over several popular benchmarks. The code for this project is publicly available at https://github.com/peiyao-w/BaFormer. △ Less

Submitted 24 May, 2024; originally announced May 2024.

Comments: 17 pages, 8 figures, 11 tables

Credibility-Aware Multi-Modal Fusion Using Probabilistic Circuits

Authors: Sahil Sidheekh, Pranuthi Tenali, Saurabh Mathur, Erik Blasch, Kristian Kersting, Sriraam Natarajan

Abstract: We consider the problem of late multi-modal fusion for discriminative learning. Motivated by noisy, multi-source domains that require understanding the reliability of each data source, we explore the notion of credibility in the context of multi-modal fusion. We propose a combination function that uses probabilistic circuits (PCs) to combine predictive distributions over individual modalities. We… ▽ More We consider the problem of late multi-modal fusion for discriminative learning. Motivated by noisy, multi-source domains that require understanding the reliability of each data source, we explore the notion of credibility in the context of multi-modal fusion. We propose a combination function that uses probabilistic circuits (PCs) to combine predictive distributions over individual modalities. We also define a probabilistic measure to evaluate the credibility of each modality via inference queries over the PC. Our experimental evaluation demonstrates that our fusion method can reliably infer credibility while maintaining competitive performance with the state-of-the-art. △ Less

Submitted 17 July, 2024; v1 submitted 5 March, 2024; originally announced March 2024.

PADTHAI-MM: A Principled Approach for Designing Trustable, Human-centered AI systems using the MAST Methodology

Authors: Nayoung Kim, Myke C. Cohen, Yang Ba, Anna Pan, Shawaiz Bhatti, Pouria Salehi, James Sung, Erik Blasch, Michelle V. Mancenido, Erin K. Chiou

Abstract: Designing for AI trustworthiness is challenging, with a lack of practical guidance despite extensive literature on trust. The Multisource AI Scorecard Table (MAST), a checklist rating system, addresses this gap in designing and evaluating AI-enabled decision support systems. We propose the Principled Approach for Designing Trustable Human-centered AI systems using MAST Methodology (PADTHAI-MM), a… ▽ More Designing for AI trustworthiness is challenging, with a lack of practical guidance despite extensive literature on trust. The Multisource AI Scorecard Table (MAST), a checklist rating system, addresses this gap in designing and evaluating AI-enabled decision support systems. We propose the Principled Approach for Designing Trustable Human-centered AI systems using MAST Methodology (PADTHAI-MM), a nine-step framework what we demonstrate through the iterative design of a text analysis platform called the REporting Assistant for Defense and Intelligence Tasks (READIT). We designed two versions of READIT, high-MAST including AI context and explanations, and low-MAST resembling a "black box" type system. Participant feedback and state-of-the-art AI knowledge was integrated in the design process, leading to a redesigned prototype tested by participants in an intelligence reporting task. Results show that MAST-guided design can improve trust perceptions, and that MAST criteria can be linked to performance, process, and purpose information, providing a practical and theory-informed basis for AI system design. △ Less

Submitted 24 January, 2024; originally announced January 2024.

Evaluating Trustworthiness of AI-Enabled Decision Support Systems: Validation of the Multisource AI Scorecard Table (MAST)

Authors: Pouria Salehi, Yang Ba, Nayoung Kim, Ahmadreza Mosallanezhad, Anna Pan, Myke C. Cohen, Yixuan Wang, Jieqiong Zhao, Shawaiz Bhatti, James Sung, Erik Blasch, Michelle V. Mancenido, Erin K. Chiou

Abstract: The Multisource AI Scorecard Table (MAST) is a checklist tool based on analytic tradecraft standards to inform the design and evaluation of trustworthy AI systems. In this study, we evaluate whether MAST is associated with people's trust perceptions in AI-enabled decision support systems (AI-DSSs). Evaluating trust in AI-DSSs poses challenges to researchers and practitioners. These challenges incl… ▽ More The Multisource AI Scorecard Table (MAST) is a checklist tool based on analytic tradecraft standards to inform the design and evaluation of trustworthy AI systems. In this study, we evaluate whether MAST is associated with people's trust perceptions in AI-enabled decision support systems (AI-DSSs). Evaluating trust in AI-DSSs poses challenges to researchers and practitioners. These challenges include identifying the components, capabilities, and potential of these systems, many of which are based on the complex deep learning algorithms that drive DSS performance and preclude complete manual inspection. We developed two interactive, AI-DSS test environments using the MAST criteria. One emulated an identity verification task in security screening, and another emulated a text summarization system to aid in an investigative reporting task. Each test environment had one version designed to match low-MAST ratings, and another designed to match high-MAST ratings, with the hypothesis that MAST ratings would be positively related to the trust ratings of these systems. A total of 177 subject matter experts were recruited to interact with and evaluate these systems. Results generally show higher MAST ratings for the high-MAST conditions compared to the low-MAST groups, and that measures of trust perception are highly correlated with the MAST ratings. We conclude that MAST can be a useful tool for designing and evaluating systems that will engender high trust perceptions, including AI-DSS that may be used to support visual screening and text summarization tasks. However, higher MAST ratings may not translate to higher joint performance. △ Less

Submitted 29 November, 2023; originally announced November 2023.

Image Prior and Posterior Conditional Probability Representation for Efficient Damage Assessment

Authors: Jie Wei, Weicong Feng, Erik Blasch, Erika Ardiles-Cruz, Haibin Ling

Abstract: It is important to quantify Damage Assessment (DA) for Human Assistance and Disaster Response (HADR) applications. In this paper, to achieve efficient and scalable DA in HADR, an image prior and posterior conditional probability (IP2CP) is developed as an effective computational imaging representation. Equipped with the IP2CP representation, the matching pre- and post-disaster images are effective… ▽ More It is important to quantify Damage Assessment (DA) for Human Assistance and Disaster Response (HADR) applications. In this paper, to achieve efficient and scalable DA in HADR, an image prior and posterior conditional probability (IP2CP) is developed as an effective computational imaging representation. Equipped with the IP2CP representation, the matching pre- and post-disaster images are effectively encoded into one image that is then processed using deep learning approaches to determine the damage levels. Two scenarios of crucial importance for the practical use of DA in HADR applications are examined: pixel-wise semantic segmentation and patch-based contrastive learning-based global damage classification. Results achieved by IP2CP in both scenarios demonstrate promising performances, showing that our IP2CP-based methods within the deep learning framework can effectively achieve data and computational efficiency, which is of utmost importance for the DA in HADR applications. △ Less

Submitted 26 October, 2023; originally announced October 2023.

Comments: 6 pages, 2 figures

MSC Class: I.4.6; I.5.3

Transparent Object Tracking with Enhanced Fusion Module

Authors: Kalyan Garigapati, Erik Blasch, Jie Wei, Haibin Ling

Abstract: Accurate tracking of transparent objects, such as glasses, plays a critical role in many robotic tasks such as robot-assisted living. Due to the adaptive and often reflective texture of such objects, traditional tracking algorithms that rely on general-purpose learned features suffer from reduced performance. Recent research has proposed to instill transparency awareness into existing general obje… ▽ More Accurate tracking of transparent objects, such as glasses, plays a critical role in many robotic tasks such as robot-assisted living. Due to the adaptive and often reflective texture of such objects, traditional tracking algorithms that rely on general-purpose learned features suffer from reduced performance. Recent research has proposed to instill transparency awareness into existing general object trackers by fusing purpose-built features. However, with the existing fusion techniques, the addition of new features causes a change in the latent space making it impossible to incorporate transparency awareness on trackers with fixed latent spaces. For example, many of the current days transformer-based trackers are fully pre-trained and are sensitive to any latent space perturbations. In this paper, we present a new feature fusion technique that integrates transparency information into a fixed feature space, enabling its use in a broader range of trackers. Our proposed fusion module, composed of a transformer encoder and an MLP module, leverages key query-based transformations to embed the transparency information into the tracking pipeline. We also present a new two-step training strategy for our fusion module to effectively merge transparency features. We propose a new tracker architecture that uses our fusion techniques to achieve superior results for transparent object tracking. Our proposed method achieves competitive results with state-of-the-art trackers on TOTB, which is the largest transparent object tracking benchmark recently released. Our results and the implementation of code will be made publicly available at https://github.com/kalyan0510/TOTEM. △ Less

Submitted 12 September, 2023; originally announced September 2023.

Comments: IEEE IROS 2023

CCTV-Gun: Benchmarking Handgun Detection in CCTV Images

Authors: Srikar Yellapragada, Zhenghong Li, Kevin Bhadresh Doshi, Purva Makarand Mhasakar, Heng Fan, Jie Wei, Erik Blasch, Bin Zhang, Haibin Ling

Abstract: Gun violence is a critical security problem, and it is imperative for the computer vision community to develop effective gun detection algorithms for real-world scenarios, particularly in Closed Circuit Television (CCTV) surveillance data. Despite significant progress in visual object detection, detecting guns in real-world CCTV images remains a challenging and under-explored task. Firearms, espec… ▽ More Gun violence is a critical security problem, and it is imperative for the computer vision community to develop effective gun detection algorithms for real-world scenarios, particularly in Closed Circuit Television (CCTV) surveillance data. Despite significant progress in visual object detection, detecting guns in real-world CCTV images remains a challenging and under-explored task. Firearms, especially handguns, are typically very small in size, non-salient in appearance, and often severely occluded or indistinguishable from other small objects. Additionally, the lack of principled benchmarks and difficulty collecting relevant datasets further hinder algorithmic development. In this paper, we present a meticulously crafted and annotated benchmark, called \textbf{CCTV-Gun}, which addresses the challenges of detecting handguns in real-world CCTV images. Our contribution is three-fold. Firstly, we carefully select and analyze real-world CCTV images from three datasets, manually annotate handguns and their holders, and assign each image with relevant challenge factors such as blur and occlusion. Secondly, we propose a new cross-dataset evaluation protocol in addition to the standard intra-dataset protocol, which is vital for gun detection in practical settings. Finally, we comprehensively evaluate both classical and state-of-the-art object detection algorithms, providing an in-depth analysis of their generalizing abilities. The benchmark will facilitate further research and development on this topic and ultimately enhance security. Code, annotations, and trained models are available at https://github.com/srikarym/CCTV-Gun. △ Less

Submitted 11 July, 2023; v1 submitted 19 March, 2023; originally announced March 2023.

A Secure Dynamic Edge Resource Federation Architecture for Cross-Domain IoT Systems

Authors: Ronghua Xu, Yu Chen, Xiaohua Li, Erik Blasch

Abstract: The fast integration of 5G communication, Artificial Intelligence (AI), and Internet-of-Things (IoT) technologies is envisioned to enable Next Generation Networks (NGNs) for diverse smart services and user-defined applications for Smart Cities. However, it is still challenging to build a scalable and efficient infrastructure that satisfies the various performance, security, and management demands… ▽ More The fast integration of 5G communication, Artificial Intelligence (AI), and Internet-of-Things (IoT) technologies is envisioned to enable Next Generation Networks (NGNs) for diverse smart services and user-defined applications for Smart Cities. However, it is still challenging to build a scalable and efficient infrastructure that satisfies the various performance, security, and management demands by heterogeneous IoT applications across multiple administrative domains. This paper presents a dynamic edge resource federation architecture, which integrates the concept of network slicing (NS) and blockchain to improve scalability, dynamicity, and security for multi-domain IoT applications. A NS-enabled dynamic edge resource federation framework adopts intelligent mechanisms to support efficient multi-domain service coordination that satisfies diverse Quality of Service (QoS) and security requirements. We propose a Hierarchical Integrated Federated Ledger (HIFL), which aims to guarantee decentralized security and privacy-preserving properties in multi-domain resource orchestration and service re-adjustment. As a secure-by-design solution, HIFL is promising to support efficient, trust and secured end-to-end IoT services. A preliminary proof-of-concept prototype has been implemented for comparing intra- and inter-domain performance expectations. △ Less

Submitted 2 August, 2022; originally announced August 2022.

Journal ref: the 4th International Workshop on Smart City Communication and Networking (SmartCityCom 2022), in conjunction with the 31st International Conference on Computer Communications and Networks (ICCCN 2022), July 27, 2022

DeFakePro: Decentralized DeepFake Attacks Detection using ENF Authentication

Authors: Deeraj Nagothu, Ronghua Xu, Yu Chen, Erik Blasch, Alexander Aved

Abstract: Advancements in generative models, like Deepfake allows users to imitate a targeted person and manipulate online interactions. It has been recognized that disinformation may cause disturbance in society and ruin the foundation of trust. This article presents DeFakePro, a decentralized consensus mechanism-based Deepfake detection technique in online video conferencing tools. Leveraging Electrical N… ▽ More Advancements in generative models, like Deepfake allows users to imitate a targeted person and manipulate online interactions. It has been recognized that disinformation may cause disturbance in society and ruin the foundation of trust. This article presents DeFakePro, a decentralized consensus mechanism-based Deepfake detection technique in online video conferencing tools. Leveraging Electrical Network Frequency (ENF), an environmental fingerprint embedded in digital media recording, affords a consensus mechanism design called Proof-of-ENF (PoENF) algorithm. The similarity in ENF signal fluctuations is utilized in the PoENF algorithm to authenticate the media broadcasted in conferencing tools. By utilizing the video conferencing setup with malicious participants to broadcast deep fake video recordings to other participants, the DeFakePro system verifies the authenticity of the incoming media in both audio and video channels. △ Less

Submitted 21 July, 2022; originally announced July 2022.

Journal ref: the IEEE IT Professional, Special Issue on Information Hygiene and the Fight against the Misinformation Info-demic, 2022

Automatic Concept Extraction for Concept Bottleneck-based Video Classification

Authors: Jeya Vikranth Jeyakumar, Luke Dickens, Luis Garcia, Yu-Hsi Cheng, Diego Ramirez Echavarria, Joseph Noor, Alessandra Russo, Lance Kaplan, Erik Blasch, Mani Srivastava

Abstract: Recent efforts in interpretable deep learning models have shown that concept-based explanation methods achieve competitive accuracy with standard end-to-end models and enable reasoning and intervention about extracted high-level visual concepts from images, e.g., identifying the wing color and beak length for bird-species classification. However, these concept bottleneck models rely on a necessary… ▽ More Recent efforts in interpretable deep learning models have shown that concept-based explanation methods achieve competitive accuracy with standard end-to-end models and enable reasoning and intervention about extracted high-level visual concepts from images, e.g., identifying the wing color and beak length for bird-species classification. However, these concept bottleneck models rely on a necessary and sufficient set of predefined concepts-which is intractable for complex tasks such as video classification. For complex tasks, the labels and the relationship between visual elements span many frames, e.g., identifying a bird flying or catching prey-necessitating concepts with various levels of abstraction. To this end, we present CoDEx, an automatic Concept Discovery and Extraction module that rigorously composes a necessary and sufficient set of concept abstractions for concept-based video classification. CoDEx identifies a rich set of complex concept abstractions from natural language explanations of videos-obviating the need to predefine the amorphous set of concepts. To demonstrate our method's viability, we construct two new public datasets that combine existing complex video classification datasets with short, crowd-sourced natural language explanations for their labels. Our method elicits inherent complex concept abstractions in natural language to generalize concept-bottleneck methods to complex tasks. △ Less

Submitted 21 June, 2022; originally announced June 2022.

Comments: 10 pages, Appendix: 2 pages

Hawkes Process Modeling of Block Arrivals in Bitcoin Blockchain

Authors: Rui Luo, Vikram Krishnamurthy, Erik Blasch

Abstract: The paper constructs a multi-variate Hawkes process model of Bitcoin block arrivals and price jumps. Hawkes processes are selfexciting point processes that can capture the self- and cross-excitation effects of block mining and Bitcoin price volatility. We use publicly available blockchain datasets to estimate the model parameters via maximum likelihood estimation. The results show that Bitcoin pri… ▽ More The paper constructs a multi-variate Hawkes process model of Bitcoin block arrivals and price jumps. Hawkes processes are selfexciting point processes that can capture the self- and cross-excitation effects of block mining and Bitcoin price volatility. We use publicly available blockchain datasets to estimate the model parameters via maximum likelihood estimation. The results show that Bitcoin price volatility boost block mining rate and Bitcoin investment return demonstrates mean reversion. Quantile-Quantile plots show that the proposed Hawkes process model is a better fit to the blockchain datasets than a Poisson process model. △ Less

Submitted 30 March, 2022; originally announced March 2022.

Mitigating Misinformation Spread on Blockchain Enabled Social Media Networks

Authors: Rui Luo, Vikram Krishnamurthy, Erik Blasch

Abstract: The paper develops a blockchain protocol for a social media network (BE-SMN) to mitigate the spread of misinformation. BE-SMN is derived based on the information transmission-time distribution by modeling the misinformation transmission as double-spend attacks on blockchain. The misinformation distribution is then incorporated into the SIR (Susceptible, Infectious, or Recovered) model, which subst… ▽ More The paper develops a blockchain protocol for a social media network (BE-SMN) to mitigate the spread of misinformation. BE-SMN is derived based on the information transmission-time distribution by modeling the misinformation transmission as double-spend attacks on blockchain. The misinformation distribution is then incorporated into the SIR (Susceptible, Infectious, or Recovered) model, which substitutes the single rate parameter in the traditional SIR model. Then, on a multi-community network, we study the propagation of misinformation numerically and show that the proposed blockchain enabled social media network outperforms the baseline network in flattening the curve of the infected population. △ Less

Submitted 1 May, 2023; v1 submitted 18 January, 2022; originally announced January 2022.

AAAI FSS-21: Artificial Intelligence in Government and Public Sector Proceedings

Authors: Mihai Boicu, Erik Blasch, Alun Preece

Abstract: Proceedings of the AAAI Fall Symposium on Artificial Intelligence in Government and Public Sector, Washington, DC, USA, November 4-6, 2021 Proceedings of the AAAI Fall Symposium on Artificial Intelligence in Government and Public Sector, Washington, DC, USA, November 4-6, 2021 △ Less

Submitted 10 December, 2021; originally announced December 2021.

Comments: Post-symposium proceedings including 9 papers

The Powerful Use of AI in the Energy Sector: Intelligent Forecasting

Authors: Erik Blasch, Haoran Li, Zhihao Ma, Yang Weng

Abstract: Artificial Intelligence (AI) techniques continue to broaden across governmental and public sectors, such as power and energy - which serve as critical infrastructures for most societal operations. However, due to the requirements of reliability, accountability, and explainability, it is risky to directly apply AI-based methods to power systems because society cannot afford cascading failures and l… ▽ More Artificial Intelligence (AI) techniques continue to broaden across governmental and public sectors, such as power and energy - which serve as critical infrastructures for most societal operations. However, due to the requirements of reliability, accountability, and explainability, it is risky to directly apply AI-based methods to power systems because society cannot afford cascading failures and large-scale blackouts, which easily cost billions of dollars. To meet society requirements, this paper proposes a methodology to develop, deploy, and evaluate AI systems in the energy sector by: (1) understanding the power system measurements with physics, (2) designing AI algorithms to forecast the need, (3) developing robust and accountable AI methods, and (4) creating reliable measures to evaluate the performance of the AI model. The goal is to provide a high level of confidence to energy utility users. For illustration purposes, the paper uses power system event forecasting (PEF) as an example, which carefully analyzes synchrophasor patterns measured by the Phasor Measurement Units (PMUs). Such a physical understanding leads to a data-driven framework that reduces the dimensionality with physics and forecasts the event with high credibility. Specifically, for dimensionality reduction, machine learning arranges physical information from different dimensions, resulting inefficient information extraction. For event forecasting, the supervised learning model fuses the results of different models to increase the confidence. Finally, comprehensive experiments demonstrate the high accuracy, efficiency, and reliability as compared to other state-of-the-art machine learning methods. △ Less

Submitted 3 November, 2021; originally announced November 2021.

Comments: Presented at AAAI FSS-21: Artificial Intelligence in Government and Public Sector, Washington, DC, USA

Certifiable Artificial Intelligence Through Data Fusion

Authors: Erik Blasch, Junchi Bin, Zheng Liu

Abstract: This paper reviews and proposes concerns in adopting, fielding, and maintaining artificial intelligence (AI) systems. While the AI community has made rapid progress, there are challenges in certifying AI systems. Using procedures from design and operational test and evaluation, there are opportunities towards determining performance bounds to manage expectations of intended use. A notional use cas… ▽ More This paper reviews and proposes concerns in adopting, fielding, and maintaining artificial intelligence (AI) systems. While the AI community has made rapid progress, there are challenges in certifying AI systems. Using procedures from design and operational test and evaluation, there are opportunities towards determining performance bounds to manage expectations of intended use. A notional use case is presented with image data fusion to support AI object recognition certifiability considering precision versus distance. △ Less

Submitted 2 November, 2021; originally announced November 2021.

Comments: Presented at AAAI FSS-21: Artificial Intelligence in Government and Public Sector, Washington, DC, USA

NIDA-CLIFGAN: Natural Infrastructure Damage Assessment through Efficient Classification Combining Contrastive Learning, Information Fusion and Generative Adversarial Networks

Authors: Jie Wei, Zhigang Zhu, Erik Blasch, Bilal Abdulrahman, Billy Davila, Shuoxin Liu, Jed Magracia, Ling Fang

Abstract: During natural disasters, aircraft and satellites are used to survey the impacted regions. Usually human experts are needed to manually label the degrees of the building damage so that proper humanitarian assistance and disaster response (HADR) can be achieved, which is labor-intensive and time-consuming. Expecting human labeling of major disasters over a wide area gravely slows down the HADR effo… ▽ More During natural disasters, aircraft and satellites are used to survey the impacted regions. Usually human experts are needed to manually label the degrees of the building damage so that proper humanitarian assistance and disaster response (HADR) can be achieved, which is labor-intensive and time-consuming. Expecting human labeling of major disasters over a wide area gravely slows down the HADR efforts. It is thus of crucial interest to take advantage of the cutting-edge Artificial Intelligence and Machine Learning techniques to speed up the natural infrastructure damage assessment process to achieve effective HADR. Accordingly, the paper demonstrates a systematic effort to achieve efficient building damage classification. First, two novel generative adversarial nets (GANs) are designed to augment data used to train the deep-learning-based classifier. Second, a contrastive learning based method using novel data structures is developed to achieve great performance. Third, by using information fusion, the classifier is effectively trained with very few training data samples for transfer learning. All the classifiers are small enough to be loaded in a smart phone or simple laptop for first responders. Based on the available overhead imagery dataset, results demonstrate data and computational efficiency with 10% of the collected data combined with a GAN reducing the time of computation from roughly half a day to about 1 hour with roughly similar classification performances. △ Less

Submitted 29 October, 2022; v1 submitted 27 October, 2021; originally announced October 2021.

UAV-Assisted Communication in Remote Disaster Areas using Imitation Learning

Authors: Alireza Shamsoshoara, Fatemeh Afghah, Erik Blasch, Jonathan Ashdown, Mehdi Bennis

Abstract: The damage to cellular towers during natural and man-made disasters can disturb the communication services for cellular users. One solution to the problem is using unmanned aerial vehicles to augment the desired communication network. The paper demonstrates the design of a UAV-Assisted Imitation Learning (UnVAIL) communication system that relays the cellular users' information to a neighbor base s… ▽ More The damage to cellular towers during natural and man-made disasters can disturb the communication services for cellular users. One solution to the problem is using unmanned aerial vehicles to augment the desired communication network. The paper demonstrates the design of a UAV-Assisted Imitation Learning (UnVAIL) communication system that relays the cellular users' information to a neighbor base station. Since the user equipment (UEs) are equipped with buffers with limited capacity to hold packets, UnVAIL alternates between different UEs to reduce the chance of buffer overflow, positions itself optimally close to the selected UE to reduce service time, and uncovers a network pathway by acting as a relay node. UnVAIL utilizes Imitation Learning (IL) as a data-driven behavioral cloning approach to accomplish an optimal scheduling solution. Results demonstrate that UnVAIL performs similar to a human expert knowledge-based planning in communication timeliness, position accuracy, and energy consumption with an accuracy of 97.52% when evaluated on a developed simulator to train the UAV. △ Less

Submitted 1 April, 2021; originally announced May 2021.

Comments: 15 pages, 14 Figures, 1 Table

TULVCAN: Terahertz Ultra-broadband Learning Vehicular Channel-Aware Networking

Authors: Chia-Hung Lin, Shih-Chun Lin, Erik Blasch

Abstract: Due to spectrum scarcity and increasing wireless capacity demands, terahertz (THz) communications at 0.1-10THz and the corresponding spectrum characterization have emerged to meet diverse service requirements in future 5G and 6G wireless systems. However, conventional compressed sensing techniques to reconstruct the original wideband spectrum with under-sampled measurements become inefficient as l… ▽ More Due to spectrum scarcity and increasing wireless capacity demands, terahertz (THz) communications at 0.1-10THz and the corresponding spectrum characterization have emerged to meet diverse service requirements in future 5G and 6G wireless systems. However, conventional compressed sensing techniques to reconstruct the original wideband spectrum with under-sampled measurements become inefficient as local spectral correlation is deliberately omitted. Recent works extend communication methods with deep learning-based algorithms but lack strong ties to THz channel properties. This paper introduces novel THz channel-aware spectrum learning solutions that fully disclose the uniqueness of THz channels when performing such ultra-broadband sensing in vehicular environments. Specifically, a joint design of spectrum compression and reconstruction is proposed through a structured sensing matrix and two-phase reconstruction based on high spreading loss and molecular absorption at THz frequencies. An end-to-end learning framework, namely compression and reconstruction network (CRNet), is further developed with the mean-square-error loss function to improve sensing accuracy while significantly reducing computational complexity. Numerical results show that the CRNet solutions outperform the latest generative adversarial network (GAN) realization with a much higher cosine and structure similarity measures, smaller learning errors, and 56% less required training overheads. This THz Ultra-broadband Learning Vehicular Channel-Aware Networking (TULVCAN) work successfully achieves effective THz spectrum learning and hence allows frequency-agile access. △ Less

Submitted 28 March, 2021; originally announced March 2021.

Comments: This paper was already accepted by the IEEE 2021 INFOCOM Workshop

Multisource AI Scorecard Table for System Evaluation

Authors: Erik Blasch, James Sung, Tao Nguyen

Abstract: The paper describes a Multisource AI Scorecard Table (MAST) that provides the developer and user of an artificial intelligence (AI)/machine learning (ML) system with a standard checklist focused on the principles of good analysis adopted by the intelligence community (IC) to help promote the development of more understandable systems and engender trust in AI outputs. Such a scorecard enables a tra… ▽ More The paper describes a Multisource AI Scorecard Table (MAST) that provides the developer and user of an artificial intelligence (AI)/machine learning (ML) system with a standard checklist focused on the principles of good analysis adopted by the intelligence community (IC) to help promote the development of more understandable systems and engender trust in AI outputs. Such a scorecard enables a transparent, consistent, and meaningful understanding of AI tools applied for commercial and government use. A standard is built on compliance and agreement through policy, which requires buy-in from the stakeholders. While consistency for testing might only exist across a standard data set, the community requires discussion on verification and validation approaches which can lead to interpretability, explainability, and proper use. The paper explores how the analytic tradecraft standards outlined in Intelligence Community Directive (ICD) 203 can provide a framework for assessing the performance of an AI system supporting various operational needs. These include sourcing, uncertainty, consistency, accuracy, and visualization. Three use cases are presented as notional examples that support security for comparative analysis. △ Less

Submitted 7 February, 2021; originally announced February 2021.

Comments: Presented at AAAI FSS-20: Artificial Intelligence in Government and Public Sector, Washington, DC, USA

Aerial Imagery Pile burn detection using Deep Learning: the FLAME dataset

Authors: Alireza Shamsoshoara, Fatemeh Afghah, Abolfazl Razi, Liming Zheng, Peter Z Fulé, Erik Blasch

Abstract: Wildfires are one of the costliest and deadliest natural disasters in the US, causing damage to millions of hectares of forest resources and threatening the lives of people and animals. Of particular importance are risks to firefighters and operational forces, which highlights the need for leveraging technology to minimize danger to people and property. FLAME (Fire Luminosity Airborne-based Machin… ▽ More Wildfires are one of the costliest and deadliest natural disasters in the US, causing damage to millions of hectares of forest resources and threatening the lives of people and animals. Of particular importance are risks to firefighters and operational forces, which highlights the need for leveraging technology to minimize danger to people and property. FLAME (Fire Luminosity Airborne-based Machine learning Evaluation) offers a dataset of aerial images of fires along with methods for fire detection and segmentation which can help firefighters and researchers to develop optimal fire management strategies. This paper provides a fire image dataset collected by drones during a prescribed burning piled detritus in an Arizona pine forest. The dataset includes video recordings and thermal heatmaps captured by infrared cameras. The captured videos and images are annotated and labeled frame-wise to help researchers easily apply their fire detection and modeling algorithms. The paper also highlights solutions to two machine learning problems: (1) Binary classification of video frames based on the presence [and absence] of fire flames. An Artificial Neural Network (ANN) method is developed that achieved a 76% classification accuracy. (2) Fire detection using segmentation methods to precisely determine fire borders. A deep learning method is designed based on the U-Net up-sampling and down-sampling approach to extract a fire mask from the video frames. Our FLAME method approached a precision of 92% and a recall of 84%. Future research will expand the technique for free burning broadcast fire using thermal images. △ Less

Submitted 27 December, 2020; originally announced December 2020.

Comments: 27 Pages, 7 Figures, 4 Tables

Hybrid Blockchain-Enabled Secure Microservices Fabric for Decentralized Multi-Domain Avionics Systems

Authors: Ronghua Xu, Yu Chen, Erik Blasch, Alexander Aved, Genshe Chen, Dan Shen

Abstract: Advancement in artificial intelligence (AI) and machine learning (ML), dynamic data driven application systems (DDDAS), and hierarchical cloud-fog-edge computing paradigm provide opportunities for enhancing multi-domain systems performance. As one example that represents multi-domain scenario, a "fly-by-feel" system utilizes DDDAS framework to support autonomous operations and improve maneuverabil… ▽ More Advancement in artificial intelligence (AI) and machine learning (ML), dynamic data driven application systems (DDDAS), and hierarchical cloud-fog-edge computing paradigm provide opportunities for enhancing multi-domain systems performance. As one example that represents multi-domain scenario, a "fly-by-feel" system utilizes DDDAS framework to support autonomous operations and improve maneuverability, safety and fuel efficiency. The DDDAS "fly-by-feel" avionics system can enhance multi-domain coordination to support domain specific operations. However, conventional enabling technologies rely on a centralized manner for data aggregation, sharing and security policy enforcement, and it incurs critical issues related to bottleneck of performance, data provenance and consistency. Inspired by the containerized microservices and blockchain technology, this paper introduces BLEM, a hybrid BLockchain-Enabled secure Microservices fabric to support decentralized, secure and efficient data fusion and multi-domain operations for avionics systems. Leveraging the fine-granularity and loose-coupling features of the microservices architecture, multidomain operations and security functionalities are decoupled into multiple containerized microservices. A hybrid blockchain fabric based on two-level committee consensus protocols is proposed to enable decentralized security architecture and support immutability, auditability and traceability for data provenience in existing multi-domain avionics system. Our evaluation results show the feasibility of the proposed BLEM mechanism to support decentralized security service and guarantee immutability, auditability and traceability for data provenience across domain boundaries. △ Less

Submitted 15 April, 2020; originally announced April 2020.

Comments: 2020 SPIE Defense + Commercial Sensing

I-ViSE: Interactive Video Surveillance as an Edge Service using Unsupervised Feature Queries

Authors: Seyed Yahya Nikouei, Yu Chen, Alexander Aved, Erik Blasch

Abstract: Situation AWareness (SAW) is essential for many mission critical applications. However, SAW is very challenging when trying to immediately identify objects of interest or zoom in on suspicious activities from thousands of video frames. This work aims at developing a queryable system to instantly select interesting content. While face recognition technology is mature, in many scenarios like public… ▽ More Situation AWareness (SAW) is essential for many mission critical applications. However, SAW is very challenging when trying to immediately identify objects of interest or zoom in on suspicious activities from thousands of video frames. This work aims at developing a queryable system to instantly select interesting content. While face recognition technology is mature, in many scenarios like public safety monitoring, the features of objects of interest may be much more complicated than face features. In addition, human operators may not be always able to provide a descriptive, simple, and accurate query. Actually, it is more often that there are only rough, general descriptions of certain suspicious objects or accidents. This paper proposes an Interactive Video Surveillance as an Edge service (I-ViSE) based on unsupervised feature queries. Adopting unsupervised methods that do not reveal any private information, the I-ViSE scheme utilizes general features of a human body and color of clothes. An I-ViSE prototype is built following the edge-fog computing paradigm and the experimental results verified the I-ViSE scheme meets the design goal of scene recognition in less than two seconds. △ Less

Submitted 9 March, 2020; originally announced March 2020.

Comments: R1 is under review by the IEEE Internet of Things Journal

Artificial Intelligence Strategies for National Security and Safety Standards

Authors: Erik Blasch, James Sung, Tao Nguyen, Chandra P. Daniel, Alisa P. Mason

Abstract: Recent advances in artificial intelligence (AI) have lead to an explosion of multimedia applications (e.g., computer vision (CV) and natural language processing (NLP)) for different domains such as commercial, industrial, and intelligence. In particular, the use of AI applications in a national security environment is often problematic because the opaque nature of the systems leads to an inability… ▽ More Recent advances in artificial intelligence (AI) have lead to an explosion of multimedia applications (e.g., computer vision (CV) and natural language processing (NLP)) for different domains such as commercial, industrial, and intelligence. In particular, the use of AI applications in a national security environment is often problematic because the opaque nature of the systems leads to an inability for a human to understand how the results came about. A reliance on 'black boxes' to generate predictions and inform decisions is potentially disastrous. This paper explores how the application of standards during each stage of the development of an AI system deployed and used in a national security environment would help enable trust. Specifically, we focus on the standards outlined in Intelligence Community Directive 203 (Analytic Standards) to subject machine outputs to the same rigorous standards as analysis performed by humans. △ Less

Submitted 3 November, 2019; originally announced November 2019.

Comments: Presented at AAAI FSS-19: Artificial Intelligence in Government and Public Sector, Arlington, Virginia, USA

Blockchain Methods for Trusted Avionics Systems

Authors: Erik Blasch, Ronghua Xu, Yu Chen, Genshe Chen, Dan Shen

Abstract: Blockchain is a popular method to ensure security for trusted systems. The benefits include an auditable method to provide decentralized security without a trusted third party, but the drawback is the large computational resources needed to process and store the ever-expanding chain of security blocks. The promise of blockchain for edge devices (e.g., internet of things) poses a variety of challen… ▽ More Blockchain is a popular method to ensure security for trusted systems. The benefits include an auditable method to provide decentralized security without a trusted third party, but the drawback is the large computational resources needed to process and store the ever-expanding chain of security blocks. The promise of blockchain for edge devices (e.g., internet of things) poses a variety of challenges and strategies before adoption. In this paper, we explore blockchain methods and examples, with experimental data to determine the merits of the capabilities. As for an aerospace example, we address a notional example for Automatic dependent surveillance-broadcast (ADS-B) from Flight24 data (https://www.flightradar24.com/) to determine whether blockchain is feasible for avionics systems. The methods are incorporated into the Lightweight Internet of Things (IoT) based Smart Public Safety (LISPS) framework. By decoupling a complex system into independent sub-tasks, the LISPS system possesses high flexibility in the design process and online maintenance. The Blockchain-enabled decentralized avionics services provide a secured data sharing and access control mechanism. The experimental results demonstrate the feasibility of the approach. △ Less

Submitted 21 October, 2019; originally announced October 2019.

Comments: Accepted and presented at 2019 IEEE NAECON Conference. arXiv admin note: text overlap with arXiv:1902.10567

Microchain: A Hybrid Consensus Mechanism for Lightweight Distributed Ledger for IoT

Authors: Ronghua Xu, Yu Chen, Erik Blasch, Genshe Chen

Abstract: A blockchain and smart contract enabled security mechanism for IoT applications has been reported recently for urban, financial, and network services. However, due to the power-intensive and a low-throughput consensus mechanism in existing blockchain, like Bitcoin and Ethereum, there are still challenges in integrating blockchain technology into resource-constrained IoT platforms. In this paper, M… ▽ More A blockchain and smart contract enabled security mechanism for IoT applications has been reported recently for urban, financial, and network services. However, due to the power-intensive and a low-throughput consensus mechanism in existing blockchain, like Bitcoin and Ethereum, there are still challenges in integrating blockchain technology into resource-constrained IoT platforms. In this paper, Microchain, based on a hybrid Proof-of-Credit (PoC)-Voting-based Chain Finality (VCF) consensus protocol, is proposed to provide a secure, scalable and lightweight distributed ledger for IoT systems. By using a bias-resistant randomness protocol and a cryptographic sortition algorithm, a random subset of nodes are selected as a final committee to perform the consensus protocol. The hybrid consensus mechanism relies on PoC, a pure Proof of stake (PoS) protocol, to determine whether or not a participant is qualified to propose a block, given a fair initial distribution of the credit assignment. The voting-based chain finality protocol is responsible for finalizing a history of blocks by resolving conflicting checkpoint and selecting a unique chain. A proof-of-conception prototype is implemented and tested on a physical network environment. The experimental results verify that the Micorchain is able to offer a partially decentralized, scalable and lightweight distributed ledger protocol for IoT applications. △ Less

Submitted 24 September, 2019; originally announced September 2019.

I-SAFE: Instant Suspicious Activity identiFication at the Edge using Fuzzy Decision Making

Authors: Seyed Yahya Nikouei, Yu Chen, Alexander Aved, Erik Blasch, Timothy R. Faughnan

Abstract: Urban imagery usually serves as forensic analysis and by design is available for incident mitigation. As more imagery collected, it is harder to narrow down to certain frames among thousands of video clips to a specific incident. A real-time, proactive surveillance system is desirable, which could instantly detect dubious personnel, identify suspicious activities, or raise momentous alerts. The re… ▽ More Urban imagery usually serves as forensic analysis and by design is available for incident mitigation. As more imagery collected, it is harder to narrow down to certain frames among thousands of video clips to a specific incident. A real-time, proactive surveillance system is desirable, which could instantly detect dubious personnel, identify suspicious activities, or raise momentous alerts. The recent proliferation of the edge computing paradigm allows more data-intensive tasks to be accomplished by smart edge devices with lightweight but powerful algorithms. This paper presents a forensic surveillance strategy by introducing an Instant Suspicious Activity identiFication at the Edge (I-SAFE) using fuzzy decision making. A fuzzy control system is proposed to mimic the decision-making process of a security officer. Decisions are made based on video features extracted by a lightweight Deep Machine Learning (DML) model. Based on the requirements from the first-line law enforcement officers, several features are selected and fuzzified to cope with the state of uncertainty that exists in the officers' decision-making process. Using features in the edge hierarchy minimizes the communication delay such that instant alerting is achieved. Additionally, leveraging the Microservices architecture, the I-SAFE scheme possesses good scalability given the increasing complexities at the network edge. Implemented as an edge-based application and tested using exemplary and various labeled dataset surveillance videos, the I-SAFE scheme raises alerts by identifying the suspicious activity in an average of 0.002 seconds. Compared to four other state-of-the-art methods over two other data sets, the experimental study verified the superiority of the I-SAFE decentralized method. △ Less

Submitted 12 September, 2019; originally announced September 2019.

Comments: Manuscript has been accepted and to be presented at the Fourth ACM/IEEE Symposium on Edge Computing, Washington DC, November 7-9, 2019

Clustered Object Detection in Aerial Images

Authors: Fan Yang, Heng Fan, Peng Chu, Erik Blasch, Haibin Ling

Abstract: Detecting objects in aerial images is challenging for at least two reasons: (1) target objects like pedestrians are very small in pixels, making them hardly distinguished from surrounding background; and (2) targets are in general sparsely and non-uniformly distributed, making the detection very inefficient. In this paper, we address both issues inspired by observing that these targets are often c… ▽ More Detecting objects in aerial images is challenging for at least two reasons: (1) target objects like pedestrians are very small in pixels, making them hardly distinguished from surrounding background; and (2) targets are in general sparsely and non-uniformly distributed, making the detection very inefficient. In this paper, we address both issues inspired by observing that these targets are often clustered. In particular, we propose a Clustered Detection (ClusDet) network that unifies object clustering and detection in an end-to-end framework. The key components in ClusDet include a cluster proposal sub-network (CPNet), a scale estimation sub-network (ScaleNet), and a dedicated detection network (DetecNet). Given an input image, CPNet produces object cluster regions and ScaleNet estimates object scales for these regions. Then, each scale-normalized cluster region is fed into DetecNet for object detection. ClusDet has several advantages over previous solutions: (1) it greatly reduces the number of chips for final object detection and hence achieves high running time efficiency, (2) the cluster-based scale estimation is more accurate than previously used single-object based ones, hence effectively improves the detection for small objects, and (3) the final DetecNet is dedicated for clustered regions and implicitly models the prior context information so as to boost detection accuracy. The proposed method is tested on three popular aerial image datasets including VisDrone, UAVDT and DOTA. In all experiments, ClusDet achieves promising performance in comparison with state-of-the-art detectors. Code will be available in \url{https://github.com/fyangneil}. △ Less

Submitted 26 August, 2019; v1 submitted 16 April, 2019; originally announced April 2019.

Decentralized Smart Surveillance through Microservices Platform

Authors: Seyed Yahya Nikouei, Ronghua Xu, Yu Chen, Alex Aved, Erik Blasch

Abstract: Connected societies require reliable measures to assure the safety, privacy, and security of members. Public safety technology has made fundamental improvements since the first generation of surveillance cameras were introduced, which aims to reduce the role of observer agents so that no abnormality goes unnoticed. While the edge computing paradigm promises solutions to address the shortcomings of… ▽ More Connected societies require reliable measures to assure the safety, privacy, and security of members. Public safety technology has made fundamental improvements since the first generation of surveillance cameras were introduced, which aims to reduce the role of observer agents so that no abnormality goes unnoticed. While the edge computing paradigm promises solutions to address the shortcomings of cloud computing, e.g., the extra communication delay and network security issues, it also introduces new challenges. One of the main concerns is the limited computing power at the edge to meet the on-site dynamic data processing. In this paper, a Lightweight IoT (Internet of Things) based Smart Public Safety (LISPS) framework is proposed on top of microservices architecture. As a computing hierarchy at the edge, the LISPS system possesses high flexibility in the design process, loose coupling to add new services or update existing functions without interrupting the normal operations, and efficient power balancing. A real-world public safety monitoring scenario is selected to verify the effectiveness of LISPS, which detects, tracks human objects and identify suspicious activities. The experimental results demonstrate the feasibility of the approach. △ Less

Submitted 11 March, 2019; originally announced March 2019.

Comments: 2019 SPIE Defense + Commercial Sensing

A Study on Smart Online Frame Forging Attacks against Video Surveillance System

Authors: Deeraj Nagothu, Jacob Schwell, Yu Chen, Erik Blasch, Sencun Zhu

Abstract: Video Surveillance Systems (VSS) have become an essential infrastructural element of smart cities by increasing public safety and countering criminal activities. A VSS is normally deployed in a secure network to prevent access from unauthorized personnel. Compared to traditional systems that continuously record video regardless of the actions in the frame, a smart VSS has the capability of capturi… ▽ More Video Surveillance Systems (VSS) have become an essential infrastructural element of smart cities by increasing public safety and countering criminal activities. A VSS is normally deployed in a secure network to prevent access from unauthorized personnel. Compared to traditional systems that continuously record video regardless of the actions in the frame, a smart VSS has the capability of capturing video data upon motion detection or object detection, and then extracts essential information and send to users. This increasing design complexity of the surveillance system, however, also introduces new security vulnerabilities. In this work, a smart, real-time frame duplication attack is investigated. We show the feasibility of forging the video streams in real-time as the camera's surroundings change. The generated frames are compared constantly and instantly to identify changes in the pixel values that could represent motion detection or changes in light intensities outdoors. An attacker (intruder) can remotely trigger the replay of some previously duplicated video streams manually or automatically, via a special quick response (QR) code or when the face of an intruder appears in the camera field of view. A detection technique is proposed by leveraging the real-time electrical network frequency (ENF) reference database to match with the power grid frequency. △ Less

Submitted 8 March, 2019; originally announced March 2019.

Comments: To Appear in the 2019 SPIE Defense + Commercial Sensing

BlendMAS: A BLockchain-ENabled Decentralized Microservices Architecture for Smart Public Safety

Authors: Ronghua Xu, Seyed Yahya Nikouei, Yu Chen, Erik Blasch, Alex Aved

Abstract: Thanks to rapid technological advances in the Internet of Things (IoT), a smart public safety (SPS) system has become feasible by integrating heterogeneous computing devices to collaboratively provide public protection services. While a service oriented architecture (SOA) has been adopted by IoT and cyber-physical systems (CPS), it is difficult for a monolithic architecture to provide scalable and… ▽ More Thanks to rapid technological advances in the Internet of Things (IoT), a smart public safety (SPS) system has become feasible by integrating heterogeneous computing devices to collaboratively provide public protection services. While a service oriented architecture (SOA) has been adopted by IoT and cyber-physical systems (CPS), it is difficult for a monolithic architecture to provide scalable and extensible services for a distributed IoT based SPS system. Furthermore, traditional security solutions rely on a centralized authority, which can be a performance bottleneck or single point failure. Inspired by microservices architecture and blockchain technology, this paper proposes a BLockchain-ENabled Decentralized Microservices Architecture for Smart public safety (BlendMAS). Within a permissioned blockchain network, a microservices based security mechanism is introduced to secure data access control in an SPS system. The functionality of security services are decoupled into separate containerized microservices that are built using a smart contract, and deployed on edge and fog computing nodes. An extensive experimental study verified that the proposed BlendMAS is able to offer a decentralized, scalable and secured data sharing and access control to distributed IoT based SPS system. △ Less

Submitted 27 February, 2019; originally announced February 2019.

Comments: Submitted to the 2019 IEEE International Conference on Blockchain (Blockchain-2019)

An Exploration of Blockchain Enabled Decentralized Capability based Access Control Strategy for Space Situation Awareness

Authors: Ronghua Xu, Yu Chen, Erik Blasch, Genshe Chen

Abstract: Space situation awareness (SSA) includes tracking of active and inactive resident space objects (RSOs) and assessing the space environment through sensor data collection and processing. To enhance SSA, the dynamic data-driven applications systems (DDDAS) framework couples on-line data with off-line models to enhance system performance. Using feedback control, sensor management, and communications… ▽ More Space situation awareness (SSA) includes tracking of active and inactive resident space objects (RSOs) and assessing the space environment through sensor data collection and processing. To enhance SSA, the dynamic data-driven applications systems (DDDAS) framework couples on-line data with off-line models to enhance system performance. Using feedback control, sensor management, and communications reliability. For information management, there is a need for identity authentication and access control to ensure the integrity of exchanged data as well as to grant authorized entities access right to data and services. Due to decentralization and heterogeneity of SSA systems, it is challenging to build an efficient centralized access control system, which could either be a performance bottleneck or the single point of failure. Inspired by the blockchain and smart contract technology, this paper introduces BlendCAC, a decentralized authentication and capability-based access control mechanism to enable effective protection for devices, services and information in SSA networks. To achieve secure identity authentication, the BlendCAC leverages the blockchain to create virtual trust zones and a robust identity-based capability token management strategy is proposed. A proof-of-concept prototype has been implemented on both resources-constrained devices and more powerful computing devices, and is tested on a private Ethereum blockchain network. The experimental results demonstrate the feasibility of the BlendCAC scheme to offer a decentralized, scalable, lightweight and fine-grained access control solution for space system towards SSA. △ Less

Submitted 24 December, 2018; v1 submitted 1 October, 2018; originally announced October 2018.

Comments: Submitted to SPIE Optical Engineering, Special Section on Sensors and Systems for Space Applications. arXiv admin note: substantial text overlap with arXiv:1804.09267

EIQIS: Toward an Event-Oriented Indexable and Queryable Intelligent Surveillance System

Authors: Seyed Yahya Nikouei, Yu Chen, Alexander Aved, Erik Blasch

Abstract: Edge computing provides the ability to link distributor users for multimedia content, while retaining the power of significant data storage and access at a centralized computer. Two requirements of significance include: what information show be processed at the edge and how the content should be stored. Answers to these questions require a combination of query-based search, access, and response as… ▽ More Edge computing provides the ability to link distributor users for multimedia content, while retaining the power of significant data storage and access at a centralized computer. Two requirements of significance include: what information show be processed at the edge and how the content should be stored. Answers to these questions require a combination of query-based search, access, and response as well as indexed-based processing, storage, and distribution. A measure of intelligence is not what is known, but is recalled, hence, future edge intelligence must provide recalled information for dynamic response. In this paper, a novel event-oriented indexable and queryable intelligent surveillance (EIQIS) system is introduced leveraging the on-site edge devices to collect the information sensed in format of frames and extracts useful features to enhance situation awareness. The design principles are discussed and a preliminary proof-of-concept prototype is built that validated the feasibility of the proposed idea. △ Less

Submitted 30 July, 2018; originally announced July 2018.

Real-Time Index Authentication for Event-Oriented Surveillance Video Query using Blockchain

Authors: Seyed Yahya Nikouei, Ronghua Xu, Deeraj Nagothu, Yu Chen, Alexander Aved, Erik Blasch

Abstract: Information from surveillance video is essential for situational awareness (SAW). Nowadays, a prohibitively large amount of surveillance data is being generated continuously by ubiquitously distributed video sensors. It is very challenging to immediately identify the objects of interest or zoom in suspicious actions from thousands of video frames. Making the big data indexable is critical to tackl… ▽ More Information from surveillance video is essential for situational awareness (SAW). Nowadays, a prohibitively large amount of surveillance data is being generated continuously by ubiquitously distributed video sensors. It is very challenging to immediately identify the objects of interest or zoom in suspicious actions from thousands of video frames. Making the big data indexable is critical to tackle this problem. It is ideal to generate pattern indexes in a real-time, on-site manner on the video streaming instead of depending on the batch processing at the cloud centers. The modern edge-fog-cloud computing paradigm allows implementation of time sensitive tasks at the edge of the network. The on-site edge devices collect the information sensed in format of frames and extracts useful features. The near-site fog nodes conduct the contextualization and classification of the features. The remote cloud center is in charge of more data intensive and computing intensive tasks. However, exchanging the index information among devices in different layers raises security concerns where an adversary can capture or tamper with features to mislead the surveillance system. In this paper, a blockchain enabled scheme is proposed to protect the index data through an encrypted secure channel between the edge and fog nodes. It reduces the chance of attacks on the small edge and fog devices. The feasibility of the proposal is validated through intensive experimental analysis. △ Less

Submitted 16 July, 2018; originally announced July 2018.

Comments: Submitted to the 1st International Workshop on BLockchain Enabled Sustainable Smart Cities (BLESS 2018), In Conjunction with the 4th IEEE Annual International Smart Cities Conference (ISC2 2018), Kansas City, MO, USA, Sept. 16-19, 2018

IR2VI: Enhanced Night Environmental Perception by Unsupervised Thermal Image Translation

Authors: Shuo Liu, Vijay John, Erik Blasch, Zheng Liu, Ying Huang

Abstract: Context enhancement is critical for night vision (NV) applications, especially for the dark night situation without any artificial lights. In this paper, we present the infrared-to-visual (IR2VI) algorithm, a novel unsupervised thermal-to-visible image translation framework based on generative adversarial networks (GANs). IR2VI is able to learn the intrinsic characteristics from VI images and inte… ▽ More Context enhancement is critical for night vision (NV) applications, especially for the dark night situation without any artificial lights. In this paper, we present the infrared-to-visual (IR2VI) algorithm, a novel unsupervised thermal-to-visible image translation framework based on generative adversarial networks (GANs). IR2VI is able to learn the intrinsic characteristics from VI images and integrate them into IR images. Since the existing unsupervised GAN-based image translation approaches face several challenges, such as incorrect mapping and lack of fine details, we propose a structure connection module and a region-of-interest (ROI) focal loss method to address the current limitations. Experimental results show the superiority of the IR2VI algorithm over baseline methods. △ Less

Submitted 25 June, 2018; originally announced June 2018.

Comments: Present at CVPR Workshops 2018

A Federated Capability-based Access Control Mechanism for Internet of Things (IoTs)

Authors: Ronghua Xu, Yu Chen, Erik Blasch, Genshe Chen

Abstract: The prevalence of Internet of Things (IoTs) allows heterogeneous embedded smart devices to collaboratively provide intelligent services with or without human intervention. While leveraging the large-scale IoT-based applications like Smart Gird and Smart Cities, IoT also incurs more concerns on privacy and security. Among the top security challenges that IoTs face is that access authorization is cr… ▽ More The prevalence of Internet of Things (IoTs) allows heterogeneous embedded smart devices to collaboratively provide intelligent services with or without human intervention. While leveraging the large-scale IoT-based applications like Smart Gird and Smart Cities, IoT also incurs more concerns on privacy and security. Among the top security challenges that IoTs face is that access authorization is critical in resource and information protection over IoTs. Traditional access control approaches, like Access Control Lists (ACL), Role-based Access Control (RBAC) and Attribute-based Access Control (ABAC), are not able to provide a scalable, manageable and efficient mechanisms to meet requirement of IoT systems. The extraordinary large number of nodes, heterogeneity as well as dynamicity, necessitate more fine-grained, lightweight mechanisms for IoT devices. In this paper, a federated capability-based access control (FedCAC) framework is proposed to enable an effective access control processes to devices, services and information in large scale IoT systems. An identity-based capability token management strategy is presented, which involves registering, propagation and revocation of the access authorization. Through delegating centralized authorization decision-making policy to local domain delegator, the access authorization process is locally conducted on the service provider that integrates situational awareness (SAW) and customized contextual conditions. Implemented and tested on both resources-constrained devices, like smart sensors and Raspberry PI, and non-resource-constrained devices, like laptops and smart phones, our experimental results demonstrate the feasibility of the proposed FedCAC approach to offer a scalable, lightweight and fine-grained access control solution to IoT systems connected to a system network. △ Less

Submitted 1 May, 2018; originally announced May 2018.

Comments: the SPIE Defense & Commercial Sensing 2018 (DCS), Conference on Sensors and Systems for Space Applications, Orlando, FL, USA, April 15, 2018. arXiv admin note: text overlap with arXiv:1804.09267

BlendCAC: A BLockchain-ENabled Decentralized Capability-based Access Control for IoTs

Authors: Ronghua Xu, Yu Chen, Erik Blasch, Genshe Chen

Abstract: The prevalence of Internet of Things (IoTs) allows heterogeneous embedded smart devices to collaboratively provide smart services with or without human intervention. While leveraging the large scale IoT based applications like Smart Gird or Smart Cities, IoTs also incur more concerns on privacy and security. Among the top security challenges that IoTs face, access authorization is critical in reso… ▽ More The prevalence of Internet of Things (IoTs) allows heterogeneous embedded smart devices to collaboratively provide smart services with or without human intervention. While leveraging the large scale IoT based applications like Smart Gird or Smart Cities, IoTs also incur more concerns on privacy and security. Among the top security challenges that IoTs face, access authorization is critical in resource sharing and information protection. One of the weaknesses in today's access control (AC) is the centralized authorization server, which can be the performance bottleneck or the single point of failure. In this paper, BlendCAC, a blockchain enabled decentralized capability based AC is proposed for the security of IoTs. The BlendCAC aims at an effective access control processes to devices, services and information in large scale IoT systems. Based on the blockchain network, a capability delegation mechanism is suggested for access permission propagation. A robust identity based capability token management strategy is proposed, which takes advantage of smart contract for registering, propagation and revocation of the access authorization. In the proposed BlendCAC scheme, IoT devices are their own master to control their resources instead of being supervised by a centralized authority. Implemented and tested on a Raspberry Pi device and on a local private blockchain network, our experimental results demonstrate the feasibility of the proposed BlendCAC approach to offer a decentralized, scalable, lightweight and fine grained AC solution to IoT systems. △ Less

Submitted 24 April, 2018; originally announced April 2018.

A Comparative Study of Object Trackers for Infrared Flying Bird Tracking

Authors: Ying Huang, Hong Zheng, Haibin Ling, Erik Blasch, Hao Yang

Abstract: Bird strikes present a huge risk for aircraft, especially since traditional airport bird surveillance is mainly dependent on inefficient human observation. Computer vision based technology has been proposed to automatically detect birds, determine bird flying trajectories, and predict aircraft takeoff delays. However, the characteristics of bird flight using imagery and the performance of existing… ▽ More Bird strikes present a huge risk for aircraft, especially since traditional airport bird surveillance is mainly dependent on inefficient human observation. Computer vision based technology has been proposed to automatically detect birds, determine bird flying trajectories, and predict aircraft takeoff delays. However, the characteristics of bird flight using imagery and the performance of existing methods applied to flying bird task are not well known. Therefore, we perform infrared flying bird tracking experiments using 12 state-of-the-art algorithms on a real BIRDSITE-IR dataset to obtain useful clues and recommend feature analysis. We also develop a Struck-scale method to demonstrate the effectiveness of multiple scale sampling adaption in handling the object of flying bird with varying shape and scale. The general analysis can be used to develop specialized bird tracking methods for airport safety, wildness and urban bird population studies. △ Less

Submitted 17 January, 2016; originally announced January 2016.