Showing 1–32 of 32 results for author: Yue, C

Enhanced Control for Diffusion Bridge in Image Restoration

Authors: Conghan Yue, Zhengwei Peng, Junlong Ma, Dongyu Zhang

Abstract: Image restoration refers to the process of restoring a damaged low-quality image back to its corresponding high-quality image. Typically, we use convolutional neural networks to directly learn the mapping from low-quality images to high-quality images achieving image restoration. Recently, a special type of diffusion bridge model has achieved more advanced results in image restoration. It can tran… ▽ More Image restoration refers to the process of restoring a damaged low-quality image back to its corresponding high-quality image. Typically, we use convolutional neural networks to directly learn the mapping from low-quality images to high-quality images achieving image restoration. Recently, a special type of diffusion bridge model has achieved more advanced results in image restoration. It can transform the direct mapping from low-quality to high-quality images into a diffusion process, restoring low-quality images through a reverse process. However, the current diffusion bridge restoration models do not emphasize the idea of conditional control, which may affect performance. This paper introduces the ECDB model enhancing the control of the diffusion bridge with low-quality images as conditions. Moreover, in response to the characteristic of diffusion models having low denoising level at larger values of \(\bm t \), we also propose a Conditional Fusion Schedule, which more effectively handles the conditional feature information of various modules. Experimental results prove that the ECDB model has achieved state-of-the-art results in many image restoration tasks, including deraining, inpainting and super-resolution. Code is avaliable at https://github.com/Hammour-steak/ECDB. △ Less

Submitted 29 August, 2024; originally announced August 2024.

Before and After Blockchain: Development and Principles of Distributed Fault-Tolerant Consensus

Authors: Huanyu Wu, Chentao Yue, Yixuan Fan, Yonghui Li, Lei Zhang

Abstract: The concept of distributed consensus gained widespread attention following the publication of "Byzantine Generals Problem" by Leslie Lamport in the 1980s. This research topic has been active and extensively studied over the last four decades, particularly since the advent of blockchain technology in 2009. Blockchain technology employs Proof-of-X (PoX) or Byzantine-fault-tolerant (BFT) systems, whe… ▽ More The concept of distributed consensus gained widespread attention following the publication of "Byzantine Generals Problem" by Leslie Lamport in the 1980s. This research topic has been active and extensively studied over the last four decades, particularly since the advent of blockchain technology in 2009. Blockchain technology employs Proof-of-X (PoX) or Byzantine-fault-tolerant (BFT) systems, where all participants follow a protocol to achieve a common state (i.e., consistency) eventually. However, because PoX consensus such as Proof-of-Work is is resource-intensive with high power consumption, most permissioned blockchains employ BFT to achieve consistency. In this article, we provide an introduction to the fundamental principles and history of distributed consensus. We then explore the well-known fault-tolerant state machine replication (SMR) in partially synchronous networks, as well as consensus protocols in asynchronous models and recently proposed DAG-based consensus. Additionally, we examine the relationship between BFT consensus and blockchain technology and discuss the following questions: What is the history and evolution of BFT? Why are BFT protocols designed in the way they are and what core components do they use? What is the connection between BFT and blockchain technology, and what are the driving needs for future BFT research? △ Less

Submitted 3 August, 2024; v1 submitted 29 July, 2024; originally announced July 2024.

Learning Interpretable Differentiable Logic Networks

Authors: Chang Yue, Niraj K. Jha

Abstract: The ubiquity of neural networks (NNs) in real-world applications, from healthcare to natural language processing, underscores their immense utility in capturing complex relationships within high-dimensional data. However, NNs come with notable disadvantages, such as their "black-box" nature, which hampers interpretability, as well as their tendency to overfit the training data. We introduce a nove… ▽ More The ubiquity of neural networks (NNs) in real-world applications, from healthcare to natural language processing, underscores their immense utility in capturing complex relationships within high-dimensional data. However, NNs come with notable disadvantages, such as their "black-box" nature, which hampers interpretability, as well as their tendency to overfit the training data. We introduce a novel method for learning interpretable differentiable logic networks (DLNs) that are architectures that employ multiple layers of binary logic operators. We train these networks by softening and differentiating their discrete components, e.g., through binarization of inputs, binary logic operations, and connections between neurons. This approach enables the use of gradient-based learning methods. Experimental results on twenty classification tasks indicate that differentiable logic networks can achieve accuracies comparable to or exceeding that of traditional NNs. Equally importantly, these networks offer the advantage of interpretability. Moreover, their relatively simple structure results in the number of logic gate-level operations during inference being up to a thousand times smaller than NNs, making them suitable for deployment on edge devices. △ Less

Submitted 4 July, 2024; originally announced July 2024.

Model-Enhanced LLM-Driven VUI Testing of VPA Apps

Authors: Suwan Li, Lei Bu, Guangdong Bai, Fuman Xie, Kai Chen, Chang Yue

Abstract: The flourishing ecosystem centered around voice personal assistants (VPA), such as Amazon Alexa, has led to the booming of VPA apps. The largest app market Amazon skills store, for example, hosts over 200,000 apps. Despite their popularity, the open nature of app release and the easy accessibility of apps also raise significant concerns regarding security, privacy and quality. Consequently, variou… ▽ More The flourishing ecosystem centered around voice personal assistants (VPA), such as Amazon Alexa, has led to the booming of VPA apps. The largest app market Amazon skills store, for example, hosts over 200,000 apps. Despite their popularity, the open nature of app release and the easy accessibility of apps also raise significant concerns regarding security, privacy and quality. Consequently, various testing approaches have been proposed to systematically examine VPA app behaviors. To tackle the inherent lack of a visible user interface in the VPA app, two strategies are employed during testing, i.e., chatbot-style testing and model-based testing. The former often lacks effective guidance for expanding its search space, while the latter falls short in interpreting the semantics of conversations to construct precise and comprehensive behavior models for apps. In this work, we introduce Elevate, a model-enhanced large language model (LLM)-driven VUI testing framework. Elevate leverages LLMs' strong capability in natural language processing to compensate for semantic information loss during model-based VUI testing. It operates by prompting LLMs to extract states from VPA apps' outputs and generate context-related inputs. During the automatic interactions with the app, it incrementally constructs the behavior model, which facilitates the LLM in generating inputs that are highly likely to discover new states. Elevate bridges the LLM and the behavior model with innovative techniques such as encoding behavior model into prompts and selecting LLM-generated inputs based on the context relevance. Elevate is benchmarked on 4,000 real-world Alexa skills, against the state-of-the-art tester Vitas. It achieves 15% higher state space coverage compared to Vitas on all types of apps, and exhibits significant advancement in efficiency. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Comments: 13 pages, 11 figures

NeurDB: An AI-powered Autonomous Data System

Authors: Beng Chin Ooi, Shaofeng Cai, Gang Chen, Yanyan Shen, Kian-Lee Tan, Yuncheng Wu, Xiaokui Xiao, Naili Xing, Cong Yue, Lingze Zeng, Meihui Zhang, Zhanhao Zhao

Abstract: In the wake of rapid advancements in artificial intelligence (AI), we stand on the brink of a transformative leap in data systems. The imminent fusion of AI and DB (AIxDB) promises a new generation of data systems, which will relieve the burden on end-users across all industry sectors by featuring AI-enhanced functionalities, such as personalized and automated in-database AI-powered analytics, sel… ▽ More In the wake of rapid advancements in artificial intelligence (AI), we stand on the brink of a transformative leap in data systems. The imminent fusion of AI and DB (AIxDB) promises a new generation of data systems, which will relieve the burden on end-users across all industry sectors by featuring AI-enhanced functionalities, such as personalized and automated in-database AI-powered analytics, self-driving capabilities for improved system performance, etc. In this paper, we explore the evolution of data systems with a focus on deepening the fusion of AI and DB. We present NeurDB, an AI-powered autonomous data system designed to fully embrace AI design in each major system component and provide in-database AI-powered analytics. We outline the conceptual and architectural overview of NeurDB, discuss its design choices and key components, and report its current development and future plan. △ Less

Submitted 4 July, 2024; v1 submitted 6 May, 2024; originally announced May 2024.

The Guesswork of Ordered Statistics Decoding: Complexity and Practical Design

Authors: Chentao Yue, Changyang She, Branka Vucetic, Yonghui Li

Abstract: This paper investigates guesswork over ordered statistics and formulates the complexity of ordered statistics decoding (OSD) in binary additive white Gaussian noise (AWGN) channels. It first develops a new upper bound of guesswork for independent sequences, by applying the Holder's inequity to Hamming shell-based subspaces. This upper bound is then extended to the ordered statistics, by constructi… ▽ More This paper investigates guesswork over ordered statistics and formulates the complexity of ordered statistics decoding (OSD) in binary additive white Gaussian noise (AWGN) channels. It first develops a new upper bound of guesswork for independent sequences, by applying the Holder's inequity to Hamming shell-based subspaces. This upper bound is then extended to the ordered statistics, by constructing the conditionally independent sequences within the ordered statistics sequences. We leverage the established bounds to formulate the best achievable decoding complexity of OSD that ensures no loss in error performance, where OSD stops immediately when the correct codeword estimate is found. We show that the average complexity of OSD at maximum decoding order can be accurately approximated by the modified Bessel function, which increases near-exponentially with code dimension. We also identify a complexity saturation threshold, where increasing the OSD decoding order beyond this threshold improves error performance without further raising decoding complexity. Finally, the paper presents insights on applying these findings to enhance the efficiency of practical decoder implementations. △ Less

Submitted 27 March, 2024; originally announced March 2024.

Comments: Submitted for peer review;19 pages;15 figures

Image Restoration Through Generalized Ornstein-Uhlenbeck Bridge

Authors: Conghan Yue, Zhengwei Peng, Junlong Ma, Shiyan Du, Pengxu Wei, Dongyu Zhang

Abstract: Diffusion models exhibit powerful generative capabilities enabling noise mapping to data via reverse stochastic differential equations. However, in image restoration, the focus is on the mapping relationship from low-quality to high-quality images. Regarding this issue, we introduce the Generalized Ornstein-Uhlenbeck Bridge (GOUB) model. By leveraging the natural mean-reverting property of the gen… ▽ More Diffusion models exhibit powerful generative capabilities enabling noise mapping to data via reverse stochastic differential equations. However, in image restoration, the focus is on the mapping relationship from low-quality to high-quality images. Regarding this issue, we introduce the Generalized Ornstein-Uhlenbeck Bridge (GOUB) model. By leveraging the natural mean-reverting property of the generalized OU process and further eliminating the variance of its steady-state distribution through the Doob's h-transform, we achieve diffusion mappings from point to point enabling the recovery of high-quality images from low-quality ones. Moreover, we unravel the fundamental mathematical essence shared by various bridge models, all of which are special instances of GOUB and empirically demonstrate the optimality of our proposed models. Additionally, we present the corresponding Mean-ODE model adept at capturing both pixel-level details and structural perceptions. Experimental outcomes showcase the state-of-the-art performance achieved by both models across diverse tasks, including inpainting, deraining, and super-resolution. Code is available at \url{https://github.com/Hammour-steak/GOUB}. △ Less

Submitted 17 May, 2024; v1 submitted 15 December, 2023; originally announced December 2023.

Comments: ICML 2024

When Distributed Consensus Meets Wireless Connected Autonomous Systems: A Review and A DAG-based Approach

Authors: Huanyu Wu, Chentao Yue, Lei Zhang, Yonghui Li, Muhammad Ali Imran

Abstract: The connected and autonomous systems (CAS) and auto-driving era is coming into our life. To support CAS applications such as AI-driven decision-making and blockchain-based smart data management platform, data and message exchange/dissemination is a fundamental element. The distributed message broadcast and forward protocols in CAS, such as vehicular ad hoc networks (VANET), can suffer from signifi… ▽ More The connected and autonomous systems (CAS) and auto-driving era is coming into our life. To support CAS applications such as AI-driven decision-making and blockchain-based smart data management platform, data and message exchange/dissemination is a fundamental element. The distributed message broadcast and forward protocols in CAS, such as vehicular ad hoc networks (VANET), can suffer from significant message loss and uncertain transmission delay, and faulty nodes might disseminate fake messages to confuse the network. Therefore, the consensus mechanism is essential in CAS with distributed structure to guaranteed correct nodes agree on the same parameter and reach consistency. However, due to the wireless nature of CAS, traditional consensus cannot be directly deployed. This article reviews several existing consensus mechanisms, including average/maximum/minimum estimation consensus mechanisms that apply on quantity, Byzantine fault tolerance consensus for request, state machine replication (SMR) and blockchain, as well as their implementations in CAS. To deploy wireless-adapted consensus, we propose a Directed Acyclic Graph (DAG)-based message structure to build a non-equivocation data dissemination protocol for CAS, which has resilience against message loss and unpredictable forwarding latency. Finally, we enhance this protocol by developing a two-dimension DAG-based strategy to achieve partial order for blockchain and total order for the distributed service model SMR. △ Less

Submitted 13 November, 2023; originally announced November 2023.

Modular Multi-Level Replanning TAMP Framework for Dynamic Environment

Authors: Tao Lin, Chengfei Yue, Ziran Liu, Xibin Cao

Abstract: Task and Motion Planning (TAMP) algorithms can generate plans that combine logic and motion aspects for robots. However, these plans are sensitive to interference and control errors. To make TAMP more applicable in real-world, we propose the modular multi-level replanning TAMP framework(MMRF), blending the probabilistic completeness of sampling-based TAMP algorithm with the robustness of reactive… ▽ More Task and Motion Planning (TAMP) algorithms can generate plans that combine logic and motion aspects for robots. However, these plans are sensitive to interference and control errors. To make TAMP more applicable in real-world, we propose the modular multi-level replanning TAMP framework(MMRF), blending the probabilistic completeness of sampling-based TAMP algorithm with the robustness of reactive replanning. MMRF generates an nominal plan from the initial state, then dynamically reconstructs this nominal plan in real-time, reorders robot manipulations. Following the logic-level adjustment, GMRF will try to replan a new motion path to ensure the updated plan is feasible at the motion level. Finally, we conducted real-world experiments involving stack and rearrange task domains. The result demonstrate MMRF's ability to swiftly complete tasks in scenarios with varying degrees of interference. △ Less

Submitted 10 March, 2024; v1 submitted 23 October, 2023; originally announced October 2023.

Efficient Near Maximum-Likelihood Reliability-Based Decoding for Short LDPC Codes

Authors: Weiyang Zhang, Chentao Yue, Yonghui Li, Branka Vucetic

Abstract: In this paper, we propose an efficient decoding algorithm for short low-density parity check (LDPC) codes by carefully combining the belief propagation (BP) decoding and order statistic decoding (OSD) algorithms. Specifically, a modified BP (mBP) algorithm is applied for a certain number of iterations prior to OSD to enhance the reliability of the received message, where an offset parameter is uti… ▽ More In this paper, we propose an efficient decoding algorithm for short low-density parity check (LDPC) codes by carefully combining the belief propagation (BP) decoding and order statistic decoding (OSD) algorithms. Specifically, a modified BP (mBP) algorithm is applied for a certain number of iterations prior to OSD to enhance the reliability of the received message, where an offset parameter is utilized in mBP to control the weight of the extrinsic information in message passing. By carefully selecting the offset parameter and the number of mBP iterations, the number of errors in the most reliable positions (MRPs) in OSD can be reduced by mBP, thereby significantly improving the overall decoding performance of error rate and complexity. Simulation results show that the proposed algorithm can approach the maximum-likelihood decoding (MLD) for short LDPC codes with only a slight increase in complexity compared to BP and a significant decrease compared to OSD. Specifically, the order-(m-1) decoding of the proposed algorithm can achieve the performance of the order-m OSD. △ Less

Submitted 1 September, 2023; v1 submitted 1 June, 2023; originally announced June 2023.

Enabling and Analyzing How to Efficiently Extract Information from Hybrid Long Documents with LLMs

Authors: Chongjian Yue, Xinrun Xu, Xiaojun Ma, Lun Du, Hengyu Liu, Zhiming Ding, Yanbing Jiang, Shi Han, Dongmei Zhang

Abstract: Large Language Models (LLMs) demonstrate exceptional performance in textual understanding and tabular reasoning tasks. However, their ability to comprehend and analyze hybrid text, containing textual and tabular data, remains underexplored. In this research, we specialize in harnessing the potential of LLMs to comprehend critical information from financial reports, which are hybrid long-documents.… ▽ More Large Language Models (LLMs) demonstrate exceptional performance in textual understanding and tabular reasoning tasks. However, their ability to comprehend and analyze hybrid text, containing textual and tabular data, remains underexplored. In this research, we specialize in harnessing the potential of LLMs to comprehend critical information from financial reports, which are hybrid long-documents. We propose an Automated Financial Information Extraction (AFIE) framework that enhances LLMs' ability to comprehend and extract information from financial reports. To evaluate AFIE, we develop a Financial Reports Numerical Extraction (FINE) dataset and conduct an extensive experimental analysis. Our framework is effectively validated on GPT-3.5 and GPT-4, yielding average accuracy increases of 53.94% and 33.77%, respectively, compared to a naive method. These results suggest that the AFIE framework offers accuracy for automated numerical extraction from complex, hybrid documents. △ Less

Submitted 7 March, 2024; v1 submitted 24 May, 2023; originally announced May 2023.

Prompt Engineering for Healthcare: Methodologies and Applications

Authors: Jiaqi Wang, Enze Shi, Sigang Yu, Zihao Wu, Chong Ma, Haixing Dai, Qiushi Yang, Yanqing Kang, Jinru Wu, Huawen Hu, Chenxi Yue, Haiyang Zhang, Yiheng Liu, Yi Pan, Zhengliang Liu, Lichao Sun, Xiang Li, Bao Ge, Xi Jiang, Dajiang Zhu, Yixuan Yuan, Dinggang Shen, Tianming Liu, Shu Zhang

Abstract: Prompt engineering is a critical technique in the field of natural language processing that involves designing and optimizing the prompts used to input information into models, aiming to enhance their performance on specific tasks. With the recent advancements in large language models, prompt engineering has shown significant superiority across various domains and has become increasingly important… ▽ More Prompt engineering is a critical technique in the field of natural language processing that involves designing and optimizing the prompts used to input information into models, aiming to enhance their performance on specific tasks. With the recent advancements in large language models, prompt engineering has shown significant superiority across various domains and has become increasingly important in the healthcare domain. However, there is a lack of comprehensive reviews specifically focusing on prompt engineering in the medical field. This review will introduce the latest advances in prompt engineering in the field of natural language processing for the medical field. First, we will provide the development of prompt engineering and emphasize its significant contributions to healthcare natural language processing applications such as question-answering systems, text summarization, and machine translation. With the continuous improvement of general large language models, the importance of prompt engineering in the healthcare domain is becoming increasingly prominent. The aim of this article is to provide useful resources and bridges for healthcare natural language processing researchers to better explore the application of prompt engineering in this field. We hope that this review can provide new ideas and inspire for research and application in medical natural language processing. △ Less

Submitted 23 March, 2024; v1 submitted 28 April, 2023; originally announced April 2023.

Cross-Dimensional Refined Learning for Real-Time 3D Visual Perception from Monocular Video

Authors: Ziyang Hong, C. Patrick Yue

Abstract: We present a novel real-time capable learning method that jointly perceives a 3D scene's geometry structure and semantic labels. Recent approaches to real-time 3D scene reconstruction mostly adopt a volumetric scheme, where a Truncated Signed Distance Function (TSDF) is directly regressed. However, these volumetric approaches tend to focus on the global coherence of their reconstructions, which le… ▽ More We present a novel real-time capable learning method that jointly perceives a 3D scene's geometry structure and semantic labels. Recent approaches to real-time 3D scene reconstruction mostly adopt a volumetric scheme, where a Truncated Signed Distance Function (TSDF) is directly regressed. However, these volumetric approaches tend to focus on the global coherence of their reconstructions, which leads to a lack of local geometric detail. To overcome this issue, we propose to leverage the latent geometric prior knowledge in 2D image features by explicit depth prediction and anchored feature generation, to refine the occupancy learning in TSDF volume. Besides, we find that this cross-dimensional feature refinement methodology can also be adopted for the semantic segmentation task by utilizing semantic priors. Hence, we proposed an end-to-end cross-dimensional refinement neural network (CDRNet) to extract both 3D mesh and 3D semantic labeling in real time. The experiment results show that this method achieves a state-of-the-art 3D perception efficiency on multiple datasets, which indicates the great potential of our method for industrial applications. △ Less

Submitted 10 September, 2023; v1 submitted 16 March, 2023; originally announced March 2023.

Comments: Accpeted to ICCV 2023 Workshops. Project page: https://hafred.github.io/cdrnet/

A Scalable Graph Neural Network Decoder for Short Block Codes

Authors: Kou Tian, Chentao Yue, Changyang She, Yonghui Li, Branka Vucetic

Abstract: In this work, we propose a novel decoding algorithm for short block codes based on an edge-weighted graph neural network (EW-GNN). The EW-GNN decoder operates on the Tanner graph with an iterative message-passing structure, which algorithmically aligns with the conventional belief propagation (BP) decoding method. In each iteration, the "weight" on the message passed along each edge is obtained fr… ▽ More In this work, we propose a novel decoding algorithm for short block codes based on an edge-weighted graph neural network (EW-GNN). The EW-GNN decoder operates on the Tanner graph with an iterative message-passing structure, which algorithmically aligns with the conventional belief propagation (BP) decoding method. In each iteration, the "weight" on the message passed along each edge is obtained from a fully connected neural network that has the reliability information from nodes/edges as its input. Compared to existing deep-learning-based decoding schemes, the EW-GNN decoder is characterised by its scalability, meaning that 1) the number of trainable parameters is independent of the codeword length, and 2) an EW-GNN decoder trained with shorter/simple codes can be directly used for longer/sophisticated codes of different code rates. Furthermore, simulation results show that the EW-GNN decoder outperforms the BP and deep-learning-based BP methods from the literature in terms of the decoding error rate. △ Less

Submitted 13 November, 2022; originally announced November 2022.

Comments: Submitted to IEEE conference for possible publication

FR: Folded Rationalization with a Unified Encoder

Authors: Wei Liu, Haozhao Wang, Jun Wang, Ruixuan Li, Chao Yue, Yuankai Zhang

Abstract: Conventional works generally employ a two-phase model in which a generator selects the most important pieces, followed by a predictor that makes predictions based on the selected pieces. However, such a two-phase model may incur the degeneration problem where the predictor overfits to the noise generated by a not yet well-trained generator and in turn, leads the generator to converge to a sub-opti… ▽ More Conventional works generally employ a two-phase model in which a generator selects the most important pieces, followed by a predictor that makes predictions based on the selected pieces. However, such a two-phase model may incur the degeneration problem where the predictor overfits to the noise generated by a not yet well-trained generator and in turn, leads the generator to converge to a sub-optimal model that tends to select senseless pieces. To tackle this challenge, we propose Folded Rationalization (FR) that folds the two phases of the rationale model into one from the perspective of text semantic extraction. The key idea of FR is to employ a unified encoder between the generator and predictor, based on which FR can facilitate a better predictor by access to valuable information blocked by the generator in the traditional two-phase model and thus bring a better generator. Empirically, we show that FR improves the F1 score by up to 10.3% as compared to state-of-the-art methods. △ Less

Submitted 20 September, 2022; v1 submitted 17 September, 2022; originally announced September 2022.

Comments: Accepted at NeurIPS 2022

SSL-WM: A Black-Box Watermarking Approach for Encoders Pre-trained by Self-supervised Learning

Authors: Peizhuo Lv, Pan Li, Shenchen Zhu, Shengzhi Zhang, Kai Chen, Ruigang Liang, Chang Yue, Fan Xiang, Yuling Cai, Hualong Ma, Yingjun Zhang, Guozhu Meng

Abstract: Recent years have witnessed tremendous success in Self-Supervised Learning (SSL), which has been widely utilized to facilitate various downstream tasks in Computer Vision (CV) and Natural Language Processing (NLP) domains. However, attackers may steal such SSL models and commercialize them for profit, making it crucial to verify the ownership of the SSL models. Most existing ownership protection s… ▽ More Recent years have witnessed tremendous success in Self-Supervised Learning (SSL), which has been widely utilized to facilitate various downstream tasks in Computer Vision (CV) and Natural Language Processing (NLP) domains. However, attackers may steal such SSL models and commercialize them for profit, making it crucial to verify the ownership of the SSL models. Most existing ownership protection solutions (e.g., backdoor-based watermarks) are designed for supervised learning models and cannot be used directly since they require that the models' downstream tasks and target labels be known and available during watermark embedding, which is not always possible in the domain of SSL. To address such a problem, especially when downstream tasks are diverse and unknown during watermark embedding, we propose a novel black-box watermarking solution, named SSL-WM, for verifying the ownership of SSL models. SSL-WM maps watermarked inputs of the protected encoders into an invariant representation space, which causes any downstream classifier to produce expected behavior, thus allowing the detection of embedded watermarks. We evaluate SSL-WM on numerous tasks, such as CV and NLP, using different SSL models both contrastive-based and generative-based. Experimental results demonstrate that SSL-WM can effectively verify the ownership of stolen SSL models in various downstream tasks. Furthermore, SSL-WM is robust against model fine-tuning, pruning, and input preprocessing attacks. Lastly, SSL-WM can also evade detection from evaluated watermark detection approaches, demonstrating its promising application in protecting the ownership of SSL models. △ Less

Submitted 29 January, 2024; v1 submitted 8 September, 2022; originally announced September 2022.

Comments: To Appear in the Network and Distributed System Security (NDSS) Symposium 2024, 26 February - 1 March 2024, San Diego, CA, USA

CTRL: Clustering Training Losses for Label Error Detection

Authors: Chang Yue, Niraj K. Jha

Abstract: In supervised machine learning, use of correct labels is extremely important to ensure high accuracy. Unfortunately, most datasets contain corrupted labels. Machine learning models trained on such datasets do not generalize well. Thus, detecting their label errors can significantly increase their efficacy. We propose a novel framework, called CTRL (Clustering TRaining Losses for label error detect… ▽ More In supervised machine learning, use of correct labels is extremely important to ensure high accuracy. Unfortunately, most datasets contain corrupted labels. Machine learning models trained on such datasets do not generalize well. Thus, detecting their label errors can significantly increase their efficacy. We propose a novel framework, called CTRL (Clustering TRaining Losses for label error detection), to detect label errors in multi-class datasets. It detects label errors in two steps based on the observation that models learn clean and noisy labels in different ways. First, we train a neural network using the noisy training dataset and obtain the loss curve for each sample. Then, we apply clustering algorithms to the training losses to group samples into two categories: cleanly-labeled and noisily-labeled. After label error detection, we remove samples with noisy labels and retrain the model. Our experimental results demonstrate state-of-the-art error detection accuracy on both image (CIFAR-10 and CIFAR-100) and tabular datasets under simulated noise. We also use a theoretical analysis to provide insights into why CTRL performs so well. △ Less

Submitted 12 September, 2023; v1 submitted 17 August, 2022; originally announced August 2022.

Invisible Backdoor Attacks Using Data Poisoning in the Frequency Domain

Authors: Chang Yue, Peizhuo Lv, Ruigang Liang, Kai Chen

Abstract: With the broad application of deep neural networks (DNNs), backdoor attacks have gradually attracted attention. Backdoor attacks are insidious, and poisoned models perform well on benign samples and are only triggered when given specific inputs, which cause the neural network to produce incorrect outputs. The state-of-the-art backdoor attack work is implemented by data poisoning, i.e., the attacke… ▽ More With the broad application of deep neural networks (DNNs), backdoor attacks have gradually attracted attention. Backdoor attacks are insidious, and poisoned models perform well on benign samples and are only triggered when given specific inputs, which cause the neural network to produce incorrect outputs. The state-of-the-art backdoor attack work is implemented by data poisoning, i.e., the attacker injects poisoned samples into the dataset, and the models trained with that dataset are infected with the backdoor. However, most of the triggers used in the current study are fixed patterns patched on a small fraction of an image and are often clearly mislabeled, which is easily detected by humans or defense methods such as Neural Cleanse and SentiNet. Also, it's difficult to be learned by DNNs without mislabeling, as they may ignore small patterns. In this paper, we propose a generalized backdoor attack method based on the frequency domain, which can implement backdoor implantation without mislabeling and accessing the training process. It is invisible to human beings and able to evade the commonly used defense methods. We evaluate our approach in the no-label and clean-label cases on three datasets (CIFAR-10, STL-10, and GTSRB) with two popular scenarios (self-supervised learning and supervised learning). The results show our approach can achieve a high attack success rate (above 90%) on all the tasks without significant performance degradation on main tasks. Also, we evaluate the bypass performance of our approach for different kinds of defenses, including the detection of training data (i.e., Activation Clustering), the preprocessing of inputs (i.e., Filtering), the detection of inputs (i.e., SentiNet), and the detection of models (i.e., Neural Cleanse). The experimental results demonstrate that our approach shows excellent robustness to such defenses. △ Less

Submitted 9 July, 2022; originally announced July 2022.

GlassDB: An Efficient Verifiable Ledger Database System Through Transparency

Authors: Cong Yue, Tien Tuan Anh Dinh, Zhongle Xie, Meihui Zhang, Gang Chen, Beng Chin Ooi, Xiaokui Xiao

Abstract: Verifiable ledger databases protect data history against malicious tampering. Existing systems, such as blockchains and certificate transparency, are based on transparency logs -- a simple abstraction allowing users to verify that a log maintained by an untrusted server is append-only. They expose a simple key-value interface. Building a practical database from transparency logs, on the other hand… ▽ More Verifiable ledger databases protect data history against malicious tampering. Existing systems, such as blockchains and certificate transparency, are based on transparency logs -- a simple abstraction allowing users to verify that a log maintained by an untrusted server is append-only. They expose a simple key-value interface. Building a practical database from transparency logs, on the other hand, remains a challenge. In this paper, we explore the design space of verifiable ledger databases along three dimensions: abstraction, threat model, and performance. We survey existing systems and identify their two limitations, namely, the lack of transaction support and the inferior efficiency. We then present GlassDB, a distributed database that addresses these limitations under a practical threat model. GlassDB inherits the verifiability of transparency logs, but supports transactions and offers high performance. It extends a ledger-like key-value store with a data structure for efficient proofs, and adds a concurrency control mechanism for transactions. GlassDB batches independent operations from concurrent transactions when updating the core data structures. In addition, we design a new benchmark for evaluating verifiable ledger databases, by extending YCSB and TPC-C benchmarks. Using this benchmark, we compare GlassDB against four baselines: reimplemented versions of three verifiable databases, and a verifiable map backed by a transparency log. Experimental results demonstrate that GlassDB is an efficient, transactional, and verifiable ledger database. △ Less

Submitted 19 February, 2023; v1 submitted 2 July, 2022; originally announced July 2022.

Ordered-Statistics Decoding with Adaptive Gaussian Elimination Reduction for Short Codes

Authors: Chentao Yue, Mahyar Shirvanimoghaddam, Branka Vucetic, Yonghui Li

Abstract: In this paper, we propose an efficient ordered-statistics decoding (OSD) algorithm with an adaptive Gaussian elimination (GE) reduction technique. The proposed decoder utilizes two decoding conditions to adaptively remove GE in OSD. The first condition determines whether GE could be skipped in the OSD process by estimating the decoding error probability. Then, the second condition is utilized to i… ▽ More In this paper, we propose an efficient ordered-statistics decoding (OSD) algorithm with an adaptive Gaussian elimination (GE) reduction technique. The proposed decoder utilizes two decoding conditions to adaptively remove GE in OSD. The first condition determines whether GE could be skipped in the OSD process by estimating the decoding error probability. Then, the second condition is utilized to identify the correct decoding result during the decoding process without GE. The proposed decoder can break the ``complexity floor'' in OSD decoders introduced by the GE overhead. Simulation results advise that when compared with the latest schemes in the literature, the proposed approach can significantly reduce the decoding complexity at high SNRs without any degradation in the error-correction capability. △ Less

Submitted 22 December, 2022; v1 submitted 22 June, 2022; originally announced June 2022.

Comments: 5 figs, 6 pages

Efficient Decoders for Short Block Length Codes in 6G URLLC

Authors: Chentao Yue, Vera Miloslavskaya, Mahyar Shirvanimoghaddam, Branka Vucetic, Yonghui Li

Abstract: This paper reviews the potential channel decoding techniques for ultra-reliable low-latency communications (URLLC). URLLC is renowned for its stringent requirements including ultra-reliability, low end-to-end transmission latency, and packet-size flexibility. These requirements exacerbate the difficulty of the physical-layer design, particularly for the channel coding and decoding schemes. To sati… ▽ More This paper reviews the potential channel decoding techniques for ultra-reliable low-latency communications (URLLC). URLLC is renowned for its stringent requirements including ultra-reliability, low end-to-end transmission latency, and packet-size flexibility. These requirements exacerbate the difficulty of the physical-layer design, particularly for the channel coding and decoding schemes. To satisfy the requirements of URLLC, decoders must exhibit superior error-rate performance \black{and} low decoding complexity. \black{Also, it is desired that decoders be universal} to accommodate various coding schemes. This paper provides a comprehensive review and comparison of different candidate decoding techniques for URLLC in terms of their error-rate performance and computational complexity for structured and random short codes. We further make recommendations of the decoder selections and suggest several potential research directions. △ Less

Submitted 22 December, 2022; v1 submitted 20 June, 2022; originally announced June 2022.

Comments: To appear in IEEE Communications Magazine

HTGN-BTW: Heterogeneous Temporal Graph Network with Bi-Time-Window Training Strategy for Temporal Link Prediction

Authors: Chongjian Yue, Lun Du, Qiang Fu, Wendong Bi, Hengyu Liu, Yu Gu, Di Yao

Abstract: With the development of temporal networks such as E-commerce networks and social networks, the issue of temporal link prediction has attracted increasing attention in recent years. The Temporal Link Prediction task of WSDM Cup 2022 expects a single model that can work well on two kinds of temporal graphs simultaneously, which have quite different characteristics and data properties, to predict whe… ▽ More With the development of temporal networks such as E-commerce networks and social networks, the issue of temporal link prediction has attracted increasing attention in recent years. The Temporal Link Prediction task of WSDM Cup 2022 expects a single model that can work well on two kinds of temporal graphs simultaneously, which have quite different characteristics and data properties, to predict whether a link of a given type will occur between two given nodes within a given time span. Our team, named as nothing here, regards this task as a link prediction task in heterogeneous temporal networks and proposes a generic model, i.e., Heterogeneous Temporal Graph Network (HTGN), to solve such temporal link prediction task with the unfixed time intervals and the diverse link types. That is, HTGN can adapt to the heterogeneity of links and the prediction with unfixed time intervals within an arbitrary given time period. To train the model, we design a Bi-Time-Window training strategy (BTW) which has two kinds of mini-batches from two kinds of time windows. As a result, for the final test, we achieved an AUC of 0.662482 on dataset A, an AUC of 0.906923 on dataset B, and won 2nd place with an Average T-scores of 0.628942. △ Less

Submitted 25 February, 2022; originally announced February 2022.

Comments: 5 pages, Second Winner Award at Temporal Link Prediction task of WSDM Cup 2022

Density Evolution Analysis of the Iterative Joint Ordered-Statistics Decoding for NOMA

Authors: Chentao Yue, Mahyar Shirvanimoghaddam, Alva Kosasih, Giyoon Park, Ok-Sun Park, Wibowo Hardjawana, Branka Vucetic, Yonghui Li

Abstract: In this paper, we develop a density evolution (DE) framework for analyzing the iterative joint decoding (JD) for non-orthogonal multiple access (NOMA) systems, where the ordered-statistics decoding (OSD) is applied to decode short block codes. We first investigate the density-transform feature of the soft-output OSD (SOSD), by deriving the density of the extrinsic log-likelihood ratio (LLR) with k… ▽ More In this paper, we develop a density evolution (DE) framework for analyzing the iterative joint decoding (JD) for non-orthogonal multiple access (NOMA) systems, where the ordered-statistics decoding (OSD) is applied to decode short block codes. We first investigate the density-transform feature of the soft-output OSD (SOSD), by deriving the density of the extrinsic log-likelihood ratio (LLR) with known densities of the priori LLR. Then, we represent the OSD-based JD by bipartite graphs (BGs), and develop the DE framework by characterizing the density-transform features of nodes over the BG. Numerical examples show that the proposed DE framework accurately tracks the evolution of LLRs during the iterative decoding, especially at moderate-to-high SNRs. Based on the DE framework, we further analyze the BER performance of the OSD-based JD, and the convergence points of the two-user and equal-power systems. △ Less

Submitted 23 December, 2021; originally announced December 2021.

Comments: 30 Pages, 12 Figures

NOMA Joint Decoding based on Soft-Output Ordered-Statistics Decoder for Short Block Codes

Authors: Chentao Yue, Alva Kosasih, Mahyar Shirvanimoghaddam, Giyoon Park, Ok-Sun Park, Wibowo Hardjawana, Branka Vucetic, Yonghui Li

Abstract: In this paper, we design the joint decoding (JD) of non-orthogonal multiple access (NOMA) systems employing short block length codes. We first proposed a low-complexity soft-output ordered-statistics decoding (LC-SOSD) based on a decoding stopping condition, derived from approximations of the a-posterior probabilities of codeword estimates. Simulation results show that LC-SOSD has the similar mutu… ▽ More In this paper, we design the joint decoding (JD) of non-orthogonal multiple access (NOMA) systems employing short block length codes. We first proposed a low-complexity soft-output ordered-statistics decoding (LC-SOSD) based on a decoding stopping condition, derived from approximations of the a-posterior probabilities of codeword estimates. Simulation results show that LC-SOSD has the similar mutual information transform property to the original SOSD with a significantly reduced complexity. Then, based on the analysis, an efficient JD receiver which combines the parallel interference cancellation (PIC) and the proposed LC-SOSD is developed for NOMA systems. Two novel techniques, namely decoding switch (DS) and decoding combiner (DC), are introduced to accelerate the convergence speed. Simulation results show that the proposed receiver can achieve a lower bit-error rate (BER) compared to the successive interference cancellation (SIC) decoding over the additive-white-Gaussian-noise (AWGN) and fading channel, with a lower complexity in terms of the number of decoding iterations. △ Less

Submitted 28 October, 2021; originally announced October 2021.

Comments: 6 pages; 5 figures

Linear-Equation Ordered-Statistics Decoding

Authors: Chentao Yue, Mahyar Shirvanimoghaddam, Giyoon Park, Ok-Sun Park, Branka Vucetic, Yonghui Li

Abstract: In this paper, we propose a new linear-equation ordered-statistics decoding (LE-OSD). Unlike the OSD, LE-OSD uses high reliable parity bits rather than information bits to recover the codeword estimates, which is equivalent to solving a system of linear equations (SLE). Only test error patterns (TEPs) that create feasible SLEs, referred to as the valid TEPs, are used to obtain different codeword e… ▽ More In this paper, we propose a new linear-equation ordered-statistics decoding (LE-OSD). Unlike the OSD, LE-OSD uses high reliable parity bits rather than information bits to recover the codeword estimates, which is equivalent to solving a system of linear equations (SLE). Only test error patterns (TEPs) that create feasible SLEs, referred to as the valid TEPs, are used to obtain different codeword estimates. We introduce several constraints on the Hamming weight of TEPs to limit the overall decoding complexity. Furthermore, we analyze the block error rate (BLER) and the computational complexity of the proposed approach. It is shown that LE-OSD has a similar performance as OSD in terms of BLER, which can asymptotically approach Maximum-likelihood (ML) performance with proper parameter selections. Simulation results demonstrate that the LE-OSD has a significantly reduced complexity compared to OSD, especially for low-rate codes, that usually require high decoding order in OSD. Nevertheless, the complexity reduction can also be observed for high-rate codes. In addition, we further improve LE-OSD by applying the decoding stopping condition and the TEP discarding condition. As shown by simulations, the improved LE-OSD has a considerably reduced complexity while maintaining the BLER performance, compared to the latest OSD approach from literature. △ Less

Submitted 21 October, 2021; originally announced October 2021.

Comments: 32 Pages, 5 figures

Efficient Training Convolutional Neural Networks on Edge Devices with Gradient-pruned Sign-symmetric Feedback Alignment

Authors: Ziyang Hong, C. Patrick Yue

Abstract: With the prosperity of mobile devices, the distributed learning approach enabling model training with decentralized data has attracted wide research. However, the lack of training capability for edge devices significantly limits the energy efficiency of distributed learning in real life. This paper describes a novel approach of training DNNs exploiting the redundancy and the weight asymmetry poten… ▽ More With the prosperity of mobile devices, the distributed learning approach enabling model training with decentralized data has attracted wide research. However, the lack of training capability for edge devices significantly limits the energy efficiency of distributed learning in real life. This paper describes a novel approach of training DNNs exploiting the redundancy and the weight asymmetry potential of conventional backpropagation. We demonstrate that with negligible classification accuracy loss, the proposed approach outperforms the prior arts by 5x in terms of energy efficiency. △ Less

Submitted 12 May, 2021; v1 submitted 4 March, 2021; originally announced March 2021.

Comments: This work is published in the Proceedings of the 9th International Conference on IT Convergence and Security (ICITCS2021)

Spitz: A Verifiable Database System

Authors: Meihui Zhang, Zhongle Xie, Cong Yue, Ziyue Zhong

Abstract: Databases in the past have helped businesses maintain and extract insights from their data. Today, it is common for a business to involve multiple independent, distrustful parties. This trend towards decentralization introduces a new and important requirement to databases: the integrity of the data, the history, and the execution must be protected. In other words, there is a need for a new class o… ▽ More Databases in the past have helped businesses maintain and extract insights from their data. Today, it is common for a business to involve multiple independent, distrustful parties. This trend towards decentralization introduces a new and important requirement to databases: the integrity of the data, the history, and the execution must be protected. In other words, there is a need for a new class of database systems whose integrity can be verified (or verifiable databases). In this paper, we identify the requirements and the design challenges of verifiable databases.We observe that the main challenges come from the need to balance data immutability, tamper evidence, and performance. We first consider approaches that extend existing OLTP and OLAP systems with support for verification. We next examine a clean-slate approach, by describing a new system, Spitz, specifically designed for efficiently supporting immutable and tamper-evident transaction management. We conduct a preliminary performance study of both approaches against a baseline system, and provide insights on their performance. △ Less

Submitted 20 August, 2020; originally announced August 2020.

CLUE: A Chinese Language Understanding Evaluation Benchmark

Authors: Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu , et al. (7 additional authors not shown)

Abstract: The advent of natural language understanding (NLU) benchmarks for English, such as GLUE and SuperGLUE allows new NLU models to be evaluated across a diverse set of tasks. These comprehensive benchmarks have facilitated a broad range of research and applications in natural language processing (NLP). The problem, however, is that most such benchmarks are limited to English, which has made it difficu… ▽ More The advent of natural language understanding (NLU) benchmarks for English, such as GLUE and SuperGLUE allows new NLU models to be evaluated across a diverse set of tasks. These comprehensive benchmarks have facilitated a broad range of research and applications in natural language processing (NLP). The problem, however, is that most such benchmarks are limited to English, which has made it difficult to replicate many of the successes in English NLU for other languages. To help remedy this issue, we introduce the first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark. CLUE is an open-ended, community-driven project that brings together 9 tasks spanning several well-established single-sentence/sentence-pair classification tasks, as well as machine reading comprehension, all on original Chinese text. To establish results on these tasks, we report scores using an exhaustive set of current state-of-the-art pre-trained Chinese models (9 in total). We also introduce a number of supplementary datasets and additional tools to help facilitate further progress on Chinese NLU. Our benchmark is released at https://www.CLUEbenchmarks.com △ Less

Submitted 5 November, 2020; v1 submitted 13 April, 2020; originally announced April 2020.

Comments: Accepted by COLING2020; 10 pages, 4 figures

A Revisit to Ordered Statistics Decoding: Distance Distribution and Decoding Rules

Authors: Chentao Yue, Mahyar Shirvanimoghaddam, Branka Vucetic, Yonghui Li

Abstract: This paper revisits the ordered statistics decoding (OSD). It provides a comprehensive analysis of the OSD algorithm by characterizing the statistical properties, evolution and the distribution of the Hamming distance and weighted Hamming distance from codeword estimates to the received sequence in the reprocessing stages of the OSD algorithm. We prove that the Hamming distance and weighted Hammin… ▽ More This paper revisits the ordered statistics decoding (OSD). It provides a comprehensive analysis of the OSD algorithm by characterizing the statistical properties, evolution and the distribution of the Hamming distance and weighted Hamming distance from codeword estimates to the received sequence in the reprocessing stages of the OSD algorithm. We prove that the Hamming distance and weighted Hamming distance distributions can be characterized as mixture models capturing the decoding error probability and code weight enumerator. Simulation and numerical results show that our proposed statistical approaches can accurately describe the distance distributions. Based on these distributions and with the aim to reduce the decoding complexity, several techniques, including stopping rules and discarding rules, are proposed, and their decoding error performance and complexity are accordingly analyzed. Simulation results for decoding various eBCH codes demonstrate that the proposed techniques can significantly reduce the decoding complexity with a negligible loss in the decoding error performance. △ Less

Submitted 6 May, 2021; v1 submitted 10 April, 2020; originally announced April 2020.

Comments: accepted by IEEE Transactions on Information Theory

Analysis of Indexing Structures for Immutable Data

Authors: Cong Yue, Zhongle Xie, Meihui Zhang, Gang Chen, Beng Chin Ooi, Sheng Wang, Xiaokui Xiao

Abstract: In emerging applications such as blockchains and collaborative data analytics, there are strong demands for data immutability, multi-version accesses, and tamper-evident controls. This leads to three new index structures for immutable data, namely Merkle Patricia Trie (MPT), Merkle Bucket Tree (MBT), and Pattern-Oriented-Split Tree (POS-Tree). Although these structures have been adopted in real ap… ▽ More In emerging applications such as blockchains and collaborative data analytics, there are strong demands for data immutability, multi-version accesses, and tamper-evident controls. This leads to three new index structures for immutable data, namely Merkle Patricia Trie (MPT), Merkle Bucket Tree (MBT), and Pattern-Oriented-Split Tree (POS-Tree). Although these structures have been adopted in real applications, there is no systematic evaluation of their pros and cons in the literature. This makes it difficult for practitioners to choose the right index structure for their applications, as there is only a limited understanding of the characteristics of each index. To alleviate the above deficiency, we present a comprehensive analysis of the existing index structures for immutable data, evaluating both their asymptotic and empirical performance. Specifically, we show that MPT, MBT, and POS-Tree are all instances of a recently proposed framework, dubbed \my{Structurally Invariant and Reusable Indexes (SIRI)}. We propose to evaluate the SIRI instances based on five essential metrics: their efficiency for four index operations (i.e., lookup, update, comparison, and merge), as well as their \my{deduplication ratios} (i.e., the size of the index with deduplication over the size without deduplication). We establish the worst-case guarantees of each index in terms of these five metrics, and we experimentally evaluate all indexes in a large variety of settings. Based on our theoretical and empirical analysis, we conclude that POS-Tree is a favorable choice for indexing immutable data. △ Less

Submitted 10 March, 2020; v1 submitted 4 March, 2020; originally announced March 2020.

Segmentation-Discarding Ordered-Statistic Decoding for Linear Block Codes

Authors: Chentao Yue, Mahyar Shirvanimoghaddam, Yonghui Li, Branka Vucetic

Abstract: In this paper, we propose an efficient reliability based segmentation-discarding decoding (SDD) algorithm for short block-length codes. A novel segmentation-discarding technique is proposed along with the stopping rule to significantly reduce the decoding complexity without a significant performance degradation compared to ordered statistics decoding (OSD). In the proposed decoder, the list of tes… ▽ More In this paper, we propose an efficient reliability based segmentation-discarding decoding (SDD) algorithm for short block-length codes. A novel segmentation-discarding technique is proposed along with the stopping rule to significantly reduce the decoding complexity without a significant performance degradation compared to ordered statistics decoding (OSD). In the proposed decoder, the list of test error patterns (TEPs) is divided into several segments according to carefully selected boundaries and every segment is checked separately during the reprocessing stage. Decoding is performed under the constraint of the discarding rule and stopping rule. Simulations results for different codes show that our proposed algorithm can significantly reduce the decoding complexity compared to the existing OSD algorithms in literature. △ Less

Submitted 21 January, 2019; v1 submitted 8 January, 2019; originally announced January 2019.

Comments: This paper has been submitted to IEEE ICC 2019

Short Block-length Codes for Ultra-Reliable Low-Latency Communications

Authors: Mahyar Shirvanimoghaddam, Mohamad Sadegh Mohamadi, Rana Abbas, Aleksandar Minja, Chentao Yue, Balazs Matuz, Guojun Han, Zihuai Lin, Yonghui Li, Sarah Johnson, Branka Vucetic

Abstract: This paper reviews the state of the art channel coding techniques for ultra-reliable low latency communication (URLLC). The stringent requirements of URLLC services, such as ultra-high reliability and low latency, have made it the most challenging feature of the fifth generation (5G) mobile systems. The problem is even more challenging for the services beyond the 5G promise, such as tele-surgery a… ▽ More This paper reviews the state of the art channel coding techniques for ultra-reliable low latency communication (URLLC). The stringent requirements of URLLC services, such as ultra-high reliability and low latency, have made it the most challenging feature of the fifth generation (5G) mobile systems. The problem is even more challenging for the services beyond the 5G promise, such as tele-surgery and factory automation, which require latencies less than 1ms and failure rate as low as $10^{-9}$. The very low latency requirements of URLLC do not allow traditional approaches such as re-transmission to be used to increase the reliability. On the other hand, to guarantee the delay requirements, the block length needs to be small, so conventional channel codes, originally designed and optimised for moderate-to-long block-lengths, show notable deficiencies for short blocks. This paper provides an overview on channel coding techniques for short block lengths and compares them in terms of performance and complexity. Several important research directions are identified and discussed in more detail with several possible solutions. △ Less

Submitted 5 September, 2018; v1 submitted 26 February, 2018; originally announced February 2018.

Comments: Accepted for publication in IEEE Communications Magazine