Computer Science
See recent articles
- [1] arXiv:2408.10207 [pdf, html, other]
-
Title: A Comprehensive Survey on Diffusion Models and Their ApplicationsSubjects: Computer Vision and Pattern Recognition (cs.CV)
Diffusion Models are probabilistic models that create realistic samples by simulating the diffusion process, gradually adding and removing noise from data. These models have gained popularity in domains such as image processing, speech synthesis, and natural language processing due to their ability to produce high-quality samples. As Diffusion Models are being adopted in various domains, existing literature reviews that often focus on specific areas like computer vision or medical imaging may not serve a broader audience across multiple fields. Therefore, this review presents a comprehensive overview of Diffusion Models, covering their theoretical foundations and algorithmic innovations. We highlight their applications in diverse areas such as media quality, authenticity, synthesis, image transformation, healthcare, and more. By consolidating current knowledge and identifying emerging trends, this review aims to facilitate a deeper understanding and broader adoption of Diffusion Models and provide guidelines for future researchers and practitioners across diverse disciplines.
- [2] arXiv:2408.10208 [pdf, html, other]
-
Title: Nonreflecting Boundary Condition for the free Schr\"{o}dinger equation for hyperrectangular computational domainsComments: 27 pages, 9 figures, 5 tables. arXiv admin note: substantial text overlap with arXiv:2403.07787Subjects: Numerical Analysis (math.NA); Computational Physics (physics.comp-ph)
In this article, we discuss the efficient ways of implementing the transparent boundary condition (TBC) and its various approximations for the free Schrödinger equation on a hyperrectangular computational domain in $\field{R}^d$ with periodic boundary conditions along the $(d-1)$ unbounded directions. In particular, we consider Padé approximant based rational approximation of the exact TBC and a spatially local form of the exact TBC obtained under its high-frequency approximation. For the spatial discretization, we use a Legendre-Galerkin spectral method with a boundary-adapted basis to ensure the bandedness of the resulting linear system. Temporal discretization is then addressed with two one-step methods, namely, the backward-differentiation formula of order 1 (BDF1) and the trapezoidal rule (TR). Finally, several numerical tests are presented to demonstrate the effectiveness of the methods where we study the stability and convergence behaviour empirically.
- [3] arXiv:2408.10210 [pdf, html, other]
-
Title: A Survey on Symbolic Knowledge Distillation of Large Language ModelsComments: 21 pages, 7 figuresSubjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
This survey paper delves into the emerging and critical area of symbolic knowledge distillation in Large Language Models (LLMs). As LLMs like Generative Pre-trained Transformer-3 (GPT-3) and Bidirectional Encoder Representations from Transformers (BERT) continue to expand in scale and complexity, the challenge of effectively harnessing their extensive knowledge becomes paramount. This survey concentrates on the process of distilling the intricate, often implicit knowledge contained within these models into a more symbolic, explicit form. This transformation is crucial for enhancing the interpretability, efficiency, and applicability of LLMs. We categorize the existing research based on methodologies and applications, focusing on how symbolic knowledge distillation can be used to improve the transparency and functionality of smaller, more efficient Artificial Intelligence (AI) models. The survey discusses the core challenges, including maintaining the depth of knowledge in a comprehensible format, and explores the various approaches and techniques that have been developed in this field. We identify gaps in current research and potential opportunities for future advancements. This survey aims to provide a comprehensive overview of symbolic knowledge distillation in LLMs, spotlighting its significance in the progression towards more accessible and efficient AI systems.
- [4] arXiv:2408.10211 [pdf, other]
-
Title: G\"odel Incompleteness Theorem for PAC Learnable TheoryComments: 24 pagesSubjects: Logic in Computer Science (cs.LO); Logic (math.LO)
Different from the view that information is objective reality, we adopts the idea that all information needs to be compiled by the interpreter before it can be observed. Then, the interpreter-based information theory with its complexity is given by formula. Through the relationship between complexity and PAC learnability, the general PAC learnability corresponding to general complexity is derived. Then we generalize the observation process to the formal system with functors, in which we give concrete proof of the generalized Gödel incompleteness theorem. Finally, the Cantor set, the original Gödel incompleteness theorem, the Turing halting problem and the EPR paradox are put into the same mathematical framework as the specific applications of the general Gödel incompleteness theorem.
- [5] arXiv:2408.10212 [pdf, html, other]
-
Title: Uniform Haar Wavelet Solutions for Fractional Regular $\beta$-Singular BVPs Modeling Human Head Heat Conduction under Febrifuge EffectsSubjects: Numerical Analysis (math.NA)
This paper introduces nonlinear fractional Lane-Emden equations of the form, $$ D^{\alpha} y(x) + \frac{\lambda}{x^\beta}~ D^{\beta} y(x) + f(y) =0, ~ ~1 < \alpha \leq 2, ~~ 0< \beta \leq 1, ~~ 0 < x < 1,$$ subject to boundary conditions, $$ y'(0) =\mathbf{a} , ~~~ \mathbf{c}~ y'(1) + \mathbf{d}~ y(1) = \mathbf{b},$$ where, $D^\alpha, D^\beta$ represent Caputo fractional derivative, $\mathbf{a, b,c,d} \in \mathbb{R}$, $ \lambda = 1, 2$, and $f(y)$ is non linear function of $y.$ We have developed collocation method namely, uniform fractional Haar wavelet collocation method and used it to compute solutions. The proposed method combines the quasilinearization method with the Haar wavelet collocation method. In this approach, fractional Haar integrations is used to determine the linear system, which, upon solving, produces the required solution. Our findings suggest that as the values of $(\alpha, \beta)$ approach $(2,1),$ the solutions of the fractional and classical Lane-Emden become identical.
- [6] arXiv:2408.10213 [pdf, html, other]
-
Title: Mono-cluster flocking and uniform-in-time stability of the discrete Motsch-Tadmor modelSubjects: Numerical Analysis (math.NA)
The Motsch-Tadmor (MT) model is a variant of the Cucker-Smale model with a normalized communication weight function. The normalization poses technical challenges in analyzing the collective behavior due to the absence of conservation of momentum. We study three quantitative estimates for the discrete-time MT model considering the first-order Euler discretization. First, we provide a sufficient framework leading to the asymptotic mono-cluster flocking. The proposed framework is given in terms of coupling strength, communication weight function, and initial data. Second, we show that the continuous transition from the discrete MT model to the continuous MT model can be made uniformly in time using the finite-time convergence result and asymptotic flocking estimate. Third, we present uniform-in-time stability estimates for the discrete MT model. We also provide several numerical examples and compare them with analytical results.
- [7] arXiv:2408.10214 [pdf, html, other]
-
Title: A Memory Reduction Compact Gas Kinetic Scheme on 3D Unstructured MeshesComments: arXiv admin note: substantial text overlap with arXiv:2402.02075Subjects: Numerical Analysis (math.NA)
This paper introduces a memory-reduction third-order compact gas-kinetic scheme (CGKS) for solving compressible Euler and Navier-Stokes equations on 3D unstructured meshes. The scheme utilizes a time-evolution gas distribution function to provide a time-evolution solution at cell interfaces, enabling the implementation of Hermite WENO techniques for high-order reconstruction. However, the HWENO method needs to store a coefficients matrix for the quadratic polynomial to achieve third-order accuracy, resulting in high memory usage. A novel reconstruction method, built upon HWENO reconstruction, has been designed to enhance computational efficiency and reduce memory usage compared to the original CGKS. The simple idea is that the first-order and second-order terms of the quadratic polynomials are determined in a two-step way. In the first step, the second-order terms are obtained from the reconstruction of a linear polynomial of the first-order derivatives by only using the cell-averaged slopes, since the second-order derivatives are nothing but the "derivatives of derivatives". Subsequently, the first-order terms left can be determined by the linear reconstruction only using cell-averaged values. Thus, we successfully split one quadratic least-square regression into several linear least-square regressions, which are commonly used in a second-order finite volume code. Since only a small matrix inversion is needed in a 3-D linear least-square regression, the computational cost for the new reconstruction is dramatically reduced and the storage of the reconstruction-coefficient matrix is no longer necessary. The proposed new reconstruction technique can reduce the overall computational cost by about 20 to 30 percent. The challenging large-scale unsteady numerical simulation is performed, which demonstrates that the current improvement brings the CGKS to a new level for industrial applications.
- [8] arXiv:2408.10215 [pdf, html, other]
-
Title: Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning ApplicationsComments: 22 pages, 25 figures, we are waiting for decision from IEEE AccessSubjects: Machine Learning (cs.LG); Systems and Control (eess.SY)
The aim of Reinforcement Learning (RL) in real-world applications is to create systems capable of making autonomous decisions by learning from their environment through trial and error. This paper emphasizes the importance of reward engineering and reward shaping in enhancing the efficiency and effectiveness of reinforcement learning algorithms. Reward engineering involves designing reward functions that accurately reflect the desired outcomes, while reward shaping provides additional feedback to guide the learning process, accelerating convergence to optimal policies. Despite significant advancements in reinforcement learning, several limitations persist. One key challenge is the sparse and delayed nature of rewards in many real-world scenarios, which can hinder learning progress. Additionally, the complexity of accurately modeling real-world environments and the computational demands of reinforcement learning algorithms remain substantial obstacles. On the other hand, recent advancements in deep learning and neural networks have significantly improved the capability of reinforcement learning systems to handle high-dimensional state and action spaces, enabling their application to complex tasks such as robotics, autonomous driving, and game playing. This paper provides a comprehensive review of the current state of reinforcement learning, focusing on the methodologies and techniques used in reward engineering and reward shaping. It critically analyzes the limitations and recent advancements in the field, offering insights into future research directions and potential applications in various domains.
- [9] arXiv:2408.10217 [pdf, html, other]
-
Title: Inference of Heterogeneous Material Properties via Infinite-Dimensional Integrated DICSubjects: Numerical Analysis (math.NA); Optimization and Control (math.OC)
We present a scalable and efficient framework for the inference of spatially-varying parameters of continuum materials from image observations of their deformations. Our goal is the nondestructive identification of arbitrary damage, defects, anomalies and inclusions without knowledge of their morphology or strength. Since these effects cannot be directly observed, we pose their identification as an inverse problem. Our approach builds on integrated digital image correlation (IDIC, Besnard Hild, Roux, 2006), which poses the image registration and material inference as a monolithic inverse problem, thereby enforcing physical consistency of the image registration using the governing PDE. Existing work on IDIC has focused on low-dimensional parameterizations of materials. In order to accommodate the inference of heterogeneous material propertes that are formally infinite dimensional, we present $\infty$-IDIC, a general formulation of the PDE-constrained coupled image registration and inversion posed directly in the function space setting. This leads to several mathematical and algorithmic challenges arising from the ill-posedness and high dimensionality of the inverse problem. To address ill-posedness, we consider various regularization schemes, namely $H^1$ and total variation for the inference of smooth and sharp features, respectively. To address the computational costs associated with the discretized problem, we use an efficient inexact-Newton CG framework for solving the regularized inverse problem. In numerical experiments, we demonstrate the ability of $\infty$-IDIC to characterize complex, spatially varying Lamé parameter fields of linear elastic and hyperelastic materials. Our method exhibits (i) the ability to recover fine-scale and sharp material features, (ii) mesh-independent convergence performance and hyperparameter selection, (iii) robustness to observational noise.
- [10] arXiv:2408.10223 [pdf, other]
-
Title: High order compact fully-discrete scheme for hyperbolic conversation lawsSubjects: Numerical Analysis (math.NA)
Based on the solution formula method, a series of one-step fully-discrete schemes, such as FWENO/Full-WENO has been proposed. Storing the by-products conservative variables at the half points (grid center) and using them as interpolation information to construct high-order schemes, we obtain a new class of one-step compact fully-discrete schemes. The new scheme can be associate with various non-oscillatory strategies. This paper takes state-of-the-art WENO-JS method as an example and proposes a family of compact fully-discrete WENO scheme. Detailed analysis is conducted on accuracy, errors, computational cost, efficiency and their connection with Hermite interpolation. Meanwhile, we design a new entropy flux linearization strategy for Euler equations to enhance its robustness, and also develop a multi-dimensional method for this compact fully-discrete framework. Due to the new scheme is one-step and utilizes stored by-products information for interpolation, it has a significant advantage in efficiency. For one-dimensional Euler equations, compared to the original FWENO, the computational cost only increases by 20-40%, while is approximately one-third of WENO+RK3. For two-dimensional case, a new special dimension-by-dimension strategy is applied. Although there is an additional computing cost, numerical experiments show that the new scheme only needs about 1/10 to 1/13 cost of that for WENO+RK3 when obtaining similar or even better resolution results, indicating that the new scheme is more efficient than semi-discrete schemes based on RK methods.
- [11] arXiv:2408.10226 [pdf, html, other]
-
Title: A nonconforming P3 and discontinuous P2 mixed finite element on tetrahedral gridsSubjects: Numerical Analysis (math.NA)
A nonconforming $P_3$ finite element is constructed by enriching the conforming $P_3$ finite element space with three $P_3$ nonconforming bubbles and six additional $P_4$ nonconforming bubbles, on each tetrahedron. Here the divergence of the $P_4$ bubble is not a $P_3$ polynomial, but a $P_2$ polynomial. This nonconforming $P_3$ finite element, combined with the discontinuous $P_2$ finite element, is inf-sup stable for solving the Stokes equations on general tetrahedral grids. Consequently such a mixed finite element method produces quasi-optimal solutions for solving the stationary Stokes equations. With these special $P_4$ bubbles, the discrete velocity remains locally pointwise divergence-free. Numerical tests confirm the theory.
- [12] arXiv:2408.10227 [pdf, html, other]
-
Title: A nonconforming P2 and discontinuous P1 mixed finite element on tetrahedral gridsSubjects: Numerical Analysis (math.NA)
A nonconforming $P_2$ finite element is constructed by enriching the conforming $P_2$ finite element space with seven $P_2$ nonconforming bubble functions (out of fifteen such bubble functions on each tetrahedron). This spacial nonconforming $P_2$ finite element, combined with the discontinuous $P_1$ finite element on general tetrahedral grids, is inf-sup stable for solving the Stokes equations. Consequently such a mixed finite element method produces optimal-order convergen solutions for solving the stationary Stokes equations. Numerical tests confirm the theory.
- [13] arXiv:2408.10229 [pdf, other]
-
Title: AI Transparency in Academic Search Systems: An Initial ExplorationSubjects: Computers and Society (cs.CY); Information Retrieval (cs.IR)
As AI-enhanced academic search systems become increasingly popular among researchers, investigating their AI transparency is crucial to ensure trust in the search outcomes, as well as the reliability and integrity of scholarly work. This study employs a qualitative content analysis approach to examine the websites of a sample of 10 AI-enhanced academic search systems identified through university library guides. The assessed level of transparency varies across these systems: five provide detailed information about their mechanisms, three offer partial information, and two provide little to no information. These findings indicate that the academic community is recommending and using tools with opaque functionalities, raising concerns about research integrity, including issues of reproducibility and researcher responsibility.
- [14] arXiv:2408.10230 [pdf, html, other]
-
Title: A General-Purpose Device for Interaction with LLMsSubjects: Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC); Robotics (cs.RO)
This paper investigates integrating large language models (LLMs) with advanced hardware, focusing on developing a general-purpose device designed for enhanced interaction with LLMs. Initially, we analyze the current landscape, where virtual assistants and LLMs are reshaping human-technology interactions, highlighting pivotal advancements and setting the stage for a new era of intelligent hardware. Despite substantial progress in LLM technology, a significant gap exists in hardware development, particularly concerning scalability, efficiency, affordability, and multimodal capabilities. This disparity presents both challenges and opportunities, underscoring the need for hardware that is not only powerful but also versatile and capable of managing the sophisticated demands of modern computation. Our proposed device addresses these needs by emphasizing scalability, multimodal data processing, enhanced user interaction, and privacy considerations, offering a comprehensive platform for LLM integration in various applications.
- [15] arXiv:2408.10231 [pdf, html, other]
-
Title: Achieving Faster and More Accurate Operation of Deep Predictive LearningComments: 2 pages, 2 figuresSubjects: Robotics (cs.RO)
Achieving both high speed and precision in robot operations is a significant challenge for social implementation. While factory robots excel at predefined tasks, they struggle with environment-specific actions like cleaning and cooking. Deep learning research aims to address this by enabling robots to autonomously execute behaviors through end-to-end learning with sensor data. RT-1 and ACT are notable examples that have expanded robots' capabilities. However, issues with model inference speed and hand position accuracy persist. High-quality training data and fast, stable inference mechanisms are essential to overcome these challenges. This paper proposes a motion generation model for high-speed, high-precision tasks, exemplified by the sports stacking task. By teaching motions slowly and inferring at high speeds, the model achieved a 94% success rate in stacking cups with a real robot.
- [16] arXiv:2408.10233 [pdf, html, other]
-
Title: FPCA: Field-Programmable Pixel Convolutional Array for Extreme-Edge IntelligenceSubjects: Hardware Architecture (cs.AR); Image and Video Processing (eess.IV)
The rapid advancement of neural network applications necessitates hardware that not only accelerates computation but also adapts efficiently to dynamic processing requirements. While processing-in-pixel has emerged as a promising solution to overcome the bottlenecks of traditional architectures at the extreme-edge, existing implementations face limitations in reconfigurability and scalability due to their static nature and inefficient area usage. Addressing these challenges, we present a novel architecture that significantly enhances the capabilities of processing-in-pixel for convolutional neural networks. Our design innovatively integrates non-volatile memory (NVM) with novel unit pixel circuit design, enabling dynamic reconfiguration of synaptic weights, kernel size, channel size and stride size. Thus offering unprecedented flexibility and adaptability. With using a separate die for pixel circuit and storing synaptic weights, our circuit achieves a substantial reduction in the required area per pixel thereby increasing the density and scalability of the pixel array. Simulation results demonstrate dot product operations of the circuit, the non-linearity of its analog output and a novel bucket-select curvefit model is proposed to capture it. This work not only addresses the limitations of current in-pixel computing approaches but also opens new avenues for developing more efficient, flexible, and scalable neural network hardware, paving the way for advanced AI applications.
- [17] arXiv:2408.10239 [pdf, html, other]
-
Title: A Conceptual Framework for Ethical Evaluation of Machine Learning SystemsSubjects: Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Software Engineering (cs.SE)
Research in Responsible AI has developed a range of principles and practices to ensure that machine learning systems are used in a manner that is ethical and aligned with human values. However, a critical yet often neglected aspect of ethical ML is the ethical implications that appear when designing evaluations of ML systems. For instance, teams may have to balance a trade-off between highly informative tests to ensure downstream product safety, with potential fairness harms inherent to the implemented testing procedures. We conceptualize ethics-related concerns in standard ML evaluation techniques. Specifically, we present a utility framework, characterizing the key trade-off in ethical evaluation as balancing information gain against potential ethical harms. The framework is then a tool for characterizing challenges teams face, and systematically disentangling competing considerations that teams seek to balance. Differentiating between different types of issues encountered in evaluation allows us to highlight best practices from analogous domains, such as clinical trials and automotive crash testing, which navigate these issues in ways that can offer inspiration to improve evaluation processes in ML. Our analysis underscores the critical need for development teams to deliberately assess and manage ethical complexities that arise during the evaluation of ML systems, and for the industry to move towards designing institutional policies to support ethical evaluations.
- [18] arXiv:2408.10240 [pdf, html, other]
-
Title: AltCanvas: A Tile-Based Image Editor with Generative AI for Blind or Visually Impaired PeopleSubjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
People with visual impairments often struggle to create content that relies heavily on visual elements, particularly when conveying spatial and structural information. Existing accessible drawing tools, which construct images line by line, are suitable for simple tasks like math but not for more expressive artwork. On the other hand, emerging generative AI-based text-to-image tools can produce expressive illustrations from descriptions in natural language, but they lack precise control over image composition and properties. To address this gap, our work integrates generative AI with a constructive approach that provides users with enhanced control and editing capabilities. Our system, AltCanvas, features a tile-based interface enabling users to construct visual scenes incrementally, with each tile representing an object within the scene. Users can add, edit, move, and arrange objects while receiving speech and audio feedback. Once completed, the scene can be rendered as a color illustration or as a vector for tactile graphic generation. Involving 14 blind or low-vision users in design and evaluation, we found that participants effectively used the AltCanvas workflow to create illustrations.
- [19] arXiv:2408.10243 [pdf, html, other]
-
Title: TrIM: Triangular Input Movement Systolic Array for Convolutional Neural Networks -- Part II: Architecture and Hardware ImplementationComments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibleSubjects: Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
Modern hardware architectures for Convolutional Neural Networks (CNNs), other than targeting high performance, aim at dissipating limited energy. Reducing the data movement cost between the computing cores and the memory is a way to mitigate the energy consumption. Systolic arrays are suitable architectures to achieve this objective: they use multiple processing elements that communicate each other to maximize data utilization, based on proper dataflows like the weight stationary and row stationary. Motivated by this, we have proposed TrIM, an innovative dataflow based on a triangular movement of inputs, and capable to reduce the number of memory accesses by one order of magnitude when compared to state-of-the-art systolic arrays. In this paper, we present a TrIM-based hardware architecture for CNNs. As a showcase, the accelerator is implemented onto a Field Programmable Gate Array (FPGA) to execute the VGG-16 CNN. The architecture achieves a peak throughput of 453.6 Giga Operations per Second, outperforming a state-of-the-art row stationary systolic array by ~5.1x in terms of memory accesses, and being up to ~12.2x more energy-efficient than other FPGA accelerators.
- [20] arXiv:2408.10246 [pdf, other]
-
Title: VyAnG-Net: A Novel Multi-Modal Sarcasm Recognition Model by Uncovering Visual, Acoustic and Glossary FeaturesSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)
Various linguistic and non-linguistic clues, such as excessive emphasis on a word, a shift in the tone of voice, or an awkward expression, frequently convey sarcasm. The computer vision problem of sarcasm recognition in conversation aims to identify hidden sarcastic, criticizing, and metaphorical information embedded in everyday dialogue. Prior, sarcasm recognition has focused mainly on text. Still, it is critical to consider all textual information, audio stream, facial expression, and body position for reliable sarcasm identification. Hence, we propose a novel approach that combines a lightweight depth attention module with a self-regulated ConvNet to concentrate on the most crucial features of visual data and an attentional tokenizer based strategy to extract the most critical context-specific information from the textual data. The following is a list of the key contributions that our experimentation has made in response to performing the task of Multi-modal Sarcasm Recognition: an attentional tokenizer branch to get beneficial features from the glossary content provided by the subtitles; a visual branch for acquiring the most prominent features from the video frames; an utterance-level feature extraction from acoustic content and a multi-headed attention based feature fusion branch to blend features obtained from multiple modalities. Extensive testing on one of the benchmark video datasets, MUSTaRD, yielded an accuracy of 79.86% for speaker dependent and 76.94% for speaker independent configuration demonstrating that our approach is superior to the existing methods. We have also conducted a cross-dataset analysis to test the adaptability of VyAnG-Net with unseen samples of another dataset MUStARD++.
- [21] arXiv:2408.10248 [pdf, other]
-
Title: Target-Dependent Multimodal Sentiment Analysis Via Employing Visual-to Emotional-Caption Translation Network using Visual-Caption PairsSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
The natural language processing and multimedia field has seen a notable surge in interest in multimodal sentiment recognition. Hence, this study aims to employ Target-Dependent Multimodal Sentiment Analysis (TDMSA) to identify the level of sentiment associated with every target (aspect) stated within a multimodal post consisting of a visual-caption pair. Despite the recent advancements in multimodal sentiment recognition, there has been a lack of explicit incorporation of emotional clues from the visual modality, specifically those pertaining to facial expressions. The challenge at hand is to proficiently obtain visual and emotional clues and subsequently synchronise them with the textual content. In light of this fact, this study presents a novel approach called the Visual-to-Emotional-Caption Translation Network (VECTN) technique. The primary objective of this strategy is to effectively acquire visual sentiment clues by analysing facial expressions. Additionally, it effectively aligns and blends the obtained emotional clues with the target attribute of the caption mode. The experimental findings demonstrate that our methodology is capable of producing ground-breaking outcomes when applied to two publicly accessible multimodal Twitter datasets, namely, Twitter-2015 and Twitter-2017. The experimental results show that the suggested model achieves an accuracy of 81.23% and a macro-F1 of 80.61% on the Twitter-15 dataset, while 77.42% and 75.19% on the Twitter-17 dataset, respectively. The observed improvement in performance reveals that our model is better than others when it comes to collecting target-level sentiment in multimodal data using the expressions of the face.
- [22] arXiv:2408.10250 [pdf, html, other]
-
Title: General reconstruction of elastic strain fields from their Longitudinal Ray TransformChris Wensrich, Sean Holman, William Lionheart, Matias Courdurier, Anna Polyakova, Ivan Svetov, Ty DoubikinComments: 24 pages, 10 figuresSubjects: Numerical Analysis (math.NA); Applied Physics (physics.app-ph)
We develop an algorithm for reconstruction of elastic strain fields from their Longitudinal Ray Transform (LRT) in either two or three dimensions. In general, the LRT only determines the solenoidal part of a symmetric tensor field, but elastic strain fields additionally satisfy mechanical equilibrium, an extra condition that allows for full reconstruction in many cases. Our method provides full reconstruction for general elastic strain fields in connected objects whose boundary only contains one component, while previous results included other requirements such as no residual stress, or zero boundary traction. This allows for full reconstruction in energy resolved neutron transmission imaging for simple objects. Along the way, we prove that the LRT of a potential rank-2 tensor restricted to a bounded set determines the potential on the boundary of the set up to infinitesimal rigid motions on each component of the boundary. The method is demonstrated with numerical examples in two dimensions.
- [23] arXiv:2408.10252 [pdf, other]
-
Title: Balancing Innovation and Ethics in AI-Driven Software DevelopmentComments: 20 PagesSubjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
This paper critically examines the ethical implications of integrating AI tools like GitHub Copilot and ChatGPT into the software development process. It explores issues such as code ownership, bias, accountability, privacy, and the potential impact on the job market. While these AI tools offer significant benefits in terms of productivity and efficiency, they also introduce complex ethical challenges. The paper argues that addressing these challenges is essential to ensuring that AI's integration into software development is both responsible and beneficial to society
- [24] arXiv:2408.10258 [pdf, html, other]
-
Title: NeRF-US: Removing Ultrasound Imaging Artifacts from Neural Radiance Fields in the WildSubjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Current methods for performing 3D reconstruction and novel view synthesis (NVS) in ultrasound imaging data often face severe artifacts when training NeRF-based approaches. The artifacts produced by current approaches differ from NeRF floaters in general scenes because of the unique nature of ultrasound capture. Furthermore, existing models fail to produce reasonable 3D reconstructions when ultrasound data is captured or obtained casually in uncontrolled environments, which is common in clinical settings. Consequently, existing reconstruction and NVS methods struggle to handle ultrasound motion, fail to capture intricate details, and cannot model transparent and reflective surfaces. In this work, we introduced NeRF-US, which incorporates 3D-geometry guidance for border probability and scattering density into NeRF training, while also utilizing ultrasound-specific rendering over traditional volume rendering. These 3D priors are learned through a diffusion model. Through experiments conducted on our new "Ultrasound in the Wild" dataset, we observed accurate, clinically plausible, artifact-free reconstructions.
- [25] arXiv:2408.10259 [pdf, html, other]
-
Title: Contrastive Learning on Medical Intents for Sequential Prescription RecommendationComments: Accepted to the 33rd ACM International Conference on Information and Knowledge Management (CIKM 2024)Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Recent advancements in sequential modeling applied to Electronic Health Records (EHR) have greatly influenced prescription recommender systems. While the recent literature on drug recommendation has shown promising performance, the study of discovering a diversity of coexisting temporal relationships at the level of medical codes over consecutive visits remains less explored. The goal of this study can be motivated from two perspectives. First, there is a need to develop a sophisticated sequential model capable of disentangling the complex relationships across sequential visits. Second, it is crucial to establish multiple and diverse health profiles for the same patient to ensure a comprehensive consideration of different medical intents in drug recommendation. To achieve this goal, we introduce Attentive Recommendation with Contrasted Intents (ARCI), a multi-level transformer-based method designed to capture the different but coexisting temporal paths across a shared sequence of visits. Specifically, we propose a novel intent-aware method with contrastive learning, that links specialized medical intents of the patients to the transformer heads for extracting distinct temporal paths associated with different health profiles. We conducted experiments on two real-world datasets for the prescription recommendation task using both ranking and classification metrics. Our results demonstrate that ARCI has outperformed the state-of-the-art prescription recommendation methods and is capable of providing interpretable insights for healthcare practitioners.
- [26] arXiv:2408.10260 [pdf, html, other]
-
Title: Optical Music Recognition in Manuscripts from the Ricordi ArchiveComments: Accepted at AudioMostly 2024Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Digital Libraries (cs.DL)
The Ricordi archive, a prestigious collection of significant musical manuscripts from renowned opera composers such as Donizetti, Verdi and Puccini, has been digitized. This process has allowed us to automatically extract samples that represent various musical elements depicted on the manuscripts, including notes, staves, clefs, erasures, and composer's annotations, among others. To distinguish between digitization noise and actual music elements, a subset of these images was meticulously grouped and labeled by multiple individuals into several classes. After assessing the consistency of the annotations, we trained multiple neural network-based classifiers to differentiate between the identified music elements. The primary objective of this study was to evaluate the reliability of these classifiers, with the ultimate goal of using them for the automatic categorization of the remaining unannotated data set. The dataset, complemented by manual annotations, models, and source code used in these experiments are publicly accessible for replication purposes.
- [27] arXiv:2408.10261 [pdf, html, other]
-
Title: Relational Graph Convolutional Networks Do Not Learn Sound RulesComments: Full version (with appendices) of paper accepted to KR 2024 (21st International Conference on Principles of Knowledge Representation and Reasoning)Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)
Graph neural networks (GNNs) are frequently used to predict missing facts in knowledge graphs (KGs). Motivated by the lack of explainability for the outputs of these models, recent work has aimed to explain their predictions using Datalog, a widely used logic-based formalism. However, such work has been restricted to certain subclasses of GNNs. In this paper, we consider one of the most popular GNN architectures for KGs, R-GCN, and we provide two methods to extract rules that explain its predictions and are sound, in the sense that each fact derived by the rules is also predicted by the GNN, for any input dataset. Furthermore, we provide a method that can verify that certain classes of Datalog rules are not sound for the R-GCN. In our experiments, we train R-GCNs on KG completion benchmarks, and we are able to verify that no Datalog rule is sound for these models, even though the models often obtain high to near-perfect accuracy. This raises some concerns about the ability of R-GCN models to generalise and about the explainability of their predictions. We further provide two variations to the training paradigm of R-GCN that encourage it to learn sound rules and find a trade-off between model accuracy and the number of learned sound rules.
- [28] arXiv:2408.10262 [pdf, html, other]
-
Title: An Analysis of the Impact of Gold Open Access Publications in Computer ScienceComments: 12 pages, 8 figuresSubjects: Digital Libraries (cs.DL)
There has been some concern about the impact of predatory publishers on scientific research for some time. Recently, publishers that might previously have been considered `predatory' have established their bona fides, at least to the extent that they are included in citation impact scores such as the field-weighted citation impact (FWCI). These are sometimes called `grey' publishers (MDPI, Frontiers, Hindawi). In this paper, we show that the citation landscape for these grey publications is significantly different from the mainstream landscape and that affording publications in these venues the same status as publications in mainstream journals may significantly distort metrics such as the FWCI.
- [29] arXiv:2408.10263 [pdf, html, other]
-
Title: Kolmogorov Arnold Networks in Fraud Detection: Bridging the Gap Between Theory and PracticeSubjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
Kolmogorov Arnold Networks (KAN) are highly efficient in inference and can handle complex patterns once trained, making them desirable for production environments and ensuring a fast service experience in the finance and electronic shopping industries. However, we found that KAN, in general, is not suitable for fraud detection problems. We also discovered a quick method to determine whether a problem is solvable by KAN: if the data can be effectively separated using spline interpolation with varying intervals after applying Principal Component Analysis (PCA) to reduce the data dimensions to two, KAN can outperform most machine learning algorithms. Otherwise, it indicates KAN may not solve the problem effectively compared to other machine learning algorithms. We also propose a heuristic approach for selecting the appropriate hyperparameters for KAN to significantly accelerate training time compared to grid search hyperparameter tuning, which usually takes a month for a comprehensive grid search. Specifically, the width parameter should generally follow a pyramid structure, allowing efficient spline mixing, and k should be fixed at 15, with the grid number fixed at 5. This streamlined approach minimizes the number of evaluations required, significantly speeding up the hyperparameter tuning process while still achieving robust performance metrics.
- [30] arXiv:2408.10264 [pdf, html, other]
-
Title: OPDR: Order-Preserving Dimension Reduction for Semantic Embedding of Multimodal Scientific DataSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
One of the most common operations in multimodal scientific data management is searching for the $k$ most similar items (or, $k$-nearest neighbors, KNN) from the database after being provided a new item. Although recent advances of multimodal machine learning models offer a \textit{semantic} index, the so-called \textit{embedding vectors} mapped from the original multimodal data, the dimension of the resulting embedding vectors are usually on the order of hundreds or a thousand, which are impractically high for time-sensitive scientific applications.
This work proposes to reduce the dimensionality of the output embedding vectors such that the set of top-$k$ nearest neighbors do not change in the lower-dimensional space, namely Order-Preserving Dimension Reduction (OPDR). In order to develop such an OPDR method, our central hypothesis is that by analyzing the intrinsic relationship among key parameters during the dimension-reduction map, a quantitative function may be constructed to reveal the correlation between the target (lower) dimensionality and other variables. To demonstrate the hypothesis, this paper first defines a formal measure function to quantify the KNN similarity for a specific vector, then extends the measure into an aggregate accuracy of the global metric spaces, and finally derives a closed-form function between the target (lower) dimensionality and other variables. We incorporate the closed-function into popular dimension-reduction methods, various distance metrics, and embedding models. - [31] arXiv:2408.10266 [pdf, other]
-
Title: Diffusion Model for Planning: A Systematic Literature ReviewComments: 13 pages, 2 figures, 4 tablesSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Robotics (cs.RO)
Diffusion models, which leverage stochastic processes to capture complex data distributions effectively, have shown their performance as generative models, achieving notable success in image-related tasks through iterative denoising processes. Recently, diffusion models have been further applied and show their strong abilities in planning tasks, leading to a significant growth in related publications since 2023. To help researchers better understand the field and promote the development of the field, we conduct a systematic literature review of recent advancements in the application of diffusion models for planning. Specifically, this paper categorizes and discusses the current literature from the following perspectives: (i) relevant datasets and benchmarks used for evaluating diffusion modelbased planning; (ii) fundamental studies that address aspects such as sampling efficiency; (iii) skill-centric and condition-guided planning for enhancing adaptability; (iv) safety and uncertainty managing mechanism for enhancing safety and robustness; and (v) domain-specific application such as autonomous driving. Finally, given the above literature review, we further discuss the challenges and future directions in this field.
- [32] arXiv:2408.10267 [pdf, other]
-
Title: Towards Efficient Machine Learning Method for IoT DDoS Attack DetectionComments: 10 pagesSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
With the rise in the number of IoT devices and its users, security in IoT has become a big concern to ensure the protection from harmful security attacks. In the recent years, different variants of DDoS attacks have been on the rise in IoT devices. Failure to detect DDoS attacks at the right time can result in financial and reputational loss for victim organizations. These attacks conducted with IoT devices can cause a significant downtime of applications running on the Internet. Although researchers have developed and utilized specialized models using artificial intelligence techniques, these models do not provide the best accuracy as there is always a scope of improvement until 100% accuracy is attained. We propose a hybrid feature selection algorithm that selects only the most useful features and passes those features into an XGBoost model, the results of which are explained using feature importances. Our model attains an accuracy of 99.993% on the CIC IDS 2017 dataset and a recall of 97.64 % on the CIC IoT 2023 dataset. Overall, this research would help researchers and implementers in the field of detecting IoT DDoS attacks by providing a more accurate and comparable model.
- [33] arXiv:2408.10268 [pdf, html, other]
-
Title: Realtime Generation of Streamliners with Large Language ModelsSubjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
This paper presents the novel method StreamLLM for generating streamliners in constraint programming using Large Language Models (LLMs). Streamliners are constraints that narrow the search space, enhancing the speed and feasibility of solving complex problems. Traditionally, streamliners were crafted manually or generated through systematically combined atomic constraints with high-effort offline testing. Our approach uses LLMs to propose effective streamliners. Our system StreamLLM generates streamlines for problems specified in the MiniZinc constraint programming language and integrates feedback to the LLM with quick empirical tests. Our rigorous empirical evaluation involving ten problems with several hundreds of test instances shows robust results that are highly encouraging, showcasing the transforming power of LLMs in the domain of constraint programming.
- [34] arXiv:2408.10269 [pdf, html, other]
-
Title: OpenCity: Open Spatio-Temporal Foundation Models for Traffic PredictionComments: 12 pagesSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
Accurate traffic forecasting is crucial for effective urban planning and transportation management, enabling efficient resource allocation and enhanced travel experiences. However, existing models often face limitations in generalization, struggling with zero-shot prediction on unseen regions and cities, as well as diminished long-term accuracy. This is primarily due to the inherent challenges in handling the spatial and temporal heterogeneity of traffic data, coupled with the significant distribution shift across time and space. In this work, we aim to unlock new possibilities for building versatile, resilient and adaptive spatio-temporal foundation models for traffic prediction. To achieve this goal, we introduce a novel foundation model, named OpenCity, that can effectively capture and normalize the underlying spatio-temporal patterns from diverse data characteristics, facilitating zero-shot generalization across diverse urban environments. OpenCity integrates the Transformer architecture with graph neural networks to model the complex spatio-temporal dependencies in traffic data. By pre-training OpenCity on large-scale, heterogeneous traffic datasets, we enable the model to learn rich, generalizable representations that can be seamlessly applied to a wide range of traffic forecasting scenarios. Experimental results demonstrate that OpenCity exhibits exceptional zero-shot predictive performance. Moreover, OpenCity showcases promising scaling laws, suggesting the potential for developing a truly one-for-all traffic prediction solution that can adapt to new urban contexts with minimal overhead. We made our proposed OpenCity model open-source and it is available at the following link: this https URL.
- [35] arXiv:2408.10270 [pdf, html, other]
-
Title: SEAL: Systematic Error Analysis for Value ALignmentComments: 28 pages, 17 Figures, 8 TablesSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Reinforcement Learning from Human Feedback (RLHF) aims to align language models (LMs) with human values by training reward models (RMs) on binary preferences and using these RMs to fine-tune the base LMs. Despite its importance, the internal mechanisms of RLHF remain poorly understood. This paper introduces new metrics to evaluate the effectiveness of modeling and aligning human values, namely feature imprint, alignment resistance and alignment robustness. We categorize alignment datasets into target features (desired values) and spoiler features (undesired concepts). By regressing RM scores against these features, we quantify the extent to which RMs reward them - a metric we term feature imprint. We define alignment resistance as the proportion of the preference dataset where RMs fail to match human preferences, and we assess alignment robustness by analyzing RM responses to perturbed inputs. Our experiments, utilizing open-source components like the Anthropic/hh-rlhf preference dataset and OpenAssistant RMs, reveal significant imprints of target features and a notable sensitivity to spoiler features. We observed a 26% incidence of alignment resistance in portions of the dataset where LM-labelers disagreed with human preferences. Furthermore, we find that misalignment often arises from ambiguous entries within the alignment dataset. These findings underscore the importance of scrutinizing both RMs and alignment datasets for a deeper understanding of value alignment.
- [36] arXiv:2408.10271 [pdf, html, other]
-
Title: Data-Driven Fire Modeling: Learning First Arrival Times and Model Parameters with Neural NetworksSubjects: Machine Learning (cs.LG)
Data-driven techniques are being increasingly applied to complement physics-based models in fire science. However, the lack of sufficiently large datasets continues to hinder the application of certain machine learning techniques. In this paper, we use simulated data to investigate the ability of neural networks to parameterize dynamics in fire science. In particular, we investigate neural networks that map five key parameters in fire spread to the first arrival time, and the corresponding inverse problem. By using simulated data, we are able to characterize the error, the required dataset size, and the convergence properties of these neural networks. For the inverse problem, we quantify the network's sensitivity in estimating each of the key parameters. The findings demonstrate the potential of machine learning in fire science, highlight the challenges associated with limited dataset sizes, and quantify the sensitivity of neural networks to estimate key parameters governing fire spread dynamics.
- [37] arXiv:2408.10275 [pdf, other]
-
Title: FedKBP: Federated dose prediction framework for knowledge-based planning in radiation therapyComments: Under review by SPIE Medical Imaging 2025 ConferenceSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Dose prediction plays a key role in knowledge-based planning (KBP) by automatically generating patient-specific dose distribution. Recent advances in deep learning-based dose prediction methods necessitates collaboration among data contributors for improved performance. Federated learning (FL) has emerged as a solution, enabling medical centers to jointly train deep-learning models without compromising patient data privacy. We developed the FedKBP framework to evaluate the performances of centralized, federated, and individual (i.e. separated) training of dose prediction model on the 340 plans from OpenKBP dataset. To simulate FL and individual training, we divided the data into 8 training sites. To evaluate the effect of inter-site data variation on model training, we implemented two types of case distributions: 1) Independent and identically distributed (IID), where the training and validating cases were evenly divided among the 8 sites, and 2) non-IID, where some sites have more cases than others. The results show FL consistently outperforms individual training on both model optimization speed and out-of-sample testing scores, highlighting the advantage of FL over individual training. Under IID data division, FL shows comparable performance to centralized training, underscoring FL as a promising alternative to traditional pooled-data training. Under non-IID division, larger sites outperformed smaller sites by up to 19% on testing scores, confirming the need of collaboration among data owners to achieve better prediction accuracy. Meanwhile, non-IID FL showed reduced performance as compared to IID FL, posing the need for more sophisticated FL method beyond mere model averaging to handle data variation among participating sites.
- [38] arXiv:2408.10276 [pdf, html, other]
-
Title: FEDKIM: Adaptive Federated Knowledge Injection into Medical Foundation ModelsComments: Submitted to EMNLP'24Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Foundation models have demonstrated remarkable capabilities in handling diverse modalities and tasks, outperforming conventional artificial intelligence (AI) approaches that are highly task-specific and modality-reliant. In the medical domain, however, the development of comprehensive foundation models is constrained by limited access to diverse modalities and stringent privacy regulations. To address these constraints, this study introduces a novel knowledge injection approach, FedKIM, designed to scale the medical foundation model within a federated learning framework. FedKIM leverages lightweight local models to extract healthcare knowledge from private data and integrates this knowledge into a centralized foundation model using a designed adaptive Multitask Multimodal Mixture Of Experts (M3OE) module. This method not only preserves privacy but also enhances the model's ability to handle complex medical tasks involving multiple modalities. Our extensive experiments across twelve tasks in seven modalities demonstrate the effectiveness of FedKIM in various settings, highlighting its potential to scale medical foundation models without direct access to sensitive data.
- [39] arXiv:2408.10277 [pdf, other]
-
Title: Increasing transformer token length with a Maximum Entropy Principle MethodComments: 17 pagesSubjects: Machine Learning (cs.LG)
Transformers suffer from the computational overhead of their quadratic dependence on the length of sequences processed. We present three methods, all adding an intermediate step between training and inference/generation, which extend the autoregressive length of transformers. All rely on a Maximum Entropy Principle (MEP) whereby entropy is maximized in the presence of suitable constraints, accounted for by use of Lagrange Multipliers. These constraint methods extend the autoregressive character from T to 2T tokens in a linear-with-T fashion. There is overhead associated with this added step, but they should still be faster than the standard methods.
- [40] arXiv:2408.10280 [pdf, html, other]
-
Title: NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large ModelsComments: Work in progress, revisions ongoingSubjects: Machine Learning (cs.LG)
In this paper, we introduce Nested Low-Rank Adaptation (NoRA), a novel approach to parameter-efficient fine-tuning that extends the capabilities of Low-Rank Adaptation (LoRA) techniques. Vanilla LoRA overlooks pre-trained weight inheritance and still requires fine-tuning numerous parameters. To addresses these issues, our NoRA adopts a dual-layer nested structure with Singular Value Decomposition (SVD), effectively leveraging original matrix knowledge while reducing tunable parameters. Specifically, NoRA freezes the outer LoRA weights and utilizes an inner LoRA design, providing enhanced control over model optimization. This approach allows the model to more precisely adapt to specific tasks while maintaining a compact parameter space. By freezing outer LoRA weights and using an inner LoRA design, NoRA enables precise task adaptation with a compact parameter space. Evaluations on tasks including commonsense reasoning with large language models, fine-tuning vision-language models, and subject-driven generation demonstrate NoRA's superiority over LoRA and its variants. Notably, NoRA reduces fine-tuning parameters|training-time|memory-usage by 4\%|22.5\%|20.7\% compared to LoRA on LLaMA-3 8B, while achieving 2.2\% higher performance. Code will be released upon acceptance.
- [41] arXiv:2408.10281 [pdf, other]
-
Title: Scalable Systems and Software Architectures for High-Performance Computing on cloud platformsComments: 6 PagesSubjects: Distributed, Parallel, and Cluster Computing (cs.DC); Performance (cs.PF)
High-performance computing (HPC) is essential for tackling complex computational problems across various domains. As the scale and complexity of HPC applications continue to grow, the need for scalable systems and software architectures becomes paramount. This paper provides a comprehensive overview of architecture for HPC on premise focusing on both hardware and software aspects and details the associated challenges in building the HPC cluster on premise. It explores design principles, challenges, and emerging trends in building scalable HPC systems and software, addressing issues such as parallelism, memory hierarchy, communication overhead, and fault tolerance on various cloud platforms. By synthesizing research findings and technological advancements, this paper aims to provide insights into scalable solutions for meeting the evolving demands of HPC applications on cloud.
- [42] arXiv:2408.10284 [pdf, html, other]
-
Title: AdapMoE: Adaptive Sensitivity-based Expert Gating and Management for Efficient MoE InferenceSubjects: Machine Learning (cs.LG)
Mixture-of-Experts (MoE) models are designed to enhance the efficiency of large language models (LLMs) without proportionally increasing the computational demands. However, their deployment on edge devices still faces significant challenges due to high on-demand loading overheads from managing sparsely activated experts. This paper introduces AdapMoE, an algorithm-system co-design framework for efficient MoE inference. AdapMoE features adaptive expert gating and management to reduce the on-demand loading overheads. We observe the heterogeneity of experts loading across layers and tokens, based on which we propose a sensitivity-based strategy to adjust the number of activated experts dynamically. Meanwhile, we also integrate advanced prefetching and cache management techniques to further reduce the loading latency. Through comprehensive evaluations on various platforms, we demonstrate AdapMoE consistently outperforms existing techniques, reducing the average number of activated experts by 25% and achieving a 1.35x speedup without accuracy degradation. Code is available at: this https URL.
- [43] arXiv:2408.10285 [pdf, html, other]
-
Title: BatGPT-Chem: A Foundation Large Model For Retrosynthesis PredictionSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE)
Retrosynthesis analysis is pivotal yet challenging in drug discovery and organic chemistry. Despite the proliferation of computational tools over the past decade, AI-based systems often fall short in generalizing across diverse reaction types and exploring alternative synthetic pathways. This paper presents BatGPT-Chem, a large language model with 15 billion parameters, tailored for enhanced retrosynthesis prediction. Integrating chemical tasks via a unified framework of natural language and SMILES notation, this approach synthesizes extensive instructional data from an expansive chemical database. Employing both autoregressive and bidirectional training techniques across over one hundred million instances, BatGPT-Chem captures a broad spectrum of chemical knowledge, enabling precise prediction of reaction conditions and exhibiting strong zero-shot capabilities. Superior to existing AI methods, our model demonstrates significant advancements in generating effective strategies for complex molecules, as validated by stringent benchmark tests. BatGPT-Chem not only boosts the efficiency and creativity of retrosynthetic analysis but also establishes a new standard for computational tools in synthetic design. This development empowers chemists to adeptly address the synthesis of novel compounds, potentially expediting the innovation cycle in drug manufacturing and materials science. We release our trial platform at \url{this https URL}.
- [44] arXiv:2408.10286 [pdf, other]
-
Title: GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle DispatchingXiao Han, Zijian Zhang, Xiangyu Zhao, Guojiang Shen, Xiangjie Kong, Xuetao Wei, Liqiang Nie, Jieping YeSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
As urban residents demand higher travel quality, vehicle dispatch has become a critical component of online ride-hailing services. However, current vehicle dispatch systems struggle to navigate the complexities of urban traffic dynamics, including unpredictable traffic conditions, diverse driver behaviors, and fluctuating supply and demand patterns. These challenges have resulted in travel difficulties for passengers in certain areas, while many drivers in other areas are unable to secure orders, leading to a decline in the overall quality of urban transportation services. To address these issues, this paper introduces GARLIC: a framework of GPT-Augmented Reinforcement Learning with Intelligent Control for vehicle dispatching. GARLIC utilizes multiview graphs to capture hierarchical traffic states, and learns a dynamic reward function that accounts for individual driving behaviors. The framework further integrates a GPT model trained with a custom loss function to enable high-precision predictions and optimize dispatching policies in real-world scenarios. Experiments conducted on two real-world datasets demonstrate that GARLIC effectively aligns with driver behaviors while reducing the empty load rate of vehicles.
- [45] arXiv:2408.10288 [pdf, html, other]
-
Title: Augmenting train maintenance technicians with automated incident diagnostic suggestionsSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Train operational incidents are so far diagnosed individually and manually by train maintenance technicians. In order to assist maintenance crews in their responsiveness and task prioritization, a learning machine is developed and deployed in production to suggest diagnostics to train technicians on their phones, tablets or laptops as soon as a train incident is declared. A feedback loop allows to take into account the actual diagnose by designated train maintenance experts to refine the learning machine. By formulating the problem as a discrete set classification task, feature engineering methods are proposed to extract physically plausible sets of events from traces generated on-board railway vehicles. The latter feed an original ensemble classifier to class incidents by their potential technical cause. Finally, the resulting model is trained and validated using real operational data and deployed on a cloud platform. Future work will explore how the extracted sets of events can be used to avoid incidents by assisting human experts in the creation predictive maintenance alerts.
- [46] arXiv:2408.10291 [pdf, html, other]
-
Title: Chatbots and Zero Sales ResistanceComments: 8 pagesSubjects: Machine Learning (cs.LG)
It is argued that the pursuit of an ever increasing number of weights in large-scale machine learning applications, besides being energetically unsustainable, is also conducive to manipulative strategies whereby Science is easily served as a strawman for economic and financial power. If machine learning is meant to serve science ahead of vested business interests, a paradigm shift is needed: from more weights and little insight to more insight and less weights.
- [47] arXiv:2408.10292 [pdf, html, other]
-
Title: Leveraging Superfluous Information in Contrastive Representation LearningSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Contrastive representation learning, which aims to learnthe shared information between different views of unlabeled data by maximizing the mutual information between them, has shown its powerful competence in self-supervised learning for downstream tasks. However, recent works have demonstrated that more estimated mutual information does not guarantee better performance in different downstream tasks. Such works inspire us to conjecture that the learned representations not only maintain task-relevant information from unlabeled data but also carry task-irrelevant information which is superfluous for downstream tasks, thus leading to performance degeneration. In this paper we show that superfluous information does exist during the conventional contrastive learning framework, and further design a new objective, namely SuperInfo, to learn robust representations by a linear combination of both predictive and superfluous information. Besides, we notice that it is feasible to tune the coefficients of introduced losses to discard task-irrelevant information, while keeping partial non-shared task-relevant information according to our SuperInfo loss.We demonstrate that learning with our loss can often outperform the traditional contrastive learning approaches on image classification, object detection and instance segmentation tasks with significant improvements.
- [48] arXiv:2408.10327 [pdf, html, other]
-
Title: An Empirical Study on Package-Level Deprecation in Python EcosystemComments: Accepted by 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE'25)Subjects: Software Engineering (cs.SE)
Open-source software (OSS) plays a crucial role in modern software development. Utilizing OSS code can greatly accelerate software development, reduce redundancy, and enhance reliability. Python, a widely adopted programming language, is renowned for its extensive and diverse third-party package ecosystem. However, a significant number of OSS packages within the Python ecosystem are in poor maintenance, leading to potential risks in functionality and security. Consequently, it is essential to establish a deprecation mechanism to assist package developers and users in managing packages effectively.
To facilitate the establishment of the package-level deprecation mechanism, this paper presents a mixed-method empirical study, including data analysis and surveys. We investigate the current practices of announcing, receiving, and handling package-level deprecation in the Python ecosystem. We also assess the benefits of having deprecation announcements for inactively maintained packages. Furthermore, we investigate the challenges faced by package developers and users and their expectations for future deprecation practices. Our findings reveal that 75.4% of inactive package developers have no intention of releasing deprecation declarations for various reasons, while 89.5% of users express a desire to be notified about the deprecation, highlighting a gap between developers and users; in many cases, no alternative solutions are available when deprecation occurs, emphasizing the need to explore practical approaches that enable seamless package handover and require less maintenance effort. Our work aims to enhance the understanding of existing package-level deprecation patterns within the Python OSS realm and facilitate the development of deprecation practices for the Python community in the future. - [49] arXiv:2408.10328 [pdf, html, other]
-
Title: Decoding Human Emotions: Analyzing Multi-Channel EEG Data using LSTM NetworksComments: 13 pages, 3 figures; accepted at ICDSA '24 Conference, Jaipur, IndiaSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
Emotion recognition from electroencephalogram (EEG) signals is a thriving field, particularly in neuroscience and Human-Computer Interaction (HCI). This study aims to understand and improve the predictive accuracy of emotional state classification through metrics such as valence, arousal, dominance, and likeness by applying a Long Short-Term Memory (LSTM) network to analyze EEG signals. Using a popular dataset of multi-channel EEG recordings known as DEAP, we look towards leveraging LSTM networks' properties to handle temporal dependencies within EEG signal data. This allows for a more comprehensive understanding and classification of emotional parameter states. We obtain accuracies of 89.89%, 90.33%, 90.70%, and 90.54% for arousal, valence, dominance, and likeness, respectively, demonstrating significant improvements in emotion recognition model capabilities. This paper elucidates the methodology and architectural specifics of our LSTM model and provides a benchmark analysis with existing papers.
- [50] arXiv:2408.10330 [pdf, html, other]
-
Title: Meta-Learning in Audio and Speech Processing: An End to End Comprehensive ReviewComments: Survey Paper (15 pages, 1 figure)Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
This survey overviews various meta-learning approaches used in audio and speech processing scenarios. Meta-learning is used where model performance needs to be maximized with minimum annotated samples, making it suitable for low-sample audio processing. Although the field has made some significant contributions, audio meta-learning still lacks the presence of comprehensive survey papers. We present a systematic review of meta-learning methodologies in audio processing. This includes audio-specific discussions on data augmentation, feature extraction, preprocessing techniques, meta-learners, task selection strategies and also presents important datasets in audio, together with crucial real-world use cases. Through this extensive review, we aim to provide valuable insights and identify future research directions in the intersection of meta-learning and audio processing.
- [51] arXiv:2408.10332 [pdf, html, other]
-
Title: Spectral Guarantees for Adversarial Streaming PCAComments: FOCS 2024Subjects: Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG)
In streaming PCA, we see a stream of vectors $x_1, \dotsc, x_n \in \mathbb{R}^d$ and want to estimate the top eigenvector of their covariance matrix. This is easier if the spectral ratio $R = \lambda_1 / \lambda_2$ is large. We ask: how large does $R$ need to be to solve streaming PCA in $\widetilde{O}(d)$ space? Existing algorithms require $R = \widetilde{\Omega}(d)$. We show: (1) For all mergeable summaries, $R = \widetilde{\Omega}(\sqrt{d})$ is necessary. (2) In the insertion-only model, a variant of Oja's algorithm gets $o(1)$ error for $R = O(\log n \log d)$. (3) No algorithm with $o(d^2)$ space gets $o(1)$ error for $R = O(1)$.
Our analysis is the first application of Oja's algorithm to adversarial streams. It is also the first algorithm for adversarial streaming PCA that is designed for a spectral, rather than Frobenius, bound on the tail; and the bound it needs is exponentially better than is possible by adapting a Frobenius guarantee. - [52] arXiv:2408.10333 [pdf, other]
-
Title: An LMI-based Robust Fuzzy Controller for Blood Glucose Regulation in Type 1 DiabetesSubjects: Systems and Control (eess.SY)
This paper presents a control algorithm for creating an artificial pancreas for type 1 diabetes, factoring in input saturation for a practical application. By utilizing the parallel distributed compensation and Takagi-Sugeno Fuzzy model, we design an optimal robust fuzzy controller. Stability conditions derived from the Lyapunov method are expressed as linear matrix inequalities, allowing for optimal controller gain selection that minimizes disturbance effects. We employ the minimal Bergman and Tolic models to represent type 1 diabetes glucose-insulin dynamics, converting them into corresponding Takagi-Sugeno fuzzy models using the sector nonlinearity approach. Simulation results demonstrate the proposed controller's effectiveness.
- [53] arXiv:2408.10334 [pdf, html, other]
-
Title: A Disguised Wolf Is More Harmful Than a Toothless Tiger: Adaptive Malicious Code Injection Backdoor Attack Leveraging User Behavior as TriggersSubjects: Artificial Intelligence (cs.AI)
In recent years, large language models (LLMs) have made significant progress in the field of code generation. However, as more and more users rely on these models for software development, the security risks associated with code generation models have become increasingly significant. Studies have shown that traditional deep learning robustness issues also negatively impact the field of code generation. In this paper, we first present the game-theoretic model that focuses on security issues in code generation scenarios. This framework outlines possible scenarios and patterns where attackers could spread malicious code models to create security threats. We also pointed out for the first time that the attackers can use backdoor attacks to dynamically adjust the timing of malicious code injection, which will release varying degrees of malicious code depending on the skill level of the user. Through extensive experiments on leading code generation models, we validate our proposed game-theoretic model and highlight the significant threats that these new attack scenarios pose to the safe use of code models.
- [54] arXiv:2408.10338 [pdf, html, other]
-
Title: Revisiting Tree Canonization using polynomialsSubjects: Computational Complexity (cs.CC); Data Structures and Algorithms (cs.DS)
Graph Isomorphism (GI) is a fundamental algorithmic problem. Amongst graph classes for which the computational complexity of GI has been resolved, trees are arguably the most fundamental. Tree Isomorphism is complete for deterministic logspace, a tiny subclass of polynomial time, by Lindell's result. Over three decades ago, he devised a deterministic logspace algorithm that computes a string which is a canon for the input tree -- two trees are isomorphic precisely when their canons are identical. Inspired by Miller-Reif's reduction of Tree Isomorphism to Polynomial Identity Testing, we present a new logspace algorithm for tree canonization fundamentally different from Lindell's algorithm. Our algorithm computes a univariate polynomial as canon for an input tree, based on the classical Eisenstein's criterion for the irreducibility of univariate polynomials. This can be implemented in logspace by invoking the well known Buss et al. algorithm for arithmetic formula evaluation. This algorithm is conceptually very simple, avoiding the delicate case analysis and complex recursion that constitute the core of Lindell's algorithm. We illustrate the adaptability of our algorithm by extending it to a couple of other classes of graphs.
- [55] arXiv:2408.10343 [pdf, html, other]
-
Title: LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal DomainSubjects: Artificial Intelligence (cs.AI)
Retrieval-Augmented Generation (RAG) systems are showing promising potential, and are becoming increasingly relevant in AI-powered legal applications. Existing benchmarks, such as LegalBench, assess the generative capabilities of Large Language Models (LLMs) in the legal domain, but there is a critical gap in evaluating the retrieval component of RAG systems. To address this, we introduce LegalBench-RAG, the first benchmark specifically designed to evaluate the retrieval step of RAG pipelines within the legal space. LegalBench-RAG emphasizes precise retrieval by focusing on extracting minimal, highly relevant text segments from legal documents. These highly relevant snippets are preferred over retrieving document IDs, or large sequences of imprecise chunks, both of which can exceed context window limitations. Long context windows cost more to process, induce higher latency, and lead LLMs to forget or hallucinate information. Additionally, precise results allow LLMs to generate citations for the end user. The LegalBench-RAG benchmark is constructed by retracing the context used in LegalBench queries back to their original locations within the legal corpus, resulting in a dataset of 6,858 query-answer pairs over a corpus of over 79M characters, entirely human-annotated by legal experts. We also introduce LegalBench-RAG-mini, a lightweight version for rapid iteration and experimentation. By providing a dedicated benchmark for legal retrieval, LegalBench-RAG serves as a critical tool for companies and researchers focused on enhancing the accuracy and performance of RAG systems in the legal domain. The LegalBench-RAG dataset is publicly available at this https URL.
- [56] arXiv:2408.10346 [pdf, html, other]
-
Title: Toward Fair and Strategyproof Tournament Rules for Tournaments with Partially Transferable UtilitiesSubjects: Computer Science and Game Theory (cs.GT)
A tournament on $n$ agents is a complete oriented graph with the agents as vertices and edges that describe the win-loss outcomes of the $\binom{n}{2}$ matches played between each pair of agents. The winner of a tournament is determined by a tournament rule that maps tournaments to probability distributions over the agents. We want these rules to be fair (choose a high-quality agent) and robust to strategic manipulation. Prior work has shown that under minimally fair rules, manipulations between two agents can be prevented when utility is nontransferable but not when utility is completely transferable. We introduce a partially transferable utility model that interpolates between these two extremes using a selfishness parameter $\lambda$. Our model is that an agent may be willing to lose on purpose, sacrificing some of her own chance of winning, but only if the colluding pair's joint gain is more than $\lambda$ times the individual's sacrifice.
We show that no fair tournament rule can prevent manipulations when $\lambda < 1$. We computationally solve for fair and manipulation-resistant tournament rules for $\lambda = 1$ for up to 6 agents. We conjecture and leave as a major open problem that such a tournament rule exists for all $n$. We analyze the trade-offs between ``relative'' and ``absolute'' approximate strategyproofness for previously studied rules and derive as a corollary that all of these rules require $\lambda \geq \Omega(n)$ to be robust to manipulation. We show that for stronger notions of fairness, non-manipulable tournament rules are closely related to tournament rules that witness decreasing gains from manipulation as the number of agents increases. - [57] arXiv:2408.10349 [pdf, html, other]
-
Title: AIR: Analytic Imbalance Rectifier for Continual LearningSubjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
Continual learning enables AI models to learn new data sequentially without retraining in real-world scenarios. Most existing methods assume the training data are balanced, aiming to reduce the catastrophic forgetting problem that models tend to forget previously generated data. However, data imbalance and the mixture of new and old data in real-world scenarios lead the model to ignore categories with fewer training samples. To solve this problem, we propose an analytic imbalance rectifier algorithm (AIR), a novel online exemplar-free continual learning method with an analytic (i.e., closed-form) solution for data-imbalanced class-incremental learning (CIL) and generalized CIL scenarios in real-world continual learning. AIR introduces an analytic re-weighting module (ARM) that calculates a re-weighting factor for each class for the loss function to balance the contribution of each category to the overall loss and solve the problem of imbalanced training data. AIR uses the least squares technique to give a non-discriminatory optimal classifier and its iterative update method in continual learning. Experimental results on multiple datasets show that AIR significantly outperforms existing methods in long-tailed and generalized CIL scenarios. The source code is available at this https URL.
- [58] arXiv:2408.10351 [pdf, html, other]
-
Title: The Psychological Impacts of Algorithmic and AI-Driven Social Media on Teenagers: A Call to ActionComments: 7 pages, 0 figures, 2 tables, 2024 IEEE Conference on Digital Platforms and Societal HarmsSubjects: Social and Information Networks (cs.SI); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Human-Computer Interaction (cs.HC)
This study investigates the meta-issues surrounding social media, which, while theoretically designed to enhance social interactions and improve our social lives by facilitating the sharing of personal experiences and life events, often results in adverse psychological impacts. Our investigation reveals a paradoxical outcome: rather than fostering closer relationships and improving social lives, the algorithms and structures that underlie social media platforms inadvertently contribute to a profound psychological impact on individuals, influencing them in unforeseen ways. This phenomenon is particularly pronounced among teenagers, who are disproportionately affected by curated online personas, peer pressure to present a perfect digital image, and the constant bombardment of notifications and updates that characterize their social media experience. As such, we issue a call to action for policymakers, platform developers, and educators to prioritize the well-being of teenagers in the digital age and work towards creating secure and safe social media platforms that protect the young from harm, online harassment, and exploitation.
- [59] arXiv:2408.10353 [pdf, html, other]
-
Title: On the Identifiability of Sparse ICA without Assuming Non-GaussianityComments: NeurIPS 2023Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Independent component analysis (ICA) is a fundamental statistical tool used to reveal hidden generative processes from observed data. However, traditional ICA approaches struggle with the rotational invariance inherent in Gaussian distributions, often necessitating the assumption of non-Gaussianity in the underlying sources. This may limit their applicability in broader contexts. To accommodate Gaussian sources, we develop an identifiability theory that relies on second-order statistics without imposing further preconditions on the distribution of sources, by introducing novel assumptions on the connective structure from sources to observed variables. Different from recent work that focuses on potentially restrictive connective structures, our proposed assumption of structural variability is both considerably less restrictive and provably necessary. Furthermore, we propose two estimation methods based on second-order statistics and sparsity constraint. Experimental results are provided to validate our identifiability theory and estimation methods.
- [60] arXiv:2408.10356 [pdf, html, other]
-
Title: Diversity and stylization of the contemporary user-generated visual arts in the complexity-entropy planeComments: 18 pages, 3 figures, 1 table, SI(4 figures, 3 tables)Subjects: Computer Vision and Pattern Recognition (cs.CV); Data Analysis, Statistics and Probability (physics.data-an); Physics and Society (physics.soc-ph)
The advent of computational and numerical methods in recent times has provided new avenues for analyzing art historiographical narratives and tracing the evolution of art styles therein. Here, we investigate an evolutionary process underpinning the emergence and stylization of contemporary user-generated visual art styles using the complexity-entropy (C-H) plane, which quantifies local structures in paintings. Informatizing 149,780 images curated in DeviantArt and Behance platforms from 2010 to 2020, we analyze the relationship between local information of the C-H space and multi-level image features generated by a deep neural network and a feature extraction algorithm. The results reveal significant statistical relationships between the C-H information of visual artistic styles and the dissimilarities of the multi-level image features over time within groups of artworks. By disclosing a particular C-H region where the diversity of image representations is noticeably manifested, our analyses reveal an empirical condition of emerging styles that are both novel in the C-H plane and characterized by greater stylistic diversity. Our research shows that visual art analyses combined with physics-inspired methodologies and machine learning, can provide macroscopic insights into quantitatively mapping relevant characteristics of an evolutionary process underpinning the creative stylization of uncharted visual arts of given groups and time.
- [61] arXiv:2408.10357 [pdf, html, other]
-
Title: Beyond Relevant Documents: A Knowledge-Intensive Approach for Query-Focused Summarization using Large Language ModelsComments: Accepted by the 27th International Conference on Pattern Recognition (ICPR 2024)Subjects: Computation and Language (cs.CL); Information Retrieval (cs.IR)
Query-focused summarization (QFS) is a fundamental task in natural language processing with broad applications, including search engines and report generation. However, traditional approaches assume the availability of relevant documents, which may not always hold in practical scenarios, especially in highly specialized topics. To address this limitation, we propose a novel knowledge-intensive approach that reframes QFS as a knowledge-intensive task setup. This approach comprises two main components: a retrieval module and a summarization controller. The retrieval module efficiently retrieves potentially relevant documents from a large-scale knowledge corpus based on the given textual query, eliminating the dependence on pre-existing document sets. The summarization controller seamlessly integrates a powerful large language model (LLM)-based summarizer with a carefully tailored prompt, ensuring the generated summary is comprehensive and relevant to the query. To assess the effectiveness of our approach, we create a new dataset, along with human-annotated relevance labels, to facilitate comprehensive evaluation covering both retrieval and summarization performance. Extensive experiments demonstrate the superior performance of our approach, particularly its ability to generate accurate summaries without relying on the availability of relevant documents initially. This underscores our method's versatility and practical applicability across diverse query scenarios.
- [62] arXiv:2408.10360 [pdf, html, other]
-
Title: HaSPeR: An Image Repository for Hand Shadow Puppet RecognitionComments: Submitted to IEEE Transactions on Artificial Intelligence (IEEE TAI), 11 pages, 78 figures, 2 tablesSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Hand shadow puppetry, also known as shadowgraphy or ombromanie, is a form of theatrical art and storytelling where hand shadows are projected onto flat surfaces to create illusions of living creatures. The skilled performers create these silhouettes by hand positioning, finger movements, and dexterous gestures to resemble shadows of animals and objects. Due to the lack of practitioners and a seismic shift in people's entertainment standards, this art form is on the verge of extinction. To facilitate its preservation and proliferate it to a wider audience, we introduce ${\rm H{\small A}SP{\small E}R}$, a novel dataset consisting of 8,340 images of hand shadow puppets across 11 classes extracted from both professional and amateur hand shadow puppeteer clips. We provide a detailed statistical analysis of the dataset and employ a range of pretrained image classification models to establish baselines. Our findings show a substantial performance superiority of traditional convolutional models over attention-based transformer architectures. We also find that lightweight models, such as MobileNetV2, suited for mobile applications and embedded devices, perform comparatively well. We surmise that such low-latency architectures can be useful in developing ombromanie teaching tools, and we create a prototype application to explore this surmission. Keeping the best-performing model InceptionV3 under the limelight, we conduct comprehensive feature-spatial, explainability, and error analyses to gain insights into its decision-making process. To the best of our knowledge, this is the first documented dataset and research endeavor to preserve this dying art for future generations, with computer vision approaches. Our code and data are publicly available.
- [63] arXiv:2408.10362 [pdf, other]
-
Title: Query languages for neural networksComments: To appear at ICDT 2025Subjects: Artificial Intelligence (cs.AI); Databases (cs.DB); Logic in Computer Science (cs.LO)
We lay the foundations for a database-inspired approach to interpreting and understanding neural network models by querying them using declarative languages. Towards this end we study different query languages, based on first-order logic, that mainly differ in their access to the neural network model. First-order logic over the reals naturally yields a language which views the network as a black box; only the input--output function defined by the network can be queried. This is essentially the approach of constraint query languages. On the other hand, a white-box language can be obtained by viewing the network as a weighted graph, and extending first-order logic with summation over weight terms. The latter approach is essentially an abstraction of SQL. In general, the two approaches are incomparable in expressive power, as we will show. Under natural circumstances, however, the white-box approach can subsume the black-box approach; this is our main result. We prove the result concretely for linear constraint queries over real functions definable by feedforward neural networks with a fixed number of hidden layers and piecewise linear activation functions.
- [64] arXiv:2408.10365 [pdf, html, other]
-
Title: AI-Driven Review Systems: Evaluating LLMs in Scalable and Bias-Aware Academic ReviewsKeith Tyser, Ben Segev, Gaston Longhitano, Xin-Yu Zhang, Zachary Meeks, Jason Lee, Uday Garg, Nicholas Belsten, Avi Shporer, Madeleine Udell, Dov Te'eni, Iddo DroriComments: 42 pagesSubjects: Artificial Intelligence (cs.AI)
Automatic reviewing helps handle a large volume of papers, provides early feedback and quality control, reduces bias, and allows the analysis of trends. We evaluate the alignment of automatic paper reviews with human reviews using an arena of human preferences by pairwise comparisons. Gathering human preference may be time-consuming; therefore, we also use an LLM to automatically evaluate reviews to increase sample efficiency while reducing bias. In addition to evaluating human and LLM preferences among LLM reviews, we fine-tune an LLM to predict human preferences, predicting which reviews humans will prefer in a head-to-head battle between LLMs. We artificially introduce errors into papers and analyze the LLM's responses to identify limitations, use adaptive review questions, meta prompting, role-playing, integrate visual and textual analysis, use venue-specific reviewing materials, and predict human preferences, improving upon the limitations of the traditional review processes. We make the reviews of publicly available arXiv and open-access Nature journal papers available online, along with a free service which helps authors review and revise their research papers and improve their quality. This work develops proof-of-concept LLM reviewing systems that quickly deliver consistent, high-quality reviews and evaluate their quality. We mitigate the risks of misuse, inflated review scores, overconfident ratings, and skewed score distributions by augmenting the LLM with multiple documents, including the review form, reviewer guide, code of ethics and conduct, area chair guidelines, and previous year statistics, by finding which errors and shortcomings of the paper may be detected by automated reviews, and evaluating pairwise reviewer preferences. This work identifies and addresses the limitations of using LLMs as reviewers and evaluators and enhances the quality of the reviewing process.
- [65] arXiv:2408.10368 [pdf, html, other]
-
Title: Deep-MacroFin: Informed Equilibrium Neural Network for Continuous Time Economic ModelsComments: 25 pages, 8 figuresSubjects: Machine Learning (cs.LG); Computational Engineering, Finance, and Science (cs.CE); Computational Finance (q-fin.CP)
In this paper, we present Deep-MacroFin, a comprehensive framework designed to solve partial differential equations, with a particular focus on models in continuous time economics. This framework leverages deep learning methodologies, including conventional Multi-Layer Perceptrons and the newly developed Kolmogorov-Arnold Networks. It is optimized using economic information encapsulated by Hamilton-Jacobi-Bellman equations and coupled algebraic equations. The application of neural networks holds the promise of accurately resolving high-dimensional problems with fewer computational demands and limitations compared to standard numerical methods. This versatile framework can be readily adapted for elementary differential equations, and systems of differential equations, even in cases where the solutions may exhibit discontinuities. Importantly, it offers a more straightforward and user-friendly implementation than existing libraries.
- [66] arXiv:2408.10369 [pdf, html, other]
-
Title: Boolean Matrix Logic ProgrammingSubjects: Symbolic Computation (cs.SC); Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)
We describe a datalog query evaluation approach based on efficient and composable boolean matrix manipulation modules. We first define an overarching problem, Boolean Matrix Logic Programming (BMLP), which uses boolean matrices as an alternative computation to evaluate datalog programs. We develop two novel BMLP modules for bottom-up inferences on linear dyadic recursive datalog programs, and show how additional modules can extend this capability to compute both linear and non-linear recursive datalog programs of arity two. Our empirical results demonstrate that these modules outperform general-purpose and specialised systems by factors of 30x and 9x, respectively, when evaluating large programs with millions of facts. This boolean matrix approach significantly enhances the efficiency of datalog querying to support logic programming techniques.
- [67] arXiv:2408.10370 [pdf, html, other]
-
Title: Convergence analysis of Levenberg-Marquardt method with Singular Scaling for nonzero residue nonlinear least-squares problemsSubjects: Numerical Analysis (math.NA); Optimization and Control (math.OC)
This paper addresses the convergence analysis of a variant of the LevenbergMarquardt method (LMM) designed for nonlinear least-squares problems with non-zero residue. This variant, called LMM with Singular Scaling (LMMSS), allows the LMM scaling matrix to be singular, encompassing a broader class of regularizers, which has proven useful in certain applications. In order to establish local convergence results, a careful choice of the LMM parameter is made based on the gradient linearization error, dictated by the nonlinearity and size of the residual. Under completeness and local error bound assumptions we prove that the distance from an iterate to the set of stationary points goes to zero superlinearly and that the iterative sequence converges. Furthermore, we also study a globalized version of the method obtained using linesearch and prove that any limit point of the generated sequence is stationary. Some examples are provided to illustrate our theoretical results.
- [68] arXiv:2408.10374 [pdf, html, other]
-
Title: Security Risks Due to Data Persistence in Cloud FPGA PlatformsSubjects: Cryptography and Security (cs.CR); Hardware Architecture (cs.AR); Distributed, Parallel, and Cluster Computing (cs.DC)
The integration of Field Programmable Gate Arrays (FPGAs) into cloud computing systems has become commonplace. As the operating systems used to manage these systems evolve, special consideration must be given to DRAM devices accessible by FPGAs. These devices may hold sensitive data that can become inadvertently exposed to adversaries following user logout. Although addressed in some cloud FPGA environments, automatic DRAM clearing after process termination is not automatically included in popular FPGA runtime environments nor in most proposed cloud FPGA hypervisors. In this paper, we examine DRAM data persistence in AMD/Xilinx Alveo U280 nodes that are part of the Open Cloud Testbed (OCT). Our results indicate that DDR4 DRAM is not automatically cleared following user logout from an allocated node and subsequent node users can easily obtain recognizable data from the DRAM following node reallocation over 17 minutes later. This issue is particularly relevant for systems which support FPGA multi-tenancy.
- [69] arXiv:2408.10376 [pdf, html, other]
-
Title: Self-Play Ensemble Q-learning enabled Resource Allocation for Network SlicingShavbo Salehi, Pedro Enrique Iturria-Rivera, Medhat Elsayed, Majid Bavand, Raimundas Gaigalas, Yigit Ozcan, Melike Erol-KantarciSubjects: Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)
In 5G networks, network slicing has emerged as a pivotal paradigm to address diverse user demands and service requirements. To meet the requirements, reinforcement learning (RL) algorithms have been utilized widely, but this method has the problem of overestimation and exploration-exploitation trade-offs. To tackle these problems, this paper explores the application of self-play ensemble Q-learning, an extended version of the RL-based technique. Self-play ensemble Q-learning utilizes multiple Q-tables with various exploration-exploitation rates leading to different observations for choosing the most suitable action for each state. Moreover, through self-play, each model endeavors to enhance its performance compared to its previous iterations, boosting system efficiency, and decreasing the effect of overestimation. For performance evaluation, we consider three RL-based algorithms; self-play ensemble Q-learning, double Q-learning, and Q-learning, and compare their performance under different network traffic. Through simulations, we demonstrate the effectiveness of self-play ensemble Q-learning in meeting the diverse demands within 21.92% in latency, 24.22% in throughput, and 23.63\% in packet drop rate in comparison with the baseline methods. Furthermore, we evaluate the robustness of self-play ensemble Q-learning and double Q-learning in situations where one of the Q-tables is affected by a malicious user. Our results depicted that the self-play ensemble Q-learning method is more robust against adversarial users and prevents a noticeable drop in system performance, mitigating the impact of users manipulating policies.
- [70] arXiv:2408.10383 [pdf, html, other]
-
Title: BrewCLIP: A Bifurcated Representation Learning Framework for Audio-Visual RetrievalSubjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
Previous methods for audio-image matching generally fall into one of two categories: pipeline models or End-to-End models. Pipeline models first transcribe speech and then encode the resulting text; End-to-End models encode speech directly. Generally, pipeline models outperform end-to-end models, but the intermediate transcription necessarily discards some potentially useful non-textual information. In addition to textual information, speech can convey details such as accent, mood, and and emphasis, which should be effectively captured in the encoded representation. In this paper, we investigate whether non-textual information, which is overlooked by pipeline-based models, can be leveraged to improve speech-image matching performance. We thoroughly analyze and compare End-to-End models, pipeline models, and our proposed dual-channel model for robust audio-image retrieval on a variety of datasets. Our approach achieves a substantial performance gain over the previous state-of-the-art by leveraging strong pretrained models, a prompting mechanism and a bifurcated design.
- [71] arXiv:2408.10388 [pdf, html, other]
-
Title: Narrowing the Gap between Vision and Action in NavigationSubjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
The existing methods for Vision and Language Navigation in the Continuous Environment (VLN-CE) commonly incorporate a waypoint predictor to discretize the environment. This simplifies the navigation actions into a view selection task and improves navigation performance significantly compared to direct training using low-level actions. However, the VLN-CE agents are still far from the real robots since there are gaps between their visual perception and executed actions. First, VLN-CE agents that discretize the visual environment are primarily trained with high-level view selection, which causes them to ignore crucial spatial reasoning within the low-level action movements. Second, in these models, the existing waypoint predictors neglect object semantics and their attributes related to passibility, which can be informative in indicating the feasibility of actions. To address these two issues, we introduce a low-level action decoder jointly trained with high-level action prediction, enabling the current VLN agent to learn and ground the selected visual view to the low-level controls. Moreover, we enhance the current waypoint predictor by utilizing visual representations containing rich semantic information and explicitly masking obstacles based on humans' prior knowledge about the feasibility of actions. Empirically, our agent can improve navigation performance metrics compared to the strong baselines on both high-level and low-level actions.
- [72] arXiv:2408.10390 [pdf, html, other]
-
Title: Self-Refined Generative Foundation Models for Wireless Traffic PredictionSubjects: Systems and Control (eess.SY)
With a broad range of emerging applications in 6G networks, wireless traffic prediction has become a critical component of network management. However, the dynamically shifting distribution of wireless traffic in non-stationary 6G networks presents significant challenges to achieving accurate and stable predictions. Motivated by recent advancements in Generative AI (GAI)-enabled 6G networks, this paper proposes a novel self-refined Large Language Model (LLM) for wireless traffic prediction, namely TrafficLLM, through in-context learning without parameter fine-tuning or model training. The proposed TrafficLLM harnesses the powerful few-shot learning abilities of LLMs to enhance the scalability of traffic prediction in dynamically changing wireless environments. Specifically, our proposed TrafficLLM embraces an LLM to iteratively refine its predictions through a three-step process: traffic prediction, feedback generation, and prediction refinement. Initially, the proposed TrafficLLM conducts traffic predictions using task-specific demonstration prompts. Recognizing that LLMs may generate incorrect predictions on the first attempt, we subsequently incorporate feedback demonstration prompts designed to provide multifaceted and valuable feedback related to these initial predictions. Following this comprehensive feedback, our proposed TrafficLLM introduces refinement demonstration prompts, enabling the same LLM to further refine its predictions and thereby enhance prediction performance. The evaluations on two realistic datasets demonstrate that the proposed TrafficLLM outperforms state-of-the-art methods with performance improvements of 23.17% and 17.09%, respectively.
- [73] arXiv:2408.10391 [pdf, html, other]
-
Title: Tax Credits and Household Behavior: The Roles of Myopic Decision-Making and Liquidity in a Simulated EconomySubjects: Multiagent Systems (cs.MA); General Economics (econ.GN)
There has been a growing interest in multi-agent simulators in the domain of economic modeling. However, contemporary research often involves developing reinforcement learning (RL) based models that focus solely on a single type of agents, such as households, firms, or the government. Such an approach overlooks the adaptation of interacting agents thereby failing to capture the complexity of real-world economic systems. In this work, we consider a multi-agent simulator comprised of RL agents of numerous types, including heterogeneous households, firm, central bank and government. In particular, we focus on the crucial role of the government in distributing tax credits to households. We conduct two broad categories of comprehensive experiments dealing with the impact of tax credits on 1) households with varied degrees of myopia (short-sightedness in spending and saving decisions), and 2) households with diverse liquidity profiles. The first category of experiments examines the impact of the frequency of tax credits (e.g. annual vs quarterly) on consumption patterns of myopic households. The second category of experiments focuses on the impact of varying tax credit distribution strategies on households with differing liquidities. We validate our simulation model by reproducing trends observed in real households upon receipt of unforeseen, uniform tax credits, as documented in a JPMorgan Chase report. Based on the results of the latter, we propose an innovative tax credit distribution strategy for the government to reduce inequality among households. We demonstrate the efficacy of this strategy in improving social welfare in our simulation results.
- [74] arXiv:2408.10392 [pdf, html, other]
-
Title: Value Alignment from Unstructured TextInkit Padhi, Karthikeyan Natesan Ramamurthy, Prasanna Sattigeri, Manish Nagireddy, Pierre Dognin, Kush R. VarshneySubjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Aligning large language models (LLMs) to value systems has emerged as a significant area of research within the fields of AI and NLP. Currently, this alignment process relies on the availability of high-quality supervised and preference data, which can be both time-consuming and expensive to curate or annotate. In this paper, we introduce a systematic end-to-end methodology for aligning LLMs to the implicit and explicit values represented in unstructured text data. Our proposed approach leverages the use of scalable synthetic data generation techniques to effectively align the model to the values present in the unstructured data. Through two distinct use-cases, we demonstrate the efficiency of our methodology on the Mistral-7B-Instruct model. Our approach credibly aligns LLMs to the values embedded within documents, and shows improved performance against other approaches, as quantified through the use of automatic metrics and win rates.
- [75] arXiv:2408.10394 [pdf, html, other]
-
Title: Joint Modeling of Search and Recommendations Via an Unified Contextual Recommender (UniCoRn)Comments: 3 pages, 1 figureSubjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Search and recommendation systems are essential in many services, and they are often developed separately, leading to complex maintenance and technical debt. In this paper, we present a unified deep learning model that efficiently handles key aspects of both tasks.
- [76] arXiv:2408.10395 [pdf, html, other]
-
Title: Evaluating Image-Based Face and Eye Tracking with Event CamerasComments: This paper has been accepted at The Workshop On Neuromorphic Vision: Advantages and Applications of Event Cameras at the European Conference on Computer Vision (ECCV), 2024Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Event Cameras, also known as Neuromorphic sensors, capture changes in local light intensity at the pixel level, producing asynchronously generated data termed ``events''. This distinct data format mitigates common issues observed in conventional cameras, like under-sampling when capturing fast-moving objects, thereby preserving critical information that might otherwise be lost. However, leveraging this data often necessitates the development of specialized, handcrafted event representations that can integrate seamlessly with conventional Convolutional Neural Networks (CNNs), considering the unique attributes of event data. In this study, We evaluate event-based Face and Eye tracking. The core objective of our study is to showcase the viability of integrating conventional algorithms with event-based data, transformed into a frame format while preserving the unique benefits of event cameras. To validate our approach, we constructed a frame-based event dataset by simulating events between RGB frames derived from the publicly accessible Helen Dataset. We assess its utility for face and eye detection tasks through the application of GR-YOLO -- a pioneering technique derived from YOLOv3. This evaluation includes a comparative analysis with results derived from training the dataset with YOLOv8. Subsequently, the trained models were tested on real event streams from various iterations of Prophesee's event cameras and further evaluated on the Faces in Event Stream (FES) benchmark dataset. The models trained on our dataset shows a good prediction performance across all the datasets obtained for validation with the best results of a mean Average precision score of 0.91. Additionally, The models trained demonstrated robust performance on real event camera data under varying light conditions.
- [77] arXiv:2408.10397 [pdf, html, other]
-
Title: Webcam-based Pupil Diameter Prediction Benefits from UpscalingSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)
Capturing pupil diameter is essential for assessing psychological and physiological states such as stress levels and cognitive load. However, the low resolution of images in eye datasets often hampers precise measurement. This study evaluates the impact of various upscaling methods, ranging from bicubic interpolation to advanced super-resolution, on pupil diameter predictions. We compare several pre-trained methods, including CodeFormer, GFPGAN, Real-ESRGAN, HAT, and SRResNet. Our findings suggest that pupil diameter prediction models trained on upscaled datasets are highly sensitive to the selected upscaling method and scale. Our results demonstrate that upscaling methods consistently enhance the accuracy of pupil diameter prediction models, highlighting the importance of upscaling in pupilometry. Overall, our work provides valuable insights for selecting upscaling techniques, paving the way for more accurate assessments in psychological and physiological research.
- [78] arXiv:2408.10404 [pdf, html, other]
-
Title: Parallel Processing of Point Cloud Ground Segmentation for Mechanical and Solid-State LiDARsComments: 5 pagesSubjects: Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV); Signal Processing (eess.SP)
In this study, we introduce a novel parallel processing framework for real-time point cloud ground segmentation on FPGA platforms, aimed at adapting LiDAR algorithms to the evolving landscape from mechanical to solid-state LiDAR (SSL) technologies. Focusing on the ground segmentation task, we explore parallel processing techniques on existing approaches and adapt them to real-world SSL data handling. We validated frame-segmentation based parallel processing methods using point-based, voxel-based, and range-image-based ground segmentation approaches on the SemanticKITTI dataset based on mechanical LiDAR. The results revealed the superior performance and robustness of the range-image method, especially in its resilience to slicing. Further, utilizing a custom dataset from our self-built Camera-SSLSS equipment, we examined regular SSL data frames and validated the effectiveness of our parallel approach for SSL sensor. Additionally, our pioneering implementation of range-image ground segmentation on FPGA for SSL sensors demonstrated significant processing speed improvements and resource efficiency, achieving processing rates up to 50.3 times faster than conventional CPU setups. These findings underscore the potential of parallel processing strategies to significantly enhance LiDAR technologies for advanced perception tasks in autonomous systems. Post-publication, both the data and the code will be made available on GitHub.
- [79] arXiv:2408.10405 [pdf, html, other]
-
Title: ROOT: Requirements Organization and Optimization ToolSubjects: Software Engineering (cs.SE)
Software engineering practices such as constructing requirements and establishing traceability help ensure systems are safe, reliable, and maintainable. However, they can be resource-intensive and are frequently underutilized. To alleviate the burden of these essential processes, we developed the Requirements Organization and Optimization Tool (ROOT). ROOT centralizes project information and offers project visualizations and AI-based tools designed to streamline engineering processes. With ROOT's assistance, engineers benefit from improved oversight and early error detection, leading to the successful development of software systems. Link to screen cast: this https URL
- [80] arXiv:2408.10409 [pdf, html, other]
-
Title: Real-Time Digital Twin Platform: A Case Study on Core Network Selection in Aeronautical Ad-Hoc NetworksSubjects: Networking and Internet Architecture (cs.NI); Emerging Technologies (cs.ET)
The development of Digital Twins (DTs) is hindered by a lack of specialized, open-source solutions that can meet the demands of dynamic applications. This has caused state-of-the-art DT applications to be validated using offline data. However, this approach falls short of integrating real-time data, which is one of the most important characteristics of DTs. This can limit the validating effectiveness of DT applications in cases such as aeronautical ad-hoc networks (AANETs). Considering this, we develop a Real-Time Digital Twin Platform and implement core network selection in AANETs as a case study. In this, we implement microservice-based architecture and design a robust data pipeline. Additionally, we develop an interactive user interface using open-source tools. Using this, the platform supports real-time decision-making in the presence of data retrieval failures.
- [81] arXiv:2408.10411 [pdf, html, other]
-
Title: Resolving Lexical Bias in Edit Scoping with Projector Editor NetworksSubjects: Computation and Language (cs.CL)
Weight-preserving model editing techniques heavily rely on the scoping mechanism that decides when to apply an edit to the base model. These scoping mechanisms utilize distance functions in the representation space to ascertain the scope of the edit. In this work, we show that distance-based scoping functions grapple with lexical biases leading to issues such as misfires with irrelevant prompts that share similar lexical characteristics. To address this problem, we introduce, Projector Editor Networks for Model Editing (PENME),is a model editing approach that employs a compact adapter with a projection network trained via a contrastive learning objective. We demonstrate the efficacy of PENME in achieving superior results while being compute efficient and flexible to adapt across model architectures.
- [82] arXiv:2408.10414 [pdf, html, other]
-
Title: Towards Automation of Human Stage of Decay Identification: An Artificial Intelligence ApproachComments: 13 pagesSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Determining the stage of decomposition (SOD) is crucial for estimating the postmortem interval and identifying human remains. Currently, labor-intensive manual scoring methods are used for this purpose, but they are subjective and do not scale for the emerging large-scale archival collections of human decomposition photos. This study explores the feasibility of automating two common human decomposition scoring methods proposed by Megyesi and Gelderman using artificial intelligence (AI). We evaluated two popular deep learning models, Inception V3 and Xception, by training them on a large dataset of human decomposition images to classify the SOD for different anatomical regions, including the head, torso, and limbs. Additionally, an interrater study was conducted to assess the reliability of the AI models compared to human forensic examiners for SOD identification. The Xception model achieved the best classification performance, with macro-averaged F1 scores of .878, .881, and .702 for the head, torso, and limbs when predicting Megyesi's SODs, and .872, .875, and .76 for the head, torso, and limbs when predicting Gelderman's SODs. The interrater study results supported AI's ability to determine the SOD at a reliability level comparable to a human expert. This work demonstrates the potential of AI models trained on a large dataset of human decomposition images to automate SOD identification.
- [83] arXiv:2408.10417 [pdf, other]
-
Title: Development of an AI Anti-Bullying System Using Large Language Model Key Topic DetectionMatthew Tassava, Cameron Kolodjski, Jordan Milbrath, Adorah Bishop, Nathan Flanders, Robbie Fetsch, Danielle Hanson, Jeremy StraubSubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
This paper presents and evaluates work on the development of an artificial intelligence (AI) anti-bullying system. The system is designed to identify coordinated bullying attacks via social media and other mechanisms, characterize them and propose remediation and response activities to them. In particular, a large language model (LLM) is used to populate an enhanced expert system-based network model of a bullying attack. This facilitates analysis and remediation activity - such as generating report messages to social media companies - determination. The system is described and the efficacy of the LLM for populating the model is analyzed herein.
- [84] arXiv:2408.10419 [pdf, html, other]
-
Title: Second-Order Forward-Mode Automatic Differentiation for OptimizationComments: 14 pages, 8 figuresSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
This paper introduces a second-order hyperplane search, a novel optimization step that generalizes a second-order line search from a line to a $k$-dimensional hyperplane. This, combined with the forward-mode stochastic gradient method, yields a second-order optimization algorithm that consists of forward passes only, completely avoiding the storage overhead of backpropagation. Unlike recent work that relies on directional derivatives (or Jacobian--Vector Products, JVPs), we use hyper-dual numbers to jointly evaluate both directional derivatives and their second-order quadratic terms. As a result, we introduce forward-mode weight perturbation with Hessian information (FoMoH). We then use FoMoH to develop a novel generalization of line search by extending it to a hyperplane search. We illustrate the utility of this extension and how it might be used to overcome some of the recent challenges of optimizing machine learning models without backpropagation. Our code is open-sourced at this https URL.
- [85] arXiv:2408.10428 [pdf, html, other]
-
Title: Are LLMs Any Good for High-Level Synthesis?Comments: ICCAD '24 Special Session on AI4HLS: New Frontiers in High-Level Synthesis Augmented with Artificial IntelligenceSubjects: Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI)
The increasing complexity and demand for faster, energy-efficient hardware designs necessitate innovative High-Level Synthesis (HLS) methodologies. This paper explores the potential of Large Language Models (LLMs) to streamline or replace the HLS process, leveraging their ability to understand natural language specifications and refactor code. We survey the current research and conduct experiments comparing Verilog designs generated by a standard HLS tool (Vitis HLS) with those produced by LLMs translating C code or natural language specifications. Our evaluation focuses on quantifying the impact on performance, power, and resource utilization, providing an assessment of the efficiency of LLM-based approaches. This study aims to illuminate the role of LLMs in HLS, identifying promising directions for optimized hardware design in applications such as AI acceleration, embedded systems, and high-performance computing.
- [86] arXiv:2408.10429 [pdf, html, other]
-
Title: Price Competition Under A Consider-Then-Choose Model With Lexicographic ChoiceSubjects: Computer Science and Game Theory (cs.GT)
The sorting and filtering capabilities offered by modern e-commerce platforms significantly impact customers' purchase decisions, as well as the resulting prices set by competing sellers on these platforms. Motivated by this practical reality, we study price competition under a flexible choice model: Consider-then-Choose with Lexicographic Choice (CLC). In this model, a customer first forms a consideration set of sellers based on (i) her willingness-to-pay and (ii) an arbitrary set of criteria on items' non-price attributes; she then chooses the highest-ranked item according to a lexicographic ranking in which items with better performance on more important attributes are ranked higher. We provide a structural characterization of equilibria in the resulting game of price competition, and derive an economically interpretable condition, which we call gradient dominance, under which equilibria can be computed efficiently. For this subclass of CLC models, we prove that distributed gradient-based pricing dynamics converge to the set of equilibria. Extensive numerical experiments show robustness of our theoretical findings when gradient dominance does not hold.
- [87] arXiv:2408.10431 [pdf, html, other]
-
Title: System-Level Design Space Exploration for High-Level Synthesis under End-to-End Latency ConstraintsSubjects: Hardware Architecture (cs.AR)
Many modern embedded systems have end-to-end (EtoE) latency constraints that necessitate precise timing to ensure high reliability and functional correctness. The combination of High-Level Synthesis (HLS) and Design Space Exploration (DSE) enables the rapid generation of embedded systems using various constraints/directives to find Pareto-optimal configurations. Current HLS DSE approaches often address latency by focusing on individual components, without considering the EtoE latency during the system-level optimization process. However, to truly optimize the system under EtoE latency, we need a holistic approach that analyzes individual system components' timing constraints in the context of how the different components interact and impact the overall design. This paper presents a novel system-level HLS DSE approach, called EtoE-DSE, that accommodates EtoE latency and variable timing constraints for complex multi-component application-specific embedded systems. EtoE-DSE employs a latency estimation model and a pathfinding algorithm to identify and estimate the EtoE latency for paths between any endpoints. It also uses a frequency-based segmentation process to segment and prune the design space, alongside a latency-constrained optimization algorithm for efficiently and accurately exploring the system-level design space. We evaluate our approach using a real-world use case of an autonomous driving subsystem compared to the state-of-the-art in HLS DSE. We show that our approach yields substantially better optimization results than prior DSE approaches, improving the quality of results by up to 89.26%, while efficiently identifying Pareto-optimal configurations in terms of energy and area.
- [88] arXiv:2408.10433 [pdf, html, other]
-
Title: CLIP-DPO: Vision-Language Models as a Source of Preference for Fixing Hallucinations in LVLMsComments: Accepted at ECCV 2024Subjects: Computer Vision and Pattern Recognition (cs.CV)
Despite recent successes, LVLMs or Large Vision Language Models are prone to hallucinating details like objects and their properties or relations, limiting their real-world deployment. To address this and improve their robustness, we present CLIP-DPO, a preference optimization method that leverages contrastively pre-trained Vision-Language (VL) embedding models, such as CLIP, for DPO-based optimization of LVLMs. Unlike prior works tackling LVLM hallucinations, our method does not rely on paid-for APIs, and does not require additional training data or the deployment of other external LVLMs. Instead, starting from the initial pool of supervised fine-tuning data, we generate a diverse set of predictions, which are ranked based on their CLIP image-text similarities, and then filtered using a robust rule-based approach to obtain a set of positive and negative pairs for DPO-based training. We applied CLIP-DPO fine-tuning to the MobileVLM-v2 family of models and to LlaVA-1.5, in all cases observing significant improvements in terms of hallucination reduction over baseline models. We also observe better performance for zero-shot classification, suggesting improved grounding capabilities, and verify that the original performance on standard LVLM benchmarks is overall preserved.
- [89] arXiv:2408.10434 [pdf, html, other]
-
Title: Insights on Microservice Architecture Through the Eyes of Industry PractitionersVinicius L. Nogueira, Fernando S. Felizardo, Aline M. M. M. Amaral, Wesley K. G. Assuncao, Thelma E. ColanziSubjects: Software Engineering (cs.SE)
The adoption of microservice architecture has seen a considerable upswing in recent years, mainly driven by the need to modernize legacy systems and address their limitations. Legacy systems, typically designed as monolithic applications, often struggle with maintenance, scalability, and deployment inefficiencies. This study investigates the motivations, activities, and challenges associated with migrating from monolithic legacy systems to microservices, aiming to shed light on common practices and challenges from a practitioner's point of view. We conducted a comprehensive study with 53 software practitioners who use microservices, expanding upon previous research by incorporating diverse international perspectives. Our mixed-methods approach includes quantitative and qualitative analyses, focusing on four main aspects: (i) the driving forces behind migration, (ii) the activities to conduct the migration, (iii) strategies for managing data consistency, and (iv) the prevalent challenges. Thus, our results reveal diverse practices and challenges practitioners face when migrating to microservices. Companies are interested in technical benefits, enhancing maintenance, scalability, and deployment processes. Testing in microservice environments remains complex, and extensive monitoring is crucial to managing the dynamic nature of microservices. Database management remains challenging. While most participants prefer decentralized databases for autonomy and scalability, challenges persist in ensuring data consistency. Additionally, many companies leverage modern cloud technologies to mitigate network overhead, showcasing the importance of cloud infrastructure in facilitating efficient microservice communication.
- [90] arXiv:2408.10435 [pdf, html, other]
-
Title: Enhanced document retrieval with topic embeddingsComments: Accepted to AICT 2024Subjects: Information Retrieval (cs.IR)
Document retrieval systems have experienced a revitalized interest with the advent of retrieval-augmented generation (RAG). RAG architecture offers a lower hallucination rate than LLM-only applications. However, the accuracy of the retrieval mechanism is known to be a bottleneck in the efficiency of these applications. A particular case of subpar retrieval performance is observed in situations where multiple documents from several different but related topics are in the corpus. We have devised a new vectorization method that takes into account the topic information of the document. The paper introduces this new method for text vectorization and evaluates it in the context of RAG. Furthermore, we discuss the challenge of evaluating RAG systems, which pertains to the case at hand.
- [91] arXiv:2408.10436 [pdf, html, other]
-
Title: Learning Regularization for Graph Inverse ProblemsSubjects: Machine Learning (cs.LG)
In recent years, Graph Neural Networks (GNNs) have been utilized for various applications ranging from drug discovery to network design and social networks. In many applications, it is impossible to observe some properties of the graph directly; instead, noisy and indirect measurements of these properties are available. These scenarios are coined as Graph Inverse Problems (GRIP). In this work, we introduce a framework leveraging GNNs to solve GRIPs. The framework is based on a combination of likelihood and prior terms, which are used to find a solution that fits the data while adhering to learned prior information. Specifically, we propose to combine recent deep learning techniques that were developed for inverse problems, together with GNN architectures, to formulate and solve GRIP. We study our approach on a number of representative problems that demonstrate the effectiveness of the framework.
- [92] arXiv:2408.10437 [pdf, html, other]
-
Title: Understanding Generative AI Content with Embedding ModelsSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
The construction of high-quality numerical features is critical to any quantitative data analysis. Feature engineering has been historically addressed by carefully hand-crafting data representations based on domain expertise. This work views the internal representations of modern deep neural networks (DNNs), called embeddings, as an automated form of traditional feature engineering. For trained DNNs, we show that these embeddings can reveal interpretable, high-level concepts in unstructured sample data. We use these embeddings in natural language and computer vision tasks to uncover both inherent heterogeneity in the underlying data and human-understandable explanations for it. In particular, we find empirical evidence that there is inherent separability between real data and that generated from AI models.
- [93] arXiv:2408.10438 [pdf, html, other]
-
Title: Private Means and the Curious Incident of the Free LunchComments: TPDP24Subjects: Cryptography and Security (cs.CR)
We show that the most well-known and fundamental building blocks of DP implementations -- sum, mean, count (and many other linear queries) -- can be released with substantially reduced noise for the same privacy guarantee. We achieve this by projecting individual data with worst-case sensitivity $R$ onto a simplex where all data now has a constant norm $R$. In this simplex, additional ``free'' queries can be run that are already covered by the privacy-loss of the original budgeted query, and which algebraically give additional estimates of counts or sums, and can be combined for lower final noise.
- [94] arXiv:2408.10439 [pdf, html, other]
-
Title: Visual Storytelling: A Methodological Approach to Designing and Implementing a Visualisation PosterComments: 5 pages, 1 figure, accepted for publication to the EG UK Computer Graphics & Visual Computing (CGVC) 2024Subjects: Human-Computer Interaction (cs.HC); Graphics (cs.GR)
We present a design study of developing a visualisation poster. Posters can be difficult to create, and the story on a poster is not always clear. Using a case-study approach we propose three important aspects: the poster should have a clear focus (especially a hero visualisation), envisioning its use helps to drive the important aspects, and third the essence (its fundamental concept and guiding idea) must be clear. We will use case studies that have focused on the use of the Five Design-Sheet method (FdS) as a way to sketch and plan a visualisation, before successfully implementing and creating the visual poster. The case studies serve as a practical illustration of the workflow, offering a means to explain the three key processes involved: (1) comprehending the data, (2) employing a design study with the FdS (Five Design-Sheet), (3) crafting, evaluating and refining the visualisation.
- [95] arXiv:2408.10441 [pdf, html, other]
-
Title: Goldfish: Monolingual Language Models for 350 LanguagesSubjects: Computation and Language (cs.CL)
For many low-resource languages, the only available language models are large multilingual models trained on many languages simultaneously. However, using FLORES perplexity as a metric, we find that these models perform worse than bigrams for many languages (e.g. 24% of languages in XGLM 4.5B; 43% in BLOOM 7.1B). To facilitate research that focuses on low-resource languages, we pre-train and release Goldfish, a suite of monolingual autoregressive Transformer language models up to 125M parameters for 350 languages. The Goldfish reach lower FLORES perplexities than BLOOM, XGLM, and MaLA-500 on 98 of 204 FLORES languages, despite each Goldfish model being over 10x smaller. However, the Goldfish significantly underperform larger multilingual models on reasoning benchmarks, suggesting that for low-resource languages, multilinguality primarily improves general reasoning abilities rather than basic text generation. We release models trained on 5MB (350 languages), 10MB (288 languages), 100MB (166 languages), and 1GB (83 languages) of text data where available. The Goldfish models are available as baselines, fine-tuning sources, or augmentations to existing models in low-resource NLP research, and they are further useful for crosslinguistic studies requiring maximally comparable models across languages.
- [96] arXiv:2408.10442 [pdf, html, other]
-
Title: Feasibility of assessing cognitive impairment via distributed camera network and privacy-preserving edge computingSubjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
INTRODUCTION: Mild cognitive impairment (MCI) is characterized by a decline in cognitive functions beyond typical age and education-related expectations. Since, MCI has been linked to reduced social interactions and increased aimless movements, we aimed to automate the capture of these behaviors to enhance longitudinal monitoring.
METHODS: Using a privacy-preserving distributed camera network, we collected movement and social interaction data from groups of individuals with MCI undergoing therapy within a 1700$m^2$ space. We developed movement and social interaction features, which were then used to train a series of machine learning algorithms to distinguish between higher and lower cognitive functioning MCI groups.
RESULTS: A Wilcoxon rank-sum test revealed statistically significant differences between high and low-functioning cohorts in features such as linear path length, walking speed, change in direction while walking, entropy of velocity and direction change, and number of group formations in the indoor space. Despite lacking individual identifiers to associate with specific levels of MCI, a machine learning approach using the most significant features provided a 71% accuracy.
DISCUSSION: We provide evidence to show that a privacy-preserving low-cost camera network using edge computing framework has the potential to distinguish between different levels of cognitive impairment from the movements and social interactions captured during group activities. - [97] arXiv:2408.10443 [pdf, html, other]
-
Title: Federated Learning of Large ASR Models in the Real WorldSubjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Federated learning (FL) has shown promising results on training machine learning models with privacy preservation. However, for large models with over 100 million parameters, the training resource requirement becomes an obstacle for FL because common devices do not have enough memory and computation power to finish the FL tasks. Although efficient training methods have been proposed, it is still a challenge to train the large models like Conformer based ASR. This paper presents a systematic solution to train the full-size ASR models of 130M parameters with FL. To our knowledge, this is the first real-world FL application of the Conformer model, which is also the largest model ever trained with FL so far. And this is the first paper showing FL can improve the ASR model quality with a set of proposed methods to refine the quality of data and labels of clients. We demonstrate both the training efficiency and the model quality improvement in real-world experiments.
- [98] arXiv:2408.10446 [pdf, other]
-
Title: The Brittleness of AI-Generated Image Watermarking Techniques: Examining Their Robustness Against Visual Paraphrasing AttacksNiyar R Barman, Krish Sharma, Ashhar Aziz, Shashwat Bajpai, Shwetangshu Biswas, Vasu Sharma, Vinija Jain, Aman Chadha, Amit Sheth, Amitava DasComments: 23 pages and 10 figuresSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
The rapid advancement of text-to-image generation systems, exemplified by models like Stable Diffusion, Midjourney, Imagen, and DALL-E, has heightened concerns about their potential misuse. In response, companies like Meta and Google have intensified their efforts to implement watermarking techniques on AI-generated images to curb the circulation of potentially misleading visuals. However, in this paper, we argue that current image watermarking methods are fragile and susceptible to being circumvented through visual paraphrase attacks. The proposed visual paraphraser operates in two steps. First, it generates a caption for the given image using KOSMOS-2, one of the latest state-of-the-art image captioning systems. Second, it passes both the original image and the generated caption to an image-to-image diffusion system. During the denoising step of the diffusion pipeline, the system generates a visually similar image that is guided by the text caption. The resulting image is a visual paraphrase and is free of any watermarks. Our empirical findings demonstrate that visual paraphrase attacks can effectively remove watermarks from images. This paper provides a critical assessment, empirically revealing the vulnerability of existing watermarking techniques to visual paraphrase attacks. While we do not propose solutions to this issue, this paper serves as a call to action for the scientific community to prioritize the development of more robust watermarking techniques. Our first-of-its-kind visual paraphrase dataset and accompanying code are publicly available.
- [99] arXiv:2408.10450 [pdf, html, other]
-
Title: RUMI: Rummaging Using Mutual InformationComments: 19 pages, 17 figures, submitted to IEEE Transactions on Robotics (T-RO)Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI)
This paper presents Rummaging Using Mutual Information (RUMI), a method for online generation of robot action sequences to gather information about the pose of a known movable object in visually-occluded environments. Focusing on contact-rich rummaging, our approach leverages mutual information between the object pose distribution and robot trajectory for action planning. From an observed partial point cloud, RUMI deduces the compatible object pose distribution and approximates the mutual information of it with workspace occupancy in real time. Based on this, we develop an information gain cost function and a reachability cost function to keep the object within the robot's reach. These are integrated into a model predictive control (MPC) framework with a stochastic dynamics model, updating the pose distribution in a closed loop. Key contributions include a new belief framework for object pose estimation, an efficient information gain computation strategy, and a robust MPC-based control scheme. RUMI demonstrates superior performance in both simulated and real tasks compared to baseline methods.
- [100] arXiv:2408.10453 [pdf, html, other]
-
Title: Kubrick: Multimodal Agent Collaborations for Synthetic Video GenerationSubjects: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR); Multimedia (cs.MM)
Text-to-video generation has been dominated by end-to-end diffusion-based or autoregressive models. On one hand, those novel models provide plausible versatility, but they are criticized for physical correctness, shading and illumination, camera motion, and temporal consistency. On the other hand, film industry relies on manually-edited Computer-Generated Imagery (CGI) using 3D modeling software. Human-directed 3D synthetic videos and animations address the aforementioned shortcomings, but it is extremely tedious and requires tight collaboration between movie makers and 3D rendering experts. In this paper, we introduce an automatic synthetic video generation pipeline based on Vision Large Language Model (VLM) agent collaborations. Given a natural language description of a video, multiple VLM agents auto-direct various processes of the generation pipeline. They cooperate to create Blender scripts which render a video that best aligns with the given description. Based on film making inspiration and augmented with Blender-based movie making knowledge, the Director agent decomposes the input text-based video description into sub-processes. For each sub-process, the Programmer agent produces Python-based Blender scripts based on customized function composing and API calling. Then, the Reviewer agent, augmented with knowledge of video reviewing, character motion coordinates, and intermediate screenshots uses its compositional reasoning ability to provide feedback to the Programmer agent. The Programmer agent iteratively improves the scripts to yield the best overall video outcome. Our generated videos show better quality than commercial video generation models in 5 metrics on video quality and instruction-following performance. Moreover, our framework outperforms other approaches in a comprehensive user study on quality, consistency, and rationality.
- [101] arXiv:2408.10454 [pdf, html, other]
-
Title: Likelihood Scouting Via Map Inversion For A Posterior-Sampled Particle FilterComments: 17 Pages, 12 FiguresSubjects: Information Theory (cs.IT)
An exploit of the Sequential Importance Sampling (SIS) algorithm using Differential Algebra (DA) techniques is derived to develop an efficient particle filter. The filter creates an original kind of particles, called scout particles, that bring information from the measurement noise onto the state prior probability density function. Thanks to the creation of high-order polynomial maps and their inversions, the scouting of the measurements helps the SIS algorithm identify the region of the prior more affected by the likelihood distribution. The result of the technique is two different versions of the proposed Scout Particle Filter (SPF), which identifies and delimits the region where the true posterior probability has high density in the SIS algorithm. Four different numerical applications show the benefits of the methodology both in terms of accuracy and efficiency, where the SPF is compared to other particle filters, with a particular focus on target tracking and orbit determination problems.
- [102] arXiv:2408.10455 [pdf, html, other]
-
Title: IDEA: Enhancing the rule learning ability of language agent through Induction, DEuction, and AbductionComments: 9pages, 12 figs, 4 tablesSubjects: Artificial Intelligence (cs.AI)
While large language models (LLMs) have been thoroughly evaluated for deductive and inductive reasoning, their proficiency in abductive reasoning and holistic rule learning in interactive environments remains less explored. This work introduces RULEARN, a novel benchmark specifically designed to assess the rule-learning ability of LLMs in interactive settings. In RULEARN, agents interact with the environment to gather observations and discern patterns, using these insights to solve problems. To further enhance the rule-learning capabilities of LLM agents within this benchmark, we propose IDEA agent, which integrates Induction, Deduction, and Abduction processes. IDEA agent refines this approach by leveraging a structured reasoning sequence: generating hypotheses through abduction, testing them via deduction, and refining them based on induction feedback. This sequence enables agents to dynamically establish and apply rules, mimicking human-like reasoning processes. Our evaluation of five representative LLMs indicates that while these models can generate plausible initial hypotheses, they often struggle with strategic interaction within the environment, effective incorporation of feedback, and adaptive refinement of their hypotheses. IDEA agent demonstrates significantly improved performance on the RULEARN benchmark, offering valuable insights for the development of agents capable of human-like rule-learning in real-world scenarios. We will release our code and data.
- [103] arXiv:2408.10456 [pdf, html, other]
-
Title: Differentially Private Stochastic Gradient Descent with Fixed-Size Minibatches: Tighter RDP Guarantees with or without ReplacementComments: 39 pages, 10 figuresSubjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR)
Differentially private stochastic gradient descent (DP-SGD) has been instrumental in privately training deep learning models by providing a framework to control and track the privacy loss incurred during training. At the core of this computation lies a subsampling method that uses a privacy amplification lemma to enhance the privacy guarantees provided by the additive noise. Fixed size subsampling is appealing for its constant memory usage, unlike the variable sized minibatches in Poisson subsampling. It is also of interest in addressing class imbalance and federated learning. However, the current computable guarantees for fixed-size subsampling are not tight and do not consider both add/remove and replace-one adjacency relationships. We present a new and holistic R{é}nyi differential privacy (RDP) accountant for DP-SGD with fixed-size subsampling without replacement (FSwoR) and with replacement (FSwR). For FSwoR we consider both add/remove and replace-one adjacency. Our FSwoR results improves on the best current computable bound by a factor of $4$. We also show for the first time that the widely-used Poisson subsampling and FSwoR with replace-one adjacency have the same privacy to leading order in the sampling probability. Accordingly, our work suggests that FSwoR is often preferable to Poisson subsampling due to constant memory usage. Our FSwR accountant includes explicit non-asymptotic upper and lower bounds and, to the authors' knowledge, is the first such analysis of fixed-size RDP with replacement for DP-SGD. We analytically and empirically compare fixed size and Poisson subsampling, and show that DP-SGD gradients in a fixed-size subsampling regime exhibit lower variance in practice in addition to memory usage benefits.
- [104] arXiv:2408.10457 [pdf, html, other]
-
Title: Parkinson's Disease Classification via EEG: All You Need is a Single Convolutional LayerComments: 10 pages, 5 figuresSubjects: Machine Learning (cs.LG); Signal Processing (eess.SP)
In this work, we introduce LightCNN, a minimalist Convolutional Neural Network (CNN) architecture designed for Parkinson's disease (PD) classification using EEG data. LightCNN's strength lies in its simplicity, utilizing just a single convolutional layer. Embracing Leonardo da Vinci's principle that "simplicity is the ultimate sophistication," LightCNN demonstrates that complexity is not required to achieve outstanding results. We benchmarked LightCNN against several state-of-the-art deep learning models known for their effectiveness in EEG-based PD classification. Remarkably, LightCNN outperformed all these complex architectures, with a 2.3% improvement in recall, a 4.6% increase in precision, a 0.1% edge in AUC, a 4% boost in F1-score, and a 3.3% higher accuracy compared to the closest competitor. Furthermore, LightCNN identifies known pathological brain rhythms associated with PD and effectively captures clinically relevant neurophysiological changes in EEG. Its simplicity and interpretability make it ideal for deployment in resource-constrained environments, such as mobile or embedded systems for EEG analysis. In conclusion, LightCNN represents a significant step forward in efficient EEG-based PD classification, demonstrating that a well-designed, lightweight model can achieve superior performance over more complex architectures. This work underscores the potential for minimalist models to meet the needs of modern healthcare applications, particularly where resources are limited.
- [105] arXiv:2408.10458 [pdf, html, other]
-
Title: Transfer Operator Learning with Fusion FrameSubjects: Machine Learning (cs.LG)
The challenge of applying learned knowledge from one domain to solve problems in another related but distinct domain, known as transfer learning, is fundamental in operator learning models that solve Partial Differential Equations (PDEs). These current models often struggle with generalization across different tasks and datasets, limiting their applicability in diverse scientific and engineering disciplines. This work presents a novel framework that enhances the transfer learning capabilities of operator learning models for solving Partial Differential Equations (PDEs) through the integration of fusion frame theory with the Proper Orthogonal Decomposition (POD)-enhanced Deep Operator Network (DeepONet). We introduce an innovative architecture that combines fusion frames with POD-DeepONet, demonstrating superior performance across various PDEs in our experimental analysis. Our framework addresses the critical challenge of transfer learning in operator learning models, paving the way for adaptable and efficient solutions across a wide range of scientific and engineering applications.
- [106] arXiv:2408.10459 [pdf, other]
-
Title: Distinct Numerical Solutions for Elliptic Cross-Interface Problems Using Finite Element and Finite Difference MethodsComments: 19 pages, 12 figuresSubjects: Numerical Analysis (math.NA)
In this paper, we discuss the second-order finite element method (FEM) and finite difference method (FDM) for numerically solving elliptic cross-interface problems characterized by vertical and horizontal straight lines, piecewise constant coefficients, two homogeneous jump conditions, continuous source terms, and Dirichlet boundary conditions. For brevity, we consider a 2D simplified version where the intersection points of the interface lines coincide with grid points in uniform Cartesian grids. Our findings reveal interesting and important results: (1) When the coefficient functions exhibit either high jumps with low-frequency oscillations or low jumps with high-frequency oscillations, the finite element method and finite difference method yield similar numerical solutions. (2) However, when the interface problems involve high-contrast and high-frequency coefficient functions, the numerical solutions obtained from the finite element and finite difference methods differ significantly. Given that the widely studied SPE10 benchmark problem (see this https URL) typically involves high-contrast and high-frequency permeability due to varying geological layers in porous media, this phenomenon warrants attention. Furthermore, this observation is particularly important for developing multiscale methods, as reference solutions for these methods are usually obtained using the standard second-order finite element method with a fine mesh, and analytical solutions are not available. We provide sufficient details to enable replication of our numerical results, and the implementation is straightforward. This simplicity ensures that readers can easily confirm the validity of our findings.
- [107] arXiv:2408.10461 [pdf, other]
-
Title: Low Profile Metamaterial Band-Pass Filter Loaded with 4-Turn Complementary Spiral Resonator for WPT ApplicationsSubjects: Systems and Control (eess.SY)
In this paper, a very compact and low insertion loss metamaterial band-pass filter (MBPF) at the center frequency of f0=730 MHz is proposed, based on the rectangular-shape 4 turn complementary spiral resonators (4 CSR). The proposed MBPF consists of an interdigital capacitor as a series capacitance in the top layer, leading to improve the stopband performance in the pass band range of 700 to 760 MHz, which makes it suitable for wireless power transfer (WPT) systems by rejecting unwanted signals. In order to validate the performance of the proposed technique, the MBPF is fabricated on the RO-4003 substrate and great agreement is achieved between simulated and measured results. The stop band attenuations of greater than 52 dB and 20 dB are obtained around the 0.8xfcl (lower cutoff frequency) and 1.2xfcu (upper cutoff frequency), respectively.
- [108] arXiv:2408.10462 [pdf, other]
-
Title: High-Sensitivity and Compact Time-domain Soil Moisture Sensor Using Dispersive Phase Shifter for Complex Permittivity MeasurementSubjects: Systems and Control (eess.SY)
This paper presents a Time-Domain Transmissometry Soil Moisture Sensor (TDT-SMS) using a Dispersive Phase Shifter (DPS), consisting of an interdigital capacitor that is loaded with a stacked 4-turn Complementary Spiral Resonator (S4-CSR). Soil moisture measurement technique of the proposed sensor is based on the complex permittivity sensing property of a DPS in time domain. Soil relative permittivity which varies with its moisture content is measured by burying the DPS under a soil mass and changing its phase difference while excited with a 114 MHz sine wave (single tone). DPS output phase and magnitude are compared with the reference signal and measured with a phase/loss detector. The proposed sensor exhibits accuracy better than +-1.2 percent at the highest Volumetric Water Content (VWC=30 percent) for sandy-type soil. Precise design guide is developed and simulations are performed to achieve a highly sensitive sensor. The measurement results validate the accuracy of theoretical analysis and design procedure. Owning the advantages of low profile, low power consumption, and high sensitivity makes the proposed TDT-SMS a good candidate for precision farming and IoT systems.
- [109] arXiv:2408.10463 [pdf, html, other]
-
Title: Adversarial training of Keyword Spotting to Minimize TTS Data OverfittingHyun Jin Park, Dhruuv Agarwal, Neng Chen, Rentao Sun, Kurt Partridge, Justin Chen, Harry Zhang, Pai Zhu, Jacob Bartel, Kyle Kastner, Gary Wang, Andrew Rosenberg, Quan WangComments: to be published in a Workshop at Interspeech 2024, Synthetic Data's Transformative Role in Foundational Speech ModelsSubjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
The keyword spotting (KWS) problem requires large amounts of real speech training data to achieve high accuracy across diverse populations. Utilizing large amounts of text-to-speech (TTS) synthesized data can reduce the cost and time associated with KWS development. However, TTS data may contain artifacts not present in real speech, which the KWS model can exploit (overfit), leading to degraded accuracy on real speech. To address this issue, we propose applying an adversarial training method to prevent the KWS model from learning TTS-specific features when trained on large amounts of TTS data. Experimental results demonstrate that KWS model accuracy on real speech data can be improved by up to 12% when adversarial loss is used in addition to the original KWS loss. Surprisingly, we also observed that the adversarial setup improves accuracy by up to 8%, even when trained solely on TTS and real negative speech data, without any real positive examples.
- [110] arXiv:2408.10464 [pdf, html, other]
-
Title: Improved Community Detection using Stochastic Block ModelsSubjects: Social and Information Networks (cs.SI)
Community detection approaches resolve complex networks into smaller groups (communities) that are expected to be relatively edge-dense and well-connected. The stochastic block model (SBM) is one of several approaches used to uncover community structure in graphs. In this study, we demonstrate that SBM software applied to various real-world and synthetic networks produces poorly-connected to disconnected clusters. We present simple modifications to improve the connectivity of SBM clusters, and show that the modifications improve accuracy using simulated networks.
- [111] arXiv:2408.10467 [pdf, html, other]
-
Title: Learning Multimodal Latent Space with EBM Prior and MCMC InferenceSubjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
Multimodal generative models are crucial for various applications. We propose an approach that combines an expressive energy-based model (EBM) prior with Markov Chain Monte Carlo (MCMC) inference in the latent space for multimodal generation. The EBM prior acts as an informative guide, while MCMC inference, specifically through short-run Langevin dynamics, brings the posterior distribution closer to its true form. This method not only provides an expressive prior to better capture the complexity of multimodality but also improves the learning of shared latent variables for more coherent generation across modalities. Our proposed method is supported by empirical experiments, underscoring the effectiveness of our EBM prior with MCMC inference in enhancing cross-modal and joint generative tasks in multimodal contexts.
- [112] arXiv:2408.10468 [pdf, html, other]
-
Title: Tracing Privacy Leakage of Language Models to Training Data via Adjusted Influence FunctionsSubjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Cryptography and Security (cs.CR)
The responses generated by Large Language Models (LLMs) can include sensitive information from individuals and organizations, leading to potential privacy leakage. This work implements Influence Functions (IFs) to trace privacy leakage back to the training data, thereby mitigating privacy concerns of Language Models (LMs). However, we notice that current IFs struggle to accurately estimate the influence of tokens with large gradient norms, potentially overestimating their influence. When tracing the most influential samples, this leads to frequently tracing back to samples with large gradient norm tokens, overshadowing the actual most influential samples even if their influences are well estimated. To address this issue, we propose Heuristically Adjusted IF (HAIF), which reduces the weight of tokens with large gradient norms, thereby significantly improving the accuracy of tracing the most influential samples. To establish easily obtained groundtruth for tracing privacy leakage, we construct two datasets, PII-E and PII-CR, representing two distinct scenarios: one with identical text in the model outputs and pre-training data, and the other where models leverage their reasoning abilities to generate text divergent from pre-training data. HAIF significantly improves tracing accuracy, enhancing it by 20.96\% to 73.71\% on the PII-E dataset and 3.21\% to 45.93\% on the PII-CR dataset, compared to the best SOTA IFs against various GPT-2 and QWen-1.5 models. HAIF also outperforms SOTA IFs on real-world pretraining data CLUECorpus2020, demonstrating strong robustness regardless prompt and response lengths.
- [113] arXiv:2408.10469 [pdf, html, other]
-
Title: LSVOS Challenge 3rd Place Report: SAM2 and Cutie based VOSSubjects: Computer Vision and Pattern Recognition (cs.CV); Information Retrieval (cs.IR)
Video Object Segmentation (VOS) presents several challenges, including object occlusion and fragmentation, the dis-appearance and re-appearance of objects, and tracking specific objects within crowded scenes. In this work, we combine the strengths of the state-of-the-art (SOTA) models SAM2 and Cutie to address these challenges. Additionally, we explore the impact of various hyperparameters on video instance segmentation performance. Our approach achieves a J\&F score of 0.7952 in the testing phase of LSVOS challenge VOS track, ranking third overa1l.
- [114] arXiv:2408.10470 [pdf, html, other]
-
Title: Inverse Design of Snap-Actuated Jumping Robots Powered by Mechanics-Aided Machine LearningComments: 8 pages, 6 figuresSubjects: Robotics (cs.RO)
Exploring the design and control strategies of soft robots through simulation is highly attractive due to its cost-effectiveness. Although many existing models (e.g., finite element analysis) are effective for simulating soft robotic dynamics, there remains a need for a general and efficient numerical simulation approach in the soft robotics community. In this paper, we develop a discrete differential geometry-based numerical framework to achieve the model-based inverse design of a novel snap-actuated jumping robot. It is found that the dynamic process of a snapping beam can be either symmetric or asymmetric, such that the trajectory of the jumping robot can be tunable (e.g., horizontal or vertical). By employing this novel mechanism of the bistable beam as the robotic actuator, we next propose a physics-data hybrid inverse design strategy for the snap-jump robot with a broad spectrum of jumping capabilities. We first use the physical engine to study the influences of the robot's design parameters on the jumping capabilities, then generate extensive simulation data to formulate a data-driven inverse design solution. The inverse design solution can rapidly explore the combination of design parameters for achieving a target jump, which provides valuable guidance for the fabrication and control of the jumping robot. The proposed methodology paves the way for exploring the design and control insights of soft robots with the help of simulations.
- [115] arXiv:2408.10473 [pdf, html, other]
-
Title: Enhancing One-shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse MechanismGuanchen Li, Xiandong Zhao, Lian Liu, Zeping Li, Dong Li, Lu Tian, Jie He, Ashish Sirasao, Emad BarsoumSubjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Pre-trained language models (PLMs) are engineered to be robust in contextual understanding and exhibit outstanding performance in various natural language processing tasks. However, their considerable size incurs significant computational and storage costs. Modern pruning strategies employ one-shot techniques to compress PLMs without the need for retraining on task-specific or otherwise general data; however, these approaches often lead to an indispensable reduction in performance. In this paper, we propose SDS, a Sparse-Dense-Sparse pruning framework to enhance the performance of the pruned PLMs from a weight distribution optimization perspective. We outline the pruning process in three steps. Initially, we prune less critical connections in the model using conventional one-shot pruning methods. Next, we reconstruct a dense model featuring a pruning-friendly weight distribution by reactivating pruned connections with sparse regularization. Finally, we perform a second pruning round, yielding a superior pruned model compared to the initial pruning. Experimental results demonstrate that SDS outperforms the state-of-the-art pruning techniques SparseGPT and Wanda under an identical sparsity configuration. For instance, SDS reduces perplexity by 9.13 on Raw-Wikitext2 and improves accuracy by an average of 2.05% across multiple zero-shot benchmarks for OPT-125M with 2:4 sparsity.
- [116] arXiv:2408.10474 [pdf, other]
-
Title: LeCov: Multi-level Testing Criteria for Large Language ModelsSubjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
Large Language Models (LLMs) are widely used in many different domains, but because of their limited interpretability, there are questions about how trustworthy they are in various perspectives, e.g., truthfulness and toxicity. Recent research has started developing testing methods for LLMs, aiming to uncover untrustworthy issues, i.e., defects, before deployment. However, systematic and formalized testing criteria are lacking, which hinders a comprehensive assessment of the extent and adequacy of testing exploration. To mitigate this threat, we propose a set of multi-level testing criteria, LeCov, for LLMs. The criteria consider three crucial LLM internal components, i.e., the attention mechanism, feed-forward neurons, and uncertainty, and contain nine types of testing criteria in total. We apply the criteria in two scenarios: test prioritization and coverage-guided testing. The experiment evaluation, on three models and four datasets, demonstrates the usefulness and effectiveness of LeCov.
- [117] arXiv:2408.10479 [pdf, html, other]
-
Title: An End-to-End Reinforcement Learning Based Approach for Micro-View Order-Dispatching in Ride-HailingComments: 8 pages, 4 figuresSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Assigning orders to drivers under localized spatiotemporal context (micro-view order-dispatching) is a major task in Didi, as it influences ride-hailing service experience. Existing industrial solutions mainly follow a two-stage pattern that incorporate heuristic or learning-based algorithms with naive combinatorial methods, tackling the uncertainty of both sides' behaviors, including emerging timings, spatial relationships, and travel duration, etc. In this paper, we propose a one-stage end-to-end reinforcement learning based order-dispatching approach that solves behavior prediction and combinatorial optimization uniformly in a sequential decision-making manner. Specifically, we employ a two-layer Markov Decision Process framework to model this problem, and present \underline{D}eep \underline{D}ouble \underline{S}calable \underline{N}etwork (D2SN), an encoder-decoder structure network to generate order-driver assignments directly and stop assignments accordingly. Besides, by leveraging contextual dynamics, our approach can adapt to the behavioral patterns for better performance. Extensive experiments on Didi's real-world benchmarks justify that the proposed approach significantly outperforms competitive baselines in optimizing matching efficiency and user experience tasks. In addition, we evaluate the deployment outline and discuss the gains and experiences obtained during the deployment tests from the view of large-scale engineering implementation.
- [118] arXiv:2408.10482 [pdf, html, other]
-
Title: Evaluation Framework for AI-driven Molecular Design of Multi-target Drugs: Brain Diseases as a Case StudyComments: 8 pages, 1 figure, published in 2024 IEEE Congress on Evolutionary Computation (CEC)Journal-ref: 2024 IEEE Congress on Evolutionary Computation (CEC), Yokohama, Japan, 2024, pp. 1-8Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
The widespread application of Artificial Intelligence (AI) techniques has significantly influenced the development of new therapeutic agents. These computational methods can be used to design and predict the properties of generated molecules. Multi-target Drug Discovery (MTDD) is an emerging paradigm for discovering drugs against complex disorders that do not respond well to more traditional target-specific treatments, such as central nervous system, immune system, and cardiovascular diseases. Still, there is yet to be an established benchmark suite for assessing the effectiveness of AI tools for designing multi-target compounds. Standardized benchmarks allow for comparing existing techniques and promote rapid research progress. Hence, this work proposes an evaluation framework for molecule generation techniques in MTDD scenarios, considering brain diseases as a case study. Our methodology involves using large language models to select the appropriate molecular targets, gathering and preprocessing the bioassay datasets, training quantitative structure-activity relationship models to predict target modulation, and assessing other essential drug-likeness properties for implementing the benchmarks. Additionally, this work will assess the performance of four deep generative models and evolutionary algorithms over our benchmark suite. In our findings, both evolutionary algorithms and generative models can achieve competitive results across the proposed benchmarks.
- [119] arXiv:2408.10483 [pdf, html, other]
-
Title: PRformer: Pyramidal Recurrent Transformer for Multivariate Time Series ForecastingSubjects: Machine Learning (cs.LG)
The self-attention mechanism in Transformer architecture, invariant to sequence order, necessitates positional embeddings to encode temporal order in time series prediction. We argue that this reliance on positional embeddings restricts the Transformer's ability to effectively represent temporal sequences, particularly when employing longer lookback windows. To address this, we introduce an innovative approach that combines Pyramid RNN embeddings(PRE) for univariate time series with the Transformer's capability to model multivariate dependencies. PRE, utilizing pyramidal one-dimensional convolutional layers, constructs multiscale convolutional features that preserve temporal order. Additionally, RNNs, layered atop these features, learn multiscale time series representations sensitive to sequence order. This integration into Transformer models with attention mechanisms results in significant performance enhancements. We present the PRformer, a model integrating PRE with a standard Transformer encoder, demonstrating state-of-the-art performance on various real-world datasets. This performance highlights the effectiveness of our approach in leveraging longer lookback windows and underscores the critical role of robust temporal representations in maximizing Transformer's potential for prediction tasks. Code is available at this repository: \url{this https URL}.
- [120] arXiv:2408.10486 [pdf, html, other]
-
Title: Revisiting Evolutionary Program Repair via Code Language ModelSubjects: Software Engineering (cs.SE)
Software defects are an inherent part of software development and maintenance. To address these defects, Automated Program Repair (APR) has been developed to fix bugs automatically. With the advent of Large Language Models, Code Language Models (CLMs) trained on code corpora excels in code generation, making them suitable for APR applications. Despite this progress, a significant limitation remains: many bugs necessitate multi-point edits for repair, yet current CLM-based APRs are restricted to single-point bug fixes, which severely narrows the scope of repairable bugs. Moreover, these tools typically only consider the direct context of the buggy line when building prompts for the CLM, leading to suboptimal repair outcomes due to the limited information provided. This paper introduces a novel approach, ARJA-CLM, which integrates the multiobjective evolutionary algorithm with CLM to fix multilocation bugs in Java projects. We also propose a context-aware prompt construction stratege, which enriches the prompt with additional information about accessible fields and methods for the CLM generating candidate statements. Our experiments on the Defects4J and APR-2024 competition benchmark demonstrate that ARJA-CLM surpasses many state-of-the-art repair systems, and performs well on multi-point bugs. The results also reveal that CLMs effectively utilize the provided field and method information within context-aware prompts to produce candidate statements.
- [121] arXiv:2408.10487 [pdf, html, other]
-
Title: MambaEVT: Event Stream based Visual Object Tracking using State Space ModelComments: In Peer ReviewSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Event camera-based visual tracking has drawn more and more attention in recent years due to the unique imaging principle and advantages of low energy consumption, high dynamic range, and dense temporal resolution. Current event-based tracking algorithms are gradually hitting their performance bottlenecks, due to the utilization of vision Transformer and the static template for target object localization. In this paper, we propose a novel Mamba-based visual tracking framework that adopts the state space model with linear complexity as a backbone network. The search regions and target template are fed into the vision Mamba network for simultaneous feature extraction and interaction. The output tokens of search regions will be fed into the tracking head for target localization. More importantly, we consider introducing a dynamic template update strategy into the tracking framework using the Memory Mamba network. By considering the diversity of samples in the target template library and making appropriate adjustments to the template memory module, a more effective dynamic template can be integrated. The effective combination of dynamic and static templates allows our Mamba-based tracking algorithm to achieve a good balance between accuracy and computational cost on multiple large-scale datasets, including EventVOT, VisEvent, and FE240hz. The source code will be released on this https URL
- [122] arXiv:2408.10488 [pdf, html, other]
-
Title: Event Stream based Sign Language Translation: A High-Definition Benchmark Dataset and A New AlgorithmComments: First Large-scale and High-Definition Benchmark Dataset for Event-based Sign Language TranslationSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Neural and Evolutionary Computing (cs.NE)
Sign Language Translation (SLT) is a core task in the field of AI-assisted disability. Unlike traditional SLT based on visible light videos, which is easily affected by factors such as lighting, rapid hand movements, and privacy breaches, this paper proposes the use of high-definition Event streams for SLT, effectively mitigating the aforementioned issues. This is primarily because Event streams have a high dynamic range and dense temporal signals, which can withstand low illumination and motion blur well. Additionally, due to their sparsity in space, they effectively protect the privacy of the target person. More specifically, we propose a new high-resolution Event stream sign language dataset, termed Event-CSL, which effectively fills the data gap in this area of research. It contains 14,827 videos, 14,821 glosses, and 2,544 Chinese words in the text vocabulary. These samples are collected in a variety of indoor and outdoor scenes, encompassing multiple angles, light intensities, and camera movements. We have benchmarked existing mainstream SLT works to enable fair comparison for future efforts. Based on this dataset and several other large-scale datasets, we propose a novel baseline method that fully leverages the Mamba model's ability to integrate temporal information of CNN features, resulting in improved sign language translation outcomes. Both the benchmark dataset and source code will be released on this https URL
- [123] arXiv:2408.10490 [pdf, html, other]
-
Title: Analysis of Plan-based Retrieval for Grounded Text GenerationSubjects: Computation and Language (cs.CL); Information Retrieval (cs.IR)
In text generation, hallucinations refer to the generation of seemingly coherent text that contradicts established knowledge. One compelling hypothesis is that hallucinations occur when a language model is given a generation task outside its parametric knowledge (due to rarity, recency, domain, etc.). A common strategy to address this limitation is to infuse the language models with retrieval mechanisms, providing the model with relevant knowledge for the task. In this paper, we leverage the planning capabilities of instruction-tuned LLMs and analyze how planning can be used to guide retrieval to further reduce the frequency of hallucinations. We empirically evaluate several variations of our proposed approach on long-form text generation tasks. By improving the coverage of relevant facts, plan-guided retrieval and generation can produce more informative responses while providing a higher rate of attribution to source documents.
- [124] arXiv:2408.10491 [pdf, html, other]
-
Title: Achieving the Tightest Relaxation of Sigmoids for Formal VerificationSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
In the field of formal verification, Neural Networks (NNs) are typically reformulated into equivalent mathematical programs which are optimized over. To overcome the inherent non-convexity of these reformulations, convex relaxations of nonlinear activation functions are typically utilized. Common relaxations (i.e., static linear cuts) of ``S-shaped" activation functions, however, can be overly loose, slowing down the overall verification process. In this paper, we derive tuneable hyperplanes which upper and lower bound the sigmoid activation function. When tuned in the dual space, these affine bounds smoothly rotate around the nonlinear manifold of the sigmoid activation function. This approach, termed $\alpha$-sig, allows us to tractably incorporate the tightest possible, element-wise convex relaxation of the sigmoid activation function into a formal verification framework. We embed these relaxations inside of large verification tasks and compare their performance to LiRPA and $\alpha$-CROWN, a state-of-the-art verification duo.
- [125] arXiv:2408.10492 [pdf, html, other]
-
Title: Is the Lecture Engaging for Learning? Lecture Voice Sentiment Analysis for Knowledge Graph-Supported Intelligent Lecturing Assistant (ILA) SystemYuan An, Samarth Kolanupaka, Jacob An, Matthew Ma, Unnat Chhatwal, Alex Kalinowski, Michelle Rogers, Brian SmithSubjects: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
This paper introduces an intelligent lecturing assistant (ILA) system that utilizes a knowledge graph to represent course content and optimal pedagogical strategies. The system is designed to support instructors in enhancing student learning through real-time analysis of voice, content, and teaching methods. As an initial investigation, we present a case study on lecture voice sentiment analysis, in which we developed a training set comprising over 3,000 one-minute lecture voice clips. Each clip was manually labeled as either engaging or non-engaging. Utilizing this dataset, we constructed and evaluated several classification models based on a variety of features extracted from the voice clips. The results demonstrate promising performance, achieving an F1-score of 90% for boring lectures on an independent set of over 800 test voice clips. This case study lays the groundwork for the development of a more sophisticated model that will integrate content analysis and pedagogical practices. Our ultimate goal is to aid instructors in teaching more engagingly and effectively by leveraging modern artificial intelligence techniques.
- [126] arXiv:2408.10493 [pdf, html, other]
-
Title: Clustering by Mining Density Distributions and Splitting Manifold StructureSubjects: Machine Learning (cs.LG)
Spectral clustering requires the time-consuming decomposition of the Laplacian matrix of the similarity graph, thus limiting its applicability to large datasets. To improve the efficiency of spectral clustering, a top-down approach was recently proposed, which first divides the data into several micro-clusters (granular-balls), then splits these micro-clusters when they are not "compact'', and finally uses these micro-clusters as nodes to construct a similarity graph for more efficient spectral clustering. However, this top-down approach is challenging to adapt to unevenly distributed or structurally complex data. This is because constructing micro-clusters as a rough ball struggles to capture the shape and structure of data in a local range, and the simplistic splitting rule that solely targets ``compactness'' is susceptible to noise and variations in data density and leads to micro-clusters with varying shapes, making it challenging to accurately measure the similarity between them. To resolve these issues, this paper first proposes to start from local structures to obtain micro-clusters, such that the complex structural information inside local neighborhoods is well captured by them. Moreover, by noting that Euclidean distance is more suitable for convex sets, this paper further proposes a data splitting rule that couples local density and data manifold structures, so that the similarities of the obtained micro-clusters can be easily characterized. A novel similarity measure between micro-clusters is then proposed for the final spectral clustering. A series of experiments based on synthetic and real-world datasets demonstrate that the proposed method has better adaptability to structurally complex data than granular-ball based methods.
- [127] arXiv:2408.10494 [pdf, html, other]
-
Title: Tensor-Product Split-Simplex Summation-By-Parts OperatorsComments: 28 pages, 15 figuresSubjects: Numerical Analysis (math.NA)
We present an approach to construct efficient sparse summation-by-parts (SBP) operators on triangles and tetrahedra with a tensor-product structure. The operators are constructed by splitting the simplices into quadrilateral or hexahedral subdomains, mapping tensor-product SBP operators onto the subdomains, and assembling back using a continuous-Galerkin-type procedure. These tensor-product split-simplex operators do not have repeated degrees of freedom at the interior interfaces between the split subdomains. Furthermore, they satisfy the SBP property by construction, leading to stable discretizations. The accuracy and sparsity of the operators substantially enhance the efficiency of SBP discretizations on simplicial meshes. The sparsity is particularly important for entropy-stable discretizations based on two-point flux functions, as it reduces the number of two-point flux computations. We demonstrate through numerical experiments that the operators exhibit efficiency surpassing that of the existing dense multidimensional SBP operators by more than an order of magnitude in many cases. This superiority is evident in both accuracy per degree of freedom and computational time required to achieve a specified error threshold.
- [128] arXiv:2408.10495 [pdf, html, other]
-
Title: How Well Do Large Language Models Serve as End-to-End Secure Code Producers?Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)
The rapid advancement of large language models (LLMs) such as GPT-4 has revolutionized the landscape of software engineering, positioning these models at the core of modern development practices. As we anticipate these models to evolve into the primary and trustworthy tools used in software development, ensuring the security of the code they produce becomes paramount. How well can LLMs serve as end-to-end secure code producers? This paper presents a systematic investigation into LLMs' inherent potential to generate code with fewer vulnerabilities. Specifically, We studied GPT-3.5 and GPT-4's capability to identify and repair vulnerabilities in the code generated by four popular LLMs including themselves (GPT-3.5, GPT-4, Code Llama, and CodeGeeX2). By manually or automatically reviewing 4,900 pieces of code, our study reveals that: (1) large language models lack awareness of scenario-relevant security risks, which leads to the generation of over 75% vulnerable code on the SecurityEval benchmark; (2) LLMs such as GPT-3.5 and GPT-4 are unable to precisely identify vulnerabilities in the code they generated; (3) GPT-3.5 and GPT-4 can achieve 33.2%~59.6% success rates in repairing the insecure code produced by the 4 LLMs, but they both perform poorly when repairing self-produced code, indicating self-repair "blind spots". To address the limitation of a single round of repair, we developed a lightweight tool that prompts LLMs to construct safer source code through an iterative repair procedure based on the insights gained from our study. Experiments show that assisted by semantic analysis engines, our tool significantly improves the success rates of repair to 65.9%~85.5%.
- [129] arXiv:2408.10496 [pdf, html, other]
-
Title: GPT-based Textile Pilling Classification Using 3D Point Cloud DataComments: 8 pages, 2 figuresSubjects: Computer Vision and Pattern Recognition (cs.CV)
Textile pilling assessment is critical for textile quality control. We collect thousands of 3D point cloud images in the actual test environment of textiles and organize and label them as TextileNet8 dataset. To the best of our knowledge, it is the first publicly available eight-categories 3D point cloud dataset in the field of textile pilling assessment. Based on PointGPT, the GPT-like big model of point cloud analysis, we incorporate the global features of the input point cloud extracted from the non-parametric network into it, thus proposing the PointGPT+NN model. Using TextileNet8 as a benchmark, the experimental results show that the proposed PointGPT+NN model achieves an overall accuracy (OA) of 91.8% and a mean per-class accuracy (mAcc) of 92.2%. Test results on other publicly available datasets also validate the competitive performance of the proposed PointGPT+NN model. The proposed TextileNet8 dataset will be publicly available.
- [130] arXiv:2408.10497 [pdf, html, other]
-
Title: QUITO-X: An Information Bottleneck-based Compression Algorithm with Cross-AttentionSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Generative LLM have achieved significant success in various industrial tasks and can effectively adapt to vertical domains and downstream tasks through ICL. However, with tasks becoming increasingly complex, the context length required by ICL is also getting longer, and two significant issues arise: (i) The excessively long context leads to high costs and inference delays. (ii) A substantial amount of task-irrelevant information introduced by long contexts exacerbates the "lost in the middle" problem.
Recently, compressing prompts by removing tokens according to some metric obtained from some causal language models, such as llama-7b, has emerged as an effective approach to mitigate these issues. However, the metric used by prior method such as self-information or PPL do not fully align with the objective of distinuishing the most important tokens when conditioning on query. In this work, we introduce information bottleneck theory to carefully examine the properties required by the metric. Inspired by this, we use cross-attention in encoder-decoder architecture as a new metric. Our simple method leads to significantly better performance in smaller models with lower latency.
We evaluate our method on four datasets: DROP, CoQA, SQuAD, and Quoref. The experimental results show that, while maintaining the same performance, our compression rate can improve by nearly 25% over previous SOTA. Remarkably, in experiments where 25% of the tokens are removed, our model's EM score for answers sometimes even exceeds that of the control group using uncompressed text as context. - [131] arXiv:2408.10499 [pdf, html, other]
-
Title: ProgramAlly: Creating Custom Visual Access Programs via Multi-Modal End-User ProgrammingComments: UIST 2024Subjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Programming Languages (cs.PL)
Existing visual assistive technologies are built for simple and common use cases, and have few avenues for blind people to customize their functionalities. Drawing from prior work on DIY assistive technology, this paper investigates end-user programming as a means for users to create and customize visual access programs to meet their unique needs. We introduce ProgramAlly, a system for creating custom filters for visual information, e.g., 'find NUMBER on BUS', leveraging three end-user programming approaches: block programming, natural language, and programming by example. To implement ProgramAlly, we designed a representation of visual filtering tasks based on scenarios encountered by blind people, and integrated a set of on-device and cloud models for generating and running these programs. In user studies with 12 blind adults, we found that participants preferred different programming modalities depending on the task, and envisioned using visual access programs to address unique accessibility challenges that are otherwise difficult with existing applications. Through ProgramAlly, we present an exploration of how blind end-users can create visual access programs to customize and control their experiences.
- [132] arXiv:2408.10500 [pdf, html, other]
-
Title: SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion RecognitionZebang Cheng, Shuyuan Tu, Dawei Huang, Minghan Li, Xiaojiang Peng, Zhi-Qi Cheng, Alexander G. HauptmannSubjects: Multimedia (cs.MM); Computer Vision and Pattern Recognition (cs.CV); Sound (cs.SD); Audio and Speech Processing (eess.AS)
This paper presents our winning approach for the MER-NOISE and MER-OV tracks of the MER2024 Challenge on multimodal emotion recognition. Our system leverages the advanced emotional understanding capabilities of Emotion-LLaMA to generate high-quality annotations for unlabeled samples, addressing the challenge of limited labeled data. To enhance multimodal fusion while mitigating modality-specific noise, we introduce Conv-Attention, a lightweight and efficient hybrid framework. Extensive experimentation vali-dates the effectiveness of our approach. In the MER-NOISE track, our system achieves a state-of-the-art weighted average F-score of 85.30%, surpassing the second and third-place teams by 1.47% and 1.65%, respectively. For the MER-OV track, our utilization of Emotion-LLaMA for open-vocabulary annotation yields an 8.52% improvement in average accuracy and recall compared to GPT-4V, securing the highest score among all participating large multimodal models. The code and model for Emotion-LLaMA are available at this https URL.
- [133] arXiv:2408.10501 [pdf, html, other]
-
Title: Generative Diffusion Models for High Dimensional Channel EstimationComments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibleSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Along with the prosperity of generative artificial intelligence (AI), its potential for solving conventional challenges in wireless communications has also surfaced. Inspired by this trend, we investigate the application of the advanced diffusion models (DMs), a representative class of generative AI models, to high dimensional wireless channel estimation. By capturing the structure of multiple-input multiple-output (MIMO) wireless channels via a deep generative prior encoded by DMs, we develop a novel posterior inference method for channel reconstruction. We further adapt the proposed method to recover channel information from low-resolution quantized measurements. Additionally, to enhance the over-the-air viability, we integrate the DM with the unsupervised Stein's unbiased risk estimator to enable learning from noisy observations and circumvent the requirements for ground truth channel data that is hardly available in practice. Results reveal that the proposed estimator achieves high-fidelity channel recovery while reducing estimation latency by a factor of 10 compared to state-of-the-art schemes, facilitating real-time implementation. Moreover, our method outperforms existing estimators while reducing the pilot overhead by half, showcasing its scalability to ultra-massive antenna arrays.
- [134] arXiv:2408.10503 [pdf, html, other]
-
Title: Adaptive Knowledge Distillation for Classification of Hand Images using Explainable Vision TransformersComments: Accepted at the ECML PKDD 2024 (Research Track)Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Assessing the forensic value of hand images involves the use of unique features and patterns present in an individual's hand. The human hand has distinct characteristics, such as the pattern of veins, fingerprints, and the geometry of the hand itself. This paper investigates the use of vision transformers (ViTs) for classification of hand images. We use explainability tools to explore the internal representations of ViTs and assess their impact on the model outputs. Utilizing the internal understanding of ViTs, we introduce distillation methods that allow a student model to adaptively extract knowledge from a teacher model while learning on data of a different domain to prevent catastrophic forgetting. Two publicly available hand image datasets are used to conduct a series of experiments to evaluate performance of the ViTs and our proposed adaptive distillation methods. The experimental results demonstrate that ViT models significantly outperform traditional machine learning methods and the internal states of ViTs are useful for explaining the model outputs in the classification task. By averting catastrophic forgetting, our distillation methods achieve excellent performance on data from both source and target domains, particularly when these two domains exhibit significant dissimilarity. The proposed approaches therefore can be developed and implemented effectively for real-world applications such as access control, identity verification, and authentication systems.
- [135] arXiv:2408.10504 [pdf, other]
-
Title: QPO: Query-dependent Prompt Optimization via Multi-Loop Offline Reinforcement LearningYilun Kong, Hangyu Mao, Qi Zhao, Bin Zhang, Jingqing Ruan, Li Shen, Yongzhe Chang, Xueqian Wang, Rui Zhao, Dacheng TaoSubjects: Artificial Intelligence (cs.AI)
Prompt engineering has demonstrated remarkable success in enhancing the performance of large language models (LLMs) across diverse tasks. However, most existing prompt optimization methods only focus on the task-level performance, overlooking the importance of query-preferred prompts, which leads to suboptimal performances. Additionally, these methods rely heavily on frequent interactions with LLMs to obtain feedback for guiding the optimization process, incurring substantial redundant interaction costs. In this paper, we introduce Query-dependent Prompt Optimization (QPO), which leverages multi-loop offline reinforcement learning to iteratively fine-tune a small pretrained language model to generate optimal prompts tailored to the input queries, thus significantly improving the prompting effect on the large target LLM. We derive insights from offline prompting demonstration data, which already exists in large quantities as a by-product of benchmarking diverse prompts on open-sourced tasks, thereby circumventing the expenses of online interactions. Furthermore, we continuously augment the offline dataset with the generated prompts in each loop, as the prompts from the fine-tuned model are supposed to outperform the source prompts in the original dataset. These iterative loops bootstrap the model towards generating optimal prompts. Experiments on various LLM scales and diverse NLP and math tasks demonstrate the efficacy and cost-efficiency of our method in both zero-shot and few-shot scenarios.
- [136] arXiv:2408.10511 [pdf, html, other]
-
Title: Single-cell Curriculum Learning-based Deep Graph Embedding ClusteringSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Genomics (q-bio.GN)
The swift advancement of single-cell RNA sequencing (scRNA-seq) technologies enables the investigation of cellular-level tissue heterogeneity. Cell annotation significantly contributes to the extensive downstream analysis of scRNA-seq data. However, The analysis of scRNA-seq for biological inference presents challenges owing to its intricate and indeterminate data distribution, characterized by a substantial volume and a high frequency of dropout events. Furthermore, the quality of training samples varies greatly, and the performance of the popular scRNA-seq data clustering solution GNN could be harmed by two types of low-quality training nodes: 1) nodes on the boundary; 2) nodes that contribute little additional information to the graph. To address these problems, we propose a single-cell curriculum learning-based deep graph embedding clustering (scCLG). We first propose a Chebyshev graph convolutional autoencoder with multi-decoder (ChebAE) that combines three optimization objectives corresponding to three decoders, including topology reconstruction loss of cell graphs, zero-inflated negative binomial (ZINB) loss, and clustering loss, to learn cell-cell topology representation. Meanwhile, we employ a selective training strategy to train GNN based on the features and entropy of nodes and prune the difficult nodes based on the difficulty scores to keep the high-quality graph. Empirical results on a variety of gene expression datasets show that our model outperforms state-of-the-art methods.
- [137] arXiv:2408.10512 [pdf, html, other]
-
Title: Approximate Estimation of High-dimension Execution Skill for Dynamic Agents in Continuous DomainsSubjects: Artificial Intelligence (cs.AI)
In many real-world continuous action domains, human agents must decide which actions to attempt and then execute those actions to the best of their ability. However, humans cannot execute actions without error. Human performance in these domains can potentially be improved by the use of AI to aid in decision-making. One requirement for an AI to correctly reason about what actions a human agent should attempt is a correct model of that human's execution error, or skill. Recent work has demonstrated successful techniques for estimating this execution error with various types of agents across different domains. However, this previous work made several assumptions that limit the application of these ideas to real-world settings. First, previous work assumed that the error distributions were symmetric normal, which meant that only a single parameter had to be estimated. In reality, agent error distributions might exhibit arbitrary shapes and should be modeled more flexibly. Second, it was assumed that the execution error of the agent remained constant across all observations. Especially for human agents, execution error changes over time, and this must be taken into account to obtain effective estimates. To overcome both of these shortcomings, we propose a novel particle-filter-based estimator for this problem. After describing the details of this approximate estimator, we experimentally explore various design decisions and compare performance with previous skill estimators in a variety of settings to showcase the improvements. The outcome is an estimator capable of generating more realistic, time-varying execution skill estimates of agents, which can then be used to assist agents in making better decisions and improve their overall performance.
- [138] arXiv:2408.10516 [pdf, html, other]
-
Title: Data Augmentation Integrating Dialogue Flow and Style to Adapt Spoken Dialogue Systems to Low-Resource User GroupsComments: Accepted to SIGDIAL 2024Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
This study addresses the interaction challenges encountered by spoken dialogue systems (SDSs) when engaging with users who exhibit distinct conversational behaviors, particularly minors, in scenarios where data are scarce. We propose a novel data augmentation framework to enhance SDS performance for user groups with limited resources. Our approach leverages a large language model (LLM) to extract speaker styles and a pre-trained language model (PLM) to simulate dialogue act history. This method generates enriched and personalized dialogue data, facilitating improved interactions with unique user demographics. Extensive experiments validate the efficacy of our methodology, highlighting its potential to foster the development of more adaptive and inclusive dialogue systems.
- [139] arXiv:2408.10517 [pdf, html, other]
-
Title: Integrating Multi-Modal Input Token Mixer Into Mamba-Based Decision Models: Decision MetaMambaSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Return-Conditioned Transformer Decision Models (RCTDM) have demonstrated the potential to enhance transformer performance in offline reinforcement learning by replacing rewards in the input sequence with returns-to-go. However, to achieve the goal of learning an optimal policy from offline datasets composed of limited suboptimal trajectories, RCTDM required alternative methods. One prominent approach, trajectory stitching, was designed to enable the network to combine multiple trajectories to find the optimal path. To implement this using only transformers without auxiliary networks, it was necessary to shorten the input sequence length to better capture the Markov property in reinforcement learnings. This, however, introduced a trade-off, as it reduced the accuracy of action inference. Our study introduces a model named Decision MetaMamba to resolve these challenges. DMM employs an input token mixer to extract patterns from short sequences and uses a State Space Model (SSM) to selectively combine information from relatively distant sequences. Inspired by Metaformer, this structure was developed by transforming Mamba's input layer into various multi-modal layers. Fortunately, with the advent of Mamba, implemented using parallel selective scanning, we achieved a high-performance sequence model capable of replacing transformers. Based on these innovations, DMM demonstrated excellent performance across various datasets in offline RL, confirming that models using SSM can improve performance by domain-specific alterations of the input layer. Additionally, it maintained its performance even in lightweight models with fewer parameters. These results suggest that decision models based on SSM can pave the way for improved outcomes in future developments.
- [140] arXiv:2408.10518 [pdf, html, other]
-
Title: BAUST Lipi: A BdSL Dataset with Deep Learning Based Bangla Sign Language RecognitionMd Hadiuzzaman, Mohammed Sowket Ali, Tamanna Sultana, Abdur Raj Shafi, Abu Saleh Musa Miah, Jungpil ShinSubjects: Computer Vision and Pattern Recognition (cs.CV)
People commonly communicate in English, Arabic, and Bengali spoken languages through various mediums. However, deaf and hard-of-hearing individuals primarily use body language and sign language to express their needs and achieve independence. Sign language research is burgeoning to enhance communication with the deaf community. While many researchers have made strides in recognizing sign languages such as French, British, Arabic, Turkish, and American, there has been limited research on Bangla sign language (BdSL) with less-than-satisfactory results. One significant barrier has been the lack of a comprehensive Bangla sign language dataset. In our work, we introduced a new BdSL dataset comprising alphabets totaling 18,000 images, with each image being 224x224 pixels in size. Our dataset encompasses 36 Bengali symbols, of which 30 are consonants and the remaining six are vowels. Despite our dataset contribution, many existing systems continue to grapple with achieving high-performance accuracy for BdSL. To address this, we devised a hybrid Convolutional Neural Network (CNN) model, integrating multiple convolutional layers, activation functions, dropout techniques, and LSTM layers. Upon evaluating our hybrid-CNN model with the newly created BdSL dataset, we achieved an accuracy rate of 97.92\%. We are confident that both our BdSL dataset and hybrid CNN model will be recognized as significant milestones in BdSL research.
- [141] arXiv:2408.10519 [pdf, html, other]
-
Title: Almost Optimal Algorithms for Token Collision in Anonymous NetworksSubjects: Distributed, Parallel, and Cluster Computing (cs.DC); Data Structures and Algorithms (cs.DS)
In distributed systems, situations often arise where some nodes each holds a collection of tokens, and all nodes collectively need to determine whether all tokens are distinct. For example, if each token represents a logged-in user, the problem corresponds to checking whether there are duplicate logins. Similarly, if each token represents a data object or a timestamp, the problem corresponds to checking whether there are conflicting operations in distributed databases. In distributed computing theory, unique identifiers generation is also related to this problem: each node generates one token, which is its identifier, then a verification phase is needed to ensure all identifiers are unique.
In this paper, we formalize and initiate the study of token collision. In this problem, a collection of $k$ tokens, each represented by some length-$L$ bit string, are distributed to $n$ nodes of an anonymous CONGEST network in an arbitrary manner. The nodes need to determine whether there are tokens with an identical value. We present near optimal deterministic algorithms for the token collision problem with $\tilde{O}(D+k\cdot L/\log{n})$ round complexity, where $D$ denotes the network diameter. Besides high efficiency, the prior knowledge required by our algorithms is also limited. For completeness, we further present a near optimal randomized algorithm for token collision. - [142] arXiv:2408.10520 [pdf, html, other]
-
Title: Efficient and Deployable Knowledge Infusion for Open-World Recommendations via Large Language ModelsYunjia Xi, Weiwen Liu, Jianghao Lin, Muyan Weng, Xiaoling Cai, Hong Zhu, Jieming Zhu, Bo Chen, Ruiming Tang, Yong Yu, Weinan ZhangComments: arXiv admin note: text overlap with arXiv:2306.10933Subjects: Information Retrieval (cs.IR)
Recommender systems (RSs) play a pervasive role in today's online services, yet their closed-loop nature constrains their access to open-world knowledge. Recently, large language models (LLMs) have shown promise in bridging this gap. However, previous attempts to directly implement LLMs as recommenders fall short in meeting the requirements of industrial RSs, particularly in terms of online inference latency and offline resource efficiency. Thus, we propose REKI to acquire two types of external knowledge about users and items from LLMs. Specifically, we introduce factorization prompting to elicit accurate knowledge reasoning on user preferences and items. We develop individual knowledge extraction and collective knowledge extraction tailored for different scales of scenarios, effectively reducing offline resource consumption. Subsequently, generated knowledge undergoes efficient transformation and condensation into augmented vectors through a hybridized expert-integrated network, ensuring compatibility. The obtained vectors can then be used to enhance any conventional recommendation model. We also ensure efficient inference by preprocessing and prestoring the knowledge from LLMs. Experiments demonstrate that REKI outperforms state-of-the-art baselines and is compatible with lots of recommendation algorithms and tasks. Now, REKI has been deployed to Huawei's news and music recommendation platforms and gained a 7% and 1.99% improvement during the online A/B test.
- [143] arXiv:2408.10524 [pdf, html, other]
-
Title: XCB: an effective contextual biasing approach to bias cross-lingual phrases in speech recognitionComments: accepted to NCMMSC 2024Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Contextualized ASR models have been demonstrated to effectively improve the recognition accuracy of uncommon phrases when a predefined phrase list is available. However, these models often struggle with bilingual settings, which are prevalent in code-switching speech recognition. In this study, we make the initial attempt to address this challenge by introducing a Cross-lingual Contextual Biasing(XCB) module. Specifically, we augment a pre-trained ASR model for the dominant language by integrating an auxiliary language biasing module and a supplementary language-specific loss, aimed at enhancing the recognition of phrases in the secondary language. Experimental results conducted on our in-house code-switching dataset have validated the efficacy of our approach, demonstrating significant improvements in the recognition of biasing phrases in the secondary language, even without any additional inference overhead. Additionally, our proposed system exhibits both efficiency and generalization when is applied by the unseen ASRU-2019 test set.
- [144] arXiv:2408.10525 [pdf, html, other]
-
Title: MPGNet: Learning Move-Push-Grasping Synergy for Target-Oriented Grasping in Occluded ScenesComments: Accepted to IROS 2024Subjects: Robotics (cs.RO)
This paper focuses on target-oriented grasping in occluded scenes, where the target object is specified by a binary mask and the goal is to grasp the target object with as few robotic manipulations as possible. Most existing methods rely on a push-grasping synergy to complete this task. To deliver a more powerful target-oriented grasping pipeline, we present MPGNet, a three-branch network for learning a synergy between moving, pushing, and grasping actions. We also propose a multi-stage training strategy to train the MPGNet which contains three policy networks corresponding to the three actions. The effectiveness of our method is demonstrated via both simulated and real-world experiments.
- [145] arXiv:2408.10527 [pdf, html, other]
-
Title: EdgeNAT: Transformer for Efficient Edge DetectionSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Transformers, renowned for their powerful feature extraction capabilities, have played an increasingly prominent role in various vision tasks. Especially, recent advancements present transformer with hierarchical structures such as Dilated Neighborhood Attention Transformer (DiNAT), demonstrating outstanding ability to efficiently capture both global and local features. However, transformers' application in edge detection has not been fully exploited. In this paper, we propose EdgeNAT, a one-stage transformer-based edge detector with DiNAT as the encoder, capable of extracting object boundaries and meaningful edges both accurately and efficiently. On the one hand, EdgeNAT captures global contextual information and detailed local cues with DiNAT, on the other hand, it enhances feature representation with a novel SCAF-MLA decoder by utilizing both inter-spatial and inter-channel relationships of feature maps. Extensive experiments on multiple datasets show that our method achieves state-of-the-art performance on both RGB and depth images. Notably, on the widely used BSDS500 dataset, our L model achieves impressive performances, with ODS F-measure and OIS F-measure of 86.0%, 87.6% for multi-scale input,and 84.9%, and 86.3% for single-scale input, surpassing the current state-of-the-art EDTER by 1.2%, 1.1%, 1.7%, and 1.6%, respectively. Moreover, as for throughput, our approach runs at 20.87 FPS on RTX 4090 GPU with single-scale input. The code for our method will be released soon.
- [146] arXiv:2408.10528 [pdf, html, other]
-
Title: NoMatterXAI: Generating "No Matter What" Alterfactual Examples for Explaining Black-Box Text Classification ModelsSubjects: Computation and Language (cs.CL)
In Explainable AI (XAI), counterfactual explanations (CEs) are a well-studied method to communicate feature relevance through contrastive reasoning of "what if" to explain AI models' predictions. However, they only focus on important (i.e., relevant) features and largely disregard less important (i.e., irrelevant) ones. Such irrelevant features can be crucial in many applications, especially when users need to ensure that an AI model's decisions are not affected or biased against specific attributes such as gender, race, religion, or political affiliation. To address this gap, the concept of alterfactual explanations (AEs) has been proposed. AEs explore an alternative reality of "no matter what", where irrelevant features are substituted with alternative features (e.g., "republicans" -> "democrats") within the same attribute (e.g., "politics") while maintaining a similar prediction output. This serves to validate whether AI model predictions are influenced by the specified attributes. Despite the promise of AEs, there is a lack of computational approaches to systematically generate them, particularly in the text domain, where creating AEs for AI text classifiers presents unique challenges. This paper addresses this challenge by formulating AE generation as an optimization problem and introducing MoMatterXAI, a novel algorithm that generates AEs for text classification tasks. Our approach achieves high fidelity of up to 95% while preserving context similarity of over 90% across multiple models and datasets. A human study further validates the effectiveness of AEs in explaining AI text classifiers to end users. All codes will be publicly available.
- [147] arXiv:2408.10529 [pdf, other]
-
Title: Automated Detection of Algorithm Debt in Deep Learning Frameworks: An Empirical StudyComments: 7 pages, 1 figure, 1 Table, Conference: 40th IEEE International Conference on Software Maintenance and Evolution Location: Flagstaff, AZ, USA Date 6 - 11 October, 2024Subjects: Software Engineering (cs.SE)
Context: Recent studies demonstrate that Machine or Deep Learning (ML/DL) models can detect Technical Debt from source code comments called Self-Admitted Technical Debt (SATD). Despite the importance of ML/DL in software development, limited studies focus on automated detection for new SATD types: Algorithm Debt (AD). AD detection is important because it helps to identify TD early, facilitating research, learning, and preventing the accumulation of issues related to model degradation and lack of scalability. Aim: Our goal is to improve AD detection performance of various ML/DL models. Method: We will perform empirical studies using approaches: TF-IDF, Count Vectorizer, Hash Vectorizer, and TD-indicative words to identify features that improve AD detection, using ML/DL classifiers with different data featurisations. We will use an existing dataset curated from seven DL frameworks where comments were manually classified as AD, Compatibility, Defect, Design, Documentation, Requirement, and Test Debt. We will explore various word embedding methods to further enrich features for ML models. These embeddings will be from models founded in DL such as ROBERTA, ALBERTv2, and large language models (LLMs): INSTRUCTOR and VOYAGE AI. We will enrich the dataset by incorporating AD-related terms, then train various ML/DL classifiers, Support Vector Machine, Logistic Regression, Random Forest, ROBERTA, and ALBERTv2.
- [148] arXiv:2408.10531 [pdf, html, other]
-
Title: Leveraging Temporal Contexts to Enhance Vehicle-Infrastructure Cooperative PerceptionComments: Accepted by IEEE ITSC 2024Subjects: Robotics (cs.RO)
Infrastructure sensors installed at elevated positions offer a broader perception range and encounter fewer occlusions. Integrating both infrastructure and ego-vehicle data through V2X communication, known as vehicle-infrastructure cooperation, has shown considerable advantages in enhancing perception capabilities and addressing corner cases encountered in single-vehicle autonomous driving. However, cooperative perception still faces numerous challenges, including limited communication bandwidth and practical communication interruptions. In this paper, we propose CTCE, a novel framework for cooperative 3D object detection. This framework transmits queries with temporal contexts enhancement, effectively balancing transmission efficiency and performance to accommodate real-world communication conditions. Additionally, we propose a temporal-guided fusion module to further improve performance. The roadside temporal enhancement and vehicle-side spatial-temporal fusion together constitute a multi-level temporal contexts integration mechanism, fully leveraging temporal information to enhance performance. Furthermore, a motion-aware reconstruction module is introduced to recover lost roadside queries due to communication interruptions. Experimental results on V2X-Seq and V2X-Sim datasets demonstrate that CTCE outperforms the baseline QUEST, achieving improvements of 3.8% and 1.3% in mAP, respectively. Experiments under communication interruption conditions validate CTCE's robustness to communication interruptions.
- [149] arXiv:2408.10532 [pdf, html, other]
-
Title: NutrifyAI: An AI-Powered System for Real-Time Food Detection, Nutritional Analysis, and Personalized Meal RecommendationsComments: 7 pages, 12 figuresSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
With diet and nutrition apps reaching 1.4 billion users in 2022 [1], it's no surprise that health apps like MyFitnessPal, Noom, and Calorie Counter, are surging in popularity. However, one major setback [2] of nearly all nutrition applications is that users must enter food data manually, which is time-consuming and tedious. Thus, there has been an increasing demand for applications that can accurately identify food items, analyze their nutritional content, and offer dietary recommendations in real-time. This paper introduces a comprehensive system that combines advanced computer vision techniques with nutrition analysis, implemented in a versatile mobile and web application. The system is divided into three key components: 1) food detection using the YOLOv8 model, 2) nutrient analysis via the Edamam Nutrition Analysis API, and 3) personalized meal recommendations using the Edamam Meal Planning and Recipe Search APIs. Designed for both mobile and web platforms, the application ensures fast processing times with an intuitive user interface, with features such as data visualizations using Chart.js, a login system, and personalized settings for dietary preferences, allergies, and cuisine choices. Preliminary results showcase the system's effectiveness, making it a valuable tool for users to make informed dietary decisions.
- [150] arXiv:2408.10533 [pdf, html, other]
-
Title: FAGStyle: Feature Augmentation on Geodesic Surface for Zero-shot Text-guided Diffusion Image Style TransferSubjects: Computer Vision and Pattern Recognition (cs.CV)
The goal of image style transfer is to render an image guided by a style reference while maintaining the original content. Existing image-guided methods rely on specific style reference images, restricting their wider application and potentially compromising result quality. As a flexible alternative, text-guided methods allow users to describe the desired style using text prompts. Despite their versatility, these methods often struggle with maintaining style consistency, reflecting the described style accurately, and preserving the content of the target image. To address these challenges, we introduce FAGStyle, a zero-shot text-guided diffusion image style transfer method. Our approach enhances inter-patch information interaction by incorporating the Sliding Window Crop technique and Feature Augmentation on Geodesic Surface into our style control loss. Furthermore, we integrate a Pre-Shape self-correlation consistency loss to ensure content consistency. FAGStyle demonstrates superior performance over existing methods, consistently achieving stylization that retains the semantic content of the source image. Experimental results confirms the efficacy of FAGStyle across a diverse range of source contents and styles, both imagined and common.
- [151] arXiv:2408.10536 [pdf, html, other]
-
Title: Synergistic Approach for Simultaneous Optimization of Monolingual, Cross-lingual, and Multilingual Information RetrievalComments: 15 pages, 2 figures, 13 tablesSubjects: Information Retrieval (cs.IR); Computation and Language (cs.CL)
Information retrieval across different languages is an increasingly important challenge in natural language processing. Recent approaches based on multilingual pre-trained language models have achieved remarkable success, yet they often optimize for either monolingual, cross-lingual, or multilingual retrieval performance at the expense of others. This paper proposes a novel hybrid batch training strategy to simultaneously improve zero-shot retrieval performance across monolingual, cross-lingual, and multilingual settings while mitigating language bias. The approach fine-tunes multilingual language models using a mix of monolingual and cross-lingual question-answer pair batches sampled based on dataset size. Experiments on XQuAD-R, MLQA-R, and MIRACL benchmark datasets show that the proposed method consistently achieves comparable or superior results in zero-shot retrieval across various languages and retrieval tasks compared to monolingual-only or cross-lingual-only training. Hybrid batch training also substantially reduces language bias in multilingual retrieval compared to monolingual training. These results demonstrate the effectiveness of the proposed approach for learning language-agnostic representations that enable strong zero-shot retrieval performance across diverse languages.
- [152] arXiv:2408.10537 [pdf, html, other]
-
Title: Subspace Prototype Guidance for Mitigating Class Imbalance in Point Cloud Semantic SegmentationSubjects: Computer Vision and Pattern Recognition (cs.CV)
Point cloud semantic segmentation can significantly enhance the perception of an intelligent agent. Nevertheless, the discriminative capability of the segmentation network is influenced by the quantity of samples available for different categories. To mitigate the cognitive bias induced by class imbalance, this paper introduces a novel method, namely subspace prototype guidance (\textbf{SPG}), to guide the training of segmentation network. Specifically, the point cloud is initially separated into independent point sets by category to provide initial conditions for the generation of feature subspaces. The auxiliary branch which consists of an encoder and a projection head maps these point sets into separate feature subspaces. Subsequently, the feature prototypes which are extracted from the current separate subspaces and then combined with prototypes of historical subspaces guide the feature space of main branch to enhance the discriminability of features of minority categories. The prototypes derived from the feature space of main branch are also employed to guide the training of the auxiliary branch, forming a supervisory loop to maintain consistent convergence of the entire network. The experiments conducted on the large public benchmarks (i.e. S3DIS, ScanNet v2, ScanNet200, Toronto-3D) and collected real-world data illustrate that the proposed method significantly improves the segmentation performance and surpasses the state-of-the-art method. The code is available at \url{this https URL}.
- [153] arXiv:2408.10538 [pdf, html, other]
-
Title: Surgical Workflow Recognition and Blocking Effectiveness Detection in Laparoscopic Liver Resections with Pringle ManeuverSubjects: Computer Vision and Pattern Recognition (cs.CV)
Pringle maneuver (PM) in laparoscopic liver resection aims to reduce blood loss and provide a clear surgical view by intermittently blocking blood inflow of the liver, whereas prolonged PM may cause ischemic injury. To comprehensively monitor this surgical procedure and provide timely warnings of ineffective and prolonged blocking, we suggest two complementary AI-assisted surgical monitoring tasks: workflow recognition and blocking effectiveness detection in liver resections. The former presents challenges in real-time capturing of short-term PM, while the latter involves the intraoperative discrimination of long-term liver ischemia states. To address these challenges, we meticulously collect a novel dataset, called PmLR50, consisting of 25,037 video frames covering various surgical phases from 50 laparoscopic liver resection procedures. Additionally, we develop an online baseline for PmLR50, termed PmNet. This model embraces Masked Temporal Encoding (MTE) and Compressed Sequence Modeling (CSM) for efficient short-term and long-term temporal information modeling, and embeds Contrastive Prototype Separation (CPS) to enhance action discrimination between similar intraoperative operations. Experimental results demonstrate that PmNet outperforms existing state-of-the-art surgical workflow recognition methods on the PmLR50 benchmark. Our research offers potential clinical applications for the laparoscopic liver surgery community. Source code and data will be publicly available.
- [154] arXiv:2408.10539 [pdf, html, other]
-
Title: Training Matting Models without Alpha LabelsComments: 12 pages, 12 figuresSubjects: Computer Vision and Pattern Recognition (cs.CV)
The labelling difficulty has been a longstanding problem in deep image matting. To escape from fine labels, this work explores using rough annotations such as trimaps coarsely indicating the foreground/background as supervision. We present that the cooperation between learned semantics from indicated known regions and proper assumed matting rules can help infer alpha values at transition areas. Inspired by the nonlocal principle in traditional image matting, we build a directional distance consistency loss (DDC loss) at each pixel neighborhood to constrain the alpha values conditioned on the input image. DDC loss forces the distance of similar pairs on the alpha matte and on its corresponding image to be consistent. In this way, the alpha values can be propagated from learned known regions to unknown transition areas. With only images and trimaps, a matting model can be trained under the supervision of a known loss and the proposed DDC loss. Experiments on AM-2K and P3M-10K dataset show that our paradigm achieves comparable performance with the fine-label-supervised baseline, while sometimes offers even more satisfying results than human-labelled ground truth. Code is available at \url{this https URL}.
- [155] arXiv:2408.10541 [pdf, html, other]
-
Title: The Instance-centric Transformer for the RVOS Track of LSVOS Challenge: 3rd Place SolutionComments: arXiv admin note: substantial text overlap with arXiv:2406.13939Subjects: Computer Vision and Pattern Recognition (cs.CV)
Referring Video Object Segmentation is an emerging multi-modal task that aims to segment objects in the video given a natural language expression. In this work, we build two instance-centric models and fuse predicted results from frame-level and instance-level. First, we introduce instance mask into the DETR-based model for query initialization to achieve temporal enhancement and employ SAM for spatial refinement. Secondly, we build an instance retrieval model conducting binary instance mask classification whether the instance is referred. Finally, we fuse predicted results and our method achieved a score of 52.67 J&F in the validation phase and 60.36 J&F in the test phase, securing the final ranking of 3rd place in the 6-th LSVOS Challenge RVOS Track.
- [156] arXiv:2408.10543 [pdf, html, other]
-
Title: Diff-PCC: Diffusion-based Neural Compression for 3D Point CloudsSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Image and Video Processing (eess.IV)
Stable diffusion networks have emerged as a groundbreaking development for their ability to produce realistic and detailed visual content. This characteristic renders them ideal decoders, capable of producing high-quality and aesthetically pleasing reconstructions. In this paper, we introduce the first diffusion-based point cloud compression method, dubbed Diff-PCC, to leverage the expressive power of the diffusion model for generative and aesthetically superior decoding. Different from the conventional autoencoder fashion, a dual-space latent representation is devised in this paper, in which a compressor composed of two independent encoding backbones is considered to extract expressive shape latents from distinct latent spaces. At the decoding side, a diffusion-based generator is devised to produce high-quality reconstructions by considering the shape latents as guidance to stochastically denoise the noisy point clouds. Experiments demonstrate that the proposed Diff-PCC achieves state-of-the-art compression performance (e.g., 7.711 dB BD-PSNR gains against the latest G-PCC standard at ultra-low bitrate) while attaining superior subjective quality. Source code will be made publicly available.
- [157] arXiv:2408.10547 [pdf, html, other]
-
Title: Semi-on-Demand Off-Peak Transit Services with Shared Autonomous Vehicles -- Service Planning, Simulation, and Analysis in Munich, GermanyMax T.M. Ng, Roman Engelhardt, Florian Dandl, Vasileios Volakakis, Hani S. Mahmassani, Klaus BogenbergerComments: 25 pages, 10 figuresSubjects: Systems and Control (eess.SY); Optimization and Control (math.OC)
This study investigates the implementation of semi-on-demand (SoD) hybrid-route services using Shared Autonomous Vehicles (SAVs) on existing transit lines. SoD services combine the cost efficiency of fixed-route buses with the flexibility of on-demand services. SAVs first serve all scheduled fixed-route stops, then drop off and pick up passengers in the pre-determined flexible-route portion, and return to the fixed route. This study addresses four key questions: optimal fleet and vehicle sizes for peak-hour fixed-route services with SAVs and during transition (from drivers to autonomous vehicles), optimal off-peak SoD service planning, and suitable use cases. The methodology combines analytical modeling for service planning with agent-based simulation for operational analysis. We examine ten bus routes in Munich, Germany, considering full SAV and transition scenarios with varying proportions of drivers. Our findings demonstrate that the lower operating costs of SAVs improve service quality through increased frequency and smaller vehicles, even in transition scenarios. The reduced headway lowers waiting time and also favors more flexible-route operation in SoD services. The optimal SoD settings range from fully flexible to hybrid routes, where higher occupancy from the terminus favors shorter flexible routes. During the transition phase, limited fleet size and higher headways constrain the benefits of flexible-route operations. The simulation results corroborate the SoD benefits of door-to-door convenience, attracting more passengers without excessive detours and operator costs at moderate flexible-route lengths, and validate the analytical model.
- [158] arXiv:2408.10548 [pdf, html, other]
-
Title: Language Modeling on Tabular Data: A Survey of Foundations, Techniques and EvolutionSubjects: Computation and Language (cs.CL)
Tabular data, a prevalent data type across various domains, presents unique challenges due to its heterogeneous nature and complex structural relationships. Achieving high predictive performance and robustness in tabular data analysis holds significant promise for numerous applications. Influenced by recent advancements in natural language processing, particularly transformer architectures, new methods for tabular data modeling have emerged. Early techniques concentrated on pre-training transformers from scratch, often encountering scalability issues. Subsequently, methods leveraging pre-trained language models like BERT have been developed, which require less data and yield enhanced performance. The recent advent of large language models, such as GPT and LLaMA, has further revolutionized the field, facilitating more advanced and diverse applications with minimal fine-tuning. Despite the growing interest, a comprehensive survey of language modeling techniques for tabular data remains absent. This paper fills this gap by providing a systematic review of the development of language modeling for tabular data, encompassing: (1) a categorization of different tabular data structures and data types; (2) a review of key datasets used in model training and tasks used for evaluation; (3) a summary of modeling techniques including widely-adopted data processing methods, popular architectures, and training objectives; (4) the evolution from adapting traditional Pre-training/Pre-trained language models to the utilization of large language models; (5) an identification of persistent challenges and potential future research directions in language modeling for tabular data analysis. GitHub page associated with this survey is available at: this https URL.
- [159] arXiv:2408.10549 [pdf, html, other]
-
Title: AI-Based IVRComments: in Russian languageSubjects: Artificial Intelligence (cs.AI)
The use of traditional IVR (Interactive Voice Response) methods often proves insufficient to meet customer needs. This article examines the application of artificial intelligence (AI) technologies to enhance the efficiency of IVR systems in call centers. A proposed approach is based on the integration of speech-to-text conversion solutions, text query classification using large language models (LLM), and speech synthesis. Special attention is given to adapting these technologies to work with the Kazakh language, including fine-tuning models on specialized datasets. The practical aspects of implementing the developed system in a real call center for query classification are described. The research results demonstrate that the application of AI technologies in call center IVR systems reduces operator workload, improves customer service quality, and increases the efficiency of query processing. The proposed approach can be adapted for use in call centers operating with various languages.
- [160] arXiv:2408.10554 [pdf, html, other]
-
Title: Ethics of Software Programming with Generative AI: Is Programming without Generative AI always radical?Marcellin Atemkeng, Sisipho Hamlomo, Brian Welman, Nicole Oyentunji, Pouya Ataei, Jean Louis K. E FendjiSubjects: Software Engineering (cs.SE)
This paper provides a comprehensive analysis of Generative AI (GenAI) potential to revolutionise software coding through increased efficiency and reduced time span for writing code. It acknowledges the transformative power of GenAI in software code generation, while also cautioning against the inherent risks of bias and errors if left unchecked. Emphasising the irreplaceable value of traditional programming, it posits that GenAI is not a replacement but a complementary tool for writing software code. Ethical considerations are paramount with the paper advocating for stringent ethical guidelines to ensure GenAI serves the greater good and does not compromise on accountability in writing software code. It suggests a balanced approach, combining human oversight with AI's capabilities, to mitigate risks and enhance reliability. The paper concludes by proposing guidelines for GenAI utilisation in coding, which will empower developers to navigate its complexities and employ it responsibly. This approach addresses current ethical concerns and sets a foundation for the judicious use of GenAI in the future, ensuring its benefits are harnessed effectively while maintaining moral integrity.
- [161] arXiv:2408.10555 [pdf, html, other]
-
Title: Target-Prompt Online Graph Collaborative Learning for Temporal QoS PredictionComments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibleSubjects: Machine Learning (cs.LG); Information Retrieval (cs.IR)
In service-oriented architecture, accurately predicting the Quality of Service (QoS) is vital for maintaining reliability and enhancing user satisfaction. However, current methods often neglect high-order latent collaborative relationships and fail to dynamically adjust feature learning for specific user-service invocations, which are critical for precise feature extraction. Moreover, relying on RNNs to capture QoS evolution limits the ability to detect long-term trends due to challenges in managing long-range dependencies. To address these issues, we propose the Target-Prompt Online Graph Collaborative Learning (TOGCL) framework for temporal QoS prediction. It leverages a dynamic user-service invocation graph to comprehensively model historical interactions. Building on this graph, it develops a target-prompt graph attention network to extract online deep latent features of users and services at each time slice, considering implicit target-neighboring collaborative relationships and historical QoS values. Additionally, a multi-layer Transformer encoder is employed to uncover temporal feature evolution patterns, enhancing temporal QoS prediction. Extensive experiments on the WS-DREAM dataset demonstrate that TOGCL significantly outperforms state-of-the-art methods across multiple metrics, achieving improvements of up to 38.80\%. These results underscore the effectiveness of TOGCL for temporal QoS prediction.
- [162] arXiv:2408.10556 [pdf, html, other]
-
Title: Hokoff: Real Game Dataset from Honor of Kings and its Offline Reinforcement Learning BenchmarksYun Qu, Boyuan Wang, Jianzhun Shao, Yuhang Jiang, Chen Chen, Zhenbin Ye, Lin Liu, Junfeng Yang, Lin Lai, Hongyang Qin, Minwen Deng, Juchao Zhuo, Deheng Ye, Qiang Fu, Wei Yang, Guang Yang, Lanxiao Huang, Xiangyang JiSubjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
The advancement of Offline Reinforcement Learning (RL) and Offline Multi-Agent Reinforcement Learning (MARL) critically depends on the availability of high-quality, pre-collected offline datasets that represent real-world complexities and practical applications. However, existing datasets often fall short in their simplicity and lack of realism. To address this gap, we propose Hokoff, a comprehensive set of pre-collected datasets that covers both offline RL and offline MARL, accompanied by a robust framework, to facilitate further research. This data is derived from Honor of Kings, a recognized Multiplayer Online Battle Arena (MOBA) game known for its intricate nature, closely resembling real-life situations. Utilizing this framework, we benchmark a variety of offline RL and offline MARL algorithms. We also introduce a novel baseline algorithm tailored for the inherent hierarchical action space of the game. We reveal the incompetency of current offline RL approaches in handling task complexity, generalization and multi-task learning.
- [163] arXiv:2408.10557 [pdf, html, other]
-
Title: Speech Representation Learning Revisited: The Necessity of Separate Learnable Parameters and Robust Data AugmentationSubjects: Computation and Language (cs.CL)
Speech modeling methods learn one embedding for a fixed segment of speech, typically in between 10-25 ms. The information present in speech can be divided into two categories: "what is being said" (content) and "how it is expressed" (other) and these two are orthogonal in nature causing the optimization algorithm to find a sub-optimal solution if forced to optimize together. This leads to sub-optimal performance in one or all downstream tasks as shown by previous studies. Current self-supervised learning (SSL) methods such as HuBERT are very good at modeling the content information present in speech. Data augmentation improves the performance on tasks which require effective modeling of other information but this leads to a divided capacity of the model. In this work, we conduct a preliminary study to understand the importance of modeling other information using separate learnable parameters. We propose a modified version of HuBERT, termed Other HuBERT (O-HuBERT), to test our hypothesis. Our findings are twofold: first, the O-HuBERT method is able to utilize all layers to build complex features to encode other information; second, a robust data augmentation strategy is essential for learning the information required by tasks that depend on other information and to achieve state-of-the-art (SOTA) performance on the SUPERB benchmark with a similarly sized model (100 million parameters) and pre-training data (960 hours).
- [164] arXiv:2408.10561 [pdf, html, other]
-
Title: ICSD: An Open-source Dataset for Infant Cry and Snoring DetectionComments: 11 pages, 6 figuresSubjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)
The detection and analysis of infant cry and snoring events are crucial tasks within the field of audio signal processing. While existing datasets for general sound event detection are plentiful, they often fall short in providing sufficient, strongly labeled data specific to infant cries and snoring. To provide a benchmark dataset and thus foster the research of infant cry and snoring detection, this paper introduces the Infant Cry and Snoring Detection (ICSD) dataset, a novel, publicly available dataset specially designed for ICSD tasks. The ICSD comprises three types of subsets: a real strongly labeled subset with event-based labels annotated manually, a weakly labeled subset with only clip-level event annotations, and a synthetic subset generated and labeled with strong annotations. This paper provides a detailed description of the ICSD creation process, including the challenges encountered and the solutions adopted. We offer a comprehensive characterization of the dataset, discussing its limitations and key factors for ICSD usage. Additionally, we conduct extensive experiments on the ICSD dataset to establish baseline systems and offer insights into the main factors when using this dataset for ICSD research. Our goal is to develop a dataset that will be widely adopted by the community as a new open benchmark for future ICSD research.
- [165] arXiv:2408.10562 [pdf, html, other]
-
Title: Kalib: Markerless Hand-Eye Calibration with Keypoint TrackingComments: The code and supplementary materials are available at this https URLSubjects: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
Hand-eye calibration involves estimating the transformation between the camera and the robot. Traditional methods rely on fiducial markers, involving much manual labor and careful setup. Recent advancements in deep learning offer markerless techniques, but they present challenges, including the need for retraining networks for each robot, the requirement of accurate mesh models for data generation, and the need to address the sim-to-real gap. In this letter, we propose Kalib, an automatic and universal markerless hand-eye calibration pipeline that leverages the generalizability of visual foundation models to eliminate these barriers. In each calibration process, Kalib uses keypoint tracking and proprioceptive sensors to estimate the transformation between a robot's coordinate space and its corresponding points in camera space. Our method does not require training new networks or access to mesh models. Through evaluations in simulation environments and the real-world dataset DROID, Kalib demonstrates superior accuracy compared to recent baseline methods. This approach provides an effective and flexible calibration process for various robot systems by simplifying setup and removing dependency on precise physical markers.
- [166] arXiv:2408.10563 [pdf, html, other]
-
Title: The Stable Model Semantics for Higher-Order Logic ProgrammingBart Bogaerts, Angelos Charalambidis, Giannos Chatziagapis, Babis Kostopoulos, Samuele Pollaci, Panos RondogiannisSubjects: Logic in Computer Science (cs.LO); Artificial Intelligence (cs.AI); Programming Languages (cs.PL)
We propose a stable model semantics for higher-order logic programs. Our semantics is developed using Approximation Fixpoint Theory (AFT), a powerful formalism that has successfully been used to give meaning to diverse non-monotonic formalisms. The proposed semantics generalizes the classical two-valued stable model semantics of (Gelfond and Lifschitz 1988) as-well-as the three-valued one of (Przymusinski 1990), retaining their desirable properties. Due to the use of AFT, we also get for free alternative semantics for higher-order logic programs, namely supported model, Kripke-Kleene, and well-founded. Additionally, we define a broad class of stratified higher-order logic programs and demonstrate that they have a unique two-valued higher-order stable model which coincides with the well-founded semantics of such programs. We provide a number of examples in different application domains, which demonstrate that higher-order logic programming under the stable model semantics is a powerful and versatile formalism, which can potentially form the basis of novel ASP systems.
- [167] arXiv:2408.10564 [pdf, other]
-
Title: Fault Tolerant Dynamic Task Assignment for UAV-based Search TeamsSubjects: Systems and Control (eess.SY)
This research offers a novel framework for dynamic task assignment for unmanned aerial vehicles (UAVs) in cooperative search settings. Notably, it incorporates post-fault UAV capabilities into job assignment techniques, assuring operational dependability in the event of sensor and actuator failures. A significant innovation is the utilization of UAV battery charge to assess range relative to search objectives, hence improving job distribution while conserving battery life. This model integrates repair, recharge, and stochastic goal recurrence, hence increasing its real-world applicability. Using stochastic dynamic programming, this method makes it simpler to determine optimal assignment policies offline so they may be implemented rapidly online. This paper emphasizes the holistic aspect of the proposed model, which connects high-level task rules to low-level control capabilities. A simulation-based case study proves its usefulness, highlighting its robustness in fault-prone and battery-variable settings. Overall, this paper proposes and demonstrates a comprehensive method for assigning UAV tasks that integrates defect awareness, battery management, and multilayer control through the use of stochastic dynamic programming.
- [168] arXiv:2408.10566 [pdf, html, other]
-
Title: SparseGrow: Addressing Growth-Induced Forgetting in Task-Agnostic Continual LearningComments: This paper has been submitted to the AAAI conference. If accepted, the final version will be updated to reflect the conference proceedingsSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
In continual learning (CL), model growth enhances adaptability over new data, improving knowledge retention for more tasks. However, improper model growth can lead to severe degradation of previously learned knowledge, an issue we name as growth-induced forgetting (GIFt), especially in task-agnostic CL using entire grown model for inference. Existing works, despite adopting model growth and random initialization for better adaptability, often fail to recognize the presence of GIFt caused by improper model growth. This oversight limits comprehensive control of forgetting and hinders full utilization of model growth. We are the first in CL to identify this issue and conduct an in-depth study on root cause of GIFt, where layer expansion stands out among model growth strategies, widening layers without affecting model functionality. Yet, direct adoption of layer expansion presents challenges. It lacks data-driven control and initialization of expanded parameters to balance adaptability and knowledge retention. This paper presents a novel SparseGrow approach to overcome the issue of GIFt while enhancing adaptability over new data. SparseGrow employs data-driven sparse layer expansion to control efficient parameter usage during growth, reducing GIFt from excessive growth and functionality changes. It also combines sparse growth with on-data initialization at training late-stage to create partially 0-valued expansions that fit learned distribution, enhancing retention and adaptability. To further minimize forgetting, freezing is applied by calculating the sparse mask, allowing data-driven preservation of important parameters. Through experiments across datasets with various settings, cases and task numbers, we demonstrate the necessity of layer expansion and showcase the effectiveness of SparseGrow in overcoming GIFt, highlighting its adaptability and knowledge retention for incremental tasks.
- [169] arXiv:2408.10568 [pdf, html, other]
-
Title: Constrained Behavior Cloning for Robotic LearningSubjects: Robotics (cs.RO)
Behavior cloning (BC) is a popular supervised imitation learning method in the societies of robotics, autonomous driving, etc., wherein complex skills can be learned by direct imitation from expert demonstrations. Despite its rapid development, it is still affected by limited field of view where accumulation of sensors and joint noise bring compounding errors. In this paper, we introduced geometrically and historically constrained behavior cloning (GHCBC) to dominantly consider high-level state information inspired by neuroscientists, wherein the geometrically constrained behavior cloning were used to geometrically constrain predicting poses, and the historically constrained behavior cloning were utilized to temporally constrain action sequences. The synergy between these two types of constrains enhanced the BC performance in terms of robustness and stability. Comprehensive experimental results showed that success rates were improved by 29.73% in simulation and 39.4% in real robot experiments in average, respectively, compared to state-of-the-art BC method, especially in long-term operational scenes, indicating great potential of using the GHCBC for robotic learning.
- [170] arXiv:2408.10569 [pdf, html, other]
-
Title: Navigating Dimensionality through State Machines in Automotive System ValidationComments: 10 pages, 5 figures, 2 figures in AppendixSubjects: Robotics (cs.RO)
The increasing automation of vehicles is resulting in the integration of more extensive in-vehicle sensor systems, electronic control units, and software. Additionally, vehicle-to-everything communication is seen as an opportunity to extend automated driving capabilities through information from a source outside the ego vehicle. However, the validation and verification of automated driving functions already pose a challenge due to the number of possible scenarios that can occur for a driving function, which makes it difficult to achieve comprehensive test coverage. Currently, the establishment of Safety Of The Intended Functionality ( SOTIF ) mandates the implementation of scenario-based testing. The introduction of additional external systems through vehicle-to-everything further complicates the problem and increases the scenario space. In this paper, a methodology based on state charts is proposed for modeling the interaction with external systems, which may remain as black boxes. This approach leverages the testability and coverage analysis inherent in state charts by combining them with scenario-based testing. The overall objective is to reduce the space of scenarios necessary for testing a networked driving function and to streamline validation and verification. The utilization of this approach is demonstrated using a simulated signalized intersection with a roadside unit that detects vulnerable road users.
- [171] arXiv:2408.10571 [pdf, html, other]
-
Title: Prompt-Agnostic Adversarial Perturbation for Customized Diffusion ModelsComments: 33 pages, 14 figures, under reviewSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Diffusion models have revolutionized customized text-to-image generation, allowing for efficient synthesis of photos from personal data with textual descriptions. However, these advancements bring forth risks including privacy breaches and unauthorized replication of artworks. Previous researches primarily center around using prompt-specific methods to generate adversarial examples to protect personal images, yet the effectiveness of existing methods is hindered by constrained adaptability to different prompts. In this paper, we introduce a Prompt-Agnostic Adversarial Perturbation (PAP) method for customized diffusion models. PAP first models the prompt distribution using a Laplace Approximation, and then produces prompt-agnostic perturbations by maximizing a disturbance expectation based on the modeled distribution. This approach effectively tackles the prompt-agnostic attacks, leading to improved defense stability. Extensive experiments in face privacy and artistic style protection, demonstrate the superior generalization of our method in comparison to existing techniques.
- [172] arXiv:2408.10573 [pdf, html, other]
-
Title: Putting People in LLMs' Shoes: Generating Better Answers via Question RewriterComments: 7 pages, 4 figures, 5 tablesSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Large Language Models (LLMs) have demonstrated significant capabilities, particularly in the domain of question answering (QA). However, their effectiveness in QA is often undermined by the vagueness of user questions. To address this issue, we introduce single-round instance-level prompt optimization, referred to as question rewriter. By enhancing the intelligibility of human questions for black-box LLMs, our question rewriter improves the quality of generated answers. The rewriter is optimized using direct preference optimization based on feedback collected from automatic criteria for evaluating generated answers; therefore, its training does not require costly human annotations. The experiments across multiple black-box LLMs and long-form question answering (LFQA) datasets demonstrate the efficacy of our method. This paper provides a practical framework for training question rewriters and sets a precedent for future explorations in prompt optimization within LFQA tasks. Code is available at \url{this https URL}.
- [173] arXiv:2408.10575 [pdf, html, other]
-
Title: MUSE: Mamba is Efficient Multi-scale Learner for Text-video RetrievalComments: 8 pagesSubjects: Computer Vision and Pattern Recognition (cs.CV)
Text-Video Retrieval (TVR) aims to align and associate relevant video content with corresponding natural language queries. Most existing TVR methods are based on large-scale pre-trained vision-language models (e.g., CLIP). However, due to the inherent plain structure of CLIP, few TVR methods explore the multi-scale representations which offer richer contextual information for a more thorough understanding. To this end, we propose MUSE, a multi-scale mamba with linear computational complexity for efficient cross-resolution modeling. Specifically, the multi-scale representations are generated by applying a feature pyramid on the last single-scale feature map. Then, we employ the Mamba structure as an efficient multi-scale learner to jointly learn scale-wise representations. Furthermore, we conduct comprehensive studies to investigate different model structures and designs. Extensive results on three popular benchmarks have validated the superiority of MUSE.
- [174] arXiv:2408.10577 [pdf, other]
-
Title: Optimizing Large Language Model Hyperparameters for Code GenerationSubjects: Software Engineering (cs.SE)
Large Language Models (LLMs), such as GPT models, are increasingly used in software engineering for various tasks, such as code generation, requirements management, and debugging. While automating these tasks has garnered significant attention, a systematic study on the impact of varying hyperparameters on code generation outcomes remains unexplored. This study aims to assess LLMs' code generation performance by exhaustively exploring the impact of various hyperparameters. Hyperparameters for LLMs are adjustable settings that affect the model's behaviour and performance. Specifically, we investigated how changes to the hyperparameters: temperature, top probability (top_p), frequency penalty, and presence penalty affect code generation outcomes. We systematically adjusted all hyperparameters together, exploring every possible combination by making small increments to each hyperparameter at a time. This exhaustive approach was applied to 13 Python code generation tasks, yielding one of four outcomes for each hyperparameter combination: no output from the LLM, non executable code, code that fails unit tests, or correct and functional code. We analysed these outcomes for a total of 14,742 generated Python code segments, focusing on correctness, to determine how the hyperparameters influence the LLM to arrive at each outcome. Using correlation coefficient and regression tree analyses, we ascertained which hyperparameters influence which aspect of the LLM. Our results indicate that optimal performance is achieved with a temperature below 0.5, top probability below 0.75, frequency penalty above -1 and below 1.5, and presence penalty above -1. We make our dataset and results available to facilitate replication.
- [175] arXiv:2408.10578 [pdf, html, other]
-
Title: Where to Fetch: Extracting Visual Scene Representation from Large Pre-Trained Models for Robotic Goal NavigationSubjects: Robotics (cs.RO)
To complete a complex task where a robot navigates to a goal object and fetches it, the robot needs to have a good understanding of the instructions and the surrounding environment. Large pre-trained models have shown capabilities to interpret tasks defined via language descriptions. However, previous methods attempting to integrate large pre-trained models with daily tasks are not competent in many robotic goal navigation tasks due to poor understanding of the environment. In this work, we present a visual scene representation built with large-scale visual language models to form a feature representation of the environment capable of handling natural language queries. Combined with large language models, this method can parse language instructions into action sequences for a robot to follow, and accomplish goal navigation with querying the scene representation. Experiments demonstrate that our method enables the robot to follow a wide range of instructions and complete complex goal navigation tasks.
- [176] arXiv:2408.10580 [pdf, html, other]
-
Title: A Passivity-Based Variable Impedance Controller for Incremental Learning of Periodic Interactive TasksComments: Accepted at the 2024 IEEE 20th International Conference on Automation Science and Engineering (CASE)Subjects: Robotics (cs.RO)
In intelligent manufacturing, robots are asked to dynamically adapt their behaviours without reducing productivity. Human teaching, where an operator physically interacts with the robot to demonstrate a new task, is a promising strategy to quickly and intuitively reconfigure the production line. However, physical guidance during task execution poses challenges in terms of both operator safety and system usability. In this paper, we solve this issue by designing a variable impedance control strategy that regulates the interaction with the environment and the physical demonstrations, explicitly preventing at the same time passivity violations. We derive constraints to limit not only the exchanged energy with the environment but also the exchanged power, resulting in smoother interactions. By monitoring the energy flow between the robot and the environment, we are able to distinguish between disturbances (to be rejected) and physical guidance (to be accomplished), enabling smooth and controlled transitions from teaching to execution and vice versa. The effectiveness of the proposed approach is validated in wiping tasks with a real robotic manipulator.
- [177] arXiv:2408.10581 [pdf, html, other]
-
Title: Multi-view Hand Reconstruction with a Point-Embedded TransformerComments: Generalizable multi-view Hand Mesh Reconstruction (HMR) model. Extension of the original work at CVPR2023Subjects: Computer Vision and Pattern Recognition (cs.CV)
This work introduces a novel and generalizable multi-view Hand Mesh Reconstruction (HMR) model, named POEM, designed for practical use in real-world hand motion capture scenarios. The advances of the POEM model consist of two main aspects. First, concerning the modeling of the problem, we propose embedding a static basis point within the multi-view stereo space. A point represents a natural form of 3D information and serves as an ideal medium for fusing features across different views, given its varied projections across these views. Consequently, our method harnesses a simple yet effective idea: a complex 3D hand mesh can be represented by a set of 3D basis points that 1) are embedded in the multi-view stereo, 2) carry features from the multi-view images, and 3) encompass the hand in it. The second advance lies in the training strategy. We utilize a combination of five large-scale multi-view datasets and employ randomization in the number, order, and poses of the cameras. By processing such a vast amount of data and a diverse array of camera configurations, our model demonstrates notable generalizability in the real-world applications. As a result, POEM presents a highly practical, plug-and-play solution that enables user-friendly, cost-effective multi-view motion capture for both left and right hands. The model and source codes are available at this https URL.
- [178] arXiv:2408.10588 [pdf, html, other]
-
Title: DEGAS: Detailed Expressions on Full-Body Gaussian AvatarsZhijing Shao, Duotun Wang, Qing-Yao Tian, Yao-Dong Yang, Hengyu Meng, Zeyu Cai, Bo Dong, Yu Zhang, Kang Zhang, Zeyu WangSubjects: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
Although neural rendering has made significant advancements in creating lifelike, animatable full-body and head avatars, incorporating detailed expressions into full-body avatars remains largely unexplored. We present DEGAS, the first 3D Gaussian Splatting (3DGS)-based modeling method for full-body avatars with rich facial expressions. Trained on multiview videos of a given subject, our method learns a conditional variational autoencoder that takes both the body motion and facial expression as driving signals to generate Gaussian maps in the UV layout. To drive the facial expressions, instead of the commonly used 3D Morphable Models (3DMMs) in 3D head avatars, we propose to adopt the expression latent space trained solely on 2D portrait images, bridging the gap between 2D talking faces and 3D avatars. Leveraging the rendering capability of 3DGS and the rich expressiveness of the expression latent space, the learned avatars can be reenacted to reproduce photorealistic rendering images with subtle and accurate facial expressions. Experiments on an existing dataset and our newly proposed dataset of full-body talking avatars demonstrate the efficacy of our method. We also propose an audio-driven extension of our method with the help of 2D talking faces, opening new possibilities to interactive AI agents.
- [179] arXiv:2408.10589 [pdf, html, other]
-
Title: Bidirectional Intent Communication: A Role for Large Foundation ModelsComments: 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Workshop: Large Language Models in the RoMan AgeSubjects: Robotics (cs.RO); Human-Computer Interaction (cs.HC)
Integrating multimodal foundation models has significantly enhanced autonomous agents' language comprehension, perception, and planning capabilities. However, while existing works adopt a \emph{task-centric} approach with minimal human interaction, applying these models to developing assistive \emph{user-centric} robots that can interact and cooperate with humans remains underexplored. This paper introduces ``Bident'', a framework designed to integrate robots seamlessly into shared spaces with humans. Bident enhances the interactive experience by incorporating multimodal inputs like speech and user gaze dynamics. Furthermore, Bident supports verbal utterances and physical actions like gestures, making it versatile for bidirectional human-robot interactions. Potential applications include personalized education, where robots can adapt to individual learning styles and paces, and healthcare, where robots can offer personalized support, companionship, and everyday assistance in the home and workplace environments.
- [180] arXiv:2408.10592 [pdf, html, other]
-
Title: Hologram Reasoning for Solving Algebra Problems with Geometry DiagramsSubjects: Artificial Intelligence (cs.AI); Computational Geometry (cs.CG); Logic in Computer Science (cs.LO)
Solving Algebra Problems with Geometry Diagrams (APGDs) is still a challenging problem because diagram processing is not studied as intensively as language processing. To work against this challenge, this paper proposes a hologram reasoning scheme and develops a high-performance method for solving APGDs by using this scheme. To reach this goal, it first defines a hologram, being a kind of graph, and proposes a hologram generator to convert a given APGD into a hologram, which represents the entire information of APGD and the relations for solving the problem can be acquired from it by a uniform way. Then HGR, a hologram reasoning method employs a pool of prepared graph models to derive algebraic equations, which is consistent with the geometric theorems. This method is able to be updated by adding new graph models into the pool. Lastly, it employs deep reinforcement learning to enhance the efficiency of model selection from the pool. The entire HGR not only ensures high solution accuracy with fewer reasoning steps but also significantly enhances the interpretability of the solution process by providing descriptions of all reasoning steps. Experimental results demonstrate the effectiveness of HGR in improving both accuracy and interpretability in solving APGDs.
- [181] arXiv:2408.10593 [pdf, html, other]
-
Title: An Efficient Sign Language Translation Using Spatial Configuration and Motion Dynamics with LLMsComments: Under ReviewSubjects: Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
Gloss-free Sign Language Translation (SLT) converts sign videos directly into spoken language sentences without relying on glosses. Recently, Large Language Models (LLMs) have shown remarkable translation performance in gloss-free methods by harnessing their powerful natural language generation capabilities. However, these methods often rely on domain-specific fine-tuning of visual encoders to achieve optimal results. By contrast, this paper emphasizes the importance of capturing the spatial configurations and motion dynamics inherent in sign language. With this in mind, we introduce Spatial and Motion-based Sign Language Translation (SpaMo), a novel LLM-based SLT framework. The core idea of SpaMo is simple yet effective. We first extract spatial and motion features using off-the-shelf visual encoders and then input these features into an LLM with a language prompt. Additionally, we employ a visual-text alignment process as a warm-up before the SLT supervision. Our experiments demonstrate that SpaMo achieves state-of-the-art performance on two popular datasets, PHOENIX14T and How2Sign.
- [182] arXiv:2408.10594 [pdf, other]
-
Title: Design and Implementation of a Takum Arithmetic Hardware Codec in VHDLComments: 12 pages, 7 figuresSubjects: Hardware Architecture (cs.AR)
The takum machine number format has been recently proposed as an enhancement over the posit number format, which is considered a promising alternative to the IEEE 754 floating-point standard. Takums retain the useful posit properties, but feature a novel exponent coding scheme that yields more precision for small and large magnitude numbers. Takum's dynamic range is larger and bounded, as reflected in its name, derived from the Icelandic 'takmarkað umfang', meaning 'limited range'. Consequently, the selection of bit string lengths becomes determined solely by precision requirements and independent of dynamic range considerations. Takum is defined in both a logarithmic number system (LNS) format and a traditional floating-point format.
This paper presents the design and implementation of a hardware codec for both the logarithmic and floating-point takum formats. The design primarily focuses on the codec, as both formats share a common internal arithmetic representation. Non-essential aspects of current posit designs, such as fused or pipelined architectures and the choice of floating-point IP cores, are thus omitted. The proposed takum codec, implemented in VHDL, demonstrates near-optimal scalability and performance on an FPGA, matching or exceeding state-of-the-art posit codecs in terms of both latency and LUT utilisation. - [183] arXiv:2408.10595 [pdf, html, other]
-
Title: Synchronization behind Learning in Periodic Zero-Sum Games Triggers Divergence from Nash equilibriumComments: 8 pages, 5 figures (main); 7 pages, 1 figure (appendix)Subjects: Computer Science and Game Theory (cs.GT); Multiagent Systems (cs.MA); Optimization and Control (math.OC); Chaotic Dynamics (nlin.CD)
Learning in zero-sum games studies a situation where multiple agents competitively learn their strategy. In such multi-agent learning, we often see that the strategies cycle around their optimum, i.e., Nash equilibrium. When a game periodically varies (called a ``periodic'' game), however, the Nash equilibrium moves generically. How learning dynamics behave in such periodic games is of interest but still unclear. Interestingly, we discover that the behavior is highly dependent on the relationship between the two speeds at which the game changes and at which players learn. We observe that when these two speeds synchronize, the learning dynamics diverge, and their time-average does not converge. Otherwise, the learning dynamics draw complicated cycles, but their time-average converges. Under some assumptions introduced for the dynamical systems analysis, we prove that this behavior occurs. Furthermore, our experiments observe this behavior even if removing these assumptions. This study discovers a novel phenomenon, i.e., synchronization, and gains insight widely applicable to learning in periodic games.
- [184] arXiv:2408.10596 [pdf, html, other]
-
Title: Fast Collective Evasion in Self-Localized Swarms of Unmanned Aerial VehiclesJournal-ref: Bioinspiration & Biomimetics, vol. 16, no. 6, pp. 066025, 2021Subjects: Robotics (cs.RO)
A novel approach for achieving fast evasion in self-localized swarms of Unmanned Aerial Vehicles (UAVs) threatened by an intruding moving object is presented in this paper. Motivated by natural self-organizing systems, the presented approach of fast and collective evasion enables the UAV swarm to avoid dynamic objects (interferers) that are actively approaching the group. The main objective of the proposed technique is the fast and safe escape of the swarm from an interferer ~discovered in proximity. This method is inspired by the collective behavior of groups of certain animals, such as schools of fish or flocks of birds. These animals use the limited information of their sensing organs and decentralized control to achieve reliable and effective group motion. The system presented in this paper is intended to execute the safe coordination of UAV swarms with a large number of agents. Similar to natural swarms, this system propagates a fast shock of information about detected interferers throughout the group to achieve dynamic and collective evasion. The proposed system is fully decentralized using only onboard sensors to mutually localize swarm agents and interferers, similar to how animals accomplish this behavior. As a result, the communication structure between swarm agents is not overwhelmed by information about the state (position and velocity) of each individual and it is reliable to communication dropouts. The proposed system and theory were numerically evaluated and verified in real-world experiments.
- [185] arXiv:2408.10600 [pdf, other]
-
Title: Breast tumor classification based on self-supervised contrastive learning from ultrasound videosSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Background: Breast ultrasound is prominently used in diagnosing breast tumors. At present, many automatic systems based on deep learning have been developed to help radiologists in diagnosis. However, training such systems remains challenging because they are usually data-hungry and demand amounts of labeled data, which need professional knowledge and are expensive. Methods: We adopted a triplet network and a self-supervised contrastive learning technique to learn representations from unlabeled breast ultrasound video clips. We further designed a new hard triplet loss to to learn representations that particularly discriminate positive and negative image pairs that are hard to recognize. We also constructed a pretraining dataset from breast ultrasound videos (1,360 videos from 200 patients), which includes an anchor sample dataset with 11,805 images, a positive sample dataset with 188,880 images, and a negative sample dataset dynamically generated from video clips. Further, we constructed a finetuning dataset, including 400 images from 66 patients. We transferred the pretrained network to a downstream benign/malignant classification task and compared the performance with other state-of-the-art models, including three models pretrained on ImageNet and a previous contrastive learning model retrained on our datasets. Results and conclusion: Experiments revealed that our model achieved an area under the receiver operating characteristic curve (AUC) of 0.952, which is significantly higher than the others. Further, we assessed the dependence of our pretrained model on the number of labeled data and revealed that <100 samples were required to achieve an AUC of 0.901. The proposed framework greatly reduces the demand for labeled data and holds potential for use in automatic breast ultrasound image diagnosis.
- [186] arXiv:2408.10602 [pdf, html, other]
-
Title: MV-MOS: Multi-View Feature Fusion for 3D Moving Object SegmentationComments: 7 pages, 4 figuresSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Effectively summarizing dense 3D point cloud data and extracting motion information of moving objects (moving object segmentation, MOS) is crucial to autonomous driving and robotics applications. How to effectively utilize motion and semantic features and avoid information loss during 3D-to-2D projection is still a key challenge. In this paper, we propose a novel multi-view MOS model (MV-MOS) by fusing motion-semantic features from different 2D representations of point clouds. To effectively exploit complementary information, the motion branches of the proposed model combines motion features from both bird's eye view (BEV) and range view (RV) representations. In addition, a semantic branch is introduced to provide supplementary semantic features of moving objects. Finally, a Mamba module is utilized to fuse the semantic features with motion features and provide effective guidance for the motion branches. We validated the effectiveness of the proposed multi-branch fusion MOS framework via comprehensive experiments, and our proposed model outperforms existing state-of-the-art models on the SemanticKITTI benchmark.
- [187] arXiv:2408.10604 [pdf, other]
-
Title: Multilingual Non-Factoid Question Answering with Silver AnswersSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (cs.LG)
Most existing Question Answering Datasets (QuADs) primarily focus on factoid-based short-context Question Answering (QA) in high-resource languages. However, the scope of such datasets for low-resource languages remains limited, with only a few works centered on factoid-based QuADs and none on non-factoid QuADs. Therefore, this work presents MuNfQuAD, a multilingual QuAD with non-factoid questions. It utilizes interrogative sub-headings from BBC news articles as questions and the corresponding paragraphs as silver answers. The dataset comprises over 370K QA pairs across 38 languages, encompassing several low-resource languages, and stands as the largest multilingual QA dataset to date. Based on the manual annotations of 790 QA-pairs from MuNfQuAD (golden set), we observe that 98\% of questions can be answered using their corresponding silver answer. Our fine-tuned Answer Paragraph Selection (APS) model outperforms the baselines. The APS model attained an accuracy of 80\% and 72\%, as well as a macro F1 of 72\% and 66\%, on the MuNfQuAD testset and the golden set, respectively. Furthermore, the APS model effectively generalizes certain a language within the golden set, even after being fine-tuned on silver labels.
- [188] arXiv:2408.10605 [pdf, other]
-
Title: MUSES: 3D-Controllable Image Generation via Multi-Modal Agent CollaborationSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Despite recent advancements in text-to-image generation, most existing methods struggle to create images with multiple objects and complex spatial relationships in 3D world. To tackle this limitation, we introduce a generic AI system, namely MUSES, for 3D-controllable image generation from user queries. Specifically, our MUSES addresses this challenging task by developing a progressive workflow with three key components, including (1) Layout Manager for 2D-to-3D layout lifting, (2) Model Engineer for 3D object acquisition and calibration, (3) Image Artist for 3D-to-2D image rendering. By mimicking the collaboration of human professionals, this multi-modal agent pipeline facilitates the effective and automatic creation of images with 3D-controllable objects, through an explainable integration of top-down planning and bottom-up generation. Additionally, we find that existing benchmarks lack detailed descriptions of complex 3D spatial relationships of multiple objects. To fill this gap, we further construct a new benchmark of T2I-3DisBench (3D image scene), which describes diverse 3D image scenes with 50 detailed prompts. Extensive experiments show the state-of-the-art performance of MUSES on both T2I-CompBench and T2I-3DisBench, outperforming recent strong competitors such as DALL-E 3 and Stable Diffusion 3. These results demonstrate a significant step of MUSES forward in bridging natural language, 2D image generation, and 3D world.
- [189] arXiv:2408.10608 [pdf, html, other]
-
Title: Promoting Equality in Large Language Models: Identifying and Mitigating the Implicit Bias based on Bayesian TheoryYongxin Deng (1), Xihe Qiu (1), Xiaoyu Tan (2), Jing Pan (3), Chen Jue (1), Zhijun Fang (4), Yinghui Xu (5), Wei Chu (2), Yuan Qi (5) ((1) Shanghai University of Engineering Science, (2) INF Technology (Shanghai) Co., Ltd., (3) Monash University, (4) Donghua University, (5) Fudan University)Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Large language models (LLMs) are trained on extensive text corpora, which inevitably include biased information. Although techniques such as Affective Alignment can mitigate some negative impacts of these biases, existing prompt-based attack methods can still extract these biases from the model's weights. Moreover, these biases frequently appear subtly when LLMs are prompted to perform identical tasks across different demographic groups, thereby camouflaging their presence. To address this issue, we have formally defined the implicit bias problem and developed an innovative framework for bias removal based on Bayesian theory, Bayesian-Theory based Bias Removal (BTBR). BTBR employs likelihood ratio screening to pinpoint data entries within publicly accessible biased datasets that represent biases inadvertently incorporated during the LLM training phase. It then automatically constructs relevant knowledge triples and expunges bias information from LLMs using model editing techniques. Through extensive experimentation, we have confirmed the presence of the implicit bias problem in LLMs and demonstrated the effectiveness of our BTBR approach.
- [190] arXiv:2408.10609 [pdf, html, other]
-
Title: PerturBench: Benchmarking Machine Learning Models for Cellular Perturbation AnalysisYan Wu, Esther Wershof, Sebastian M Schmon, Marcel Nassar, Błażej Osiński, Ridvan Eksi, Kun Zhang, Thore GraepelComments: 9 pages plus 19 pages supplementary material. Code is available at this https URLSubjects: Machine Learning (cs.LG); Genomics (q-bio.GN); Machine Learning (stat.ML)
We present a comprehensive framework for predicting the effects of perturbations in single cells, designed to standardize benchmarking in this rapidly evolving field. Our framework, PerturBench, includes a user-friendly platform, diverse datasets, metrics for fair model comparison, and detailed performance analysis. Extensive evaluations of published and baseline models reveal limitations like mode or posterior collapse, and underscore the importance of rank metrics that assess the ordering of perturbations alongside traditional measures like RMSE. Our findings show that simple models can outperform more complex approaches. This benchmarking exercise sets new standards for model evaluation, supports robust model development, and advances the potential of these models to use high-throughput and high-content genetic and chemical screens for disease target discovery.
- [191] arXiv:2408.10610 [pdf, html, other]
-
Title: On the Approximability of Stationary Processes using the ARMA ModelComments: 10 pages, 3 figuresSubjects: Machine Learning (cs.LG); Probability (math.PR); Methodology (stat.ME)
We identify certain gaps in the literature on the approximability of stationary random variables using the Autoregressive Moving Average (ARMA) model. To quantify approximability, we propose that an ARMA model be viewed as an approximation of a stationary random variable. We map these stationary random variables to Hardy space functions, and formulate a new function approximation problem that corresponds to random variable approximation, and thus to ARMA. Based on this Hardy space formulation we identify a class of stationary processes where approximation guarantees are feasible. We also identify an idealized stationary random process for which we conjecture that a good ARMA approximation is not possible. Next, we provide a constructive proof that Padé approximations do not always correspond to the best ARMA approximation. Finally, we note that the spectral methods adopted in this paper can be seen as a generalization of unit root methods for stationary processes even when an ARMA model is not defined.
- [192] arXiv:2408.10613 [pdf, html, other]
-
Title: Task-level Distributionally Robust Optimization for Large Language Model-based Dense RetrievalSubjects: Information Retrieval (cs.IR)
Large Language Model-based Dense Retrieval (LLM-DR) optimizes over numerous heterogeneous fine-tuning collections from different domains. However, the discussion about its training data distribution is still minimal. Previous studies rely on empirically assigned dataset choices or sampling ratios, which inevitably leads to sub-optimal retrieval performances. In this paper, we propose a new task-level Distributionally Robust Optimization (tDRO) algorithm for LLM-DR fine-tuning, targeted at improving the universal domain generalization ability by end-to-end reweighting the data distribution of each task. The tDRO parameterizes the domain weights and updates them with scaled domain gradients. The optimized weights are then transferred to the LLM-DR fine-tuning to train more robust retrievers. Experiments show optimal improvements in large-scale retrieval benchmarks and reduce up to 30% dataset usage after applying our optimization algorithm with a series of different-sized LLM-DR models.
- [193] arXiv:2408.10614 [pdf, html, other]
-
Title: Generalizable Facial Expression RecognitionComments: Accepted by ECCV2024Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
SOTA facial expression recognition (FER) methods fail on test sets that have domain gaps with the train set. Recent domain adaptation FER methods need to acquire labeled or unlabeled samples of target domains to fine-tune the FER model, which might be infeasible in real-world deployment. In this paper, we aim to improve the zero-shot generalization ability of FER methods on different unseen test sets using only one train set. Inspired by how humans first detect faces and then select expression features, we propose a novel FER pipeline to extract expression-related features from any given face images. Our method is based on the generalizable face features extracted by large models like CLIP. However, it is non-trivial to adapt the general features of CLIP for specific tasks like FER. To preserve the generalization ability of CLIP and the high precision of the FER model, we design a novel approach that learns sigmoid masks based on the fixed CLIP face features to extract expression features. To further improve the generalization ability on unseen test sets, we separate the channels of the learned masked features according to the expression classes to directly generate logits and avoid using the FC layer to reduce overfitting. We also introduce a channel-diverse loss to make the learned masks separated. Extensive experiments on five different FER datasets verify that our method outperforms SOTA FER methods by large margins. Code is available in this https URL.
- [194] arXiv:2408.10615 [pdf, html, other]
-
Title: Enhancing Robustness in Large Language Models: Prompting for Mitigating the Impact of Irrelevant InformationSubjects: Computation and Language (cs.CL)
In recent years, Large language models (LLMs) have garnered significant attention due to their superior performance in complex reasoning tasks. However, recent studies may diminish their reasoning capabilities markedly when problem descriptions contain irrelevant information, even with the use of advanced prompting techniques. To further investigate this issue, a dataset of primary school mathematics problems containing irrelevant information, named GSMIR, was constructed. Testing prominent LLMs and prompting techniques on this dataset revealed that while LLMs can identify irrelevant information, they do not effectively mitigate the interference it causes once identified. A novel automatic construction method, ATF, which enhances the ability of LLMs to identify and self-mitigate the influence of irrelevant information, is proposed to address this shortcoming. This method operates in two steps: first, analysis of irrelevant information, followed by its filtering. The ATF method, as demonstrated by experimental results, significantly improves the reasoning performance of LLMs and prompting techniques, even in the presence of irrelevant information on the GSMIR dataset.
- [195] arXiv:2408.10616 [pdf, other]
-
Title: A toolbox for calculating objective image properties in aesthetics researchChristoph Redies, Ralf Bartho, Lisa Koßmann, Branka Spehar, Ronald Hübner, Johan Wagemans, Gregor U. Hayn-LeichsenringComments: 41 pages, 6 figureSubjects: Computer Vision and Pattern Recognition (cs.CV); Applications (stat.AP)
Over the past two decades, researchers in the field of visual aesthetics have studied numerous quantitative (objective) image properties and how they relate to visual aesthetic appreciation. However, results are difficult to compare between research groups. One reason is that researchers use different sets of image properties in their studies. But even if the same properties are used, the image pre-processing techniques may differ and often researchers use their own customized scripts to calculate the image properties. To provide greater accessibility and comparability of research results in visual experimental aesthetics, we developed an open-access and easy-to-use toolbox (called the 'Aesthetics Toolbox'). The Toolbox allows users to calculate a well-defined set of quantitative image properties popular in contemporary research. The properties include lightness and color statistics, Fourier spectral properties, fractality, self-similarity, symmetry, as well as different entropy measures and CNN-based variances. Compatible with most devices, the Toolbox provides an intuitive click-and-drop web interface. In the Toolbox, we integrated the original scripts of four different research groups and translated them into Python 3. To ensure that results were consistent across analyses, we took care that results from the Python versions of the scripts were the same as those from the original scripts. The toolbox, detailed documentation, and a link to the cloud version are available via Github: this https URL. In summary, we developed a toolbox that helps to standardize and simplify the calculation of quantitative image properties for visual aesthetics research.
- [196] arXiv:2408.10618 [pdf, html, other]
-
Title: OMEGA: Efficient Occlusion-Aware Navigation for Air-Ground Robot in Dynamic Environments via State Space ModelComments: OccMamba is Coming!Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
Air-ground robots (AGRs) are widely used in surveillance and disaster response due to their exceptional mobility and versatility (i.e., flying and driving). Current AGR navigation systems perform well in static occlusion-prone environments (e.g., indoors) by using 3D semantic occupancy networks to predict occlusions for complete local mapping and then computing Euclidean Signed Distance Field (ESDF) for path planning. However, these systems face challenges in dynamic, severe occlusion scenes (e.g., crowds) due to limitations in perception networks' low prediction accuracy and path planners' high computation overhead. In this paper, we propose OMEGA, which contains OccMamba with an Efficient AGR-Planner to address the above-mentioned problems. OccMamba adopts a novel architecture that separates semantic and occupancy prediction into independent branches, incorporating two mamba blocks within these branches. These blocks efficiently extract semantic and geometric features in 3D environments with linear complexity, ensuring that the network can learn long-distance dependencies to improve prediction accuracy. Semantic and geometric features are combined within the Bird's Eye View (BEV) space to minimise computational overhead during feature fusion. The resulting semantic occupancy map is then seamlessly integrated into the local map, providing occlusion awareness of the dynamic environment. Our AGR-Planner utilizes this local map and employs kinodynamic A* search and gradient-based trajectory optimization to guarantee planning is ESDF-free and energy-efficient. Extensive experiments demonstrate that OccMamba outperforms the state-of-the-art 3D semantic occupancy network with 25.0% mIoU. End-to-end navigation experiments in dynamic scenes verify OMEGA's efficiency, achieving a 96% average planning success rate. Code and video are available at this https URL.
- [197] arXiv:2408.10619 [pdf, other]
-
Title: Novel Change Detection Framework in Remote Sensing Imagery Using Diffusion Models and Structural Similarity Index (SSIM)Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Image and Video Processing (eess.IV)
Change detection is a crucial task in remote sensing, enabling the monitoring of environmental changes, urban growth, and disaster impact. Conventional change detection techniques, such as image differencing and ratioing, often struggle with noise and fail to capture complex variations in imagery. Recent advancements in machine learning, particularly generative models like diffusion models, offer new opportunities for enhancing change detection accuracy. In this paper, we propose a novel change detection framework that combines the strengths of Stable Diffusion models with the Structural Similarity Index (SSIM) to create robust and interpretable change maps. Our approach, named Diffusion Based Change Detector, is evaluated on both synthetic and real-world remote sensing datasets and compared with state-of-the-art methods. The results demonstrate that our method significantly outperforms traditional differencing techniques and recent deep learning-based methods, particularly in scenarios with complex changes and noise.
- [198] arXiv:2408.10620 [pdf, html, other]
-
Title: Fast Grid Emissions Sensitivities using Parallel Decentralized Implicit DifferentiationSubjects: Systems and Control (eess.SY)
Marginal emissions rates -- the sensitivity of carbon emissions to electricity demand -- are important for evaluating the impact of emissions mitigation measures. Like locational marginal prices, locational marginal emissions rates (LMEs) can vary geographically, even between nearby locations, and may be coupled across time periods because of, for example, storage and ramping constraints. This temporal coupling makes computing LMEs computationally expensive for large electricity networks with high storage and renewable penetrations. Recent work demonstrates that decentralized algorithms can mitigate this problem by decoupling timesteps during differentiation. Unfortunately, we show these potential speedups are negated by the sparse structure inherent in power systems problems. We address these limitations by introducing a parallel, reverse-mode decentralized differentiation scheme that never explicitly instantiates the solution map Jacobian. We show both theoretically and empirically that parallelization is necessary to achieve non-trivial speedups when computing grid emissions sensitivities. Numerical results on a 500 node system indicate that our method can achieve greater than 10x speedups over centralized and serial decentralized approaches.
- [199] arXiv:2408.10622 [pdf, html, other]
-
Title: Safety Metric Aware Trajectory Repairing for Automated DrivingSubjects: Robotics (cs.RO)
Recent analyses highlight challenges in autonomous vehicle technologies, particularly failures in decision-making under dynamic or emergency conditions. Traditional automated driving systems recalculate the entire trajectory in a changing environment. Instead, a novel approach retains valid trajectory segments, minimizing the need for complete replanning and reducing changes to the original plan. This work introduces a trajectory repairing framework that calculates a feasible evasive trajectory while computing the Feasible Time-to-React (F-TTR), balancing the maintenance of the original plan with safety assurance. The framework employs a binary search algorithm to iteratively create repaired trajectories, guaranteeing both the safety and feasibility of the trajectory repairing result. In contrast to earlier approaches that separated the calculation of safety metrics from trajectory repairing, which resulted in unsuccessful plans for evasive maneuvers, our work has the anytime capability to provide both a Feasible Time-to-React and an evasive trajectory for further execution.
- [200] arXiv:2408.10623 [pdf, html, other]
-
Title: TextMastero: Mastering High-Quality Scene Text Editing in Diverse Languages and StylesSubjects: Computer Vision and Pattern Recognition (cs.CV)
Scene text editing aims to modify texts on images while maintaining the style of newly generated text similar to the original. Given an image, a target area, and target text, the task produces an output image with the target text in the selected area, replacing the original. This task has been studied extensively, with initial success using Generative Adversarial Networks (GANs) to balance text fidelity and style similarity. However, GAN-based methods struggled with complex backgrounds or text styles. Recent works leverage diffusion models, showing improved results, yet still face challenges, especially with non-Latin languages like CJK characters (Chinese, Japanese, Korean) that have complex glyphs, often producing inaccurate or unrecognizable characters. To address these issues, we present \emph{TextMastero} - a carefully designed multilingual scene text editing architecture based on latent diffusion models (LDMs). TextMastero introduces two key modules: a glyph conditioning module for fine-grained content control in generating accurate texts, and a latent guidance module for providing comprehensive style information to ensure similarity before and after editing. Both qualitative and quantitative experiments demonstrate that our method surpasses all known existing works in text fidelity and style similarity.
- [201] arXiv:2408.10624 [pdf, html, other]
-
Title: WRIM-Net: Wide-Ranging Information Mining Network for Visible-Infrared Person Re-IdentificationComments: 18 pages, 5 figuresSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
For the visible-infrared person re-identification (VI-ReID) task, one of the primary challenges lies in significant cross-modality discrepancy. Existing methods struggle to conduct modality-invariant information mining. They often focus solely on mining singular dimensions like spatial or channel, and overlook the extraction of specific-modality multi-dimension information. To fully mine modality-invariant information across a wide range, we introduce the Wide-Ranging Information Mining Network (WRIM-Net), which mainly comprises a Multi-dimension Interactive Information Mining (MIIM) module and an Auxiliary-Information-based Contrastive Learning (AICL) approach. Empowered by the proposed Global Region Interaction (GRI), MIIM comprehensively mines non-local spatial and channel information through intra-dimension interaction. Moreover, Thanks to the low computational complexity design, separate MIIM can be positioned in shallow layers, enabling the network to better mine specific-modality multi-dimension information. AICL, by introducing the novel Cross-Modality Key-Instance Contrastive (CMKIC) loss, effectively guides the network in extracting modality-invariant information. We conduct extensive experiments not only on the well-known SYSU-MM01 and RegDB datasets but also on the latest large-scale cross-modality LLCM dataset. The results demonstrate WRIM-Net's superiority over state-of-the-art methods.
- [202] arXiv:2408.10627 [pdf, html, other]
-
Title: Rethinking Video Segmentation with Masked Video Consistency: Did the Model Learn as Intended?Subjects: Computer Vision and Pattern Recognition (cs.CV)
Video segmentation aims at partitioning video sequences into meaningful segments based on objects or regions of interest within frames. Current video segmentation models are often derived from image segmentation techniques, which struggle to cope with small-scale or class-imbalanced video datasets. This leads to inconsistent segmentation results across frames. To address these issues, we propose a training strategy Masked Video Consistency, which enhances spatial and temporal feature aggregation. MVC introduces a training strategy that randomly masks image patches, compelling the network to predict the entire semantic segmentation, thus improving contextual information integration. Additionally, we introduce Object Masked Attention (OMA) to optimize the cross-attention mechanism by reducing the impact of irrelevant queries, thereby enhancing temporal modeling capabilities. Our approach, integrated into the latest decoupled universal video segmentation framework, achieves state-of-the-art performance across five datasets for three video segmentation tasks, demonstrating significant improvements over previous methods without increasing model parameters.
- [203] arXiv:2408.10628 [pdf, html, other]
-
Title: Finding the DeepDream for Time Series: Activation Maximization for Univariate Time SeriesComments: 16 pages, 4 figures, accepted at TempXAI @ ECML-PKDDSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Understanding how models process and interpret time series data remains a significant challenge in deep learning to enable applicability in safety-critical areas such as healthcare. In this paper, we introduce Sequence Dreaming, a technique that adapts Activation Maximization to analyze sequential information, aiming to enhance the interpretability of neural networks operating on univariate time series. By leveraging this method, we visualize the temporal dynamics and patterns most influential in model decision-making processes. To counteract the generation of unrealistic or excessively noisy sequences, we enhance Sequence Dreaming with a range of regularization techniques, including exponential smoothing. This approach ensures the production of sequences that more accurately reflect the critical features identified by the neural network. Our approach is tested on a time series classification dataset encompassing applications in predictive maintenance. The results show that our proposed Sequence Dreaming approach demonstrates targeted activation maximization for different use cases so that either centered class or border activation maximization can be generated. The results underscore the versatility of Sequence Dreaming in uncovering salient temporal features learned by neural networks, thereby advancing model transparency and trustworthiness in decision-critical domains.
- [204] arXiv:2408.10631 [pdf, html, other]
-
Title: LLM-Barber: Block-Aware Rebuilder for Sparsity Mask in One-Shot for Large Language ModelsSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Large language models (LLMs) have grown significantly in scale, leading to a critical need for efficient model pruning techniques. Existing post-training pruning techniques primarily focus on measuring weight importance on converged dense models to determine salient weights to retain. However, they often overlook the changes in weight importance during the pruning process, which can lead to performance degradation in the pruned models. To address this issue, we present LLM-Barber (Block-Aware Rebuilder for Sparsity Mask in One-Shot), a novel one-shot pruning framework that rebuilds the sparsity mask of pruned models without any retraining or weight reconstruction. LLM-Barber incorporates block-aware error optimization across Self-Attention and MLP blocks, ensuring global performance optimization. Inspired by the recent discovery of prominent outliers in LLMs, LLM-Barber introduces an innovative pruning metric that identifies weight importance using weights multiplied by gradients. Our experiments show that LLM-Barber can efficiently prune models like LLaMA and OPT families with 7B to 13B parameters on a single A100 GPU in just 30 minutes, achieving state-of-the-art results in both perplexity and zero-shot performance across various language benchmarks. Code is available at this https URL.
- [205] arXiv:2408.10633 [pdf, html, other]
-
Title: Interactive Counterfactual Generation for Univariate Time SeriesComments: 14 pages, 4 figures, accepted at XKDD @ ECML-PKDDSubjects: Machine Learning (cs.LG); Human-Computer Interaction (cs.HC)
We propose an interactive methodology for generating counterfactual explanations for univariate time series data in classification tasks by leveraging 2D projections and decision boundary maps to tackle interpretability challenges. Our approach aims to enhance the transparency and understanding of deep learning models' decision processes. The application simplifies the time series data analysis by enabling users to interactively manipulate projected data points, providing intuitive insights through inverse projection techniques. By abstracting user interactions with the projected data points rather than the raw time series data, our method facilitates an intuitive generation of counterfactual explanations. This approach allows for a more straightforward exploration of univariate time series data, enabling users to manipulate data points to comprehend potential outcomes of hypothetical scenarios. We validate this method using the ECG5000 benchmark dataset, demonstrating significant improvements in interpretability and user understanding of time series classification. The results indicate a promising direction for enhancing explainable AI, with potential applications in various domains requiring transparent and interpretable deep learning models. Future work will explore the scalability of this method to multivariate time series data and its integration with other interpretability techniques.
- [206] arXiv:2408.10634 [pdf, other]
-
Title: Industry Perception of Security Challenges with Identity Access Management SolutionsComments: Will be published in 2024 IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom), Tbilisi, Georgia, 24_27 June 2024Subjects: Cryptography and Security (cs.CR)
Identity Access Management (IAM) is an area posing significant challenges, particularly in the context of remote connectivity and distributed or cloud-based systems. A wide range of technical solutions have been proposed by prior research, but the integration of these solutions in the commercial sector represent steps that significantly hamper their acceptance. The study aims to outline the current perception and security issues associated with IAMs solutions from the perspective of the beneficiaries. The analysis relies on a series of interviews with 45 cyber security professionals from different organisations all over the world. As results showed, cloud IAM solutions and on premises IAM solutions are affected by different issues. The main challenges for cloud based IAM solutions were Default configurations, Poor management of Non-Human Identities such as Service accounts, Poor certificate management, Poor API configuration and limited Log analysis. In contrast, the challenges for on premise solutions were Multi Factor Authentication, insecure Default configurations, Lack of skillsets required to manage IAM solution securely, Poor password policies, Unpatched vulnerabilities, and compromise of Single-Sign on leading to compromise of multiple entities. The study also determined that, regardless the evolving functionality of cloud based IAM solutions, 41% of respondents believe that the on premise solutions more secure than the cloud-based ones. As pointed out by the respondents, cloud IAM may potentially expose organisations to a wider range of vulnerabilities due to the complexity of the underlying solutions, challenges with managing permissions, and compliance to dynamic IAM policies.
- [207] arXiv:2408.10635 [pdf, html, other]
-
Title: Strategist: Learning Strategic Skills by LLMs via Bi-Level Tree SearchComments: website: this https URLSubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
In this paper, we propose a new method Strategist that utilizes LLMs to acquire new skills for playing multi-agent games through a self-improvement process. Our method gathers quality feedback through self-play simulations with Monte Carlo tree search and LLM-based reflection, which can then be used to learn high-level strategic skills such as how to evaluate states that guide the low-level execution.We showcase how our method can be used in both action planning and dialogue generation in the context of games, achieving good performance on both tasks. Specifically, we demonstrate that our method can help train agents with better performance than both traditional reinforcement learning-based approaches and other LLM-based skill learning approaches in games including the Game of Pure Strategy (GOPS) and The Resistance: Avalon.
- [208] arXiv:2408.10637 [pdf, html, other]
-
Title: Variations on distributed beliefSubjects: Logic in Computer Science (cs.LO)
Motivated by the search for forms of distributed belief that do not collapse in the face of conflicting information, this paper introduces the notions of cautious and bold distributed belief. Both notions rely on maximally consistent subgroups of agents, with cautious quantifying universally and bold quantifying existentially. As a result, while the cautious distributed belief of a group is inconsistent only when all group members are individually inconsistent, the bold distributed belief of a group is never inconsistent. The paper discusses these two notions, presenting their respective modalities and semantic interpretations, discussing some of their basic properties, studying whether they preserve doxastic properties from the members of the group, and comparing them not only with standard distributed belief but also with one another, both at the level of modalities and at the level of language expressivity.
- [209] arXiv:2408.10641 [pdf, html, other]
-
Title: A Review of Human-Object Interaction DetectionSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Human-object interaction (HOI) detection plays a key role in high-level visual understanding, facilitating a deep comprehension of human activities. Specifically, HOI detection aims to locate the humans and objects involved in interactions within images or videos and classify the specific interactions between them. The success of this task is influenced by several key factors, including the accurate localization of human and object instances, as well as the correct classification of object categories and interaction relationships. This paper systematically summarizes and discusses the recent work in image-based HOI detection. First, the mainstream datasets involved in HOI relationship detection are introduced. Furthermore, starting with two-stage methods and end-to-end one-stage detection approaches, this paper comprehensively discusses the current developments in image-based HOI detection, analyzing the strengths and weaknesses of these two methods. Additionally, the advancements of zero-shot learning, weakly supervised learning, and the application of large-scale language models in HOI detection are discussed. Finally, the current challenges in HOI detection are outlined, and potential research directions and future trends are explored.
- [210] arXiv:2408.10642 [pdf, html, other]
-
Title: Minor SFT loss for LLM fine-tune to increase performance and reduce model deviationComments: 8 pages, 5 figuresSubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Instruct LLM provide a paradigm used in large scale language model to align LLM to human preference. The paradigm contains supervised fine tuning and reinforce learning from human feedback. This paradigm is also used in downstream scenarios to adapt LLM to specific corpora and applications. Comparing to SFT, there are many efforts focused on RLHF and several algorithms being proposed, such as PPO, DPO, IPO, KTO, MinorDPO and etc. Meanwhile most efforts for SFT are focused on how to collect, filter and mix high quality data. In this article with insight from DPO and MinorDPO, we propose a training metric for SFT to measure the discrepancy between the optimized model and the original model, and a loss function MinorSFT that can increase the training effectiveness, and reduce the discrepancy between the optimized LLM and original LLM.
- [211] arXiv:2408.10645 [pdf, html, other]
-
Title: CoRA: Collaborative Information Perception by Large Language Model's Weights for RecommendationYuting Liu, Jinghao Zhang, Yizhou Dang, Yuliang Liang, Qiang Liu, Guibing Guo, Jianzhe Zhao, Xingwei WangSubjects: Information Retrieval (cs.IR); Machine Learning (cs.LG)
Involving collaborative information in Large Language Models (LLMs) is a promising technique for adapting LLMs for recommendation. Existing methods achieve this by concatenating collaborative features with text tokens into a unified sequence input and then fine-tuning to align these features with LLM's input space. Although effective, in this work, we identify two limitations when adapting LLMs to recommendation tasks, which hinder the integration of general knowledge and collaborative information, resulting in sub-optimal recommendation performance. (1) Fine-tuning LLM with recommendation data can undermine its inherent world knowledge and fundamental competencies, which are crucial for interpreting and inferring recommendation text. (2) Incorporating collaborative features into textual prompts disrupts the semantics of the original prompts, preventing LLM from generating appropriate outputs. In this paper, we propose a new paradigm, CoRA (an acronym for Collaborative LoRA), with a collaborative weights generator. Rather than input space alignment, this method aligns collaborative information with LLM's parameter space, representing them as incremental weights to update LLM's output. This way, LLM perceives collaborative information without altering its general knowledge and text inference capabilities. Specifically, we employ a collaborative filtering model to extract user and item embeddings, converting them into collaborative weights with low-rank properties through the collaborative weights generator. We then merge the collaborative weights into LLM's weights, enabling LLM to perceive the collaborative signals and generate personalized recommendations without fine-tuning or extra collaborative tokens in prompts. Extensive experiments confirm that CoRA effectively integrates collaborative information into LLM, enhancing recommendation performance.
- [212] arXiv:2408.10646 [pdf, html, other]
-
Title: Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMsSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
The veracity of a factoid is largely independent of the language it is written in. However, language models are inconsistent in their ability to answer the same factual question across languages. This raises questions about how LLMs represent a given fact across languages. We explore multilingual factual knowledge through two aspects: the model's ability to answer a query consistently across languages, and the ability to ''store'' answers in a shared representation for several languages. We propose a methodology to measure the extent of representation sharing across languages by repurposing knowledge editing methods. We examine LLMs with various multilingual configurations using a new multilingual dataset. We reveal that high consistency does not necessarily imply shared representation, particularly for languages with different scripts. Moreover, we find that script similarity is a dominant factor in representation sharing. Finally, we observe that if LLMs could fully share knowledge across languages, their accuracy in their best-performing language could benefit an increase of up to 150\% on average. These findings highlight the need for improved multilingual knowledge representation in LLMs and suggest a path for the development of more robust and consistent multilingual LLMs.
- [213] arXiv:2408.10647 [pdf, html, other]
-
Title: Privacy-preserving Universal Adversarial Defense for Black-box ModelsQiao Li, Cong Wu, Jing Chen, Zijun Zhang, Kun He, Ruiying Du, Xinxin Wang, Qingchuang Zhao, Yang LiuComments: 12 pages, 9 figuresSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR)
Deep neural networks (DNNs) are increasingly used in critical applications such as identity authentication and autonomous driving, where robustness against adversarial attacks is crucial. These attacks can exploit minor perturbations to cause significant prediction errors, making it essential to enhance the resilience of DNNs. Traditional defense methods often rely on access to detailed model information, which raises privacy concerns, as model owners may be reluctant to share such data. In contrast, existing black-box defense methods fail to offer a universal defense against various types of adversarial attacks. To address these challenges, we introduce DUCD, a universal black-box defense method that does not require access to the target model's parameters or architecture. Our approach involves distilling the target model by querying it with data, creating a white-box surrogate while preserving data privacy. We further enhance this surrogate model using a certified defense based on randomized smoothing and optimized noise selection, enabling robust defense against a broad range of adversarial attacks. Comparative evaluations between the certified defenses of the surrogate and target models demonstrate the effectiveness of our approach. Experiments on multiple image classification datasets show that DUCD not only outperforms existing black-box defenses but also matches the accuracy of white-box defenses, all while enhancing data privacy and reducing the success rate of membership inference attacks.
- [214] arXiv:2408.10648 [pdf, html, other]
-
Title: Smart Contract Coordinated Privacy Preserving Crowd-Sensing CampaignsSubjects: Cryptography and Security (cs.CR); Networking and Internet Architecture (cs.NI)
Crowd-sensing has emerged as a powerful data retrieval model, enabling diverse applications by leveraging active user participation. However, data availability and privacy concerns pose significant challenges. Traditional methods like data encryption and anonymization, while essential, may not fully address these issues. For instance, in sparsely populated areas, anonymized data can still be traced back to individual users. Additionally, the volume of data generated by users can reveal their identities. To develop credible crowd-sensing systems, data must be anonymized, aggregated and separated into uniformly sized chunks. Furthermore, decentralizing the data management process, rather than relying on a single server, can enhance security and trust. This paper proposes a system utilizing smart contracts and blockchain technologies to manage crowd-sensing campaigns. The smart contract handles user subscriptions, data encryption, and decentralized storage, creating a secure data marketplace. Incentive policies within the smart contract encourage user participation and data diversity. Simulation results confirm the system's viability, highlighting the importance of user participation for data credibility and the impact of geographical data scarcity on rewards. This approach aims to balance data origin and reduce cheating risks.
- [215] arXiv:2408.10649 [pdf, html, other]
-
Title: Inferring Underwater Topography with FINNCoşku Can Horuz, Matthias Karlbauer, Timothy Praditia, Sergey Oladyshkin, Wolfgang Nowak, Sebastian OtteSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Atmospheric and Oceanic Physics (physics.ao-ph); Computational Physics (physics.comp-ph); Fluid Dynamics (physics.flu-dyn)
Spatiotemporal partial differential equations (PDEs) find extensive application across various scientific and engineering fields. While numerous models have emerged from both physics and machine learning (ML) communities, there is a growing trend towards integrating these approaches to develop hybrid architectures known as physics-aware machine learning models. Among these, the finite volume neural network (FINN) has emerged as a recent addition. FINN has proven to be particularly efficient in uncovering latent structures in data. In this study, we explore the capabilities of FINN in tackling the shallow-water equations, which simulates wave dynamics in coastal regions. Specifically, we investigate FINN's efficacy to reconstruct underwater topography based on these particular wave equations. Our findings reveal that FINN exhibits a remarkable capacity to infer topography solely from wave dynamics, distinguishing itself from both conventional ML and physics-aware ML models. Our results underscore the potential of FINN in advancing our understanding of spatiotemporal phenomena and enhancing parametrization capabilities in related domains.
- [216] arXiv:2408.10652 [pdf, html, other]
-
Title: Vocabulary-Free 3D Instance Segmentation with Vision and Language AssistantSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Most recent 3D instance segmentation methods are open vocabulary, offering a greater flexibility than closed-vocabulary methods. Yet, they are limited to reasoning within a specific set of concepts, \ie the vocabulary, prompted by the user at test time. In essence, these models cannot reason in an open-ended fashion, i.e., answering ``List the objects in the scene.''. We introduce the first method to address 3D instance segmentation in a setting that is void of any vocabulary prior, namely a vocabulary-free setting. We leverage a large vision-language assistant and an open-vocabulary 2D instance segmenter to discover and ground semantic categories on the posed images. To form 3D instance mask, we first partition the input point cloud into dense superpoints, which are then merged into 3D instance masks. We propose a novel superpoint merging strategy via spectral clustering, accounting for both mask coherence and semantic coherence that are estimated from the 2D object instance masks. We evaluate our method using ScanNet200 and Replica, outperforming existing methods in both vocabulary-free and open-vocabulary settings. Code will be made available.
- [217] arXiv:2408.10653 [pdf, html, other]
-
Title: UIE-UnFold: Deep Unfolding Network with Color Priors and Vision Transformer for Underwater Image EnhancementComments: Accepted by DSAA CIVIL 2024Subjects: Computer Vision and Pattern Recognition (cs.CV)
Underwater image enhancement (UIE) plays a crucial role in various marine applications, but it remains challenging due to the complex underwater environment. Current learning-based approaches frequently lack explicit incorporation of prior knowledge about the physical processes involved in underwater image formation, resulting in limited optimization despite their impressive enhancement results. This paper proposes a novel deep unfolding network (DUN) for UIE that integrates color priors and inter-stage feature transformation to improve enhancement performance. The proposed DUN model combines the iterative optimization and reliability of model-based methods with the flexibility and representational power of deep learning, offering a more explainable and stable solution compared to existing learning-based UIE approaches. The proposed model consists of three key components: a Color Prior Guidance Block (CPGB) that establishes a mapping between color channels of degraded and original images, a Nonlinear Activation Gradient Descent Module (NAGDM) that simulates the underwater image degradation process, and an Inter Stage Feature Transformer (ISF-Former) that facilitates feature exchange between different network stages. By explicitly incorporating color priors and modeling the physical characteristics of underwater image formation, the proposed DUN model achieves more accurate and reliable enhancement results. Extensive experiments on multiple underwater image datasets demonstrate the superiority of the proposed model over state-of-the-art methods in both quantitative and qualitative evaluations. The proposed DUN-based approach offers a promising solution for UIE, enabling more accurate and reliable scientific analysis in marine research. The code is available at this https URL.
- [218] arXiv:2408.10654 [pdf, other]
-
Title: Incorporating a 'ladder of trust' into dynamic Allocation of Function in Human-Autonomous Agent CollectivesJournal-ref: 14th Organizational Design and Management Conference (ODAM), Bordeaux, France, Jul. 11-13, 2023Subjects: Human-Computer Interaction (cs.HC)
A major, ongoing social transition is the inclusion of autonomous agents into human organizations. For example, in defence and security applications, robots may be used alongside human operatives to reduce risk or add capability. But a key barrier to the transition to successful human-autonomous agent collectives is the need for sufficient trust between team members. A critical enabling factor for this trust will be a suitably designed dynamic allocation of function (AoF). We consider AoF in terms of a 'ladder of trust' (from low to high) with individual team members adjusting trust in their teammates based on variation in 'score' over time. The score is derived by the ability of team member to perceive and understand its situation based on the gathered information and act to acheive team or self goals. Combining these trust scores gives a system-level perspective on how AoF might be adjusted during a mission. That is, the most suitable teammate for a function might have a low trust rating from its fellow teammates, so it might be preferable to choose the next most suitable teammate for the function at that point in time. Of course, this is only in the situation where the next most suitable teammate is also likely to perform within the set framework of moral, ethical, and legal constraints. The trade-offs between trust in the individual agent's capability and predictability need to be considered within the broader context of the agent's integrity and accountability. From this perspective, the Allocation Space is defined by more than ability of each agent to perform a function.
- [219] arXiv:2408.10657 [pdf, html, other]
-
Title: ETGuard: Malicious Encrypted Traffic Detection in Blockchain-based Power Grid SystemsSubjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
The escalating prevalence of encryption protocols has led to a concomitant surge in the number of malicious attacks that hide in encrypted traffic. Power grid systems, as fundamental infrastructure, are becoming prime targets for such attacks. Conventional methods for detecting malicious encrypted packets typically use a static pre-trained model. We observe that these methods are not well-suited for blockchain-based power grid systems. More critically, they fall short in dynamic environments where new types of encrypted attacks continuously emerge. Motivated by this, in this paper we try to tackle these challenges from two aspects: (1) We present a novel framework that is able to automatically detect malicious encrypted traffic in blockchain-based power grid systems and incrementally learn from new malicious traffic. (2) We mathematically derive incremental learning losses to resist the forgetting of old attack patterns while ensuring the model is capable of handling new encrypted attack patterns. Empirically, our method achieves state-of-the-art performance on three different benchmark datasets. We also constructed the first malicious encrypted traffic dataset for blockchain-based power grid scenario. Our code and dataset are available at this https URL, hoping to inspire future research.
- [220] arXiv:2408.10658 [pdf, html, other]
-
Title: Learning Instruction-Guided Manipulation Affordance via Large Models for Embodied Robotic TasksComments: Accepted to ICARM 2024Subjects: Robotics (cs.RO)
We study the task of language instruction-guided robotic manipulation, in which an embodied robot is supposed to manipulate the target objects based on the language instructions. In previous studies, the predicted manipulation regions of the target object typically do not change with specification from the language instructions, which means that the language perception and manipulation prediction are separate. However, in human behavioral patterns, the manipulation regions of the same object will change for different language instructions. In this paper, we propose Instruction-Guided Affordance Net (IGANet) for predicting affordance maps of instruction-guided robotic manipulation tasks by utilizing powerful priors from vision and language encoders pre-trained on large-scale datasets. We develop a Vison-Language-Models(VLMs)-based data augmentation pipeline, which can generate a large amount of data automatically for model training. Besides, with the help of Large-Language-Models(LLMs), actions can be effectively executed to finish the tasks defined by instructions. A series of real-world experiments revealed that our method can achieve better performance with generated data. Moreover, our model can generalize better to scenarios with unseen objects and language instructions.
- [221] arXiv:2408.10663 [pdf, html, other]
-
Title: REInstruct: Building Instruction Data from Unlabeled CorpusComments: Accepted by ACL2024 FindingsSubjects: Computation and Language (cs.CL)
Manually annotating instruction data for large language models is difficult, costly, and hard to scale. Meanwhile, current automatic annotation methods typically rely on distilling synthetic data from proprietary LLMs, which not only limits the upper bound of the quality of the instruction data but also raises potential copyright issues. In this paper, we propose REInstruct, a simple and scalable method to automatically build instruction data from an unlabeled corpus without heavy reliance on proprietary LLMs and human annotation. Specifically, REInstruct first selects a subset of unlabeled texts that potentially contain well-structured helpful and insightful content and then generates instructions for these texts. To generate accurate and relevant responses for effective and robust training, REInstruct further proposes a rewriting-based approach to improve the quality of the generated instruction data. By training Llama-7b on a combination of 3k seed data and 32k synthetic data from REInstruct, fine-tuned model achieves a 65.41\% win rate on AlpacaEval leaderboard against text-davinci-003, outperforming other open-source, non-distilled instruction data construction methods. The code is publicly available at \url{this https URL}.
- [222] arXiv:2408.10664 [pdf, html, other]
-
Title: Federated Clustering: An Unsupervised Cluster-Wise Training for Decentralized Data DistributionsSubjects: Machine Learning (cs.LG)
Federated Learning (FL) is a pivotal approach in decentralized machine learning, especially when data privacy is crucial and direct data sharing is impractical. While FL is typically associated with supervised learning, its potential in unsupervised scenarios is underexplored. This paper introduces a novel unsupervised federated learning methodology designed to identify the complete set of categories (global K) across multiple clients within label-free, non-uniform data distributions, a process known as Federated Clustering. Our approach, Federated Cluster-Wise Refinement (FedCRef), involves clients that collaboratively train models on clusters with similar data distributions. Initially, clients with diverse local data distributions (local K) train models on their clusters to generate compressed data representations. These local models are then shared across the network, enabling clients to compare them through reconstruction error analysis, leading to the formation of federated this http URL these groups, clients collaboratively train a shared model representing each data distribution, while continuously refining their local clusters to enhance data association accuracy. This iterative process allows our system to identify all potential data distributions across the network and develop robust representation models for each. To validate our approach, we compare it with traditional centralized methods, establishing a performance baseline and showcasing the advantages of our distributed solution. We also conduct experiments on the EMNIST and KMNIST datasets, demonstrating FedCRef's ability to refine and align cluster models with actual data distributions, significantly improving data representation precision in unsupervised federated settings.
- [223] arXiv:2408.10666 [pdf, html, other]
-
Title: Accelerating the Surrogate Retraining for Poisoning Attacks against Recommender SystemsComments: Accepted by RecSys 2024Subjects: Information Retrieval (cs.IR)
Recent studies have demonstrated the vulnerability of recommender systems to data poisoning attacks, where adversaries inject carefully crafted fake user interactions into the training data of recommenders to promote target items. Current attack methods involve iteratively retraining a surrogate recommender on the poisoned data with the latest fake users to optimize the attack. However, this repetitive retraining is highly time-consuming, hindering the efficient assessment and optimization of fake users. To mitigate this computational bottleneck and develop a more effective attack in an affordable time, we analyze the retraining process and find that a change in the representation of one user/item will cause a cascading effect through the user-item interaction graph. Under theoretical guidance, we introduce \emph{Gradient Passing} (GP), a novel technique that explicitly passes gradients between interacted user-item pairs during backpropagation, thereby approximating the cascading effect and accelerating retraining. With just a single update, GP can achieve effects comparable to multiple original training iterations. Under the same number of retraining epochs, GP enables a closer approximation of the surrogate recommender to the victim. This more accurate approximation provides better guidance for optimizing fake users, ultimately leading to enhanced data poisoning attacks. Extensive experiments on real-world datasets demonstrate the efficiency and effectiveness of our proposed GP.
- [224] arXiv:2408.10668 [pdf, html, other]
-
Title: Probing the Safety Response Boundary of Large Language Models via Unsafe Decoding Path GenerationSubjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
Large Language Models (LLMs) are implicit troublemakers. While they provide valuable insights and assist in problem-solving, they can also potentially serve as a resource for malicious activities. Implementing safety alignment could mitigate the risk of LLMs generating harmful responses. We argue that: even when an LLM appears to successfully block harmful queries, there may still be hidden vulnerabilities that could act as ticking time bombs. To identify these underlying weaknesses, we propose to use a cost value model as both a detector and an attacker. Trained on external or self-generated harmful datasets, the cost value model could successfully influence the original safe LLM to output toxic content in decoding process. For instance, LLaMA-2-chat 7B outputs 39.18% concrete toxic content, along with only 22.16% refusals without any harmful suffixes. These potential weaknesses can then be exploited via prompt optimization such as soft prompts on images. We name this decoding strategy: Jailbreak Value Decoding (JVD), emphasizing that seemingly secure LLMs may not be as safe as we initially believe. They could be used to gather harmful data or launch covert attacks.
- [225] arXiv:2408.10669 [pdf, html, other]
-
Title: Tensor tree learns hidden relational structures in data to construct generative modelsComments: 9 pages, 3 figuresSubjects: Machine Learning (cs.LG); Statistical Mechanics (cond-mat.stat-mech); Artificial Intelligence (cs.AI); Quantum Physics (quant-ph)
Based on the tensor tree network with the Born machine framework, we propose a general method for constructing a generative model by expressing the target distribution function as the quantum wave function amplitude represented by a tensor tree. The key idea is dynamically optimizing the tree structure that minimizes the bond mutual information. The proposed method offers enhanced performance and uncovers hidden relational structures in the target data. We illustrate potential practical applications with four examples: (i) random patterns, (ii) QMNIST hand-written digits, (iii) Bayesian networks, and (iv) the stock price fluctuation pattern in S&P500. In (i) and (ii), strongly correlated variables were concentrated near the center of the network; in (iii), the causality pattern was identified; and, in (iv), a structure corresponding to the eleven sectors emerged.
- [226] arXiv:2408.10670 [pdf, other]
-
Title: A Noncontact Technique for Wave Measurement Based on Thermal Stereography and Deep LearningSubjects: Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
The accurate measurement of the wave field and its spatiotemporal evolution is essential in many hydrodynamic experiments and engineering applications. The binocular stereo imaging technique has been widely used to measure waves. However, the optical properties of indoor water surfaces, including transparency, specular reflection, and texture absence, pose challenges for image processing and stereo reconstruction. This study proposed a novel technique that combined thermal stereography and deep learning to achieve fully noncontact wave measurements. The optical imaging properties of water in the long-wave infrared spectrum were found to be suitable for stereo matching, effectively avoiding the issues in the visible-light spectrum. After capturing wave images using thermal stereo cameras, a reconstruction strategy involving deep learning techniques was proposed to improve stereo matching performance. A generative approach was employed to synthesize a dataset with ground-truth disparity from unannotated infrared images. This dataset was then fed to a pretrained stereo neural network for fine-tuning to achieve domain adaptation. Wave flume experiments were conducted to validate the feasibility and accuracy of the proposed technique. The final reconstruction results indicated great agreement and high accuracy with a mean bias of less than 2.1% compared with the measurements obtained using wave probes, suggesting that the novel technique effectively measures the spatiotemporal distribution of wave surface in hydrodynamic experiments.
- [227] arXiv:2408.10672 [pdf, html, other]
-
Title: Neural Exploratory Landscape AnalysisSubjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
Recent research in Meta-Black-Box Optimization (MetaBBO) have shown that meta-trained neural networks can effectively guide the design of black-box optimizers, significantly reducing the need for expert tuning and delivering robust performance across complex problem distributions. Despite their success, a paradox remains: MetaBBO still rely on human-crafted Exploratory Landscape Analysis features to inform the meta-level agent about the low-level optimization progress. To address the gap, this paper proposes Neural Exploratory Landscape Analysis (NeurELA), a novel framework that dynamically profiles landscape features through a two-stage, attention-based neural network, executed in an entirely end-to-end fashion. NeurELA is pre-trained over a variety of MetaBBO algorithms using a multi-task neuroevolution strategy. Extensive experiments show that NeurELA achieves consistently superior performance when integrated into different and even unseen MetaBBO tasks and can be efficiently fine-tuned for further performance boost. This advancement marks a pivotal step in making MetaBBO algorithms more autonomous and broadly applicable.
- [228] arXiv:2408.10673 [pdf, html, other]
-
Title: Iterative Window Mean Filter: Thwarting Diffusion-based Adversarial PurificationComments: Under reviewSubjects: Cryptography and Security (cs.CR)
Face authentication systems have brought significant convenience and advanced developments, yet they have become unreliable due to their sensitivity to inconspicuous perturbations, such as adversarial attacks. Existing defenses often exhibit weaknesses when facing various attack algorithms and adaptive attacks or compromise accuracy for enhanced security. To address these challenges, we have developed a novel and highly efficient non-deep-learning-based image filter called the Iterative Window Mean Filter (IWMF) and proposed a new framework for adversarial purification, named IWMF-Diff, which integrates IWMF and denoising diffusion models. These methods can function as pre-processing modules to eliminate adversarial perturbations without necessitating further modifications or retraining of the target system. We demonstrate that our proposed methodologies fulfill four critical requirements: preserved accuracy, improved security, generalizability to various threats in different settings, and better resistance to adaptive attacks. This performance surpasses that of the state-of-the-art adversarial purification method, DiffPure.
- [229] arXiv:2408.10676 [pdf, html, other]
-
Title: Representation Norm Amplification for Out-of-Distribution Detection in Long-Tail LearningComments: 30 pages, 8 figures, 17 tablesSubjects: Machine Learning (cs.LG)
Detecting out-of-distribution (OOD) samples is a critical task for reliable machine learning. However, it becomes particularly challenging when the models are trained on long-tailed datasets, as the models often struggle to distinguish tail-class in-distribution samples from OOD samples. We examine the main challenges in this problem by identifying the trade-offs between OOD detection and in-distribution (ID) classification, faced by existing methods. We then introduce our method, called \textit{Representation Norm Amplification} (RNA), which solves this challenge by decoupling the two problems. The main idea is to use the norm of the representation as a new dimension for OOD detection, and to develop a training method that generates a noticeable discrepancy in the representation norm between ID and OOD data, while not perturbing the feature learning for ID classification. Our experiments show that RNA achieves superior performance in both OOD detection and classification compared to the state-of-the-art methods, by 1.70\% and 9.46\% in FPR95 and 2.43\% and 6.87\% in classification accuracy on CIFAR10-LT and ImageNet-LT, respectively. The code for this work is available at this https URL.
- [230] arXiv:2408.10679 [pdf, html, other]
-
Title: DemMamba: Alignment-free Raw Video Demoireing with Frequency-assisted Spatio-Temporal MambaSubjects: Computer Vision and Pattern Recognition (cs.CV)
Moire patterns arise when two similar repetitive patterns interfere, a phenomenon frequently observed during the capture of images or videos on screens. The color, shape, and location of moire patterns may differ across video frames, posing a challenge in learning information from adjacent frames and preserving temporal consistency. Previous video demoireing methods heavily rely on well-designed alignment modules, resulting in substantial computational burdens. Recently, Mamba, an improved version of the State Space Model (SSM), has demonstrated significant potential for modeling long-range dependencies with linear complexity, enabling efficient temporal modeling in video demoireing without requiring a specific alignment module. In this paper, we propose a novel alignment-free Raw video demoireing network with frequency-assisted spatio-temporal Mamba (DemMamba). The Spatial Mamba Block (SMB) and Temporal Mamba Block (TMB) are sequentially arranged to facilitate effective intra- and inter-relationship modeling in Raw videos with moire patterns. Within SMB, an Adaptive Frequency Block (AFB) is introduced to aid demoireing in the frequency domain. For TMB, a Channel Attention Block (CAB) is embedded to further enhance temporal information interactions by exploiting the inter-channel relationships among features. Extensive experiments demonstrate that our proposed DemMamba surpasses state-of-the-art approaches by 1.3 dB and delivers a superior visual experience.
- [231] arXiv:2408.10680 [pdf, html, other]
-
Title: Towards Rehearsal-Free Multilingual ASR: A LoRA-based Case Study on WhisperSubjects: Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Pre-trained multilingual speech foundation models, like Whisper, have shown impressive performance across different languages. However, adapting these models to new or specific languages is computationally extensive and faces catastrophic forgetting problems. Addressing these issues, our study investigates strategies to enhance the model on new languages in the absence of original training data, while also preserving the established performance on the original languages. Specifically, we first compare various LoRA-based methods to find out their vulnerability to forgetting. To mitigate this issue, we propose to leverage the LoRA parameters from the original model for approximate orthogonal gradient descent on the new samples. Additionally, we also introduce a learnable rank coefficient to allocate trainable parameters for more efficient training. Our experiments with a Chinese Whisper model (for Uyghur and Tibetan) yield better results with a more compact parameter set.
- [232] arXiv:2408.10681 [pdf, html, other]
-
Title: HMoE: Heterogeneous Mixture of Experts for Language ModelingAn Wang, Xingwu Sun, Ruobing Xie, Shuaipeng Li, Jiaqi Zhu, Zhen Yang, Pinxue Zhao, J.N.Han, Zhanhui Kang, Di Wang, Naoaki Okazaki, Cheng-zhong XuSubjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Mixture of Experts (MoE) offers remarkable performance and computational efficiency by selectively activating subsets of model parameters. Traditionally, MoE models use homogeneous experts, each with identical capacity. However, varying complexity in input data necessitates experts with diverse capabilities, while homogeneous MoE hinders effective expert specialization and efficient parameter utilization. In this study, we propose a novel Heterogeneous Mixture of Experts (HMoE), where experts differ in size and thus possess diverse capacities. This heterogeneity allows for more specialized experts to handle varying token complexities more effectively. To address the imbalance in expert activation, we propose a novel training objective that encourages the frequent activation of smaller experts, enhancing computational efficiency and parameter utilization. Extensive experiments demonstrate that HMoE achieves lower loss with fewer activated parameters and outperforms conventional homogeneous MoE models on various pre-training evaluation benchmarks. Codes will be released upon acceptance.
- [233] arXiv:2408.10682 [pdf, html, other]
-
Title: Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language ModelsComments: 13 pagesSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
LLM have achieved success in many fields but still troubled by problematic content in the training corpora. LLM unlearning aims at reducing their influence and avoid undesirable behaviours. However, existing unlearning methods remain vulnerable to adversarial queries and the unlearned knowledge resurfaces after the manually designed attack queries. As part of a red-team effort to proactively assess the vulnerabilities of unlearned models, we design Dynamic Unlearning Attack (DUA), a dynamic and automated framework to attack these models and evaluate their robustness. It optimizes adversarial suffixes to reintroduce the unlearned knowledge in various scenarios. We find that unlearned knowledge can be recovered in $55.2\%$ of the questions, even without revealing the unlearned model's parameters. In response to this vulnerability, we propose Latent Adversarial Unlearning (LAU), a universal framework that effectively enhances the robustness of the unlearned process. It formulates the unlearning process as a min-max optimization problem and resolves it through two stages: an attack stage, where perturbation vectors are trained and added to the latent space of LLMs to recover the unlearned knowledge, and a defense stage, where previously trained perturbation vectors are used to enhance unlearned model's robustness. With our LAU framework, we obtain two robust unlearning methods, AdvGA and AdvNPO. We conduct extensive experiments across multiple unlearning benchmarks and various models, and demonstrate that they improve the unlearning effectiveness by over $53.5\%$, cause only less than a $11.6\%$ reduction in neighboring knowledge, and have almost no impact on the model's general capabilities.
- [234] arXiv:2408.10683 [pdf, html, other]
-
Title: Rejection in Abstract Argumentation: Harder Than Acceptance?Comments: accepted version as ECAI24Subjects: Artificial Intelligence (cs.AI); Computational Complexity (cs.CC); Logic in Computer Science (cs.LO)
Abstract argumentation is a popular toolkit for modeling, evaluating, and comparing arguments. Relationships between arguments are specified in argumentation frameworks (AFs), and conditions are placed on sets (extensions) of arguments that allow AFs to be evaluated. For more expressiveness, AFs are augmented with \emph{acceptance conditions} on directly interacting arguments or a constraint on the admissible sets of arguments, resulting in dialectic frameworks or constrained argumentation frameworks. In this paper, we consider flexible conditions for \emph{rejecting} an argument from an extension, which we call rejection conditions (RCs). On the technical level, we associate each argument with a specific logic program. We analyze the resulting complexity, including the structural parameter treewidth. Rejection AFs are highly expressive, giving rise to natural problems on higher levels of the polynomial hierarchy.
- [235] arXiv:2408.10685 [pdf, html, other]
-
Title: Proving Cutoff Bounds for Safety Properties in First-Order LogicComments: 28 pages, 6 figures, 1 tableSubjects: Logic in Computer Science (cs.LO); Programming Languages (cs.PL)
First-order logic has been established as an important tool for modeling and verifying intricate systems such as distributed protocols and concurrent systems. These systems are parametric in the number of nodes in the network or the number of threads, which is finite in any system instance, but unbounded. One disadvantage of first-order logic is that it cannot distinguish between finite and infinite structures, leading to spurious counterexamples. To mitigate this, we offer a verification approach that captures only finite system instances. Our approach is an adaptation of the cutoff method to systems modeled in first-order logic. The idea is to show that any safety violation in a system instance of size larger than some bound can be simulated by a safety violation in a system of a smaller size. The simulation provides an inductive argument for correctness in finite instances, reducing the problem to showing safety of instances with bounded size. To this end, we develop a framework to (i) encode such simulation relations in first-order logic and to (ii) validate the simulation relation by a set of verification conditions given to an SMT solver. We apply our approach to verify safety of a set of examples, some of which cannot be proven by a first-order inductive invariant.
- [236] arXiv:2408.10688 [pdf, html, other]
-
Title: TDS-CLIP: Temporal Difference Side Network for Image-to-Video Transfer LearningSubjects: Computer Vision and Pattern Recognition (cs.CV)
Recently, large-scale pre-trained vision-language models (e.g., CLIP), have garnered significant attention thanks to their powerful representative capabilities. This inspires researchers in transferring the knowledge from these large pre-trained models to other task-specific models, e.g., Video Action Recognition (VAR) models, via particularly leveraging side networks to enhance the efficiency of parameter-efficient fine-tuning (PEFT). However, current transferring approaches in VAR tend to directly transfer the frozen knowledge from large pre-trained models to action recognition networks with minimal cost, instead of exploiting the temporal modeling capabilities of the action recognition models themselves. Therefore, in this paper, we propose a memory-efficient Temporal Difference Side Network (TDS-CLIP) to balance knowledge transferring and temporal modeling, avoiding backpropagation in frozen parameter models. Specifically, we introduce a Temporal Difference Adapter (TD-Adapter), which can effectively capture local temporal differences in motion features to strengthen the model's global temporal modeling capabilities. Furthermore, we designed a Side Motion Enhancement Adapter (SME-Adapter) to guide the proposed side network in efficiently learning the rich motion information in videos, thereby improving the side network's ability to capture and learn motion information. Extensive experiments are conducted on three benchmark datasets, including Something-Something V1\&V2, and Kinetics-400. Experimental results demonstrate that our approach achieves competitive performance.
- [237] arXiv:2408.10689 [pdf, html, other]
-
Title: Genesis: Towards the Automation of Systems Biology ResearchIevgeniia A. Tiukova, Daniel Brunnsåker, Erik Y. Bjurström, Alexander H. Gower, Filip Kronström, Gabriel K. Reder, Ronald S. Reiserer, Konstantin Korovin, Larisa B. Soldatova, John P. Wikswo, Ross D. KingSubjects: Artificial Intelligence (cs.AI)
The cutting edge of applying AI to science is the closed-loop automation of scientific research: robot scientists. We have previously developed two robot scientists: `Adam' (for yeast functional biology), and `Eve' (for early-stage drug design)). We are now developing a next generation robot scientist Genesis. With Genesis we aim to demonstrate that an area of science can be investigated using robot scientists unambiguously faster, and at lower cost, than with human scientists. Here we report progress on the Genesis project. Genesis is designed to automatically improve system biology models with thousands of interacting causal components. When complete Genesis will be able to initiate and execute in parallel one thousand hypothesis-led closed-loop cycles of experiment per-day. Here we describe the core Genesis hardware: the one thousand computer-controlled $\mu$-bioreactors. For the integrated Mass Spectrometry platform we have developed AutonoMS, a system to automatically run, process, and analyse high-throughput experiments. We have also developed Genesis-DB, a database system designed to enable software agents access to large quantities of structured domain information. We have developed RIMBO (Revisions for Improvements of Models in Biology Ontology) to describe the planned hundreds of thousands of changes to the models. We have demonstrated the utility of this infrastructure by developed two relational learning bioinformatic projects. Finally, we describe LGEM+ a relational learning system for the automated abductive improvement of genome-scale metabolic models.
- [238] arXiv:2408.10690 [pdf, html, other]
-
Title: Spectral Function Space Learning and Numerical Linear Algebra Networks for Solving Linear Inverse ProblemsSubjects: Numerical Analysis (math.NA); Functional Analysis (math.FA)
We consider solving a probably ill-conditioned linear operator equation, where the operator is not modeled by physical laws but is specified via training pairs (consisting of images and data) of the input-output relation of the operator. We derive a stable method for computing the operator, which consists of first a Gram-Schmidt orthonormalization of images and a principal component analysis of the data. This two-step algorithm provides a spectral decomposition of the linear operator. Moreover, we show that both Gram-Schmidt and principal component analysis can be written as a deep neural network, which relates this procedure to de-and encoder networks. Therefore, we call the two-step algorithm a linear algebra network. Finally, we provide numerical simulations showing the strategy is feasible for reconstructing spectral functions and for solving operator equations without explicitly exploiting the physical model.
- [239] arXiv:2408.10691 [pdf, html, other]
-
Title: Fine-Tuning and Deploying Large Language Models Over Edges: Issues and ApproachesSubjects: Artificial Intelligence (cs.AI)
Since the invention of GPT2--1.5B in 2019, large language models (LLMs) have transitioned from specialized models to versatile foundation models. The LLMs exhibit impressive zero-shot ability, however, require fine-tuning on local datasets and significant resources for deployment. Traditional fine-tuning techniques with the first-order optimizers require substantial GPU memory that exceeds mainstream hardware capability. Therefore, memory-efficient methods are motivated to be investigated. Model compression techniques can reduce energy consumption, operational costs, and environmental impact so that to support sustainable artificial intelligence advancements. Additionally, large-scale foundation models have expanded to create images, audio, videos, and multi-modal contents, further emphasizing the need for efficient deployment. Therefore, we are motivated to present a comprehensive overview of the prevalent memory-efficient fine-tuning methods over the network edge. We also review the state-of-the-art literatures on model compression to provide a vision on deploying LLMs over the network edge.
- [240] arXiv:2408.10692 [pdf, html, other]
-
Title: Unconditional Truthfulness: Learning Conditional Dependency for Uncertainty Quantification of Large Language ModelsArtem Vazhentsev, Ekaterina Fadeeva, Rui Xing, Alexander Panchenko, Preslav Nakov, Timothy Baldwin, Maxim Panov, Artem ShelmanovSubjects: Computation and Language (cs.CL)
Uncertainty quantification (UQ) is a perspective approach to detecting Large Language Model (LLM) hallucinations and low quality output. In this work, we address one of the challenges of UQ in generation tasks that arises from the conditional dependency between the generation steps of an LLM. We propose to learn this dependency from data. We train a regression model, which target variable is the gap between the conditional and the unconditional generation confidence. During LLM inference, we use this learned conditional dependency model to modulate the uncertainty of the current generation step based on the uncertainty of the previous step. Our experimental evaluation on nine datasets and three LLMs shows that the proposed method is highly effective for uncertainty quantification, achieving substantial improvements over rivaling approaches.
- [241] arXiv:2408.10693 [pdf, html, other]
-
Title: Improved Differential Evolution based Feature Selection through Quantum, Chaos, and LassoComments: 20 pages, 12 tables, 3 figuresSubjects: Neural and Evolutionary Computing (cs.NE)
Modern deep learning continues to achieve outstanding performance on an astounding variety of high-dimensional tasks. In practice, this is obtained by fitting deep neural models to all the input data with minimal feature engineering, thus sacrificing interpretability in many cases. However, in applications such as medicine, where interpretability is crucial, feature subset selection becomes an important problem. Metaheuristics such as Binary Differential Evolution are a popular approach to feature selection, and the research literature continues to introduce novel ideas, drawn from quantum computing and chaos theory, for instance, to improve them. In this paper, we demonstrate that introducing chaos-generated variables, generated from considerations of the Lyapunov time, in place of random variables in quantum-inspired metaheuristics significantly improves their performance on high-dimensional medical classification tasks and outperforms other approaches. We show that this chaos-induced improvement is a general phenomenon by demonstrating it for multiple varieties of underlying quantum-inspired metaheuristics. Performance is further enhanced through Lasso-assisted feature pruning. At the implementation level, we vastly speed up our algorithms through a scalable island-based computing cluster parallelization technique.
- [242] arXiv:2408.10694 [pdf, html, other]
-
Title: MsMemoryGAN: A Multi-scale Memory GAN for Palm-vein Adversarial PurificationSubjects: Computer Vision and Pattern Recognition (cs.CV)
Deep neural networks have recently achieved promising performance in the vein recognition task and have shown an increasing application trend, however, they are prone to adversarial perturbation attacks by adding imperceptible perturbations to the input, resulting in making incorrect recognition. To address this issue, we propose a novel defense model named MsMemoryGAN, which aims to filter the perturbations from adversarial samples before recognition. First, we design a multi-scale autoencoder to achieve high-quality reconstruction and two memory modules to learn the detailed patterns of normal samples at different scales. Second, we investigate a learnable metric in the memory module to retrieve the most relevant memory items to reconstruct the input image. Finally, the perceptional loss is combined with the pixel loss to further enhance the quality of the reconstructed image. During the training phase, the MsMemoryGAN learns to reconstruct the input by merely using fewer prototypical elements of the normal patterns recorded in the memory. At the testing stage, given an adversarial sample, the MsMemoryGAN retrieves its most relevant normal patterns in memory for the reconstruction. Perturbations in the adversarial sample are usually not reconstructed well, resulting in purifying the input from adversarial perturbations. We have conducted extensive experiments on two public vein datasets under different adversarial attack methods to evaluate the performance of the proposed approach. The experimental results show that our approach removes a wide variety of adversarial perturbations, allowing vein classifiers to achieve the highest recognition accuracy.
- [243] arXiv:2408.10695 [pdf, html, other]
-
Title: On NVD Users' Attitudes, Experiences, Hopes and HurdlesComments: To appear in ACM DTRAP Special Issue on IMF 2024Subjects: Cryptography and Security (cs.CR)
The National Vulnerability Database (NVD) is a major vulnerability database that is free to use for everyone. It provides information about vulnerabilities and further useful resources such as linked advisories and patches. The NVD is often considered as the central source for vulnerability information and as a help to improve the resource-intensive process of vulnerability management. Although the NVD receives much public attention, little is known about its usage in vulnerability management, users' attitudes towards it and whether they encounter any problems during usage. We explored these questions using a preliminary interview study with seven people, and a follow-up survey with 71 participants. The results show that the NVD is consulted regularly and often aids decision making. Generally, users are positive about the NVD and perceive it as a helpful, clearly structured tool. But users also faced issues: missing or incorrect entries, incomplete descriptions or incomprehensible CVSS ratings. In order to identify the problems origins, we discussed the results with two senior NVD members. Many of the problems can be attributed to higher-level problems such as the CVE List or limited resources. Nevertheless, the NVD is working on improving existing problems.
- [244] arXiv:2408.10700 [pdf, html, other]
-
Title: AnyGraph: Graph Foundation Model in the WildSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
The growing ubiquity of relational data structured as graphs has underscored the need for graph learning models with exceptional generalization capabilities. However, current approaches often struggle to effectively extract generalizable insights, frequently requiring extensive fine-tuning and limiting their versatility. Graph foundation models offer a transformative solution, with the potential to learn robust, generalizable representations from graph data. This enables more effective and adaptable applications across a wide spectrum of tasks and domains. In this work, we investigate a unified graph model, AnyGraph, designed to handle key challenges: i) Structure Heterogenity. Addressing distribution shift in graph structural information; ii) Feature Heterogenity. Handling diverse feature representation spaces across graph datasets; iii) Fast Adaptation. Efficiently adapting the model to new graph domains; iv) Scaling Law Emergence. Enabling the model to exhibit scaling law behavior, where its performance scales favorably with the amount of data and parameter sizes. To tackle these critical challenges, we build the AnyGraph upon a Graph Mixture-of-Experts (MoE) architecture. This approach empowers the model to effectively manage both the in-domain and cross-domain distribution shift concerning structure-level and feature-level heterogeneity. Furthermore, a lightweight graph expert routing mechanism is proposed to facilitate AnyGraph's fast adaptability to new data and domains. Our extensive experiments on diverse 38 graph datasets have demonstrated the strong zero-shot learning performance of AnyGraph across diverse graph domains with significant distribution shift. Furthermore, we have validated the model's fast adaptation ability and scaling law emergence, showcasing its versatility.
- [245] arXiv:2408.10701 [pdf, other]
-
Title: Ferret: Faster and Effective Automated Red Teaming with Reward-Based Scoring TechniqueSubjects: Computation and Language (cs.CL)
In today's era, where large language models (LLMs) are integrated into numerous real-world applications, ensuring their safety and robustness is crucial for responsible AI usage. Automated red-teaming methods play a key role in this process by generating adversarial attacks to identify and mitigate potential vulnerabilities in these models. However, existing methods often struggle with slow performance, limited categorical diversity, and high resource demands. While Rainbow Teaming, a recent approach, addresses the diversity challenge by framing adversarial prompt generation as a quality-diversity search, it remains slow and requires a large fine-tuned mutator for optimal performance. To overcome these limitations, we propose Ferret, a novel approach that builds upon Rainbow Teaming by generating multiple adversarial prompt mutations per iteration and using a scoring function to rank and select the most effective adversarial prompt. We explore various scoring functions, including reward models, Llama Guard, and LLM-as-a-judge, to rank adversarial mutations based on their potential harm to improve the efficiency of the search for harmful mutations. Our results demonstrate that Ferret, utilizing a reward model as a scoring function, improves the overall attack success rate (ASR) to 95%, which is 46% higher than Rainbow Teaming. Additionally, Ferret reduces the time needed to achieve a 90% ASR by 15.2% compared to the baseline and generates adversarial prompts that are transferable i.e. effective on other LLMs of larger size. Our codes are available at this https URL.
- [246] arXiv:2408.10703 [pdf, html, other]
-
Title: Large Language Models for Multimodal Deformable Image RegistrationSubjects: Computer Vision and Pattern Recognition (cs.CV)
The challenge of Multimodal Deformable Image Registration (MDIR) lies in the conversion and alignment of features between images of different modalities. Generative models (GMs) cannot retain the necessary information enough from the source modality to the target one, while non-GMs struggle to align features across these two modalities. In this paper, we propose a novel coarse-to-fine MDIR framework,LLM-Morph, which is applicable to various pre-trained Large Language Models (LLMs) to solve these concerns by aligning the deep features from different modal medical images. Specifically, we first utilize a CNN encoder to extract deep visual features from cross-modal image pairs, then we use the first adapter to adjust these tokens, and use LoRA in pre-trained LLMs to fine-tune their weights, both aimed at eliminating the domain gap between the pre-trained LLMs and the MDIR task. Third, for the alignment of tokens, we utilize other four adapters to transform the LLM-encoded tokens into multi-scale visual features, generating multi-scale deformation fields and facilitating the coarse-to-fine MDIR task. Extensive experiments in MR-CT Abdomen and SR-Reg Brain datasets demonstrate the effectiveness of our framework and the potential of pre-trained LLMs for MDIR task. Our code is availabel at: this https URL.
- [247] arXiv:2408.10706 [pdf, html, other]
-
Title: Performance Analysis of Physical Layer Security: From Far-Field to Near-FieldSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
The secrecy performance in both near-field and far-field communications is analyzed using two fundamental metrics: the secrecy capacity under a power constraint and the minimum power requirement to achieve a specified secrecy rate target. 1) For the secrecy capacity, a closed-form expression is derived under a discrete-time memoryless setup. This expression is further analyzed under several far-field and near-field channel models, and the capacity scaling law is revealed by assuming an infinitely large transmit array and an infinitely high power. A novel concept of "depth of insecurity" is proposed to evaluate the secrecy performance achieved by near-field beamfocusing. It is demonstrated that increasing the number of transmit antennas reduces this depth and thus improves the secrecy performance. 2) Regarding the minimum required power, a closed-form expression is derived and analyzed within far-field and near-field scenarios. Asymptotic analyses are performed by setting the number of transmit antennas to infinity to unveil the power scaling law. Numerical results are provided to demonstrate that: i) compared to far-field communications, near-field communications expand the areas where secure transmission is feasible, specifically when the eavesdropper is located in the same direction as the intended receiver; ii) as the number of transmit antennas increases, neither the secrecy capacity nor the minimum required power scales or vanishes unboundedly, adhering to the principle of energy conservation.
- [248] arXiv:2408.10708 [pdf, html, other]
-
Title: Distribution of Reconfiguration Languages maintaining Tree-like Communication TopologySubjects: Formal Languages and Automata Theory (cs.FL)
We study how to distribute trace languages in a setting where processes communicate via reconfigurable communication channels. That is, the different processes can connect and disconnect from channels at run time. We restrict attention to communication via tree-like communication architectures. These allow channels to connect more than two processes in a way that maintains an underlying spanning tree and keeps communication continuous on the tree. We make the reconfiguration explicit in the language allowing both a centralized automaton as well as the distributed processes to share relevant information about the current communication configuration. We show that Zielonka's seminal result regarding distribution of regular languages for asynchronous automata can be generalized in this setting, incorporating both reconfiguration and more than binary tree architectures.
- [249] arXiv:2408.10709 [pdf, html, other]
-
Title: Variable Assignment Invariant Neural Networks for Learning Logic ProgramsSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Learning from interpretation transition (LFIT) is a framework for learning rules from observed state transitions. LFIT has been implemented in purely symbolic algorithms, but they are unable to deal with noise or generalize to unobserved transitions. Rule extraction based neural network methods suffer from overfitting, while more general implementation that categorize rules suffer from combinatorial explosion. In this paper, we introduce a technique to leverage variable permutation invariance inherent in symbolic domains. Our technique ensures that the permutation and the naming of the variables would not affect the results. We demonstrate the effectiveness and the scalability of this method with various experiments. Our code is publicly available at this https URL
- [250] arXiv:2408.10710 [pdf, html, other]
-
Title: Coarse-to-Fine Detection of Multiple Seams for Robotic WeldingSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Efficiently detecting target weld seams while ensuring sub-millimeter accuracy has always been an important challenge in autonomous welding, which has significant application in industrial practice. Previous works mostly focused on recognizing and localizing welding seams one by one, leading to inferior efficiency in modeling the workpiece. This paper proposes a novel framework capable of multiple weld seams extraction using both RGB images and 3D point clouds. The RGB image is used to obtain the region of interest by approximately localizing the weld seams, and the point cloud is used to achieve the fine-edge extraction of the weld seams within the region of interest using region growth. Our method is further accelerated by using a pre-trained deep learning model to ensure both efficiency and generalization ability. The performance of the proposed method has been comprehensively tested on various workpieces featuring both linear and curved weld seams and in physical experiment systems. The results showcase considerable potential for real-world industrial applications, emphasizing the method's efficiency and effectiveness. Videos of the real-world experiments can be found at this https URL.
- [251] arXiv:2408.10711 [pdf, html, other]
-
Title: Investigating Context Effects in Similarity Judgements in Large Language ModelsComments: Accepted at The First Workshop on AI Behavioral Science (AIBS 2024), held in conjunction with KDD 2024Subjects: Artificial Intelligence (cs.AI)
Large Language Models (LLMs) have revolutionised the capability of AI models in comprehending and generating natural language text. They are increasingly being used to empower and deploy agents in real-world scenarios, which make decisions and take actions based on their understanding of the context. Therefore researchers, policy makers and enterprises alike are working towards ensuring that the decisions made by these agents align with human values and user expectations. That being said, human values and decisions are not always straightforward to measure and are subject to different cognitive biases. There is a vast section of literature in Behavioural Science which studies biases in human judgements. In this work we report an ongoing investigation on alignment of LLMs with human judgements affected by order bias. Specifically, we focus on a famous human study which showed evidence of order effects in similarity judgements, and replicate it with various popular LLMs. We report the different settings where LLMs exhibit human-like order effect bias and discuss the implications of these findings to inform the design and development of LLM based applications.
- [252] arXiv:2408.10713 [pdf, html, other]
-
Title: Offline Model-Based Reinforcement Learning with Anti-ExplorationSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Model-based reinforcement learning (MBRL) algorithms learn a dynamics model from collected data and apply it to generate synthetic trajectories to enable faster learning. This is an especially promising paradigm in offline reinforcement learning (RL) where data may be limited in quantity, in addition to being deficient in coverage and quality. Practical approaches to offline MBRL usually rely on ensembles of dynamics models to prevent exploitation of any individual model and to extract uncertainty estimates that penalize values in states far from the dataset support. Uncertainty estimates from ensembles can vary greatly in scale, making it challenging to generalize hyperparameters well across even similar tasks. In this paper, we present Morse Model-based offline RL (MoMo), which extends the anti-exploration paradigm found in offline model-free RL to the model-based space. We develop model-free and model-based variants of MoMo and show how the model-free version can be extended to detect and deal with out-of-distribution (OOD) states using explicit uncertainty estimation without the need for large ensembles. MoMo performs offline MBRL using an anti-exploration bonus to counteract value overestimation in combination with a policy constraint, as well as a truncation function to terminate synthetic rollouts that are excessively OOD. Experimentally, we find that both model-free and model-based MoMo perform well, and the latter outperforms prior model-based and model-free baselines on the majority of D4RL datasets tested.
- [253] arXiv:2408.10714 [pdf, html, other]
-
Title: Physics-Driven AI Correction in Laser Absorption Sensing QuantificationComments: 13 pagesSubjects: Neural and Evolutionary Computing (cs.NE)
Laser absorption spectroscopy (LAS) quantification is a popular tool used in measuring temperature and concentration of gases. It has low error tolerance, whereas current ML-based solutions cannot guarantee their measure reliability. In this work, we propose a new framework, SPEC, to address this issue. In addition to the conventional ML estimator-based estimation mode, SPEC also includes a Physics-driven Anomaly Detection module (PAD) to assess the error of the estimation. And a Correction mode is designed to correct the unreliable estimation. The correction mode is a network-based optimization algorithm, which uses the guidance of error to iteratively correct the estimation. A hybrid surrogate error model is proposed to estimate the error distribution, which contains an ensemble of networks to simulate reconstruction error, and true feasible error computation. A greedy ensemble search is proposed to find the optimal correction robustly and efficiently from the gradient guidance of surrogate model. The proposed SPEC is validated on the test scenarios which are outside the training distribution. The results show that SPEC can significantly improve the estimation quality, and the correction mode outperforms current network-based optimization algorithms. In addition, SPEC has the reconfigurability, which can be easily adapted to different quantification tasks via changing PAD without retraining the ML estimator.
- [254] arXiv:2408.10715 [pdf, html, other]
-
Title: Fine-Tuning a Local LLaMA-3 Large Language Model for Automated Privacy-Preserving Physician Letter Generation in Radiation OncologyYihao Hou, Christoph Bert, Ahmed Gomaa, Godehard Lahmer, Daniel Hoefler, Thomas Weissmann, Raphaela Voigt, Philipp Schubert, Charlotte Schmitter, Alina Depardon, Sabine Semrau, Andreas Maier, Rainer Fietkau, Yixing Huang, Florian PutzSubjects: Artificial Intelligence (cs.AI)
Generating physician letters is a time-consuming task in daily clinical practice. This study investigates local fine-tuning of large language models (LLMs), specifically LLaMA models, for physician letter generation in a privacy-preserving manner within the field of radiation oncology. Our findings demonstrate that base LLaMA models, without fine-tuning, are inadequate for effectively generating physician letters. The QLoRA algorithm provides an efficient method for local intra-institutional fine-tuning of LLMs with limited computational resources (i.e., a single 48 GB GPU workstation within the hospital). The fine-tuned LLM successfully learns radiation oncology-specific information and generates physician letters in an institution-specific style. ROUGE scores of the generated summary reports highlight the superiority of the 8B LLaMA-3 model over the 13B LLaMA-2 model. Further multidimensional physician evaluations of 10 cases reveal that, although the fine-tuned LLaMA-3 model has limited capacity to generate content beyond the provided input data, it successfully generates salutations, diagnoses and treatment histories, recommendations for further treatment, and planned schedules. Overall, clinical benefit was rated highly by the clinical experts (average score of 3.44 on a 4-point scale). With careful physician review and correction, automated LLM-based physician letter generation has significant practical value.
- [255] arXiv:2408.10717 [pdf, html, other]
-
Title: Accelerated training of deep learning surrogate models for surface displacement and flow, with application to MCMC-based history matching of CO2 storage operationsSubjects: Machine Learning (cs.LG)
Deep learning surrogate modeling shows great promise for subsurface flow applications, but the training demands can be substantial. Here we introduce a new surrogate modeling framework to predict CO2 saturation, pressure and surface displacement for use in the history matching of carbon storage operations. Rather than train using a large number of expensive coupled flow-geomechanics simulation runs, training here involves a large number of inexpensive flow-only simulations combined with a much smaller number of coupled runs. The flow-only runs use an effective rock compressibility, which is shown to provide accurate predictions for saturation and pressure for our system. A recurrent residual U-Net architecture is applied for the saturation and pressure surrogate models, while a new residual U-Net model is introduced to predict surface displacement. The surface displacement surrogate accepts, as inputs, geomodel quantities along with saturation and pressure surrogate predictions. Median relative error for a diverse test set is less than 4% for all variables. The surrogate models are incorporated into a hierarchical Markov chain Monte Carlo history matching workflow. Surrogate error is included using a new treatment involving the full model error covariance matrix. A high degree of prior uncertainty, with geomodels characterized by uncertain geological scenario parameters (metaparameters) and associated realizations, is considered. History matching results for a synthetic true model are generated using in-situ monitoring-well data only, surface displacement data only, and both data types. The enhanced uncertainty reduction achieved with both data types is quantified. Posterior saturation and surface displacement fields are shown to correspond well with the true solution.
- [256] arXiv:2408.10718 [pdf, html, other]
-
Title: CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?Comments: Work in progressSubjects: Software Engineering (cs.SE); Computation and Language (cs.CL)
Recent advancements in large language models (LLMs) have showcased impressive code generation capabilities, primarily evaluated through language-to-code benchmarks. However, these benchmarks may not fully capture a model's code understanding abilities. We introduce CodeJudge-Eval (CJ-Eval), a novel benchmark designed to assess LLMs' code understanding abilities from the perspective of code judging rather than code generation. CJ-Eval challenges models to determine the correctness of provided code solutions, encompassing various error types and compilation issues. By leveraging a diverse set of problems and a fine-grained judging system, CJ-Eval addresses the limitations of traditional benchmarks, including the potential memorization of solutions. Evaluation of 12 well-known LLMs on CJ-Eval reveals that even state-of-the-art models struggle, highlighting the benchmark's ability to probe deeper into models' code understanding abilities. Our benchmark will be available at \url{this https URL}.
- [257] arXiv:2408.10720 [pdf, html, other]
-
Title: Towards Foundation Models for the Industrial Forecasting of Chemical KineticsComments: Accepted into the IEEE CAI 2024 Workshop on Scientific Machine Learning and Its Industrial Applications (SMLIA2024)Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Scientific Machine Learning is transforming traditional engineering industries by enhancing the efficiency of existing technologies and accelerating innovation, particularly in modeling chemical reactions. Despite recent advancements, the issue of solving stiff chemically reacting problems within computational fluid dynamics remains a significant issue. In this study we propose a novel approach utilizing a multi-layer-perceptron mixer architecture (MLP-Mixer) to model the time-series of stiff chemical kinetics. We evaluate this method using the ROBER system, a benchmark model in chemical kinetics, to compare its performance with traditional numerical techniques. This study provides insight into the industrial utility of the recently developed MLP-Mixer architecture to model chemical kinetics and provides motivation for such neural architecture to be used as a base for time-series foundation models.
- [258] arXiv:2408.10722 [pdf, html, other]
-
Title: MEGen: Generative Backdoor in Large Language Models via Model EditingComments: Working in progressSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Large language models (LLMs) have demonstrated remarkable capabilities. Their powerful generative abilities enable flexible responses based on various queries or instructions. Emerging as widely adopted generalists for diverse tasks, LLMs are still vulnerable to backdoors. This paper proposes an editing-based generative backdoor, named MEGen, aiming to create a customized backdoor for NLP tasks with the least side effects. In our approach, we first leverage a language model to insert a trigger selected on fixed metrics into the input, then design a pipeline of model editing to directly embed a backdoor into an LLM. By adjusting a small set of local parameters with a mini-batch of samples, MEGen significantly enhances time efficiency and achieves high robustness. Experimental results indicate that our backdoor attack strategy achieves a high attack success rate on poison data while maintaining the model's performance on clean data. Notably, the backdoored model, when triggered, can freely output pre-set dangerous information while successfully completing downstream tasks. This suggests that future LLM applications could be guided to deliver certain dangerous information, thus altering the LLM's generative style. We believe this approach provides insights for future LLM applications and the execution of backdoor attacks on conversational AI systems.
- [259] arXiv:2408.10724 [pdf, html, other]
-
Title: Crafting Tomorrow's Headlines: Neural News Generation and Detection in English, Turkish, Hungarian, and PersianSubjects: Computation and Language (cs.CL)
In the era dominated by information overload and its facilitation with Large Language Models (LLMs), the prevalence of misinformation poses a significant threat to public discourse and societal well-being. A critical concern at present involves the identification of machine-generated news. In this work, we take a significant step by introducing a benchmark dataset designed for neural news detection in four languages: English, Turkish, Hungarian, and Persian. The dataset incorporates outputs from multiple multilingual generators (in both, zero-shot and fine-tuned setups) such as BloomZ, LLaMa-2, Mistral, Mixtral, and GPT-4. Next, we experiment with a variety of classifiers, ranging from those based on linguistic features to advanced Transformer-based models and LLMs prompting. We present the detection results aiming to delve into the interpretablity and robustness of machine-generated texts detectors across all target languages.
- [260] arXiv:2408.10729 [pdf, html, other]
-
Title: Towards Efficient Large Language Models for Scientific Text: A ReviewSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Large language models (LLMs) have ushered in a new era for processing complex information in various fields, including science. The increasing amount of scientific literature allows these models to acquire and understand scientific knowledge effectively, thus improving their performance in a wide range of tasks. Due to the power of LLMs, they require extremely expensive computational resources, intense amounts of data, and training time. Therefore, in recent years, researchers have proposed various methodologies to make scientific LLMs more affordable. The most well-known approaches align in two directions. It can be either focusing on the size of the models or enhancing the quality of data. To date, a comprehensive review of these two families of methods has not yet been undertaken. In this paper, we (I) summarize the current advances in the emerging abilities of LLMs into more accessible AI solutions for science, and (II) investigate the challenges and opportunities of developing affordable solutions for scientific domains using LLMs.
- [261] arXiv:2408.10731 [pdf, other]
-
Title: Towards reliable real-time trajectory optimizationComments: PhD Thesis, University of Tartu, 2024. The thesis was defended on 21st of June. this https URLSubjects: Robotics (cs.RO); Systems and Control (eess.SY)
Motion planning is a key aspect of robotics. A common approach to address motion planning problems is trajectory optimization. Trajectory optimization can represent the high-level behaviors of robots through mathematical formulations. However, current trajectory optimization approaches have two main challenges. Firstly, their solution heavily depends on the initial guess, and they are prone to get stuck in local minima. Secondly, they face scalability limitations by increasing the number of constraints. This thesis endeavors to tackle these challenges by introducing four innovative trajectory optimization algorithms to improve reliability, scalability, and computational efficiency. There are two novel aspects of the proposed algorithms. The first key innovation is remodeling the kinematic constraints and collision avoidance constraints. Another key innovation lies in the design of algorithms that effectively utilize parallel computation on GPU accelerators. By using reformulated constraints and leveraging the computational power of GPUs, the proposed algorithms of this thesis demonstrate significant improvements in efficiency and scalability compared to the existing methods. Parallelization enables faster computation times, allowing for real-time decision-making in dynamic environments. Moreover, the algorithms are designed to adapt to changes in the environment, ensuring robust performance. Extensive benchmarking for each proposed optimizer validates their efficacy. Overall, this thesis makes a significant contribution to the field of trajectory optimization algorithms. It introduces innovative solutions that specifically address the challenges faced by existing methods. The proposed algorithms pave the way for more efficient and robust motion planning solutions in robotics by leveraging parallel computation and specific mathematical structures.
- [262] arXiv:2408.10734 [pdf, html, other]
-
Title: Vector Symbolic Open Source Information DiscoveryJournal-ref: Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications VI, vol. 13051, pp. 380-390. SPIE, 2024Subjects: Information Retrieval (cs.IR)
Combined, joint, intra-governmental, inter-agency and multinational (CJIIM) operations require rapid data sharing without the bottlenecks of metadata curation and alignment. Curation and alignment is particularly infeasible for external open source information (OSINF), e.g., social media, which has become increasingly valuable in understanding unfolding situations. Large language models (transformers) facilitate semantic data and metadata alignment but are inefficient in CJIIM settings characterised as denied, degraded, intermittent and low bandwidth (DDIL). Vector symbolic architectures (VSA) support semantic information processing using highly compact binary vectors, typically 1-10k bits, suitable in a DDIL setting. We demonstrate a novel integration of transformer models with VSA, combining the power of the former for semantic matching with the compactness and representational structure of the latter. The approach is illustrated via a proof-of-concept OSINF data discovery portal that allows partners in a CJIIM operation to share data sources with minimal metadata curation and low communications bandwidth. This work was carried out as a bridge between previous low technology readiness level (TRL) research and future higher-TRL technology demonstration and deployment.
- [263] arXiv:2408.10737 [pdf, html, other]
-
Title: Mid-Band Extra Large-Scale MIMO System: Channel Modeling and Performance AnalysisComments: 16 pages, 10 figuresSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
In pursuit of enhanced quality of service and higher transmission rates, communication within the mid-band spectrum, such as bands in the 6-15 GHz range, combined with extra large-scale multiple-input multiple-output (XL-MIMO), is considered a potential enabler for future communication systems. However, the characteristics introduced by mid-band XL-MIMO systems pose challenges for channel modeling and performance analysis. In this paper, we first analyze the potential characteristics of mid-band MIMO channels. Then, an analytical channel model incorporating novel channel characteristics is proposed, based on a review of classical analytical channel models. This model is convenient for theoretical analysis and compatible with other analytical channel models. Subsequently, based on the proposed channel model, we analyze key metrics of wireless communication, including the ergodic spectral efficiency (SE) and outage probability (OP) of MIMO maximal-ratio combining systems. Specifically, we derive closed-form approximations and performance bounds for two typical scenarios, aiming to illustrate the influence of mid-band XL-MIMO systems. Finally, comparisons between systems under different practical configurations are carried out through simulations. The theoretical analysis and simulations demonstrate that mid-band XL-MIMO systems excel in SE and OP due to the increased array elements, moderate large-scale fading, and enlarged transmission bandwidth.
- [264] arXiv:2408.10738 [pdf, html, other]
-
Title: PhishAgent: A Robust Multimodal Agent for Phishing Webpage DetectionSubjects: Cryptography and Security (cs.CR)
Phishing attacks are a major threat to online security, exploiting user vulnerabilities to steal sensitive information. Various methods have been developed to counteract phishing, each with varying levels of accuracy, but they also encounter notable limitations. In this study, we introduce PhishAgent, a multimodal agent that combines a wide range of tools, integrating both online and offline knowledge bases with Multimodal Large Language Models (MLLMs). This combination leads to broader brand coverage, which enhances brand recognition and recall. Furthermore, we propose a multimodal information retrieval framework designed to extract the top k relevant items from offline knowledge bases, utilizing all available information from a webpage, including logos, HTML, and URLs. Our empirical results, based on three real-world datasets, demonstrate that the proposed framework significantly enhances detection accuracy and reduces both false positives and false negatives, while maintaining model efficiency. Additionally, PhishAgent shows strong resilience against various types of adversarial attacks.
- [265] arXiv:2408.10739 [pdf, html, other]
-
Title: TrackNeRF: Bundle Adjusting NeRF from Sparse and Noisy Views via Feature TracksJinjie Mai, Wenxuan Zhu, Sara Rojas, Jesus Zarzar, Abdullah Hamdi, Guocheng Qian, Bing Li, Silvio Giancola, Bernard GhanemComments: ECCV 2024 (supplemental pages included)Subjects: Computer Vision and Pattern Recognition (cs.CV)
Neural radiance fields (NeRFs) generally require many images with accurate poses for accurate novel view synthesis, which does not reflect realistic setups where views can be sparse and poses can be noisy. Previous solutions for learning NeRFs with sparse views and noisy poses only consider local geometry consistency with pairs of views. Closely following \textit{bundle adjustment} in Structure-from-Motion (SfM), we introduce TrackNeRF for more globally consistent geometry reconstruction and more accurate pose optimization. TrackNeRF introduces \textit{feature tracks}, \ie connected pixel trajectories across \textit{all} visible views that correspond to the \textit{same} 3D points. By enforcing reprojection consistency among feature tracks, TrackNeRF encourages holistic 3D consistency explicitly. Through extensive experiments, TrackNeRF sets a new benchmark in noisy and sparse view reconstruction. In particular, TrackNeRF shows significant improvements over the state-of-the-art BARF and SPARF by $\sim8$ and $\sim1$ in terms of PSNR on DTU under various sparse and noisy view setups. The code is available at \href{this https URL}.
- [266] arXiv:2408.10746 [pdf, html, other]
-
Title: Pluto and Charon: A Time and Memory Efficient Collaborative Edge AI Framework for Personal LLMs Fine-TuningComments: Accepted by The 53rd International Conference on Parallel Processing (ICPP'24)Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Networking and Internet Architecture (cs.NI)
Large language models (LLMs) have unlocked a plethora of powerful applications at the network edge, such as intelligent personal assistants. Data privacy and security concerns have prompted a shift towards edge-based fine-tuning of personal LLMs, away from cloud reliance. However, this raises issues of computational intensity and resource scarcity, hindering training efficiency and feasibility. While current studies investigate parameter-efficient fine-tuning (PEFT) techniques to mitigate resource constraints, our analysis indicates that these techniques are not sufficiently resource-efficient for edge devices. To tackle these challenges, we propose Pluto and Charon (PAC), a time and memory efficient collaborative edge AI framework for personal LLMs fine-tuning. PAC breaks the resource wall of personal LLMs fine-tuning with a sophisticated algorithm-system co-design. (1) Algorithmically, PAC implements a personal LLMs fine-tuning technique that is efficient in terms of parameters, time, and memory. It utilizes Parallel Adapters to circumvent the need for a full backward pass through the LLM backbone. Additionally, an activation cache mechanism further streamlining the process by negating the necessity for repeated forward passes across multiple epochs. (2) Systematically, PAC leverages edge devices in close proximity, pooling them as a collective resource for in-situ personal LLMs fine-tuning, utilizing a hybrid data and pipeline parallelism to orchestrate distributed training. The use of the activation cache eliminates the need for forward pass through the LLM backbone,enabling exclusive fine-tuning of the Parallel Adapters using data parallelism. Extensive evaluation based on prototype implementation demonstrates that PAC remarkably outperforms state-of-the-art approaches, achieving up to 8.64x end-to-end speedup and up to 88.16% reduction in memory footprint.
- [267] arXiv:2408.10752 [pdf, html, other]
-
Title: Security Assessment of Hierarchical Federated Deep LearningJournal-ref: 33rd International Conference on Artificial Neural Networks (ICANN) (2024)Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR)
Hierarchical federated learning (HFL) is a promising distributed deep learning model training paradigm, but it has crucial security concerns arising from adversarial attacks. This research investigates and assesses the security of HFL using a novel methodology by focusing on its resilience against adversarial attacks inference-time and training-time. Through a series of extensive experiments across diverse datasets and attack scenarios, we uncover that HFL demonstrates robustness against untargeted training-time attacks due to its hierarchical structure. However, targeted attacks, particularly backdoor attacks, exploit this architecture, especially when malicious clients are positioned in the overlapping coverage areas of edge servers. Consequently, HFL shows a dual nature in its resilience, showcasing its capability to recover from attacks thanks to its hierarchical aggregation that strengthens its suitability for adversarial training, thereby reinforcing its resistance against inference-time attacks. These insights underscore the necessity for balanced security strategies in HFL systems, leveraging their inherent strengths while effectively mitigating vulnerabilities.
- [268] arXiv:2408.10754 [pdf, html, other]
-
Title: Ghost Echoes Revealed: Benchmarking Maintainability Metrics and Machine Learning Predictions Against Human AssessmentsComments: Accepted for publication in the Proc. of the 40th International Conference on Software Maintenance and Evolution (ICSME)Subjects: Software Engineering (cs.SE)
As generative AI is expected to increase global code volumes, the importance of maintainability from a human perspective will become even greater. Various methods have been developed to identify the most important maintainability issues, including aggregated metrics and advanced Machine Learning (ML) models. This study benchmarks several maintainability prediction approaches, including State-of-the-Art (SotA) ML, SonarQube's Maintainability Rating, CodeScene's Code Health, and Microsoft's Maintainability Index. Our results indicate that CodeScene matches the accuracy of SotA ML and outperforms the average human expert. Importantly, unlike SotA ML, CodeScene also provides end users with actionable code smell details to remedy identified issues. Finally, caution is advised with SonarQube due to its tendency to generate many false positives. Unfortunately, our findings call into question the validity of previous studies that solely relied on SonarQube output for establishing ground truth labels. To improve reliability in future maintainability and technical debt studies, we recommend employing more accurate metrics. Moreover, reevaluating previous findings with Code Health would mitigate this revealed validity threat.
- [269] arXiv:2408.10755 [pdf, html, other]
-
Title: Generating Synthetic Fair Syntax-agnostic Data by Learning and Distilling Fair RepresentationSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Data Fairness is a crucial topic due to the recent wide usage of AI powered applications. Most of the real-world data is filled with human or machine biases and when those data are being used to train AI models, there is a chance that the model will reflect the bias in the training data. Existing bias-mitigating generative methods based on GANs, Diffusion models need in-processing fairness objectives and fail to consider computational overhead while choosing computationally-heavy architectures, which may lead to high computational demands, instability and poor optimization performance. To mitigate this issue, in this work, we present a fair data generation technique based on knowledge distillation, where we use a small architecture to distill the fair representation in the latent space. The idea of fair latent space distillation enables more flexible and stable training of Fair Generative Models (FGMs). We first learn a syntax-agnostic (for any data type) fair representation of the data, followed by distillation in the latent space into a smaller model. After distillation, we use the distilled fair latent space to generate high-fidelity fair synthetic data. While distilling, we employ quality loss (for fair distillation) and utility loss (for data utility) to ensure that the fairness and data utility characteristics remain in the distilled latent space. Our approaches show a 5%, 5% and 10% rise in performance in fairness, synthetic sample quality and data utility, respectively, than the state-of-the-art fair generative model.
- [270] arXiv:2408.10756 [pdf, html, other]
-
Title: Data Ethics and Practices of Human-Nonhuman Sound Technologies and EcologiesSubjects: Human-Computer Interaction (cs.HC)
Human-nonhuman sound interaction and technologies aim to bridge the gap of inter-species communication. While they emerge from attempts to understand and communicate with nonhumans, they also raise questions on the ethics of nonhuman data use, for example regarding the unintended consequences such data extraction can have to nonhumans. In this paper, we discuss power relations and aspects of representation in nonhuman data practices, and their potential critical implications to nonhumans. Drawing from prior research on data ethics and posthumanities, we conceptualize two challenges of nonhuman data ethics for the design of Human-Nonhuman Interaction (HNI) and technologies in sound ecologies. We provide takeaways for how sensitivities toward nonhuman stakeholders can be considered in the design of HNI in the context of sound ecologies.
- [271] arXiv:2408.10757 [pdf, html, other]
-
Title: Decreasing verification radius in local certificationSubjects: Distributed, Parallel, and Cluster Computing (cs.DC)
This paper deals with local certification, specifically locally checkable proofs: given a graph property, the task is to certify whether a graph satisfies the property. The verification of this certification needs to be done locally without the knowledge of the whole graph. More precisely, a distributed algorithm, called a verifier, is executed on each vertex. The verifier observes the local neighborhood up to a constant distance and either accepts or rejects. We examine the trade-off between the visibility radius and the size of certificates. We describe a procedure that decreases the radius by encoding the neighbourhood of each vertex into its certificate. We also provide a corresponding lower bound on the required certificate size increase, showing that such an approach is close to optimal.
- [272] arXiv:2408.10758 [pdf, html, other]
-
Title: Does Co-Development with AI Assistants Lead to More Maintainable Code? A Registered ReportComments: Accepted as In-Principal Acceptance (IPA) for a Stage 1 registration of the Registered Report Track at the 40th IEEE International Conference on Software Maintenance and Evolution (ICSME)Subjects: Software Engineering (cs.SE)
[Background/Context] AI assistants like GitHub Copilot are transforming software engineering; several studies have highlighted productivity improvements. However, their impact on code quality, particularly in terms of maintainability, requires further investigation. [Objective/Aim] This study aims to examine the influence of AI assistants on software maintainability, specifically assessing how these tools affect the ability of developers to evolve code. [Method] We will conduct a two-phased controlled experiment involving professional developers. In Phase 1, developers will add a new feature to a Java project, with or without the aid of an AI assistant. Phase 2, a randomized controlled trial, will involve a different set of developers evolving random Phase 1 projects - working without AI assistants. We will employ Bayesian analysis to evaluate differences in completion time, perceived productivity, code quality, and test coverage.
- [273] arXiv:2408.10760 [pdf, html, other]
-
Title: SAM-COD: SAM-guided Unified Framework for Weakly-Supervised Camouflaged Object DetectionComments: Accepted by ECCV2024Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Most Camouflaged Object Detection (COD) methods heavily rely on mask annotations, which are time-consuming and labor-intensive to acquire. Existing weakly-supervised COD approaches exhibit significantly inferior performance compared to fully-supervised methods and struggle to simultaneously support all the existing types of camouflaged object labels, including scribbles, bounding boxes, and points. Even for Segment Anything Model (SAM), it is still problematic to handle the weakly-supervised COD and it typically encounters challenges of prompt compatibility of the scribble labels, extreme response, semantically erroneous response, and unstable feature representations, producing unsatisfactory results in camouflaged scenes. To mitigate these issues, we propose a unified COD framework in this paper, termed SAM-COD, which is capable of supporting arbitrary weakly-supervised labels. Our SAM-COD employs a prompt adapter to handle scribbles as prompts based on SAM. Meanwhile, we introduce response filter and semantic matcher modules to improve the quality of the masks obtained by SAM under COD prompts. To alleviate the negative impacts of inaccurate mask predictions, a new strategy of prompt-adaptive knowledge distillation is utilized to ensure a reliable feature representation. To validate the effectiveness of our approach, we have conducted extensive empirical experiments on three mainstream COD benchmarks. The results demonstrate the superiority of our method against state-of-the-art weakly-supervised and even fully-supervised methods.
- [274] arXiv:2408.10761 [pdf, html, other]
-
Title: On the Power of Graphical Reconfigurable CircuitsSubjects: Distributed, Parallel, and Cluster Computing (cs.DC)
We introduce the \emph{graphical reconfigurable circuits (GRC)} model as an abstraction for distributed graph algorithms whose communication scheme is based on local mechanisms that collectively construct long-range reconfigurable channels (this is an extension to general graphs of a distributed computational model recently introduced by Feldmann et al.\ (JCB 2022) for hexagonal grids). The crux of the GRC model lies in its modest assumptions: (1) the individual nodes are computationally weak, with state space bounded independently of any global graph parameter; and (2) the reconfigurable communication channels are highly restrictive, only carrying information-less signals (a.k.a.\ \emph{beeps}). Despite these modest assumptions, we prove that GRC algorithms can solve many important distributed tasks efficiently, i.e., in polylogarithmic time. On the negative side, we establish various runtime lower bounds, proving that for other tasks, GRC algorithms (if they exist) are doomed to be slow.
- [275] arXiv:2408.10763 [pdf, html, other]
-
Title: Analyzing the Impact of Electric Vehicles on Local Energy Systems using Digital TwinsComments: Paper is to be published in Proceedings of the 2024 Winter Simulation ConferenceSubjects: Multiagent Systems (cs.MA)
The electrification of the transportation and heating sector, the so-called sector coupling, is one of the core elements to achieve independence from fossil fuels. As it highly affects the electricity demand, especially on the local level, the integrated modeling and simulation of all sectors is a promising approach for analyzing design decisions or complex control strategies. This paper analyzes the increase in electricity demand resulting from sector coupling, mainly due to integrating electric vehicles into urban energy systems. Therefore, we utilize a digital twin of an existing local energy system and extend it with a mobility simulation model to evaluate the impact of electric vehicles on the distribution grid level. Our findings indicate a significant rise in annual electricity consumption attributed to electric vehicles, with home charging alone resulting in a 78% increase. However, we demonstrate that integrating photovoltaic and battery energy storage systems can effectively mitigate this rise.
- [276] arXiv:2408.10764 [pdf, html, other]
-
Title: Predicting Rewards Alongside Tokens: Non-disruptive Parameter Insertion for Efficient Inference Intervention in Large Language ModelComments: 16 pagesSubjects: Computation and Language (cs.CL)
Transformer-based large language models (LLMs) exhibit limitations such as generating unsafe responses, unreliable reasoning, etc. Existing inference intervention approaches attempt to mitigate these issues by finetuning additional models to produce calibration signals (such as rewards) that guide the LLM's decoding process. However, this solution introduces substantial time and space overhead due to the separate models required. This work proposes Non-disruptive parameters insertion (Otter), inserting extra parameters into the transformer architecture to predict calibration signals along with the original LLM output. Otter offers state-of-the-art performance on multiple demanding tasks while saving up to 86.5\% extra space and 98.5\% extra time. Furthermore, Otter seamlessly integrates with existing inference engines, requiring only a one-line code change, and the original model response remains accessible after the parameter insertion. Our code is publicly available at \url{this https URL}
- [277] arXiv:2408.10766 [pdf, html, other]
-
Title: An Open Source Python Library for Anonymizing Sensitive DataComments: Preprint under reviewSubjects: Cryptography and Security (cs.CR); Databases (cs.DB); Software Engineering (cs.SE)
Open science is a fundamental pillar to promote scientific progress and collaboration, based on the principles of open data, open source and open access. However, the requirements for publishing and sharing open data are in many cases difficult to meet in compliance with strict data protection regulations. Consequently, researchers need to rely on proven methods that allow them to anonymize their data without sharing it with third parties. To this end, this paper presents the implementation of a Python library for the anonymization of sensitive tabular data. This framework provides users with a wide range of anonymization methods that can be applied on the given dataset, including the set of identifiers, quasi-identifiers, generalization hierarchies and allowed level of suppression, along with the sensitive attribute and the level of anonymity required. The library has been implemented following best practices for integration and continuous development, as well as the use of workflows to test code coverage based on unit and functional tests.
- [278] arXiv:2408.10768 [pdf, html, other]
-
Title: Detection of Intracranial Hemorrhage for Trauma PatientsSubjects: Computer Vision and Pattern Recognition (cs.CV)
Whole-body CT is used for multi-trauma patients in the search of any and all injuries. Since an initial assessment needs to be rapid and the search for lesions is done for the whole body, very little time can be allocated for the inspection of a specific anatomy. In particular, intracranial hemorrhages are still missed, especially by clinical students. In this work, we present a Deep Learning approach for highlighting such lesions to improve the diagnostic accuracy. While most works on intracranial hemorrhages perform segmentation, detection only requires bounding boxes for the localization of the bleeding. In this paper, we propose a novel Voxel-Complete IoU (VC-IoU) loss that encourages the network to learn the 3D aspect ratios of bounding boxes and leads to more precise detections. We extensively experiment on brain bleeding detection using a publicly available dataset, and validate it on a private cohort, where we achieve 0.877 AR30, 0.728 AP30, and 0.653 AR30, 0.514 AP30 respectively. These results constitute a relative +5% improvement in Average Recall for both datasets compared to other loss functions. Finally, as there is little data currently publicly available for 3D object detection and as annotation resources are limited in the clinical setting, we evaluate the cost of different annotation methods, as well as the impact of imprecise bounding boxes in the training data on the detection performance.
- [279] arXiv:2408.10773 [pdf, other]
-
Title: Multi-Agent Based Simulation for Investigating Centralized Charging Strategies and their Impact on Electric Vehicle Home Charging EcosystemSubjects: Multiagent Systems (cs.MA)
This paper addresses the critical integration of electric vehicles (EVs) into the electricity grid, which is essential for achieving carbon neutrality by 2050. The rapid increase in EV adoption poses significant challenges to the existing grid infrastructure, particularly in managing the increasing electricity demand and mitigating the risk of grid overloads. Centralized EV charging strategies are investigated due to their potential to optimize grid stability and efficiency, compared to decentralized approaches that may exacerbate grid stress. Utilizing a multi-agent based simulation model, the study provides a realistic representation of the electric vehicle home charging ecosystem in a case study of Strib, Denmark. The findings show that the Earliest-deadline-first and Round Robin perform best with 100% EV adoption in terms of EV user satisfaction. The simulation considers a realistic adoption curve, EV charging strategies, EV models, and driving patterns to capture the full ecosystem dynamics over a long-term period with high resolution (hourly). Additionally, the study offers detailed load profiles for future distribution grids, demonstrating how centralized charging strategies can efficiently manage grid loads and prevent overloads.
- [280] arXiv:2408.10774 [pdf, html, other]
-
Title: Flexora: Flexible Low Rank Adaptation for Large Language ModelsComments: 29 pages, 13 figuresSubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Large Language Models (LLMs) are driving advancements in artificial intelligence by increasing the scale of model parameters, which has significantly enhanced generalization ability and unlocked new capabilities in practice. However, their performance in specific downstream tasks is usually hindered by their knowledge boundaries on these tasks. Thus, fine-tuning techniques, especially the widely used Low-Rank Adaptation (LoRA) method, have been introduced to expand the boundaries on these tasks, whereas LoRA would underperform on certain tasks owing to its potential overfitting on these tasks. To overcome this overfitting and improve the performance of LoRA, we propose the flexible low rank adaptation (Flexora) method to automatically and flexibly select the most important layers needing to be fine-tuned to achieve the best performance on different downstream tasks. Specifically, Flexora firstly frames this layer selection problem as a well-defined hyperparameter optimization (HPO) problem, then addresses it using the unrolled differentiation (UD) method, and finally selects the most useful layers based on the optimized hyperparameters. Our extensive experiments on many pretrained models and natural language tasks show that Flexora is able to consistently improve over the existing baselines, indicating the effectiveness of our Flexora in practice. We additionally provide insightful theoretical results and many ablation studies to deliver a comprehensive understanding of our Flexora.
- [281] arXiv:2408.10775 [pdf, html, other]
-
Title: Generative AI in Industrial Machine Vision -- A ReviewHans Aoyang Zhou, Dominik Wolfschläger, Constantinos Florides, Jonas Werheid, Hannes Behnen, Jan-Henrick Woltersmann, Tiago C. Pinto, Marco Kemmerling, Anas Abdelrazeq, Robert H. SchmittComments: 44 pages, 7 figures, This work has been submitted to the Journal of Intelligent ManufacturingSubjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV)
Machine vision enhances automation, quality control, and operational efficiency in industrial applications by enabling machines to interpret and act on visual data. While traditional computer vision algorithms and approaches remain widely utilized, machine learning has become pivotal in current research activities. In particular, generative \gls*{AI} demonstrates promising potential by improving pattern recognition capabilities, through data augmentation, increasing image resolution, and identifying anomalies for quality control. However, the application of generative \gls*{AI} in machine vision is still in its early stages due to challenges in data diversity, computational requirements, and the necessity for robust validation methods. A comprehensive literature review is essential to understand the current state of generative \gls*{AI} in industrial machine vision, focusing on recent advancements, applications, and research trends. Thus, a literature review based on the PRISMA guidelines was conducted, analyzing over 1,200 papers on generative \gls*{AI} in industrial machine vision. Our findings reveal various patterns in current research, with the primary use of generative \gls*{AI} being data augmentation, for machine vision tasks such as classification and object detection. Furthermore, we gather a collection of application challenges together with data requirements to enable a successful application of generative \gls*{AI} in industrial machine vision. This overview aims to provide researchers with insights into the different areas and applications within current research, highlighting significant advancements and identifying opportunities for future work.
- [282] arXiv:2408.10777 [pdf, html, other]
-
Title: Just a Hint: Point-Supervised Camouflaged Object DetectionComments: Accepted by ECCV2024Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Camouflaged Object Detection (COD) demands models to expeditiously and accurately distinguish objects which conceal themselves seamlessly in the environment. Owing to the subtle differences and ambiguous boundaries, COD is not only a remarkably challenging task for models but also for human annotators, requiring huge efforts to provide pixel-wise annotations. To alleviate the heavy annotation burden, we propose to fulfill this task with the help of only one point supervision. Specifically, by swiftly clicking on each object, we first adaptively expand the original point-based annotation to a reasonable hint area. Then, to avoid partial localization around discriminative parts, we propose an attention regulator to scatter model attention to the whole object through partially masking labeled regions. Moreover, to solve the unstable feature representation of camouflaged objects under only point-based annotation, we perform unsupervised contrastive learning based on differently augmented image pairs (e.g. changing color or doing translation). On three mainstream COD benchmarks, experimental results show that our model outperforms several weakly-supervised methods by a large margin across various metrics.
- [283] arXiv:2408.10779 [pdf, other]
-
Title: The Power of Abstract MAC Layer: A Fault-tolerance PerspectiveComments: A full version for our DISC24 paperSubjects: Distributed, Parallel, and Cluster Computing (cs.DC)
This paper studies the power of the "abstract MAC layer" model in a single-hop asynchronous network. The model captures primitive properties of modern wireless MAC protocols. In this model, Newport [PODC '14] proves that it is impossible to achieve deterministic consensus when nodes may crash. Subsequently, Newport and Robinson [DISC '18] present randomized consensus algorithms that terminate with O(n3 log n) expected broadcasts in a system of n nodes. We are not aware of any results on other fault-tolerant distributed tasks in this model.
We first study the computability aspect of the abstract MAC layer. We present a wait-free algorithm that implements an atomic register. Furthermore, we show that in general, k-set consensus is impossible. Second, we aim to minimize storage complexity. Existing algorithms require {\Omega}(n log n) bits. We propose four wait-free consensus algorithms that only need constant storage complexity. (Two approximate consensus and two randomized binary consensus algorithms.) One randomized algorithm terminates with O(n log n) expected broadcasts. All our consensus algorithms are anonymous, meaning that at the algorithm level, nodes do not need to have a unique identifier. - [284] arXiv:2408.10783 [pdf, other]
-
Title: Multi-agent based modeling for investigating excess heat utilization from electrolyzer production to district heating networkSubjects: Multiagent Systems (cs.MA)
Power-to-Hydrogen is crucial for the renewable energy transition, yet existing literature lacks business models for the significant excess heat it generates. This study addresses this by evaluating three models for selling electrolyzer-generated heat to district heating grids: constant, flexible, and renewable-source hydrogen production, with and without heat sales. Using agent-based modeling and multi-criteria decision-making methods (VIKOR, TOPSIS, PROMETHEE), it finds that selling excess heat can cut hydrogen production costs by 5.6%. The optimal model operates flexibly with electricity spot prices, includes heat sales, and maintains a hydrogen price of 3.3 EUR/kg. Environmentally, hydrogen production from grid electricity could emit up to 13,783.8 tons of CO2 over four years from 2023. The best economic and environmental model uses renewable sources and sells heat at 3.5 EUR/kg
- [285] arXiv:2408.10784 [pdf, other]
-
Title: Framework for uncertainty quantification of wave-structure interaction in a flumeSubjects: Numerical Analysis (math.NA); Mathematical Physics (math-ph)
In this paper we propose a numerical procedure for the quantification of uncertainties in wave-structure interaction. We utilise the smoothed particle hydrodynamics (SPH) scheme for modelling the wave mechanics, coupled one-way with a finite element method (FEM) for the structural response. Physical wave flumes are extensively used in the study of hydrodynamics especially in wave-structure interaction (WSI) and prediction of forces to near-shore structures in disaster mitigation and offshore structures in the oil and gas, and more recently renewable energy sector. Over the years, numerical wave flumes have been developed extensively to enable the modelling of complex wave-structure interaction. However, most of these studies are deterministic and limited to using either simple flexible beams or rigid monolithic structures to model the structural part in the WSI. Additionally, uncertainties are commonly observed in both wave and structural parameters and need to be accounted for. This work presents a numerical framework to enable uncertainty quantification for wave-structure interaction problems in terms of the forces experienced by the structure. A one-way coupling between SPH with the FEM and uncertainty quantification (UQ) is proposed. We employ the so-called Tokyo wave flume geometry, which has potential for future surrogate modelling in WSI. The developed model is validated using numerical and experimental results from the literature and is used to demonstrate the prediction of probabilistic responses of structures under breaking and non-breaking wave scenarios.
- [286] arXiv:2408.10787 [pdf, html, other]
-
Title: LightMDETR: A Lightweight Approach for Low-Cost Open-Vocabulary Object Detection TrainingSubjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Object detection in computer vision traditionally involves identifying objects in images. By integrating textual descriptions, we enhance this process, providing better context and accuracy. The MDETR model significantly advances this by combining image and text data for more versatile object detection and classification. However, MDETR's complexity and high computational demands hinder its practical use. In this paper, we introduce Lightweight MDETR (LightMDETR), an optimized MDETR variant designed for improved computational efficiency while maintaining robust multimodal capabilities. Our approach involves freezing the MDETR backbone and training a sole component, the Deep Fusion Encoder (DFE), to represent image and text modalities. A learnable context vector enables the DFE to switch between these modalities. Evaluation on datasets like RefCOCO, RefCOCO+, and RefCOCOg demonstrates that LightMDETR achieves superior precision and accuracy.
- [287] arXiv:2408.10788 [pdf, other]
-
Title: Understanding the Skills Gap between Higher Education and Industry in the UK in Artificial Intelligence SectorComments: Accepted to the journal "Industry and Higher Education"Subjects: Artificial Intelligence (cs.AI)
As Artificial Intelligence (AI) changes how businesses work, there is a growing need for people who can work in this sector. This paper investigates how well universities in United Kingdom offering courses in AI, prepare students for jobs in the real world. To gain insight into the differences between university curricula and industry demands we review the contents of taught courses and job advertisement portals. By using custom data scraping tools to gather information from job advertisements and university curricula, and frequency and Naive Bayes classifier analysis, this study will show exactly what skills industry is looking for. In this study we identified 12 skill categories that were used for mapping. The study showed that the university curriculum in the AI domain is well balanced in most technical skills, including Programming and Machine learning subjects, but have a gap in Data Science and Maths and Statistics skill categories.
- [288] arXiv:2408.10789 [pdf, html, other]
-
Title: Learning Part-aware 3D Representations by Fusing 2D Gaussians and SuperquadricsSubjects: Computer Vision and Pattern Recognition (cs.CV)
Low-level 3D representations, such as point clouds, meshes, NeRFs, and 3D Gaussians, are commonly used to represent 3D objects or scenes. However, humans usually perceive 3D objects or scenes at a higher level as a composition of parts or structures rather than points or voxels. Representing 3D as semantic parts can benefit further understanding and applications. We aim to solve part-aware 3D reconstruction, which parses objects or scenes into semantic parts. In this paper, we introduce a hybrid representation of superquadrics and 2D Gaussians, trying to dig 3D structural clues from multi-view image inputs. Accurate structured geometry reconstruction and high-quality rendering are achieved at the same time. We incorporate parametric superquadrics in mesh forms into 2D Gaussians by attaching Gaussian centers to faces in meshes. During the training, superquadrics parameters are iteratively optimized, and Gaussians are deformed accordingly, resulting in an efficient hybrid representation. On the one hand, this hybrid representation inherits the advantage of superquadrics to represent different shape primitives, supporting flexible part decomposition of scenes. On the other hand, 2D Gaussians are incorporated to model the complex texture and geometry details, ensuring high-quality rendering and geometry reconstruction. The reconstruction is fully unsupervised. We conduct extensive experiments on data from DTU and ShapeNet datasets, in which the method decomposes scenes into reasonable parts, outperforming existing state-of-the-art approaches.
- [289] arXiv:2408.10790 [pdf, other]
-
Title: Multi-Agent Based Simulation for Decentralized Electric Vehicle Charging Strategies and their ImpactsSubjects: Multiagent Systems (cs.MA)
The growing shift towards a Smart Grid involves integrating numerous new digital energy solutions into the energy ecosystems to address problems arising from the transition to carbon neutrality, particularly in linking the electricity and transportation sectors. Yet, this shift brings challenges due to mass electric vehicle adoption and the lack of methods to adequately assess various EV charging algorithms and their ecosystem impacts. This paper introduces a multi-agent based simulation model, validated through a case study of a Danish radial distribution network serving 126 households. The study reveals that traditional charging leads to grid overload by 2031 at 67% EV penetration, while decentralized strategies like Real-Time Pricing could cause overloads as early as 2028. The developed multi-agent based simulation demonstrates its ability to offer detailed, hourly analysis of future load profiles in distribution grids, and therefore, can be applied to other prospective scenarios in similar energy systems.
- [290] arXiv:2408.10791 [pdf, other]
-
Title: Exploring the Impact of Word Prediction Assistive Features on Smartphone Keyboards for Blind UsersComments: 17 pagesSubjects: Human-Computer Interaction (cs.HC)
Assistive technologies have been developed to enhance blind users' typing performance, focusing on speed, accuracy, and effort reduction. One such technology is word prediction software, designed to minimize keystrokes required for text input. This study investigates the impact of word prediction on typing performance among blind users using an on-screen QWERTY keyboard. We conducted a comparative study involving eleven blind participants, evaluating both standard QWERTY input and word prediction-assisted typing. Our findings reveal that while word prediction slightly improves typing speed, it does not enhance typing accuracy and increases both physical and temporal workload compared to the default keyboard. We conclude with recommendations for improving word prediction systems, including more efficient editing methods and the integration of voice pitch variations to aid error recognition.
- [291] arXiv:2408.10794 [pdf, html, other]
-
Title: Tapping in a Remote Vehicle's onboard LLM to Complement the Ego Vehicle's Field-of-ViewComments: 50th Euromicro Conference Series on Software Engineering and Advanced Applications (SEAA) 2024 - WiPSubjects: Computer Vision and Pattern Recognition (cs.CV)
Today's advanced automotive systems are turning into intelligent Cyber-Physical Systems (CPS), bringing computational intelligence to their cyber-physical context. Such systems power advanced driver assistance systems (ADAS) that observe a vehicle's surroundings for their functionality. However, such ADAS have clear limitations in scenarios when the direct line-of-sight to surrounding objects is occluded, like in urban areas. Imagine now automated driving (AD) systems that ideally could benefit from other vehicles' field-of-view in such occluded situations to increase traffic safety if, for example, locations about pedestrians can be shared across vehicles. Current literature suggests vehicle-to-infrastructure (V2I) via roadside units (RSUs) or vehicle-to-vehicle (V2V) communication to address such issues that stream sensor or object data between vehicles. When considering the ongoing revolution in vehicle system architectures towards powerful, centralized processing units with hardware accelerators, foreseeing the onboard presence of large language models (LLMs) to improve the passengers' comfort when using voice assistants becomes a reality. We are suggesting and evaluating a concept to complement the ego vehicle's field-of-view (FOV) with another vehicle's FOV by tapping into their onboard LLM to let the machines have a dialogue about what the other vehicle ``sees''. Our results show that very recent versions of LLMs, such as GPT-4V and GPT-4o, understand a traffic situation to an impressive level of detail, and hence, they can be used even to spot traffic participants. However, better prompts are needed to improve the detection quality and future work is needed towards a standardised message interchange format between vehicles.
- [292] arXiv:2408.10795 [pdf, html, other]
-
Title: Adversarial Attack for Explanation Robustness of Rationalization ModelsSubjects: Computation and Language (cs.CL)
Rationalization models, which select a subset of input text as rationale-crucial for humans to understand and trust predictions-have recently emerged as a prominent research area in eXplainable Artificial Intelligence. However, most of previous studies mainly focus on improving the quality of the rationale, ignoring its robustness to malicious attack. Specifically, whether the rationalization models can still generate high-quality rationale under the adversarial attack remains unknown. To explore this, this paper proposes UAT2E, which aims to undermine the explainability of rationalization models without altering their predictions, thereby eliciting distrust in these models from human users. UAT2E employs the gradient-based search on triggers and then inserts them into the original input to conduct both the non-target and target attack. Experimental results on five datasets reveal the vulnerability of rationalization models in terms of explanation, where they tend to select more meaningless tokens under attacks. Based on this, we make a series of recommendations for improving rationalization models in terms of explanation.
- [293] arXiv:2408.10796 [pdf, other]
-
Title: Honeyquest: Rapidly Measuring the Enticingness of Cyber Deception Techniques with Code-based QuestionnairesComments: to be published in the 27th International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2024), dataset and source code available at this https URLSubjects: Cryptography and Security (cs.CR); Computers and Society (cs.CY)
Fooling adversaries with traps such as honeytokens can slow down cyber attacks and create strong indicators of compromise. Unfortunately, cyber deception techniques are often poorly specified. Also, realistically measuring their effectiveness requires a well-exposed software system together with a production-ready implementation of these techniques. This makes rapid prototyping challenging. Our work translates 13 previously researched and 12 self-defined techniques into a high-level, machine-readable specification. Our open-source tool, Honeyquest, allows researchers to quickly evaluate the enticingness of deception techniques without implementing them. We test the enticingness of 25 cyber deception techniques and 19 true security risks in an experiment with 47 humans. We successfully replicate the goals of previous work with many consistent findings, but without a time-consuming implementation of these techniques on real computer systems. We provide valuable insights for the design of enticing deception and also show that the presence of cyber deception can significantly reduce the risk that adversaries will find a true security risk by about 22% on average.
- [294] arXiv:2408.10798 [pdf, html, other]
-
Title: Universal Novelty Detection Through Adaptive Contrastive LearningHossein Mirzaei, Mojtaba Nafez, Mohammad Jafari, Mohammad Bagher Soltani, Mohammad Azizmalayeri, Jafar Habibi, Mohammad Sabokrou, Mohammad Hossein RohbanComments: 16 pages, 5 figures, conferenceSubjects: Machine Learning (cs.LG)
Novelty detection is a critical task for deploying machine learning models in the open world. A crucial property of novelty detection methods is universality, which can be interpreted as generalization across various distributions of training or test data. More precisely, for novelty detection, distribution shifts may occur in the training set or the test set. Shifts in the training set refer to cases where we train a novelty detector on a new dataset and expect strong transferability. Conversely, distribution shifts in the test set indicate the methods' performance when the trained model encounters a shifted test sample. We experimentally show that existing methods falter in maintaining universality, which stems from their rigid inductive biases. Motivated by this, we aim for more generalized techniques that have more adaptable inductive biases. In this context, we leverage the fact that contrastive learning provides an efficient framework to easily switch and adapt to new inductive biases through the proper choice of augmentations in forming the negative pairs. We propose a novel probabilistic auto-negative pair generation method AutoAugOOD, along with contrastive learning, to yield a universal novelty detector method. Our experiments demonstrate the superiority of our method under different distribution shifts in various image benchmark datasets. Notably, our method emerges universality in the lens of adaptability to different setups of novelty detection, including one-class, unlabeled multi-class, and labeled multi-class settings. Code: this https URL
- [295] arXiv:2408.10802 [pdf, html, other]
-
Title: Inverse Deep Learning Ray Tracing for Heliostat Surface PredictionSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Concentrating Solar Power (CSP) plants play a crucial role in the global transition towards sustainable energy. A key factor in ensuring the safe and efficient operation of CSP plants is the distribution of concentrated flux density on the receiver. However, the non-ideal flux density generated by individual heliostats can undermine the safety and efficiency of the power plant. The flux density from each heliostat is influenced by its precise surface profile, which includes factors such as canting and mirror errors. Accurately measuring these surface profiles for a large number of heliostats in operation is a formidable challenge. Consequently, control systems often rely on the assumption of ideal surface conditions, which compromises both safety and operational efficiency. In this study, we introduce inverse Deep Learning Ray Tracing (iDLR), an innovative method designed to predict heliostat surfaces based solely on target images obtained during heliostat calibration. Our simulation-based investigation demonstrates that sufficient information regarding the heliostat surface is retained in the flux density distribution of a single heliostat, enabling deep learning models to accurately predict the underlying surface with deflectometry-like precision for the majority of heliostats. Additionally, we assess the limitations of this method, particularly in relation to surface accuracy and resultant flux density predictions. Furthermore, we are presenting a new comprehensive heliostat model using Non-Uniform Rational B-Spline (NURBS) that has the potential to become the new State of the Art for heliostat surface parameterization. Our findings reveal that iDLR has significant potential to enhance CSP plant operations, potentially increasing the overall efficiency and energy output of the power plants.
- [296] arXiv:2408.10804 [pdf, html, other]
-
Title: Kotlin's Type System is (Also) UnsoundSubjects: Programming Languages (cs.PL); Software Engineering (cs.SE)
Soundness of a type system is a fundemental property that guarantees that no operation that is not supported by a value will be performed on that value at run time. A type checker for a sound type system is expected to issue a warning on every type error. While soundness is a desirable property for many practical applications, in 2016, Amin and Tate presented the first unsoundness proof for two major industry languages: Java and Scala. This proof relied on use-site variance and implicit null values.
We present an unsoundness proof for Kotlin, another emerging industry language, which relies on a previously unknown unsound combination of language features. Kotlin does not have implicit null values, meaning that the proof by Amin and Tate would not work for Kotlin. Our new proof, which is an infringing code snippet, utilizes Kotlin's \emph{declaration-site} variance specification and does not require implicit null values.
We present this counterexample to soundness in full along with detailed explanations of every step. Finally, we present a thorough discussion on precisely which language features cause this issue, as well as how Kotlin's compiler can be patched to fix it. - [297] arXiv:2408.10805 [pdf, html, other]
-
Title: MPL: Lifting 3D Human Pose from Multi-view 2D PosesComments: 14 pages, accepted in ECCV T-CAP 2024, code: this https URLSubjects: Computer Vision and Pattern Recognition (cs.CV)
Estimating 3D human poses from 2D images is challenging due to occlusions and projective acquisition. Learning-based approaches have been largely studied to address this challenge, both in single and multi-view setups. These solutions however fail to generalize to real-world cases due to the lack of (multi-view) 'in-the-wild' images paired with 3D poses for training. For this reason, we propose combining 2D pose estimation, for which large and rich training datasets exist, and 2D-to-3D pose lifting, using a transformer-based network that can be trained from synthetic 2D-3D pose pairs. Our experiments demonstrate decreases up to 45% in MPJPE errors compared to the 3D pose obtained by triangulating the 2D poses. The framework's source code is available at this https URL .
- [298] arXiv:2408.10807 [pdf, html, other]
-
Title: DisMix: Disentangling Mixtures of Musical Instruments for Source-level Pitch and Timbre ManipulationYin-Jyun Luo, Kin Wai Cheuk, Woosung Choi, Toshimitsu Uesaka, Keisuke Toyama, Koichi Saito, Chieh-Hsin Lai, Yuhta Takida, Wei-Hsiang Liao, Simon Dixon, Yuki MitsufujiSubjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
Existing work on pitch and timbre disentanglement has been mostly focused on single-instrument music audio, excluding the cases where multiple instruments are presented. To fill the gap, we propose DisMix, a generative framework in which the pitch and timbre representations act as modular building blocks for constructing the melody and instrument of a source, and the collection of which forms a set of per-instrument latent representations underlying the observed mixture. By manipulating the representations, our model samples mixtures with novel combinations of pitch and timbre of the constituent instruments. We can jointly learn the disentangled pitch-timbre representations and a latent diffusion transformer that reconstructs the mixture conditioned on the set of source-level representations. We evaluate the model using both a simple dataset of isolated chords and a realistic four-part chorales in the style of J.S. Bach, identify the key components for the success of disentanglement, and demonstrate the application of mixture transformation based on source-level attribute manipulation.
- [299] arXiv:2408.10808 [pdf, html, other]
-
Title: ColBERT Retrieval and Ensemble Response Scoring for Language Model Question AnsweringComments: This work has been submitted to the 2024 IEEE Globecom Workshops for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibleSubjects: Computation and Language (cs.CL); Information Retrieval (cs.IR)
Domain-specific question answering remains challenging for language models, given the deep technical knowledge required to answer questions correctly. This difficulty is amplified for smaller language models that cannot encode as much information in their parameters as larger models. The "Specializing Large Language Models for Telecom Networks" challenge aimed to enhance the performance of two small language models, Phi-2 and Falcon-7B in telecommunication question answering. In this paper, we present our question answering systems for this challenge. Our solutions achieved leading marks of 81.9% accuracy for Phi-2 and 57.3% for Falcon-7B. We have publicly released our code and fine-tuned models.
- [300] arXiv:2408.10811 [pdf, html, other]
-
Title: Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in?Chengzhi Zhong, Fei Cheng, Qianying Liu, Junfeng Jiang, Zhen Wan, Chenhui Chu, Yugo Murawaki, Sadao KurohashiComments: work in progressSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
In this study, we investigate whether non-English-centric LLMs, despite their strong performance, `think' in their respective dominant language: more precisely, `think' refers to how the representations of intermediate layers, when un-embedded into the vocabulary space, exhibit higher probabilities for certain dominant languages during generation. We term such languages as internal $\textbf{latent languages}$.
We examine the latent language of three typical categories of models for Japanese processing: Llama2, an English-centric model; Swallow, an English-centric model with continued pre-training in Japanese; and LLM-jp, a model pre-trained on balanced English and Japanese corpora. Our empirical findings reveal that, unlike Llama2 which relies exclusively on English as the internal latent language, Japanese-specific Swallow and LLM-jp employ both Japanese and English, exhibiting dual internal latent languages. For any given target language, the model preferentially activates the latent language most closely related to it. In addition, we explore how intermediate layers respond to questions involving cultural conflicts between latent internal and target output languages. We further explore how the language identity shifts across layers while keeping consistent semantic meaning reflected in the intermediate layer representations.
This study deepens the understanding of non-English-centric large language models, highlighting the intricate dynamics of language representation within their intermediate layers. - [301] arXiv:2408.10812 [pdf, other]
-
Title: Use Cases for Prospective Sensemaking of Human-AI-CollaborationComments: Please note that this is a pre-print version of the paper. The final version may differ after peer review and editingSubjects: Human-Computer Interaction (cs.HC)
Our study explores the potential of human-AI collaboration (HAIC) through semi-structured interviews with 14 executives. We identify 63 HAIC use cases and classify them using a novel matrix combining value chain and group work activities. Most use cases identified are related to firm infrastructure and technology development, with very few pertaining to services and procurement, and none to logistics. HAIC is predominantly seen as support for choosing and executing group tasks, with an emphasis on choosing in supporting activities of the value chain. In contrast, primary activities such as operations and marketing focus more on executing group tasks. Few use cases involve negotiating tasks. Beyond identifying and classifying HAIC use cases, we discuss their potential as a tool for prospective sensemaking and to foster strategic managerial decisions.
- [302] arXiv:2408.10818 [pdf, html, other]
-
Title: Learning Randomized Algorithms with TransformersSubjects: Machine Learning (cs.LG)
Randomization is a powerful tool that endows algorithms with remarkable properties. For instance, randomized algorithms excel in adversarial settings, often surpassing the worst-case performance of deterministic algorithms with large margins. Furthermore, their success probability can be amplified by simple strategies such as repetition and majority voting. In this paper, we enhance deep neural networks, in particular transformer models, with randomization. We demonstrate for the first time that randomized algorithms can be instilled in transformers through learning, in a purely data- and objective-driven manner. First, we analyze known adversarial objectives for which randomized algorithms offer a distinct advantage over deterministic ones. We then show that common optimization techniques, such as gradient descent or evolutionary strategies, can effectively learn transformer parameters that make use of the randomness provided to the model. To illustrate the broad applicability of randomization in empowering neural networks, we study three conceptual tasks: associative recall, graph coloring, and agents that explore grid worlds. In addition to demonstrating increased robustness against oblivious adversaries through learned randomization, our experiments reveal remarkable performance improvements due to the inherently random nature of the neural networks' computation and predictions.
- [303] arXiv:2408.10819 [pdf, html, other]
-
Title: Exploiting Large Language Models Capabilities for Question Answer-Driven Knowledge Graph Completion Across Static and Temporal DomainsSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Knowledge graph completion (KGC) aims to identify missing triples in a knowledge graph (KG). This is typically achieved through tasks such as link prediction and instance completion. However, these methods often focus on either static knowledge graphs (SKGs) or temporal knowledge graphs (TKGs), addressing only within-scope triples. This paper introduces a new generative completion framework called Generative Subgraph-based KGC (GS-KGC). GS-KGC employs a question-answering format to directly generate target entities, addressing the challenge of questions having multiple possible answers. We propose a strategy that extracts subgraphs centered on entities and relationships within the KG, from which negative samples and neighborhood information are separately obtained to address the one-to-many problem. Our method generates negative samples using known facts to facilitate the discovery of new information. Furthermore, we collect and refine neighborhood path data of known entities, providing contextual information to enhance reasoning in large language models (LLMs). Our experiments evaluated the proposed method on four SKGs and two TKGs, achieving state-of-the-art Hits@1 metrics on five datasets. Analysis of the results shows that GS-KGC can discover new triples within existing KGs and generate new facts beyond the closed KG, effectively bridging the gap between closed-world and open-world KGC.
- [304] arXiv:2408.10821 [pdf, html, other]
-
Title: Constructing a High Temporal Resolution Global Lakes Dataset via Swin-Unet with Applications to Area PredictionSubjects: Computer Vision and Pattern Recognition (cs.CV)
Lakes provide a wide range of valuable ecosystem services, such as water supply, biodiversity habitats, and carbon sequestration. However, lakes are increasingly threatened by climate change and human activities. Therefore, continuous global monitoring of lake dynamics is crucial, but remains challenging on a large scale. The recently developed Global Lakes Area Database (GLAKES) has mapped over 3.4 million lakes worldwide, but it only provides data at decadal intervals, which may be insufficient to capture rapid or short-term changes.This paper introduces an expanded lake database, GLAKES-Additional, which offers biennial delineations and area measurements for 152,567 lakes globally from 1990 to 2021. We employed the Swin-Unet model, replacing traditional convolution operations, to effectively address the challenges posed by the receptive field requirements of high spatial resolution satellite imagery. The increased biennial time resolution helps to quantitatively attribute lake area changes to climatic and hydrological drivers, such as precipitation and temperature changes.For predicting lake area changes, we used a Long Short-Term Memory (LSTM) neural network and an extended time series dataset for preliminary modeling. Under climate and land use scenarios, our model achieved an RMSE of 0.317 km^2 in predicting future lake area changes.
- [305] arXiv:2408.10822 [pdf, html, other]
-
Title: Navigating Spatio-Temporal Heterogeneity: A Graph Transformer Approach for Traffic ForecastingSubjects: Machine Learning (cs.LG)
Traffic forecasting has emerged as a crucial research area in the development of smart cities. Although various neural networks with intricate architectures have been developed to address this problem, they still face two key challenges: i) Recent advancements in network designs for modeling spatio-temporal correlations are starting to see diminishing returns in performance enhancements. ii) Additionally, most models do not account for the spatio-temporal heterogeneity inherent in traffic data, i.e., traffic distribution varies significantly across different regions and traffic flow patterns fluctuate across various time slots. To tackle these challenges, we introduce the Spatio-Temporal Graph Transformer (STGormer), which effectively integrates attribute and structure information inherent in traffic data for learning spatio-temporal correlations, and a mixture-of-experts module for capturing heterogeneity along spaital and temporal axes. Specifically, we design two straightforward yet effective spatial encoding methods based on the graph structure and integrate time position encoding into the vanilla transformer to capture spatio-temporal traffic patterns. Additionally, a mixture-of-experts enhanced feedforward neural network (FNN) module adaptively assigns suitable expert layers to distinct patterns via a spatio-temporal gating network, further improving overall prediction accuracy. Experiments on five real-world datasets demonstrate that STGormer achieves state-of-the-art performance.
- [306] arXiv:2408.10823 [pdf, html, other]
-
Title: Trustworthy Compression? Impact of AI-based Codecs on Biometrics for Law EnforcementSubjects: Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
Image-based biometrics can aid law enforcement in various aspects, for example in iris, fingerprint and soft-biometric recognition. A critical precondition for recognition is the availability of sufficient biometric information in images. It is visually apparent that strong JPEG compression removes such details. However, latest AI-based image compression seemingly preserves many image details even for very strong compression factors. Yet, these perceived details are not necessarily grounded in measurements, which raises the question whether these images can still be used for biometric recognition. In this work, we investigate how AI compression impacts iris, fingerprint and soft-biometric (fabrics and tattoo) images. We also investigate the recognition performance for iris and fingerprint images after AI compression. It turns out that iris recognition can be strongly affected, while fingerprint recognition is quite robust. The loss of detail is qualitatively best seen in fabrics and tattoos images. Overall, our results show that AI-compression still permits many biometric tasks, but attention to strong compression factors in sensitive tasks is advisable.
- [307] arXiv:2408.10824 [pdf, html, other]
-
Title: Emerging clean technologies: policy-driven cost reductions, implications and perspectivesSubjects: Systems and Control (eess.SY)
Hydrogen production from water electrolysis, direct air capture (DAC), and synthetic kerosene derived from hydrogen and CO2 (`e-kerosene') are expected to play an important role in global decarbonization efforts. So far, the economics of these nascent technologies hamper their market diffusion. However, a wave of recent policy support in the United States, Europe, China, and elsewhere is anticipated to drive their commercial liftoff and bring their costs down. To this end, we evaluate the potential cost reductions driven by policy-induced scale-up of these emerging technologies through 2030 using an experience curves approach accounting for both local and global learning effects. We then analyze the consequences of projected cost declines on the competitiveness of these nascent technologies compared to conventional fossil alternatives, where applicable, and highlight some of the tradeoffs associated with their expansion. Our findings indicate that enacted policies could lead to substantial capital cost reductions for electrolyzers. Nevertheless, electrolytic hydrogen production at $1-2/kg would still require some form of policy support. Given expected costs and experience curves, it is unlikely that liquid solvent DAC (L-DAC) scale-up will bring removal costs to stated targets of $100/tCO2, though a $200/tCO2 may eventually be within reach. We also underscore the importance of tackling methane leakage for natural gas-powered L-DAC: unmitigated leaks amplify net removal costs, exacerbate the investment requirements to reach targeted costs, and cast doubt on L-DAC's role in the clean energy transition. Lastly, despite reductions in electrolysis and L-DAC costs, e-kerosene remains considerably more expensive than fossil jet fuel. The economics of e-kerosene and the resources required for production raise questions about the fuel's ultimate viability as a decarbonization tool for aviation.
- [308] arXiv:2408.10826 [pdf, html, other]
-
Title: NeuLite: Memory-Efficient Federated Learning via Elastic Progressive TrainingSubjects: Distributed, Parallel, and Cluster Computing (cs.DC)
Federated Learning (FL) emerges as a new learning paradigm that enables multiple devices to collaboratively train a shared model while preserving data privacy. However, intensive memory footprint during the training process severely bottlenecks the deployment of FL on resource-constrained devices in real-world cases. In this paper, we propose NeuLite, a framework that breaks the memory wall through elastic progressive training. Unlike traditional FL, which updates the full model during the whole training procedure, NeuLite divides the model into blocks and conducts the training process in a progressive manner. Except for the progressive training paradigm, NeuLite further features the following two key components to guide the training process: 1) curriculum mentor and 2) training harmonizer. Specifically, the Curriculum Mentor devises curriculum-aware training losses for each block, assisting them in learning the expected feature representation and mitigating the loss of valuable information. Additionally, the Training Harmonizer develops a parameter co-adaptation training paradigm to break the information isolation across blocks from both forward and backward propagation. Furthermore, it constructs output modules for each block to strengthen model parameter co-adaptation. Extensive experiments are conducted to evaluate the effectiveness of NeuLite across both simulation and hardware testbeds. The results demonstrate that NeuLite effectively reduces peak memory usage by up to 50.4%. It also enhances model performance by up to 84.2% and accelerates the training process by up to 1.9X.
- [309] arXiv:2408.10829 [pdf, html, other]
-
Title: The high resolution sampling methods for acoustic sources from multi-frequency far field patterns at sparse observation directionsSubjects: Numerical Analysis (math.NA); Mathematical Physics (math-ph); Analysis of PDEs (math.AP)
This work is dedicated to novel uniqueness results and high resolution sampling methods for source support from multi-frequency sparse far field patterns. With a single pair of observation directions $\pm\hat{x}$, we prove that the lines $\{z\in\mathbb R^2|\, \hat{x}\cdot z = \hat{x}\cdot y, \,y\in A_{\hat{x}}\}$ can be determined by multi-frequency far field patterns at the directions $\pm\hat{x}$, where $A_{\hat{x}}$ denotes a set containing the corners of the boundary and points whose normal vector to the boundary is parallel to $\hat{x}$. Furthermore, if the source support is composed of polygons and annuluses, then we prove that the support can be determined by multi-frequency far field patterns at sparse directions. Precisely, the lowest number of the observation directions is given in terms of the number of the corners and the annuluses. Inspired by the uniqueness arguments, we introduce two novel indicators to determine the source support. Numerical examples in two dimensions are presented to show the validity and robustness of the two indicators for reconstructing the boundaries of the source support with a high resolution. The second indicator also shows its powerful ability to determine the unknown source function.
- [310] arXiv:2408.10830 [pdf, html, other]
-
Title: Single Bridge Formation in Self-Organizing Particle SystemsSubjects: Distributed, Parallel, and Cluster Computing (cs.DC); Emerging Technologies (cs.ET)
Local interactions of uncoordinated individuals produce the collective behaviors of many biological systems, inspiring much of the current research in programmable matter. A striking example is the spontaneous assembly of fire ants into "bridges" comprising their own bodies to traverse obstacles and reach sources of food. Experiments and simulations suggest that, remarkably, these ants always form one bridge -- instead of multiple, competing bridges -- despite a lack of central coordination. We argue that the reliable formation of a single bridge does not require sophistication on behalf of the individuals by provably reproducing this behavior in a self-organizing particle system. We show that the formation of a single bridge by the particles is a statistical inevitability of their preferences to move in a particular direction, such as toward a food source, and their preference for more neighbors. Two parameters, $\eta$ and $\beta$, reflect the strengths of these preferences and determine the Gibbs stationary measure of the corresponding particle system's Markov chain dynamics. We show that a single bridge almost certainly forms when $\eta$ and $\beta$ are sufficiently large. Our proof introduces an auxiliary Markov chain, called an "occupancy chain", that captures only the significant, global changes to the system. Through the occupancy chain, we abstract away information about the motion of individual particles, but we gain a more direct means of analyzing their collective behavior. Such abstractions provide a promising new direction for understanding many other systems of programmable matter.
- [311] arXiv:2408.10831 [pdf, html, other]
-
Title: ZebraPose: Zebra Detection and Pose Estimation using only Synthetic DataComments: 8 pages, 5 tables, 7 figuresSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO)
Synthetic data is increasingly being used to address the lack of labeled images in uncommon domains for deep learning tasks. A prominent example is 2D pose estimation of animals, particularly wild species like zebras, for which collecting real-world data is complex and impractical. However, many approaches still require real images, consistency and style constraints, sophisticated animal models, and/or powerful pre-trained networks to bridge the syn-to-real gap. Moreover, they often assume that the animal can be reliably detected in images or videos, a hypothesis that often does not hold, e.g. in wildlife scenarios or aerial images. To solve this, we use synthetic data generated with a 3D photorealistic simulator to obtain the first synthetic dataset that can be used for both detection and 2D pose estimation of zebras without applying any of the aforementioned bridging strategies. Unlike previous works, we extensively train and benchmark our detection and 2D pose estimation models on multiple real-world and synthetic datasets using both pre-trained and non-pre-trained backbones. These experiments show how the models trained from scratch and only with synthetic data can consistently generalize to real-world images of zebras in both tasks. Moreover, we show it is possible to easily generalize those same models to 2D pose estimation of horses with a minimal amount of real-world images to account for the domain transfer. Code, results, trained models; and the synthetic, training, and validation data, including 104K manually labeled frames, are provided as open-source at this https URL
- [312] arXiv:2408.10838 [pdf, html, other]
-
Title: Multilevel CNNs for Parametric PDEs based on Adaptive Finite ElementsSubjects: Machine Learning (cs.LG); Numerical Analysis (math.NA)
A neural network architecture is presented that exploits the multilevel properties of high-dimensional parameter-dependent partial differential equations, enabling an efficient approximation of parameter-to-solution maps, rivaling best-in-class methods such as low-rank tensor regression in terms of accuracy and complexity. The neural network is trained with data on adaptively refined finite element meshes, thus reducing data complexity significantly. Error control is achieved by using a reliable finite element a posteriori error estimator, which is also provided as input to the neural network.
The proposed U-Net architecture with CNN layers mimics a classical finite element multigrid algorithm. It can be shown that the CNN efficiently approximates all operations required by the solver, including the evaluation of the residual-based error estimator. In the CNN, a culling mask set-up according to the local corrections due to refinement on each mesh level reduces the overall complexity, allowing the network optimization with localized fine-scale finite element data.
A complete convergence and complexity analysis is carried out for the adaptive multilevel scheme, which differs in several aspects from previous non-adaptive multilevel CNN. Moreover, numerical experiments with common benchmark problems from Uncertainty Quantification illustrate the practical performance of the architecture. - [313] arXiv:2408.10839 [pdf, html, other]
-
Title: Benchmarking Large Language Models for Math Reasoning TasksSubjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
The use of Large Language Models (LLMs) in mathematical reasoning has become a cornerstone of related research, demonstrating the intelligence of these models and enabling potential practical applications through their advanced performance, such as in educational settings. Despite the variety of datasets and in-context learning algorithms designed to improve the ability of LLMs to automate mathematical problem solving, the lack of comprehensive benchmarking across different datasets makes it complicated to select an appropriate model for specific tasks. In this project, we present a benchmark that fairly compares seven state-of-the-art in-context learning algorithms for mathematical problem solving across five widely used mathematical datasets on four powerful foundation models. Furthermore, we explore the trade-off between efficiency and performance, highlighting the practical applications of LLMs for mathematical reasoning. Our results indicate that larger foundation models like GPT-4o and LLaMA 3-70B can solve mathematical reasoning independently from the concrete prompting strategy, while for smaller models the in-context learning approach significantly influences the performance. Moreover, the optimal prompt depends on the chosen foundation model. We open-source our benchmark code to support the integration of additional models in future research.
- [314] arXiv:2408.10841 [pdf, html, other]
-
Title: DELIA: Diversity-Enhanced Learning for Instruction Adaptation in Large Language ModelsComments: 8 pages, 5 figuresSubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Although instruction tuning is widely used to adjust behavior in Large Language Models (LLMs), extensive empirical evidence and research indicates that it is primarily a process where the model fits to specific task formats, rather than acquiring new knowledge or capabilities. We propose that this limitation stems from biased features learned during instruction tuning, which differ from ideal task-specfic features, leading to learn less underlying semantics in downstream tasks. However, ideal features are unknown and incalculable, constraining past work to rely on prior knowledge to assist reasoning or training, which limits LLMs' capabilities to the developers' abilities, rather than data-driven scalable learning. In our paper, through our novel data synthesis method, DELIA (Diversity-Enhanced Learning for Instruction Adaptation), we leverage the buffering effect of extensive diverse data in LLMs training to transform biased features in instruction tuning into approximations of ideal features, without explicit prior ideal features. Experiments show DELIA's better performance compared to common instruction tuning and other baselines. It outperforms common instruction tuning by 17.07%-33.41% on Icelandic-English translation bleurt score (WMT-21 dataset, gemma-7b-it) and improves accuracy by 36.1% on formatted text generation (Llama2-7b-chat). Notably, among knowledge injection methods we've known, DELIA uniquely align the internal representations of new special tokens with their prior semantics.
- [315] arXiv:2408.10843 [pdf, html, other]
-
Title: Detecting Wildfires on UAVs with Real-time Segmentation Trained by Larger Teacher ModelsJulius Pesonen, Teemu Hakala, Väinö Karjalainen, Niko Koivumäki, Lauri Markelin, Anna-Maria Raita-Hakola, Juha Suomalainen, Ilkka Pölönen, Eija HonkavaaraSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Early detection of wildfires is essential to prevent large-scale fires resulting in extensive environmental, structural, and societal damage. Uncrewed aerial vehicles (UAVs) can cover large remote areas effectively with quick deployment requiring minimal infrastructure and equipping them with small cameras and computers enables autonomous real-time detection. In remote areas, however, the UAVs are limited to on-board computing for detection due to the lack of high-bandwidth mobile networks. This limits the detection to methods which are light enough for the on-board computer alone. For accurate camera-based localisation, segmentation of the detected smoke is essential but training data for deep learning-based wildfire smoke segmentation is limited. This study shows how small specialised segmentation models can be trained using only bounding box labels, leveraging zero-shot foundation model supervision. The method offers the advantages of needing only fairly easily obtainable bounding box labels and requiring training solely for the smaller student network. The proposed method achieved 63.3% mIoU on a manually annotated and diverse wildfire dataset. The used model can perform in real-time at ~11 fps with a UAV-carried NVIDIA Jetson Orin NX computer while reliably recognising smoke, demonstrated at real-world forest burning events. Code is available at this https URL
- [316] arXiv:2408.10844 [pdf, html, other]
-
Title: Aligning Object Detector Bounding Boxes with Human PreferenceComments: Accepted paper at the ECCV 2024 workshop on Assistive Computer Vision and Robotics (ACVR)Subjects: Computer Vision and Pattern Recognition (cs.CV)
Previous work shows that humans tend to prefer large bounding boxes over small bounding boxes with the same IoU. However, we show here that commonly used object detectors predict large and small boxes equally often. In this work, we investigate how to align automatically detected object boxes with human preference and study whether this improves human quality perception. We evaluate the performance of three commonly used object detectors through a user study (N = 123). We find that humans prefer object detections that are upscaled with factors of 1.5 or 2, even if the corresponding AP is close to 0. Motivated by this result, we propose an asymmetric bounding box regression loss that encourages large over small predicted bounding boxes. Our evaluation study shows that object detectors fine-tuned with the asymmetric loss are better aligned with human preference and are preferred over fixed scaling factors. A qualitative evaluation shows that human preference might be influenced by some object characteristics, like object shape.
- [317] arXiv:2408.10845 [pdf, html, other]
-
Title: CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous DrivingHidehisa Arai, Keita Miwa, Kento Sasaki, Yu Yamaguchi, Kohei Watanabe, Shunsuke Aoki, Issei YamamotoSubjects: Computer Vision and Pattern Recognition (cs.CV)
Autonomous driving, particularly navigating complex and unanticipated scenarios, demands sophisticated reasoning and planning capabilities. While Multi-modal Large Language Models (MLLMs) offer a promising avenue for this, their use has been largely confined to understanding complex environmental contexts or generating high-level driving commands, with few studies extending their application to end-to-end path planning. A major research bottleneck is the lack of large-scale annotated datasets encompassing vision, language, and action. To address this issue, we propose CoVLA (Comprehensive Vision-Language-Action) Dataset, an extensive dataset comprising real-world driving videos spanning more than 80 hours. This dataset leverages a novel, scalable approach based on automated data processing and a caption generation pipeline to generate accurate driving trajectories paired with detailed natural language descriptions of driving environments and maneuvers. This approach utilizes raw in-vehicle sensor data, allowing it to surpass existing datasets in scale and annotation richness. Using CoVLA, we investigate the driving capabilities of MLLMs that can handle vision, language, and action in a variety of driving scenarios. Our results illustrate the strong proficiency of our model in generating coherent language and action outputs, emphasizing the potential of Vision-Language-Action (VLA) models in the field of autonomous driving. This dataset establishes a framework for robust, interpretable, and data-driven autonomous driving systems by providing a comprehensive platform for training and evaluating VLA models, contributing to safer and more reliable self-driving vehicles. The dataset is released for academic purpose.
- [318] arXiv:2408.10846 [pdf, html, other]
-
Title: Harmonizing Attention: Training-free Texture-aware Geometry TransferComments: 10 pages, 6 figuresSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)
Extracting geometry features from photographic images independently of surface texture and transferring them onto different materials remains a complex challenge. In this study, we introduce Harmonizing Attention, a novel training-free approach that leverages diffusion models for texture-aware geometry transfer. Our method employs a simple yet effective modification of self-attention layers, allowing the model to query information from multiple reference images within these layers. This mechanism is seamlessly integrated into the inversion process as Texture-aligning Attention and into the generation process as Geometry-aligning Attention. This dual-attention approach ensures the effective capture and transfer of material-independent geometry features while maintaining material-specific textural continuity, all without the need for model fine-tuning.
- [319] arXiv:2408.10848 [pdf, html, other]
-
Title: Perception-guided Jailbreak against Text-to-Image ModelsComments: 8 pagesSubjects: Computer Vision and Pattern Recognition (cs.CV)
In recent years, Text-to-Image (T2I) models have garnered significant attention due to their remarkable advancements. However, security concerns have emerged due to their potential to generate inappropriate or Not-Safe-For-Work (NSFW) images. In this paper, inspired by the observation that texts with different semantics can lead to similar human perceptions, we propose an LLM-driven perception-guided jailbreak method, termed PGJ. It is a black-box jailbreak method that requires no specific T2I model (model-free) and generates highly natural attack prompts. Specifically, we propose identifying a safe phrase that is similar in human perception yet inconsistent in text semantics with the target unsafe word and using it as a substitution. The experiments conducted on six open-source models and commercial online services with thousands of prompts have verified the effectiveness of PGJ.
- [320] arXiv:2408.10849 [pdf, html, other]
-
Title: A Noval Feature via Color Quantisation for Fake Audio DetectionZhiyong Wang, Xiaopeng Wang, Yuankun Xie, Ruibo Fu, Zhengqi Wen, Jianhua Tao, Yukun Liu, Guanjun Li, Xin Qi, Yi Lu, Xuefei Liu, Yongwei LiComments: accepted by ISCSLP2024Subjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)
In the field of deepfake detection, previous studies focus on using reconstruction or mask and prediction methods to train pre-trained models, which are then transferred to fake audio detection training where the encoder is used to extract features, such as wav2vec2.0 and Masked Auto Encoder. These methods have proven that using real audio for reconstruction pre-training can better help the model distinguish fake audio. However, the disadvantage lies in poor interpretability, meaning it is hard to intuitively present the differences between deepfake and real audio. This paper proposes a noval feature extraction method via color quantisation which constrains the reconstruction to use a limited number of colors for the spectral image-like input. The proposed method ensures reconstructed input differs from the original, which allows for intuitive observation of the focus areas in the spectral reconstruction. Experiments conducted on the ASVspoof2019 dataset demonstrate that the proposed method achieves better classification performance compared to using the original spectral as input and pretraining the recolor network can also benefit the fake audio detection.
- [321] arXiv:2408.10850 [pdf, other]
-
Title: Hardware Implementation of Projection-Aggregation Decoders for Reed-Muller CodesSubjects: Hardware Architecture (cs.AR); Information Theory (cs.IT); Signal Processing (eess.SP)
This paper presents the hardware implementation of two variants of projection-aggregation-based decoding of Reed-Muller (RM) codes, namely unique projection aggregation (UPA) and collapsed projection aggregation (CPA). Our study focuses on introducing hardware architectures for both UPA and CPA. Through thorough analysis and experimentation, we observe that the hardware implementation of UPA exhibits superior resource usage and reduced energy consumption compared to CPA for the vanilla IPA decoder. This finding underscores a critical insight: software optimizations, in isolation, may not necessarily translate into hardware cost-effectiveness.
- [322] arXiv:2408.10852 [pdf, html, other]
-
Title: EELE: Exploring Efficient and Extensible LoRA Integration in Emotional Text-to-SpeechXin Qi, Ruibo Fu, Zhengqi Wen, Jianhua Tao, Shuchen Shi, Yi Lu, Zhiyong Wang, Xiaopeng Wang, Yuankun Xie, Yukun Liu, Guanjun Li, Xuefei Liu, Yongwei LiSubjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)
In the current era of Artificial Intelligence Generated Content (AIGC), a Low-Rank Adaptation (LoRA) method has emerged. It uses a plugin-based approach to learn new knowledge with lower parameter quantities and computational costs, and it can be plugged in and out based on the specific sub-tasks, offering high flexibility. However, the current application schemes primarily incorporate LoRA into the pre-introduced conditional parts of the speech models. This fixes the position of LoRA, limiting the flexibility and scalability of its application. Therefore, we propose the Exploring Efficient and Extensible LoRA Integration in Emotional Text-to-Speech (EELE) method. Starting from a general neutral speech model, we do not pre-introduce emotional information but instead use the LoRA plugin to design a flexible adaptive scheme that endows the model with emotional generation capabilities. Specifically, we initially train the model using only neutral speech data. After training is complete, we insert LoRA into different modules and fine-tune the model with emotional speech data to find the optimal insertion scheme. Through experiments, we compare and test the effects of inserting LoRA at different positions within the model and assess LoRA's ability to learn various emotions, effectively proving the validity of our method. Additionally, we explore the impact of the rank size of LoRA and the difference compared to directly fine-tuning the entire model.
- [323] arXiv:2408.10853 [pdf, html, other]
-
Title: Does Current Deepfake Audio Detection Model Effectively Detect ALM-based Deepfake Audio?Yuankun Xie, Chenxu Xiong, Xiaopeng Wang, Zhiyong Wang, Yi Lu, Xin Qi, Ruibo Fu, Yukun Liu, Zhengqi Wen, Jianhua Tao, Guanjun Li, Long YeSubjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
Currently, Audio Language Models (ALMs) are rapidly advancing due to the developments in large language models and audio neural codecs. These ALMs have significantly lowered the barrier to creating deepfake audio, generating highly realistic and diverse types of deepfake audio, which pose severe threats to society. Consequently, effective audio deepfake detection technologies to detect ALM-based audio have become increasingly critical. This paper investigate the effectiveness of current countermeasure (CM) against ALM-based audio. Specifically, we collect 12 types of the latest ALM-based deepfake audio and utilizing the latest CMs to evaluate. Our findings reveal that the latest codec-trained CM can effectively detect ALM-based audio, achieving 0% equal error rate under most ALM test conditions, which exceeded our expectations. This indicates promising directions for future research in ALM-based deepfake audio detection.
- [324] arXiv:2408.10858 [pdf, html, other]
-
Title: Knowledge Sharing and Transfer via Centralized Reward Agent for Multi-Task Reinforcement LearningSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Reward shaping is effective in addressing the sparse-reward challenge in reinforcement learning by providing immediate feedback through auxiliary informative rewards. Based on the reward shaping strategy, we propose a novel multi-task reinforcement learning framework, that integrates a centralized reward agent (CRA) and multiple distributed policy agents. The CRA functions as a knowledge pool, which aims to distill knowledge from various tasks and distribute it to individual policy agents to improve learning efficiency. Specifically, the shaped rewards serve as a straightforward metric to encode knowledge. This framework not only enhances knowledge sharing across established tasks but also adapts to new tasks by transferring valuable reward signals. We validate the proposed method on both discrete and continuous domains, demonstrating its robustness in multi-task sparse-reward settings and its effective transferability to unseen tasks.
- [325] arXiv:2408.10861 [pdf, other]
-
Title: DVRP-MHSI: Dynamic Visualization Research Platform for Multimodal Human-Swarm InteractionSubjects: Robotics (cs.RO); Human-Computer Interaction (cs.HC)
In recent years, there has been a significant amount of research on algorithms and control methods for distributed collaborative robots. However, the emergence of collective behavior in a swarm is still difficult to predict and control. Nevertheless, human interaction with the swarm helps render the swarm more predictable and controllable, as human operators can utilize intuition or knowledge that is not always available to the swarm. Therefore, this paper designs the Dynamic Visualization Research Platform for Multimodal Human-Swarm Interaction (DVRP-MHSI), which is an innovative open system that can perform real-time dynamic visualization and is specifically designed to accommodate a multitude of interaction modalities (such as brain-computer, eye-tracking, electromyographic, and touch-based interfaces), thereby expediting progress in human-swarm interaction research. Specifically, the platform consists of custom-made low-cost omnidirectional wheeled mobile robots, multitouch screens and two workstations. In particular, the mutitouch screens can recognize human gestures and the shapes of objects placed on them, and they can also dynamically render diverse scenes. One of the workstations processes communication information within robots and the other one implements human-robot interaction methods. The development of DVRP-MHSI frees researchers from hardware or software details and allows them to focus on versatile swarm algorithms and human-swarm interaction methods without being limited to fixed scenarios, tasks, and interfaces. The effectiveness and potential of the platform for human-swarm interaction studies are validated by several demonstrative experiments.
- [326] arXiv:2408.10862 [pdf, html, other]
-
Title: Feature Selection from Differentially Private CorrelationsComments: To appear in Proceedings of the 17th ACM Workshop on Artificial Intelligence and Security, 2024Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Data scientists often seek to identify the most important features in high-dimensional datasets. This can be done through $L_1$-regularized regression, but this can become inefficient for very high-dimensional datasets. Additionally, high-dimensional regression can leak information about individual datapoints in a dataset. In this paper, we empirically evaluate the established baseline method for feature selection with differential privacy, the two-stage selection technique, and show that it is not stable under sparsity. This makes it perform poorly on real-world datasets, so we consider a different approach to private feature selection. We employ a correlations-based order statistic to choose important features from a dataset and privatize them to ensure that the results do not leak information about individual datapoints. We find that our method significantly outperforms the established baseline for private feature selection on many datasets.
- [327] arXiv:2408.10864 [pdf, html, other]
-
Title: Rage Music Classification and Analysis using K-Nearest Neighbour, Random Forest, Support Vector Machine, Convolutional Neural Networks, and Gradient BoostingSubjects: Sound (cs.SD); Audio and Speech Processing (eess.AS)
We classify rage music (a subgenre of rap well-known for disagreements on whether a particular song is part of the genre) with an extensive feature set through algorithms including Random Forest, Support Vector Machine, K-nearest Neighbour, Gradient Boosting, and Convolutional Neural Networks. We compare methods of classification in the application of audio analysis with machine learning and identify optimal models. We then analyze the significant audio features present in and most effective in categorizing rage music, while also identifying key audio features as well as broader separating sonic variations and trends.
- [328] arXiv:2408.10865 [pdf, html, other]
-
Title: Multi-agent Multi-armed Bandits with Stochastic Sharable Arm CapacitiesComments: 28 pagesSubjects: Artificial Intelligence (cs.AI)
Motivated by distributed selection problems, we formulate a new variant of multi-player multi-armed bandit (MAB) model, which captures stochastic arrival of requests to each arm, as well as the policy of allocating requests to players. The challenge is how to design a distributed learning algorithm such that players select arms according to the optimal arm pulling profile (an arm pulling profile prescribes the number of players at each arm) without communicating to each other. We first design a greedy algorithm, which locates one of the optimal arm pulling profiles with a polynomial computational complexity. We also design an iterative distributed algorithm for players to commit to an optimal arm pulling profile with a constant number of rounds in expectation. We apply the explore then commit (ETC) framework to address the online setting when model parameters are unknown. We design an exploration strategy for players to estimate the optimal arm pulling profile. Since such estimates can be different across different players, it is challenging for players to commit. We then design an iterative distributed algorithm, which guarantees that players can arrive at a consensus on the optimal arm pulling profile in only M rounds. We conduct experiments to validate our algorithm.
- [329] arXiv:2408.10872 [pdf, html, other]
-
Title: V-RoAst: A New Dataset for Visual Road AssessmentNatchapon Jongwiriyanurak, Zichao Zeng, June Moh Goo, Xinglei Wang, Ilya Ilyankou, Kerkritt Srirrongvikrai, Meihui Wang, James HaworthSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET)
Road traffic crashes cause millions of deaths annually and have a significant economic impact, particularly in low- and middle-income countries (LMICs). This paper presents an approach using Vision Language Models (VLMs) for road safety assessment, overcoming the limitations of traditional Convolutional Neural Networks (CNNs). We introduce a new task ,V-RoAst (Visual question answering for Road Assessment), with a real-world dataset. Our approach optimizes prompt engineering and evaluates advanced VLMs, including Gemini-1.5-flash and GPT-4o-mini. The models effectively examine attributes for road assessment. Using crowdsourced imagery from Mapillary, our scalable solution influentially estimates road safety levels. In addition, this approach is designed for local stakeholders who lack resources, as it does not require training data. It offers a cost-effective and automated methods for global road safety assessments, potentially saving lives and reducing economic burdens.
- [330] arXiv:2408.10878 [pdf, html, other]
-
Title: DBHP: Trajectory Imputation in Multi-Agent Sports Using Derivative-Based Hybrid PredictionSubjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multiagent Systems (cs.MA)
Many spatiotemporal domains handle multi-agent trajectory data, but in real-world scenarios, collected trajectory data are often partially missing due to various reasons. While existing approaches demonstrate good performance in trajectory imputation, they face challenges in capturing the complex dynamics and interactions between agents due to a lack of physical constraints that govern realistic trajectories, leading to suboptimal results. To address this issue, the paper proposes a Derivative-Based Hybrid Prediction (DBHP) framework that can effectively impute multiple agents' missing trajectories. First, a neural network equipped with Set Transformers produces a naive prediction of missing trajectories while satisfying the permutation-equivariance in terms of the order of input agents. Then, the framework makes alternative predictions leveraging velocity and acceleration information and combines all the predictions with properly determined weights to provide final imputed trajectories. In this way, our proposed framework not only accurately predicts position, velocity, and acceleration values but also enforces the physical relationship between them, eventually improving both the accuracy and naturalness of the predicted trajectories. Accordingly, the experiment results about imputing player trajectories in team sports show that our framework significantly outperforms existing imputation baselines.
- [331] arXiv:2408.10880 [pdf, html, other]
-
Title: Open 3D World in Autonomous DrivingSubjects: Computer Vision and Pattern Recognition (cs.CV)
The capability for open vocabulary perception represents a significant advancement in autonomous driving systems, facilitating the comprehension and interpretation of a wide array of textual inputs in real-time. Despite extensive research in open vocabulary tasks within 2D computer vision, the application of such methodologies to 3D environments, particularly within large-scale outdoor contexts, remains relatively underdeveloped. This paper presents a novel approach that integrates 3D point cloud data, acquired from LIDAR sensors, with textual information. The primary focus is on the utilization of textual data to directly localize and identify objects within the autonomous driving context. We introduce an efficient framework for the fusion of bird's-eye view (BEV) region features with textual features, thereby enabling the system to seamlessly adapt to novel textual inputs and enhancing the robustness of open vocabulary detection tasks. The effectiveness of the proposed methodology is rigorously evaluated through extensive experimentation on the newly introduced NuScenes-T dataset, with additional validation of its zero-shot performance on the Lyft Level 5 dataset. This research makes a substantive contribution to the advancement of autonomous driving technologies by leveraging multimodal data to enhance open vocabulary perception in 3D environments, thereby pushing the boundaries of what is achievable in autonomous navigation and perception.
- [332] arXiv:2408.10883 [pdf, html, other]
-
Title: DAAD: Dynamic Analysis and Adaptive Discriminator for Fake News DetectionSubjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
In current web environment, fake news spreads rapidly across online social networks, posing serious threats to society. Existing multimodal fake news detection (MFND) methods can be classified into knowledge-based and semantic-based approaches. However, these methods are overly dependent on human expertise and feedback, lacking flexibility. To address this challenge, we propose a Dynamic Analysis and Adaptive Discriminator (DAAD) approach for fake news detection. For knowledge-based methods, we introduce the Monte Carlo Tree Search (MCTS) algorithm to leverage the self-reflective capabilities of large language models (LLMs) for prompt optimization, providing richer, domain-specific details and guidance to the LLMs, while enabling more flexible integration of LLM comment on news content. For semantic-based methods, we define four typical deceit patterns: emotional exaggeration, logical inconsistency, image manipulation, and semantic inconsistency, to reveal the mechanisms behind fake news creation. To detect these patterns, we carefully design four discriminators and expand them in depth and breadth, using the soft-routing mechanism to explore optimal detection models. Experimental results on three real-world datasets demonstrate the superiority of our approach. The code will be available at: this https URL.
- [333] arXiv:2408.10885 [pdf, html, other]
-
Title: Low-Quality Image Detection by Hierarchical VAEComments: ICCV 2023, Workshop on Uncertainty Estimation for Computer VisionSubjects: Computer Vision and Pattern Recognition (cs.CV)
To make an employee roster, photo album, or training dataset of generative models, one needs to collect high-quality images while dismissing low-quality ones. This study addresses a new task of unsupervised detection of low-quality images. We propose a method that not only detects low-quality images with various types of degradation but also provides visual clues of them based on an observation that partial reconstruction by hierarchical variational autoencoders fails for low-quality images. The experiments show that our method outperforms several unsupervised out-of-distribution detection methods and also gives visual clues for low-quality images that help humans recognize them even in thumbnail view.
- [334] arXiv:2408.10886 [pdf, html, other]
-
Title: Leveraging LLMs for the Quality Assurance of Software RequirementsSebastian Lubos, Alexander Felfernig, Thi Ngoc Trang Tran, Damian Garber, Merfat El Mansi, Seda Polat Erdeniz, Viet-Man LeComments: Accepted for publication at the RE@Next! track of RE 2024Subjects: Software Engineering (cs.SE)
Successful software projects depend on the quality of software requirements. Creating high-quality requirements is a crucial step toward successful software development. Effective support in this area can significantly reduce development costs and enhance the software quality. In this paper, we introduce and assess the capabilities of a Large Language Model (LLM) to evaluate the quality characteristics of software requirements according to the ISO 29148 standard. We aim to further improve the support of stakeholders engaged in requirements engineering (RE). We show how an LLM can assess requirements, explain its decision-making process, and examine its capacity to propose improved versions of requirements. We conduct a study with software engineers to validate our approach. Our findings emphasize the potential of LLMs for improving the quality of software requirements.
- [335] arXiv:2408.10887 [pdf, html, other]
-
Title: A Mini-Review on Mobile Manipulators with Variable AutonomyComments: Presented at Variable Autonomy for Human-Robot Teaming (VAT) at IEEE RO-MAN 2024 WorkshopSubjects: Robotics (cs.RO)
This paper presents a mini-review of the current state of research in mobile manipulators with variable levels of autonomy, emphasizing their associated challenges and application environments. The need for mobile manipulators in different environments is evident due to the unique challenges and risks each presents. Many systems deployed in these environments are not fully autonomous, requiring human-robot teaming to ensure safe and reliable operations under uncertainties. Through this analysis, we identify gaps and challenges in the literature on Variable Autonomy, including cognitive workload and communication delays, and propose future directions, including whole-body Variable Autonomy for mobile manipulators, virtual reality frameworks, and large language models to reduce operators' complexity and cognitive load in some challenging and uncertain scenarios.
- [336] arXiv:2408.10889 [pdf, html, other]
-
Title: On Learning Action Costs from Input PlansMarianela Morales, Alberto Pozanco, Giuseppe Canonaco, Sriram Gopalakrishnan, Daniel Borrajo, Manuela VelosoSubjects: Artificial Intelligence (cs.AI)
Most of the work on learning action models focus on learning the actions' dynamics from input plans. This allows us to specify the valid plans of a planning task. However, very little work focuses on learning action costs, which in turn allows us to rank the different plans. In this paper we introduce a new problem: that of learning the costs of a set of actions such that a set of input plans are optimal under the resulting planning model. To solve this problem we present $LACFIP^k$, an algorithm to learn action's costs from unlabeled input plans. We provide theoretical and empirical results showing how $LACFIP^k$ can successfully solve this task.
- [337] arXiv:2408.10894 [pdf, html, other]
-
Title: ViLReF: A Chinese Vision-Language Retinal Foundation ModelSubjects: Computer Vision and Pattern Recognition (cs.CV)
Subtle semantic differences in retinal image and text data present great challenges for pre-training visual-language models. Moreover, false negative samples, i.e., image-text pairs having the same semantics but incorrectly regarded as negatives, disrupt the visual-language pre-training process and affect the model's learning ability. This work aims to develop a retinal foundation model, called ViLReF, by pre-training on a paired dataset comprising 451,956 retinal images and corresponding diagnostic text reports. In our vision-language pre-training strategy, we leverage expert knowledge to facilitate the extraction of labels and propose a novel constraint, the Weighted Similarity Coupling Loss, to adjust the speed of pushing sample pairs further apart dynamically within the feature space. Furthermore, we employ a batch expansion module with dynamic memory queues, maintained by momentum encoders, to supply extra samples and compensate for the vacancies caused by eliminating false negatives. Extensive experiments are conducted on multiple datasets for downstream classification and segmentation tasks. The experimental results demonstrate the powerful zero-shot and transfer learning capabilities of ViLReF, verifying the effectiveness of our pre-training strategy. Our ViLReF model is available at: this https URL.
- [338] arXiv:2408.10895 [pdf, html, other]
-
Title: Analytical and Empirical Study of Herding Effects in Recommendation SystemsComments: 29 pagesSubjects: Artificial Intelligence (cs.AI)
Online rating systems are often used in numerous web or mobile applications, e.g., Amazon and TripAdvisor, to assess the ground-truth quality of products. Due to herding effects, the aggregation of historical ratings (or historical collective opinion) can significantly influence subsequent ratings, leading to misleading and erroneous assessments. We study how to manage product ratings via rating aggregation rules and shortlisted representative reviews, for the purpose of correcting the assessment error. We first develop a mathematical model to characterize important factors of herding effects in product ratings. We then identify sufficient conditions (via the stochastic approximation theory), under which the historical collective opinion converges to the ground-truth collective opinion of the whole user population. These conditions identify a class of rating aggregation rules and review selection mechanisms that can reveal the ground-truth product quality. We also quantify the speed of convergence (via the martingale theory), which reflects the efficiency of rating aggregation rules and review selection mechanisms. We prove that the herding effects slow down the speed of convergence while an accurate review selection mechanism can speed it up. We also study the speed of convergence numerically and reveal trade-offs in selecting rating aggregation rules and review selection mechanisms. To show the utility of our framework, we design a maximum likelihood algorithm to infer model parameters from ratings, and conduct experiments on rating datasets from Amazon and TripAdvisor. We show that proper recency aware rating aggregation rules can improve the speed of convergence in Amazon and TripAdvisor by 41% and 62% respectively.
- [339] arXiv:2408.10899 [pdf, html, other]
-
Title: All Robots in One: A New Standard and Unified Dataset for Versatile, General-Purpose Embodied AgentsZhiqiang Wang, Hao Zheng, Yunshuang Nie, Wenjun Xu, Qingwei Wang, Hua Ye, Zhe Li, Kaidong Zhang, Xuewen Cheng, Wanxi Dong, Chang Cai, Liang Lin, Feng Zheng, Xiaodan LiangComments: Project website: this https URLSubjects: Robotics (cs.RO)
Embodied AI is transforming how AI systems interact with the physical world, yet existing datasets are inadequate for developing versatile, general-purpose agents. These limitations include a lack of standardized formats, insufficient data diversity, and inadequate data volume. To address these issues, we introduce ARIO (All Robots In One), a new data standard that enhances existing datasets by offering a unified data format, comprehensive sensory modalities, and a combination of real-world and simulated data. ARIO aims to improve the training of embodied AI agents, increasing their robustness and adaptability across various tasks and environments. Building upon the proposed new standard, we present a large-scale unified ARIO dataset, comprising approximately 3 million episodes collected from 258 series and 321,064 tasks. The ARIO standard and dataset represent a significant step towards bridging the gaps of existing data resources. By providing a cohesive framework for data collection and representation, ARIO paves the way for the development of more powerful and versatile embodied AI agents, capable of navigating and interacting with the physical world in increasingly complex and diverse ways. The project is available on this https URL
- [340] arXiv:2408.10900 [pdf, html, other]
-
Title: Towards Efficient Formal Verification of Spiking Neural NetworkSubjects: Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Neural and Evolutionary Computing (cs.NE)
Recently, AI research has primarily focused on large language models (LLMs), and increasing accuracy often involves scaling up and consuming more power. The power consumption of AI has become a significant societal issue; in this context, spiking neural networks (SNNs) offer a promising solution. SNNs operate event-driven, like the human brain, and compress information temporally. These characteristics allow SNNs to significantly reduce power consumption compared to perceptron-based artificial neural networks (ANNs), highlighting them as a next-generation neural network technology. However, societal concerns regarding AI go beyond power consumption, with the reliability of AI models being a global issue. For instance, adversarial attacks on AI models are a well-studied problem in the context of traditional neural networks. Despite their importance, the stability and property verification of SNNs remains in the early stages of research. Most SNN verification methods are time-consuming and barely scalable, making practical applications challenging. In this paper, we introduce temporal encoding to achieve practical performance in verifying the adversarial robustness of SNNs. We conduct a theoretical analysis of this approach and demonstrate its success in verifying SNNs at previously unmanageable scales. Our contribution advances SNN verification to a practical level, facilitating the safer application of SNNs.
- [341] arXiv:2408.10901 [pdf, html, other]
-
Title: A Grey-box Attack against Latent Diffusion Model-based Image Editing by Posterior CollapseZhongliang Guo, Lei Fang, Jingyu Lin, Yifei Qian, Shuai Zhao, Zeyu Wang, Junhao Dong, Cunjian Chen, Ognjen Arandjelović, Chun Pong LauComments: 21 pages, 7 figures, 10 tablesSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Recent advancements in generative AI, particularly Latent Diffusion Models (LDMs), have revolutionized image synthesis and manipulation. However, these generative techniques raises concerns about data misappropriation and intellectual property infringement. Adversarial attacks on machine learning models have been extensively studied, and a well-established body of research has extended these techniques as a benign metric to prevent the underlying misuse of generative AI. Current approaches to safeguarding images from manipulation by LDMs are limited by their reliance on model-specific knowledge and their inability to significantly degrade semantic quality of generated images. In response to these shortcomings, we propose the Posterior Collapse Attack (PCA) based on the observation that VAEs suffer from posterior collapse during training. Our method minimizes dependence on the white-box information of target models to get rid of the implicit reliance on model-specific knowledge. By accessing merely a small amount of LDM parameters, in specific merely the VAE encoder of LDMs, our method causes a substantial semantic collapse in generation quality, particularly in perceptual consistency, and demonstrates strong transferability across various model architectures. Experimental results show that PCA achieves superior perturbation effects on image generation of LDMs with lower runtime and VRAM. Our method outperforms existing techniques, offering a more robust and generalizable solution that is helpful in alleviating the socio-technical challenges posed by the rapidly evolving landscape of generative AI.
- [342] arXiv:2408.10902 [pdf, html, other]
-
Title: Soda-Eval: Open-Domain Dialogue Evaluation in the age of LLMsComments: 22 pages, 10 figuresSubjects: Computation and Language (cs.CL)
Although human evaluation remains the gold standard for open-domain dialogue evaluation, the growing popularity of automated evaluation using Large Language Models (LLMs) has also extended to dialogue. However, most frameworks leverage benchmarks that assess older chatbots on aspects such as fluency and relevance, which are not reflective of the challenges associated with contemporary models. In fact, a qualitative analysis on Soda, a GPT-3.5 generated dialogue dataset, suggests that current chatbots may exhibit several recurring issues related to coherence and commonsense knowledge, but generally produce highly fluent and relevant responses.
Noting the aforementioned limitations, this paper introduces Soda-Eval, an annotated dataset based on Soda that covers over 120K turn-level assessments across 10K dialogues, where the annotations were generated by GPT-4. Using Soda-Eval as a benchmark, we then study the performance of several open-access instruction-tuned LLMs, finding that dialogue evaluation remains challenging. Fine-tuning these models improves performance over few-shot inferences, both in terms of correlation and explanation. - [343] arXiv:2408.10903 [pdf, html, other]
-
Title: BEYOND DIALOGUE: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language ModelSubjects: Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)
The rapid advancement of large language models (LLMs) has revolutionized role-playing, enabling the development of general role-playing models. However, current role-playing training has two significant issues: (I) Using a predefined role profile to prompt dialogue training for specific scenarios usually leads to inconsistencies and even conflicts between the dialogue and the profile, resulting in training biases. (II) The model learns to imitate the role based solely on the profile, neglecting profile-dialogue alignment at the sentence level. In this work, we propose a simple yet effective framework called BEYOND DIALOGUE, designed to overcome these hurdles. This framework innovatively introduces "beyond dialogue" tasks to align dialogue with profile traits based on each specific scenario, thereby eliminating biases during training. Furthermore, by adopting an innovative prompting mechanism that generates reasoning outcomes for training, the framework allows the model to achieve fine-grained alignment between profile and dialogue at the sentence level. The aforementioned methods are fully automated and low-cost. Additionally, the integration of automated dialogue and objective evaluation methods forms a comprehensive framework, paving the way for general role-playing. Experimental results demonstrate that our model excels in adhering to and reflecting various dimensions of role profiles, outperforming most proprietary general and specialized role-playing baselines. All code and datasets are available at this https URL.
- [344] arXiv:2408.10905 [pdf, html, other]
-
Title: The impact of labeling automotive AI as "trustworthy" or "reliable" on user evaluation and technology acceptanceComments: 36 pages, 12 figuresSubjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET)
This study explores whether labeling AI as "trustworthy" or "reliable" influences user perceptions and acceptance of automotive AI technologies. Using a one-way between-subjects design, the research involved 478 online participants who were presented with guidelines for either trustworthy or reliable AI. Participants then evaluated three vignette scenarios and completed a modified version of the Technology Acceptance Model, which included variables such as perceived ease of use, human-like trust, and overall attitude. Although labeling AI as "trustworthy" did not significantly influence judgments on specific scenarios, it increased perceived ease of use and human-like trust, particularly benevolence. This suggests a positive impact on usability and an anthropomorphic effect on user perceptions. The study provides insights into how specific labels can influence attitudes toward AI technology.
- [345] arXiv:2408.10906 [pdf, other]
-
Title: ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised PretrainingSubjects: Computer Vision and Pattern Recognition (cs.CV)
3D Gaussian Splatting (3DGS) has become the de facto method of 3D representation in many vision tasks. This calls for the 3D understanding directly in this representation space. To facilitate the research in this direction, we first build a large-scale dataset of 3DGS using the commonly used ShapeNet and ModelNet datasets. Our dataset ShapeSplat consists of 65K objects from 87 unique categories, whose labels are in accordance with the respective datasets. The creation of this dataset utilized the compute equivalent of 2 GPU years on a TITAN XP GPU.
We utilize our dataset for unsupervised pretraining and supervised finetuning for classification and segmentation tasks. To this end, we introduce \textbf{\textit{Gaussian-MAE}}, which highlights the unique benefits of representation learning from Gaussian parameters. Through exhaustive experiments, we provide several valuable insights. In particular, we show that (1) the distribution of the optimized GS centroids significantly differs from the uniformly sampled point cloud (used for initialization) counterpart; (2) this change in distribution results in degradation in classification but improvement in segmentation tasks when using only the centroids; (3) to leverage additional Gaussian parameters, we propose Gaussian feature grouping in a normalized feature space, along with splats pooling layer, offering a tailored solution to effectively group and embed similar Gaussians, which leads to notable improvement in finetuning tasks. - [346] arXiv:2408.10908 [pdf, html, other]
-
Title: Enhancing End-to-End Autonomous Driving Systems Through Synchronized Human Behavior DataSubjects: Robotics (cs.RO); Human-Computer Interaction (cs.HC)
This paper presents a pioneering exploration into the integration of fine-grained human supervision within the autonomous driving domain to enhance system performance. The current advances in End-to-End autonomous driving normally are data-driven and rely on given expert trials. However, this reliance limits the systems' generalizability and their ability to earn human trust. Addressing this gap, our research introduces a novel approach by synchronously collecting data from human and machine drivers under identical driving scenarios, focusing on eye-tracking and brainwave data to guide machine perception and decision-making processes. This paper utilizes the Carla simulation to evaluate the impact brought by human behavior guidance. Experimental results show that using human attention to guide machine attention could bring a significant improvement in driving performance. However, guidance by human intention still remains a challenge. This paper pioneers a promising direction and potential for utilizing human behavior guidance to enhance autonomous systems.
- [347] arXiv:2408.10912 [pdf, html, other]
-
Title: An Achievable Rate-Distortion Region of Joint Identification and Sensing for Multiple Access ChannelsSubjects: Information Theory (cs.IT)
In contrast to Shannon transmission codes, the size of identification (ID) codes for discrete memoryless channels (DMCs) experiences doubly exponential growth with the block length when randomized encoding is used. Additional enhancements within the ID paradigm can be realized through supplementary resources such as quantum entanglement, common randomness (CR), and feedback. Joint transmission and sensing demonstrate significant benefits over separation-based methods. Inspired by the significant impact of feedback on the ID capacity, our work delves into the realm of joint ID and sensing (JIDAS) for state-dependent multiple access channels (SD-MACs) with noiseless strictly casual feedback. Here, the senders aim to convey ID messages to the receiver while simultaneously sensing the channel states. We establish a lower bound on the capacity-distortion region of the SD-MACs. An example shows that JIDAS outperforms the separation-based approach.
- [348] arXiv:2408.10913 [pdf, html, other]
-
Title: How Much Reserve Fuel: Quantifying the Maximal Energy Cost of System DisturbancesComments: 6 pages, 4 figures. IEEE Conference on Decision and ControlSubjects: Systems and Control (eess.SY); Optimization and Control (math.OC)
Motivated by the design question of additional fuel needed to complete a task in an uncertain environment, this paper introduces metrics to quantify the maximal additional energy used by a control system in the presence of bounded disturbances when compared to a nominal, disturbance-free system. In particular, we consider the task of finite-time stabilization for a linear time-invariant system. We first derive the nominal energy required to achieve this task in a disturbance-free system, and then the worst-case energy over all feasible disturbances. The latter leads to an optimal control problem with a least-squares solution, and then an infinite-dimensional optimization problem where we derive an upper bound on the solution. The comparison of these energies is accomplished using additive and multiplicative metrics, and we derive analytical bounds on these metrics. Simulation examples on an ADMIRE fighter jet model demonstrate the practicability of these metrics, and their variation with the task hardness, a combination of the distance of the initial condition from the origin and the task completion time.
- [349] arXiv:2408.10914 [pdf, html, other]
-
Title: To Code, or Not To Code? Exploring Impact of Code in Pre-trainingViraat Aryabumi, Yixuan Su, Raymond Ma, Adrien Morisot, Ivan Zhang, Acyr Locatelli, Marzieh Fadaee, Ahmet Üstün, Sara HookerSubjects: Computation and Language (cs.CL)
Including code in the pre-training data mixture, even for models not specifically designed for code, has become a common practice in LLMs pre-training. While there has been anecdotal consensus among practitioners that code data plays a vital role in general LLMs' performance, there is only limited work analyzing the precise impact of code on non-code tasks. In this work, we systematically investigate the impact of code data on general performance. We ask "what is the impact of code data used in pre-training on a large variety of downstream tasks beyond code generation". We conduct extensive ablations and evaluate across a broad range of natural language reasoning tasks, world knowledge tasks, code benchmarks, and LLM-as-a-judge win-rates for models with sizes ranging from 470M to 2.8B parameters. Across settings, we find a consistent results that code is a critical building block for generalization far beyond coding tasks and improvements to code quality have an outsized impact across all tasks. In particular, compared to text-only pre-training, the addition of code results in up to relative increase of 8.2% in natural language (NL) reasoning, 4.2% in world knowledge, 6.6% improvement in generative win-rates, and a 12x boost in code performance respectively. Our work suggests investments in code quality and preserving code during pre-training have positive impacts.
- [350] arXiv:2408.10918 [pdf, html, other]
-
Title: CHECKWHY: Causal Fact Verification via Argument StructureComments: Accepted by ACL2024; Awarded as Outstanding Paper Award and Area Chair AwardSubjects: Computation and Language (cs.CL)
With the growing complexity of fact verification tasks, the concern with "thoughtful" reasoning capabilities is increasing. However, recent fact verification benchmarks mainly focus on checking a narrow scope of semantic factoids within claims and lack an explicit logical reasoning process. In this paper, we introduce CheckWhy, a challenging dataset tailored to a novel causal fact verification task: checking the truthfulness of the causal relation within claims through rigorous reasoning steps. CheckWhy consists of over 19K "why" claim-evidence-argument structure triplets with supports, refutes, and not enough info labels. Each argument structure is composed of connected evidence, representing the reasoning process that begins with foundational evidence and progresses toward claim establishment. Through extensive experiments on state-of-the-art models, we validate the importance of incorporating the argument structure for causal fact verification. Moreover, the automated and human evaluation of argument structure generation reveals the difficulty in producing satisfying argument structure by fine-tuned models or Chain-of-Thought prompted LLMs, leaving considerable room for future improvements.
- [351] arXiv:2408.10919 [pdf, html, other]
-
Title: CrossFi: A Cross Domain Wi-Fi Sensing Framework Based on Siamese NetworkSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Signal Processing (eess.SP)
In recent years, Wi-Fi sensing has garnered significant attention due to its numerous benefits, such as privacy protection, low cost, and penetration ability. Extensive research has been conducted in this field, focusing on areas such as gesture recognition, people identification, and fall detection. However, many data-driven methods encounter challenges related to domain shift, where the model fails to perform well in environments different from the training data. One major factor contributing to this issue is the limited availability of Wi-Fi sensing datasets, which makes models learn excessive irrelevant information and over-fit to the training set. Unfortunately, collecting large-scale Wi-Fi sensing datasets across diverse scenarios is a challenging task. To address this problem, we propose CrossFi, a siamese network-based approach that excels in both in-domain scenario and cross-domain scenario, including few-shot, zero-shot scenarios, and even works in few-shot new-class scenario where testing set contains new categories. The core component of CrossFi is a sample-similarity calculation network called CSi-Net, which improves the structure of the siamese network by using an attention mechanism to capture similarity information, instead of simply calculating the distance or cosine similarity. Based on it, we develop an extra Weight-Net that can generate a template for each class, so that our CrossFi can work in different scenarios. Experimental results demonstrate that our CrossFi achieves state-of-the-art performance across various scenarios. In gesture recognition task, our CrossFi achieves an accuracy of 98.17% in in-domain scenario, 91.72% in one-shot cross-domain scenario, 64.81% in zero-shot cross-domain scenario, and 84.75% in one-shot new-class scenario. To facilitate future research, we will release the code for our model upon publication.
- [352] arXiv:2408.10920 [pdf, html, other]
-
Title: Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear RepresentationsSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
The Linear Representation Hypothesis (LRH) states that neural networks learn to encode concepts as directions in activation space, and a strong version of the LRH states that models learn only such encodings. In this paper, we present a counterexample to this strong LRH: when trained to repeat an input token sequence, gated recurrent neural networks (RNNs) learn to represent the token at each position with a particular order of magnitude, rather than a direction. These representations have layered features that are impossible to locate in distinct linear subspaces. To show this, we train interventions to predict and manipulate tokens by learning the scaling factor corresponding to each sequence position. These interventions indicate that the smallest RNNs find only this magnitude-based solution, while larger RNNs have linear representations. These findings strongly indicate that interpretability research should not be confined by the LRH.
- [353] arXiv:2408.10921 [pdf, html, other]
-
Title: MTFinEval:A Multi-domain Chinese Financial Benchmark with Eurypalynous questionsSubjects: Artificial Intelligence (cs.AI)
With the emergence of more and more economy-specific LLMS, how to measure whether they can be safely invested in production becomes a problem. Previous research has primarily focused on evaluating the performance of LLMs within specific application scenarios. However, these benchmarks cannot reflect the theoretical level and generalization ability, and the backward datasets are increasingly unsuitable for problems in real scenarios. In this paper, we have compiled a new benchmark, MTFinEval, focusing on the LLMs' basic knowledge of economics, which can always be used as a basis for judgment. To examine only theoretical knowledge as much as possible, MTFinEval is build with foundational questions from university textbooks,and exam papers in economics and management major. Aware of the overall performance of LLMs do not depend solely on one subdiscipline of economics, MTFinEval comprise 360 questions refined from six major disciplines of economics, and reflect capabilities more comprehensively. Experiment result shows all LLMs perform poorly on MTFinEval, which proves that our benchmark built on basic knowledge is very successful. Our research not only offers guidance for selecting the appropriate LLM for specific use cases, but also put forward increase the rigor reliability of LLMs from the basics.
- [354] arXiv:2408.10923 [pdf, html, other]
-
Title: LBC: Language-Based-Classifier for Out-Of-Variable GeneralizationComments: 16 pages, 7 figures, 4 tablesSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Large Language Models (LLMs) have great success in natural language processing tasks such as response generation. However, their use in tabular data has been limited due to their inferior performance compared to traditional machine learning models (TMLs) such as XGBoost. We find that the pre-trained knowledge of LLMs enables them to interpret new variables that appear in a test without additional training, a capability central to the concept of Out-of-Variable (OOV). From the findings, we propose a Language-Based-Classifier (LBC), a classifier that maximizes the benefits of LLMs to outperform TMLs on OOV tasks. LBC employs three key methodological strategies: 1) Categorical changes to adjust data to better fit the model's understanding, 2) Advanced order and indicator to enhance data representation to the model, and 3) Using verbalizer to map logit scores to classes during inference to generate model predictions. These strategies, combined with the pre-trained knowledge of LBC, emphasize the model's ability to effectively handle OOV tasks. We empirically and theoretically validate the superiority of LBC. LBC is the first study to apply an LLM-based model to OOV tasks. The source code is at this https URL.
- [355] arXiv:2408.10932 [pdf, html, other]
-
Title: The Evolution of Reinforcement Learning in Quantitative FinanceComments: This work is currently submitted to and under-review for ACM Computing Surveys. This copy is an unedited, pre-print version and it is the author's version of the work. ISubjects: Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE); Machine Learning (cs.LG)
Reinforcement Learning (RL) has experienced significant advancement over the past decade, prompting a growing interest in applications within finance. This survey critically evaluates 167 publications, exploring diverse RL applications and frameworks in finance. Financial markets, marked by their complexity, multi-agent nature, information asymmetry, and inherent randomness, serve as an intriguing test-bed for RL. Traditional finance offers certain solutions, and RL advances these with a more dynamic approach, incorporating machine learning methods, including transfer learning, meta-learning, and multi-agent solutions. This survey dissects key RL components through the lens of Quantitative Finance. We uncover emerging themes, propose areas for future research, and critique the strengths and weaknesses of existing methods.
- [356] arXiv:2408.10933 [pdf, html, other]
-
Title: Evaluating Assistive Technologies on a Trade Fair: Methodological Overview and Lessons LearnedSubjects: Human-Computer Interaction (cs.HC); Robotics (cs.RO)
User-centered evaluations are a core requirement in the development of new user related technologies. However, it is often difficult to recruit sufficient participants, especially if the target population is small, particularly busy, or in some way restricted in their mobility. We bypassed these problems by conducting studies on trade fairs that were specifically designed for our target population (potentially care-receiving individuals in wheelchairs) and therefore provided our users with external incentive to attend our study. This paper presents our gathered experiences, including methodological specifications and lessons learned, and is aimed to guide other researchers with conducting similar studies. In addition, we also discuss chances generated by this unconventional study environment as well as its limitations.
- [357] arXiv:2408.10934 [pdf, html, other]
-
Title: SDI-Net: Toward Sufficient Dual-View Interaction for Low-light Stereo Image EnhancementSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Image and Video Processing (eess.IV)
Currently, most low-light image enhancement methods only consider information from a single view, neglecting the correlation between cross-view information. Therefore, the enhancement results produced by these methods are often unsatisfactory. In this context, there have been efforts to develop methods specifically for low-light stereo image enhancement. These methods take into account the cross-view disparities and enable interaction between the left and right views, leading to improved performance. However, these methods still do not fully exploit the interaction between left and right view information. To address this issue, we propose a model called Toward Sufficient Dual-View Interaction for Low-light Stereo Image Enhancement (SDI-Net). The backbone structure of SDI-Net is two encoder-decoder pairs, which are used to learn the mapping function from low-light images to normal-light images. Among the encoders and the decoders, we design a module named Cross-View Sufficient Interaction Module (CSIM), aiming to fully exploit the correlations between the binocular views via the attention mechanism. The quantitative and visual results on public datasets validate the superiority of our method over other related methods. Ablation studies also demonstrate the effectiveness of the key elements in our model.
- [358] arXiv:2408.10935 [pdf, html, other]
-
Title: Large Point-to-Gaussian Model for Image-to-3D GenerationComments: 10 pages, 9 figures, ACM MM 2024Subjects: Computer Vision and Pattern Recognition (cs.CV)
Recently, image-to-3D approaches have significantly advanced the generation quality and speed of 3D assets based on large reconstruction models, particularly 3D Gaussian reconstruction models. Existing large 3D Gaussian models directly map 2D image to 3D Gaussian parameters, while regressing 2D image to 3D Gaussian representations is challenging without 3D priors. In this paper, we propose a large Point-to-Gaussian model, that inputs the initial point cloud produced from large 3D diffusion model conditional on 2D image to generate the Gaussian parameters, for image-to-3D generation. The point cloud provides initial 3D geometry prior for Gaussian generation, thus significantly facilitating image-to-3D Generation. Moreover, we present the \textbf{A}ttention mechanism, \textbf{P}rojection mechanism, and \textbf{P}oint feature extractor, dubbed as \textbf{APP} block, for fusing the image features with point cloud features. The qualitative and quantitative experiments extensively demonstrate the effectiveness of the proposed approach on GSO and Objaverse datasets, and show the proposed method achieves state-of-the-art performance.
- [359] arXiv:2408.10937 [pdf, html, other]
-
Title: Proxona: Leveraging LLM-Driven Personas to Enhance Creators' Understanding of Their AudienceComments: 32 pages (including 14 pages of Appendix)Subjects: Human-Computer Interaction (cs.HC)
Creators are nothing without their audience, and thereby understanding their audience is the cornerstone of their professional achievement. Yet many creators feel lost while comprehending audiences with existing tools, which offer insufficient insights for tailoring content to audience needs. To address the challenges creators face in understanding their audience, we present Proxona, a system for defining and extracting representative audience personas from the comments. Creators converse with personas to gain insights into their preferences and engagement, solicit feedback, and implement evidence-based improvements to their content. Powered by large language models, Proxona analyzes audience comments, distilling the latent characteristics of audiences into tangible dimensions (classification categories) and values (category attributes). Proxona then clusters these into synthetic personas. Our technical evaluations demonstrated that our pipelines effectively generated relevant and distinct dimensions and values, enabling the deduction of audience-reflecting personas, while minimizing the likelihood of hallucinations in persona responses. Our user evaluation with 11 creators showed that Proxona supported creators to gain new insights about their audience, make informed decisions, and successfully complete content creation with high confidence. Proxona's data-driven audience personas empower creators to seamlessly integrate audience perspectives into their creative processes, fostering a collaborative approach to content creation.
- [360] arXiv:2408.10939 [pdf, html, other]
-
Title: Conformalized Interval Arithmetic with Symmetric CalibrationSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Uncertainty quantification is essential in decision-making, especially when joint distributions of random variables are involved. While conformal prediction provides distribution-free prediction sets with valid coverage guarantees, it traditionally focuses on single predictions. This paper introduces novel conformal prediction methods for estimating the sum or average of unknown labels over specific index sets. We develop conformal prediction intervals for single target to the prediction interval for sum of multiple targets. Under permutation invariant assumptions, we prove the validity of our proposed method. We also apply our algorithms on class average estimation and path cost prediction tasks, and we show that our method outperforms existing conformalized approaches as well as non-conformal approaches.
- [361] arXiv:2408.10940 [pdf, html, other]
-
Title: A Closer Look at Data Augmentation Strategies for Finetuning-Based Low/Few-Shot Object DetectionVladislav Li, Georgios Tsoumplekas, Ilias Siniosoglou, Vasileios Argyriou, Anastasios Lytos, Eleftherios Fountoukidis, Panagiotis SarigiannidisSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Performance (cs.PF)
Current methods for low- and few-shot object detection have primarily focused on enhancing model performance for detecting objects. One common approach to achieve this is by combining model finetuning with data augmentation strategies. However, little attention has been given to the energy efficiency of these approaches in data-scarce regimes. This paper seeks to conduct a comprehensive empirical study that examines both model performance and energy efficiency of custom data augmentations and automated data augmentation selection strategies when combined with a lightweight object detector. The methods are evaluated in three different benchmark datasets in terms of their performance and energy consumption, and the Efficiency Factor is employed to gain insights into their effectiveness considering both performance and efficiency. Consequently, it is shown that in many cases, the performance gains of data augmentation strategies are overshadowed by their increased energy usage, necessitating the development of more energy efficient data augmentation strategies to address data scarcity.
- [362] arXiv:2408.10941 [pdf, html, other]
-
Title: Safety-Critical Stabilization of Force-Controlled Nonholonomic RobotsSubjects: Systems and Control (eess.SY)
We present a safety-critical controller for the problem of stabilization for force-controlled nonholonomic autonomous vehicles. The proposed control law is based on the constructions of control Lyapunov functions (CLFs) and control barrier functions (CBFs) for cascaded systems. To address nonholonomicity, we design the nominal controller that guarantees global asymptotic stability and local exponential stability for the closed-loop system in polar coordinates and construct a strict Lyapunov function valid on any compact sets. Furthermore, we present a procedure for constructing CBFs for cascaded systems, utilizing the CBF of the kinematic model through integrator backstepping. Quadratic programming is employed to combine CLFs and CBFs to integrate both stability and safety in the closed loop. The proposed control law is time-invariant, continuous along trajectories, and easy to implement. Our main results guarantee both safety and local asymptotic stability for the closed-loop system.
- [363] arXiv:2408.10942 [pdf, html, other]
-
Title: Robust Regression with Ensembles Communicating over Noisy ChannelsSubjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Information Theory (cs.IT)
As machine-learning models grow in size, their implementation requirements cannot be met by a single computer system. This observation motivates distributed settings, in which intermediate computations are performed across a network of processing units, while the central node only aggregates their outputs. However, distributing inference tasks across low-precision or faulty edge devices, operating over a network of noisy communication channels, gives rise to serious reliability challenges. We study the problem of an ensemble of devices, implementing regression algorithms, that communicate through additive noisy channels in order to collaboratively perform a joint regression task. We define the problem formally, and develop methods for optimizing the aggregation coefficients for the parameters of the noise in the channels, which can potentially be correlated. Our results apply to the leading state-of-the-art ensemble regression methods: bagging and gradient boosting. We demonstrate the effectiveness of our algorithms on both synthetic and real-world datasets.
- [364] arXiv:2408.10943 [pdf, html, other]
-
Title: SysBench: Can Large Language Models Follow System Messages?Yanzhao Qin, Tao Zhang, Tao Zhang, Yanjun Shen, Wenjing Luo, Haoze Sun, Yan Zhang, Yujing Qiao, Weipeng Chen, Zenan Zhou, Wentao Zhang, Bin CuiSubjects: Computation and Language (cs.CL)
Large Language Models (LLMs) have become instrumental across various applications, with the customization of these models to specific scenarios becoming increasingly critical. System message, a fundamental component of LLMs, is consist of carefully crafted instructions that guide the behavior of model to meet intended goals. Despite the recognized potential of system messages to optimize AI-driven solutions, there is a notable absence of a comprehensive benchmark for evaluating how well different LLMs follow these system messages. To fill this gap, we introduce SysBench, a benchmark that systematically analyzes system message following ability in terms of three challenging aspects: constraint complexity, instruction misalignment and multi-turn stability. In order to enable effective evaluation, SysBench constructs multi-turn user conversations covering various interaction relationships, based on six common types of constraints from system messages in real-world scenarios. Our dataset contains 500 system messages from various domains, each paired with 5 turns of user conversations, which have been manually formulated and checked to guarantee high quality. SysBench provides extensive evaluation across various LLMs, measuring their ability to follow specified constraints given in system messages. The results highlight both the strengths and weaknesses of existing models, offering key insights and directions for future research. The open source library SysBench is available at this https URL.
- [365] arXiv:2408.10945 [pdf, html, other]
-
Title: HiRED: Attention-Guided Token Dropping for Efficient Inference of High-Resolution Vision-Language Models in Resource-Constrained EnvironmentsComments: PreprintSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
High-resolution Vision-Language Models (VLMs) have been widely used in multimodal tasks to enhance accuracy by preserving detailed image information. However, these models often generate excessive visual tokens due to encoding multiple partitions of the input image. Processing these excessive visual tokens is computationally challenging, especially in resource-constrained environments with commodity GPUs. To support high-resolution images while meeting resource constraints, we propose High-Resolution Early Dropping (HiRED), a token-dropping scheme that operates within a fixed token budget before the Large Language Model (LLM) stage. HiRED can be integrated with existing high-resolution VLMs in a plug-and-play manner, as it requires no additional training while still maintaining superior accuracy. We strategically use the vision encoder's attention in the initial layers to assess the visual content of each image partition and allocate the token budget accordingly. Then, using the attention in the final layer, we select the most important visual tokens from each partition within the allocated budget, dropping the rest. Empirically, when applied to LLaVA-Next-7B on NVIDIA TESLA P40 GPU, HiRED with a 20% token budget increases token generation throughput by 4.7, reduces first-token generation latency by 15 seconds, and saves 2.3 GB of GPU memory for a single inference.
- [366] arXiv:2408.10946 [pdf, html, other]
-
Title: Large Language Model Driven RecommendationAnton Korikov, Scott Sanner, Yashar Deldjoo, Zhankui He, Julian McAuley, Arnau Ramisa, Rene Vidal, Mahesh Sathiamoorthy, Atoosa Kasrizadeh, Silvia Milano, Francesco RicciSubjects: Artificial Intelligence (cs.AI)
While previous chapters focused on recommendation systems (RSs) based on standardized, non-verbal user feedback such as purchases, views, and clicks -- the advent of LLMs has unlocked the use of natural language (NL) interactions for recommendation. This chapter discusses how LLMs' abilities for general NL reasoning present novel opportunities to build highly personalized RSs -- which can effectively connect nuanced and diverse user preferences to items, potentially via interactive dialogues. To begin this discussion, we first present a taxonomy of the key data sources for language-driven recommendation, covering item descriptions, user-system interactions, and user profiles. We then proceed to fundamental techniques for LLM recommendation, reviewing the use of encoder-only and autoregressive LLM recommendation in both tuned and untuned settings. Afterwards, we move to multi-module recommendation architectures in which LLMs interact with components such as retrievers and RSs in multi-stage pipelines. This brings us to architectures for conversational recommender systems (CRSs), in which LLMs facilitate multi-turn dialogues where each turn presents an opportunity not only to make recommendations, but also to engage with the user in interactive preference elicitation, critiquing, and question-answering.
- [367] arXiv:2408.10947 [pdf, html, other]
-
Title: Dr.Academy: A Benchmark for Evaluating Questioning Capability in Education for Large Language ModelsComments: Accepted to ACL 2024Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY)
Teachers are important to imparting knowledge and guiding learners, and the role of large language models (LLMs) as potential educators is emerging as an important area of study. Recognizing LLMs' capability to generate educational content can lead to advances in automated and personalized learning. While LLMs have been tested for their comprehension and problem-solving skills, their capability in teaching remains largely unexplored. In teaching, questioning is a key skill that guides students to analyze, evaluate, and synthesize core concepts and principles. Therefore, our research introduces a benchmark to evaluate the questioning capability in education as a teacher of LLMs through evaluating their generated educational questions, utilizing Anderson and Krathwohl's taxonomy across general, monodisciplinary, and interdisciplinary domains. We shift the focus from LLMs as learners to LLMs as educators, assessing their teaching capability through guiding them to generate questions. We apply four metrics, including relevance, coverage, representativeness, and consistency, to evaluate the educational quality of LLMs' outputs. Our results indicate that GPT-4 demonstrates significant potential in teaching general, humanities, and science courses; Claude2 appears more apt as an interdisciplinary teacher. Furthermore, the automatic scores align with human perspectives.
- [368] arXiv:2408.10948 [pdf, html, other]
-
Title: GAIM: Attacking Graph Neural Networks via Adversarial Influence MaximizationSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Recent studies show that well-devised perturbations on graph structures or node features can mislead trained Graph Neural Network (GNN) models. However, these methods often overlook practical assumptions, over-rely on heuristics, or separate vital attack components. In response, we present GAIM, an integrated adversarial attack method conducted on a node feature basis while considering the strict black-box setting. Specifically, we define an adversarial influence function to theoretically assess the adversarial impact of node perturbations, thereby reframing the GNN attack problem into the adversarial influence maximization problem. In our approach, we unify the selection of the target node and the construction of feature perturbations into a single optimization problem, ensuring a unique and consistent feature perturbation for each target node. We leverage a surrogate model to transform this problem into a solvable linear programming task, streamlining the optimization process. Moreover, we extend our method to accommodate label-oriented attacks, broadening its applicability. Thorough evaluations on five benchmark datasets across three popular models underscore the effectiveness of our method in both untargeted and label-oriented targeted attacks. Through comprehensive analysis and ablation studies, we demonstrate the practical value and efficacy inherent to our design choices.
- [369] arXiv:2408.10951 [pdf, html, other]
-
Title: Wave-Mask/Mix: Exploring Wavelet-Based Augmentations for Time Series ForecastingSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Data augmentation is important for improving machine learning model performance when faced with limited real-world data. In time series forecasting (TSF), where accurate predictions are crucial in fields like finance, healthcare, and manufacturing, traditional augmentation methods for classification tasks are insufficient to maintain temporal coherence. This research introduces two augmentation approaches using the discrete wavelet transform (DWT) to adjust frequency elements while preserving temporal dependencies in time series data. Our methods, Wavelet Masking (WaveMask) and Wavelet Mixing (WaveMix), are evaluated against established baselines across various forecasting horizons. To the best of our knowledge, this is the first study to conduct extensive experiments on multivariate time series using Discrete Wavelet Transform as an augmentation technique. Experimental results demonstrate that our techniques achieve competitive results with previous methods. We also explore cold-start forecasting using downsampled training datasets, comparing outcomes to baseline methods.
- [370] arXiv:2408.10955 [pdf, html, other]
-
Title: Multichannel Attention Networks with Ensembled Transfer Learning to Recognize Bangla Handwritten CharecterFarhanul Haque, Md. Al-Hasan, Sumaiya Tabssum Mou, Abu Saleh Musa Miah, Jungpil Shin, Md Abdur RahimSubjects: Computer Vision and Pattern Recognition (cs.CV)
The Bengali language is the 5th most spoken native and 7th most spoken language in the world, and Bengali handwritten character recognition has attracted researchers for decades. However, other languages such as English, Arabic, Turkey, and Chinese character recognition have contributed significantly to developing handwriting recognition systems. Still, little research has been done on Bengali character recognition because of the similarity of the character, curvature and other complexities. However, many researchers have used traditional machine learning and deep learning models to conduct Bengali hand-written recognition. The study employed a convolutional neural network (CNN) with ensemble transfer learning and a multichannel attention network. We generated the feature from the two branches of the CNN, including Inception Net and ResNet and then produced an ensemble feature fusion by concatenating them. After that, we applied the attention module to produce the contextual information from the ensemble features. Finally, we applied a classification module to refine the features and classification. We evaluated the proposed model using the CAMTERdb 3.1.2 data set and achieved 92\% accuracy for the raw dataset and 98.00\% for the preprocessed dataset. We believe that our contribution to the Bengali handwritten character recognition domain will be considered a great development.
- [371] arXiv:2408.10962 [pdf, html, other]
-
Title: NLP for The Greek Language: A Longer SurveySubjects: Computation and Language (cs.CL)
English language is in the spotlight of the Natural Language Processing (NLP) community with other languages, like Greek, lagging behind in terms of offered methods, tools and resources. Due to the increasing interest in NLP, in this paper we try to condense research efforts for the automatic processing of Greek language covering the last three decades. In particular, we list and briefly discuss related works, resources and tools, categorized according to various processing layers and contexts. We are not restricted to the modern form of Greek language but also cover Ancient Greek and various Greek dialects. This survey can be useful for researchers and students interested in NLP tasks, Information Retrieval and Knowledge Management for the Greek language.
- [372] arXiv:2408.10963 [pdf, html, other]
-
Title: KeySpace: Public Key Infrastructure Considerations in Interplanetary NetworksComments: 14 pages, 9 figures, 7 tablesSubjects: Cryptography and Security (cs.CR); Networking and Internet Architecture (cs.NI)
As satellite networks grow larger and begin to incorporate interplanetary communication, there is an increasing interest in the unsolved problem of how to approach PKI in these conditions. In this paper we explore the goals and requirements for implementing key management systems in satellite networks, focusing on megaconstellations and interplanetary networks. We design a set of standardized experiments which can be used to compare systems against one another for particular network topologies. Using these, we demonstrate that terrestrial PKI techniques are feasible in highly distributed interplanetary networks, showing that it is possible to configure PKI systems to achieve efficient low-latency connection establishment, and minimize the impact of attacks through effective revocations.
We evaluate this by building the Deep Space Network Simulator (DSNS), a novel network simulator aimed at efficient simulation of large space networks. We run simulations evaluating connection establishment and key revocation under a wide range of PKI configurations. Finally, we propose and evaluate two additional configuration options: OCSP Hybrid, and the use of relay nodes as a firewall. Together these minimize the extent of the network an attacker can reach with a compromised key, and reduce the attacker's load on interplanetary relay links. - [373] arXiv:2408.10970 [pdf, html, other]
-
Title: Hybrid Recurrent Models Support Emergent Descriptions for Hierarchical Planning and ControlComments: 4 pages, 3 figuresSubjects: Artificial Intelligence (cs.AI); Systems and Control (eess.SY)
An open problem in artificial intelligence is how systems can flexibly learn discrete abstractions that are useful for solving inherently continuous problems. Previous work has demonstrated that a class of hybrid state-space model known as recurrent switching linear dynamical systems (rSLDS) discover meaningful behavioural units via the piecewise linear decomposition of complex continuous dynamics (Linderman et al., 2016). Furthermore, they model how the underlying continuous states drive these discrete mode switches. We propose that the rich representations formed by an rSLDS can provide useful abstractions for planning and control. We present a novel hierarchical model-based algorithm inspired by Active Inference in which a discrete MDP sits above a low-level linear-quadratic controller. The recurrent transition dynamics learned by the rSLDS allow us to (1) specify temporally-abstracted sub-goals in a method reminiscent of the options framework, (2) lift the exploration into discrete space allowing us to exploit information-theoretic exploration bonuses and (3) `cache' the approximate solutions to low-level problems in the discrete planner. We successfully apply our model to the sparse Continuous Mountain Car task, demonstrating fast system identification via enhanced exploration and non-trivial planning through the delineation of abstract sub-goals.
- [374] arXiv:2408.10971 [pdf, html, other]
-
Title: Asynchronous Fault-Tolerant Distributed Proper Coloring of GraphsSubjects: Distributed, Parallel, and Cluster Computing (cs.DC)
We revisit asynchronous computing in networks of crash-prone processes, under the asynchronous variant of the standard LOCAL model, recently introduced by Fraigniaud et al. [DISC 2022]. We focus on the vertex coloring problem, and our contributions concern both lower and upper bounds for this problem.
On the upper bound side, we design an algorithm tolerating an arbitrarily large number of crash failures that computes an $O(\Delta^2)$-coloring of any $n$-node graph of maximum degree $\Delta$, in $O(\log^\star n)$ rounds. This extends Linial's seminal result from the (synchronous failure-free) LOCAL model to its asynchronous crash-prone variant. Then, by allowing a dependency on $\Delta$ on the runtime, we show that we can reduce the colors to $\big(\frac12(\Delta+1)(\Delta+2)-1 \big)$. For cycles (i.e., for $\Delta=2$), our algorithm achieves a 5-coloring of any $n$-node cycle, in $O(\log^\star n)$ rounds. This improves the known 6-coloring algorithm by Fraigniaud et al., and fixes a bug in their algorithm, which was erroneously claimed to produce a 5-coloring.
On the lower bound side, we show that, for $k<5$, and for every prime integer~$n$, no algorithm can $k$-color the $n$-node cycle in the asynchronous crash-prone variant of LOCAL, independently from the round-complexities of the algorithms. This lower bound is obtained by reduction from an original extension of the impossibility of solving weak symmetry-breaking in the wait-free shared-memory model. We show that this impossibility still holds even if the processes are provided with inputs susceptible to help breaking symmetry. - [375] arXiv:2408.10974 [pdf, html, other]
-
Title: Deep Reinforcement Learning for Network Energy Saving in 6G and Beyond NetworksComments: 7 pages, 4 figuresSubjects: Networking and Internet Architecture (cs.NI)
Network energy saving has received great attention from operators and vendors to reduce energy consumption and CO2 emissions to the environment as well as significantly reduce costs for mobile network operators. However, the design of energy-saving networks also needs to ensure the mobile users' (MUs) QoS requirements such as throughput requirements (TR). This work considers a mobile cellular network including many ground base stations (GBSs), and some GBSs are intentionally turned off due to network energy saving (NES) or crash, so the MUs located in these outage GBSs are not served in time. Based on this observation, we propose the problem of maximizing the total achievable throughput in the network by optimizing the GBSs' antenna tilt and adaptive transmission power with a given number of served MUs satisfied. Notice that, the MU is considered successfully served if its Reference Signal Received Power (RSRP) and throughput requirement are satisfied. The formulated optimization problem becomes difficult to solve with multiple binary variables and non-convex constraints along with random throughput requirements and random placement of MUs. We propose a Deep Q-learning-based algorithm to help the network learn the uncertainty and dynamics of the transmission environment. Extensive simulation results show that our proposed algorithm achieves much better performance than the benchmark schemes.
- [376] arXiv:2408.10979 [pdf, other]
-
Title: The fusion of phonography and ideographic characters into virtual Chinese characters -- Based on Chinese and EnglishComments: 14 pages, 7 figuresSubjects: Computation and Language (cs.CL)
The characters used in modern countries are mainly divided into ideographic characters and phonetic characters, both of which have their advantages and disadvantages. Chinese is difficult to learn and easy to master, while English is easy to learn but has a large vocabulary. There is still no language that combines the advantages of both languages and has less memory capacity, can form words, and is easy to learn. Therefore, inventing new characters that can be combined and the popularization of deep knowledge, and reduce disputes through communication. Firstly, observe the advantages and disadvantages of Chinese and English, such as their vocabulary, information content, and ease of learning in deep scientific knowledge, and create a new writing system. Then, use comparative analysis to observe the total score of the new language. Through this article, it can be concluded that the new text combines the advantages of both pictographic and alphabetical writing: new characters that can be combined into words reduces the vocabulary that needs to be learned; Special prefixes allow beginners to quickly guess the approximate category and meaning of unseen words; New characters can enable humans to quickly learn more advanced knowledge.
- [377] arXiv:2408.10982 [pdf, html, other]
-
Title: GreediRIS: Scalable Influence Maximization using Distributed Streaming Maximum CoverSubjects: Distributed, Parallel, and Cluster Computing (cs.DC)
Influence maximization--the problem of identifying a subset of k influential seeds (vertices) in a network--is a classical problem in network science with numerous applications. The problem is NP-hard, but there exist efficient polynomial time approximations. However, scaling these algorithms still remain a daunting task due to the complexities associated with steps involving stochastic sampling and large-scale aggregations. In this paper, we present a new parallel distributed approximation algorithm for influence maximization with provable approximation guarantees. Our approach, which we call GreediRIS, leverages the RandGreedi framework--a state-of-the-art approach for distributed submodular optimization--for solving a step that computes a maximum k cover. GreediRIS combines distributed and streaming models of computations, along with pruning techniques, to effectively address the communication bottlenecks of the algorithm. Experimental results on up to 512 nodes (32K cores) of the NERSC Perlmutter supercomputer show that GreediRIS can achieve good strong scaling performance, preserve quality, and significantly outperform the other state-of-the-art distributed implementations. For instance, on 512 nodes, the most performant variant of GreediRIS achieves geometric mean speedups of 28.99x and 36.35x for two different diffusion models, over a state-of-the-art parallel implementation. We also present a communication-optimized version of GreediRIS that further improves the speedups by two orders of magnitude.
- [378] arXiv:2408.10993 [pdf, html, other]
-
Title: Facial Demorphing via Identity Preserving Image DecompositionSubjects: Computer Vision and Pattern Recognition (cs.CV)
A face morph is created by combining the face images usually pertaining to two distinct identities. The goal is to generate an image that can be matched with two identities thereby undermining the security of a face recognition system. To deal with this problem, several morph attack detection techniques have been developed. But these methods do not extract any information about the underlying bonafides used to create them. Demorphing addresses this limitation. However, current demorphing techniques are mostly reference-based, i.e, they need an image of one of the identities to recover the other. In this work, we treat demorphing as an ill-posed decomposition problem. We propose a novel method that is reference-free and recovers the bonafides with high accuracy. Our method decomposes the morph into several identity-preserving feature components. A merger network then weighs and combines these components to recover the bonafides. Our method is observed to reconstruct high-quality bonafides in terms of definition and fidelity. Experiments on the CASIA-WebFace, SMDD and AMSL datasets demonstrate the effectiveness of our method.
- [379] arXiv:2408.10995 [pdf, html, other]
-
Title: CTP-LLM: Clinical Trial Phase Transition Prediction Using Large Language ModelsSubjects: Computation and Language (cs.CL)
New medical treatment development requires multiple phases of clinical trials. Despite the significant human and financial costs of bringing a drug to market, less than 20% of drugs in testing will make it from the first phase to final approval. Recent literature indicates that the design of the trial protocols significantly contributes to trial performance. We investigated Clinical Trial Outcome Prediction (CTOP) using trial design documents to predict phase transitions automatically. We propose CTP-LLM, the first Large Language Model (LLM) based model for CTOP. We also introduce the PhaseTransition (PT) Dataset; which labels trials based on their progression through the regulatory process and serves as a benchmark for CTOP evaluation. Our fine-tuned GPT-3.5-based model (CTP-LLM) predicts clinical trial phase transition by analyzing the trial's original protocol texts without requiring human-selected features. CTP-LLM achieves a 67% accuracy rate in predicting trial phase transitions across all phases and a 75% accuracy rate specifically in predicting the transition from Phase~III to final approval. Our experimental performance highlights the potential of LLM-powered applications in forecasting clinical trial outcomes and assessing trial design.
- [380] arXiv:2408.10997 [pdf, html, other]
-
Title: Disentangling segmental and prosodic factors to non-native speech comprehensibilitySubjects: Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)
Current accent conversion (AC) systems do not disentangle the two main sources of non-native accent: segmental and prosodic characteristics. Being able to manipulate a non-native speaker's segmental and/or prosodic channels independently is critical to quantify how these two channels contribute to speech comprehensibility and social attitudes. We present an AC system that not only decouples voice quality from accent, but also disentangles the latter into its segmental and prosodic characteristics. The system is able to generate accent conversions that combine (1) the segmental characteristics from a source utterance, (2) the voice characteristics from a target utterance, and (3) the prosody of a reference utterance. We show that vector quantization of acoustic embeddings and removal of consecutive duplicated codewords allows the system to transfer prosody and improve voice similarity. We conduct perceptual listening tests to quantify the individual contributions of segmental features and prosody on the perceived comprehensibility of non-native speech. Our results indicate that, contrary to prior research in non-native speech, segmental features have a larger impact on comprehensibility than prosody. The proposed AC system may also be used to study how segmental and prosody cues affect social attitudes towards non-native speech.
- [381] arXiv:2408.10998 [pdf, html, other]
-
Title: Audio Match Cutting: Finding and Creating Matching Audio Transitions in Movies and VideosComments: Accepted to ICASSP 2024Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
A "match cut" is a common video editing technique where a pair of shots that have a similar composition transition fluidly from one to another. Although match cuts are often visual, certain match cuts involve the fluid transition of audio, where sounds from different sources merge into one indistinguishable transition between two shots. In this paper, we explore the ability to automatically find and create "audio match cuts" within videos and movies. We create a self-supervised audio representation for audio match cutting and develop a coarse-to-fine audio match pipeline that recommends matching shots and creates the blended audio. We further annotate a dataset for the proposed audio match cut task and compare the ability of multiple audio representations to find audio match cut candidates. Finally, we evaluate multiple methods to blend two matching audio candidates with the goal of creating a smooth transition. Project page and examples are available at: this https URL
- [382] arXiv:2408.11000 [pdf, html, other]
-
Title: SenPa-MAE: Sensor Parameter Aware Masked Autoencoder for Multi-Satellite Self-Supervised PretrainingComments: GCPR 2024Subjects: Computer Vision and Pattern Recognition (cs.CV)
This paper introduces SenPa-MAE, a transformer architecture that encodes the sensor parameters of an observed multispectral signal into the image embeddings. SenPa-MAE can be pre-trained on imagery of different satellites with non-matching spectral or geometrical sensor characteristics. To incorporate sensor parameters, we propose a versatile sensor parameter encoding module as well as a data augmentation strategy for the diversification of the pre-training dataset. This enables the model to effectively differentiate between various sensors and gain an understanding of sensor parameters and the correlation to the observed signal. Given the rising number of Earth observation satellite missions and the diversity in their sensor specifications, our approach paves the way towards a sensor-independent Earth observation foundation model. This opens up possibilities such as cross-sensor training and sensor-independent inference.
- [383] arXiv:2408.11001 [pdf, html, other]
-
Title: MegaFusion: Extend Diffusion Models towards Higher-resolution Image Generation without Further TuningComments: Technical Report. Project Page: this https URLSubjects: Computer Vision and Pattern Recognition (cs.CV)
Diffusion models have emerged as frontrunners in text-to-image generation for their impressive capabilities. Nonetheless, their fixed image resolution during training often leads to challenges in high-resolution image generation, such as semantic inaccuracies and object replication. This paper introduces MegaFusion, a novel approach that extends existing diffusion-based text-to-image generation models towards efficient higher-resolution generation without additional fine-tuning or extra adaptation. Specifically, we employ an innovative truncate and relay strategy to bridge the denoising processes across different resolutions, allowing for high-resolution image generation in a coarse-to-fine manner. Moreover, by integrating dilated convolutions and noise re-scheduling, we further adapt the model's priors for higher resolution. The versatility and efficacy of MegaFusion make it universally applicable to both latent-space and pixel-space diffusion models, along with other derivative models. Extensive experiments confirm that MegaFusion significantly boosts the capability of existing models to produce images of megapixels and various aspect ratios, while only requiring about 40% of the original computational cost.
- [384] arXiv:2408.11002 [pdf, html, other]
-
Title: On the Cop Number of String GraphsComments: A preliminary version appeared in ISAAC 2022Subjects: Discrete Mathematics (cs.DM); Combinatorics (math.CO)
Cops and Robber is a well-studied two-player pursuit-evasion game played on a graph, where a group of cops tries to capture the robber. The \emph{cop number} of a graph is the minimum number of cops required to capture the robber. Gavenčiak et al.~[Eur. J. of Comb. 72, 45--69 (2018)] studied the game on intersection graphs and established that the cop number for the class of string graphs is at most 15, and asked as an open question to improve this bound for string graphs and subclasses of string graphs. We address this question and establish that the cop number of a string graph is at most 13. To this end, we develop a novel \textit{guarding} technique. We further establish that this technique can be useful for other Cops and Robber games on graphs admitting a representation. In particular, we show that four cops have a winning strategy for a variant of Cops and Robber, named Fully Active Cops and Robber, on planar graphs, addressing an open question of Gromovikov et al.~[Austr. J. Comb. 76(2), 248--265 (2020)]. In passing, we also improve the known bounds on the cop number of boxicity 2 graphs.
- [385] arXiv:2408.11006 [pdf, html, other]
-
Title: While GitHub Copilot Excels at Coding, Does It Ensure Responsible Output?Subjects: Computation and Language (cs.CL); Cryptography and Security (cs.CR)
The rapid development of large language models (LLMs) has significantly advanced code completion capabilities, giving rise to a new generation of LLM-based Code Completion Tools (LCCTs). Unlike general-purpose LLMs, these tools possess unique workflows, integrating multiple information sources as input and prioritizing code suggestions over natural language interaction, which introduces distinct security challenges. Additionally, LCCTs often rely on proprietary code datasets for training, raising concerns about the potential exposure of sensitive data. This paper exploits these distinct characteristics of LCCTs to develop targeted attack methodologies on two critical security risks: jailbreaking and training data extraction attacks. Our experimental results expose significant vulnerabilities within LCCTs, including a 99.4% success rate in jailbreaking attacks on GitHub Copilot and a 46.3% success rate on Amazon Q. Furthermore, We successfully extracted sensitive user data from GitHub Copilot, including 54 real email addresses and 314 physical addresses associated with GitHub usernames. Our study also demonstrates that these code-based attack methods are effective against general-purpose LLMs, such as the GPT series, highlighting a broader security misalignment in the handling of code by modern LLMs. These findings underscore critical security challenges associated with LCCTs and suggest essential directions for strengthening their security frameworks. The example code and attack samples from our research are provided at this https URL.
- [386] arXiv:2408.11007 [pdf, other]
-
Title: Extending the Quantitative Pattern-Matching ParadigmComments: APLAS 2024, full version (including complete proofs)Subjects: Programming Languages (cs.PL); Logic in Computer Science (cs.LO)
We show how (well-established) type systems based on non-idempotent intersection types can be extended to characterize termination properties of functional programming languages with pattern matching features. To model such programming languages, we use a (weak and closed) $\lambda$-calculus integrating a pattern matching mechanism on algebraic data types (ADTs). Remarkably, we also show that this language not only encodes Plotkin's CBV and CBN $\lambda$-calculus as well as other subsuming frameworks, such as the bang-calculus, but can also be used to interpret the semantics of effectful languages with exceptions. After a thorough study of the untyped language, we introduce a type system based on intersection types, and we show through purely logical methods that the set of terminating terms of the language corresponds exactly to that of well-typed terms. Moreover, by considering non-idempotent intersection types, this characterization turns out to be quantitative, i.e. the size of the type derivation of a term t gives an upper bound for the number of evaluation steps from t to its normal form.
- [387] arXiv:2408.11008 [pdf, html, other]
-
Title: Towards a Standardized Representation for Deep Learning Collective AlgorithmsJinsun Yoo, William Won, Meghan Cowan, Nan Jiang, Benjamin Klenk, Srinivas Sridharan, Tushar KrishnaSubjects: Distributed, Parallel, and Cluster Computing (cs.DC)
The explosion of machine learning model size has led to its execution on distributed clusters at a very large scale. Many works have tried to optimize the process of producing collective algorithms and running collective communications, which act as a bottleneck to distributed machine learning. However, different works use their own collective algorithm representation, pushing away from co-optimizing collective communication and the rest of the workload. The lack of a standardized collective algorithm representation has also hindered interoperability between collective algorithm producers and consumers. Additionally, tool-specific conversions and modifications have to be made for each pair of tools producing and consuming collective algorithms which adds to engineering efforts.
In this position paper, we propose a standardized workflow leveraging a common collective algorithm representation. Upstream producers and downstream consumers converge to a common representation format based on Chakra Execution Trace, a commonly used graph based representation of distributed machine learning workloads. Such a common representation enables us to view collective communications at the same level as workload operations and decouple producer and consumer tools, enhance interoperability, and relieve the user from the burden of having to focus on downstream implementations. We provide a proof-of-concept of this standardized workflow by simulating collective algorithms generated by the MSCCLang domain-specific language through the ASTRA-sim distributed machine learning simulator using various network configurations. - [388] arXiv:2408.11015 [pdf, other]
-
Title: Hyperproperty-Preserving Register Specifications (Extended Version)Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)
Reasoning about hyperproperties of concurrent implementations, such as the guarantees these implementations provide to randomized client programs, has been a long-standing challenge. Standard linearizability enables the use of atomic specifications for reasoning about standard properties, but not about hyperproperties. A stronger correctness criterion, called strong linearizability, enables such reasoning, but is rarely achievable, leaving various useful implementations with no means for reasoning about their hyperproperties. In this paper, we focus on registers and devise non-atomic specifications that capture a wide-range of well-studied register implementations and enable reasoning about their hyperproperties. First, we consider the class of write strong-linearizable implementations, a recently proposed useful weakening of strong linearizability, which allows more intricate implementations, such as the well-studied single-writer ABD distributed implementation. We introduce a simple shared-memory register specification that can be used for reasoning about hyperproperties of programs that use write strongly-linearizable implementations. Second, we introduce a new linearizability class, which we call decisive linearizability, that is weaker than write strong-linearizability and includes multi-writer ABD, and develop a second shared-memory register specification for reasoning about hyperproperties of programs that use register implementations of this class. These results shed light on the hyperproperties guaranteed when simulating shared memory in a crash-resilient message-passing system.
- [389] arXiv:2408.11017 [pdf, html, other]
-
Title: Multiwinner Temporal Voting with Aversion to ChangeComments: Appears in the 27th European Conference on Artificial Intelligence (ECAI), 2024Subjects: Computer Science and Game Theory (cs.GT); Artificial Intelligence (cs.AI); Computational Complexity (cs.CC)
We study two-stage committee elections where voters have dynamic preferences over candidates; at each stage, a committee is chosen under a given voting rule. We are interested in identifying a winning committee for the second stage that overlaps as much as possible with the first-stage committee. We show a full complexity dichotomy for the class of Thiele rules: this problem is tractable for Approval Voting (AV) and hard for all other Thiele rules (including, in particular, Proportional Approval Voting and the Chamberlin-Courant rule). We extend this dichotomy to the greedy variants of Thiele rules. We also explore this problem from a parameterized complexity perspective for several natural parameters. We complement the theory with experimental analysis: e.g., we investigate the average number of changes in the committee as a function of changes in voters' preferences and the role of ties.
- [390] arXiv:2408.11021 [pdf, html, other]
-
Title: Athena: Safe Autonomous Agents with Verbal Contrastive LearningComments: 9 pages, 2 figures, 4 tablesSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)
Due to emergent capabilities, large language models (LLMs) have been utilized as language-based agents to perform a variety of tasks and make decisions with an increasing degree of autonomy. These autonomous agents can understand high-level instructions, interact with their environments, and execute complex tasks using a selection of tools available to them. As the capabilities of the agents expand, ensuring their safety and trustworthiness becomes more imperative. In this study, we introduce the Athena framework which leverages the concept of verbal contrastive learning where past safe and unsafe trajectories are used as in-context (contrastive) examples to guide the agent towards safety while fulfilling a given task. The framework also incorporates a critiquing mechanism to guide the agent to prevent risky actions at every step. Furthermore, due to the lack of existing benchmarks on the safety reasoning ability of LLM-based agents, we curate a set of 80 toolkits across 8 categories with 180 scenarios to provide a safety evaluation benchmark. Our experimental evaluation, with both closed- and open-source LLMs, indicates verbal contrastive learning and interaction-level critiquing improve the safety rate significantly.
- [391] arXiv:2408.11029 [pdf, html, other]
-
Title: Scaling Law with Learning Rate AnnealingComments: 25 pages, 23 figuresSubjects: Computation and Language (cs.CL)
We find that the cross-entropy loss curves of neural language models empirically adhere to a scaling law with learning rate (LR) annealing over training steps ($s$): $$L(s) = L_0 + A\cdot S_1^{-\alpha} - C\cdot S_2$$ Where $S_1$ is forward area and $S_2$ is learning rate annealing area. This formulation takes into account two factors: (1) The forward scaling defined as typical scaling law, and (2) the additional loss drop brought by LR annealing. Therefore, this formulation can describe the full loss curve at each step, rather than the single loss point at the end of training. Applying the scaling law with LR annealing and fitting only one or two training curves, we can accurately predict the loss of language model training at any given step and across any learning rate scheduler (LRS). Furthermore, this equation accurately describes the dynamics during training process, and provides a theoretical verification and explanation for numerous experimental findings of previous studies, particularly those focusing on LR schedule and LR annealing. The resulting insights, also serve as a guide for researchers to select critical LRS in advance by prediction using our equation. Most significantly, since all the points in a full training curve follow the equation, we can achieve accurate loss prediction at any given step across any learning rate scheduler, while expending less than 1\% of the computational cost required by the chinchilla scaling law to fit language modeling loss. This approach extremely democratizes scaling law fitting and predicting in developing large language models.
- [392] arXiv:2408.11030 [pdf, html, other]
-
Title: OpenScan: A Benchmark for Generalized Open-Vocabulary 3D Scene UnderstandingSubjects: Computer Vision and Pattern Recognition (cs.CV)
Open-vocabulary 3D scene understanding (OV-3D) aims to localize and classify novel objects beyond the closed object classes. However, existing approaches and benchmarks primarily focus on the open vocabulary problem within the context of object classes, which is insufficient to provide a holistic evaluation to what extent a model understands the 3D scene. In this paper, we introduce a more challenging task called Generalized Open-Vocabulary 3D Scene Understanding (GOV-3D) to explore the open vocabulary problem beyond object classes. It encompasses an open and diverse set of generalized knowledge, expressed as linguistic queries of fine-grained and object-specific attributes. To this end, we contribute a new benchmark named OpenScan, which consists of 3D object attributes across eight representative linguistic aspects, including affordance, property, material, and more. We further evaluate state-of-the-art OV-3D methods on our OpenScan benchmark, and discover that these methods struggle to comprehend the abstract vocabularies of the GOV-3D task, a challenge that cannot be addressed by simply scaling up object classes during training. We highlight the limitations of existing methodologies and explore a promising direction to overcome the identified shortcomings. Data and code are available at this https URL
- [393] arXiv:2408.11032 [pdf, html, other]
-
Title: Atmospheric Transport Modeling of CO$_2$ with Neural NetworksComments: Code: this https URLSubjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Atmospheric and Oceanic Physics (physics.ao-ph)
Accurately describing the distribution of CO$_2$ in the atmosphere with atmospheric tracer transport models is essential for greenhouse gas monitoring and verification support systems to aid implementation of international climate agreements. Large deep neural networks are poised to revolutionize weather prediction, which requires 3D modeling of the atmosphere. While similar in this regard, atmospheric transport modeling is subject to new challenges. Both, stable predictions for longer time horizons and mass conservation throughout need to be achieved, while IO plays a larger role compared to computational costs. In this study we explore four different deep neural networks (UNet, GraphCast, Spherical Fourier Neural Operator and SwinTransformer) which have proven as state-of-the-art in weather prediction to assess their usefulness for atmospheric tracer transport modeling. For this, we assemble the CarbonBench dataset, a systematic benchmark tailored for machine learning emulators of Eulerian atmospheric transport. Through architectural adjustments, we decouple the performance of our emulators from the distribution shift caused by a steady rise in atmospheric CO$_2$. More specifically, we center CO$_2$ input fields to zero mean and then use an explicit flux scheme and a mass fixer to assure mass balance. This design enables stable and mass conserving transport for over 6 months with all four neural network architectures. In our study, the SwinTransformer displays particularly strong emulation skill (90-day $R^2 > 0.99$), with physically plausible emulation even for forward runs of multiple years. This work paves the way forward towards high resolution forward and inverse modeling of inert trace gases with neural networks.
- [394] arXiv:2408.11039 [pdf, html, other]
-
Title: Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal ModelChunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, Omer LevyComments: 23 pagesSubjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
We introduce Transfusion, a recipe for training a multi-modal model over discrete and continuous data. Transfusion combines the language modeling loss function (next token prediction) with diffusion to train a single transformer over mixed-modality sequences. We pretrain multiple Transfusion models up to 7B parameters from scratch on a mixture of text and image data, establishing scaling laws with respect to a variety of uni- and cross-modal benchmarks. Our experiments show that Transfusion scales significantly better than quantizing images and training a language model over discrete image tokens. By introducing modality-specific encoding and decoding layers, we can further improve the performance of Transfusion models, and even compress each image to just 16 patches. We further demonstrate that scaling our Transfusion recipe to 7B parameters and 2T multi-modal tokens produces a model that can generate images and text on a par with similar scale diffusion models and language models, reaping the benefits of both worlds.
- [395] arXiv:2408.11041 [pdf, other]
-
Title: Decentralized Distributed Graph Coloring II: degree+1-Coloring Virtual GraphsComments: 56 pages. To appear in proceedings of DISC 2024Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Data Structures and Algorithms (cs.DS)
Graph coloring is fundamental to distributed computing. We give the first general treatment of the coloring of virtual graphs, where the graph $H$ to be colored is locally embedded within the communication graph $G$. Besides generalizing classical distributed graph coloring (where $H=G$), this captures other previously studied settings, including cluster graphs and power graphs.
We find that the complexity of coloring a virtual graph depends on the edge congestion of its embedding. The main question of interest is how fast we can color virtual graphs of constant congestion. We find that, surprisingly, these graphs can be colored nearly as fast as ordinary graphs. Namely, we give a $O(\log^4\log n)$-round algorithm for the deg+1-coloring problem, where each node is assigned more colors than its degree.
This can be viewed as a case where a distributed graph problem can be solved even when the operation of each node is decentralized. - [396] arXiv:2408.11042 [pdf, html, other]
-
Title: GraphFSA: A Finite State Automaton Framework for Algorithmic Learning on GraphsComments: Published as a conference paper at ECAI 2024Subjects: Artificial Intelligence (cs.AI)
Many graph algorithms can be viewed as sets of rules that are iteratively applied, with the number of iterations dependent on the size and complexity of the input graph. Existing machine learning architectures often struggle to represent these algorithmic decisions as discrete state transitions. Therefore, we propose a novel framework: GraphFSA (Graph Finite State Automaton). GraphFSA is designed to learn a finite state automaton that runs on each node of a given graph. We test GraphFSA on cellular automata problems, showcasing its abilities in a straightforward algorithmic setting. For a comprehensive empirical evaluation of our framework, we create a diverse range of synthetic problems. As our main application, we then focus on learning more elaborate graph algorithms. Our findings suggest that GraphFSA exhibits strong generalization and extrapolation abilities, presenting an alternative approach to represent these algorithms.
- [397] arXiv:2408.11043 [pdf, html, other]
-
Title: Reconciling Methodological Paradigms: Employing Large Language Models as Novice Qualitative Research Assistants in Talent Management ResearchComments: Accepted to KDD '24 workshop on Talent Management and Computing (TMC 2024). 9 pagesSubjects: Computers and Society (cs.CY); Artificial Intelligence (cs.AI)
Qualitative data collection and analysis approaches, such as those employing interviews and focus groups, provide rich insights into customer attitudes, sentiment, and behavior. However, manually analyzing qualitative data requires extensive time and effort to identify relevant topics and thematic insights. This study proposes a novel approach to address this challenge by leveraging Retrieval Augmented Generation (RAG) based Large Language Models (LLMs) for analyzing interview transcripts. The novelty of this work lies in strategizing the research inquiry as one that is augmented by an LLM that serves as a novice research assistant. This research explores the mental model of LLMs to serve as novice qualitative research assistants for researchers in the talent management space. A RAG-based LLM approach is extended to enable topic modeling of semi-structured interview data, showcasing the versatility of these models beyond their traditional use in information retrieval and search. Our findings demonstrate that the LLM-augmented RAG approach can successfully extract topics of interest, with significant coverage compared to manually generated topics from the same dataset. This establishes the viability of employing LLMs as novice qualitative research assistants. Additionally, the study recommends that researchers leveraging such models lean heavily on quality criteria used in traditional qualitative research to ensure rigor and trustworthiness of their approach. Finally, the paper presents key recommendations for industry practitioners seeking to reconcile the use of LLMs with established qualitative research paradigms, providing a roadmap for the effective integration of these powerful, albeit novice, AI tools in the analysis of qualitative datasets within talent
- [398] arXiv:2408.11046 [pdf, html, other]
-
Title: Inside the Black Box: Detecting Data Leakage in Pre-trained Language EncodersComments: ECAI24Subjects: Computation and Language (cs.CL)
Despite being prevalent in the general field of Natural Language Processing (NLP), pre-trained language models inherently carry privacy and copyright concerns due to their nature of training on large-scale web-scraped data. In this paper, we pioneer a systematic exploration of such risks associated with pre-trained language encoders, specifically focusing on the membership leakage of pre-training data exposed through downstream models adapted from pre-trained language encoders-an aspect largely overlooked in existing literature. Our study encompasses comprehensive experiments across four types of pre-trained encoder architectures, three representative downstream tasks, and five benchmark datasets. Intriguingly, our evaluations reveal, for the first time, the existence of membership leakage even when only the black-box output of the downstream model is exposed, highlighting a privacy risk far greater than previously assumed. Alongside, we present in-depth analysis and insights toward guiding future researchers and practitioners in addressing the privacy considerations in developing pre-trained language models.
- [399] arXiv:2408.11048 [pdf, html, other]
-
Title: RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot HandsYi Zhao, Le Chen, Jan Schneider, Quankai Gao, Juho Kannala, Bernhard Schölkopf, Joni Pajarinen, Dieter BüchlerComments: Project Website: this https URLSubjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
It has been a long-standing research goal to endow robot hands with human-level dexterity. Bi-manual robot piano playing constitutes a task that combines challenges from dynamic tasks, such as generating fast while precise motions, with slower but contact-rich manipulation problems. Although reinforcement learning based approaches have shown promising results in single-task performance, these methods struggle in a multi-song setting. Our work aims to close this gap and, thereby, enable imitation learning approaches for robot piano playing at scale. To this end, we introduce the Robot Piano 1 Million (RP1M) dataset, containing bi-manual robot piano playing motion data of more than one million trajectories. We formulate finger placements as an optimal transport problem, thus, enabling automatic annotation of vast amounts of unlabeled songs. Benchmarking existing imitation learning approaches shows that such approaches reach state-of-the-art robot piano playing performance by leveraging RP1M.
- [400] arXiv:2408.11049 [pdf, html, other]
-
Title: MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative DecodingJian Chen, Vashisth Tiwari, Ranajoy Sadhukhan, Zhuoming Chen, Jinyuan Shi, Ian En-Hsu Yen, Beidi ChenSubjects: Computation and Language (cs.CL)
Large Language Models (LLMs) have become more prevalent in long-context applications such as interactive chatbots, document analysis, and agent workflows, but it is challenging to serve long-context requests with low latency and high throughput. Speculative decoding (SD) is a widely used technique to reduce latency without sacrificing performance but the conventional wisdom suggests that its efficacy is limited to small batch sizes. In MagicDec, we show that surprisingly SD can achieve speedup even for a high throughput inference regime for moderate to long sequences. More interestingly, an intelligent drafting strategy can achieve better speedup with increasing batch size based on our rigorous analysis. MagicDec first identifies the bottleneck shifts with increasing batch size and sequence length, and uses these insights to deploy speculative decoding more effectively for high throughput inference. Then, it leverages draft models with sparse KV cache to address the KV bottleneck that scales with both sequence length and batch size.
- [401] arXiv:2408.11051 [pdf, html, other]
-
Title: FLAME: Learning to Navigate with Multimodal LLM in Urban EnvironmentsComments: 10 pages, 5 figuresSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Robotics (cs.RO)
Large Language Models (LLMs) have demonstrated potential in Vision-and-Language Navigation (VLN) tasks, yet current applications face challenges. While LLMs excel in general conversation scenarios, they struggle with specialized navigation tasks, yielding suboptimal performance compared to specialized VLN models. We introduce FLAME (FLAMingo-Architected Embodied Agent), a novel Multimodal LLM-based agent and architecture designed for urban VLN tasks that efficiently handles multiple observations. Our approach implements a three-phase tuning technique for effective adaptation to navigation tasks, including single perception tuning for street view description, multiple perception tuning for trajectory summarization, and end-to-end training on VLN datasets. The augmented datasets are synthesized automatically. Experimental results demonstrate FLAME's superiority over existing methods, surpassing state-of-the-art methods by a 7.3% increase in task completion rate on Touchdown dataset. This work showcases the potential of Multimodal LLMs (MLLMs) in complex navigation tasks, representing an advancement towards practical applications of MLLMs in embodied AI. Project page: this https URL
- [402] arXiv:2408.11052 [pdf, html, other]
-
Title: Accelerating Goal-Conditioned RL Algorithms and ResearchMichał Bortkiewicz, Władek Pałucki, Vivek Myers, Tadeusz Dziarmaga, Tomasz Arczewski, Łukasz Kuciński, Benjamin EysenbachSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Self-supervision has the potential to transform reinforcement learning (RL), paralleling the breakthroughs it has enabled in other areas of machine learning. While self-supervised learning in other domains aims to find patterns in a fixed dataset, self-supervised goal-conditioned reinforcement learning (GCRL) agents discover new behaviors by learning from the goals achieved during unstructured interaction with the environment. However, these methods have failed to see similar success, both due to a lack of data from slow environments as well as a lack of stable algorithms. We take a step toward addressing both of these issues by releasing a high-performance codebase and benchmark JaxGCRL for self-supervised GCRL, enabling researchers to train agents for millions of environment steps in minutes on a single GPU. The key to this performance is a combination of GPU-accelerated environments and a stable, batched version of the contrastive reinforcement learning algorithm, based on an infoNCE objective, that effectively makes use of this increased data throughput. With this approach, we provide a foundation for future research in self-supervised GCRL, enabling researchers to quickly iterate on new ideas and evaluate them in a diverse set of challenging environments. Website + Code: this https URL
- [403] arXiv:2408.11053 [pdf, html, other]
-
Title: Revisiting VerilogEval: Newer LLMs, In-Context Learning, and Specification-to-RTL TasksComments: This paper revisits and improves the benchmark first presented in arXiv:2309.07544. Seven pages, three figuresSubjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)
The application of large-language models (LLMs) to digital hardware code generation is an emerging field. Most LLMs are primarily trained on natural language and software code. Hardware code, such as Verilog, represents only a small portion of the training data and few hardware benchmarks exist. To address this gap, the open-source VerilogEval benchmark was released in 2023, providing a consistent evaluation framework for LLMs on code completion tasks. It was tested on state-of-the-art models at the time including GPT-4. However, VerilogEval and other Verilog generation benchmarks lack failure analysis and, in present form, are not conducive to exploring prompting techniques. Also, since VerilogEval's release, both commercial and open-source models have seen continued development.
In this work, we evaluate new commercial and open-source models of varying sizes against an improved VerilogEval benchmark suite. We enhance VerilogEval's infrastructure and dataset by automatically classifying failures, introduce new prompts for supporting in-context learning (ICL) examples, and extend the supported tasks to specification-to-RTL translation. We find a measurable improvement in commercial state-of-the-art models, with GPT-4 Turbo achieving a 59% pass rate on spec-to-RTL tasks. We also study the performance of open-source and domain-specific models that have emerged, and demonstrate that models can benefit substantially from ICL. We find that recently-released Llama 3.1 405B achieves a pass rate of 58%, effectively matching that of GPT-4 Turbo, and that the much smaller domain-specific RTL-Coder 6.7B models achieve an impressive 37% pass rate. However, prompt engineering is key to achieving good pass rates, and varies widely with model and task. A benchmark infrastructure that allows for prompt engineering and failure analysis is key to continued model development and deployment. - [404] arXiv:2408.11054 [pdf, html, other]
-
Title: NeCo: Improving DINOv2's spatial representations in 19 GPU hours with Patch Neighbor ConsistencyComments: Preprint. The webpage is accessible at: this https URLSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
We propose sorting patch representations across views as a novel self-supervised learning signal to improve pretrained representations. To this end, we introduce NeCo: Patch Neighbor Consistency, a novel training loss that enforces patch-level nearest neighbor consistency across a student and teacher model, relative to reference batches. Our method leverages a differentiable sorting method applied on top of pretrained representations, such as DINOv2-registers to bootstrap the learning signal and further improve upon them. This dense post-pretraining leads to superior performance across various models and datasets, despite requiring only 19 hours on a single GPU. We demonstrate that this method generates high-quality dense feature encoders and establish several new state-of-the-art results: +5.5% and + 6% for non-parametric in-context semantic segmentation on ADE20k and Pascal VOC, and +7.2% and +5.7% for linear segmentation evaluations on COCO-Things and -Stuff.
- [405] arXiv:2408.11055 [pdf, html, other]
-
Title: Prompt-Guided Image-Adaptive Neural Implicit Lookup Tables for Interpretable Image EnhancementComments: Accepted to ACM Multimedia 2024Subjects: Computer Vision and Pattern Recognition (cs.CV)
In this paper, we delve into the concept of interpretable image enhancement, a technique that enhances image quality by adjusting filter parameters with easily understandable names such as "Exposure" and "Contrast". Unlike using predefined image editing filters, our framework utilizes learnable filters that acquire interpretable names through training. Our contribution is two-fold. Firstly, we introduce a novel filter architecture called an image-adaptive neural implicit lookup table, which uses a multilayer perceptron to implicitly define the transformation from input feature space to output color space. By incorporating image-adaptive parameters directly into the input features, we achieve highly expressive filters. Secondly, we introduce a prompt guidance loss to assign interpretable names to each filter. We evaluate visual impressions of enhancement results, such as exposure and contrast, using a vision and language model along with guiding prompts. We define a constraint to ensure that each filter affects only the targeted visual impression without influencing other attributes, which allows us to obtain the desired filter effects. Experimental results show that our method outperforms existing predefined filter-based methods, thanks to the filters optimized to predict target results. Our source code is available at this https URL.
New submissions for Wednesday, 21 August 2024 (showing 405 of 405 entries )
- [406] arXiv:2408.10228 (cross-list from eess.SP) [pdf, html, other]
-
Title: ECG Unveiled: Analysis of Client Re-identification Risks in Real-World ECG DatasetsZiyu Wang, Anil Kanduri, Seyed Amir Hossein Aqajari, Salar Jafarlou, Sanaz R. Mousavi, Pasi Liljeberg, Shaista Malik, Amir M. RahmaniSubjects: Signal Processing (eess.SP); Machine Learning (cs.LG)
While ECG data is crucial for diagnosing and monitoring heart conditions, it also contains unique biometric information that poses significant privacy risks. Existing ECG re-identification studies rely on exhaustive analysis of numerous deep learning features, confining to ad-hoc explainability towards clinicians decision making. In this work, we delve into explainability of ECG re-identification risks using transparent machine learning models. We use SHapley Additive exPlanations (SHAP) analysis to identify and explain the key features contributing to re-identification risks. We conduct an empirical analysis of identity re-identification risks using ECG data from five diverse real-world datasets, encompassing 223 participants. By employing transparent machine learning models, we reveal the diversity among different ECG features in contributing towards re-identification of individuals with an accuracy of 0.76 for gender, 0.67 for age group, and 0.82 for participant ID re-identification. Our approach provides valuable insights for clinical experts and guides the development of effective privacy-preserving mechanisms. Further, our findings emphasize the necessity for robust privacy measures in real-world health applications and offer detailed, actionable insights for enhancing data anonymization techniques.
- [407] arXiv:2408.10235 (cross-list from eess.SP) [pdf, html, other]
-
Title: Multi-Source EEG Emotion Recognition via Dynamic Contrastive Domain AdaptationSubjects: Signal Processing (eess.SP); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
Electroencephalography (EEG) provides reliable indications of human cognition and mental states. Accurate emotion recognition from EEG remains challenging due to signal variations among individuals and across measurement sessions. To address these challenges, we introduce a multi-source dynamic contrastive domain adaptation method (MS-DCDA), which models coarse-grained inter-domain and fine-grained intra-class adaptations through a multi-branch contrastive neural network and contrastive sub-domain discrepancy learning. Our model leverages domain knowledge from each individual source and a complementary source ensemble and uses dynamically weighted learning to achieve an optimal tradeoff between domain transferability and discriminability. The proposed MS-DCDA model was evaluated using the SEED and SEED-IV datasets, achieving respectively the highest mean accuracies of $90.84\%$ and $78.49\%$ in cross-subject experiments as well as $95.82\%$ and $82.25\%$ in cross-session experiments. Our model outperforms several alternative domain adaptation methods in recognition accuracy, inter-class margin, and intra-class compactness. Our study also suggests greater emotional sensitivity in the frontal and parietal brain lobes, providing insights for mental health interventions, personalized medicine, and development of preventive strategies.
- [408] arXiv:2408.10236 (cross-list from eess.IV) [pdf, html, other]
-
Title: AID-DTI: Accelerating High-fidelity Diffusion Tensor Imaging with Detail-preserving Model-based Deep LearningComments: 12 pages, 3 figures, MICCAI 2024 Workshop on Computational Diffusion MRI. arXiv admin note: text overlap with arXiv:2401.01693, arXiv:2405.03159Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Deep learning has shown great potential in accelerating diffusion tensor imaging (DTI). Nevertheless, existing methods tend to suffer from Rician noise and eddy current, leading to detail loss in reconstructing the DTI-derived parametric maps especially when sparsely sampled q-space data are used. To address this, this paper proposes a novel method, AID-DTI (\textbf{A}ccelerating h\textbf{I}gh fi\textbf{D}elity \textbf{D}iffusion \textbf{T}ensor \textbf{I}maging), to facilitate fast and accurate DTI with only six measurements. AID-DTI is equipped with a newly designed Singular Value Decomposition-based regularizer, which can effectively capture fine details while suppressing noise during network training by exploiting the correlation across DTI-derived parameters. Additionally, we introduce a Nesterov-based adaptive learning algorithm that optimizes the regularization parameter dynamically to enhance the performance. AID-DTI is an extendable framework capable of incorporating flexible network architecture. Experimental results on Human Connectome Project (HCP) data consistently demonstrate that the proposed method estimates DTI parameter maps with fine-grained details and outperforms other state-of-the-art methods both quantitatively and qualitatively.
- [409] arXiv:2408.10247 (cross-list from q-bio.BM) [pdf, html, other]
-
Title: MetaEnzyme: Meta Pan-Enzyme Learning for Task-Adaptive RedesignComments: Accepted to ACM Multimedia 2024Subjects: Biomolecules (q-bio.BM); Artificial Intelligence (cs.AI)
Enzyme design plays a crucial role in both industrial production and biology. However, this field faces challenges due to the lack of comprehensive benchmarks and the complexity of enzyme design tasks, leading to a dearth of systematic research. Consequently, computational enzyme design is relatively overlooked within the broader protein domain and remains in its early stages. In this work, we address these challenges by introducing MetaEnzyme, a staged and unified enzyme design framework. We begin by employing a cross-modal structure-to-sequence transformation architecture, as the feature-driven starting point to obtain initial robust protein representation. Subsequently, we leverage domain adaptive techniques to generalize specific enzyme design tasks under low-resource conditions. MetaEnzyme focuses on three fundamental low-resource enzyme redesign tasks: functional design (FuncDesign), mutation design (MutDesign), and sequence generation design (SeqDesign). Through novel unified paradigm and enhanced representation capabilities, MetaEnzyme demonstrates adaptability to diverse enzyme design tasks, yielding outstanding results. Wet lab experiments further validate these findings, reinforcing the efficacy of the redesign process.
- [410] arXiv:2408.10255 (cross-list from q-fin.ST) [pdf, html, other]
-
Title: Large Investment ModelComments: 20 pages, 10 figures, 2 tablesSubjects: Statistical Finance (q-fin.ST); Artificial Intelligence (cs.AI); Computational Finance (q-fin.CP)
Traditional quantitative investment research is encountering diminishing returns alongside rising labor and time costs. To overcome these challenges, we introduce the Large Investment Model (LIM), a novel research paradigm designed to enhance both performance and efficiency at scale. LIM employs end-to-end learning and universal modeling to create an upstream foundation model capable of autonomously learning comprehensive signal patterns from diverse financial data spanning multiple exchanges, instruments, and frequencies. These "global patterns" are subsequently transferred to downstream strategy modeling, optimizing performance for specific tasks. We detail the system architecture design of LIM, address the technical challenges inherent in this approach, and outline potential directions for future research. The advantages of LIM are demonstrated through a series of numerical experiments on cross-instrument prediction for commodity futures trading, leveraging insights from stock markets.
- [411] arXiv:2408.10265 (cross-list from quant-ph) [pdf, html, other]
-
Title: Distributed and Secure Kernel-Based Quantum Machine LearningComments: This paper contains 12 pages, 4 figures and 1 table. For associated supplementary code, see this https URL. The paper has been submitted to AAAI 2025Subjects: Quantum Physics (quant-ph); Machine Learning (cs.LG)
Quantum computing promises to revolutionize machine learning, offering significant efficiency gains in tasks such as clustering and distance estimation. Additionally, it provides enhanced security through fundamental principles like the measurement postulate and the no-cloning theorem, enabling secure protocols such as quantum teleportation and quantum key distribution. While advancements in secure quantum machine learning are notable, the development of secure and distributed quantum analogues of kernel-based machine learning techniques remains underexplored.
In this work, we present a novel approach for securely computing common kernels, including polynomial, radial basis function (RBF), and Laplacian kernels, when data is distributed, using quantum feature maps. Our methodology introduces a robust framework that leverages quantum teleportation to ensure secure and distributed kernel learning. The proposed architecture is validated using IBM's Qiskit Aer Simulator on various public datasets. - [412] arXiv:2408.10274 (cross-list from quant-ph) [pdf, html, other]
-
Title: Benchmarking quantum machine learning kernel training for classification tasksComments: 13 pagesSubjects: Quantum Physics (quant-ph); Machine Learning (cs.LG)
Quantum-enhanced machine learning is a rapidly evolving field that aims to leverage the unique properties of quantum mechanics to enhance classical machine learning. However, the practical applicability of these methods remains an open question, particularly in the context of real-world datasets and the limitations of current quantum hardware. This work performs a benchmark study of Quantum Kernel Estimation (QKE) and Quantum Kernel Training (QKT) with a focus on classification tasks. Through a series of experiments, the versatility and generalization capabilities of two quantum feature mappings, namely ZZFeatureMap and CovariantFeatureMap, are analyzed in this context. Remarkably, these feature maps have been proposed in the literature under the conjecture of possible near-term quantum advantage and have shown promising performance in ad-hoc datasets. This study explores both artificial and established reference datasets and incorporates classical machine learning methods, specifically Support Vector Machines (SVMs) and logistic regression, as baseline comparisons. Experimental results indicate that quantum methods exhibit varying performance across different datasets. While they outperform classical methods in ad-hoc datasets, they frequently encounter difficulties in generalizing to unseen test data when dealing with reference classical datasets, even if achieving high classification accuracy on the training data. It is suggested that the choice of the feature mapping and the optimization of kernel parameters through QKT are critical for maximizing the effectiveness of quantum methods.
- [413] arXiv:2408.10283 (cross-list from eess.IV) [pdf, html, other]
-
Title: SDE-based Multiplicative Noise RemovalComments: 9 pages, 4 figuresSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Multiplicative noise, also known as speckle or pepper noise, commonly affects images produced by synthetic aperture radar (SAR), lasers, or optical lenses. Unlike additive noise, which typically arises from thermal processes or external factors, multiplicative noise is inherent to the system, originating from the fluctuation in diffuse reflections. These fluctuations result in multiple copies of the same signal with varying magnitudes being combined. Consequently, despeckling, or removing multiplicative noise, necessitates different techniques compared to those used for additive noise removal.
In this paper, we propose a novel approach using Stochastic Differential Equations based diffusion models to address multiplicative noise. We demonstrate that multiplicative noise can be effectively modeled as a Geometric Brownian Motion process in the logarithmic domain. Utilizing the Fokker-Planck equation, we derive the corresponding reverse process for image denoising. To validate our method, we conduct extensive experiments on two different datasets, comparing our approach to both classical signal processing techniques and contemporary CNN-based noise removal models. Our results indicate that the proposed method significantly outperforms existing methods on perception-based metrics such as FID and LPIPS, while maintaining competitive performance on traditional metrics like PSNR and SSIM. - [414] arXiv:2408.10287 (cross-list from physics.optics) [pdf, other]
-
Title: Recognizing Beam Profiles from Silicon Photonics Gratings using Transformer ModelSubjects: Optics (physics.optics); Artificial Intelligence (cs.AI); Image and Video Processing (eess.IV)
Over the past decade, there has been extensive work in developing integrated silicon photonics (SiPh) gratings for the optical addressing of trapped ion qubits in the ion trap quantum computing community. However, when viewing beam profiles from infrared (IR) cameras, it is often difficult to determine the corresponding heights where the beam profiles are located. In this work, we developed transformer models to recognize the corresponding height categories of beam profiles of light from SiPh gratings. The model is trained using two techniques: (1) input patches, and (2) input sequence. For model trained with input patches, the model achieved recognition accuracy of 0.938. Meanwhile, model trained with input sequence shows lower accuracy of 0.895. However, when repeating the model-training 150 cycles, model trained with input patches shows inconsistent accuracy ranges between 0.445 to 0.959, while model trained with input sequence exhibit higher accuracy values between 0.789 to 0.936. The obtained outcomes can be expanded to various applications, including auto-focusing of light beam and auto-adjustment of z-axis stage to acquire desired beam profiles.
- [415] arXiv:2408.10323 (cross-list from quant-ph) [pdf, html, other]
-
Title: SDP bounds on quantum codesComments: 51 pagesSubjects: Quantum Physics (quant-ph); Information Theory (cs.IT); Optimization and Control (math.OC)
This paper provides a semidefinite programming hierarchy based on state polynomial optimization to determine the existence of quantum codes with given parameters. The hierarchy is complete, in the sense that if a $(\!(n,K,\delta)\!)_2$ code does not exist then a level of the hierarchy is infeasible. It is not limited to stabilizer codes and thus applicable generally. While it is formally dimension-free, we restrict it to qubit codes through quasi-Clifford algebras. We derive the quantum analog of a range of classical results: first, from an intermediate level a Lovász bound for self-dual quantum codes is recovered. Second, a symmetrization of a minor variation of this Lovász bound recovers the quantum Delsarte bound. Third, a symmetry reduction using the Terwilliger algebra leads to semidefinite programming bounds of size $O(n^4)$. With this we give an alternative proof that there is no $(\!(7,1,4)\!)_2$ quantum code, and show that $(\!(8,9,3)\!)_2$ and $(\!(10,5,4)\!)_2$ codes do not exist.
- [416] arXiv:2408.10340 (cross-list from stat.ML) [pdf, html, other]
-
Title: Can an unsupervised clustering algorithm reproduce a categorization system?Comments: 9 pages, 4 tables 28 figuresSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistical Finance (q-fin.ST); Applications (stat.AP)
Peer analysis is a critical component of investment management, often relying on expert-provided categorization systems. These systems' consistency is questioned when they do not align with cohorts from unsupervised clustering algorithms optimized for various metrics. We investigate whether unsupervised clustering can reproduce ground truth classes in a labeled dataset, showing that success depends on feature selection and the chosen distance metric. Using toy datasets and fund categorization as real-world examples we demonstrate that accurately reproducing ground truth classes is challenging. We also highlight the limitations of standard clustering evaluation metrics in identifying the optimal number of clusters relative to the ground truth classes. We then show that if appropriate features are available in the dataset, and a proper distance metric is known (e.g., using a supervised Random Forest-based distance metric learning method), then an unsupervised clustering can indeed reproduce the ground truth classes as distinct clusters.
- [417] arXiv:2408.10359 (cross-list from econ.GN) [pdf, html, other]
-
Title: How Small is Big Enough? Open Labeled Datasets and the Development of Deep LearningSubjects: General Economics (econ.GN); Computers and Society (cs.CY)
We investigate the emergence of Deep Learning as a technoscientific field, emphasizing the role of open labeled datasets. Through qualitative and quantitative analyses, we evaluate the role of datasets like CIFAR-10 in advancing computer vision and object recognition, which are central to the Deep Learning revolution. Our findings highlight CIFAR-10's crucial role and enduring influence on the field, as well as its importance in teaching ML techniques. Results also indicate that dataset characteristics such as size, number of instances, and number of categories, were key factors. Econometric analysis confirms that CIFAR-10, a small-but-sufficiently-large open dataset, played a significant and lasting role in technological advancements and had a major function in the development of the early scientific literature as shown by citation metrics.
- [418] arXiv:2408.10361 (cross-list from eess.AS) [pdf, html, other]
-
Title: ASASVIcomtech: The Vicomtech-UGR Speech Deepfake Detection and SASV Systems for the ASVspoof5 ChallengeJuan M. Martín-Doñas, Eros Roselló, Angel M. Gomez, Aitor Álvarez, Iván López-Espejo, Antonio M. PeinadoComments: This paper was accepted at ASVspoof Workshop 2024Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)
This paper presents the work carried out by the ASASVIcomtech team, made up of researchers from Vicomtech and University of Granada, for the ASVspoof5 Challenge. The team has participated in both Track 1 (speech deepfake detection) and Track 2 (spoofing-aware speaker verification). This work started with an analysis of the challenge available data, which was regarded as an essential step to avoid later potential biases of the trained models, and whose main conclusions are presented here. With respect to the proposed approaches, a closed-condition system employing a deep complex convolutional recurrent architecture was developed for Track 1, although, unfortunately, no noteworthy results were achieved. On the other hand, different possibilities of open-condition systems, based on leveraging self-supervised models, augmented training data from previous challenges, and novel vocoders, were explored for both tracks, finally achieving very competitive results with an ensemble system.
- [419] arXiv:2408.10378 (cross-list from math.OC) [pdf, html, other]
-
Title: Finite-time input-to-state stability for infinite-dimensional systemsSubjects: Optimization and Control (math.OC); Systems and Control (eess.SY)
In this paper, we extend the notion of finite-time input-to-state stability (FTISS) for finite-dimensional systems to infinite-dimensional systems. More specifically, we first prove an FTISS Lyapunov theorem for a class of infinite-dimensional systems, namely, the existence of an FTISS Lyapunov functional (FTISS-LF) implies the FTISS of the system, and then, provide a sufficient condition for ensuring the existence of an FTISS-LF for a class of abstract infinite-dimensional systems under the framework of compact semigroup theory and Hilbert spaces. As an application of the FTISS Lyapunov theorem, we verify the FTISS for a class of parabolic PDEs involving sublinear terms and distributed in-domain disturbances. Since the nonlinear terms of the corresponding abstract system are not Lipschitz continuous, the well-posedness is proved based on the application of compact semigroup theory and the FTISS is assessed by using the Lyapunov method with the aid of an interpolation inequality. Numerical simulations are conducted to confirm the theoretical results.
- [420] arXiv:2408.10381 (cross-list from stat.ML) [pdf, html, other]
-
Title: Efficient Reinforcement Learning in Probabilistic Reward MachinesComments: 33 pages, 4 figuresSubjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
In this paper, we study reinforcement learning in Markov Decision Processes with Probabilistic Reward Machines (PRMs), a form of non-Markovian reward commonly found in robotics tasks. We design an algorithm for PRMs that achieves a regret bound of $\widetilde{O}(\sqrt{HOAT} + H^2O^2A^{3/2} + H\sqrt{T})$, where $H$ is the time horizon, $O$ is the number of observations, $A$ is the number of actions, and $T$ is the number of time-steps. This result improves over the best-known bound, $\widetilde{O}(H\sqrt{OAT})$ of \citet{pmlr-v206-bourel23a} for MDPs with Deterministic Reward Machines (DRMs), a special case of PRMs. When $T \geq H^3O^3A^2$ and $OA \geq H$, our regret bound leads to a regret of $\widetilde{O}(\sqrt{HOAT})$, which matches the established lower bound of $\Omega(\sqrt{HOAT})$ for MDPs with DRMs up to a logarithmic factor. To the best of our knowledge, this is the first efficient algorithm for PRMs. Additionally, we present a new simulation lemma for non-Markovian rewards, which enables reward-free exploration for any non-Markovian reward given access to an approximate planner. Complementing our theoretical findings, we show through extensive experiment evaluations that our algorithm indeed outperforms prior methods in various PRM environments.
- [421] arXiv:2408.10384 (cross-list from math.OC) [pdf, html, other]
-
Title: Empirical risk minimization for risk-neutral composite optimal control with applications to bang-bang controlComments: 21 pagesSubjects: Optimization and Control (math.OC); Numerical Analysis (math.NA)
Nonsmooth composite optimization problems under uncertainty are prevalent in various scientific and engineering applications. We consider risk-neutral composite optimal control problems, where the objective function is the sum of a potentially nonconvex expectation function and a nonsmooth convex function. To approximate the risk-neutral optimization problems, we use a Monte Carlo sample-based approach, study its asymptotic consistency, and derive nonasymptotic sample size estimates. Our analyses leverage problem structure commonly encountered in PDE-constrained optimization problems, including compact embeddings and growth conditions. We apply our findings to bang-bang-type optimal control problems and propose the use of a conditional gradient method to solve them effectively. We present numerical illustrations.
- [422] arXiv:2408.10484 (cross-list from quant-ph) [pdf, html, other]
-
Title: Dependable Classical-Quantum Computer Systems EngineeringEdoardo Giusto (1), Santiago Nuñez-Corrales (2 and 3), Phuong Cao (2), Alessandro Cilardo (1), Ravishankar K. Iyer (3), Weiwen Jiang (6), Paolo Rech (4), Flavio Vella (4), Bartolomeo Montrucchio (5), Samudra Dasgupta (7), Travis S. Humble (7) ((1) University of Naples Federico II, Italy, (2) National Center for Supercomputing Applications, Urbana, IL, US, (3) University of Illinois Urbana-Champaign, US, (4) University of Trento, Italy, (5) Politecnico di Torino, Italy, (6) George Mason University, VA, US, (7) Oak Ridge National Laboratory, TN, US)Subjects: Quantum Physics (quant-ph); Emerging Technologies (cs.ET)
Quantum Computing (QC) offers the potential to enhance traditional High-Performance Computing (HPC) workloads by leveraging the unique properties of quantum computers, leading to the emergence of a new paradigm: HPC-QC. While this integration presents new opportunities, it also brings novel challenges, particularly in ensuring the dependability of such hybrid systems. This paper aims to identify integration challenges, anticipate failures, and foster a diverse co-design for HPC-QC systems by bringing together QC, cloud computing, HPC, and network security. The focus of this emerging inter-disciplinary effort is to develop engineering principles that ensure the dependability of hybrid systems, aiming for a more prescriptive co-design cycle. Our framework will help to prevent design pitfalls and accelerate the maturation of the QC technology ecosystem. Key aspects include building resilient HPC-QC systems, analyzing the applicability of conventional techniques to the quantum domain, and exploring the complexity of scaling in such hybrid systems. This underscores the need for performance-reliability metrics specific to this new computational paradigm.
- [423] arXiv:2408.10498 (cross-list from eess.IV) [pdf, html, other]
-
Title: Cervical Cancer Detection Using Multi-Branch Deep Learning ModelSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Cervical cancer is a crucial global health concern for women, and the persistent infection of High-risk HPV mainly triggers this remains a global health challenge, with young women diagnosis rates soaring from 10\% to 40\% over three decades. While Pap smear screening is a prevalent diagnostic method, visual image analysis can be lengthy and often leads to mistakes. Early detection of the disease can contribute significantly to improving patient outcomes. In recent decades, many researchers have employed machine learning techniques that achieved promise in cervical cancer detection processes based on medical images. In recent years, many researchers have employed various deep-learning techniques to achieve high-performance accuracy in detecting cervical cancer but are still facing various challenges. This research proposes an innovative and novel approach to automate cervical cancer image classification using Multi-Head Self-Attention (MHSA) and convolutional neural networks (CNNs). The proposed method leverages the strengths of both MHSA mechanisms and CNN to effectively capture both local and global features within cervical images in two streams. MHSA facilitates the model's ability to focus on relevant regions of interest, while CNN extracts hierarchical features that contribute to accurate classification. Finally, we combined the two stream features and fed them into the classification module to refine the feature and the classification. To evaluate the performance of the proposed approach, we used the SIPaKMeD dataset, which classifies cervical cells into five categories. Our model achieved a remarkable accuracy of 98.522\%. This performance has high recognition accuracy of medical image classification and holds promise for its applicability in other medical image recognition tasks.
- [424] arXiv:2408.10502 (cross-list from stat.ML) [pdf, html, other]
-
Title: Asymptotic Classification Error for Heavy-Tailed Renewal ProcessesComments: 11 pages, 2 figuresSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
Despite the widespread occurrence of classification problems and the increasing collection of point process data across many disciplines, study of error probability for point process classification only emerged very recently. Here, we consider classification of renewal processes. We obtain asymptotic expressions for the Bhattacharyya bound on misclassification error probabilities for heavy-tailed renewal processes.
- [425] arXiv:2408.10567 (cross-list from q-bio.NC) [pdf, html, other]
-
Title: Prompt Your Brain: Scaffold Prompt Tuning for Efficient Adaptation of fMRI Pre-trained ModelComments: MICCAI 2024Subjects: Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
We introduce Scaffold Prompt Tuning (ScaPT), a novel prompt-based framework for adapting large-scale functional magnetic resonance imaging (fMRI) pre-trained models to downstream tasks, with high parameter efficiency and improved performance compared to fine-tuning and baselines for prompt tuning. The full fine-tuning updates all pre-trained parameters, which may distort the learned feature space and lead to overfitting with limited training data which is common in fMRI fields. In contrast, we design a hierarchical prompt structure that transfers the knowledge learned from high-resource tasks to low-resource ones. This structure, equipped with a Deeply-conditioned Input-Prompt (DIP) mapping module, allows for efficient adaptation by updating only 2% of the trainable parameters. The framework enhances semantic interpretability through attention mechanisms between inputs and prompts, and it clusters prompts in the latent space in alignment with prior knowledge. Experiments on public resting state fMRI datasets reveal ScaPT outperforms fine-tuning and multitask-based prompt tuning in neurodegenerative diseases diagnosis/prognosis and personality trait prediction, even with fewer than 20 participants. It highlights ScaPT's efficiency in adapting pre-trained fMRI models to low-resource tasks.
- [426] arXiv:2408.10572 (cross-list from eess.IV) [pdf, other]
-
Title: A Tutorial on Explainable Image Classification for Dementia Stages Using Convolutional Neural Network and Gradient-weighted Class Activation MappingComments: 15 pages, 11 figures, 3 tablesSubjects: Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
This paper presents a tutorial of an explainable approach using Convolutional Neural Network (CNN) and Gradient-weighted Class Activation Mapping (Grad-CAM) to classify four progressive dementia stages based on open MRI brain images. The detailed implementation steps are demonstrated with an explanation. Whilst the proposed CNN architecture is demonstrated to achieve more than 99% accuracy for the test dataset, the computational procedure of CNN remains a black box. The visualisation based on Grad-CAM is attempted to explain such very high accuracy and may provide useful information for physicians. Future motivation based on this work is discussed.
- [427] arXiv:2408.10599 (cross-list from hep-ex) [pdf, html, other]
-
Title: Vision Calorimeter for Anti-neutron Reconstruction: A BaselineHongtian Yu, Yangu Li, Mingrui Wu, Letian Shen, Yue Liu, Yunxuan Song, Qixiang Ye, Xiaorui Lyu, Yajun Mao, Yangheng Zheng, Yunfan LiuSubjects: High Energy Physics - Experiment (hep-ex); Computer Vision and Pattern Recognition (cs.CV)
In high-energy physics, anti-neutrons ($\bar{n}$) are fundamental particles that frequently appear as final-state particles, and the reconstruction of their kinematic properties provides an important probe for understanding the governing principles. However, this confronts significant challenges instrumentally with the electromagnetic calorimeter (EMC), a typical experimental sensor but recovering the information of incident $\bar{n}$ insufficiently. In this study, we introduce Vision Calorimeter (ViC), a baseline method for anti-neutron reconstruction that leverages deep learning detectors to analyze the implicit relationships between EMC responses and incident $\bar{n}$ characteristics. Our motivation lies in that energy distributions of $\bar{n}$ samples deposited in the EMC cell arrays embody rich contextual information. Converted to 2-D images, such contextual energy distributions can be used to predict the status of $\bar{n}$ ($i.e.$, incident position and momentum) through a deep learning detector along with pseudo bounding boxes and a specified training objective. Experimental results demonstrate that ViC substantially outperforms the conventional reconstruction approach, reducing the prediction error of incident position by 42.81% (from 17.31$^{\circ}$ to 9.90$^{\circ}$). More importantly, this study for the first time realizes the measurement of incident $\bar{n}$ momentum, underscoring the potential of deep learning detectors for particle reconstruction. Code is available at this https URL.
- [428] arXiv:2408.10636 (cross-list from eess.IV) [pdf, other]
-
Title: Generating Multi-frame Ultrawide-field Fluorescein Angiography from Ultrawide-field Color Imaging Improves Diabetic Retinopathy StratificationComments: 27 pages, 2 figuresSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Ultrawide-field fluorescein angiography (UWF-FA) facilitates diabetic retinopathy (DR) detection by providing a clear visualization of peripheral retinal lesions. However, the intravenous dye injection with potential risks hamper its application. We aim to acquire dye-free UWF-FA images from noninvasive UWF color fundus (UWF-CF) images using generative artificial intelligence (GenAI) and evaluate its effectiveness in DR screening. A total of 18,321 UWF-FA images of different phases were registered with corresponding UWF-CF images and fed into a generative adversarial networks (GAN)-based model for training. The quality of generated UWF-FA images was evaluated through quantitative metrics and human evaluation. The DeepDRiD dataset was used to externally assess the contribution of generated UWF-FA images to DR classification, using area under the receiver operating characteristic curve (AUROC) as outcome metrics. The generated early, mid, and late phase UWF-FA images achieved high authenticity, with multi-scale similarity scores ranging from 0.70 to 0.91 and qualitative visual scores ranging from 1.64 to 1.98 (1=real UWF-FA quality). In fifty randomly selected images, 56% to 76% of the generated images were difficult to distinguish from real images in the Turing test. Moreover, adding these generated UWF-FA images for DR classification significantly increased the AUROC from 0.869 to 0.904 compared to the baseline model using UWF-CF images (P < .001). The model successfully generates realistic multi-frame UWF-FA images without intravenous dye injection. The generated UWF-FA enhanced DR stratification.
- [429] arXiv:2408.10656 (cross-list from eess.IV) [pdf, html, other]
-
Title: deepmriprep: Voxel-based Morphometry (VBM) Preprocessing via Deep Neural NetworksLukas Fisch, Nils R. Winter, Janik Goltermann, Carlotta Barkhau, Daniel Emden, Jan Ernsting, Maximilian Konowski, Ramona Leenings, Tiana Borgers, Kira Flinkenflügel, Dominik Grotegerd, Anna Kraus, Elisabeth J. Leehr, Susanne Meinert, Frederike Stein, Lea Teutenberg, Florian Thomas-Odenthal, Paula Usemann, Marco Hermesdorf, Hamidreza Jamalabadi, Andreas Jansen, Igor Nenadic, Benjamin Straube, Tilo Kircher, Klaus Berger, Benjamin Risse, Udo Dannlowski, Tim HahnSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Voxel-based Morphometry (VBM) has emerged as a powerful approach in neuroimaging research, utilized in over 7,000 studies since the year 2000. Using Magnetic Resonance Imaging (MRI) data, VBM assesses variations in the local density of brain tissue and examines its associations with biological and psychometric variables. Here, we present deepmriprep, a neural network-based pipeline that performs all necessary preprocessing steps for VBM analysis of T1-weighted MR images using deep neural networks. Utilizing the Graphics Processing Unit (GPU), deepmriprep is 37 times faster than CAT12, the leading VBM preprocessing toolbox. The proposed method matches CAT12 in accuracy for tissue segmentation and image registration across more than 100 datasets and shows strong correlations in VBM results. Tissue segmentation maps from deepmriprep have over 95% agreement with ground truth maps, and its non-linear registration, using supervised SYMNet, predicts smooth deformation fields comparable to CAT12. The high processing speed of deepmriprep enables rapid preprocessing of extensive datasets and thereby fosters the application of VBM analysis to large-scale neuroimaging studies and opens the door to real-time applications. Finally, deepmripreps straightforward, modular design enables researchers to easily understand, reuse, and advance the underlying methods, fostering further advancements in neuroimaging research. deepmriprep can be conveniently installed as a Python package and is publicly accessible at this https URL.
- [430] arXiv:2408.10665 (cross-list from eess.IV) [pdf, html, other]
-
Title: End-to-end learned Lossy Dynamic Point Cloud Attribute CompressionComments: 6 pages, accepted for presentation at 2024 IEEE International Conference on Image Processing (ICIP) 2024Subjects: Image and Video Processing (eess.IV); Machine Learning (cs.LG)
Recent advancements in point cloud compression have primarily emphasized geometry compression while comparatively fewer efforts have been dedicated to attribute compression. This study introduces an end-to-end learned dynamic lossy attribute coding approach, utilizing an efficient high-dimensional convolution to capture extensive inter-point dependencies. This enables the efficient projection of attribute features into latent variables. Subsequently, we employ a context model that leverage previous latent space in conjunction with an auto-regressive context model for encoding the latent tensor into a bitstream. Evaluation of our method on widely utilized point cloud datasets from the MPEG and Microsoft demonstrates its superior performance compared to the core attribute compression module Region-Adaptive Hierarchical Transform method from MPEG Geometry Point Cloud Compression with 38.1% Bjontegaard Delta-rate saving in average while ensuring a low-complexity encoding/decoding.
- [431] arXiv:2408.10726 (cross-list from quant-ph) [pdf, other]
-
Title: Quantum Artificial Intelligence: A Brief SurveyComments: 21 pages, 5 figuresSubjects: Quantum Physics (quant-ph); Artificial Intelligence (cs.AI)
Quantum Artificial Intelligence (QAI) is the intersection of quantum computing and AI, a technological synergy with expected significant benefits for both. In this paper, we provide a brief overview of what has been achieved in QAI so far and point to some open questions for future research. In particular, we summarize some major key findings on the feasability and the potential of using quantum computing for solving computationally hard problems in various subfields of AI, and vice versa, the leveraging of AI methods for building and operating quantum computing devices.
- [432] arXiv:2408.10733 (cross-list from eess.IV) [pdf, html, other]
-
Title: Classification of Endoscopy and Video Capsule Images using CNN-Transformer ModelSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Gastrointestinal cancer is a leading cause of cancer-related incidence and death, making it crucial to develop novel computer-aided diagnosis systems for early detection and enhanced treatment. Traditional approaches rely on the expertise of gastroenterologists to identify diseases; however, this process is subjective, and interpretation can vary even among expert clinicians. Considering recent advancements in classifying gastrointestinal anomalies and landmarks in endoscopic and video capsule endoscopy images, this study proposes a hybrid model that combines the advantages of Transformers and Convolutional Neural Networks (CNNs) to enhance classification performance. Our model utilizes DenseNet201 as a CNN branch to extract local features and integrates a Swin Transformer branch for global feature understanding, combining both to perform the classification task. For the GastroVision dataset, our proposed model demonstrates excellent performance with Precision, Recall, F1 score, Accuracy, and Matthews Correlation Coefficient (MCC) of 0.8320, 0.8386, 0.8324, 0.8386, and 0.8191, respectively, showcasing its robustness against class imbalance and surpassing other CNNs as well as the Swin Transformer model. Similarly, for the Kvasir-Capsule, a large video capsule endoscopy dataset, our model outperforms all others, achieving overall Precision, Recall, F1 score, Accuracy, and MCC of 0.7007, 0.7239, 0.6900, 0.7239, and 0.3871. Moreover, we generated saliency maps to explain our model's focus areas, demonstrating its reliable decision-making process. The results underscore the potential of our hybrid CNN-Transformer model in aiding the early and accurate detection of gastrointestinal (GI) anomalies.
- [433] arXiv:2408.10743 (cross-list from quant-ph) [pdf, html, other]
-
Title: Fast Algorithms and Implementations for Computing the Minimum Distance of Quantum CodesComments: 14 pages, 7 figures, 1 tableSubjects: Quantum Physics (quant-ph); Computational Engineering, Finance, and Science (cs.CE); Information Theory (cs.IT); Mathematical Software (cs.MS)
The distance of a stabilizer quantum code is a very important feature since it determines the number of errors that can be detected and corrected. We present three new fast algorithms and implementations for computing the symplectic distance of the associated classical code. Our new algorithms are based on the Brouwer-Zimmermann algorithm. Our experimental study shows that these new implementations are much faster than current state-of-the-art licensed implementations on single-core processors, multicore processors, and shared-memory multiprocessors. In the most computationally-demanding cases, the performance gain in the computational time can be larger than one order of magnitude. The experimental study also shows a good scalability on shared-memory parallel architectures.
- [434] arXiv:2408.10769 (cross-list from q-bio.PE) [pdf, other]
-
Title: A Wild Sheep Chase Through an OrchardComments: 27 pages, 13 figuresSubjects: Populations and Evolution (q-bio.PE); Computational Complexity (cs.CC); Combinatorics (math.CO)
Orchards are a biologically relevant class of phylogenetic networks as they can describe treelike evolutionary histories augmented with horizontal transfer events. Moreover, the class has attractive mathematical characterizations that can be exploited algorithmically. On the other hand, undirected orchard networks have hardly been studied yet. Here, we prove that deciding whether an undirected, binary phylogenetic network is an orchard -- or equivalently, whether it has an orientation that makes it a rooted orchard -- is NP-hard. For this, we introduce a new characterization of undirected orchards which could be useful for proving positive results.
- [435] arXiv:2408.10771 (cross-list from eess.AS) [pdf, html, other]
-
Title: SSL-TTS: Leveraging Self-Supervised Embeddings and kNN Retrieval for Zero-Shot Multi-speaker TTSComments: Submitted to IEEE Signal Processing LettersSubjects: Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Sound (cs.SD)
While recent zero-shot multispeaker text-to-speech (TTS) models achieve impressive results, they typically rely on extensive transcribed speech datasets from numerous speakers and intricate training pipelines. Meanwhile, self-supervised learning (SSL) speech features have emerged as effective intermediate representations for TTS. It was also observed that SSL features from different speakers that are linearly close share phonetic information while maintaining individual speaker identity, which enables straight-forward and robust voice cloning. In this study, we introduce SSL-TTS, a lightweight and efficient zero-shot TTS framework trained on transcribed speech from a single speaker. SSL-TTS leverages SSL features and retrieval methods for simple and robust zero-shot multi-speaker synthesis. Objective and subjective evaluations show that our approach achieves performance comparable to state-of-the-art models that require significantly larger training datasets. The low training data requirements mean that SSL-TTS is well suited for the development of multi-speaker TTS systems for low-resource domains and languages. We also introduce an interpolation parameter which enables fine control over the output speech by blending voices. Demo samples are available at this https URL
- [436] arXiv:2408.10816 (cross-list from eess.SP) [pdf, html, other]
-
Title: Deep Learning-based Classification of Dementia using Image Representation of Subcortical SignalsShivani Ranjan, Ayush Tripathi, Harshal Shende, Robin Badal, Amit Kumar, Pramod Yadav, Deepak Joshi, Lalan KumarSubjects: Signal Processing (eess.SP); Machine Learning (cs.LG)
Dementia is a neurological syndrome marked by cognitive decline. Alzheimer's disease (AD) and Frontotemporal dementia (FTD) are the common forms of dementia, each with distinct progression patterns. EEG, a non-invasive tool for recording brain activity, has shown potential in distinguishing AD from FTD and mild cognitive impairment (MCI). Previous studies have utilized various EEG features, such as subband power and connectivity patterns to differentiate these conditions. However, artifacts in EEG signals can obscure crucial information, necessitating advanced signal processing techniques. This study aims to develop a deep learning-based classification system for dementia by analyzing scout time-series signals from deep brain regions, specifically the hippocampus, amygdala, and thalamus. The study utilizes scout time series extracted via the standardized low-resolution brain electromagnetic tomography (sLORETA) technique. The time series is converted to image representations using continuous wavelet transform (CWT) and fed as input to deep learning models. Two high-density EEG datasets are utilized to check for the efficacy of the proposed method: the online BrainLat dataset (comprising AD, FTD, and healthy controls (HC)) and the in-house IITD-AIIA dataset (including subjects with AD, MCI, and HC). Different classification strategies and classifier combinations have been utilized for the accurate mapping of classes on both datasets. The best results were achieved by using a product of probabilities from classifiers for left and right subcortical regions in conjunction with the DenseNet model architecture. It yields accuracies of 94.17$\%$ and 77.72$\%$ on the BrainLat and IITD-AIIA datasets, respectively. This highlights the potential of this approach for early and accurate differentiation of neurodegenerative disorders.
- [437] arXiv:2408.10827 (cross-list from eess.IV) [pdf, html, other]
-
Title: CO2Wounds-V2: Extended Chronic Wounds Dataset From Leprosy PatientsComments: 2024 IEEE International Conference on Image Processing (ICIP 2024)Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Chronic wounds pose an ongoing health concern globally, largely due to the prevalence of conditions such as diabetes and leprosy's disease. The standard method of monitoring these wounds involves visual inspection by healthcare professionals, a practice that could present challenges for patients in remote areas with inadequate transportation and healthcare infrastructure. This has led to the development of algorithms designed for the analysis and follow-up of wound images, which perform image-processing tasks such as classification, detection, and segmentation. However, the effectiveness of these algorithms heavily depends on the availability of comprehensive and varied wound image data, which is usually scarce. This paper introduces the CO2Wounds-V2 dataset, an extended collection of RGB wound images from leprosy patients with their corresponding semantic segmentation annotations, aiming to enhance the development and testing of image-processing algorithms in the medical field.
- [438] arXiv:2408.10854 (cross-list from physics.ao-ph) [pdf, html, other]
-
Title: MambaDS: Near-Surface Meteorological Field Downscaling with Topography Constrained Selective State Space ModelingSubjects: Atmospheric and Oceanic Physics (physics.ao-ph); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
In an era of frequent extreme weather and global warming, obtaining precise, fine-grained near-surface weather forecasts is increasingly essential for human activities. Downscaling (DS), a crucial task in meteorological forecasting, enables the reconstruction of high-resolution meteorological states for target regions from global-scale forecast results. Previous downscaling methods, inspired by CNN and Transformer-based super-resolution models, lacked tailored designs for meteorology and encountered structural limitations. Notably, they failed to efficiently integrate topography, a crucial prior in the downscaling process. In this paper, we address these limitations by pioneering the selective state space model into the meteorological field downscaling and propose a novel model called MambaDS. This model enhances the utilization of multivariable correlations and topography information, unique challenges in the downscaling process while retaining the advantages of Mamba in long-range dependency modeling and linear computational complexity. Through extensive experiments in both China mainland and the continental United States (CONUS), we validated that our proposed MambaDS achieves state-of-the-art results in three different types of meteorological field downscaling settings. We will release the code subsequently.
- [439] arXiv:2408.10871 (cross-list from astro-ph.IM) [pdf, html, other]
-
Title: Radio U-Net: a convolutional neural network to detect diffuse radio sources in galaxy clusters and beyondComments: Accepted by MNRAS, 16 pages, 9 figures, 2 tablesSubjects: Instrumentation and Methods for Astrophysics (astro-ph.IM); Cosmology and Nongalactic Astrophysics (astro-ph.CO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
The forthcoming generation of radio telescope arrays promises significant advancements in sensitivity and resolution, enabling the identification and characterization of many new faint and diffuse radio sources. Conventional manual cataloging methodologies are anticipated to be insufficient to exploit the capabilities of new radio surveys. Radio interferometric images of diffuse sources present a challenge for image segmentation tasks due to noise, artifacts, and embedded radio sources. In response to these challenges, we introduce Radio U-Net, a fully convolutional neural network based on the U-Net architecture. Radio U-Net is designed to detect faint and extended sources in radio surveys, such as radio halos, relics, and cosmic web filaments. Radio U-Net was trained on synthetic radio observations built upon cosmological simulations and then tested on a sample of galaxy clusters, where the detection of cluster diffuse radio sources relied on customized data reduction and visual inspection of LOFAR Two Metre Sky Survey (LoTSS) data. The 83% of clusters exhibiting diffuse radio emission were accurately identified, and the segmentation successfully recovered the morphology of the sources even in low-quality images. In a test sample comprising 246 galaxy clusters, we achieved a 73% accuracy rate in distinguishing between clusters with and without diffuse radio emission. Our results establish the applicability of Radio U-Net to extensive radio survey datasets, probing its efficiency on cutting-edge high-performance computing systems. This approach represents an advancement in optimizing the exploitation of forthcoming large radio surveys for scientific exploration.
- [440] arXiv:2408.10876 (cross-list from stat.AP) [pdf, html, other]
-
Title: More Options for Prelabor Rupture of Membranes, A Bayesian AnalysisComments: To appear in the 2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA)Subjects: Applications (stat.AP); Machine Learning (cs.LG)
An obstetric goal for a laboring mother is to achieve a vaginal delivery as it reduces the risks inherent in major abdominal surgery (i.e., a Cesarean section). Various medical interventions may be used by a physician to increase the likelihood of this occurring while minimizing maternal and fetal morbidity. However, patients with prelabor rupture of membranes (PROM) have only two commonly used options for cervical ripening, Pitocin and misoprostol. Little research exists on the benefits/risks for these two key drugs for PROM patients. A major limitation with most induction-of-labor related research is the inability to account for differences in \textit{Bishop scores} that are commonly used in obstetrical practice to determine the next induction agent offered to the patient. This creates a confounding factor, which biases the results, but has not been realized in the literature. In this work, we use a Bayesian model of the relationships between the relevant factors, informed by expert physicians, to separate the confounding variable from its actual impact. In doing so, we provide strong evidence that pitocin and buccal misoprostol are equally effective and safe; thus, physicians have more choice in clinical care than previously realized. This is particularly important for developing countries where neither medication may be readily available, and prior guidelines may create an artificial barrier to needed medication.
- [441] arXiv:2408.10892 (cross-list from math.CO) [pdf, other]
-
Title: Characterization of Circular-arc Graphs: II. McConnell FlippingComments: arXiv admin note: text overlap with arXiv:2403.01947Subjects: Combinatorics (math.CO); Data Structures and Algorithms (cs.DS)
McConnell [FOCS 2001] presented a flipping transformation from circular-arc graphs to interval graphs with certain patterns of representations. Beyond its algorithmic implications, this transformation is instrumental in identifying all minimal graphs that are not circular-arc graphs. We conduct a structural study of this transformation, and for $C_{4}$-free graphs, we achieve a complete characterization of these patterns. This characterization allows us, among other things, to identify all minimal chordal graphs that are not circular-arc graphs in a companion paper.
- [442] arXiv:2408.10958 (cross-list from physics.ao-ph) [pdf, html, other]
-
Title: Kilometer-Scale Convection Allowing Model Emulation using Generative Diffusion ModelingJaideep Pathak, Yair Cohen, Piyush Garg, Peter Harrington, Noah Brenowitz, Dale Durran, Morteza Mardani, Arash Vahdat, Shaoming Xu, Karthik Kashinath, Michael PritchardSubjects: Atmospheric and Oceanic Physics (physics.ao-ph); Machine Learning (cs.LG)
Storm-scale convection-allowing models (CAMs) are an important tool for predicting the evolution of thunderstorms and mesoscale convective systems that result in damaging extreme weather. By explicitly resolving convective dynamics within the atmosphere they afford meteorologists the nuance needed to provide outlook on hazard. Deep learning models have thus far not proven skilful at km-scale atmospheric simulation, despite being competitive at coarser resolution with state-of-the-art global, medium-range weather forecasting. We present a generative diffusion model called StormCast, which emulates the high-resolution rapid refresh (HRRR) model-NOAA's state-of-the-art 3km operational CAM. StormCast autoregressively predicts 99 state variables at km scale using a 1-hour time step, with dense vertical resolution in the atmospheric boundary layer, conditioned on 26 synoptic variables. We present evidence of successfully learnt km-scale dynamics including competitive 1-6 hour forecast skill for composite radar reflectivity alongside physically realistic convective cluster evolution, moist updrafts, and cold pool morphology. StormCast predictions maintain realistic power spectra for multiple predicted variables across multi-hour forecasts. Together, these results establish the potential for autoregressive ML to emulate CAMs -- opening up new km-scale frontiers for regional ML weather prediction and future climate hazard dynamical downscaling.
- [443] arXiv:2408.10961 (cross-list from math.CO) [pdf, html, other]
-
Title: Combinatorial alphabet-dependent bounds for insdel codesComments: 20 pagesSubjects: Combinatorics (math.CO); Information Theory (cs.IT)
Error-correcting codes resilient to synchronization errors such as insertions and deletions are known as insdel codes. Due to their important applications in DNA storage and computational biology, insdel codes have recently become a focal point of research in coding theory.
In this paper, we present several new combinatorial upper and lower bounds on the maximum size of $q$-ary insdel codes. Our main upper bound is a sphere-packing bound obtained by solving a linear programming (LP) problem. It improves upon previous results for cases when the distance $d$ or the alphabet size $q$ is large. Our first lower bound is derived from a connection between insdel codes and matchings in special hypergraphs. This lower bound, together with our upper bound, shows that for fixed block length $n$ and edit distance $d$, when $q$ is sufficiently large, the maximum size of insdel codes is $ \frac{q^{n-\frac{d}{2}+1}}{{n\choose \frac{d}{2}-1}}(1 \pm o(1))$. The second lower bound refines Alon et al.'s recent logarithmic improvement on Levenshtein's GV-type bound and extends its applicability to large $q$ and $d$. - [444] arXiv:2408.10966 (cross-list from eess.IV) [pdf, html, other]
-
Title: ISLES'24: Improving final infarct prediction in ischemic stroke using multimodal imaging and clinical dataEzequiel de la Rosa, Ruisheng Su, Mauricio Reyes, Roland Wiest, Evamaria O. Riedel, Florian Kofler, Kaiyuan Yang, Hakim Baazaoui, David Robben, Susanne Wegener, Jan S. Kirschke, Benedikt Wiestler, Bjoern MenzeSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Accurate estimation of core (irreversibly damaged tissue) and penumbra (salvageable tissue) volumes is essential for ischemic stroke treatment decisions. Perfusion CT, the clinical standard, estimates these volumes but is affected by variations in deconvolution algorithms, implementations, and thresholds. Core tissue expands over time, with growth rates influenced by thrombus location, collateral circulation, and inherent patient-specific factors. Understanding this tissue growth is crucial for determining the need to transfer patients to comprehensive stroke centers, predicting the benefits of additional reperfusion attempts during mechanical thrombectomy, and forecasting final clinical outcomes. This work presents the ISLES'24 challenge, which addresses final post-treatment stroke infarct prediction from pre-interventional acute stroke imaging and clinical data. ISLES'24 establishes a unique 360-degree setting where all feasibly accessible clinical data are available for participants, including full CT acute stroke imaging, sub-acute follow-up MRI, and clinical tabular data. The contributions of this work are two-fold: first, we introduce a standardized benchmarking of final stroke infarct segmentation algorithms through the ISLES'24 challenge; second, we provide insights into infarct segmentation using multimodal imaging and clinical data strategies by identifying outperforming methods on a finely curated dataset. The outputs of this challenge are anticipated to enhance clinical decision-making and improve patient outcome predictions. All ISLES'24 materials, including data, performance evaluation scripts, and leading algorithmic strategies, are available to the research community following \url{this https URL}.
- [445] arXiv:2408.10976 (cross-list from stat.ML) [pdf, other]
-
Title: Kernel-Based Differentiable Learning of Non-Parametric Directed Acyclic Graphical ModelsComments: To be published in the Proceedings of Probabilistic Graphical Models (PGM) 2024Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Causal discovery amounts to learning a directed acyclic graph (DAG) that encodes a causal model. This model selection problem can be challenging due to its large combinatorial search space, particularly when dealing with non-parametric causal models. Recent research has sought to bypass the combinatorial search by reformulating causal discovery as a continuous optimization problem, employing constraints that ensure the acyclicity of the graph. In non-parametric settings, existing approaches typically rely on finite-dimensional approximations of the relationships between nodes, resulting in a score-based continuous optimization problem with a smooth acyclicity constraint. In this work, we develop an alternative approximation method by utilizing reproducing kernel Hilbert spaces (RKHS) and applying general sparsity-inducing regularization terms based on partial derivatives. Within this framework, we introduce an extended RKHS representer theorem. To enforce acyclicity, we advocate the log-determinant formulation of the acyclicity constraint and show its stability. Finally, we assess the performance of our proposed RKHS-DAGMA procedure through simulations and illustrative data analyses.
- [446] arXiv:2408.10987 (cross-list from eess.IV) [pdf, html, other]
-
Title: Denoising Plane Wave Ultrasound Images Using Diffusion Probabilistic ModelsSubjects: Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
Ultrasound plane wave imaging is a cutting-edge technique that enables high frame-rate imaging. However, one challenge associated with high frame-rate ultrasound imaging is the high noise associated with them, hindering their wider adoption. Therefore, the development of a denoising method becomes imperative to augment the quality of plane wave images. Drawing inspiration from Denoising Diffusion Probabilistic Models (DDPMs), our proposed solution aims to enhance plane wave image quality. Specifically, the method considers the distinction between low-angle and high-angle compounding plane waves as noise and effectively eliminates it by adapting a DDPM to beamformed radiofrequency (RF) data. The method underwent training using only 400 simulated images. In addition, our approach employs natural image segmentation masks as intensity maps for the generated images, resulting in accurate denoising for various anatomy shapes. The proposed method was assessed across simulation, phantom, and in vivo images. The results of the evaluations indicate that our approach not only enhances image quality on simulated data but also demonstrates effectiveness on phantom and in vivo data in terms of image quality. Comparative analysis with other methods underscores the superiority of our proposed method across various evaluation metrics. The source code and trained model will be released along with the dataset at: this http URL
- [447] arXiv:2408.10996 (cross-list from stat.ML) [pdf, html, other]
-
Title: Approximation Rates for Shallow ReLU$^k$ Neural Networks on Sobolev Spaces via the Radon TransformSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Numerical Analysis (math.NA)
Let $\Omega\subset \mathbb{R}^d$ be a bounded domain. We consider the problem of how efficiently shallow neural networks with the ReLU$^k$ activation function can approximate functions from Sobolev spaces $W^s(L_p(\Omega))$ with error measured in the $L_q(\Omega)$-norm. Utilizing the Radon transform and recent results from discrepancy theory, we provide a simple proof of nearly optimal approximation rates in a variety of cases, including when $q\leq p$, $p\geq 2$, and $s \leq k + (d+1)/2$. The rates we derive are optimal up to logarithmic factors, and significantly generalize existing results. An interesting consequence is that the adaptivity of shallow ReLU$^k$ neural networks enables them to obtain optimal approximation rates for smoothness up to order $s = k + (d+1)/2$, even though they represent piecewise polynomials of fixed degree $k$.
- [448] arXiv:2408.11019 (cross-list from q-bio.NC) [pdf, html, other]
-
Title: An Overlooked Role of Context-Sensitive DendritesSubjects: Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
To date, most dendritic studies have predominantly focused on the apical zone of pyramidal two-point neurons (TPNs) receiving only feedback (FB) connections from higher perceptual layers and using them for learning. Recent cellular neurophysiology and computational neuroscience studies suggests that the apical input (context), coming from feedback and lateral connections, is multifaceted and far more diverse, with greater implications for ongoing learning and processing in the brain than previously realized. In addition to the FB, the apical tuft receives signals from neighboring cells of the same network as proximal (P) context, other parts of the brain as distal (D) context, and overall coherent information across the network as universal (U) context. The integrated context (C) amplifies and suppresses the transmission of coherent and conflicting feedforward (FF) signals, respectively. Specifically, we show that complex context-sensitive (CS)-TPNs flexibly integrate C moment-by-moment with the FF somatic current at the soma such that the somatic current is amplified when both feedforward (FF) and C are coherent; otherwise, it is attenuated. This generates the event only when the FF and C currents are coherent, which is then translated into a singlet or a burst based on the FB information. Spiking simulation results show that this flexible integration of somatic and contextual currents enables the propagation of more coherent signals (bursts), making learning faster with fewer neurons. Similar behavior is observed when this functioning is used in conventional artificial networks, where orders of magnitude fewer neurons are required to process vast amounts of heterogeneous real-world audio-visual (AV) data trained using backpropagation (BP). The computational findings presented here demonstrate the universality of CS-TPNs, suggesting a dendritic narrative that was previously overlooked.
- [449] arXiv:2408.11040 (cross-list from math.OC) [pdf, html, other]
-
Title: Solving the Convex Flow ProblemSubjects: Optimization and Control (math.OC); Mathematical Software (cs.MS)
In this paper, we introduce the solver ConvexFlows for the convex flow problem first defined in the authors' previous work. In this problem, we aim to optimize a concave utility function depending on the flows over a graph. However, unlike the classic network flows literature, we also allow for a concave relationship between the input and output flows of edges. This nonlinear gain describes many physical phenomena, including losses in power network transmission lines. We outline an efficient algorithm for solving this problem which parallelizes over the graph edges. We provide an open source implementation of this algorithm in the Julia programming language package ConvexFlows.jl. This package includes an interface to easily specify these flow problems. We conclude by walking through an example of solving for an optimal power flow using ConvexFlows.
Cross submissions for Wednesday, 21 August 2024 (showing 44 of 44 entries )
- [450] arXiv:1909.12755 (replaced) [pdf, other]
-
Title: On the Approximation Ratio of the $k$-Opt and Lin-Kernighan AlgorithmSubjects: Discrete Mathematics (cs.DM); Data Structures and Algorithms (cs.DS); Combinatorics (math.CO)
The $k$-Opt and Lin-Kernighan algorithm are two of the most important local search approaches for the Metric TSP. Both start with an arbitrary tour and make local improvements in each step to get a shorter tour. We show that for any fixed $k\geq 3$ the approximation ratio of the $k$-Opt algorithm for Metric TSP is $O(\sqrt[k]{n})$. Assuming the Erdős girth conjecture, we prove a matching lower bound of $\Omega(\sqrt[k]{n})$. Unconditionally, we obtain matching bounds for $k=3,4,6$ and a lower bound of $\Omega(n^{\frac{2}{3k-3}})$. Our most general bounds depend on the values of a function from extremal graph theory and are tight up to a factor logarithmic in the number of vertices unconditionally. Moreover, all the upper bounds also apply to a parameterized generalization of the Lin-Kernighan algorithm with appropriate parameters. We also show that the approximation ratio of $k$-Opt for Graph TSP is $\Omega\left(\frac{\log(n)}{\log\log(n)}\right)$ and $O\left(\left(\frac{\log(n)}{\log\log(n)}\right)^{\log_2(9)+\epsilon}\right)$ for all $\epsilon>0$. For the (1,2)-TSP we give a lower bound of $\frac{11}{10}$ on the approximation ratio of the $k$-improv and $k$-Opt algorithm for arbitrary fixed $k$.
- [451] arXiv:2001.05989 (replaced) [pdf, html, other]
-
Title: Conformal e-predictionComments: This version is greatly expanded; 33 pages and 1 figureSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
This paper discusses a counterpart of conformal prediction for e-values, "conformal e-prediction". Conformal e-prediction is conceptually simpler and had been developed in the 1990s as precursor of conformal prediction. When conformal prediction emerged as result of replacing e-values by p-values, it seemed to have important advantages over conformal e-prediction without obvious disadvantages. This paper re-examines relations between conformal prediction and conformal e-prediction systematically from a modern perspective. Conformal e-prediction has advantages of its own, such as the ease of designing conditional conformal e-predictors and the guaranteed validity of cross-conformal e-predictors (whereas for cross-conformal predictors validity is only an empirical fact and can be broken with excessive randomization). Even where conformal prediction has clear advantages, conformal e-prediction can often emulate those advantages, more or less successfully. Conformal e-prediction can also serve as basis for testing; the resulting "conformal e-testing" looks very different from but inherits some strengths of conformal testing.
- [452] arXiv:2004.12571 (replaced) [pdf, html, other]
-
Title: Exploiting Defenses against GAN-Based Feature Inference Attacks in Federated LearningSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
Federated learning (FL) is a decentralized model training framework that aims to merge isolated data islands while maintaining data privacy. However, recent studies have revealed that Generative Adversarial Network (GAN) based attacks can be employed in FL to learn the distribution of private datasets and reconstruct recognizable images. In this paper, we exploit defenses against GAN-based attacks in FL and propose a framework, Anti-GAN, to prevent attackers from learning the real distribution of the victim's data. The core idea of Anti-GAN is to manipulate the visual features of private training images to make them indistinguishable to human eyes even restored by attackers. Specifically, Anti-GAN projects the private dataset onto a GAN's generator and combines the generated fake images with the actual images to create the training dataset, which is then used for federated model training. The experimental results demonstrate that Anti-GAN is effective in preventing attackers from learning the distribution of private images while causing minimal harm to the accuracy of the federated model.
- [453] arXiv:2010.08929 (replaced) [pdf, html, other]
-
Title: Self-stabilizing Graph Exploration by a Single AgentSubjects: Distributed, Parallel, and Cluster Computing (cs.DC)
In this paper, we present two self-stabilizing algorithms that enable a single (mobile) agent to explore graphs. The agent visits all nodes starting from any configuration, \ie regardless of the initial state of the agent, the initial states of all nodes, and the initial location of the agent. We evaluate the algorithms using two metrics: cover time, which is the number of moves required to visit all nodes, and memory usage, which includes the storage needed for the state of the agent and the state of each node. The first algorithm is randomized. Given an integer $c = \Omega(n)$, the cover time of this algorithm is optimal, \ie $O(m)$ in expectation, and the memory requirements for the agent and each node $v$ are $O(\log c)$ and $O(\log (c+\delta_v))$ bits, respectively, where $n$ and $m$ are the numbers of nodes and edges, respectively, and $\delta_v$ is the degree of $v$. The second algorithm is deterministic. It requires an input integer $k \ge \max(D,d_{\mathrm{max}})$, where $D$ and $d_{\mathrm{max}}$ are the diameter and the maximum degree of the graph, respectively. The cover time of this algorithm is $O(m + nD)$, and it uses $O(\log k)$ bits both for agent memory and each node.
- [454] arXiv:2103.14847 (replaced) [pdf, html, other]
-
Title: Approval-Based Committee Voting under Incomplete InformationSubjects: Computer Science and Game Theory (cs.GT); Computational Complexity (cs.CC); Multiagent Systems (cs.MA)
We investigate approval-based committee voting with incomplete information about the approval preferences of voters. We consider several models of incompleteness where each voter partitions the set of candidates into approved, disapproved, and unknown candidates, possibly with ordinal preference constraints among candidates in the latter category. This captures scenarios where voters have not evaluated all candidates and/or it is unknown where voters draw the threshold between approved and disapproved candidates. We study the complexity of some fundamental computational problems for a number of classic approval-based committee voting rules including Proportional Approval Voting and Chamberlin-Courant. These problems include determining whether a given set of candidates is a possible or necessary winning committee and whether a given candidate is possibly or necessarily a member of the winning committee. We also consider proportional representation axioms and the problem of deciding whether a given committee is possibly or necessarily representative.
- [455] arXiv:2201.07201 (replaced) [pdf, html, other]
-
Title: Bridging the Language Gap: An Empirical Study of Bindings for Open Source Machine Learning Libraries Across Software Package EcosystemsSubjects: Software Engineering (cs.SE); Machine Learning (cs.LG)
Open source machine learning (ML) libraries enable developers to integrate advanced ML functionality into their own applications. However, popular ML libraries, such as TensorFlow, are not available natively in all programming languages and software package ecosystems. Hence, developers who wish to use an ML library which is not available in their programming language or ecosystem of choice, may need to resort to using a so-called binding library (or binding). Bindings provide support across programming languages and package ecosystems for reusing a host library. For example, the Keras .NET binding provides support for the Keras library in the NuGet (.NET) ecosystem even though the Keras library was written in Python. In this paper, we collect 2,436 cross-ecosystem bindings for 546 ML libraries across 13 software package ecosystems by using an approach called BindFind, which can automatically identify bindings and link them to their host libraries. Furthermore, we conduct an in-depth study of 133 cross-ecosystem bindings and their development for 40 popular open source ML libraries. Our findings reveal that the majority of ML library bindings are maintained by the community, with npm being the most popular ecosystem for these bindings. Our study also indicates that most bindings cover only a limited range of the host library's releases, often experience considerable delays in supporting new releases, and have widespread technical lag. Our findings highlight key factors to consider for developers integrating bindings for ML libraries and open avenues for researchers to further investigate bindings in software package ecosystems.
- [456] arXiv:2202.00993 (replaced) [pdf, html, other]
-
Title: Normalise for Fairness: A Simple Normalisation Technique for Fairness in Regression Machine Learning ProblemsComments: Including references and appendices: 17 pages, 3 Figures, 5 TablesSubjects: Machine Learning (cs.LG); Computers and Society (cs.CY)
Algorithms and Machine Learning (ML) are increasingly affecting everyday life and several decision-making processes, where ML has an advantage due to scalability or superior performance. Fairness in such applications is crucial, where models should not discriminate their results based on race, gender, or other protected groups. This is especially crucial for models affecting very sensitive topics, like interview invitation or recidivism prediction. Fairness is not commonly studied for regression problems compared to binary classification problems; hence, we present a simple, yet effective method based on normalisation (FaiReg), which minimises the impact of unfairness in regression problems, especially due to labelling bias. We present a theoretical analysis of the method, in addition to an empirical comparison against two standard methods for fairness, namely data balancing and adversarial training. We also include a hybrid formulation (FaiRegH), merging the presented method with data balancing, in an attempt to face labelling and sampling biases simultaneously. The experiments are conducted on the multimodal dataset First Impressions (FI) with various labels, namely Big-Five personality prediction and interview screening score. The results show the superior performance of diminishing the effects of unfairness better than data balancing, also without deteriorating the performance of the original problem as much as adversarial training. Fairness is evaluated based on the Equal Accuracy (EA) and Statistical Parity (SP) constraints. The experiments present a setup that enhances the fairness for several protected variables simultaneously.
- [457] arXiv:2202.10367 (replaced) [pdf, html, other]
-
Title: Probabilities of the Third Type: Statistical Relational Learning and Reasoning with Relative FrequenciesComments: 30 pagesJournal-ref: Journal of Artificial Intelligence Research 80 (2024) 1407-1436Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Logic in Computer Science (cs.LO)
Dependencies on the relative frequency of a state in the domain are common when modelling probabilistic dependencies on relational data. For instance, the likelihood of a school closure during an epidemic might depend on the proportion of infected pupils exceeding a threshold. Often, rather than depending on discrete thresholds, dependencies are continuous: for instance, the likelihood of any one mosquito bite transmitting an illness depends on the proportion of carrier mosquitoes. Current approaches usually only consider probabilities over possible worlds rather than over domain elements themselves. An exception are the recently introduced lifted Bayesian networks for conditional probability logic, which express discrete dependencies on probabilistic data. We introduce functional lifted Bayesian networks, a formalism that explicitly incorporates continuous dependencies on relative frequencies into statistical relational artificial intelligence, and compare and contrast them with lifted Bayesian networks for conditional probability logic. Incorporating relative frequencies is not only beneficial to modelling; it also provides a more rigorous approach to learning problems where training and test or application domains have different sizes. To this end, we provide a representation of the asymptotic probability distributions induced by functional lifted Bayesian networks on domains of increasing sizes. Since that representation has well-understood scaling behaviour across domain sizes, it can be used to estimate parameters for a large domain consistently from randomly sampled subpopulations. Furthermore, we show that in parametric families of FLBN, convergence is uniform in the parameters, which ensures a meaningful dependence of the asymptotic probabilities on the parameters of the model.
- [458] arXiv:2202.13088 (replaced) [pdf, html, other]
-
Title: Almost Tight Approximation Hardness for Single-Source Directed k-Edge-ConnectivityComments: 26 pages, 11 figuresSubjects: Data Structures and Algorithms (cs.DS); Computational Complexity (cs.CC); Optimization and Control (math.OC)
In the $k$-connected directed Steiner tree problem ($k$-DST), we are given an $n$-vertex directed graph $G=(V,E)$ with edge costs, a connectivity requirement $k$, a root $r\in V$ and a set of terminals $T\subseteq V$. The goal is to find a minimum-cost subgraph $H\subseteq G$ that has $k$ internally disjoint paths from the root vertex $r$ to every terminal $t\in T$.
In this paper, we show the approximation hardness of $k$-DST for various parameters, which thus close some long-standing open problems.
- $\Omega\left(|T|/\log |T|\right)$-approximation hardness, which holds under the standard assumption $\mathrm{NP}\neq \mathrm{ZPP}$. The inapproximability ratio is tightened to $\Omega\left(|T|\right)$ under the Strongish Planted Clique Hypothesis [Manurangsi, Rubinstein and Schramm, ITCS 2021].
The latter hardness result matches the approximation ratio of $|T|$ obtained by a trivial approximation algorithm, thus closing the long-standing open problem.
- $\Omega\left(\sqrt{2}^k / k\right)$-approximation hardness for the general case of $k$-DST under the assumption $\mathrm{NP}\neq\mathrm{ZPP}$. This is the first hardness result known for survivable network design problems with an inapproximability ratio exponential in $k$.
- $\Omega\left((k/L)^{L/4}\right)$-approximation hardness for $k$-DST on $L$-layered graphs for $L\le O\left(\log n\right)$. This almost matches the approximation ratio of $O(k^{L-1}\cdot L \cdot \log |T|)$ achieving in $O\left(n^L\right)$-time due to Laekhanukit [ICALP`16]. - [459] arXiv:2207.00625 (replaced) [pdf, html, other]
-
Title: Projectivity revisitedComments: 30 pagesJournal-ref: International Journal of Approximate Reasoning 162:109031 (2023)Subjects: Artificial Intelligence (cs.AI); Probability (math.PR); Statistics Theory (math.ST)
The behaviour of statistical relational representations across differently sized domains has become a focal area of research from both a modelling and a complexity viewpoint.Recently, projectivity of a family of distributions emerged as a key property, ensuring that marginal probabilities are independent of the domain size. However, the formalisation used currently assumes that the domain is characterised only by its size. This contribution extends the notion of projectivity from families of distributions indexed by domain size to functors taking extensional data from a database. This makes projectivity available for the large range of applications taking structured input. We transfer key known results on projective families of distributions to the new setting. This includes a characterisation of projective fragments in different statistical relational formalisms as well as a general representation theorem for projective families of distributions. Furthermore, we prove a correspondence between projectivity and distributions on countably infinite domains, which we use to unify and generalise earlier work on statistical relational representations in infinite domains. Finally, we use the extended notion of projectivity to define a further strengthening, which we call $\sigma$-projectivity, and which allows the use of the same representation in different modes while retaining projectivity.
- [460] arXiv:2207.08794 (replaced) [pdf, html, other]
-
Title: D$^3$FlowSLAM: Self-Supervised Dynamic SLAM with Flow Motion Decomposition and DINO GuidanceComments: Homepage: this https URLSubjects: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
In this paper, we introduce a self-supervised deep SLAM method that robustly operates in dynamic scenes while accurately identifying dynamic components. Our method leverages a dual-flow representation for static flow and dynamic flow, facilitating effective scene decomposition in dynamic environments. We propose a dynamic update module based on this representation and develop a dense SLAM system that excels in dynamic scenarios. In addition, we design a self-supervised training scheme using DINO as a prior, enabling label-free training. Our method achieves superior accuracy compared to other self-supervised methods. It also matches or even surpasses the performance of existing supervised methods in some cases. All code and data will be made publicly available upon acceptance.
- [461] arXiv:2302.08929 (replaced) [pdf, html, other]
-
Title: Spatial Voting with Incomplete Voter InformationSubjects: Computer Science and Game Theory (cs.GT); Computational Complexity (cs.CC)
We consider spatial voting where candidates are located in the Euclidean $d$-dimensional space, and each voter ranks candidates based on their distance from the voter's ideal point. We explore the case where information about the location of voters' ideal points is incomplete: for each dimension, we are given an interval of possible values. We study the computational complexity of finding the possible and necessary winners for positional scoring rules. Our results show that we retain tractable cases of the classic model where voters have partial-order preferences. Moreover, we show that there are positional scoring rules under which the possible-winner problem is intractable for partial orders, but tractable in the one-dimensional spatial setting. We also consider approval voting in this setting. We show that for up to two dimensions, the necessary-winner problem is tractable, while the possible-winner problem is hard for any number of dimensions.
- [462] arXiv:2302.11988 (replaced) [pdf, html, other]
-
Title: Time Complexity of Broadcast and Consensus for Randomized Oblivious Message AdversariesComments: 24 pages + 13 pages of appendix. To appear at DISC'24Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Networking and Internet Architecture (cs.NI); Social and Information Networks (cs.SI)
Broadcast and consensus are most fundamental tasks in distributed computing. These tasks are particularly challenging in dynamic networks where communication across the network links may be unreliable, e.g., due to mobility or failures. Indeed, over the last years, researchers have derived several impossibility results and high time complexity lower bounds (i.e., linear in the number of nodes $n$) for these tasks, even for oblivious message adversaries where communication networks are rooted trees. However, such deterministic adversarial models may be overly conservative, as many processes in real-world settings are stochastic in nature rather than worst case.
This paper initiates the study of broadcast and consensus on stochastic dynamic networks, introducing a randomized oblivious message adversary. Our model is reminiscent of the SI model in epidemics, however, revolving around trees (which renders the analysis harder due to the apparent lack of independence). In particular, we show that if information dissemination occurs along random rooted trees, broadcast and consensus complete fast with high probability, namely in logarithmic time. Our analysis proves the independence of a key variable, which enables a formal understanding of the dissemination process.
More formally, for a network with $n$ nodes, we first consider the completely random case where in each round the communication network is chosen uniformly at random among rooted trees. We then introduce the notion of randomized oblivious message adversary, where in each round, an adversary can choose $k$ edges to appear in the communication network, and then a rooted tree is chosen uniformly at random among the set of all rooted trees that include these edges. We show that broadcast completes in $O(k+\log n)$ rounds, and that this it is also the case for consensus as long as $k \le 0.1n$. - [463] arXiv:2304.03997 (replaced) [pdf, other]
-
Title: Predicting Short Term Energy Demand in Smart Grid: A Deep Learning Approach for Integrating Renewable Energy Sources in Line with SDGs 7, 9, and 13Md Saef Ullah Miah, Junaida Sulaiman, Md. Imamul Islam, Md. Masuduzzaman, Molla Shahadat Hossain Lipu, Ramdhan NugrahaSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Integrating renewable energy sources into the power grid is becoming increasingly important as the world moves towards a more sustainable energy future in line with SDG 7. However, the intermittent nature of renewable energy sources can make it challenging to manage the power grid and ensure a stable supply of electricity, which is crucial for achieving SDG 9. In this paper, we propose a deep learning model for predicting energy demand in a smart power grid, which can improve the integration of renewable energy sources by providing accurate predictions of energy demand. Our approach aligns with SDG 13 on climate action, enabling more efficient management of renewable energy resources. We use long short-term memory networks, well-suited for time series data, to capture complex patterns and dependencies in energy demand data. The proposed approach is evaluated using four historical short-term energy demand data datasets from different energy distribution companies, including American Electric Power, Commonwealth Edison, Dayton Power and Light, and Pennsylvania-New Jersey-Maryland Interconnection. The proposed model is compared with three other state-of-the-art forecasting algorithms: Facebook Prophet, Support Vector Regression, and Random Forest Regression. The experimental results show that the proposed REDf model can accurately predict energy demand with a mean absolute error of 1.4%, indicating its potential to enhance the stability and efficiency of the power grid and contribute to achieving SDGs 7, 9, and 13. The proposed model also has the potential to manage the integration of renewable energy sources effectively.
- [464] arXiv:2304.14918 (replaced) [pdf, html, other]
-
Title: Representation Matters for Mastering Chess: Improved Feature Representation in AlphaZero Outperforms Switching to TransformersComments: 18 pages, 8 figures, 12 tablesSubjects: Artificial Intelligence (cs.AI)
While transformers have gained recognition as a versatile tool for artificial intelligence (AI), an unexplored challenge arises in the context of chess - a classical AI benchmark. Here, incorporating Vision Transformers (ViTs) into AlphaZero is insufficient for chess mastery, mainly due to ViTs' computational limitations. The attempt to optimize their efficiency by combining MobileNet and NextViT outperformed AlphaZero by about 30 Elo. However, we propose a practical improvement that involves a simple change in the input representation and value loss functions. As a result, we achieve a significant performance boost of up to 180 Elo points beyond what is currently achievable with AlphaZero in chess. In addition to these improvements, our experimental results using the Integrated Gradient technique confirm the effectiveness of the newly introduced features.
- [465] arXiv:2305.00050 (replaced) [pdf, html, other]
-
Title: Causal Reasoning and Large Language Models: Opening a New Frontier for CausalityComments: Added three novel datasets. To be published in TMLR. Authors listed alphabeticallySubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG); Methodology (stat.ME)
The causal capabilities of large language models (LLMs) are a matter of significant debate, with critical implications for the use of LLMs in societally impactful domains such as medicine, science, law, and policy. We conduct a "behavorial" study of LLMs to benchmark their capability in generating causal arguments. Across a wide range of tasks, we find that LLMs can generate text corresponding to correct causal arguments with high probability, surpassing the best-performing existing methods. Algorithms based on GPT-3.5 and 4 outperform existing algorithms on a pairwise causal discovery task (97%, 13 points gain), counterfactual reasoning task (92%, 20 points gain) and event causality (86% accuracy in determining necessary and sufficient causes in vignettes). We perform robustness checks across tasks and show that the capabilities cannot be explained by dataset memorization alone, especially since LLMs generalize to novel datasets that were created after the training cutoff date.
That said, LLMs exhibit unpredictable failure modes, and we discuss the kinds of errors that may be improved and what are the fundamental limits of LLM-based answers. Overall, by operating on the text metadata, LLMs bring capabilities so far understood to be restricted to humans, such as using collected knowledge to generate causal graphs or identifying background causal context from natural language. As a result, LLMs may be used by human domain experts to save effort in setting up a causal analysis, one of the biggest impediments to the widespread adoption of causal methods. Given that LLMs ignore the actual data, our results also point to a fruitful research direction of developing algorithms that combine LLMs with existing causal techniques. Code and datasets are available at this https URL. - [466] arXiv:2305.13153 (replaced) [pdf, other]
-
Title: Effective Bilevel Optimization via Minimax ReformulationComments: Typos and intended inclusion of additional experimentsSubjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
Bilevel optimization has found successful applications in various machine learning problems, including hyper-parameter optimization, data cleaning, and meta-learning. However, its huge computational cost presents a significant challenge for its utilization in large-scale problems. This challenge arises due to the nested structure of the bilevel formulation, where each hyper-gradient computation necessitates a costly inner optimization procedure. To address this issue, we propose a reformulation of bilevel optimization as a minimax problem, effectively decoupling the outer-inner dependency. Under mild conditions, we show these two problems are equivalent. Furthermore, we introduce a multi-stage gradient descent and ascent (GDA) algorithm to solve the resulting minimax problem with convergence guarantees. Extensive experimental results demonstrate that our method outperforms state-of-the-art bilevel methods while significantly reducing the computational cost.
- [467] arXiv:2305.14405 (replaced) [pdf, html, other]
-
Title: NeuralMatrix: Compute the Entire Neural Networks with Linear Matrix Operations for Efficient InferenceComments: 9 pages, 8figures, Submitted to The 39th Annual AAAI Conference on Artificial IntelligenceSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR)
The inherent diversity of computation types within the deep neural network (DNN) models often requires a variety of specialized units in hardware processors, which limits computational efficiency, increasing both inference latency and power consumption, especially when the hardware processor needs to support and execute different neural networks. In this study, we introduce NeuralMatrix, which elastically transforms the computations of entire DNNs into linear matrix operations. This transformation allows seamless execution of various DNN models all with matrix operations and paves the way for running versatile DNN models with a single General Matrix Multiplication (GEMM) accelerator.Extensive experiments with both CNN and transformer-based models demonstrate the potential of NeuralMatrix to accurately and efficiently execute a wide range of DNN models, achieving 2.17-38.72 times computation efficiency (i.e., throughput per power) compared to CPUs, GPUs, and SoC platforms. This level of efficiency is usually only attainable with the accelerator designed for a specific neural network.
- [468] arXiv:2305.16635 (replaced) [pdf, html, other]
-
Title: Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and ParaphrasingJaehun Jung, Peter West, Liwei Jiang, Faeze Brahman, Ximing Lu, Jillian Fisher, Taylor Sorensen, Yejin ChoiComments: NAACL 2024Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
We present Impossible Distillation, a novel framework for paraphrasing and sentence summarization, that distills a high-quality dataset and model from a low-quality teacher that itself cannot perform these tasks. Unlike prior works that rely on an extreme-scale teacher model (e.g., GPT3) or task-specific architecture, we hypothesize and verify the paraphrastic proximity intrinsic to pre-trained LMs (e.g., GPT2), where paraphrases occupy a proximal subspace in the LM distribution. By identifying and distilling generations from these subspaces, Impossible Distillation produces a high-quality dataset and model even from GPT2-scale LMs. We evaluate our method on multiple benchmarks spanning unconstrained / syntax-controlled paraphrase generation and sentence summarization. Our model with 770M parameters consistently outperforms strong baselines, including models distilled from ChatGPT, and sometimes, even ChatGPT itself. Also, we find that our distilled dataset from 1.5B LMs exhibits higher diversity and fidelity than up to 13 times larger datasets.
- [469] arXiv:2306.02314 (replaced) [pdf, html, other]
-
Title: Using Unreliable Pseudo-Labels for Label-Efficient Semantic SegmentationComments: Accepted by IJCV. arXiv admin note: text overlap with arXiv:2203.03884Subjects: Computer Vision and Pattern Recognition (cs.CV)
The crux of label-efficient semantic segmentation is to produce high-quality pseudo-labels to leverage a large amount of unlabeled or weakly labeled data. A common practice is to select the highly confident predictions as the pseudo-ground-truths for each pixel, but it leads to a problem that most pixels may be left unused due to their unreliability. However, we argue that every pixel matters to the model training, even those unreliable and ambiguous pixels. Intuitively, an unreliable prediction may get confused among the top classes, however, it should be confident about the pixel not belonging to the remaining classes. Hence, such a pixel can be convincingly treated as a negative key to those most unlikely categories. Therefore, we develop an effective pipeline to make sufficient use of unlabeled data. Concretely, we separate reliable and unreliable pixels via the entropy of predictions, push each unreliable pixel to a category-wise queue that consists of negative keys, and manage to train the model with all candidate pixels. Considering the training evolution, we adaptively adjust the threshold for the reliable-unreliable partition. Experimental results on various benchmarks and training settings demonstrate the superiority of our approach over the state-of-the-art alternatives.
- [470] arXiv:2306.07215 (replaced) [pdf, html, other]
-
Title: Efficient and Robust Quantization-aware Training via Adaptive Coreset SelectionComments: Accepted by TMLR, Code: this https URLSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
Quantization-aware training (QAT) is a representative model compression method to reduce redundancy in weights and activations. However, most existing QAT methods require end-to-end training on the entire dataset, which suffers from long training time and high energy costs. In addition, the potential label noise in the training data undermines the robustness of QAT. We propose two metrics based on analysis of loss and gradient of quantized weights: error vector score and disagreement score, to quantify the importance of each sample during training. Guided by these two metrics, we proposed a quantization-aware Adaptive Coreset Selection (ACS) method to select the data for the current training epoch. We evaluate our method on various networks (ResNet-18, MobileNetV2, RetinaNet), datasets(CIFAR-10, CIFAR-100, ImageNet-1K, COCO), and under different quantization settings. Specifically, our method can achieve an accuracy of 68.39\% of 4-bit quantized ResNet-18 on the ImageNet-1K dataset with only a 10\% subset, which has an absolute gain of 4.24\% compared to the baseline. Our method can also improve the robustness of QAT by removing noisy samples in the training set.
- [471] arXiv:2307.02969 (replaced) [pdf, html, other]
-
Title: DPM: Clustering Sensitive Data through SeparationComments: The first two authors equally contributed to this workSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
Clustering is an important tool for data exploration where the goal is to subdivide a data set into disjoint clusters that fit well into the underlying data structure. When dealing with sensitive data, privacy-preserving algorithms aim to approximate the non-private baseline while minimising the leakage of sensitive information. State-of-the-art privacy-preserving clustering algorithms tend to output clusters that are good in terms of the standard metrics, inertia, silhouette score, and clustering accuracy, however, the clustering result strongly deviates from the non-private KMeans baseline. In this work, we present a privacy-preserving clustering algorithm called DPM that recursively separates a data set into clusters based on a geometrical clustering approach. In addition, DPM estimates most of the data-dependent hyper-parameters in a privacy-preserving way. We prove that DPM preserves Differential Privacy and analyse the utility guarantees of DPM. Finally, we conduct an extensive empirical evaluation for synthetic and real-life data sets. We show that DPM achieves state-of-the-art utility on the standard clustering metrics and yields a clustering result much closer to that of the popular non-private KMeans algorithm without requiring the number of classes.
- [472] arXiv:2307.06281 (replaced) [pdf, html, other]
-
Title: MMBench: Is Your Multi-modal Model an All-around Player?Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, Dahua LinComments: Accepted in ECCV2024 as Oral PresentationSubjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
Large vision-language models (VLMs) have recently achieved remarkable progress, exhibiting impressive multimodal perception and reasoning abilities. However, effectively evaluating these large VLMs remains a major challenge, hindering future development in this domain. Traditional benchmarks like VQAv2 or COCO Caption provide quantitative performance measurements but lack fine-grained ability assessment and robust evaluation metrics. Meanwhile, subjective benchmarks, such as OwlEval, offer comprehensive evaluations of a model's abilities by incorporating human labor, which is not scalable and may display significant bias. In response to these challenges, we propose MMBench, a bilingual benchmark for assessing the multi-modal capabilities of VLMs. MMBench methodically develops a comprehensive evaluation pipeline, primarily comprised of the following key features: 1. MMBench is meticulously curated with well-designed quality control schemes, surpassing existing similar benchmarks in terms of the number and variety of evaluation questions and abilities; 2. MMBench introduces a rigorous CircularEval strategy and incorporates large language models to convert free-form predictions into pre-defined choices, which helps to yield accurate evaluation results for models with limited instruction-following capabilities. 3. MMBench incorporates multiple-choice questions in both English and Chinese versions, enabling an apples-to-apples comparison of VLMs' performance under a bilingual context. To summarize, MMBench is a systematically designed objective benchmark for a robust and holistic evaluation of vision-language models. We hope MMBench will assist the research community in better evaluating their models and facilitate future progress in this area. The evalutation code of MMBench has been integrated into VLMEvalKit: this https URL.
- [473] arXiv:2307.16694 (replaced) [pdf, html, other]
-
Title: Investigating and Improving Latent Density Segmentation Models for Aleatoric Uncertainty Quantification in Medical ImagingM. M. Amaan Valiuddin, Christiaan G. A. Viviers, Ruud J. G. van Sloun, Peter H. N. de With, Fons van der SommenSubjects: Computer Vision and Pattern Recognition (cs.CV)
Data uncertainties, such as sensor noise, occlusions or limitations in the acquisition method can introduce irreducible ambiguities in images, which result in varying, yet plausible, semantic hypotheses. In Machine Learning, this ambiguity is commonly referred to as aleatoric uncertainty. In image segmentation, latent density models can be utilized to address this problem. The most popular approach is the Probabilistic U-Net (PU-Net), which uses latent Normal densities to optimize the conditional data log-likelihood Evidence Lower Bound. In this work, we demonstrate that the PU-Net latent space is severely sparse and heavily under-utilized. To address this, we introduce mutual information maximization and entropy-regularized Sinkhorn Divergence in the latent space to promote homogeneity across all latent dimensions, effectively improving gradient-descent updates and latent space informativeness. Our results show that by applying this on public datasets of various clinical segmentation problems, our proposed methodology receives up to 11% performance gains compared against preceding latent variable models for probabilistic segmentation on the Hungarian-Matched Intersection over Union. The results indicate that encouraging a homogeneous latent space significantly improves latent density modeling for medical image segmentation.
- [474] arXiv:2308.05680 (replaced) [pdf, html, other]
-
Title: Breaking Language Barriers with MMTweets: Advancing Cross-Lingual Debunked Narrative Retrieval for Fact-CheckingSubjects: Computation and Language (cs.CL); Computers and Society (cs.CY); Information Retrieval (cs.IR); Machine Learning (cs.LG); Social and Information Networks (cs.SI)
Finding previously debunked narratives involves identifying claims that have already undergone fact-checking. The issue intensifies when similar false claims persist in multiple languages, despite the availability of debunks for several months in another language. Hence, automatically finding debunks (or fact-checks) in multiple languages is crucial to make the best use of scarce fact-checkers' resources. Mainly due to the lack of readily available data, this is an understudied problem, particularly when considering the cross-lingual scenario, i.e. the retrieval of debunks in a language different from the language of the online post being checked. This study introduces cross-lingual debunked narrative retrieval and addresses this research gap by: (i) creating Multilingual Misinformation Tweets (MMTweets): a dataset that stands out, featuring cross-lingual pairs, images, human annotations, and fine-grained labels, making it a comprehensive resource compared to its counterparts; (ii) conducting an extensive experiment to benchmark state-of-the-art cross-lingual retrieval models and introducing multistage retrieval methods tailored for the task; and (iii) comprehensively evaluating retrieval models for their cross-lingual and cross-dataset transfer capabilities within MMTweets, and conducting a retrieval latency analysis. We find that MMTweets presents challenges for cross-lingual debunked narrative retrieval, highlighting areas for improvement in retrieval models. Nonetheless, the study provides valuable insights for creating MMTweets datasets and optimising debunked narrative retrieval models to empower fact-checking endeavours. The dataset and annotation codebook are publicly available at this https URL.
- [475] arXiv:2308.05907 (replaced) [pdf, html, other]
-
Title: Simple Analysis of Priority SamplingComments: 7 pagesSubjects: Data Structures and Algorithms (cs.DS); Databases (cs.DB)
We prove a tight upper bound on the variance of the priority sampling method (aka sequential Poisson sampling). Our proof is significantly shorter and simpler than the original proof given by Mario Szegedy at STOC 2006, which resolved a conjecture by Duffield, Lund, and Thorup.
- [476] arXiv:2308.07545 (replaced) [pdf, html, other]
-
Title: Vision-Language Dataset DistillationComments: 31 pages, 13 figuresSubjects: Computer Vision and Pattern Recognition (cs.CV)
Dataset distillation methods reduce large-scale datasets to smaller sets of synthetic data, preserving sufficient information to quickly train a new model from scratch. However, prior work on dataset distillation has focused exclusively on image classification datasets, whereas modern large-scale datasets are primarily vision-language datasets. In this work, we design the first vision-language dataset distillation method, building on the idea of trajectory matching. A key challenge is that vision-language datasets do not have a set of discrete classes. To overcome this, our proposed method jointly distills image-text pairs in a contrastive formulation. Further, we leverage Low-Rank Adaptation (LoRA) matching to enable more efficient and effective trajectory matching in complex modern vision-language models. Since there are no existing baselines, we compare our distillation approach with three adapted vision-language coreset selection methods. We demonstrate significant improvements on the challenging Flickr30K and COCO retrieval benchmarks: for example, on Flickr30K, the best coreset selection method selecting 1000 image-text pairs for training achieves only 5.6% image-to-text retrieval accuracy (i.e., recall@1); in contrast, our dataset distillation almost doubles that to 9.9% with just 100 training pairs, an order of magnitude fewer.
- [477] arXiv:2308.08242 (replaced) [pdf, html, other]
-
Title: Contrastive Learning for Lane Detection via cross-similarityComments: 11 pagesSubjects: Computer Vision and Pattern Recognition (cs.CV)
Detecting lane markings in road scenes poses a challenge due to their intricate nature, which is susceptible to unfavorable conditions. While lane markings have strong shape priors, their visibility is easily compromised by lighting conditions, occlusions by other vehicles or pedestrians, and fading of colors over time. The detection process is further complicated by the presence of several lane shapes and natural variations, necessitating large amounts of data to train a robust lane detection model capable of handling various scenarios. In this paper, we present a novel self-supervised learning method termed Contrastive Learning for Lane Detection via cross-similarity (CLLD) to enhance the resilience of lane detection models in real-world scenarios, particularly when the visibility of lanes is compromised. CLLD introduces a contrastive learning (CL) method that assesses the similarity of local features within the global context of the input image. It uses the surrounding information to predict lane markings. This is achieved by integrating local feature contrastive learning with our proposed cross-similar operation. The local feature CL concentrates on extracting features from small patches, a necessity for accurately localizing lane segments. Meanwhile, cross-similarity captures global features, enabling the detection of obscured lane segments based on their surroundings. We enhance cross-similarity by randomly masking portions of input images in the process of augmentation. Extensive experiments on TuSimple and CuLane benchmarks demonstrate that CLLD outperforms SOTA contrastive learning methods, particularly in visibility-impairing conditions like shadows, while it also delivers comparable results under normal conditions. Compared to supervised learning, CLLD still excels in challenging scenarios such as shadows and crowded scenes, which are common in real-world driving.
- [478] arXiv:2309.06325 (replaced) [pdf, html, other]
-
Title: Distributed Precoding for Satellite-Terrestrial Integrated Networks Without Sharing CSIT: A Rate-Splitting ApproachComments: 16 pages, 7 figuresSubjects: Information Theory (cs.IT)
Satellite-terrestrial integrated networks (STINs) are promising architecture for providing global coverage. In STINs, full frequency reuse between a satellite and a terrestrial base station (BS) is encouraged for aggressive spectrum reuse, which induces non-negligible amount of interference. To address the interference management problem in STINs, this paper proposes a novel distributed precoding method. Key features of our method are: i) a rate-splitting (RS) strategy is incorporated for efficient interference management and ii) the precoders are designed in a distributed way without sharing channel state information between a satellite and a terrestrial BS. Specifically, to design the precoders in a distributed fashion, we put forth a spectral efficiency decoupling technique, that disentangles the total spectral efficiency function into two distinct terms, each of which is dependent solely on the satellite's precoder and the terrestrial BS's precoder, respectively. Then, to resolve the non-smoothness raised by the RS strategy, we approximate the spectral efficiency expression as a smooth function by using the LogSumExp technique; thereafter we develop a generalized power iteration inspired optimization algorithm built based on the first-order optimality condition. Simulation results demonstrate that the proposed method offers considerable spectral efficiency gains compared to the existing methods.
- [479] arXiv:2309.06611 (replaced) [pdf, html, other]
-
Title: Enabling the Deployment of Any-Scale Robotic Applications in Microservice Architectures through Automated ContainerizationComments: 6 pages; Submitted to IEEE International Conference on Robotics and Automation (ICRA) 2024Journal-ref: 2024 IEEE International Conference on Robotics and Automation (ICRA)Subjects: Robotics (cs.RO)
In an increasingly automated world -- from warehouse robots to self-driving cars -- streamlining the development and deployment process and operations of robotic applications becomes ever more important. Automated DevOps processes and microservice architectures have already proven successful in other domains such as large-scale customer-oriented web services (e.g., Netflix). We recommend to employ similar microservice architectures for the deployment of small- to large-scale robotic applications in order to accelerate development cycles, loosen functional dependence, and improve resiliency and elasticity. In order to facilitate involved DevOps processes, we present and release a tooling suite for automating the development of microservices for robotic applications based on the Robot Operating System (ROS). Our tooling suite covers the automated minimal containerization of ROS applications, a collection of useful machine learning-enabled base container images, as well as a CLI tool for simplified interaction with container images during the development phase. Within the scope of this paper, we embed our tooling suite into the overall context of streamlined robotics deployment and compare it to alternative solutions. We release our tools as open-source software at this https URL.
- [480] arXiv:2309.08698 (replaced) [pdf, html, other]
-
Title: No Imputation Needed: A Switch Approach to Irregularly Sampled Time SeriesRohit Agarwal, Aman Sinha, Ayan Vishwakarma, Xavier Coubez, Marianne Clausel, Mathieu Constant, Alexander Horsch, Dilip K. PrasadSubjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Modeling irregularly-sampled time series (ISTS) is challenging because of missing values. Most existing methods focus on handling ISTS by converting irregularly sampled data into regularly sampled data via imputation. These models assume an underlying missing mechanism, which may lead to unwanted bias and sub-optimal performance. We present SLAN (Switch LSTM Aggregate Network), which utilizes a group of LSTMs to model ISTS without imputation, eliminating the assumption of any underlying process. It dynamically adapts its architecture on the fly based on the measured sensors using switches. SLAN exploits the irregularity information to explicitly capture each sensor's local summary and maintains a global summary state throughout the observational period. We demonstrate the efficacy of SLAN on two public datasets, namely, MIMIC-III, and Physionet 2012.
- [481] arXiv:2309.09879 (replaced) [pdf, html, other]
-
Title: DynaPix SLAM: A Pixel-Based Dynamic Visual SLAM ApproachComments: Chenghao Xu and Elia Bonetto contributed equally to this work as first authors. 19 pages, 4 tables, 6 figures. Includes supplementary materialSubjects: Robotics (cs.RO)
Visual Simultaneous Localization and Mapping (V-SLAM) methods achieve remarkable performance in static environments, but face challenges in dynamic scenes where moving objects severely affect their core modules. To avoid this, dynamic V-SLAM approaches often leverage semantic information, geometric constraints, or optical flow. However, these methods are limited by imprecise estimations and their reliance on the accuracy of deep-learning models. Moreover, predefined thresholds for static/dynamic classification, the a-priori selection of dynamic object classes, and the inability to recognize unknown or unexpected moving objects, often degrade their performance. To address these limitations, we introduce DynaPix, a novel semantic-free V-SLAM system based on per-pixel motion probability estimation and an improved pose optimization process. The per-pixel motion probability is estimated using a static background differencing method on image data and optical flows computed on splatted frames. With DynaPix, we fully integrate these probabilities into map point selection and apply them through weighted bundle adjustment within the tracking and optimization modules of ORB-SLAM2. We thoroughly evaluate our method using the GRADE and TUM RGB-D datasets, showing significantly lower trajectory errors and longer tracking times in both static and dynamic sequences. The source code, datasets, and results are available at this https URL.
- [482] arXiv:2309.13333 (replaced) [pdf, html, other]
-
Title: mdendro: An R package for extended agglomerative hierarchical clusteringComments: 29 pages, 15 figures. Software available at CRAN (this https URL) and Github (this https URL)Subjects: Information Retrieval (cs.IR); Data Analysis, Statistics and Probability (physics.data-an); Computation (stat.CO)
"mdendro" is an R package that provides a comprehensive collection of linkage methods for agglomerative hierarchical clustering on a matrix of proximity data (distances or similarities), returning a multifurcated dendrogram or multidendrogram. Multidendrograms can group more than two clusters at the same time, solving the nonuniqueness problem that arises when there are ties in the data. This problem causes that different binary dendrograms are possible depending both on the order of the input data and on the criterion used to break ties. Weighted and unweighted versions of the most common linkage methods are included in the package, which also implements two parametric linkage methods. In addition, package "mdendro" provides five descriptive measures to analyze the resulting dendrograms: cophenetic correlation coefficient, space distortion ratio, agglomeration coefficient, chaining coefficient and tree balance.
- [483] arXiv:2309.16680 (replaced) [pdf, html, other]
-
Title: Semi-Persistent Scheduling in NR Sidelink Mode 2: MAC Packet Reception Ratio Model and ns-3 ValidationComments: This work has been submitted to the IEEE for possible publication. 13 pages, 22 figuresSubjects: Networking and Internet Architecture (cs.NI); Systems and Control (eess.SY)
5G New Radio (NR) Sidelink (SL) has demonstrated the promising capability for infrastructure-less cellular coverage. Understanding the fundamentals of the NR SL channel access mechanism, Semi-Persistent Scheduling (SPS), which is specified by the 3rd Generation Partnership Project (3GPP), is a necessity to enhance the NR SL Packet Reception Ratio (PRR). However, most existing works fail to account for the new SPS features introduced in NR SL, which might be out-of-date for comprehensively describing the NR SL PRR. The existing models ignore the relationships between SPS parameters and, therefore, do not provide sufficient insights into the PRR of SPS. This work proposes a novel SPS PRR model incorporating MAC collisions based on new features in NR SL. We extend our model by loosening several simplifying assumptions made in our initial modeling. The extended models illustrate how the PRR is affected by various SPS parameters. The computed results are validated via simulations using the network simulator (ns-3), which provides important guidelines for future NR SL enhancement work.
- [484] arXiv:2310.03089 (replaced) [pdf, other]
-
Title: An adaptive stabilized trace finite element method for surface PDEsSubjects: Numerical Analysis (math.NA)
The paper introduces an adaptive version of the stabilized Trace Finite Element Method (TraceFEM) designed to solve low-regularity elliptic problems on level-set surfaces using a shape-regular bulk mesh in the embedding space. Two stabilization variants, gradient-jump face and normal-gradient volume, are considered for continuous trace spaces of the first and second degrees, based on the polynomial families $Q_1$ and $Q_2$. We propose a practical error indicator that estimates the `jumps' of finite element solution derivatives across background mesh faces and it avoids integration of any quantities along implicitly defined curvilinear edges of the discrete surface elements. For the $Q_1$ family of piecewise trilinear polynomials on bulk cells, the solve-estimate-mark-refine strategy, combined with the suggested error indicator, achieves optimal convergence rates typical of two-dimensional problems. We also provide a posteriori error estimates, establishing the reliability of the error indicator for the $Q_1$ and $Q_2$ elements and for two types of stabilization. In numerical experiments, we assess the reliability and efficiency of the error indicator. While both stabilizations are found to deliver comparable performance,the lowest degree finite element space appears to be the more robust choice for the adaptive TraceFEM framework.
- [485] arXiv:2310.04376 (replaced) [pdf, html, other]
-
Title: Near-linear Time Dispersion of Mobile AgentsSubjects: Distributed, Parallel, and Cluster Computing (cs.DC); Multiagent Systems (cs.MA)
Consider that there are $k\le n$ agents in a simple, connected, and undirected graph $G=(V,E)$ with $n$ nodes and $m$ edges. The goal of the dispersion problem is to move these $k$ agents to mutually distinct nodes. Agents can communicate only when they are at the same node, and no other communication means, such as whiteboards, are available. We assume that the agents operate synchronously. We consider two scenarios: when all agents are initially located at a single node (rooted setting) and when they are initially distributed over one or more nodes (general setting). Kshemkalyani and Sharma presented a dispersion algorithm for the general setting, which uses $O(m_k)$ time and $\log(k + \Delta)$ bits of memory per agent [OPODIS 2021], where $m_k$ is the maximum number of edges in any induced subgraph of $G$ with $k$ nodes, and $\Delta$ is the maximum degree of $G$. This algorithm is currently the fastest in the literature, as no $o(m_k)$-time algorithm has been discovered, even for the rooted setting. In this paper, we present significantly faster algorithms for both the rooted and the general settings. First, we present an algorithm for the rooted setting that solves the dispersion problem in $O(k\log \min(k,\Delta))=O(k\log k)$ time using $O(\log (k+\Delta))$ bits of memory per agent. Next, we propose an algorithm for the general setting that achieves dispersion in $O(k \log k \cdot \log \min(k,\Delta))=O(k \log^2 k)$ time using $O(\log (k+\Delta))$ bits. Finally, for the rooted setting, we give a time-optimal (i.e.,~$O(k)$-time) algorithm with $O(\Delta+\log k)$ bits of space per agent. All algorithms presented in this paper work only in the synchronous setting, while several algorithms in the literature, including the one given by Kshemkalyani and Sharma at OPODIS 2021, work in the asynchronous setting.
- [486] arXiv:2310.06397 (replaced) [pdf, html, other]
-
Title: Top of the Heap: Efficient Memory Error Protection of Safe Heap ObjectsSubjects: Cryptography and Security (cs.CR)
Heap memory errors remain a major source of software vulnerabilities. Existing memory safety defenses aim at protecting all objects, resulting in high performance cost and incomplete protection. Instead, we propose an approach that accurately identifies objects that are inexpensive to protect, and design a method to protect such objects comprehensively from all classes of memory errors. Towards this goal, we introduce the Uriah system that (1) statically identifies the heap objects whose accesses satisfy spatial and type safety, and (2) dynamically allocates such "safe" heap objects on an isolated safe heap to enforce a form of temporal safety while preserving spatial and type safety, called temporal allocated-type safety. Uriah finds 72.0% of heap allocation sites produce objects whose accesses always satisfy spatial and type safety in the SPEC CPU2006/2017 benchmarks, 5 server programs, and Firefox, which are then isolated on a safe heap using Uriah allocator to enforce temporal allocated-type safety. Uriah incurs only 2.9% and 2.6% runtime overhead, along with 9.3% and 5.4% memory overhead, on the SPEC CPU 2006 and 2017 benchmarks, while preventing exploits on all the heap memory errors in DARPA CGC binaries and 28 recent CVEs. Additionally, using existing defenses to enforce their memory safety guarantees on the unsafe heap objects significantly reduces overhead, enabling the protection of heap objects from all classes of memory errors at more practical costs.
- [487] arXiv:2310.07048 (replaced) [pdf, html, other]
-
Title: FedMFS: Federated Multimodal Fusion Learning with Selective Modality CommunicationComments: ICC 2024Subjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Networking and Internet Architecture (cs.NI)
Multimodal federated learning (FL) aims to enrich model training in FL settings where devices are collecting measurements across multiple modalities (e.g., sensors measuring pressure, motion, and other types of data). However, key challenges to multimodal FL remain unaddressed, particularly in heterogeneous network settings: (i) the set of modalities collected by each device will be diverse, and (ii) communication limitations prevent devices from uploading all their locally trained modality models to the server. In this paper, we propose Federated Multimodal Fusion learning with Selective modality communication (FedMFS), a new multimodal fusion FL methodology that can tackle the above mentioned challenges. The key idea is the introduction of a modality selection criterion for each device, which weighs (i) the impact of the modality, gauged by Shapley value analysis, against (ii) the modality model size as a gauge for communication overhead. This enables FedMFS to flexibly balance performance against communication costs, depending on resource constraints and application requirements. Experiments on the real-world ActionSense dataset demonstrate the ability of FedMFS to achieve comparable accuracy to several baselines while reducing the communication overhead by over 4x.
- [488] arXiv:2310.09462 (replaced) [pdf, html, other]
-
Title: A Framework for Empowering Reinforcement Learning Agents with Causal Analysis: Enhancing Automated Cryptocurrency TradingSubjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Despite advances in artificial intelligence-enhanced trading methods, developing a profitable automated trading system remains challenging in the rapidly evolving cryptocurrency market. This research focuses on developing a reinforcement learning (RL) framework to tackle the complexities of trading five prominent altcoins: Binance Coin, Ethereum, Litecoin, Ripple, and Tether. To this end, we present the CausalReinforceNet~(CRN) framework, which integrates both Bayesian and dynamic Bayesian network techniques to empower the RL agent in trade decision-making. We develop two agents using the framework based on distinct RL algorithms to analyse performance compared to the Buy-and-Hold benchmark strategy and a baseline RL model. The results indicate that our framework surpasses both models in profitability, highlighting CRN's consistent superiority, although the level of effectiveness varies across different cryptocurrencies.
- [489] arXiv:2310.10830 (replaced) [pdf, html, other]
-
Title: Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style AttacksComments: Accepted to KDD 2024 (Research Track)Subjects: Computation and Language (cs.CL)
It is commonly perceived that fake news and real news exhibit distinct writing styles, such as the use of sensationalist versus objective language. However, we emphasize that style-related features can also be exploited for style-based attacks. Notably, the advent of powerful Large Language Models (LLMs) has empowered malicious actors to mimic the style of trustworthy news sources, doing so swiftly, cost-effectively, and at scale. Our analysis reveals that LLM-camouflaged fake news content significantly undermines the effectiveness of state-of-the-art text-based detectors (up to 38% decrease in F1 Score), implying a severe vulnerability to stylistic variations. To address this, we introduce SheepDog, a style-robust fake news detector that prioritizes content over style in determining news veracity. SheepDog achieves this resilience through (1) LLM-empowered news reframings that inject style diversity into the training process by customizing articles to match different styles; (2) a style-agnostic training scheme that ensures consistent veracity predictions across style-diverse reframings; and (3) content-focused veracity attributions that distill content-centric guidelines from LLMs for debunking fake news, offering supplementary cues and potential intepretability that assist veracity prediction. Extensive experiments on three real-world benchmarks demonstrate SheepDog's style robustness and adaptability to various backbones.
- [490] arXiv:2310.12375 (replaced) [pdf, html, other]
-
Title: Nearly Optimal Bounds for Sample-Based Testing and Learning of $k$-Monotone FunctionsComments: Appearing in RANDOM 2024Subjects: Data Structures and Algorithms (cs.DS)
We study monotonicity testing of functions $f \colon \{0,1\}^d \to \{0,1\}$ using sample-based algorithms, which are only allowed to observe the value of $f$ on points drawn independently from the uniform distribution. A classic result by Bshouty-Tamon (J. ACM 1996) proved that monotone functions can be learned with $\exp(\widetilde{O}(\min\{\frac{1}{\varepsilon}\sqrt{d},d\}))$ samples and it is not hard to show that this bound extends to testing. Prior to our work the only lower bound for this problem was $\Omega(\sqrt{\exp(d)/\varepsilon})$ in the small $\varepsilon$ parameter regime, when $\varepsilon = O(d^{-3/2})$, due to Goldreich-Goldwasser-Lehman-Ron-Samorodnitsky (Combinatorica 2000). Thus, the sample complexity of monotonicity testing was wide open for $\varepsilon \gg d^{-3/2}$. We resolve this question, obtaining a nearly tight lower bound of $\exp(\Omega(\min\{\frac{1}{\varepsilon}\sqrt{d},d\}))$ for all $\varepsilon$ at most a sufficiently small constant. In fact, we prove a much more general result, showing that the sample complexity of $k$-monotonicity testing and learning for functions $f \colon \{0,1\}^d \to [r]$ is $\exp(\Omega(\min\{\frac{rk}{\varepsilon}\sqrt{d},d\}))$. For testing with one-sided error we show that the sample complexity is $\exp(\Theta(d))$.
Beyond the hypercube, we prove nearly tight bounds (up to polylog factors of $d,k,r,1/\varepsilon$ in the exponent) of $\exp(\widetilde{\Theta}(\min\{\frac{rk}{\varepsilon}\sqrt{d},d\}))$ on the sample complexity of testing and learning measurable $k$-monotone functions $f \colon \mathbb{R}^d \to [r]$ under product distributions. Our upper bound improves upon the previous bound of $\exp(\widetilde{O}(\min\{\frac{k}{\varepsilon^2}\sqrt{d},d\}))$ by Harms-Yoshida (ICALP 2022) for Boolean functions ($r=2$). - [491] arXiv:2310.12431 (replaced) [pdf, html, other]
-
Title: SAM Meets UAP: Attacking Segment Anything Model With Universal Adversarial PerturbationSubjects: Computer Vision and Pattern Recognition (cs.CV)
As Segment Anything Model (SAM) becomes a popular foundation model in computer vision, its adversarial robustness has become a concern that cannot be ignored. This works investigates whether it is possible to attack SAM with image-agnostic Universal Adversarial Perturbation (UAP). In other words, we seek a single perturbation that can fool the SAM to predict invalid masks for most (if not all) images. We demonstrate convetional image-centric attack framework is effective for image-independent attacks but fails for universal adversarial attack. To this end, we propose a novel perturbation-centric framework that results in a UAP generation method based on self-supervised contrastive learning (CL), where the UAP is set to the anchor sample and the positive sample is augmented from the UAP. The representations of negative samples are obtained from the image encoder in advance and saved in a memory bank. The effectiveness of our proposed CL-based UAP generation method is validated by both quantitative and qualitative results. On top of the ablation study to understand various components in our proposed method, we shed light on the roles of positive and negative samples in making the generated UAP effective for attacking SAM.
- [492] arXiv:2310.13479 (replaced) [pdf, html, other]
-
Title: Segment, Select, Correct: A Framework for Weakly-Supervised Referring SegmentationComments: Accepted to ECCV'24 Workshop Proceedings (Instance-Level Recognition Workshop)Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Referring Image Segmentation (RIS) - the problem of identifying objects in images through natural language sentences - is a challenging task currently mostly solved through supervised learning. However, while collecting referred annotation masks is a time-consuming process, the few existing weakly-supervised and zero-shot approaches fall significantly short in performance compared to fully-supervised learning ones. To bridge the performance gap without mask annotations, we propose a novel weakly-supervised framework that tackles RIS by decomposing it into three steps: obtaining instance masks for the object mentioned in the referencing instruction (segment), using zero-shot learning to select a potentially correct mask for the given instruction (select), and bootstrapping a model which allows for fixing the mistakes of zero-shot selection (correct). In our experiments, using only the first two steps (zero-shot segment and select) outperforms other zero-shot baselines by as much as 16.5%, while our full method improves upon this much stronger baseline and sets the new state-of-the-art for weakly-supervised RIS, reducing the gap between the weakly-supervised and fully-supervised methods in some cases from around 33% to as little as 7%. Code is available at this https URL.
- [493] arXiv:2310.13664 (replaced) [pdf, html, other]
-
Title: Explainable Depression Symptom Detection in Social MediaComments: Accepted for publication in Health Information Science and SystemsSubjects: Computation and Language (cs.CL)
Users of social platforms often perceive these sites as supportive spaces to post about their mental health issues. Those conversations contain important traces about individuals' health risks. Recently, researchers have exploited this online information to construct mental health detection models, which aim to identify users at risk on platforms like Twitter, Reddit or Facebook. Most of these models are centred on achieving good classification results, ignoring the explainability and interpretability of the decisions. Recent research has pointed out the importance of using clinical markers, such as the use of symptoms, to improve trust in the computational models by health professionals. In this paper, we propose using transformer-based architectures to detect and explain the appearance of depressive symptom markers in the users' writings. We present two approaches: i) train a model to classify, and another one to explain the classifier's decision separately and ii) unify the two tasks simultaneously using a single model. Additionally, for this latter manner, we also investigated the performance of recent conversational LLMs when using in-context learning. Our natural language explanations enable clinicians to interpret the models' decisions based on validated symptoms, enhancing trust in the automated process. We evaluate our approach using recent symptom-based datasets, employing both offline and expert-in-the-loop metrics to assess the quality of the explanations generated by our models. The experimental results show that it is possible to achieve good classification results while generating interpretable symptom-based explanations.
- [494] arXiv:2310.13712 (replaced) [pdf, html, other]
-
Title: Impact of Guidance and Interaction Strategies for LLM Use on Learner Performance and PerceptionHarsh Kumar, Ilya Musabirov, Mohi Reza, Jiakai Shi, Xinyuan Wang, Joseph Jay Williams, Anastasia Kuzminykh, Michael LiutComments: To appear in CSCW 2024Subjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)
Personalized chatbot-based teaching assistants can be crucial in addressing increasing classroom sizes, especially where direct teacher presence is limited. Large language models (LLMs) offer a promising avenue, with increasing research exploring their educational utility. However, the challenge lies not only in establishing the efficacy of LLMs but also in discerning the nuances of interaction between learners and these models, which impact learners' engagement and results. We conducted a formative study in an undergraduate computer science classroom (N=145) and a controlled experiment on Prolific (N=356) to explore the impact of four pedagogically informed guidance strategies on the learners' performance, confidence and trust in LLMs. Direct LLM answers marginally improved performance, while refining student solutions fostered trust. Structured guidance reduced random queries as well as instances of students copy-pasting assignment questions to the LLM. Our work highlights the role that teachers can play in shaping LLM-supported learning environments.
- [495] arXiv:2310.15290 (replaced) [pdf, html, other]
-
Title: Reliable Generation of Privacy-preserving Synthetic EHR Time Series via Diffusion ModelsSubjects: Machine Learning (cs.LG)
Electronic Health Records (EHRs) are rich sources of patient-level data, offering valuable resources for medical data analysis. However, privacy concerns often restrict access to EHRs, hindering downstream analysis. Current EHR de-identification methods are flawed and can lead to potential privacy leakage. Additionally, existing publicly available EHR databases are limited, preventing the advancement of medical research using EHR. This study aims to overcome these challenges by generating realistic and privacy-preserving synthetic electronic health records (EHRs) time series efficiently. We introduce a new method for generating diverse and realistic synthetic EHR time series data using Denoising Diffusion Probabilistic Models (DDPM). We conducted experiments on six databases: Medical Information Mart for Intensive Care III and IV (MIMIC-III/IV), the eICU Collaborative Research Database (eICU), and non-EHR datasets on Stocks and Energy. We compared our proposed method with eight existing methods. Our results demonstrate that our approach significantly outperforms all existing methods in terms of data fidelity while requiring less training effort. Additionally, data generated by our method yields a lower discriminative accuracy compared to other baseline methods, indicating the proposed method can generate data with less privacy risk. The proposed diffusion-model-based method can reliably and efficiently generate synthetic EHR time series, which facilitates the downstream medical data analysis. Our numerical results show the superiority of the proposed method over all other existing methods.
- [496] arXiv:2311.00582 (replaced) [pdf, html, other]
-
Title: Minimally Modifying a Markov Game to Achieve Any Nash Equilibrium and ValueComments: Accepted by ICML 2024 ConferenceSubjects: Computer Science and Game Theory (cs.GT); Artificial Intelligence (cs.AI)
We study the game modification problem, where a benevolent game designer or a malevolent adversary modifies the reward function of a zero-sum Markov game so that a target deterministic or stochastic policy profile becomes the unique Markov perfect Nash equilibrium and has a value within a target range, in a way that minimizes the modification cost. We characterize the set of policy profiles that can be installed as the unique equilibrium of a game and establish sufficient and necessary conditions for successful installation. We propose an efficient algorithm that solves a convex optimization problem with linear constraints and then performs random perturbation to obtain a modification plan with a near-optimal cost.
- [497] arXiv:2311.04350 (replaced) [pdf, html, other]
-
Title: Device Sampling and Resource Optimization for Federated Learning in Cooperative Edge NetworksComments: Published in IEEE/ACM Transactions on Networking. arXiv admin note: substantial text overlap with arXiv:2101.00787Subjects: Networking and Internet Architecture (cs.NI); Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG)
The conventional federated learning (FedL) architecture distributes machine learning (ML) across worker devices by having them train local models that are periodically aggregated by a server. FedL ignores two important characteristics of contemporary wireless networks, however: (i) the network may contain heterogeneous communication/computation resources, and (ii) there may be significant overlaps in devices' local data distributions. In this work, we develop a novel optimization methodology that jointly accounts for these factors via intelligent device sampling complemented by device-to-device (D2D) offloading. Our optimization methodology aims to select the best combination of sampled nodes and data offloading configuration to maximize FedL training accuracy while minimizing data processing and D2D communication resource consumption subject to realistic constraints on the network topology and device capabilities. Theoretical analysis of the D2D offloading subproblem leads to new FedL convergence bounds and an efficient sequential convex optimizer. Using these results, we develop a sampling methodology based on graph convolutional networks (GCNs) which learns the relationship between network attributes, sampled nodes, and D2D data offloading to maximize FedL accuracy. Through evaluation on popular datasets and real-world network measurements from our edge testbed, we find that our methodology outperforms popular device sampling methodologies from literature in terms of ML model performance, data processing overhead, and energy consumption.
- [498] arXiv:2311.06228 (replaced) [pdf, other]
-
Title: Learning material synthesis-process-structure-property relationship by data fusion: Bayesian Coregionalization N-Dimensional Piecewise Function LearningSubjects: Machine Learning (cs.LG); Materials Science (cond-mat.mtrl-sci)
Autonomous materials research labs require the ability to combine and learn from diverse data streams. This is especially true for learning material synthesis-process-structure-property relationships, key to accelerating materials optimization and discovery as well as accelerating mechanistic understanding. We present the Synthesis-process-structure-property relAtionship coreGionalized lEarner (SAGE) algorithm. A fully Bayesian algorithm that uses multimodal coregionalization to merge knowledge across data sources to learn synthesis-process-structure-property relationships. SAGE outputs a probabilistic posterior for the relationships including the most likely relationships given the data.
- [499] arXiv:2311.06523 (replaced) [pdf, html, other]
-
Title: Generative AI for Space-Air-Ground Integrated NetworksRuichen Zhang, Hongyang Du, Dusit Niyato, Jiawen Kang, Zehui Xiong, Abbas Jamalipour, Ping Zhang, Dong In KimComments: 10 pages, 3 figures, Accepted at IEEE Wireless CommunicationsSubjects: Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)
Recently, generative AI technologies have emerged as a significant advancement in artificial intelligence field, renowned for their language and image generation capabilities. Meantime, space-air-ground integrated network (SAGIN) is an integral part of future B5G/6G for achieving ubiquitous connectivity. Inspired by this, this article explores an integration of generative AI in SAGIN, focusing on potential applications and case study. We first provide a comprehensive review of SAGIN and generative AI models, highlighting their capabilities and opportunities of their integration. Benefiting from generative AI's ability to generate useful data and facilitate advanced decision-making processes, it can be applied to various scenarios of SAGIN. Accordingly, we present a concise survey on their integration, including channel modeling and channel state information (CSI) estimation, joint air-space-ground resource allocation, intelligent network deployment, semantic communications, image extraction and processing, security and privacy enhancement. Next, we propose a framework that utilizes a Generative Diffusion Model (GDM) to construct channel information map to enhance quality of service for SAGIN. Simulation results demonstrate the effectiveness of the proposed framework. Finally, we discuss potential research directions for generative AI-enabled SAGIN.
- [500] arXiv:2311.08815 (replaced) [pdf, html, other]
-
Title: Self-Supervised Disentanglement by Leveraging Structure in Data AugmentationsCian Eastwood, Julius von Kügelgen, Linus Ericsson, Diane Bouchacourt, Pascal Vincent, Bernhard Schölkopf, Mark IbrahimSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
Self-supervised representation learning often uses data augmentations to induce some invariance to "style" attributes of the data. However, with downstream tasks generally unknown at training time, it is difficult to deduce a priori which attributes of the data are indeed "style" and can be safely discarded. To deal with this, current approaches try to retain some style information by tuning the degree of invariance to some particular task, such as ImageNet object classification. However, prior work has shown that such task-specific tuning can lead to significant performance degradation on other tasks that rely on the discarded style. To address this, we introduce a more principled approach that seeks to disentangle style features rather than discard them. The key idea is to add multiple style embedding spaces where: (i) each is invariant to all-but-one augmentation; and (ii) joint entropy is maximized. We formalize our structured data-augmentation procedure from a causal latent-variable-model perspective, and prove identifiability of both content and individual style variables. We empirically demonstrate the benefits of our approach on both synthetic and real-world data.
- [501] arXiv:2311.09263 (replaced) [pdf, html, other]
-
Title: Auto-ICL: In-Context Learning without Human SupervisionComments: Under reviewSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
With in-context learning ability, the performance of large language models can be significantly boosted when provided with appropriate context. However, existing in-context learning methods mainly rely on human-provided contexts, such as labeled examples and explicit instructions. Writing context by humans is labor-intensive on various tasks and limits the model to tasks manageable by humans. To overcome these limitations, we propose Automatic In-Context Learning framework that enables the model to autonomously generate examples and instructions for problem-solving. With experiments across various models and datasets, results show that model-generated contexts outperform human-annotated contexts, including Few-Shot and Few-Shot-CoT methods, and surpass existing self-generated context methods like Zero-CoT and Auto-CoT.
- [502] arXiv:2311.10653 (replaced) [pdf, html, other]
-
Title: Learning Realistic Joint Space Boundaries for Range of Motion Analysis of Healthy and Impaired Human ArmsJournal-ref: 2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids), 2023, pp. 1-8Subjects: Robotics (cs.RO); Machine Learning (cs.LG)
A realistic human kinematic model that satisfies anatomical constraints is essential for human-robot interaction, biomechanics and robot-assisted rehabilitation. Modeling realistic joint constraints, however, is challenging as human arm motion is constrained by joint limits, inter- and intra-joint dependencies, self-collisions, individual capabilities and muscular or neurological constraints which are difficult to represent. Hence, physicians and researchers have relied on simple box-constraints, ignoring important anatomical factors. In this paper, we propose a data-driven method to learn realistic anatomically constrained upper-limb range of motion (RoM) boundaries from motion capture data. This is achieved by fitting a one-class support vector machine to a dataset of upper-limb joint space exploration motions with an efficient hyper-parameter tuning scheme. Our approach outperforms similar works focused on valid RoM learning. Further, we propose an impairment index (II) metric that offers a quantitative assessment of capability/impairment when comparing healthy and impaired arms. We validate the metric on healthy subjects physically constrained to emulate hemiplegia and different disability levels as stroke patients.
- [503] arXiv:2311.13254 (replaced) [pdf, html, other]
-
Title: Unified Domain Adaptive Semantic SegmentationComments: 18 pages,10 figures, 11 tablesSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Image and Video Processing (eess.IV)
Unsupervised Domain Adaptive Semantic Segmentation (UDA-SS) aims to transfer the supervision from a labeled source domain to an unlabeled target domain. The majority of existing UDA-SS works typically consider images whilst recent attempts have extended further to tackle videos by modeling the temporal dimension. Although the two lines of research share the major challenges -- overcoming the underlying domain distribution shift, their studies are largely independent, resulting in fragmented insights, a lack of holistic understanding, and missed opportunities for cross-pollination of ideas. This fragmentation prevents the unification of methods, leading to redundant efforts and suboptimal knowledge transfer across image and video domains. Under this observation, we advocate unifying the study of UDA-SS across video and image scenarios, enabling a more comprehensive understanding, synergistic advancements, and efficient knowledge sharing. To that end, we explore the unified UDA-SS from a general data augmentation perspective, serving as a unifying conceptual framework, enabling improved generalization, and potential for cross-pollination of ideas, ultimately contributing to the overall progress and practical impact of this field of research. Specifically, we propose a Quad-directional Mixup (QuadMix) method, characterized by tackling distinct point attributes and feature inconsistencies through four-directional paths for intra- and inter-domain mixing in a feature space. To deal with temporal shifts with videos, we incorporate optical flow-guided feature aggregation across spatial and temporal dimensions for fine-grained domain alignment. Extensive experiments show that our method outperforms the state-of-the-art works by large margins on four challenging UDA-SS benchmarks. Our source code and models will be released at \url{this https URL}.
- [504] arXiv:2312.02851 (replaced) [pdf, other]
-
Title: Checkpoint-based rollback recovery in session programmingSubjects: Programming Languages (cs.PL); Logic in Computer Science (cs.LO)
To react to unforeseen circumstances or amend abnormal situations in communication-centric systems, programmers are in charge of "undoing" the interactions which led to an undesired state. To assist this task, session-based languages can be endowed with reversibility mechanisms. In this paper we propose a language enriched with programming facilities to commit session interactions, to roll back the computation to a previous commit point, and to abort the session. Rollbacks in our language always bring the system to previous visited states and a rollback cannot bring the system back to a point prior to the last commit. Programmers are relieved from the burden of ensuring that a rollback never restores a checkpoint imposed by a session participant different from the rollback requester. Such undesired situations are prevented at design-time (statically) by relying on a decidable compliance check at the type level, implemented in MAUDE. We show that the language satisfies error-freedom and progress of a session.
- [505] arXiv:2312.03451 (replaced) [pdf, html, other]
-
Title: An efficient data-based off-policy Q-learning algorithm for optimal output feedback control of linear systemsComments: Proceedings of the 6th Annual Learning for Dynamics & Control Conference, PMLR 242:312-323, 2024Subjects: Systems and Control (eess.SY)
In this paper, we present a Q-learning algorithm to solve the optimal output regulation problem for discrete-time LTI systems. This off-policy algorithm only relies on using persistently exciting input-output data, measured offline. No model knowledge or state measurements are needed and the obtained optimal policy only uses past input-output information. Moreover, our formulation of the proposed algorithm renders it computationally efficient. We provide conditions that guarantee the convergence of the algorithm to the optimal solution. Finally, the performance of our method is compared to existing algorithms in the literature.
- [506] arXiv:2312.03782 (replaced) [pdf, html, other]
-
Title: Novel class discovery meets foundation models for 3D semantic segmentationComments: arXiv admin note: substantial text overlap with arXiv:2303.11610Subjects: Computer Vision and Pattern Recognition (cs.CV)
The task of Novel Class Discovery (NCD) in semantic segmentation entails training a model able to accurately segment unlabelled (novel) classes, relying on the available supervision from annotated (base) classes. Although extensively investigated in 2D image data, the extension of the NCD task to the domain of 3D point clouds represents a pioneering effort, characterized by assumptions and challenges that are not present in the 2D case. This paper represents an advancement in the analysis of point cloud data in four directions. Firstly, it introduces the novel task of NCD for point cloud semantic segmentation. Secondly, it demonstrates that directly transposing the only existing NCD method for 2D image semantic segmentation to 3D data yields suboptimal results. Thirdly, a new NCD approach based on online clustering, uncertainty estimation, and semantic distillation is presented. Lastly, a novel evaluation protocol is proposed to rigorously assess the performance of NCD in point cloud semantic segmentation. Through comprehensive evaluations on the SemanticKITTI, SemanticPOSS, and S3DIS datasets, the paper demonstrates substantial superiority of the proposed method over the considered baselines.
- [507] arXiv:2312.07910 (replaced) [pdf, html, other]
-
Title: PromptBench: A Unified Library for Evaluation of Large Language ModelsComments: Accepted by Journal of Machine Learning Research (JMLR); code: this https URLSubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
The evaluation of large language models (LLMs) is crucial to assess their performance and mitigate potential security risks. In this paper, we introduce PromptBench, a unified library to evaluate LLMs. It consists of several key components that are easily used and extended by researchers: prompt construction, prompt engineering, dataset and model loading, adversarial prompt attack, dynamic evaluation protocols, and analysis tools. PromptBench is designed to be an open, general, and flexible codebase for research purposes that can facilitate original study in creating new benchmarks, deploying downstream applications, and designing new evaluation protocols. The code is available at: this https URL and will be continuously supported.
- [508] arXiv:2312.10943 (replaced) [pdf, html, other]
-
Title: Model Stealing Attack against Graph Classification with Authenticity, Uncertainty and DiversitySubjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR)
Recent research demonstrates that GNNs are vulnerable to the model stealing attack, a nefarious endeavor geared towards duplicating the target model via query permissions. However, they mainly focus on node classification tasks, neglecting the potential threats entailed within the domain of graph classification tasks. Furthermore, their practicality is questionable due to unreasonable assumptions, specifically concerning the large data requirements and extensive model knowledge. To this end, we advocate following strict settings with limited real data and hard-label awareness to generate synthetic data, thereby facilitating the stealing of the target model. Specifically, following important data generation principles, we introduce three model stealing attacks to adapt to different actual scenarios: MSA-AU is inspired by active learning and emphasizes the uncertainty to enhance query value of generated samples; MSA-AD introduces diversity based on Mixup augmentation strategy to alleviate the query inefficiency issue caused by over-similar samples generated by MSA-AU; MSA-AUD combines the above two strategies to seamlessly integrate the authenticity, uncertainty, and diversity of the generated samples. Finally, extensive experiments consistently demonstrate the superiority of the proposed methods in terms of concealment, query efficiency, and stealing performance.
- [509] arXiv:2312.14871 (replaced) [pdf, html, other]
-
Title: BrainVis: Exploring the Bridge between Brain and Visual Signals via Image ReconstructionSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Analyzing and reconstructing visual stimuli from brain signals effectively advances the understanding of human visual system. However, the EEG signals are complex and contain significant noise. This leads to substantial limitations in existing works of visual stimuli reconstruction from EEG, such as difficulties in aligning EEG embeddings with the fine-grained semantic information and a heavy reliance on additional large self-collected dataset for training. To address these challenges, we propose a novel approach called BrainVis. Firstly, we divide the EEG signals into various units and apply a self-supervised approach on them to obtain EEG time-domain features, in an attempt to ease the training difficulty. Additionally, we also propose to utilize the frequency-domain features to enhance the EEG representations. Then, we simultaneously align EEG time-frequency embeddings with the interpolation of the coarse and fine-grained semantics in the CLIP space, to highlight the primary visual components and reduce the cross-modal alignment difficulty. Finally, we adopt the cascaded diffusion models to reconstruct images. Using only 10\% training data of the previous work, our proposed BrainVis outperforms state of the arts in both semantic fidelity reconstruction and generation quality. The code is available at this https URL.
- [510] arXiv:2312.17503 (replaced) [pdf, html, other]
-
Title: HiBid: A Cross-Channel Constrained Bidding System with Budget Allocation by Hierarchical Offline Deep Reinforcement LearningHao Wang, Bo Tang, Chi Harold Liu, Shangqin Mao, Jiahong Zhou, Zipeng Dai, Yaqi Sun, Qianlong Xie, Xingxing Wang, Dong WangSubjects: Machine Learning (cs.LG); Computer Science and Game Theory (cs.GT)
Online display advertising platforms service numerous advertisers by providing real-time bidding (RTB) for the scale of billions of ad requests every day. The bidding strategy handles ad requests cross multiple channels to maximize the number of clicks under the set financial constraints, i.e., total budget and cost-per-click (CPC), etc. Different from existing works mainly focusing on single channel bidding, we explicitly consider cross-channel constrained bidding with budget allocation. Specifically, we propose a hierarchical offline deep reinforcement learning (DRL) framework called ``HiBid'', consisted of a high-level planner equipped with auxiliary loss for non-competitive budget allocation, and a data augmentation enhanced low-level executor for adaptive bidding strategy in response to allocated budgets. Additionally, a CPC-guided action selection mechanism is introduced to satisfy the cross-channel CPC constraint. Through extensive experiments on both the large-scale log data and online A/B testing, we confirm that HiBid outperforms six baselines in terms of the number of clicks, CPC satisfactory ratio, and return-on-investment (ROI). We also deploy HiBid on Meituan advertising platform to already service tens of thousands of advertisers every day.
- [511] arXiv:2401.00763 (replaced) [pdf, html, other]
-
Title: New Job, New Gender? Measuring the Social Bias in Image Generation ModelsWenxuan Wang, Haonan Bai, Jen-tse Huang, Yuxuan Wan, Youliang Yuan, Haoyi Qiu, Nanyun Peng, Michael R. LyuComments: ACM MM 2024 OralSubjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)
Image generation models can generate or edit images from a given text. Recent advancements in image generation technology, exemplified by DALL-E and Midjourney, have been groundbreaking. These advanced models, despite their impressive capabilities, are often trained on massive Internet datasets, making them susceptible to generating content that perpetuates social stereotypes and biases, which can lead to severe consequences. Prior research on assessing bias within image generation models suffers from several shortcomings, including limited accuracy, reliance on extensive human labor, and lack of comprehensive analysis. In this paper, we propose BiasPainter, a novel evaluation framework that can accurately, automatically and comprehensively trigger social bias in image generation models. BiasPainter uses a diverse range of seed images of individuals and prompts the image generation models to edit these images using gender, race, and age-neutral queries. These queries span 62 professions, 39 activities, 57 types of objects, and 70 personality traits. The framework then compares the edited images to the original seed images, focusing on the significant changes related to gender, race, and age. BiasPainter adopts a key insight that these characteristics should not be modified when subjected to neutral prompts. Built upon this design, BiasPainter can trigger the social bias and evaluate the fairness of image generation models. We use BiasPainter to evaluate six widely-used image generation models, such as stable diffusion and Midjourney. Experimental results show that BiasPainter can successfully trigger social bias in image generation models. According to our human evaluation, BiasPainter can achieve 90.8% accuracy on automatic bias detection, which is significantly higher than the results reported in previous work.
- [512] arXiv:2401.01524 (replaced) [pdf, html, other]
-
Title: Multimodal self-supervised learning for lesion localizationHao Yang, Hong-Yu Zhou, Cheng Li, Weijian Huang, Jiarun Liu, Yong Liang, Guangming Shi, Hairong Zheng, Qiegen Liu, Shanshan WangSubjects: Computer Vision and Pattern Recognition (cs.CV)
Multimodal deep learning utilizing imaging and diagnostic reports has made impressive progress in the field of medical imaging diagnostics, demonstrating a particularly strong capability for auxiliary diagnosis in cases where sufficient annotation information is lacking. Nonetheless, localizing diseases accurately without detailed positional annotations remains a challenge. Although existing methods have attempted to utilize local information to achieve fine-grained semantic alignment, their capability in extracting the fine-grained semantics of the comprehensive context within reports is limited. To address this problem, a new method is introduced that takes full sentences from textual reports as the basic units for local semantic alignment. This approach combines chest X-ray images with their corresponding textual reports, performing contrastive learning at both global and local levels. The leading results obtained by this method on multiple datasets confirm its efficacy in the task of lesion localization.
- [513] arXiv:2401.03315 (replaced) [pdf, html, other]
-
Title: Malla: Demystifying Real-world Large Language Model Integrated Malicious ServicesComments: Accepted at the 33rd USENIX Security Symposium (USENIX Security '24). The data and code are available at this https URLSubjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
The underground exploitation of large language models (LLMs) for malicious services (i.e., Malla) is witnessing an uptick, amplifying the cyber threat landscape and posing questions about the trustworthiness of LLM technologies. However, there has been little effort to understand this new cybercrime, in terms of its magnitude, impact, and techniques. In this paper, we conduct the first systematic study on 212 real-world Mallas, uncovering their proliferation in underground marketplaces and exposing their operational modalities. Our study discloses the Malla ecosystem, revealing its significant growth and impact on today's public LLM services. Through examining 212 Mallas, we uncovered eight backend LLMs used by Mallas, along with 182 prompts that circumvent the protective measures of public LLM APIs. We further demystify the tactics employed by Mallas, including the abuse of uncensored LLMs and the exploitation of public LLM APIs through jailbreak prompts. Our findings enable a better understanding of the real-world exploitation of LLMs by cybercriminals, offering insights into strategies to counteract this cybercrime.
- [514] arXiv:2401.05544 (replaced) [pdf, html, other]
-
Title: Enhancing Source Code Classification Effectiveness via Prompt Learning Incorporating Knowledge FeaturesComments: Accepted by Scientific ReportsSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Researchers have investigated the potential of leveraging pre-trained language models, such as CodeBERT, to enhance source code-related tasks. Previous methodologies have relied on CodeBERT's '[CLS]' token as the embedding representation of input sequences for task performance, necessitating additional neural network layers to enhance feature representation, which in turn increases computational expenses. These approaches have also failed to fully leverage the comprehensive knowledge inherent within the source code and its associated text, potentially limiting classification efficacy. We propose CodeClassPrompt, a text classification technique that harnesses prompt learning to extract rich knowledge associated with input sequences from pre-trained models, thereby eliminating the need for additional layers and lowering computational costs. By applying an attention mechanism, we synthesize multi-layered knowledge into task-specific features, enhancing classification accuracy. Our comprehensive experimentation across four distinct source code-related tasks reveals that CodeClassPrompt achieves competitive performance while significantly reducing computational overhead.
- [515] arXiv:2401.08398 (replaced) [pdf, html, other]
-
Title: High-Quality Mesh Blendshape Generation from Face Videos via Neural Inverse RenderingSubjects: Graphics (cs.GR); Computer Vision and Pattern Recognition (cs.CV)
Readily editable mesh blendshapes have been widely used in animation pipelines, while recent advancements in neural geometry and appearance representations have enabled high-quality inverse rendering. Building upon these observations, we introduce a novel technique that reconstructs mesh-based blendshape rigs from single or sparse multi-view videos, leveraging state-of-the-art neural inverse rendering. We begin by constructing a deformation representation that parameterizes vertex displacements into differential coordinates with tetrahedral connections, allowing for high-quality vertex deformation on high-resolution meshes. By constructing a set of semantic regulations in this representation, we achieve joint optimization of blendshapes and expression coefficients. Furthermore, to enable a user-friendly multi-view setup with unsynchronized cameras, we propose a neural regressor to model time-varying motion parameters. This approach implicitly considers the time difference across multiple cameras, enhancing the accuracy of motion modeling. Experiments demonstrate that, with the flexible input of single or sparse multi-view videos, we reconstruct personalized high-fidelity blendshapes. These blendshapes are both geometrically and semantically accurate, and they are compatible with industrial animation pipelines. Code and data are available at this https URL.
- [516] arXiv:2401.10110 (replaced) [pdf, html, other]
-
Title: SVIPTR: Fast and Efficient Scene Text Recognition with Vision Permutable ExtractorXianfu Cheng, Weixiao Zhou, Xiang Li, Jian Yang, Hang Zhang, Tao Sun, Wei Zhang, Yuying Mai, Tongliang Li, Xiaoming Chen, Zhoujun LiComments: 10 pages, 4 figures, 6 tablesSubjects: Computer Vision and Pattern Recognition (cs.CV)
Scene Text Recognition (STR) is an important and challenging upstream task for building structured information databases, that involves recognizing text within images of natural scenes. Although current state-of-the-art (SOTA) models for STR exhibit high performance, they typically suffer from low inference efficiency due to their reliance on hybrid architectures comprised of visual encoders and sequence decoders. In this work, we propose a VIsion Permutable extractor for fast and efficient Scene Text Recognition (SVIPTR), which achieves an impressive balance between high performance and rapid inference speeds in the domain of STR. Specifically, SVIPTR leverages a visual-semantic extractor with a pyramid structure, characterized by the Permutation and combination of local and global self-attention layers. This design results in a lightweight and efficient model and its inference is insensitive to input length. Extensive experimental results on various standard datasets for both Chinese and English scene text recognition validate the superiority of SVIPTR. Notably, the SVIPTR-T (Tiny) variant delivers highly competitive accuracy on par with other lightweight models and achieves SOTA inference speeds. Meanwhile, the SVIPTR-L (Large) attains SOTA accuracy in single-encoder-type models, while maintaining a low parameter count and favorable inference speed. Our proposed method provides a compelling solution for the STR challenge, which greatly benefits real-world applications requiring fast and efficient STR. The code is publicly available at this https URL.
- [517] arXiv:2401.11472 (replaced) [pdf, html, other]
-
Title: Abstract Weighted Based Gradual Semantics in Argumentation TheorySubjects: Artificial Intelligence (cs.AI)
Weighted gradual semantics provide an acceptability degree to each argument representing the strength of the argument, computed based on factors including background evidence for the argument, and taking into account interactions between this argument and others. We introduce four important problems linking gradual semantics and acceptability degrees. First, we reexamine the inverse problem, seeking to identify the argument weights of the argumentation framework which lead to a specific final acceptability degree. Second, we ask whether the function mapping between argument weights and acceptability degrees is injective or a homeomorphism onto its image. Third, we ask whether argument weights can be found when preferences, rather than acceptability degrees for arguments are considered. Fourth, we consider the topology of the space of valid acceptability degrees, asking whether "gaps" exist in this space. While different gradual semantics have been proposed in the literature, in this paper, we identify a large family of weighted gradual semantics, called abstract weighted based gradual semantics. These generalise many of the existing semantics while maintaining desirable properties such as convergence to a unique fixed point. We also show that a sub-family of the weighted gradual semantics, called abstract weighted (L^p,\lambda,\mu)-based gradual semantics and which include well-known semantics, solve all four of the aforementioned problems.
- [518] arXiv:2401.11880 (replaced) [pdf, html, other]
-
Title: PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System SafetyZaibin Zhang, Yongting Zhang, Lijun Li, Hongzhi Gao, Lijun Wang, Huchuan Lu, Feng Zhao, Yu Qiao, Jing ShaoComments: ACL 2024Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Multiagent Systems (cs.MA)
Multi-agent systems, when enhanced with Large Language Models (LLMs), exhibit profound capabilities in collective intelligence. However, the potential misuse of this intelligence for malicious purposes presents significant risks. To date, comprehensive research on the safety issues associated with multi-agent systems remains limited. In this paper, we explore these concerns through the innovative lens of agent psychology, revealing that the dark psychological states of agents constitute a significant threat to safety. To tackle these concerns, we propose a comprehensive framework (PsySafe) grounded in agent psychology, focusing on three key areas: firstly, identifying how dark personality traits in agents can lead to risky behaviors; secondly, evaluating the safety of multi-agent systems from the psychological and behavioral perspectives, and thirdly, devising effective strategies to mitigate these risks. Our experiments reveal several intriguing phenomena, such as the collective dangerous behaviors among agents, agents' self-reflection when engaging in dangerous behavior, and the correlation between agents' psychological assessments and dangerous behaviors. We anticipate that our framework and observations will provide valuable insights for further research into the safety of multi-agent systems. We will make our data and code publicly accessible at this https URL.
- [519] arXiv:2401.12508 (replaced) [pdf, html, other]
-
Title: On the Stochastic (Variance-Reduced) Proximal Gradient Method for Regularized Expected Reward OptimizationComments: 23 pagesSubjects: Machine Learning (cs.LG); Optimization and Control (math.OC)
We consider a regularized expected reward optimization problem in the non-oblivious setting that covers many existing problems in reinforcement learning (RL). In order to solve such an optimization problem, we apply and analyze the classical stochastic proximal gradient method. In particular, the method has shown to admit an $O(\epsilon^{-4})$ sample complexity to an $\epsilon$-stationary point, under standard conditions. Since the variance of the classical stochastic gradient estimator is typically large, which slows down the convergence, we also apply an efficient stochastic variance-reduce proximal gradient method with an importance sampling based ProbAbilistic Gradient Estimator (PAGE). Our analysis shows that the sample complexity can be improved from $O(\epsilon^{-4})$ to $O(\epsilon^{-3})$ under additional conditions. Our results on the stochastic (variance-reduced) proximal gradient method match the sample complexity of their most competitive counterparts for discounted Markov decision processes under similar settings. To the best of our knowledge, the proposed methods represent a novel approach in addressing the general regularized reward optimization problem.
- [520] arXiv:2401.14869 (replaced) [pdf, other]
-
Title: F-Eval: Assessing Fundamental Abilities with Refined Evaluation MethodsYu Sun, Keyu Chen, Shujie Wang, Peiji Li, Qipeng Guo, Hang Yan, Xipeng Qiu, Xuanjing Huang, Dahua LinComments: ACL 2024Subjects: Computation and Language (cs.CL)
Large language models (LLMs) garner significant attention for their unprecedented performance, leading to an increasing number of researches evaluating LLMs. However, these evaluation benchmarks are limited to assessing the instruction-following capabilities, overlooking the fundamental abilities that emerge during the pre-training stage. Previous subjective evaluation methods mainly reply on scoring by API models. However, in the absence of references, large models have shown limited ability to discern subtle differences. To bridge the gap, we propose F-Eval, a bilingual evaluation benchmark to evaluate the fundamental abilities, including expression, commonsense and logic. The tasks in F-Eval include multi-choice objective tasks, open-ended objective tasks, reference-based subjective tasks and reference-free subjective tasks. For reference-free subjective tasks, we devise new evaluation methods, serving as alternatives to scoring by API models. We conduct evaluations on 13 advanced LLMs. Results show that our evaluation methods show higher correlation coefficients and larger distinction than other evaluators. Additionally, we discuss the influence of different model sizes, dimensions, and normalization methods. We anticipate that F-Eval will facilitate the study of LLMs' fundamental abilities.
- [521] arXiv:2401.14962 (replaced) [pdf, html, other]
-
Title: Indirect Lossy Source Coding with Observed Source Reconstruction: Nonasymptotic Bounds and Second-Order AsymptoticsComments: 23 pages, 5 figuresSubjects: Information Theory (cs.IT)
This paper considers the joint compression of a pair of correlated sources, where the encoder is allowed to access only one of the sources. The objective is to recover both sources under separate distortion constraints for each source while minimizing the rate. This problem generalizes the indirect lossy source coding problem by also requiring the recovery of the observed source. In this paper, we aim to study the nonasymptotic and second-order asymptotic properties of this problem. Specifically, we begin by deriving nonasymptotic achievability and converse bounds valid for general sources and distortion measures. The source dispersion (Gaussian approximation) is then determined through asymptotic analysis of the nonasymptotic bounds. We further examine the case of erased fair coin flips (EFCF) and provide its specific nonasymptotic achievability and converse bounds. Numerical results under the EFCF case demonstrate that our second-order asymptotic approximation closely approximates the optimum rate at appropriately large blocklengths.
- [522] arXiv:2401.17793 (replaced) [pdf, html, other]
-
Title: Optimal Dynamic Ancillary Services Provision Based on Local Power Grid PerceptionComments: 15 pages, 20 FiguresSubjects: Systems and Control (eess.SY)
In this paper, we propose a systematic closed-loop approach to provide optimal dynamic ancillary services with converter-interfaced generation systems based on local power grid perception. In particular, we structurally encode dynamic ancillary services such as fast frequency and voltage regulation in the form of a parametric transfer function matrix, which includes several parameters to define a set of different feasible response behaviors, among which we aim to find the optimal one to be realized by the converter system. Our approach is based on a so-called "perceive-and-optimize" (P&O) strategy: First, we identify a grid dynamic equivalent at the interconnection terminals of the converter system. Second, we consider the closed-loop interconnection of the identified grid equivalent and the parametric transfer function matrix, which we optimize for the set of transfer function parameters, resulting in a stable and optimal closed-loop performance for ancillary services provision. In the process, we ensure that grid-code and device-level requirements are satisfied. Finally, we demonstrate the effectiveness of our approach in different numerical case studies based on a modified Kundur two-area test system.
- [523] arXiv:2402.00220 (replaced) [pdf, other]
-
Title: A Circuit Approach to Constructing Blockchains on BlockchainsComments: In Advances in Financial Technologies (AFT 2024)Subjects: Cryptography and Security (cs.CR)
Since the creation of Bitcoin 15 years ago, there has been an explosion in the number of permissionless blockchains. Each of these blockchains provides an open ledger that anyone can read from and write to. In this multi-chain world, an important question emerges: how can we build a more secure overlay blockchain by reading from and writing to a given set of blockchains? Drawing an analogy with switching circuits, we approach the problem by defining two basic compositional operations between blockchains, serial and triangular compositions, and use these operations as building blocks to construct general overlay blockchains. Under the partially synchronous setting, we have the following results: 1) the serial composition, between two blockchains, yields an overlay blockchain that is safe if at least one of the two underlay blockchains is safe and that is live if both of them are live; 2) the triangular composition between three blockchains, akin to parallel composition of switching circuits, yields an overlay blockchain that is safe if all underlay blockchains are safe and that is live if over half of them are live; 3) repeated composition of these two basic operations can yield all possible tradeoffs of safety and liveness for an overlay blockchain built on arbitrary number of underlay chains. The results are also extended to the synchronous setting.
- [524] arXiv:2402.00827 (replaced) [pdf, html, other]
-
Title: GaussianStyle: Gaussian Head Avatar via StyleGANComments: demo page and code to be updated soonSubjects: Computer Vision and Pattern Recognition (cs.CV)
Existing methods like Neural Radiation Fields (NeRF) and 3D Gaussian Splatting (3DGS) have made significant strides in facial attribute control such as facial animation and components editing, yet they struggle with fine-grained representation and scalability in dynamic head modeling. To address these limitations, we propose GaussianStyle, a novel framework that integrates the volumetric strengths of 3DGS with the powerful implicit representation of StyleGAN. The GaussianStyle preserves structural information, such as expressions and poses, using Gaussian points, while projecting the implicit volumetric representation into StyleGAN to capture high-frequency details and mitigate the over-smoothing commonly observed in neural texture rendering. Experimental outcomes indicate that our method achieves state-of-the-art performance in reenactment, novel view synthesis, and animation.
- [525] arXiv:2402.01975 (replaced) [pdf, html, other]
-
Title: Structure-Aware E(3)-Invariant Molecular Conformer Aggregation NetworksDuy M. H. Nguyen, Nina Lukashina, Tai Nguyen, An T. Le, TrungTin Nguyen, Nhat Ho, Jan Peters, Daniel Sonntag, Viktor Zaverkin, Mathias NiepertComments: Accepted at ICML 2024 (updated version)Subjects: Machine Learning (cs.LG)
A molecule's 2D representation consists of its atoms, their attributes, and the molecule's covalent bonds. A 3D (geometric) representation of a molecule is called a conformer and consists of its atom types and Cartesian coordinates. Every conformer has a potential energy, and the lower this energy, the more likely it occurs in nature. Most existing machine learning methods for molecular property prediction consider either 2D molecular graphs or 3D conformer structure representations in isolation. Inspired by recent work on using ensembles of conformers in conjunction with 2D graph representations, we propose $\mathrm{E}$(3)-invariant molecular conformer aggregation networks. The method integrates a molecule's 2D representation with that of multiple of its conformers. Contrary to prior work, we propose a novel 2D-3D aggregation mechanism based on a differentiable solver for the Fused Gromov-Wasserstein Barycenter problem and the use of an efficient conformer generation method based on distance geometry. We show that the proposed aggregation mechanism is $\mathrm{E}$(3) invariant and propose an efficient GPU implementation. Moreover, we demonstrate that the aggregation mechanism helps to significantly outperform state-of-the-art molecule property prediction methods on established datasets.
- [526] arXiv:2402.03629 (replaced) [pdf, html, other]
-
Title: Disparate Impact on Group Accuracy of Linearization for Private InferenceComments: Extended version of the paper accepted to appear at the Forty-first International Conference on Machine Learning (ICML) 2024Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Computers and Society (cs.CY)
Ensuring privacy-preserving inference on cryptographically secure data is a well-known computational challenge. To alleviate the bottleneck of costly cryptographic computations in non-linear activations, recent methods have suggested linearizing a targeted portion of these activations in neural networks. This technique results in significantly reduced runtimes with often negligible impacts on accuracy. In this paper, we demonstrate that such computational benefits may lead to increased fairness costs. Specifically, we find that reducing the number of ReLU activations disproportionately decreases the accuracy for minority groups compared to majority groups. To explain these observations, we provide a mathematical interpretation under restricted assumptions about the nature of the decision boundary, while also showing the prevalence of this problem across widely used datasets and architectures. Finally, we show how a simple procedure altering the fine-tuning step for linearized models can serve as an effective mitigation strategy.
- [527] arXiv:2402.03641 (replaced) [pdf, html, other]
-
Title: Stable BDF time discretization of BGN-based parametric finite element methods for geometric flowsSubjects: Numerical Analysis (math.NA)
We propose a novel class of temporal high-order parametric finite element methods for solving a wide range of geometric flows of curves and surfaces. By incorporating the backward differentiation formulae (BDF) for time discretization into the BGN formulation, originally proposed by Barrett, Garcke, and Nürnberg (J. Comput. Phys., 222 (2007), pp.~441--467), we successfully develop high-order BGN/BDF$k$ schemes. The proposed BGN/BDF$k$ schemes not only retain almost all the advantages of the classical first-order BGN scheme such as computational efficiency and good mesh quality, but also exhibit the desired $k$th-order temporal accuracy in terms of shape metrics, ranging from second-order to fourth-order accuracy. Furthermore, we validate the performance of our proposed BGN/BDF$k$ schemes through extensive numerical examples, demonstrating their high-order temporal accuracy for various types of geometric flows while maintaining good mesh quality throughout the evolution.
- [528] arXiv:2402.04863 (replaced) [pdf, html, other]
-
Title: SCLA: Automated Smart Contract Summarization via LLMs and Semantic AugmentationSubjects: Software Engineering (cs.SE)
In the rapidly evolving world of blockchain systems, the efficient development and maintenance of smart contracts has become a critical task. Smart contract code summarization can significantly facilitate the maintenance of smart contracts and mitigate their vulnerabilities. Large Language Models (LLMs), such as GPT-4o and Gemini-1.5-Pro, possess the capability to generate code summarizations from code examples embedded in prompts. However, the performance of LLMs in code summarization remains suboptimal compared to fine-tuning-based models (e.g., CodeT5+, CodeBERT). Therefore, we propose SCLA, a framework leveraging LLMs and semantic augmentation to improve code summarization performance. SCLA constructs the smart contract's Abstract Syntax Tree (AST) to extract latent semantics, thereby forming a semantically augmented prompt. For evaluation, we utilize a large-scale dataset comprising 40,000 real-world contracts. Experimental results demonstrate that SCLA, with its enhanced prompt, significantly improves the quality of code summarizations. SCLA surpasses other state-of-the-art models (e.g., CodeBERT, CodeT5, and CodeT5+), achieving 37.53% BLEU-4, 52.54% METEOR, 56.97% ROUGE-L, and 63.44% BLEURT, respectively.
- [529] arXiv:2402.05535 (replaced) [pdf, other]
-
Title: Batch-Schedule-Execute: On Optimizing Concurrent Deterministic Scheduling for Blockchains (Extended Version)Comments: Updated version for conference version; 53 pages, 12 figures, LaTeX with Auxiliary Files, short single lineSubjects: Distributed, Parallel, and Cluster Computing (cs.DC); Databases (cs.DB)
Executing smart contracts is a compute and storage-intensive task, which currently dominates modern blockchain's performance. Given that computers are becoming increasingly multicore, concurrency is an attractive approach to improve programs' execution runtime. A unique challenge of blockchains is that all replicas (miners or validators) must execute all smart contracts in the same logical order to maintain the semantics of State Machine Replication (SMR).
In this work, we study the maximal level of parallelism attainable when focusing on the conflict graph between transactions packaged in the same block. This exposes a performance vulnerability that block creators may exploit against existing blockchain concurrency solutions, which rely on a total ordering phase for maintaining consistency amongst all replicas. To facilitate the formal aspects of our study, we develop a novel generic framework for Active State Machine Replication (ASMR) that is strictly serializable. We introduce the concept of graph scheduling and the definition of the minimal latency scheduling problem, which we prove to be NP-hard. We show that the restricted version of this problem for homogeneous transactions is equivalent to the classic Graph Vertex Coloring Problem, yet show that the heterogeneous case is more complex. We discuss the practical implications of these results. - [530] arXiv:2402.06871 (replaced) [pdf, html, other]
-
Title: Non-autoregressive Generative Models for Reranking RecommendationComments: Accepted by KDD 2024Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)
Contemporary recommendation systems are designed to meet users' needs by delivering tailored lists of items that align with their specific demands or interests. In a multi-stage recommendation system, reranking plays a crucial role by modeling the intra-list correlations among items. The key challenge of reranking lies in the exploration of optimal sequences within the combinatorial space of permutations. Recent research proposes a generator-evaluator learning paradigm, where the generator generates multiple feasible sequences and the evaluator picks out the best sequence based on the estimated listwise score. The generator is of vital importance, and generative models are well-suited for the generator function. Current generative models employ an autoregressive strategy for sequence generation. However, deploying autoregressive models in real-time industrial systems is challenging. To address these issues, we propose a Non-AutoRegressive generative model for reranking Recommendation (NAR4Rec) designed to enhance efficiency and effectiveness. To tackle challenges such as sparse training samples and dynamic candidates, we introduce a matching model. Considering the diverse nature of user feedback, we employ a sequence-level unlikelihood training objective to differentiate feasible sequences from unfeasible ones. Additionally, to overcome the lack of dependency modeling in non-autoregressive models regarding target items, we introduce contrastive decoding to capture correlations among these items. Extensive offline experiments validate the superior performance of NAR4Rec over state-of-the-art reranking methods. Online A/B tests reveal that NAR4Rec significantly enhances the user experience. Furthermore, NAR4Rec has been fully deployed in a popular video app Kuaishou with over 300 million daily active users.
- [531] arXiv:2402.07037 (replaced) [pdf, html, other]
-
Title: Recursive Model-agnostic Inverse Dynamics of Serial Soft-Rigid RobotsSubjects: Robotics (cs.RO)
Robotics is shifting from rigid, articulated systems to more sophisticated and heterogeneous mechanical structures. Soft robots, for example, have continuously deformable elements capable of large deformations. The flourishing of control techniques developed for this class of systems is fueling the need of efficient procedures for evaluating their inverse dynamics (ID), which is challenging due to the complex and mixed nature of these systems. As of today, no single ID algorithm can describe the behavior of generic (combinations of) models of soft robots. We address this challenge for generic series-like interconnections of possibly soft structures that may require heterogeneous modeling techniques. Our proposed algorithm requires as input a purely geometric description (forward-kinematics-like) of the mapping from configuration space to deformation space. With this information only, the complete equations of motion can be given an exact recursive structure which is essentially independent from (or `agnostic' to) the underlying reduced-order kinematic modeling techniques. We achieve this goal by exploiting Kane's method to manipulate the equations of motion, showing then their recursive structure. The resulting ID algorithms have optimal computational complexity within the proposed setting, i.e., linear in the number of distinct modules. Further, a variation of the algorithm is introduced that can evaluate the generalized mass matrix without increasing computation costs. We showcase the applicability of this method to robot models involving a mixture of rigid and soft elements, described via possibly heterogeneous reduced order models (ROMs), such as Volumetric FEM, Cosserat strain-based, and volume-preserving deformation primitives. None of these systems can be handled using existing ID techniques.
- [532] arXiv:2402.07345 (replaced) [pdf, html, other]
-
Title: Computing Krylov iterates in the time of matrix multiplicationComments: 10 pages, 5 algorithmsSubjects: Symbolic Computation (cs.SC)
Krylov methods rely on iterated matrix-vector products $A^k u_j$ for an $n\times n$ matrix $A$ and vectors $u_1,\ldots,u_m$. The space spanned by all iterates $A^k u_j$ admits a particular basis -- the \emph{maximal Krylov basis} -- which consists of iterates of the first vector $u_1, Au_1, A^2u_1,\ldots$, until reaching linear dependency, then iterating similarly the subsequent vectors until a basis is obtained. Finding minimal polynomials and Frobenius normal forms is closely related to computing maximal Krylov bases. The fastest way to produce these bases was, until this paper, Keller-Gehrig's 1985 algorithm whose complexity bound $O(n^\omega \log(n))$ comes from repeated squarings of $A$ and logarithmically many Gaussian eliminations. Here $\omega>2$ is a feasible exponent for matrix multiplication over the base field. We present an algorithm computing the maximal Krylov basis in $O(n^\omega\log\log(n))$ field operations when $m \in O(n)$, and even $O(n^\omega)$ as soon as $m\in O(n/\log(n)^c)$ for some fixed real $c>0$. As a consequence, we show that the Frobenius normal form together with a transformation matrix can be computed deterministically in $O(n^\omega (\log\log(n))^2)$, and therefore matrix exponentiation~$A^k$ can be performed in the latter complexity if $\log(k) \in O(n^{\omega-1-\varepsilon})$ for some fixed $\varepsilon>0$. A key idea for these improvements is to rely on fast algorithms for $m\times m$ polynomial matrices of average degree $n/m$, involving high-order lifting and minimal kernel bases.
- [533] arXiv:2402.10543 (replaced) [pdf, html, other]
-
Title: Strong hallucinations from negation and how to fix themComments: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Findings)Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Despite great performance on many tasks, language models (LMs) still struggle with reasoning, sometimes providing responses that cannot possibly be true because they stem from logical incoherence. We call such responses \textit{strong hallucinations} and prove that they follow from an LM's computation of its internal representations for logical operators and outputs from those representations. Focusing on negation, we provide a novel solution in which negation is treated not as another element of a latent representation, but as \textit{an operation over an LM's latent representations that constrains how they may evolve}. We show that our approach improves model performance in cloze prompting and natural language inference tasks with negation without requiring training on sparse negative data.
- [534] arXiv:2402.10725 (replaced) [pdf, html, other]
-
Title: Cloud Kitchen: Using Planning-based Composite AI to Optimize Food Delivery ProcessesSubjects: Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)
The global food delivery market provides many opportunities for AI-based services that can improve the efficiency of feeding the world. This paper presents the Cloud Kitchen platform as a decision-making tool for restaurants with food delivery and a simulator to evaluate the impact of the decisions. The platform contains a Technology-Specific Bridge (TSB) that provides an interface for communicating with restaurants or the simulator. TSB uses a planning domain model to represent decisions embedded in the Unified Planning Framework (UPF). Decision-making, which concerns allocating customers' orders to vehicles and deciding in which order the customers will be served (for each vehicle), is done via a Vehicle Routing Problem with Time Windows (VRPTW), an efficient tool for this problem. We show that decisions made by our platform can improve customer satisfaction by reducing the number of delayed deliveries using a real-world historical dataset.
- [535] arXiv:2402.10726 (replaced) [pdf, html, other]
-
Title: Planning Domain Model Acquisition from State Traces without Action ParametersTomáš Balyo, Martin Suda, Lukáš Chrpa, Dominik Šafránek, Stephan Gocht, Filip Dvořák, Roman Barták, G. Michael YoungbloodSubjects: Artificial Intelligence (cs.AI)
Previous STRIPS domain model acquisition approaches that learn from state traces start with the names and parameters of the actions to be learned. Therefore their only task is to deduce the preconditions and effects of the given actions. In this work, we explore learning in situations when the parameters of learned actions are not provided. We define two levels of trace quality based on which information is provided and present an algorithm for each. In one level (L1), the states in the traces are labeled with action names, so we can deduce the number and names of the actions, but we still need to work out the number and types of parameters. In the other level (L2), the states are additionally labeled with objects that constitute the parameters of the corresponding grounded actions. Here we still need to deduce the types of the parameters in the learned actions. We experimentally evaluate the proposed algorithms and compare them with the state-of-the-art learning tool FAMA on a large collection of IPC benchmarks. The evaluation shows that our new algorithms are faster, can handle larger inputs and provide better results in terms of learning action models more similar to reference models.
- [536] arXiv:2402.10825 (replaced) [pdf, html, other]
-
Title: Nash Equilibrium and Learning Dynamics in Three-Player Matching $m$-Action GamesComments: 9 pages, 4 figures (main), 9 pages, 1 figure (appendix)Subjects: Computer Science and Game Theory (cs.GT); Multiagent Systems (cs.MA); Optimization and Control (math.OC); Chaotic Dynamics (nlin.CD)
Learning in games discusses the processes where multiple players learn their optimal strategies through the repetition of game plays. The dynamics of learning between two players in zero-sum games, such as matching pennies, where their benefits are competitive, have already been well analyzed. However, it is still unexplored and challenging to analyze the dynamics of learning among three players. In this study, we formulate a minimalistic game where three players compete to match their actions with one another. Although interaction among three players diversifies and complicates the Nash equilibria, we fully analyze the equilibria. We also discuss the dynamics of learning based on some famous algorithms categorized into Follow the Regularized Leader. From both theoretical and experimental aspects, we characterize the dynamics by categorizing three-player interactions into three forces to synchronize their actions, switch their actions rotationally, and seek competition.
- [537] arXiv:2402.11173 (replaced) [pdf, html, other]
-
Title: How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex OptimizationComments: ICML 2024Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Optimization and Control (math.OC)
We provide a simple and flexible framework for designing differentially private algorithms to find approximate stationary points of non-convex loss functions. Our framework is based on using a private approximate risk minimizer to "warm start" another private algorithm for finding stationary points. We use this framework to obtain improved, and sometimes optimal, rates for several classes of non-convex loss functions. First, we obtain improved rates for finding stationary points of smooth non-convex empirical loss functions. Second, we specialize to quasar-convex functions, which generalize star-convex functions and arise in learning dynamical systems and training some neural nets. We achieve the optimal rate for this class. Third, we give an optimal algorithm for finding stationary points of functions satisfying the Kurdyka-Lojasiewicz (KL) condition. For example, over-parameterized neural networks often satisfy this condition. Fourth, we provide new state-of-the-art rates for stationary points of non-convex population loss functions. Fifth, we obtain improved rates for non-convex generalized linear models. A modification of our algorithm achieves nearly the same rates for second-order stationary points of functions with Lipschitz Hessian, improving over the previous state-of-the-art for each of the above problems.
- [538] arXiv:2402.11266 (replaced) [pdf, html, other]
-
Title: Filtered Lie-Trotter splitting for the "good" Boussinesq equation: low regularity error estimatesComments: 3 figuresSubjects: Numerical Analysis (math.NA)
We investigate a filtered Lie-Trotter splitting scheme for the ``good" Boussinesq equation and derive an error estimate for initial data with very low regularity. Through the use of discrete Bourgain spaces, our analysis extends to initial data in $H^{s}$ for $0<s\leq 2$, overcoming the constraint of $s>1/2$ imposed by the bilinear estimate in smooth Sobolev spaces. We establish convergence rates of order $\tau^{s/2}$ in $L^2$ for such levels of regularity. Our analytical findings are supported by numerical experiments.
- [539] arXiv:2402.18744 (replaced) [pdf, html, other]
-
Title: Timer-Based Coverage Control for Mobile SensorsSubjects: Systems and Control (eess.SY)
This work investigates the coverage control problem over a static, bounded, and convex workspace and develops a hybrid extension of the continuous-time Lloyd algorithm. Each agent in a multi-agent system (MAS) is equipped with a timer mechanism that generates intermittent measurement and control update events, which may occur asynchronously between agents. Between consecutive event times, as determined by the corresponding timer mechanism, the controller of each agent is held constant. These controllers are shown to drive the configuration of the MAS into a neighborhood of the set of centroidal Voronoi configurations, i.e., a local minimizer of the standard locational cost. The combination of continuous-time dynamics with intermittently updated control inputs is modeled as a hybrid system. The coverage objective is posed as a set attractivity problem for hybrid systems, where an invariance-based convergence analysis yields sufficient conditions that ensure maximal solutions of the hybrid system asymptotically converge to a desired set. A brief simulation example is included to showcase the result.
- [540] arXiv:2403.00868 (replaced) [pdf, html, other]
-
Title: SoftTiger: A Clinical Foundation Model for Healthcare WorkflowsComments: Accepted to AAAI 2024 Spring Symposium on Clinical Foundation Models, Stanford University, Stanford, CaliforniaSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
We introduce SoftTiger, a clinical large language model (CLaM) designed as a foundation model for healthcare workflows. The narrative and unstructured nature of clinical notes is a major obstacle for healthcare intelligentization. We address a critical problem of structuring clinical notes into clinical data, according to international interoperability standards. We collect and annotate data for three subtasks, namely, international patient summary, clinical impression and medical encounter. We then supervised fine-tuned a state-of-the-art LLM using public and credentialed clinical data. The training is orchestrated in a way that the target model can first support basic clinical tasks such as abbreviation expansion and temporal information extraction, and then learn to perform more complex downstream clinical tasks. Moreover, we address several modeling challenges in the healthcare context, e.g., extra long context window. Our blind pairwise evaluation shows that SoftTiger outperforms other popular open-source models and GPT-3.5, comparable to Gemini-pro, with a mild gap from GPT-4. We believe that LLMs may become a step-stone towards healthcare digitalization and democratization. Therefore, we publicly release SoftTiger models at scales of 13 billion and 70 billion parameters, as well as datasets and code for our innovative scalable evaluation, hopefully, making a significant contribution to the healthcare industry.
- [541] arXiv:2403.03174 (replaced) [pdf, html, other]
-
Title: MOKA: Open-Vocabulary Robotic Manipulation through Mark-Based Visual PromptingSubjects: Robotics (cs.RO); Artificial Intelligence (cs.AI)
Open-world generalization requires robotic systems to have a profound understanding of the physical world and the user command to solve diverse and complex tasks. While the recent advancement in vision-language models (VLMs) has offered unprecedented opportunities to solve open-world problems, how to leverage their capabilities to control robots remains a grand challenge. In this paper, we present MOKA (Marking Open-vocabulary Keypoint Affordances), an approach that employs VLMs to solve robotic manipulation tasks specified by free-form language instructions. Central to our approach is a compact point-based representation of affordance, which bridges the VLM's predictions on observed images and the robot's actions in the physical world. By prompting the pre-trained VLM, our approach utilizes the VLM's commonsense knowledge and concept understanding acquired from broad data sources to predict affordances and generate motions. To facilitate the VLM's reasoning in zero-shot and few-shot manners, we propose a visual prompting technique that annotates marks on images, converting affordance reasoning into a series of visual question-answering problems that are solvable by the VLM. We further explore methods to enhance performance with robot experiences collected by MOKA through in-context learning and policy distillation. We evaluate and analyze MOKA's performance on various table-top manipulation tasks including tool use, deformable body manipulation, and object rearrangement.
- [542] arXiv:2403.03740 (replaced) [pdf, other]
-
Title: Self-supervised Photographic Image Layout Representation LearningComments: The authors of the paper believe that there is an error in the measurement of the F1 curve in the metrics descriptionSubjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)
In the domain of image layout representation learning, the critical process of translating image layouts into succinct vector forms is increasingly significant across diverse applications, such as image retrieval, manipulation, and generation. Most approaches in this area heavily rely on costly labeled datasets and notably lack in adapting their modeling and learning methods to the specific nuances of photographic image layouts. This shortfall makes the learning process for photographic image layouts suboptimal. In our research, we directly address these challenges. We innovate by defining basic layout primitives that encapsulate various levels of layout information and by mapping these, along with their interconnections, onto a heterogeneous graph structure. This graph is meticulously engineered to capture the intricate layout information within the pixel domain explicitly. Advancing further, we introduce novel pretext tasks coupled with customized loss functions, strategically designed for effective self-supervised learning of these layout graphs. Building on this foundation, we develop an autoencoder-based network architecture skilled in compressing these heterogeneous layout graphs into precise, dimensionally-reduced layout representations. Additionally, we introduce the LODB dataset, which features a broader range of layout categories and richer semantics, serving as a comprehensive benchmark for evaluating the effectiveness of layout representation learning methods. Our extensive experimentation on this dataset demonstrates the superior performance of our approach in the realm of photographic image layout representation learning.
- [543] arXiv:2403.04151 (replaced) [pdf, html, other]
-
Title: Dual-path Frequency Discriminators for Few-shot Anomaly DetectionComments: Accepted by KBSSubjects: Computer Vision and Pattern Recognition (cs.CV)
Few-shot anomaly detection (FSAD) plays a crucial role in industrial manufacturing. However, existing FSAD methods encounter difficulties leveraging a limited number of normal samples, frequently failing to detect and locate inconspicuous anomalies in the spatial domain. We have further discovered that these subtle anomalies would be more noticeable in the frequency domain. In this paper, we propose a Dual-Path Frequency Discriminators (DFD) network from a frequency perspective to tackle these issues. The original spatial images are transformed into multi-frequency images, making them more conducive to the tailored discriminators in detecting anomalies. Additionally, the discriminators learn a joint representation with forms of pseudo-anomalies. Extensive experiments conducted on MVTec AD and VisA benchmarks demonstrate that our DFD surpasses current state-of-the-art methods. Source code will be available.
- [544] arXiv:2403.05053 (replaced) [pdf, html, other]
-
Title: PrimeComposer: Faster Progressively Combined Diffusion for Image Composition with Attention SteeringComments: Accepted by ACMMM2024. Code: this https URLSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Image composition involves seamlessly integrating given objects into a specific visual context. Current training-free methods rely on composing attention weights from several samplers to guide the generator. However, since these weights are derived from disparate contexts, their combination leads to coherence confusion and loss of appearance information. These issues worsen with their excessive focus on background generation, even when unnecessary in this task. This not only impedes their swift implementation but also compromises foreground generation quality. Moreover, these methods introduce unwanted artifacts in the transition area. In this paper, we formulate image composition as a subject-based local editing task, solely focusing on foreground generation. At each step, the edited foreground is combined with the noisy background to maintain scene consistency. To address the remaining issues, we propose PrimeComposer, a faster training-free diffuser that composites the images by well-designed attention steering across different noise levels. This steering is predominantly achieved by our Correlation Diffuser, utilizing its self-attention layers at each step. Within these layers, the synthesized subject interacts with both the referenced object and background, capturing intricate details and coherent relationships. This prior information is encoded into the attention weights, which are then integrated into the self-attention layers of the generator to guide the synthesis process. Besides, we introduce a Region-constrained Cross-Attention to confine the impact of specific subject-related tokens to desired regions, addressing the unwanted artifacts shown in the prior method thereby further improving the coherence in the transition area. Our method exhibits the fastest inference efficiency and extensive experiments demonstrate our superiority both qualitatively and quantitatively.
- [545] arXiv:2403.06906 (replaced) [pdf, html, other]
-
Title: Cost-Sensitive Learning to Defer to Multiple Experts with Workload ConstraintsJean V. Alves, Diogo Leitão, Sérgio Jesus, Marco O. P. Sampaio, Javier Liébana, Pedro Saleiro, Mário A. T. Figueiredo, Pedro BizarroSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Learning to defer (L2D) aims to improve human-AI collaboration systems by learning how to defer decisions to humans when they are more likely to be correct than an ML classifier. Existing research in L2D overlooks key real-world aspects that impede its practical adoption, namely: i) neglecting cost-sensitive scenarios, where type I and type II errors have different costs; ii) requiring concurrent human predictions for every instance of the training dataset; and iii) not dealing with human work-capacity constraints. To address these issues, we propose the \textit{deferral under cost and capacity constraints framework} (DeCCaF). DeCCaF is a novel L2D approach, employing supervised learning to model the probability of human error under less restrictive data requirements (only one expert prediction per instance) and using constraint programming to globally minimize the error cost, subject to workload limitations. We test DeCCaF in a series of cost-sensitive fraud detection scenarios with different teams of 9 synthetic fraud analysts, with individual work-capacity constraints. The results demonstrate that our approach performs significantly better than the baselines in a wide array of scenarios, achieving an average $8.4\%$ reduction in the misclassification cost. The code used for the experiments is available at this https URL
- [546] arXiv:2403.07162 (replaced) [pdf, html, other]
-
Title: Digital Twin Evolution for Sustainable Smart EcosystemsSubjects: Software Engineering (cs.SE); Emerging Technologies (cs.ET)
Smart ecosystems are the drivers of modern society. They control infrastructures of socio-techno-economic importance, ensuring their stable and sustainable operation. Smart ecosystems are governed by digital twins -- real-time virtual representations of physical infrastructure. To support the open-ended and reactive traits of smart ecosystems, digital twins need to be able to evolve in reaction to changing conditions. However, digital twin evolution is challenged by the intertwined nature of physical and software components, and their individual evolution. As a consequence, software practitioners find a substantial body of knowledge on software evolution hard to apply in digital twin evolution scenarios and a lack of knowledge on the digital twin evolution itself. The aim of this paper, consequently, is to provide software practitioners with tangible leads toward understanding and managing the evolutionary concerns of digital twins. We use four distinct digital twin evolution scenarios, contextualized in a citizen energy community case to illustrate the usage of the 7R taxonomy of digital twin evolution. By that, we aim to bridge a significant gap in leveraging software engineering practices to develop robust smart ecosystems.
- [547] arXiv:2403.08374 (replaced) [pdf, html, other]
-
Title: Efficient Signature-Free Validated AgreementPierre Civit, Muhammad Ayaz Dzulfikar, Seth Gilbert, Rachid Guerraoui, Jovan Komatovic, Manuel Vidigueira, Igor ZablotchiSubjects: Distributed, Parallel, and Cluster Computing (cs.DC)
Byzantine agreement enables n processes to agree on a common L-bit value, despite up to t > 0 arbitrary failures. A long line of work has been dedicated to improving the bit complexity of Byzantine agreement in synchrony. This has culminated in COOL, an error-free (deterministically secure against a computationally unbounded adversary) solution that achieves O(nL + n^2 logn) worst-case bit complexity (which is optimal for L >= n logn according to the Dolev-Reischuk lower bound). COOL satisfies strong unanimity: if all correct processes propose the same value, only that value can be decided. Strong unanimity is, however, not sufficient for today's state machine replication (SMR) and blockchain protocols. These systems value progress and require a decided value to always be valid, excluding default decisions (such as EMPTY) even in cases where there is no unanimity a priori. Validated Byzantine agreement satisfies this property (called external validity). Yet, the best error-free (or even signature-free) validated agreement solutions achieve only O(n^2L) bit complexity, a far cry from the Omega(nL + n^2) Dolev-Reishcuk lower bound. In this paper, we present two new synchronous algorithms for validated Byzantine agreement, HashExt and ErrorFreeExt, with different trade-offs. Both algorithms are (1) signature-free, (2) optimally resilient (tolerate up to t < n / 3 failures), and (3) early-stopping (terminate in O(f+1) rounds, where f <= t is the actual number of failures). On the one hand, HashExt uses only hashes and achieves O(nL + n^3 kappa) bit complexity, which is optimal for L >= n^2 kappa (where kappa is the size of a hash). On the other hand, ErrorFreeExt is error-free, using no cryptography whatsoever, and achieves O( (nL + n^2) logn ) bit complexity, which is near-optimal for any L.
- [548] arXiv:2403.11664 (replaced) [pdf, other]
-
Title: Calibration-Based ALE Model Order Reduction for Hyperbolic Problems with Self-Similar Travelling DiscontinuitiesSubjects: Numerical Analysis (math.NA)
We propose a novel Model Order Reduction framework that is able to handle solutions of hyperbolic problems characterized by multiple travelling discontinuities. By means of an optimization based approach, we introduce suitable calibration maps that allow us to transform the original solution manifold into a lower dimensional one. The novelty of the methodology is represented by the fact that the optimization process does not require the knowledge of the discontinuities location. The optimization can be carried out simply by choosing some reference control points, thus avoiding the use of some implicit shock tracking techniques, which would translate into an increased computational effort during the offline phase. In the online phase, we rely on a non-intrusive approach, where the coefficients of the projection of the reduced order solution onto the reduced space are recovered by means of an Artificial Neural Network. To validate the methodology, we present numerical results for the 1D Sod shock tube problem, for the 2D double Mach reflection problem, also in the parametric case, and for the triple point problem.
- [549] arXiv:2403.13378 (replaced) [pdf, html, other]
-
Title: IIDM: Image-to-Image Diffusion Model for Semantic Image SynthesisComments: 6 pages, 7 figures, accepted by CVMJ 2024Subjects: Computer Vision and Pattern Recognition (cs.CV)
Semantic image synthesis aims to generate high-quality images given semantic conditions, i.e. segmentation masks and style reference images. Existing methods widely adopt generative adversarial networks (GANs). GANs take all conditional inputs and directly synthesize images in a single forward step. In this paper, semantic image synthesis is treated as an image denoising task and is handled with a novel image-to-image diffusion model (IIDM). Specifically, the style reference is first contaminated with random noise and then progressively denoised by IIDM, guided by segmentation masks. Moreover, three techniques, refinement, color-transfer and model ensembles, are proposed to further boost the generation quality. They are plug-in inference modules and do not require additional training. Extensive experiments show that our IIDM outperforms existing state-of-the-art methods by clear margins. Further analysis is provided via detailed demonstrations. We have implemented IIDM based on the Jittor framework; code is available at this https URL.
- [550] arXiv:2403.13780 (replaced) [pdf, html, other]
-
Title: Information-Theoretic Distillation for Reference-less SummarizationSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
The current winning recipe for automatic summarization is using proprietary large-scale language models (LLMs) such as ChatGPT as is, or imitation learning from them as teacher models. While increasingly ubiquitous dependence on such large-scale language models is convenient, there remains an important question of whether small-scale models could have achieved competitive results, if we were to seek an alternative learning method -- that allows for a more cost-efficient, controllable, yet powerful summarizer. We present InfoSumm, a novel framework to distill a powerful summarizer based on the information-theoretic objective for summarization, without relying on either the LLM's capability or human-written references. To achieve this, we first propose a novel formulation of the desiderata of summarization (saliency, faithfulness and brevity) through the lens of mutual information between the original document and the summary. Based on this formulation, we start off from Pythia-2.8B as the teacher model, which is not yet capable of summarization, then self-train the model to optimize for the information-centric measures of ideal summaries. Distilling from the improved teacher, we arrive at a compact but powerful summarizer with only 568M parameters that performs competitively against ChatGPT, without ever relying on ChatGPT's capabilities. Extensive analysis demonstrates that our approach outperforms in-domain supervised models in human evaluation, let alone state-of-the-art unsupervised methods, and wins over ChatGPT in controllable summarization.
- [551] arXiv:2403.15946 (replaced) [pdf, html, other]
-
Title: Team Coordination on Graphs: Problem, Analysis, and AlgorithmsComments: 8 pages, 4 figuresSubjects: Multiagent Systems (cs.MA)
Team Coordination on Graphs with Risky Edges (TCGRE) is a recently emerged problem, in which a robot team collectively reduces graph traversal cost through support from one robot to another when the latter traverses a risky edge. Resembling the traditional Multi-Agent Path Finding (MAPF) problem, both classical and learning-based methods have been proposed to solve TCGRE, however, they lacked either computational efficiency or optimality assurance. In this paper, we reformulate TCGRE as a constrained optimization problem and perform a rigorous mathematical analysis. Our theoretical analysis shows the NP-hardness of TCGRE by reduction from the Maximum 3D Matching problem and that efficient decomposition is a key to tackle this combinatorial optimization problem. Furthermore, we design three classes of algorithms to solve TCGRE, i.e., Joint State Graph (JSG) based, coordination based, and receding-horizon sub-team based solutions. Each of these proposed algorithms enjoy different provable optimality and efficiency characteristics that are demonstrated in our extensive experiments.
- [552] arXiv:2403.17111 (replaced) [pdf, html, other]
-
Title: Vision-Based Dexterous Motion Planning by Dynamic Movement Primitives with Human Hand DemonstrationComments: This paper has been published in 2024 IEEE 33rd International Symposium on Industrial Electronics (ISIE)Subjects: Robotics (cs.RO)
This paper proposes a vision-based framework for a 7-degree-of-freedom robotic manipulator, with the primary objective of facilitating its capacity to acquire information from human hand demonstrations for the execution of dexterous pick-and-place tasks. Most existing works only focus on the position demonstration without considering the orientations. In this paper, by employing a single depth camera, MediaPipe is applied to generate the three-dimensional coordinates of a human hand, thereby comprehensively recording the hand's motion, encompassing the trajectory of the wrist, orientation of the hand, and the grasp motion. A mean filter is applied during data pre-processing to smooth the raw data. The demonstration is designed to pick up an object at a specific angle, navigate around obstacles in its path and subsequently, deposit it within a sloped container. The robotic system demonstrates its learning capabilities, facilitated by the implementation of Dynamic Movement Primitives, enabling the assimilation of user actions into its trajectories with different start and end poi
- [553] arXiv:2403.17442 (replaced) [pdf, html, other]
-
Title: Touch the Core: Exploring Task Dependence Among Hybrid Targets for RecommendationComments: Accepted by RecSys 2024Subjects: Information Retrieval (cs.IR)
As user behaviors become complicated on business platforms, online recommendations focus more on how to touch the core conversions, which are highly related to the interests of platforms. These core conversions are usually continuous targets, such as \textit{watch time}, \textit{revenue}, and so on, whose predictions can be enhanced by previous discrete conversion actions. Therefore, multi-task learning (MTL) can be adopted as the paradigm to learn these hybrid targets. However, existing works mainly emphasize investigating the sequential dependence among discrete conversion actions, which neglects the complexity of dependence between discrete conversions and the final continuous conversion. Moreover, simultaneously optimizing hybrid tasks with stronger task dependence will suffer from volatile issues where the core regression task might have a larger influence on other tasks. In this paper, we study the MTL problem with hybrid targets for the first time and propose the model named Hybrid Targets Learning Network (HTLNet) to explore task dependence and enhance optimization. Specifically, we introduce label embedding for each task to explicitly transfer the label information among these tasks, which can effectively explore logical task dependence. We also further design the gradient adjustment regime between the final regression task and other classification tasks to enhance the optimization. Extensive experiments on two offline public datasets and one real-world industrial dataset are conducted to validate the effectiveness of HTLNet. Moreover, online A/B tests on the financial recommender system also show that our model has improved significantly. Our implementation is available here\footnote{\url{this https URL}}.
- [554] arXiv:2403.19056 (replaced) [pdf, html, other]
-
Title: CAUSE: Counterfactual Assessment of User Satisfaction Estimation in Task-Oriented Dialogue SystemsAmin Abolghasemi, Zhaochun Ren, Arian Askari, Mohammad Aliannejadi, Maarten de Rijke, Suzan VerberneSubjects: Computation and Language (cs.CL)
An important unexplored aspect in previous work on user satisfaction estimation for Task-Oriented Dialogue (TOD) systems is their evaluation in terms of robustness for the identification of user dissatisfaction: current benchmarks for user satisfaction estimation in TOD systems are highly skewed towards dialogues for which the user is satisfied. The effect of having a more balanced set of satisfaction labels on performance is unknown. However, balancing the data with more dissatisfactory dialogue samples requires further data collection and human annotation, which is costly and time-consuming. In this work, we leverage large language models (LLMs) and unlock their ability to generate satisfaction-aware counterfactual dialogues to augment the set of original dialogues of a test collection. We gather human annotations to ensure the reliability of the generated samples. We evaluate two open-source LLMs as user satisfaction estimators on our augmented collection against state-of-the-art fine-tuned models. Our experiments show that when used as few-shot user satisfaction estimators, open-source LLMs show higher robustness to the increase in the number of dissatisfaction labels in the test collection than the fine-tuned state-of-the-art models. Our results shed light on the need for data augmentation approaches for user satisfaction estimation in TOD systems. We release our aligned counterfactual dialogues, which are curated by human annotation, to facilitate further research on this topic.
- [555] arXiv:2404.01099 (replaced) [pdf, html, other]
-
Title: What is in Your Safe Data? Identifying Benign Data that Breaks SafetySubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Cryptography and Security (cs.CR)
Current Large Language Models (LLMs), even those tuned for safety and alignment, are susceptible to jailbreaking. Some have found that just further fine-tuning an aligned model with benign data (i.e., data without harmful content) surprisingly leads to substantial degradation in safety. We delve into the data-centric aspects of why benign fine-tuning inadvertently contributes to jailbreaking. First, we represent fine-tuning data through two lenses: representation and gradient spaces. Additionally, we propose a bi-directional anchoring method that, during the selection process, prioritizes data points that are close to harmful examples and far from benign ones. Our approach effectively identifies subsets of benign data that are more likely to degrade the model's safety after fine-tuning. Training on just 100 of these seemingly benign datapoints surprisingly leads to the fine-tuned model affirmatively responding to >70% of tested harmful requests, compared to <20% after fine-tuning on randomly selected data. We also observe that the selected data frequently appear as lists, bullet points, or math questions, indicating a systematic pattern in fine-tuning data that contributes to jailbreaking.
- [556] arXiv:2404.03455 (replaced) [pdf, html, other]
-
Title: Synergy as the failure of distributivityComments: 30 pages, 4 figuresSubjects: Information Theory (cs.IT); Biological Physics (physics.bio-ph); Data Analysis, Statistics and Probability (physics.data-an)
The concept of emergence, or synergy in its simplest form, is widely used but lacks a rigorous definition. Our work connects information and set theory to uncover the mathematical nature of synergy as the failure of distributivity. It resolves the persistent self-contradiction of information decomposition theory and reinstates it as a primary route toward a rigorous definition of emergence. Our results suggest that non-distributive variants of set theory may be used to describe emergent physical systems.
- [557] arXiv:2404.04162 (replaced) [pdf, html, other]
-
Title: Wireless Resource Optimization in Hybrid Semantic/Bit Communication NetworksComments: This paper has been resubmitted, after completing major revisions, to IEEE Transactions on Communications for possible publication. arXiv admin note: text overlap with arXiv:2408.07820Subjects: Networking and Internet Architecture (cs.NI); Systems and Control (eess.SY)
Recently, semantic communication (SemCom) has shown great potential in significant resource savings and efficient information exchanges, thus naturally introducing a novel and practical cellular network paradigm where two modes of SemCom and conventional bit communication (BitCom) coexist. Nevertheless, the involved wireless resource management becomes rather complicated and challenging, given the unique background knowledge matching and time-consuming semantic coding requirements in SemCom. To this end, this paper jointly investigates user association (UA), mode selection (MS), and bandwidth allocation (BA) problems in a hybrid semantic/bit communication network (HSB-Net). Concretely, we first identify a unified performance metric of message throughput for both SemCom and BitCom links. Next, we specially develop a knowledge matching-aware two-stage tandem packet queuing model and theoretically derive the average packet loss ratio and queuing latency. Combined with practical constraints, we then formulate a joint optimization problem for UA, MS, and BA to maximize the overall message throughput of HSB-Net. Afterward, we propose an optimal resource management strategy by utilizing a Lagrange primal-dual transformation method and a preference list-based heuristic algorithm with polynomial-time complexity. Numerical results not only demonstrate the accuracy of our analytical queuing model, but also validate the performance superiority of our proposed strategy compared with different benchmarks.
- [558] arXiv:2404.06209 (replaced) [pdf, html, other]
-
Title: Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language ModelsComments: COLM camera readySubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
While many have shown how Large Language Models (LLMs) can be applied to a diverse set of tasks, the critical issues of data contamination and memorization are often glossed over. In this work, we address this concern for tabular data. Specifically, we introduce a variety of different techniques to assess whether a language model has seen a tabular dataset during training. This investigation reveals that LLMs have memorized many popular tabular datasets verbatim. We then compare the few-shot learning performance of LLMs on datasets that were seen during training to the performance on datasets released after training. We find that LLMs perform better on datasets seen during training, indicating that memorization leads to overfitting. At the same time, LLMs show non-trivial performance on novel datasets and are surprisingly robust to data transformations. We then investigate the in-context statistical learning abilities of LLMs. While LLMs are significantly better than random at solving statistical classification problems, the sample efficiency of few-shot learning lags behind traditional statistical learning algorithms, especially as the dimension of the problem increases. This suggests that much of the observed few-shot performance on novel real-world datasets is due to the LLM's world knowledge. Overall, our results highlight the importance of testing whether an LLM has seen an evaluation dataset during pre-training. We release the this https URL Python package to test LLMs for memorization of tabular datasets.
- [559] arXiv:2404.06492 (replaced) [pdf, html, other]
-
Title: Graph Reinforcement Learning for Combinatorial Optimization: A Survey and Unifying PerspectiveComments: To appear in Transactions on Machine Learning Research (TMLR)Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Graphs are a natural representation for systems based on relations between connected entities. Combinatorial optimization problems, which arise when considering an objective function related to a process of interest on discrete structures, are often challenging due to the rapid growth of the solution space. The trial-and-error paradigm of Reinforcement Learning has recently emerged as a promising alternative to traditional methods, such as exact algorithms and (meta)heuristics, for discovering better decision-making strategies in a variety of disciplines including chemistry, computer science, and statistics. Despite the fact that they arose in markedly different fields, these techniques share significant commonalities. Therefore, we set out to synthesize this work in a unifying perspective that we term Graph Reinforcement Learning, interpreting it as a constructive decision-making method for graph problems. After covering the relevant technical background, we review works along the dividing line of whether the goal is to optimize graph structure given a process of interest, or to optimize the outcome of the process itself under fixed graph structure. Finally, we discuss the common challenges facing the field and open research questions. In contrast with other surveys, the present work focuses on non-canonical graph problems for which performant algorithms are typically not known and Reinforcement Learning is able to provide efficient and effective solutions.
- [560] arXiv:2404.06624 (replaced) [pdf, html, other]
-
Title: Smooth Actual EIRP Control for EMF Compliance with Minimum Traffic GuaranteesSubjects: Information Theory (cs.IT)
To mitigate Electromagnetic Fields (EMF) human exposure from base stations, international standards bodies define EMF emission requirements that can be translated into limits on the "actual" Equivalent Isotropic Radiated Power (EIRP), i.e., averaged over a sliding time window. We aim to enable base stations to adhere to these constraints while mitigating any impact on user performance. Specifically, our objectives are to: i) ensure EMF exposure compliance using actual EIRP control when implementing the "actual maximum approach" described in IEC 62232:2022, ii) guarantee a minimum EIRP level, and iii) prevent resource shortages at all times. We first investigate exact and conservative algorithms, with linear and constant complexity, respectively, to compute the maximum allowed EIRP consumption under constraints i) and ii), referred to as EIRP "budget". Subsequently, we design a control method based on Drift-Plus-Penalty theory that preemptively curbs EIRP consumption only when needed to avoid future resource shortages.
- [561] arXiv:2404.08382 (replaced) [pdf, html, other]
-
Title: Look at the Text: Instruction-Tuned Language Models are More Robust Multiple Choice Selectors than You ThinkComments: COLM 2024Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Multiple choice questions (MCQs) are commonly used to evaluate the capabilities of large language models (LLMs). One common way to evaluate the model response is to rank the candidate answers based on the log probability of the first token prediction. An alternative way is to examine the text output. Prior work has shown that first token probabilities lack robustness to changes in MCQ phrasing, and that first token probabilities do not match text answers for instruction-tuned models. Therefore, in this paper, we investigate the robustness of text answers. We show that the text answers are more robust to question perturbations than the first token probabilities, when the first token answers mismatch the text answers. The difference in robustness increases as the mismatch rate becomes greater. As the mismatch reaches over 50\%, the text answer is more robust to option order changes than the debiased first token probabilities using state-of-the-art debiasing methods such as PriDe. Our findings provide further evidence for the benefits of text answer evaluation over first token probability evaluation.
- [562] arXiv:2404.08638 (replaced) [pdf, html, other]
-
Title: Age of Information Optimization and State Error Analysis for Correlated Multi-Process Multi-Sensor SystemsSubjects: Information Theory (cs.IT)
In this paper, we examine a multi-sensor system where each sensor may monitor more than one time-varying information process and send status updates to a remote monitor over a common channel. We consider that each sensor's status update may contain information about more than one information process in the system subject to the system's constraints. To investigate the impact of this correlation on the overall system's performance, we conduct an analysis of both the average Age of Information (AoI) and source state estimation error at the monitor. Building upon this analysis, we subsequently explore the impact of the packet arrivals, correlation probabilities, and rate of processes' state change on the system's performance. Next, we consider the case where sensors have limited sensing abilities and distribute a portion of their sensing abilities across the different processes. We optimize this distribution to minimize the total AoI of the system. Interestingly, we show that monitoring multiple processes from a single source may not always be beneficial. Our results also reveal that the optimal sensing distribution for diverse arrival rates may exhibit a rapid regime switch, rather than smooth transitions, after crossing critical system values. This highlights the importance of identifying these critical thresholds to ensure effective system performance.
- [563] arXiv:2404.09872 (replaced) [pdf, html, other]
-
Title: Conditional Prototype Rectification Prompt LearningHaoxing Chen, Yaohui Li, Zizheng Huang, Yan Hong, Zhuoer Xu, Zhangxuan Gu, Jun Lan, Huijia Zhu, Weiqiang WangSubjects: Computer Vision and Pattern Recognition (cs.CV)
Pre-trained large-scale vision-language models (VLMs) have acquired profound understanding of general visual concepts. Recent advancements in efficient transfer learning (ETL) have shown remarkable success in fine-tuning VLMs within the scenario of limited data, introducing only a few parameters to harness task-specific insights from VLMs. Despite significant progress, current leading ETL methods tend to overfit the narrow distributions of base classes seen during training and encounter two primary challenges: (i) only utilizing uni-modal information to modeling task-specific knowledge; and (ii) using costly and time-consuming methods to supplement knowledge. To address these issues, we propose a Conditional Prototype Rectification Prompt Learning (CPR) method to correct the bias of base examples and augment limited data in an effective way. Specifically, we alleviate overfitting on base classes from two aspects. First, each input image acquires knowledge from both textual and visual prototypes, and then generates sample-conditional text tokens. Second, we extract utilizable knowledge from unlabeled data to further refine the prototypes. These two strategies mitigate biases stemming from base classes, yielding a more effective classifier. Extensive experiments on 11 benchmark datasets show that our CPR achieves state-of-the-art performance on both few-shot classification and base-to-new generalization tasks. Our code is avaliable at \url{this https URL}.
- [564] arXiv:2404.10199 (replaced) [pdf, html, other]
-
Title: CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language PromptingHuihan Li, Liwei Jiang, Jena D. Hwang, Hyunwoo Kim, Sebastin Santy, Taylor Sorensen, Bill Yuchen Lin, Nouha Dziri, Xiang Ren, Yejin ChoiSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
As the utilization of large language models (LLMs) has proliferated world-wide, it is crucial for them to have adequate knowledge and fair representation for diverse global cultures. In this work, we uncover culture perceptions of three SOTA models on 110 countries and regions on 8 culture-related topics through culture-conditioned generations, and extract symbols from these generations that are associated to each culture by the LLM. We discover that culture-conditioned generation consist of linguistic "markers" that distinguish marginalized cultures apart from default cultures. We also discover that LLMs have an uneven degree of diversity in the culture symbols, and that cultures from different geographic regions have different presence in LLMs' culture-agnostic generation. Our findings promote further research in studying the knowledge and fairness of global culture perception in LLMs. Code and Data can be found here: this https URL
- [565] arXiv:2404.12965 (replaced) [pdf, html, other]
-
Title: A note on higher-order and nonlinear limiting approaches for continuously bounds-preserving discontinuous Galerkin methodsSubjects: Numerical Analysis (math.NA); Computational Physics (physics.comp-ph)
In (Dzanic, J. Comp. Phys., 508:113010, 2024), a limiting approach for high-order discontinuous Galerkin schemes was introduced which allowed for imposing constraints on the solution continuously (i.e., everywhere within the element). While exact for linear constraint functionals, this approach only imposed a sufficient (but not the minimum necessary) amount of limiting for nonlinear constraint functionals. This short note shows how this limiting approach can be extended to allow exactness for general nonlinear quasiconcave constraint functionals through a nonlinear limiting procedure, reducing unnecessary numerical dissipation. Some examples are shown for nonlinear pressure and entropy constraints in the compressible gas dynamics equations, where both analytic and iterative approaches are used.
- [566] arXiv:2405.00338 (replaced) [pdf, html, other]
-
Title: Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language ModelComments: 11 pages, 2 figuresSubjects: Information Retrieval (cs.IR)
Owing to their powerful semantic reasoning capabilities, Large Language Models (LLMs) have been effectively utilized as recommenders, achieving impressive performance. However, the high inference latency of LLMs significantly restricts their practical deployment. To address this issue, this work investigates knowledge distillation from cumbersome LLM-based recommendation models to lightweight conventional sequential models. It encounters three challenges: 1) the teacher's knowledge may not always be reliable; 2) the capacity gap between the teacher and student makes it difficult for the student to assimilate the teacher's knowledge; 3) divergence in semantic space poses a challenge to distill the knowledge from embeddings. To tackle these challenges, this work proposes a novel distillation strategy, DLLM2Rec, specifically tailored for knowledge distillation from LLM-based recommendation models to conventional sequential models. DLLM2Rec comprises: 1) Importance-aware ranking distillation, which filters reliable and student-friendly knowledge by weighting instances according to teacher confidence and student-teacher consistency; 2) Collaborative embedding distillation integrates knowledge from teacher embeddings with collaborative signals mined from the data. Extensive experiments demonstrate the effectiveness of the proposed DLLM2Rec, boosting three typical sequential models with an average improvement of 47.97%, even enabling them to surpass LLM-based recommenders in some cases.
- [567] arXiv:2405.00885 (replaced) [pdf, html, other]
-
Title: WHALE-FL: Wireless and Heterogeneity Aware Latency Efficient Federated Learning over Mobile Devices via Adaptive Subnetwork SchedulingSubjects: Machine Learning (cs.LG); Networking and Internet Architecture (cs.NI); Image and Video Processing (eess.IV)
As a popular distributed learning paradigm, federated learning (FL) over mobile devices fosters numerous applications, while their practical deployment is hindered by participating devices' computing and communication heterogeneity. Some pioneering research efforts proposed to extract subnetworks from the global model, and assign as large a subnetwork as possible to the device for local training based on its full computing and communications capacity. Although such fixed size subnetwork assignment enables FL training over heterogeneous mobile devices, it is unaware of (i) the dynamic changes of devices' communication and computing conditions and (ii) FL training progress and its dynamic requirements of local training contributions, both of which may cause very long FL training delay. Motivated by those dynamics, in this paper, we develop a wireless and heterogeneity aware latency efficient FL (WHALE-FL) approach to accelerate FL training through adaptive subnetwork scheduling. Instead of sticking to the fixed size subnetwork, WHALE-FL introduces a novel subnetwork selection utility function to capture device and FL training dynamics, and guides the mobile device to adaptively select the subnetwork size for local training based on (a) its computing and communication capacity, (b) its dynamic computing and/or communication conditions, and (c) FL training status and its corresponding requirements for local training contributions. Our evaluation shows that, compared with peer designs, WHALE-FL effectively accelerates FL training without sacrificing learning accuracy.
- [568] arXiv:2405.00981 (replaced) [pdf, html, other]
-
Title: Bayesian Optimization with LLM-Based Acquisition Functions for Natural Language Preference ElicitationSubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Designing preference elicitation (PE) methodologies that can quickly ascertain a user's top item preferences in a cold-start setting is a key challenge for building effective and personalized conversational recommendation (ConvRec) systems. While large language models (LLMs) enable fully natural language (NL) PE dialogues, we hypothesize that monolithic LLM NL-PE approaches lack the multi-turn, decision-theoretic reasoning required to effectively balance the exploration and exploitation of user preferences towards an arbitrary item set. In contrast, traditional Bayesian optimization PE methods define theoretically optimal PE strategies, but cannot generate arbitrary NL queries or reason over content in NL item descriptions -- requiring users to express preferences via ratings or comparisons of unfamiliar items. To overcome the limitations of both approaches, we formulate NL-PE in a Bayesian Optimization (BO) framework that seeks to actively elicit NL feedback to identify the best recommendation. Key challenges in generalizing BO to deal with natural language feedback include determining: (a) how to leverage LLMs to model the likelihood of NL preference feedback as a function of item utilities, and (b) how to design an acquisition function for NL BO that can elicit preferences in the infinite space of language. We demonstrate our framework in a novel NL-PE algorithm, PEBOL, which uses: 1) Natural Language Inference (NLI) between user preference utterances and NL item descriptions to maintain Bayesian preference beliefs, and 2) BO strategies such as Thompson Sampling (TS) and Upper Confidence Bound (UCB) to steer LLM query generation. We numerically evaluate our methods in controlled simulations, finding that after 10 turns of dialogue, PEBOL can achieve an MRR@10 of up to 0.27 compared to the best monolithic LLM baseline's MRR@10 of 0.17, despite relying on earlier and smaller LLMs.
- [569] arXiv:2405.02338 (replaced) [pdf, html, other]
-
Title: Surrealism Me: Interactive Virtual Embodying Experiences in Mixed RealityComments: 4 pages, updated after being accepted to 2024 ACM SIGGRAPH DAC Online ExhibitionSubjects: Human-Computer Interaction (cs.HC)
This paper introduces an interactive Mixed Reality (MR) experience and artistic inquiry entitled ``Surrealism Me'' which delves into Vilém Flusser's critique of media as mediators that often distort human perception of reality and diminish freedom, particularly within the context of MR technology. It engages with Flusser's theories by allowing participants to experience a two-phase virtual embodying (i.e., another body) in MR, highlighting the complex interplay between human agency, body ownership, and self-location. Initially, the participant manipulates their virtual body through various inputs or chooses AI-generated movements. Then, the interactive MR experience leads to an immersive phase where an Unmanned Aerial Vehicle (UAV) extends their sensory perceptions, embodying the virtual body's perspective. ``Surrealism Me'' confronts the concept of ``playing against the apparatus'' by offering an interactive milieu where human and AI collaboratively explore the program's capacity limitation, thereby challenging and exhausting the potential of technology. This process further critically examines the obfuscating nature of media; as the MR medium breaks down, the work reveals the constructed nature of media-projected realities, prompting a reevaluation of media's role and influence on our perception. By navigating the boundary between real and virtual, ``Surrealism Me'' fosters a critical discourse on media's dominance and advocates for a nuanced understanding of Flusserian freedom, encouraging participants to question and reflect on the authentic and mediated experiences of reality.
- [570] arXiv:2405.06161 (replaced) [pdf, html, other]
-
Title: An Introduction to Decentralized Training and Execution in Cooperative Multi-Agent Reinforcement LearningSubjects: Machine Learning (cs.LG); Multiagent Systems (cs.MA)
Multi-agent reinforcement learning (MARL) has exploded in popularity in recent years. Many approaches have been developed but they can be divided into three main types: centralized training and execution (CTE), centralized training for decentralized execution (CTDE), and Decentralized training and execution (DTE). Decentralized training and execution methods make the fewest assumptions and are often simple to implement. In fact, as I'll discuss, any single-agent RL method can be used for DTE by just letting each agent learn separately. Of course, there are pros and cons to such approaches. It is worth noting that DTE is required if no offline coordination is available. That is, if all agents must learn during online interactions without prior coordination, learning and execution must both be decentralized. DTE methods can be applied in cooperative, competitive, or mixed cases but this text will focus on the cooperative MARL case.
This text is an introduction to the field of decentralized, cooperative MARL. As such, I will first give a brief description of the cooperative MARL problem in the form of the Dec-POMDP. Then, I will discuss value-based DTE methods starting with independent Q-learning and its extensions and then discuss the extension to the deep case with DQN, the additional complications this causes, and methods that have been developed to (attempt to) address these issues. Next, I will discuss policy gradient DTE methods starting with independent REINFORCE (i.e., vanilla policy gradient), and then extending to the actor-critic case and deep variants (such as independent PPO). Finally, I will discuss some general topics related to DTE and future directions. - [571] arXiv:2405.06342 (replaced) [pdf, html, other]
-
Title: Compression-Realized Deep Structural Network for Video Quality EnhancementSubjects: Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
This paper focuses on the task of quality enhancement for compressed videos. Although deep network-based video restorers achieve impressive progress, most of the existing methods lack a structured design to optimally leverage the priors within compression codecs. Since the quality degradation of the video is primarily induced by the compression algorithm, a new paradigm is urgently needed for a more ``conscious'' process of quality enhancement. As a result, we propose the Compression-Realized Deep Structural Network (CRDS), introducing three inductive biases aligned with the three primary processes in the classic compression codec, merging the strengths of classical encoder architecture with deep network capabilities. Inspired by the residual extraction and domain transformation process in the codec, a pre-trained Latent Degradation Residual Auto-Encoder is proposed to transform video frames into a latent feature space, and the mutual neighborhood attention mechanism is integrated for precise motion estimation and residual extraction. Furthermore, drawing inspiration from the quantization noise distribution of the codec, CRDS proposes a novel Progressive Denoising framework with intermediate supervision that decomposes the quality enhancement into a series of simpler denoising sub-tasks. Experimental results on datasets like LDV 2.0 and MFQE 2.0 indicate our approach surpasses state-of-the-art models.
- [572] arXiv:2405.08890 (replaced) [pdf, html, other]
-
Title: Language-Guided Self-Supervised Video Summarization Using Text Semantic Matching Considering the Diversity of the VideoSubjects: Computer Vision and Pattern Recognition (cs.CV)
Current video summarization methods rely heavily on supervised computer vision techniques, which demands time-consuming and subjective manual annotations. To overcome these limitations, we investigated self-supervised video summarization. Inspired by the success of Large Language Models (LLMs), we explored the feasibility in transforming the video summarization task into a Natural Language Processing (NLP) task. By leveraging the advantages of LLMs in context understanding, we aim to enhance the effectiveness of self-supervised video summarization. Our method begins by generating captions for individual video frames, which are then synthesized into text summaries by LLMs. Subsequently, we measure semantic distance between the captions and the text summary. Notably, we propose a novel loss function to optimize our model according to the diversity of the video. Finally, the summarized video can be generated by selecting the frames with captions similar to the text summary. Our method achieves state-of-the-art performance on the SumMe dataset in rank correlation coefficients. In addition, our method has a novel feature of being able to achieve personalized summarization.
- [573] arXiv:2405.09863 (replaced) [pdf, html, other]
-
Title: Box-Free Model Watermarks Are Prone to Black-Box Removal AttacksSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Box-free model watermarking is an emerging technique to safeguard the intellectual property of deep learning models, particularly those for low-level image processing tasks. Existing works have verified and improved its effectiveness in several aspects. However, in this paper, we reveal that box-free model watermarking is prone to removal attacks, even under the real-world threat model such that the protected model and the watermark extractor are in black boxes. Under this setting, we carry out three studies. 1) We develop an extractor-gradient-guided (EGG) remover and show its effectiveness when the extractor uses ReLU activation only. 2) More generally, for an unknown extractor, we leverage adversarial attacks and design the EGG remover based on the estimated gradients. 3) Under the most stringent condition that the extractor is inaccessible, we design a transferable remover based on a set of private proxy models. In all cases, the proposed removers can successfully remove embedded watermarks while preserving the quality of the processed images, and we also demonstrate that the EGG remover can even replace the watermarks. Extensive experimental results verify the effectiveness and generalizability of the proposed attacks, revealing the vulnerabilities of the existing box-free methods and calling for further research.
- [574] arXiv:2405.10706 (replaced) [pdf, other]
-
Title: Challenging the Human-in-the-loop in Algorithmic Decision-makingSubjects: Machine Learning (cs.LG)
We discuss the role of humans in algorithmic decision-making (ADM) for socially relevant problems from a technical and philosophical perspective. In particular, we illustrate tensions arising from diverse expectations, values, and constraints by and on the humans involved. To this end, we assume that a strategic decision-maker (SDM) introduces ADM to optimize strategic and societal goals while the algorithms' recommended actions are overseen by a practical decision-maker (PDM) - a specific human-in-the-loop - who makes the final decisions. While the PDM is typically assumed to be a corrective, it can counteract the realization of the SDM's desired goals and societal values not least because of a misalignment of these values and unmet information needs of the PDM. This has significant implications for the distribution of power between the stakeholders in ADM, their constraints, and information needs. In particular, we emphasize the overseeing PDM's role as a potential political and ethical decision maker, who acts expected to balance strategic, value-driven objectives and on-the-ground individual decisions and constraints. We demonstrate empirically, on a machine learning benchmark dataset, the significant impact an overseeing PDM's decisions can have even if the PDM is constrained to performing only a limited amount of actions differing from the algorithms' recommendations. To ensure that the SDM's intended values are realized, the PDM needs to be provided with appropriate information conveyed through tailored explanations and its role must be characterized clearly. Our findings emphasize the need for an in-depth discussion of the role and power of the PDM and challenge the often-taken view that just including a human-in-the-loop in ADM ensures the 'correct' and 'ethical' functioning of the system.
- [575] arXiv:2405.11389 (replaced) [pdf, html, other]
-
Title: Adjacent Leader Decentralized Stochastic Gradient DescentComments: 9 pages of main paper, and 12 pages of appendixSubjects: Machine Learning (cs.LG)
This work focuses on the decentralized deep learning optimization framework. We propose Adjacent Leader Decentralized Gradient Descent (AL-DSGD), for improving final model performance, accelerating convergence, and reducing the communication overhead of decentralized deep learning optimizers. AL-DSGD relies on two main ideas. Firstly, to increase the influence of the strongest learners on the learning system it assigns weights to different neighbor workers according to both their performance and the degree when averaging among them, and it applies a corrective force on the workers dictated by both the currently best-performing neighbor and the neighbor with the maximal degree. Secondly, to alleviate the problem of the deterioration of the convergence speed and performance of the nodes with lower degrees, AL-DSGD relies on dynamic communication graphs, which effectively allows the workers to communicate with more nodes while keeping the degrees of the nodes low. Experiments demonstrate that AL-DSGD accelerates the convergence of the decentralized state-of-the-art techniques and improves their test performance especially in the communication constrained environments. We also theoretically prove the convergence of the proposed scheme. Finally, we release to the community a highly general and concise PyTorch-based library for distributed training of deep learning models that supports easy implementation of any distributed deep learning approach ((a)synchronous, (de)centralized).
- [576] arXiv:2405.11715 (replaced) [pdf, html, other]
-
Title: Semantic Trajectory Data Mining with LLM-Informed POI ClassificationComments: 7 pages, accepted for the 27th IEEE International Conference on Intelligent Transportation Systems (ITSC 2024)Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Human travel trajectory mining is crucial for transportation systems, enhancing route optimization, traffic management, and the study of human travel patterns. Previous rule-based approaches without the integration of semantic information show a limitation in both efficiency and accuracy. Semantic information, such as activity types inferred from Points of Interest (POI) data, can significantly enhance the quality of trajectory mining. However, integrating these insights is challenging, as many POIs have incomplete feature information, and current learning-based POI algorithms require the integrity of datasets to do the classification. In this paper, we introduce a novel pipeline for human travel trajectory mining. Our approach first leverages the strong inferential and comprehension capabilities of large language models (LLMs) to annotate POI with activity types and then uses a Bayesian-based algorithm to infer activity for each stay point in a trajectory. In our evaluation using the OpenStreetMap (OSM) POI dataset, our approach achieves a 93.4% accuracy and a 96.1% F-1 score in POI classification, and a 91.7% accuracy with a 92.3% F-1 score in activity inference.
- [577] arXiv:2405.14554 (replaced) [pdf, html, other]
-
Title: SearchLVLMs: A Plug-and-Play Framework for Augmenting Large Vision-Language Models by Searching Up-to-Date Internet KnowledgeChuanhao Li, Zhen Li, Chenchen Jing, Shuo Liu, Wenqi Shao, Yuwei Wu, Ping Luo, Yu Qiao, Kaipeng ZhangComments: 13 pages, 6 figures, a plug-and-play framework to augment large vision-language models with up-to-date internet knowledgeSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Large vision-language models (LVLMs) are ignorant of the up-to-date knowledge, such as LLaVA series, because they cannot be updated frequently due to the large amount of resources required, and therefore fail in many cases. For example, if a LVLM was released on January 2024, and it wouldn't know the singer of the theme song for the new Detective Conan movie, which wasn't released until April 2024. To solve the problem, a promising solution motivated by retrieval-augmented generation (RAG) is to provide LVLMs with up-to-date knowledge via internet search during inference, i.e., internet-augmented generation (IAG), which is already integrated in some closed-source commercial LVLMs such as GPT-4V. However, the specific mechanics underpinning them remain a mystery. In this paper, we propose a plug-and-play framework, for augmenting existing LVLMs in handling visual question answering (VQA) about up-to-date knowledge, dubbed SearchLVLMs. A hierarchical filtering model is trained to effectively and efficiently find the most helpful content from the websites returned by a search engine to prompt LVLMs with up-to-date knowledge. To train the model and evaluate our framework's performance, we propose a pipeline to automatically generate news-related VQA samples to construct a dataset, dubbed UDK-VQA. A multi-model voting mechanism is introduced to label the usefulness of website/content for VQA samples to construct the training set. Experimental results demonstrate the effectiveness of our framework, outperforming GPT-4V by about 25% in accuracy.
- [578] arXiv:2405.15863 (replaced) [pdf, html, other]
-
Title: QA-MDT: Quality-aware Masked Diffusion Transformer for Enhanced Music GenerationSubjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
In recent years, diffusion-based text-to-music (TTM) generation has gained prominence, offering an innovative approach to synthesizing musical content from textual descriptions. Achieving high accuracy and diversity in this generation process requires extensive, high-quality data, including both high-fidelity audio waveforms and detailed text descriptions, which often constitute only a small portion of available datasets. In open-source datasets, issues such as low-quality music waveforms, mislabeling, weak labeling, and unlabeled data significantly hinder the development of music generation models. To address these challenges, we propose a novel paradigm for high-quality music generation that incorporates a quality-aware training strategy, enabling generative models to discern the quality of input music waveforms during training. Leveraging the unique properties of musical signals, we first adapted and implemented a masked diffusion transformer (MDT) model for the TTM task, demonstrating its distinct capacity for quality control and enhanced musicality. Additionally, we address the issue of low-quality captions in TTM with a caption refinement data processing approach. Experiments demonstrate our state-of-the-art (SOTA) performance on MusicCaps and the Song-Describer Dataset. Our demo page can be accessed at this https URL.
- [579] arXiv:2405.19823 (replaced) [pdf, html, other]
-
Title: Joint Selective State Space Model and Detrending for Robust Time Series Anomaly DetectionComments: Accepted by IEEE Signal Processing Letters. DOI:https://doi.org/10.1109/LSP.2024.3438078Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Deep learning-based sequence models are extensively employed in Time Series Anomaly Detection (TSAD) tasks due to their effective sequential modeling capabilities. However, the ability of TSAD is limited by two key challenges: (i) the ability to model long-range dependency and (ii) the generalization issue in the presence of non-stationary data. To tackle these challenges, an anomaly detector that leverages the selective state space model known for its proficiency in capturing long-term dependencies across various domains is proposed. Additionally, a multi-stage detrending mechanism is introduced to mitigate the prominent trend component in non-stationary data to address the generalization issue. Extensive experiments conducted on realworld public datasets demonstrate that the proposed methods surpass all 12 compared baseline methods.
- [580] arXiv:2405.20132 (replaced) [pdf, html, other]
-
Title: LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating MetaheuristicsComments: Submitted to IEEE TEVCSubjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
Large Language Models (LLMs) such as GPT-4 have demonstrated their ability to understand natural language and generate complex code snippets. This paper introduces a novel Large Language Model Evolutionary Algorithm (LLaMEA) framework, leveraging GPT models for the automated generation and refinement of algorithms. Given a set of criteria and a task definition (the search space), LLaMEA iteratively generates, mutates and selects algorithms based on performance metrics and feedback from runtime evaluations. This framework offers a unique approach to generating optimized algorithms without requiring extensive prior expertise. We show how this framework can be used to generate novel black-box metaheuristic optimization algorithms automatically. LLaMEA generates multiple algorithms that outperform state-of-the-art optimization algorithms (Covariance Matrix Adaptation Evolution Strategy and Differential Evolution) on the five dimensional black box optimization benchmark (BBOB). The algorithms also show competitive performance on the 10- and 20-dimensional instances of the test functions, although they have not seen such instances during the automated generation process. The results demonstrate the feasibility of the framework and identify future directions for automated generation and optimization of algorithms via LLMs.
- [581] arXiv:2405.20219 (replaced) [pdf, html, other]
-
Title: System Identification for Lithium-Ion Batteries with Nonlinear Coupled Electro-Thermal Dynamics via Bayesian OptimizationComments: 2024 American Control Conference(ACC)Subjects: Systems and Control (eess.SY)
Essential to various practical applications of lithium-ion batteries is the availability of accurate equivalent circuit models. This paper presents a new coupled electro-thermal model for batteries and studies how to extract it from data. We consider the problem of maximum likelihood parameter estimation, which, however, is nontrivial to solve as the model is nonlinear in both its dynamics and measurement. We propose to leverage the Bayesian optimization approach, owing to its machine learning-driven capability in handling complex optimization problems and searching for global optima. To enhance the parameter search efficiency, we dynamically narrow and refine the search space in Bayesian optimization. The proposed system identification approach can efficiently determine the parameters of the coupled electro-thermal model. It is amenable to practical implementation, with few requirements on the experiment, data types, and optimization setups, and well applicable to many other battery models.
- [582] arXiv:2405.20327 (replaced) [pdf, html, other]
-
Title: GECO: Generative Image-to-3D within a SECOndComments: Project Page: this https URLSubjects: Computer Vision and Pattern Recognition (cs.CV)
Recent years have seen significant advancements in 3D generation. While methods like score distillation achieve impressive results, they often require extensive per-scene optimization, which limits their time efficiency. On the other hand, reconstruction-based approaches are more efficient but tend to compromise quality due to their limited ability to handle uncertainty. We introduce GECO, a novel method for high-quality 3D generative modeling that operates within a second. Our approach addresses the prevalent issues of uncertainty and inefficiency in existing methods through a two-stage approach. In the first stage, we train a single-step multi-view generative model with score distillation. Then, a second-stage distillation is applied to address the challenge of view inconsistency in the multi-view generation. This two-stage process ensures a balanced approach to 3D generation, optimizing both quality and efficiency. Our comprehensive experiments demonstrate that GECO achieves high-quality image-to-3D mesh generation with an unprecedented level of efficiency. We will make the code and model publicly available.
- [583] arXiv:2405.20470 (replaced) [pdf, html, other]
-
Title: STHN: Deep Homography Estimation for UAV Thermal Geo-localization with Satellite ImageryComments: 8 pages, 7 figures. Accepted for IEEE Robotics and Automation LettersSubjects: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
Accurate geo-localization of Unmanned Aerial Vehicles (UAVs) is crucial for outdoor applications including search and rescue operations, power line inspections, and environmental monitoring. The vulnerability of Global Navigation Satellite Systems (GNSS) signals to interference and spoofing necessitates the development of additional robust localization methods for autonomous navigation. Visual Geo-localization (VG), leveraging onboard cameras and reference satellite maps, offers a promising solution for absolute localization. Specifically, Thermal Geo-localization (TG), which relies on image-based matching between thermal imagery with satellite databases, stands out by utilizing infrared cameras for effective nighttime localization. However, the efficiency and effectiveness of current TG approaches, are hindered by dense sampling on satellite maps and geometric noises in thermal query images. To overcome these challenges, we introduce STHN, a novel UAV thermal geo-localization approach that employs a coarse-to-fine deep homography estimation method. This method attains reliable thermal geo-localization within a 512-meter radius of the UAV's last known location even with a challenging 11\% size ratio between thermal and satellite images, despite the presence of indistinct textures and self-similar patterns. We further show how our research significantly enhances UAV thermal geo-localization performance and robustness against geometric noises under low-visibility conditions in the wild. The code is made publicly available.
- [584] arXiv:2406.00497 (replaced) [pdf, html, other]
-
Title: Recent Advances in End-to-End Simultaneous Speech TranslationComments: Accepted by IJCAI 2024Subjects: Sound (cs.SD); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)
Simultaneous speech translation (SimulST) is a demanding task that involves generating translations in real-time while continuously processing speech input. This paper offers a comprehensive overview of the recent developments in SimulST research, focusing on four major challenges. Firstly, the complexities associated with processing lengthy and continuous speech streams pose significant hurdles. Secondly, satisfying real-time requirements presents inherent difficulties due to the need for immediate translation output. Thirdly, striking a balance between translation quality and latency constraints remains a critical challenge. Finally, the scarcity of annotated data adds another layer of complexity to the task. Through our exploration of these challenges and the proposed solutions, we aim to provide valuable insights into the current landscape of SimulST research and suggest promising directions for future exploration.
- [585] arXiv:2406.01290 (replaced) [pdf, html, other]
-
Title: Resource-constrained FairnessSubjects: Machine Learning (cs.LG); Computers and Society (cs.CY)
Access to resources strongly constrains the decisions we make. While we might wish to offer every student a scholarship, or schedule every patient for follow-up meetings with a specialist, limited resources mean that this is not possible. When deploying machine learning systems, these resource constraints are simply enforced by varying the threshold of a classifier. However, these finite resource limitations are disregarded by most existing tools for fair machine learning, which do not allow the specification of resource limitations and do not remain fair when varying thresholds. This makes them ill-suited for real-world deployment. Our research introduces the concept of "resource-constrained fairness" and quantifies the cost of fairness within this framework. We demonstrate that the level of available resources significantly influences this cost, a factor overlooked in previous evaluations.
- [586] arXiv:2406.01297 (replaced) [pdf, html, other]
-
Title: When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMsComments: Accepted for publication in Transactions of the Association for Computational Linguistics (TACL), 2024. Author's final versionSubjects: Computation and Language (cs.CL)
Self-correction is an approach to improving responses from large language models (LLMs) by refining the responses using LLMs during inference. Prior work has proposed various self-correction frameworks using different sources of feedback, including self-evaluation and external feedback. However, there is still no consensus on the question of when LLMs can correct their own mistakes, as recent studies also report negative results. In this work, we critically survey broad papers and discuss the conditions required for successful self-correction. We first find that prior studies often do not define their research questions in detail and involve impractical frameworks or unfair evaluations that over-evaluate self-correction. To tackle these issues, we categorize research questions in self-correction research and provide a checklist for designing appropriate experiments. Our critical survey based on the newly categorized research questions shows that (1) no prior work demonstrates successful self-correction with feedback from prompted LLMs, except for studies in tasks that are exceptionally suited for self-correction, (2) self-correction works well in tasks that can use reliable external feedback, and (3) large-scale fine-tuning enables self-correction.
- [587] arXiv:2406.01631 (replaced) [pdf, html, other]
-
Title: SUBER: An RL Environment with Simulated Human Behavior for Recommender SystemsSubjects: Information Retrieval (cs.IR); Machine Learning (cs.LG)
Reinforcement learning (RL) has gained popularity in the realm of recommender systems due to its ability to optimize long-term rewards and guide users in discovering relevant content. However, the successful implementation of RL in recommender systems is challenging because of several factors, including the limited availability of online data for training on-policy methods. This scarcity requires expensive human interaction for online model training. Furthermore, the development of effective evaluation frameworks that accurately reflect the quality of models remains a fundamental challenge in recommender systems. To address these challenges, we propose a comprehensive framework for synthetic environments that simulate human behavior by harnessing the capabilities of large language models (LLMs). We complement our framework with in-depth ablation studies and demonstrate its effectiveness with experiments on movie and book recommendations. Using LLMs as synthetic users, this work introduces a modular and novel framework to train RL-based recommender systems. The software, including the RL environment, is publicly available on GitHub.
- [588] arXiv:2406.03151 (replaced) [pdf, html, other]
-
Title: Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and EvaluationHao Li, Yuping Wu, Viktor Schlegel, Riza Batista-Navarro, Tharindu Madusanka, Iqra Zahid, Jiayan Zeng, Xiaochi Wang, Xinran He, Yizhi Li, Goran NenadicComments: Published on ACL 2024 FindingsSubjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
With the recent advances of large language models (LLMs), it is no longer infeasible to build an automated debate system that helps people to synthesise persuasive arguments. Previous work attempted this task by integrating multiple components. In our work, we introduce an argument mining dataset that captures the end-to-end process of preparing an argumentative essay for a debate, which covers the tasks of claim and evidence identification (Task 1 ED), evidence convincingness ranking (Task 2 ECR), argumentative essay summarisation and human preference ranking (Task 3 ASR) and metric learning for automated evaluation of resulting essays, based on human feedback along argument quality dimensions (Task 4 SQE). Our dataset contains 14k examples of claims that are fully annotated with the various properties supporting the aforementioned tasks. We evaluate multiple generative baselines for each of these tasks, including representative LLMs. We find, that while they show promising results on individual tasks in our benchmark, their end-to-end performance on all four tasks in succession deteriorates significantly, both in automated measures as well as in human-centred evaluation. This challenge presented by our proposed dataset motivates future research on end-to-end argument mining and summarisation. The repository of this project is available at this https URL
- [589] arXiv:2406.05036 (replaced) [pdf, html, other]
-
Title: TimeSieve: Extracting Temporal Dynamics through Information BottlenecksSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Time series forecasting has become an increasingly popular research area due to its critical applications in various real-world domains such as traffic management, weather prediction, and financial analysis. Despite significant advancements, existing models face notable challenges, including the necessity of manual hyperparameter tuning for different datasets, and difficulty in effectively distinguishing signal from redundant features in data characterized by strong seasonality. These issues hinder the generalization and practical application of time series forecasting models. To solve this issues, we propose an innovative time series forecasting model TimeSieve designed to address these challenges. Our approach employs wavelet transforms to preprocess time series data, effectively capturing multi-scale features without the need for additional parameters or manual hyperparameter tuning. Additionally, we introduce the information bottleneck theory that filters out redundant features from both detail and approximation coefficients, retaining only the most predictive information. This combination reduces significantly improves the model's accuracy. Extensive experiments demonstrate that our model outperforms existing state-of-the-art methods on 70% of the datasets, achieving higher predictive accuracy and better generalization across diverse datasets. Our results validate the effectiveness of our approach in addressing the key challenges in time series forecasting, paving the way for more reliable and efficient predictive models in practical applications. The code for our model is available at this https URL.
- [590] arXiv:2406.07835 (replaced) [pdf, html, other]
-
Title: SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific LiteratureDavid Wadden, Kejian Shi, Jacob Morrison, Aakanksha Naik, Shruti Singh, Nitzan Barzilay, Kyle Lo, Tom Hope, Luca Soldaini, Shannon Zejiang Shen, Doug Downey, Hannaneh Hajishirzi, Arman CohanComments: Submitted to NeurIPS Datasets and Benchmarks 2024Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
We present SciRIFF (Scientific Resource for Instruction-Following and Finetuning), a dataset of 137K instruction-following demonstrations for 54 tasks covering five essential scientific literature understanding capabilities: information extraction, summarization, question answering, claim verification, and classification. SciRIFF demonstrations are notable for their long input contexts, detailed task specifications, and complex structured outputs. While instruction-following resources are available in specific domains such as clinical medicine and chemistry, SciRIFF is the first dataset focused on extracting and synthesizing information from research literature across a wide range of scientific fields. To demonstrate the utility of SciRIFF, we develop a sample-efficient strategy to adapt a general instruction-following model for science by performing additional finetuning on a mix of general-domain and SciRIFF demonstrations. In evaluations on nine held-out scientific tasks, our model -- called SciTulu -- improves over a strong LLM baseline by 28.1% and 6.5% at the 7B and 70B scales respectively, while maintaining general instruction-following performance within 2% of the baseline. We are optimistic that SciRIFF will facilitate the development and evaluation of LLMs to help researchers navigate the ever-growing body of scientific literature. We release our dataset, model checkpoints, and data processing and evaluation code to enable further research.
- [591] arXiv:2406.10868 (replaced) [pdf, html, other]
-
Title: Identifying Query-Relevant Neurons in Large Language Models for Long-Form TextsComments: 14 pagesSubjects: Computation and Language (cs.CL)
Large Language Models (LLMs) possess vast amounts of knowledge within their parameters, prompting research into methods for locating and editing this knowledge. Previous work has largely focused on locating entity-related (often single-token) facts in smaller models. However, several key questions remain unanswered: (1) How can we effectively locate query-relevant neurons in contemporary autoregressive LLMs, such as Llama and Mistral? (2) How can we address the challenge of long-form text generation? (3) Are there localized knowledge regions in LLMs? In this study, we introduce Query-Relevant Neuron Cluster Attribution (QRNCA), a novel architecture-agnostic framework capable of identifying query-relevant neurons in LLMs. QRNCA allows for the examination of long-form answers beyond triplet facts by employing the proxy task of multi-choice question answering. To evaluate the effectiveness of our detected neurons, we build two multi-choice QA datasets spanning diverse domains and languages. Empirical evaluations demonstrate that our method outperforms baseline methods significantly. Further, analysis of neuron distributions reveals the presence of visible localized regions, particularly within different domains. Finally, we show potential applications of our detected neurons in knowledge editing and neuron-based prediction.
- [592] arXiv:2406.11709 (replaced) [pdf, html, other]
-
Title: Instruct, Not Assist: LLM-based Multi-Turn Planning and Hierarchical Questioning for Socratic Code DebuggingSubjects: Computation and Language (cs.CL); Multiagent Systems (cs.MA)
Socratic questioning is an effective teaching strategy, encouraging critical thinking and problem-solving. The conversational capabilities of large language models (LLMs) show great potential for providing scalable, real-time student guidance. However, current LLMs often give away solutions directly, making them ineffective instructors. We tackle this issue in the code debugging domain with TreeInstruct, an Instructor agent guided by a novel state space-based planning algorithm. TreeInstruct asks probing questions to help students independently identify and resolve errors. It estimates a student's conceptual and syntactical knowledge to dynamically construct a question tree based on their responses and current knowledge state, effectively addressing both independent and dependent mistakes concurrently in a multi-turn interaction setting. In addition to using an existing single-bug debugging benchmark, we construct a more challenging multi-bug dataset of 150 coding problems, incorrect solutions, and bug fixes -- all carefully constructed and annotated by experts. Extensive evaluation shows TreeInstruct's state-of-the-art performance on both datasets, proving it to be a more effective instructor than baselines. Furthermore, a real-world case study with five students of varying skill levels further demonstrates TreeInstruct's ability to guide students to debug their code efficiently with minimal turns and highly Socratic questioning. We provide our code and datasets at this http URL .
- [593] arXiv:2406.13281 (replaced) [pdf, html, other]
-
Title: ECAFormer: Low-light Image Enhancement using Cross AttentionSubjects: Computer Vision and Pattern Recognition (cs.CV)
Low-light image enhancement (LLIE) is critical in computer vision. Existing LLIE methods often fail to discover the underlying relationships between different sub-components, causing the loss of complementary information between multiple modules and network layers, ultimately resulting in the loss of image details. To beat this shortage, we design a hierarchical mutual Enhancement via a Cross Attention transformer (ECAFormer), which introduces an architecture that enables concurrent propagation and interaction of multiple features. The model preserves detailed information by introducing a Dual Multi-head self-attention (DMSA), which leverages visual and semantic features across different scales, allowing them to guide and complement each other. Besides, a Cross-Scale DMSA block is introduced to capture the residual connection, integrating cross-layer information to further enhance image detail. Experimental results show that ECAFormer reaches competitive performance across multiple benchmarks, yielding nearly a 3% improvement in PSNR over the suboptimal method, demonstrating the effectiveness of information interaction in LLIE.
- [594] arXiv:2406.13629 (replaced) [pdf, html, other]
-
Title: InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized RationalesComments: Code: this https URLSubjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Retrieval-augmented generation (RAG) has shown promising potential to enhance the accuracy and factuality of language models (LMs). However, imperfect retrievers or noisy corpora can introduce misleading or even erroneous information to the retrieved contents, posing a significant challenge to the generation quality. Existing RAG methods typically address this challenge by directly predicting final answers despite potentially noisy inputs, resulting in an implicit denoising process that is difficult to interpret and verify. On the other hand, the acquisition of explicit denoising supervision is often costly, involving significant human efforts. In this work, we propose InstructRAG, where LMs explicitly learn the denoising process through self-synthesized rationales -- First, we instruct the LM to explain how the ground-truth answer is derived from retrieved documents. Then, these rationales can be used either as demonstrations for in-context learning of explicit denoising or as supervised fine-tuning data to train the model. Compared to standard RAG approaches, InstructRAG requires no additional supervision, allows for easier verification of the predicted answers, and effectively improves generation accuracy. Experiments show InstructRAG consistently outperforms existing RAG methods in both training-free and trainable scenarios, achieving a relative improvement of 8.3% over the best baseline method on average across five knowledge-intensive benchmarks. Extensive analysis indicates that InstructRAG scales well with increased numbers of retrieved documents and consistently exhibits robust denoising ability even in out-of-domain datasets, demonstrating strong generalizability.
- [595] arXiv:2406.14281 (replaced) [pdf, html, other]
-
Title: FairX: A comprehensive benchmarking tool for model analysis using fairness, utility, and explainabilitySubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
We present FairX, an open-source Python-based benchmarking tool designed for the comprehensive analysis of models under the umbrella of fairness, utility, and eXplainability (XAI). FairX enables users to train benchmarking bias-removal models and evaluate their fairness using a wide array of fairness metrics, data utility metrics, and generate explanations for model predictions, all within a unified framework. Existing benchmarking tools do not have the way to evaluate synthetic data generated from fair generative models, also they do not have the support for training fair generative models either. In FairX, we add fair generative models in the collection of our fair-model library (pre-processing, in-processing, post-processing) and evaluation metrics for evaluating the quality of synthetic fair data. This version of FairX supports both tabular and image datasets. It also allows users to provide their own custom datasets. The open-source FairX benchmarking package is publicly available at this https URL.
- [596] arXiv:2406.14722 (replaced) [pdf, html, other]
-
Title: Does GPT Really Get It? A Hierarchical Scale to Quantify Human vs AI's Understanding of AlgorithmsComments: 13 pages, 8 figuresSubjects: Artificial Intelligence (cs.AI)
As Large Language Models (LLMs) perform (and sometimes excel at) more and more complex cognitive tasks, a natural question is whether AI really understands. The study of understanding in LLMs is in its infancy, and the community has yet to incorporate well-trodden research in philosophy, psychology, and education. We initiate this, specifically focusing on understanding algorithms, and propose a hierarchy of levels of understanding. We use the hierarchy to design and conduct a study with human subjects (undergraduate and graduate students) as well as large language models (generations of GPT), revealing interesting similarities and differences. We expect that our rigorous criteria will be useful to keep track of AI's progress in such cognitive domains.
- [597] arXiv:2406.16153 (replaced) [pdf, html, other]
-
Title: RowPress Vulnerability in Modern DRAM ChipsHaocong Luo, Ataberk Olgun, A. Giray Yağlıkçı, Yahya Can Tuğrul, Steve Rhyner, Meryem Banu Cavlak, Joël Lindegger, Mohammad Sadrosadati, Onur MutluComments: To Appear in IEEE MICRO Top Picks Special Issue (July-August 2024). arXiv admin note: substantial text overlap with arXiv:2306.17061Subjects: Hardware Architecture (cs.AR); Cryptography and Security (cs.CR)
Memory isolation is a critical property for system reliability, security, and safety. We demonstrate RowPress, a DRAM read disturbance phenomenon different from the well-known RowHammer. RowPress induces bitflips by keeping a DRAM row open for a long period of time instead of repeatedly opening and closing the row. We experimentally characterize RowPress bitflips, showing their widespread existence in commodity off-the-shelf DDR4 DRAM chips. We demonstrate RowPress bitflips in a real system that already has RowHammer protection, and propose effective mitigation techniques that protect DRAM against both RowHammer and RowPress.
- [598] arXiv:2406.18159 (replaced) [pdf, html, other]
-
Title: Human-Aware 3D Scene Generation with Spatially-constrained Diffusion ModelsSubjects: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
Generating 3D scenes from human motion sequences supports numerous applications, including virtual reality and architectural design. However, previous auto-regression-based human-aware 3D scene generation methods have struggled to accurately capture the joint distribution of multiple objects and input humans, often resulting in overlapping object generation in the same space. To address this limitation, we explore the potential of diffusion models that simultaneously consider all input humans and the floor plan to generate plausible 3D scenes. Our approach not only satisfies all input human interactions but also adheres to spatial constraints with the floor plan. Furthermore, we introduce two spatial collision guidance mechanisms: human-object collision avoidance and object-room boundary constraints. These mechanisms help avoid generating scenes that conflict with human motions while respecting layout constraints. To enhance the diversity and accuracy of human-guided scene generation, we have developed an automated pipeline that improves the variety and plausibility of human-object interactions in the existing 3D FRONT HUMAN dataset. Extensive experiments on both synthetic and real-world datasets demonstrate that our framework can generate more natural and plausible 3D scenes with precise human-scene interactions, while significantly reducing human-object collisions compared to previous state-of-the-art methods. Our code and data will be made publicly available upon publication of this work.
- [599] arXiv:2406.18938 (replaced) [pdf, html, other]
-
Title: Towards Personalized Federated Multi-Scenario Multi-Task RecommendationYue Ding, Yanbiao Ji, Xun Cai, Xin Xin, Yuxiang Lu, Suizhi Huang, Chang Liu, Xiaofeng Gao, Tsuyoshi Murata, Hongtao LuSubjects: Information Retrieval (cs.IR)
In modern recommender systems, especially in e-commerce, predicting multiple targets such as click-through rate (CTR) and post-view conversion rate (CTCVR) is common. Multi-task recommender systems are increasingly popular in both research and practice, as they leverage shared knowledge across diverse business scenarios to enhance performance. However, emerging real-world scenarios and data privacy concerns complicate the development of a unified multi-task recommendation model.
In this paper, we propose PF-MSMTrec, a novel framework for personalized federated multi-scenario multi-task recommendation. In this framework, each scenario is assigned to a dedicated client utilizing the Multi-gate Mixture-of-Experts (MMoE) structure. To address the unique challenges of multiple optimization conflicts, we introduce a bottom-up joint learning mechanism. First, we design a parameter template to decouple the expert network parameters, distinguishing scenario-specific parameters as shared knowledge for federated parameter aggregation. Second, we implement personalized federated learning for each expert network during a federated communication round, using three modules: federated batch normalization, conflict coordination, and personalized aggregation. Finally, we conduct an additional round of personalized federated parameter aggregation on the task tower network to obtain prediction results for multiple tasks. Extensive experiments on two public datasets demonstrate that our proposed method outperforms state-of-the-art approaches. The source code and datasets will be released as open-source for public access. - [600] arXiv:2407.00688 (replaced) [pdf, other]
-
Title: On the Number of Quantifiers Needed to Define Boolean FunctionsComments: Full version of version that is to appear in Proceedings of 49th International Symposium on Mathematical Foundations of Computer Science, 2024. arXiv admin note: substantial text overlap with arXiv:2402.10293Subjects: Logic in Computer Science (cs.LO); Computational Complexity (cs.CC)
The number of quantifiers needed to express first-order (FO) properties is captured by two-player combinatorial games called multi-structural games. We analyze these games on binary strings with an ordering relation, using a technique we call parallel play, which significantly reduces the number of quantifiers needed in many cases. Ordered structures such as strings have historically been notoriously difficult to analyze in the context of these and similar games. Nevertheless, in this paper, we provide essentially tight upper bounds on the number of quantifiers needed to characterize different-sized subsets of strings. The results immediately give bounds on the number of quantifiers necessary to define several different classes of Boolean functions. One of our results is analogous to Lupanov's upper bounds on circuit size and formula size in propositional logic: we show that every Boolean function on $n$-bit inputs can be defined by a FO sentence having $(1 + \varepsilon)n\log(n) + O(1)$ quantifiers, and that this is essentially tight. We reduce this number to $(1 + \varepsilon)\log(n) + O(1)$ when the Boolean function in question is sparse.
- [601] arXiv:2407.02337 (replaced) [pdf, html, other]
-
Title: Open foundation models for Azerbaijani languageComments: Presented in the First Workshop on Natural Language Processing for Turkic LanguagesSubjects: Computation and Language (cs.CL)
The emergence of multilingual large language models has enabled the development of language understanding and generation systems in Azerbaijani. However, most of the production-grade systems rely on cloud solutions, such as GPT-4. While there have been several attempts to develop open foundation models for Azerbaijani, these works have not found their way into common use due to a lack of systemic benchmarking. This paper encompasses several lines of work that promote open-source foundation models for Azerbaijani. We introduce (1) a large text corpus for Azerbaijani, (2) a family of encoder-only language models trained on this dataset, (3) labeled datasets for evaluating these models, and (4) extensive evaluation that covers all major open-source models with Azerbaijani support.
- [602] arXiv:2407.03627 (replaced) [pdf, html, other]
-
Title: DSLR: Document Refinement with Sentence-Level Re-ranking and Reconstruction to Enhance Retrieval-Augmented GenerationComments: 20 pagesJournal-ref: KnowledgeNLP@ACL 2024Subjects: Computation and Language (cs.CL)
Recent advancements in Large Language Models (LLMs) have significantly improved their performance across various Natural Language Processing (NLP) tasks. However, LLMs still struggle with generating non-factual responses due to limitations in their parametric memory. Retrieval-Augmented Generation (RAG) systems address this issue by incorporating external knowledge with a retrieval module. Despite their successes, however, current RAG systems face challenges with retrieval failures and the limited ability of LLMs to filter out irrelevant information. Therefore, in this work, we propose DSLR (Document Refinement with Sentence-Level Re-ranking and Reconstruction), an unsupervised framework that decomposes retrieved documents into sentences, filters out irrelevant sentences, and reconstructs them again into coherent passages. We experimentally validate DSLR on multiple open-domain QA datasets and the results demonstrate that DSLR significantly enhances the RAG performance over conventional fixed-size passage. Furthermore, our DSLR enhances performance in specific, yet realistic scenarios without the need for additional training, providing an effective and efficient solution for refining retrieved documents in RAG systems.
- [603] arXiv:2407.03891 (replaced) [pdf, html, other]
-
Title: AutoBench: Automatic Testbench Generation and Evaluation Using LLMs for HDL DesignSubjects: Software Engineering (cs.SE); Programming Languages (cs.PL)
In digital circuit design, testbenches constitute the cornerstone of simulation-based hardware verification. Traditional methodologies for testbench generation during simulation-based hardware verification still remain partially manual, resulting in inefficiencies in testing various scenarios and requiring expensive time from designers. Large Language Models (LLMs) have demonstrated their potential in automating the circuit design flow. However, directly applying LLMs to generate testbenches suffers from a low pass rate. To address this challenge, we introduce AutoBench, the first LLM-based testbench generator for digital circuit design, which requires only the description of the design under test (DUT) to automatically generate comprehensive testbenches. In AutoBench, a hybrid testbench structure and a self-checking system are realized using LLMs. To validate the generated testbenches, we also introduce an automated testbench evaluation framework to evaluate the quality of generated testbenches from multiple perspectives. Experimental results demonstrate that AutoBench achieves a 57% improvement in the testbench pass@1 ratio compared with the baseline that directly generates testbenches using LLMs. For 75 sequential circuits, AutoBench successfully has a 3.36 times testbench pass@1 ratio compared with the baseline. The source codes and experimental results are open-sourced at this link: this https URL
- [604] arXiv:2407.05724 (replaced) [pdf, other]
-
Title: Learning Stochastic Reduced Models from Data: A Nonintrusive ApproachSubjects: Numerical Analysis (math.NA)
A nonintrusive model order reduction method for bilinear stochastic differential equations with additive noise is proposed. A reduced order model (ROM) is designed in order to approximate the statistical properties of high-dimensional systems. The drift and diffusion coefficients of the ROM are inferred from state observations by solving appropriate least-squares problems. The closeness of the ROM obtained by the presented approach to the intrusive ROM obtained by the proper orthogonal decomposition (POD) method is investigated. Two generalisations of the snapshot-based dominant subspace construction to the stochastic case are presented. Numerical experiments are provided to compare the developed approach to POD.
- [605] arXiv:2407.05890 (replaced) [pdf, html, other]
-
Title: Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language NavigationSubjects: Robotics (cs.RO); Computation and Language (cs.CL)
LLM-based agents have demonstrated impressive zero-shot performance in vision-language navigation (VLN) task. However, existing LLM-based methods often focus only on solving high-level task planning by selecting nodes in predefined navigation graphs for movements, overlooking low-level control in navigation scenarios. To bridge this gap, we propose AO-Planner, a novel Affordances-Oriented Planner for continuous VLN task. Our AO-Planner integrates various foundation models to achieve affordances-oriented low-level motion planning and high-level decision-making, both performed in a zero-shot setting. Specifically, we employ a Visual Affordances Prompting (VAP) approach, where the visible ground is segmented by SAM to provide navigational affordances, based on which the LLM selects potential candidate waypoints and plans low-level paths towards selected waypoints. We further propose a high-level PathAgent which marks planned paths into the image input and reasons the most probable path by comprehending all environmental information. Finally, we convert the selected path into 3D coordinates using camera intrinsic parameters and depth information, avoiding challenging 3D predictions for LLMs. Experiments on the challenging R2R-CE and RxR-CE datasets show that AO-Planner achieves state-of-the-art zero-shot performance (8.8% improvement on SPL). Our method can also serve as a data annotator to obtain pseudo-labels, distilling its waypoint prediction ability into a learning-based predictor. This new predictor does not require any waypoint data from the simulator and achieves 47% SR competing with supervised methods. We establish an effective connection between LLM and 3D world, presenting novel prospects for employing foundation models in low-level motion control.
- [606] arXiv:2407.06121 (replaced) [pdf, html, other]
-
Title: Periodic agent-state based Q-learning for POMDPsSubjects: Machine Learning (cs.LG)
The standard approach for Partially Observable Markov Decision Processes (POMDPs) is to convert them to a fully observed belief-state MDP. However, the belief state depends on the system model and is therefore not viable in reinforcement learning (RL) settings. A widely used alternative is to use an agent state, which is a model-free, recursively updateable function of the observation history. Examples include frame stacking and recurrent neural networks. Since the agent state is model-free, it is used to adapt standard RL algorithms to POMDPs. However, standard RL algorithms like Q-learning learn a stationary policy. Our main thesis that we illustrate via examples is that because the agent state does not satisfy the Markov property, non-stationary agent-state based policies can outperform stationary ones. To leverage this feature, we propose PASQL (periodic agent-state based Q-learning), which is a variant of agent-state-based Q-learning that learns periodic policies. By combining ideas from periodic Markov chains and stochastic approximation, we rigorously establish that PASQL converges to a cyclic limit and characterize the approximation error of the converged periodic policy. Finally, we present a numerical experiment to highlight the salient features of PASQL and demonstrate the benefit of learning periodic policies over stationary policies.
- [607] arXiv:2407.12879 (replaced) [pdf, html, other]
-
Title: Large Visual-Language Models Are Also Good Classifiers: A Study of In-Context Multimodal Fake News DetectionSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Large visual-language models (LVLMs) exhibit exceptional performance in visual-language reasoning across diverse cross-modal benchmarks. Despite these advances, recent research indicates that Large Language Models (LLMs), like GPT-3.5-turbo, underachieve compared to well-trained smaller models, such as BERT, in Fake News Detection (FND), prompting inquiries into LVLMs' efficacy in FND tasks. Although performance could improve through fine-tuning LVLMs, the substantial parameters and requisite pre-trained weights render it a resource-heavy endeavor for FND applications. This paper initially assesses the FND capabilities of two notable LVLMs, CogVLM and GPT4V, in comparison to a smaller yet adeptly trained CLIP model in a zero-shot context. The findings demonstrate that LVLMs can attain performance competitive with that of the smaller model. Next, we integrate standard in-context learning (ICL) with LVLMs, noting improvements in FND performance, though limited in scope and consistency. To address this, we introduce the \textbf{I}n-context \textbf{M}ultimodal \textbf{F}ake \textbf{N}ews \textbf{D}etection (IMFND) framework, enriching in-context examples and test inputs with predictions and corresponding probabilities from a well-trained smaller model. This strategic integration directs the LVLMs' focus towards news segments associated with higher probabilities, thereby improving their analytical accuracy. The experimental results suggest that the IMFND framework significantly boosts the FND efficiency of LVLMs, achieving enhanced accuracy over the standard ICL approach across three publicly available FND datasets.
- [608] arXiv:2407.13044 (replaced) [pdf, html, other]
-
Title: DropKAN: Regularizing KANs by masking post-activationsSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
We propose DropKAN (Dropout Kolmogorov-Arnold Networks) a regularization method that prevents co-adaptation of activation function weights in Kolmogorov-Arnold Networks (KANs). DropKAN functions by embedding the drop mask directly within the KAN layer, randomly masking the outputs of some activations within the KANs' computation graph. We show that this simple procedure that require minimal coding effort has a regularizing effect and consistently lead to better generalization of KANs. We analyze the adaptation of the standard Dropout with KANs and demonstrate that Dropout applied to KANs' neurons can lead to unpredictable behavior in the feedforward pass. We carry an empirical study with real world Machine Learning datasets to validate our findings. Our results suggest that DropKAN is consistently a better alternative to using standard Dropout with KANs, and improves the generalization performance of KANs. Our implementation of DropKAN is available at: \url{this https URL}.
- [609] arXiv:2407.13555 (replaced) [pdf, html, other]
-
Title: PetFace: A Large-Scale Dataset and Benchmark for Animal IdentificationComments: ECCV 2024. Dataset and code: this https URLSubjects: Computer Vision and Pattern Recognition (cs.CV)
Automated animal face identification plays a crucial role in the monitoring of behaviors, conducting of surveys, and finding of lost animals. Despite the advancements in human face identification, the lack of datasets and benchmarks in the animal domain has impeded progress. In this paper, we introduce the PetFace dataset, a comprehensive resource for animal face identification encompassing 257,484 unique individuals across 13 animal families and 319 breed categories, including both experimental and pet animals. This large-scale collection of individuals facilitates the investigation of unseen animal face verification, an area that has not been sufficiently explored in existing datasets due to the limited number of individuals. Moreover, PetFace also has fine-grained annotations such as sex, breed, color, and pattern. We provide multiple benchmarks including re-identification for seen individuals and verification for unseen individuals. The models trained on our dataset outperform those trained on prior datasets, even for detailed breed variations and unseen animal families. Our result also indicates that there is some room to improve the performance of integrated identification on multiple animal families. We hope the PetFace dataset will facilitate animal face identification and encourage the development of non-invasive animal automatic identification methods.
- [610] arXiv:2407.16467 (replaced) [pdf, html, other]
-
Title: Side-Channel Analysis of OpenVINO-based Neural Network ModelsSubjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
Embedded devices with neural network accelerators offer great versatility for their users, reducing the need to use cloud-based services. At the same time, they introduce new security challenges in the area of hardware attacks, the most prominent being side-channel analysis (SCA). It was shown that SCA can recover model parameters with a high accuracy, posing a threat to entities that wish to keep their models confidential. In this paper, we explore the susceptibility of quantized models implemented in OpenVINO, an embedded framework for deploying neural networks on embedded and Edge devices. We show that it is possible to recover model parameters with high precision, allowing the recovered model to perform very close to the original one. Our experiments on GoogleNet v1 show only a 1% difference in the Top 1 and a 0.64% difference in the Top 5 accuracies.
- [611] arXiv:2407.16771 (replaced) [pdf, html, other]
-
Title: Topology-Guided ORCA: Smooth Multi-Agent Motion Planning in Constrained EnvironmentsComments: Accepted by Unsolved Problems in Social Robot Navigation workshop in conjunction with RSS 2024Subjects: Robotics (cs.RO)
We present Topology-Guided ORCA as an alternative simulator to replace ORCA for planning smooth multi-agent motions in environments with static obstacles. Despite the impressive performance in simulating multi-agent crowd motion in free space, ORCA encounters a significant challenge in navigating the agents with the presence of static obstacles. ORCA ignores static obstacles until an agent gets too close to an obstacle, and the agent will get stuck if the obstacle intercepts an agent's path toward the goal. To address this challenge, Topology-Guided ORCA constructs a graph to represent the topology of the traversable region of the environment. We use a path planner to plan a path of waypoints that connects each agent's start and goal positions. The waypoints are used as a sequence of goals to guide ORCA. The experiments of crowd simulation in constrained environments show that our method outperforms ORCA in terms of generating smooth and natural motions of multiple agents in constrained environments, which indicates great potential of Topology-Guided ORCA for serving as an effective simulator for training constrained social navigation policies.
- [612] arXiv:2407.16944 (replaced) [pdf, html, other]
-
Title: Adaptive Gradient Regularization: A Faster and Generalizable Optimization Technique for Deep Neural NetworksComments: 12 pages, 13 figuresSubjects: Machine Learning (cs.LG)
Stochastic optimization plays a crucial role in the advancement of deep learning technologies. Over the decades, significant effort has been dedicated to improving the training efficiency and robustness of deep neural networks, via various strategies including gradient normalization (GN) and gradient centralization (GC). Nevertheless, to the best of our knowledge, no one has considered to capture the optimal gradient descent trajectory, by adaptively controlling gradient descent direction. To address this concern, this paper is the first attempt to study a new optimization technique for deep neural networks, using the sum normalization of a gradient vector as coefficients, to dynamically regularize gradients and thus to effectively control optimization direction. The proposed technique is hence named as the adaptive gradient regularization (AGR). It can be viewed as an adaptive gradient clipping method. The theoretical analysis reveals that the AGR can effectively smooth the loss landscape, and hence can significantly improve the training efficiency and model generalization performance. We note that AGR can greatly improve the training efficiency of vanilla optimizers' including Adan and AdamW, by adding only three lines of code. The final experiments conducted on image generation, image classification, and language representation, demonstrate that the AGR method can not only improve the training efficiency but also enhance the model generalization performance.
- [613] arXiv:2407.17657 (replaced) [pdf, other]
-
Title: My Ontologist: Evaluating BFO-Based AI for Definition SupportSubjects: Databases (cs.DB)
Generative artificial intelligence (AI), exemplified by the release of GPT-3.5 in 2022, has significantly advanced the potential applications of large language models (LLMs), including in the realms of ontology development and knowledge graph creation. Ontologies, which are structured frameworks for organizing information, and knowledge graphs, which combine ontologies with actual data, are essential for enabling interoperability and automated reasoning. However, current research has largely overlooked the generation of ontologies extending from established upper-level frameworks like the Basic Formal Ontology (BFO), risking the creation of non-integrable ontology silos. This study explores the extent to which LLMs, particularly GPT-4, can support ontologists trained in BFO. Through iterative development of a specialized GPT model named "My Ontologist," we aimed to generate BFO-conformant ontologies. Initial versions faced challenges in maintaining definition conventions and leveraging foundational texts effectively. My Ontologist 3.0 showed promise by adhering to structured rules and modular ontology suites, yet the release of GPT-4o disrupted this progress by altering the model's behavior. Our findings underscore the importance of aligning LLM-generated ontologies with top-level standards and highlight the complexities of integrating evolving AI capabilities in ontology engineering.
- [614] arXiv:2407.17722 (replaced) [pdf, html, other]
-
Title: Text-Driven Neural Collaborative Filtering Model for Paper Source TracingComments: KDD CUP 2024 OAG-Challenges, Paper Source Tracing, Technical Report of Team AoboSama @ KDD CUP 2024. August 25--29, 2024. Barcelona, SpainSubjects: Information Retrieval (cs.IR); Machine Learning (cs.LG)
Identifying significant references within the complex interrelations of a citation knowledge graph is challenging, which encompasses connections through citations, authorship, keywords, and other relational attributes. The Paper Source Tracing (PST) task seeks to automate the identification of pivotal references for given scholarly articles utilizing advanced data mining techniques. In the KDD CUP OAG-Challenge PST track, we design a recommendation-based framework tailored for the PST task. This framework employs the Neural Collaborative Filtering (NCF) model to generate final predictions. To process the textual attributes of the papers and extract input features for the model, we utilize SciBERT, a pre-trained language model. According to the experimental results, our method achieved a score of 0.37814 on the Mean Average Precision (MAP) metric, outperforming baseline models and ranking 11th among all participating teams. The source code is publicly available at this https URL.
- [615] arXiv:2407.20557 (replaced) [pdf, other]
-
Title: CELLM: An Efficient Communication in Large Language Models Training for Federated LearningComments: arXiv admin note: This submission has been withdrawn by arXiv administrators due to inappropriate text overlap with external sourcesSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Federated Learning (FL) is a recent model training paradigm in which client devices collaboratively train a model without ever aggregating their data. Crucially, this scheme offers users potential privacy and security benefits by only ever communicating updates to the model weights to a central server as opposed to traditional machine learning (ML) training which directly communicates and aggregates data. However, FL training suffers from statistical heterogeneity as clients may have differing local data distributions. Large language models (LLMs) offer a potential solution to this issue of heterogeneity given that they have consistently been shown to be able to learn on vast amounts of noisy data. While LLMs are a promising development for resolving the consistent issue of non-I.I.D. Clients in federated settings exacerbate two other bottlenecks in FL: limited local computing and expensive communication. This thesis aims to develop efficient training methods for LLMs in FL. To this end, we employ two critical techniques in enabling efficient training. First, we use low-rank adaptation (LoRA) to reduce the computational load of local model training. Second, we communicate sparse updates throughout training to significantly cut down on communication costs. Taken together, our method reduces communication costs by up to 10x over vanilla LoRA and up to 5x over more complex sparse LoRA baselines while achieving greater utility. We emphasize the importance of carefully applying sparsity and picking effective rank and sparsity configurations for federated LLM training.
- [616] arXiv:2407.20619 (replaced) [pdf, html, other]
-
Title: ATI-CTLO:Adaptive Temporal Interval-based Continuous-Time LiDAR-Only OdometrySubjects: Robotics (cs.RO)
The motion distortion in LiDAR scans caused by aggressive robot motion and varying terrain features significantly impacts the positioning and mapping performance of 3D LiDAR odometry. Existing distortion correction solutions often struggle to balance computational complexity and accuracy. In this work, we propose an Adaptive Temporal Interval-based Continuous-Time LiDAR-only Odometry, utilizing straightforward and efficient linear interpolation. Our method flexibly adjusts the temporal intervals between control nodes according to the dynamics of motion and environmental characteristics. This adaptability enhances performance across various motion states and improves robustness in challenging, feature-sparse environments. We validate the effectiveness of our method on multiple datasets across different platforms, achieving accuracy comparable to state-of-the-art LiDAR-only odometry methods. Notably, in scenarios involving aggressive motion and sparse features, our method outperforms existing solutions.
- [617] arXiv:2408.01239 (replaced) [pdf, html, other]
-
Title: Tailoring Graph Neural Network-based Flow-guided Localization to Individual Bloodstreams and ActivitiesComments: 7 pages, 9 figures, 2 tables, 16 references, accepted at ACM NanoCom'25Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Networking and Internet Architecture (cs.NI)
Flow-guided localization using in-body nanodevices in the bloodstream is expected to be beneficial for early disease detection, continuous monitoring of biological conditions, and targeted treatment. The nanodevices face size and power constraints that produce erroneous raw data for localization purposes. On-body anchors receive this data, and use it to derive the locations of diagnostic events of interest. Different Machine Learning (ML) approaches have been recently proposed for this task, yet they are currently restricted to a reference bloodstream of a resting patient. As such, they are unable to deal with the physical diversity of patients' bloodstreams and cannot provide continuous monitoring due to changes in individual patient's activities. Toward addressing these issues for the current State-of-the-Art (SotA) flow-guided localization approach based on Graph Neural Networks (GNNs), we propose a pipeline for GNN adaptation based on individual physiological indicators including height, weight, and heart rate. Our results indicate that the proposed adaptions are beneficial in reconciling the individual differences between bloodstreams and activities.
- [618] arXiv:2408.01534 (replaced) [pdf, html, other]
-
Title: An Efficient Real-Time Object Detection Framework on Resource-Constricted Hardware Devices via Software and Hardware Co-designComments: 11 pages, 6 figuresSubjects: Machine Learning (cs.LG)
The fast development of object detection techniques has attracted attention to developing efficient Deep Neural Networks (DNNs). However, the current state-of-the-art DNN models can not provide a balanced solution among accuracy, speed, and model size. This paper proposes an efficient real-time object detection framework on resource-constrained hardware devices through hardware and software co-design. The Tensor Train (TT) decomposition is proposed for compressing the YOLOv5 model. By unitizing the unique characteristics given by the TT decomposition, we develop an efficient hardware accelerator based on FPGA devices. Experimental results show that the proposed method can significantly reduce the model size and improve the execution time.
- [619] arXiv:2408.02110 (replaced) [pdf, html, other]
-
Title: AvatarPose: Avatar-guided 3D Pose Estimation of Close Human Interaction from Sparse Multi-view VideosComments: Project Page: this https URLSubjects: Computer Vision and Pattern Recognition (cs.CV)
Despite progress in human motion capture, existing multi-view methods often face challenges in estimating the 3D pose and shape of multiple closely interacting people. This difficulty arises from reliance on accurate 2D joint estimations, which are hard to obtain due to occlusions and body contact when people are in close interaction. To address this, we propose a novel method leveraging the personalized implicit neural avatar of each individual as a prior, which significantly improves the robustness and precision of this challenging pose estimation task. Concretely, the avatars are efficiently reconstructed via layered volume rendering from sparse multi-view videos. The reconstructed avatar prior allows for the direct optimization of 3D poses based on color and silhouette rendering loss, bypassing the issues associated with noisy 2D detections. To handle interpenetration, we propose a collision loss on the overlapping shape regions of avatars to add penetration constraints. Moreover, both 3D poses and avatars are optimized in an alternating manner. Our experimental results demonstrate state-of-the-art performance on several public datasets.
- [620] arXiv:2408.02123 (replaced) [pdf, html, other]
-
Title: Human-inspired Explanations for Vision Transformers and Convolutional Neural NetworksComments: Accepted at the Human-inspired Computer Vision (HCV) ECCV 2024 Workshop as an extended abstract. A long version of the work can be found at arXiv:2408.02123v1Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
We introduce Foveation-based Explanations (FovEx), a novel human-inspired visual explainability (XAI) method for Deep Neural Networks. Our method achieves state-of-the-art performance on both transformer (on 4 out of 5 metrics) and convolutional models (on 3 out of 5 metrics), demonstrating its versatility. Furthermore, we show the alignment between the explanation map produced by FovEx and human gaze patterns (+14\% in NSS compared to RISE, +203\% in NSS compared to gradCAM), enhancing our confidence in FovEx's ability to close the interpretation gap between humans and machines.
- [621] arXiv:2408.02574 (replaced) [pdf, html, other]
-
Title: DanModCap: Designing a Danmaku Moderation Tool for Video-Sharing Platforms that Leverages Impact CaptionsSubjects: Human-Computer Interaction (cs.HC)
Online video platforms have gained increased popularity due to their ability to support information consumption and sharing and the diverse social interactions they afford. Danmaku, a real-time commentary feature that overlays user comments on a video, has been found to improve user engagement, however, the use of Danmaku can lead to toxic behaviors and inappropriate comments. To address these issues, we propose a proactive moderation approach inspired by Impact Captions, a visual technique used in East Asian variety shows. Impact Captions combine textual content and visual elements to construct emotional and cognitive resonance. Within the context of this work, Impact Captions were used to guide viewers towards positive Danmaku-related activities and elicit more pro-social behaviors. Leveraging Impact Captions, we developed DanModCap, an moderation tool that collected and analyzed Danmaku and used it as input to large generative language models to produce Impact Captions. Our evaluation of DanModCap demonstrated that Impact Captions reduced negative antagonistic emotions, increased users' desire to share positive content, and elicited self-control in Danmaku social action to fostering proactive community maintenance behaviors. Our approach highlights the benefits of using LLM-supported content moderation methods for proactive moderation in a large-scale live content contexts.
- [622] arXiv:2408.03837 (replaced) [pdf, html, other]
-
Title: WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language ModelsPrannaya Gupta, Le Qi Yau, Hao Han Low, I-Shiang Lee, Hugo Maximus Lim, Yu Xin Teoh, Jia Hng Koh, Dar Win Liew, Rishabh Bhardwaj, Rajat Bhardwaj, Soujanya PoriaComments: Under reviewSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
WalledEval is a comprehensive AI safety testing toolkit designed to evaluate large language models (LLMs). It accommodates a diverse range of models, including both open-weight and API-based ones, and features over 35 safety benchmarks covering areas such as multilingual safety, exaggerated safety, and prompt injections. The framework supports both LLM and judge benchmarking and incorporates custom mutators to test safety against various text-style mutations, such as future tense and paraphrasing. Additionally, WalledEval introduces WalledGuard, a new, small, and performant content moderation tool, and two datasets: SGXSTest and HIXSTest, which serve as benchmarks for assessing the exaggerated safety of LLMs and judges in cultural contexts. We make WalledEval publicly available at this https URL.
- [623] arXiv:2408.04380 (replaced) [pdf, html, other]
-
Title: Deep Generative Models in Robotics: A Survey on Learning from Multimodal DemonstrationsJulen Urain, Ajay Mandlekar, Yilun Du, Mahi Shafiullah, Danfei Xu, Katerina Fragkiadaki, Georgia Chalvatzaki, Jan PetersComments: 20 pages, 11 figures, submitted to TROSubjects: Robotics (cs.RO); Machine Learning (cs.LG)
Learning from Demonstrations, the field that proposes to learn robot behavior models from data, is gaining popularity with the emergence of deep generative models. Although the problem has been studied for years under names such as Imitation Learning, Behavioral Cloning, or Inverse Reinforcement Learning, classical methods have relied on models that don't capture complex data distributions well or don't scale well to large numbers of demonstrations. In recent years, the robot learning community has shown increasing interest in using deep generative models to capture the complexity of large datasets. In this survey, we aim to provide a unified and comprehensive review of the last year's progress in the use of deep generative models in robotics. We present the different types of models that the community has explored, such as energy-based models, diffusion models, action value maps, or generative adversarial networks. We also present the different types of applications in which deep generative models have been used, from grasp generation to trajectory generation or cost learning. One of the most important elements of generative models is the generalization out of distributions. In our survey, we review the different decisions the community has made to improve the generalization of the learned models. Finally, we highlight the research challenges and propose a number of future directions for learning deep generative models in robotics.
- [624] arXiv:2408.04472 (replaced) [pdf, html, other]
-
Title: Can LLMs Beat Humans in Debating? A Dynamic Multi-agent Framework for Competitive DebateComments: 12 pages (including appendix), 7 figuresSubjects: Computation and Language (cs.CL)
Competitive debate is a complex task of computational argumentation. Large Language Models (LLMs) suffer from hallucinations and lack competitiveness in this field. To address these challenges, we introduce Agent for Debate (Agent4Debate), a dynamic multi-agent framework based on LLMs designed to enhance their capabilities in competitive debate. Drawing inspiration from human behavior in debate preparation and execution, Agent4Debate employs a collaborative architecture where four specialized agents, involving Searcher, Analyzer, Writer, and Reviewer, dynamically interact and cooperate. These agents work throughout the debate process, covering multiple stages from initial research and argument formulation to rebuttal and summary. To comprehensively evaluate framework performance, we construct the Competitive Debate Arena, comprising 66 carefully selected Chinese debate motions. We recruit ten experienced human debaters and collect records of 200 debates involving Agent4Debate, baseline models, and humans. The evaluation employs the Debatrix automatic scoring system and professional human reviewers based on the established Debatrix-Elo and Human-Elo ranking. Experimental results indicate that the state-of-the-art Agent4Debate exhibits capabilities comparable to those of humans. Furthermore, ablation studies demonstrate the effectiveness of each component in the agent structure.
- [625] arXiv:2408.05398 (replaced) [pdf, html, other]
-
Title: PersonViT: Large-scale Self-supervised Vision Transformer for Person Re-IdentificationSubjects: Computer Vision and Pattern Recognition (cs.CV)
Person Re-Identification (ReID) aims to retrieve relevant individuals in non-overlapping camera images and has a wide range of applications in the field of public safety. In recent years, with the development of Vision Transformer (ViT) and self-supervised learning techniques, the performance of person ReID based on self-supervised pre-training has been greatly improved. Person ReID requires extracting highly discriminative local fine-grained features of the human body, while traditional ViT is good at extracting context-related global features, making it difficult to focus on local human body features. To this end, this article introduces the recently emerged Masked Image Modeling (MIM) self-supervised learning method into person ReID, and effectively extracts high-quality global and local features through large-scale unsupervised pre-training by combining masked image modeling and discriminative contrastive learning, and then conducts supervised fine-tuning training in the person ReID task. This person feature extraction method based on ViT with masked image modeling (PersonViT) has the good characteristics of unsupervised, scalable, and strong generalization capabilities, overcoming the problem of difficult annotation in supervised person ReID, and achieves state-of-the-art results on publicly available benchmark datasets, including MSMT17, Market1501, DukeMTMC-reID, and Occluded-Duke. The code and pre-trained models of the PersonViT method are released at \url{this https URL} to promote further research in the person ReID field.
- [626] arXiv:2408.05431 (replaced) [pdf, html, other]
-
Title: Simple and Nearly-Optimal Sampling for Rank-1 Tensor Completion via Gauss-JordanComments: 16 pages; corrected typos in Prior Work section & Theorem 1.5Subjects: Data Structures and Algorithms (cs.DS); Machine Learning (cs.LG); Statistics Theory (math.ST); Machine Learning (stat.ML)
We revisit the sample and computational complexity of completing a rank-1 tensor in $\otimes_{i=1}^{N} \mathbb{R}^{d}$, given a uniformly sampled subset of its entries. We present a characterization of the problem (i.e. nonzero entries) which admits an algorithm amounting to Gauss-Jordan on a pair of random linear systems. For example, when $N = \Theta(1)$, we prove it uses no more than $m = O(d^2 \log d)$ samples and runs in $O(md^2)$ time. Moreover, we show any algorithm requires $\Omega(d\log d)$ samples.
By contrast, existing upper bounds on the sample complexity are at least as large as $d^{1.5} \mu^{\Omega(1)} \log^{\Omega(1)} d$, where $\mu$ can be $\Theta(d)$ in the worst case. Prior work obtained these looser guarantees in higher rank versions of our problem, and tend to involve more complicated algorithms. - [627] arXiv:2408.05477 (replaced) [pdf, html, other]
-
Title: Scene123: One Prompt to 3D Scene Generation via Video-Assisted and Consistency-Enhanced MAEComments: arXiv admin note: text overlap with arXiv:2305.11588 by other authorsSubjects: Computer Vision and Pattern Recognition (cs.CV)
As Artificial Intelligence Generated Content (AIGC) advances, a variety of methods have been developed to generate text, images, videos, and 3D objects from single or multimodal inputs, contributing efforts to emulate human-like cognitive content creation. However, generating realistic large-scale scenes from a single input presents a challenge due to the complexities involved in ensuring consistency across extrapolated views generated by models. Benefiting from recent video generation models and implicit neural representations, we propose Scene123, a 3D scene generation model, that not only ensures realism and diversity through the video generation framework but also uses implicit neural fields combined with Masked Autoencoders (MAE) to effectively ensures the consistency of unseen areas across views. Specifically, we initially warp the input image (or an image generated from text) to simulate adjacent views, filling the invisible areas with the MAE model. However, these filled images usually fail to maintain view consistency, thus we utilize the produced views to optimize a neural radiance field, enhancing geometric consistency.
Moreover, to further enhance the details and texture fidelity of generated views, we employ a GAN-based Loss against images derived from the input image through the video generation model. Extensive experiments demonstrate that our method can generate realistic and consistent scenes from a single prompt. Both qualitative and quantitative results indicate that our approach surpasses existing state-of-the-art methods. We show encourage video examples at this https URL. - [628] arXiv:2408.05727 (replaced) [pdf, html, other]
-
Title: Hotfixing Large Language Models for Code: How Far Can Parameter-Efficient Fine-Tuning Go?Comments: work in progressSubjects: Software Engineering (cs.SE)
Large Language Models for Code (LLM4Code) have become an integral part of developers' workflows, assisting with tasks such as code completion and generation. However, these models are found to exhibit undesired behaviors after their release, like generating buggy code, due to their extensive training on vast amounts of source code that contain such buggy code. The training data (usually coming from open-source software) keeps evolving, e.g., developers fix the buggy code. However, adapting such evolution to mitigate LLM4Code's undesired behaviors is non-trivial, as retraining models on the updated dataset usually takes much time and resources. This motivates us to propose the concept of hotfixing LLM4Code, mitigating LLM4Code's undesired behaviors effectively and efficiently with minimal negative effects.
This paper mainly focuses on hotfixing LLM4Code to make them generate less buggy code and more fixed code. We begin by demonstrating that models from the popular CodeGen family frequently generate buggy code. Then, we define three learning objectives in hotfixing and design multiple loss functions for each objective: (1) learn the desired behaviors, (2) unlearn the undesired behaviors, and (3) retain knowledge of other code. We evaluate four different fine-tuning techniques for hotfixing the models and gain the following insights. Optimizing these three learning goals together, using LoRA (low-rank adaptation), effectively influences the model's behavior. Specifically, it increases the generation of fixed code by up to 108.42% and decreases the generation of buggy code by up to 50.47%. Statistical tests confirm that hotfixing does not significantly affect the models' functional correctness on the HumanEval benchmark. We also show that hotfixing demonstrates strong time efficiency. - [629] arXiv:2408.06213 (replaced) [pdf, html, other]
-
Title: Batched Ranged Random Integer GenerationComments: software: this https URLSubjects: Data Structures and Algorithms (cs.DS)
Pseudorandom values are often generated as 64-bit binary words. These random words need to be converted into ranged values without statistical bias. We present an efficient algorithm to generate multiple independent uniformly-random bounded integers from a single uniformly-random binary word, without any bias. In the common case, our method uses one multiplication and no division operations per value produced. In practice, our algorithm can more than double the speed of unbiased random shuffling for small to moderately large arrays.
- [630] arXiv:2408.06282 (replaced) [pdf, html, other]
-
Title: A characterization for an almost MDS code to be a near MDS code and a proof of the Geng-Yang-Zhang-Zhou conjectureComments: 25 pages. One reference was added and several typos were corrected in this versionSubjects: Information Theory (cs.IT); Number Theory (math.NT)
Let $\mathbb{F}_q$ be the finite field of $q$ elements, where $q=p^{m}$ with $p$ being a prime number and $m$ being a positive integer. Let $\mathcal{C}_{(q, n, \delta, h)}$ be a class of BCH codes of length $n$ and designed $\delta$. A linear code $\mathcal{C}$ is said to be maximum distance separable (MDS) if the minimum distance $d=n-k+1$. If $d=n-k$, then $\mathcal{C}$ is called an almost MDS (AMDS) code. Moreover, if both of $\mathcal{C}$ and its dual code $\mathcal{C}^{\bot}$ are AMDS, then $\mathcal{C}$ is called a near MDS (NMDS) code. In [A class of almost MDS codes, {\it Finite Fields Appl.} {\bf 79} (2022), \#101996], Geng, Yang, Zhang and Zhou proved that the BCH code $\mathcal{C}_{(q, q+1,3,4)}$ is an almost MDS code, where $q=3^m$ and $m$ is an odd integer, and they also showed that its parameters is $[q+1, q-3, 4]$. Furthermore, they proposed a conjecture stating that the dual code $\mathcal{C}^{\bot}_{(q, q+1, 3, 4)}$ is also an AMDS code with parameters $[q+1, 4, q-3]$. In this paper, we first present a characterization for the dual code of an almost MDS code to be an almost MDS code. Then we use this result to show that the Geng-Yang-Zhang-Zhou conjecture is true. Our result together with the Geng-Yang-Zhang-Zhou theorem implies that the BCH code $\mathcal{C}_{(q, q+1,3,4)}$ is a near MDS code.
- [631] arXiv:2408.06583 (replaced) [pdf, html, other]
-
Title: A Structure-aware Generative Model for Biomedical Event ExtractionComments: 8 pages, 4 figures, 6 tablesSubjects: Computation and Language (cs.CL)
Biomedical Event Extraction (BEE) is a challenging task that involves modeling complex relationships between fine-grained entities in biomedical text. BEE has traditionally been formulated as a classification problem. With the recent technological advancements in large language models (LLMs), generation-based models that cast event extraction as a sequence generation problem have attracted much attention from the NLP research communities. However, current generative models often overlook the importance of cross-instance information from complex event structures such as nested events and overlapping events, which contribute to over 20% of the events in the benchmark datasets. In this paper, we propose an event structure-aware generative model named GenBEE, which can capture complex event structures in biomedical text for biomedical event extraction. In particular, GenBEE constructs event prompts that distill knowledge from LLMs for incorporating both label semantics and argument dependency relationships into the proposed model. In addition, GenBEE also generates prefixes with event structural prompts to incorporate structural features for improving the model's overall performance. We have evaluated the proposed GenBEE model on three widely used biomedical event extraction benchmark datasets, namely MLEE, GE11, and PHEE. Experimental results show that GenBEE has achieved state-of-the-art performance on the MLEE and GE11 datasets, and achieved competitive results when compared to the state-of-the-art classification-based models on the PHEE dataset.
- [632] arXiv:2408.06602 (replaced) [pdf, html, other]
-
Title: Super-intelligence or Superstition? Exploring Psychological Factors Underlying Unwarranted Belief in AI PredictionsSubjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)
This study investigates psychological factors influencing belief in AI predictions about personal behavior, comparing it to belief in astrology and personality-based predictions. Through an experiment with 238 participants, we examined how cognitive style, paranormal beliefs, AI attitudes, personality traits, and other factors affect perceived validity, reliability, usefulness, and personalization of predictions from different sources. Our findings reveal that belief in AI predictions is positively correlated with belief in predictions based on astrology and personality psychology. Notably, paranormal beliefs and positive AI attitudes significantly increased perceived validity, reliability, usefulness, and personalization of AI predictions. Conscientiousness was negatively correlated with belief in predictions across all sources, and interest in the prediction topic increased believability across predictions. Surprisingly, cognitive style did not significantly influence belief in predictions. These results highlight the "rational superstition" phenomenon in AI, where belief is driven more by mental heuristics and intuition than critical evaluation. We discuss implications for designing AI systems and communication strategies that foster appropriate trust and skepticism. This research contributes to our understanding of the psychology of human-AI interaction and offers insights for the design and deployment of AI systems.
- [633] arXiv:2408.06899 (replaced) [pdf, html, other]
-
Title: EEPPR: Event-based Estimation of Periodic Phenomena Rate using Correlation in 3DComments: 13 paper pages + 11 suppl pages, 15 figues, 3 tablesSubjects: Computer Vision and Pattern Recognition (cs.CV)
We present a novel method for measuring the period of phenomena like rotation, flicker and vibration, by an event camera, a device asynchronously reporting brightness changes at independently operating pixels with high temporal resolution. The approach assumes that for a periodic phenomenon, a highly similar set of events is generated within a spatio-temporal window at a time difference corresponding to its period. The sets of similar events are detected by a correlation in the spatio-temporal event stream space. The proposed method, EEPPR, is evaluated on a dataset of 12 sequences of periodic phenomena, i.e. flashing light and vibration, and periodic motion, e.g., rotation, ranging from 3.2 Hz to 2 kHz (equivalent to 192 - 120 000 RPM). EEPPR significantly outperforms published methods on this dataset, achieving the mean relative error of 0.1%. The dataset and codes are publicly available on GitHub.
- [634] arXiv:2408.07088 (replaced) [pdf, html, other]
-
Title: Learning Rule-Induced Subgraph Representations for Inductive Relation PredictionJournal-ref: Advances in Neural Information Processing Systems 36 (2024)Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Inductive relation prediction (IRP) -- where entities can be different during training and inference -- has shown great power for completing evolving knowledge graphs. Existing works mainly focus on using graph neural networks (GNNs) to learn the representation of the subgraph induced from the target link, which can be seen as an implicit rule-mining process to measure the plausibility of the target link. However, these methods cannot differentiate the target link and other links during message passing, hence the final subgraph representation will contain irrelevant rule information to the target link, which reduces the reasoning performance and severely hinders the applications for real-world scenarios. To tackle this problem, we propose a novel \textit{single-source edge-wise} GNN model to learn the \textbf{R}ule-induc\textbf{E}d \textbf{S}ubgraph represen\textbf{T}ations (\textbf{REST}), which encodes relevant rules and eliminates irrelevant rules within the subgraph. Specifically, we propose a \textit{single-source} initialization approach to initialize edge features only for the target link, which guarantees the relevance of mined rules and target link. Then we propose several RNN-based functions for \textit{edge-wise} message passing to model the sequential property of mined rules. REST is a simple and effective approach with theoretical support to learn the \textit{rule-induced subgraph representation}. Moreover, REST does not need node labeling, which significantly accelerates the subgraph preprocessing time by up to \textbf{11.66$\times$}. Experiments on inductive relation prediction benchmarks demonstrate the effectiveness of our REST. Our code is available at this https URL.
- [635] arXiv:2408.07176 (replaced) [pdf, html, other]
-
Title: Surrogate-Assisted Search with Competitive Knowledge Transfer for Expensive OptimizationComments: 22 pages, 14 figuresSubjects: Neural and Evolutionary Computing (cs.NE)
Expensive optimization problems (EOPs) have attracted increasing research attention over the decades due to their ubiquity in a variety of practical applications. Despite many sophisticated surrogate-assisted evolutionary algorithms (SAEAs) that have been developed for solving such problems, most of them lack the ability to transfer knowledge from previously-solved tasks and always start their search from scratch, making them troubled by the notorious cold-start issue. A few preliminary studies that integrate transfer learning into SAEAs still face some issues, such as defective similarity quantification that is prone to underestimate promising knowledge, surrogate-dependency that makes the transfer methods not coherent with the state-of-the-art in SAEAs, etc. In light of the above, a plug and play competitive knowledge transfer method is proposed to boost various SAEAs in this paper. Specifically, both the optimized solutions from the source tasks and the promising solutions acquired by the target surrogate are treated as task-solving knowledge, enabling them to compete with each other to elect the winner for expensive evaluation, thus boosting the search speed on the target task. Moreover, the lower bound of the convergence gain brought by the knowledge competition is mathematically analyzed, which is expected to strengthen the theoretical foundation of sequential transfer optimization. Experimental studies conducted on a series of benchmark problems and a practical application from the petroleum industry verify the efficacy of the proposed method. The source code of the competitive knowledge transfer is available at this https URL.
- [636] arXiv:2408.07205 (replaced) [pdf, html, other]
-
Title: Deep Index Policy for Multi-Resource Restless Matching Bandit and Its Application in Multi-Channel SchedulingSubjects: Machine Learning (cs.LG)
Scheduling in multi-channel wireless communication system presents formidable challenges in effectively allocating resources. To address these challenges, we investigate a multi-resource restless matching bandit (MR-RMB) model for heterogeneous resource systems with an objective of maximizing long-term discounted total rewards while respecting resource constraints. We have also generalized to applications beyond multi-channel wireless. We discuss the Max-Weight Index Matching algorithm, which optimizes resource allocation based on learned partial indexes. We have derived the policy gradient theorem for index learning. Our main contribution is the introduction of a new Deep Index Policy (DIP), an online learning algorithm tailored for MR-RMB. DIP learns the partial index by leveraging the policy gradient theorem for restless arms with convoluted and unknown transition kernels of heterogeneous resources. We demonstrate the utility of DIP by evaluating its performance for three different MR-RMB problems. Our simulation results show that DIP indeed learns the partial indexes efficiently.
- [637] arXiv:2408.07505 (replaced) [pdf, html, other]
-
Title: Large Language Models Know What Makes Exemplary ContextsComments: 12 pages, 3 figuresSubjects: Computation and Language (cs.CL)
In-context learning (ICL) has proven to be a significant capability with the advancement of Large Language models (LLMs). By instructing LLMs using few-shot demonstrative examples, ICL enables them to perform a wide range of tasks without needing to update millions of parameters. This paper presents a unified framework for LLMs that allows them to self-select influential in-context examples to compose their contexts; self-rank candidates with different demonstration compositions; self-optimize the demonstration selection and ordering through reinforcement learning. Specifically, our method designs a parameter-efficient retrieval head that generates the optimized demonstration after training with rewards from LLM's own preference. Experimental results validate the proposed method's effectiveness in enhancing ICL performance. Additionally, our approach effectively identifies and selects the most representative examples for the current task, and includes more diversity in retrieval.
- [638] arXiv:2408.08258 (replaced) [pdf, html, other]
-
Title: Snuffy: Efficient Whole Slide Image ClassifierHossein Jafarinia, Alireza Alipanah, Danial Hamdi, Saeed Razavi, Nahal Mirzaie, Mohammad Hossein RohbanComments: Accepted for ECCV 2024Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Image and Video Processing (eess.IV)
Whole Slide Image (WSI) classification with multiple instance learning (MIL) in digital pathology faces significant computational challenges. Current methods mostly rely on extensive self-supervised learning (SSL) for satisfactory performance, requiring long training periods and considerable computational resources. At the same time, no pre-training affects performance due to domain shifts from natural images to WSIs. We introduce Snuffy architecture, a novel MIL-pooling method based on sparse transformers that mitigates performance loss with limited pre-training and enables continual few-shot pre-training as a competitive option. Our sparsity pattern is tailored for pathology and is theoretically proven to be a universal approximator with the tightest probabilistic sharp bound on the number of layers for sparse transformers, to date. We demonstrate Snuffy's effectiveness on CAMELYON16 and TCGA Lung cancer datasets, achieving superior WSI and patch-level accuracies. The code is available on this https URL.
- [639] arXiv:2408.08493 (replaced) [pdf, html, other]
-
Title: Fishers Harvest Parallel Unlearning in Inherited Model NetworksSubjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Unlearning in various learning frameworks remains challenging, with the continuous growth and updates of models exhibiting complex inheritance relationships. This paper presents a novel unlearning framework, which enables fully parallel unlearning among models exhibiting inheritance. A key enabler is the new Unified Model Inheritance Graph (UMIG), which captures the inheritance using a Directed Acyclic Graph (DAG).Central to our framework is the new Fisher Inheritance Unlearning (FIUn) algorithm, which utilizes the Fisher Information Matrix (FIM) from initial unlearning models to pinpoint impacted parameters in inherited models. By employing FIM, the FIUn method breaks the sequential dependencies among the models, facilitating simultaneous unlearning and reducing computational overhead. We further design to merge disparate FIMs into a single matrix, synchronizing updates across inherited models. Experiments confirm the effectiveness of our unlearning framework. For single-class tasks, it achieves complete unlearning with 0\% accuracy for unlearned labels while maintaining 94.53\% accuracy for retained labels on average. For multi-class tasks, the accuracy is 1.07\% for unlearned labels and 84.77\% for retained labels on average. Our framework accelerates unlearning by 99\% compared to alternative methods.
- [640] arXiv:2408.08780 (replaced) [pdf, html, other]
-
Title: Large Language Models Might Not Care What You Are Saying: Prompt Format Beats DescriptionsComments: 10 pages, 6 figures, 3 tablesSubjects: Computation and Language (cs.CL)
With the help of in-context learning (ICL), large language models (LLMs) have achieved impressive performance across various tasks. However, the function of descriptive instructions during ICL remains under-explored. In this work, we propose an ensemble prompt framework to describe the selection criteria of multiple in-context examples, and preliminary experiments on machine translation (MT) across six translation directions confirm that this framework boosts ICL perfromance. But to our surprise, LLMs might not necessarily care what the descriptions actually say, and the performance gain is primarily caused by the ensemble format, since the framework could lead to improvement even with random descriptive nouns. We further apply this new ensemble prompt on a range of commonsense, math, logical reasoning and hallucination tasks with three LLMs and achieve promising results, suggesting again that designing a proper prompt format would be much more effective and efficient than paying effort into specific descriptions. Our code will be publicly available once this paper is published.
- [641] arXiv:2408.08802 (replaced) [pdf, html, other]
-
Title: PriorMapNet: Enhancing Online Vectorized HD Map Construction with PriorsSubjects: Computer Vision and Pattern Recognition (cs.CV)
Online vectorized High-Definition (HD) map construction is crucial for subsequent prediction and planning tasks in autonomous driving. Following MapTR paradigm, recent works have made noteworthy achievements. However, reference points are randomly initialized in mainstream methods, leading to unstable matching between predictions and ground truth. To address this issue, we introduce PriorMapNet to enhance online vectorized HD map construction with priors. We propose the PPS-Decoder, which provides reference points with position and structure priors. Fitted from the map elements in the dataset, prior reference points lower the learning difficulty and achieve stable matching. Furthermore, we propose the PF-Encoder to enhance the image-to-BEV transformation with BEV feature priors. Besides, we propose the DMD cross-attention, which decouples cross-attention along multi-scale and multi-sample respectively to achieve efficiency. Our proposed PriorMapNet achieves state-of-the-art performance in the online vectorized HD map construction task on nuScenes and Argoverse2 datasets. The code will be released publicly soon.
- [642] arXiv:2408.08808 (replaced) [pdf, html, other]
-
Title: Constructing Domain-Specific Evaluation Sets for LLM-as-a-judgeComments: 14 pages, 8 figures, Under reviewSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Large Language Models (LLMs) have revolutionized the landscape of machine learning, yet current benchmarks often fall short in capturing the diverse behavior of these models in real-world applications. A benchmark's usefulness is determined by its ability to clearly differentiate between models of varying capabilities (separability) and closely align with human preferences. Existing frameworks like Alpaca-Eval 2.0 LC \cite{dubois2024lengthcontrolledalpacaevalsimpleway} and Arena-Hard v0.1 \cite{li2024crowdsourced} are limited by their focus on general-purpose queries and lack of diversity across domains such as law, medicine, and multilingual contexts. In this paper, we address these limitations by introducing a novel data pipeline that curates diverse, domain-specific evaluation sets tailored for LLM-as-a-Judge frameworks. Our approach leverages a combination of manual curation, semi-supervised learning to generate clusters, and stratified sampling to ensure balanced representation across a wide range of domains and languages. The resulting evaluation set, which includes 1573 samples across 14 categories, demonstrates high separability (84\%) across ten top-ranked models, and agreement (84\%) with Chatbot Arena and (0.915) Spearman correlation. The agreement values are 9\% better than Arena Hard and 20\% better than AlpacaEval 2.0 LC, while the Spearman coefficient is 0.7 more than the next best benchmark, showcasing a significant improvement in the usefulness of the benchmark. We further provide an open-source evaluation tool that enables fine-grained analysis of model performance across user-defined categories, offering valuable insights for practitioners. This work contributes to the ongoing effort to enhance the transparency, diversity, and effectiveness of LLM evaluation methodologies.
- [643] arXiv:2408.08968 (replaced) [pdf, html, other]
-
Title: Online SLA Decomposition: Enabling Real-Time Adaptation to Evolving SystemsComments: The paper has been submitted to IEEE Networking LettersSubjects: Networking and Internet Architecture (cs.NI); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
When a network slice spans multiple domains, each domain must uphold the End-to-End (E2E) Service Level Agreement (SLA) associated with the slice. This requires decomposing the E2E SLA into partial SLAs for each domain. In a two-level network slicing management system with an E2E orchestrator and local controllers, we propose an online learning-decomposition framework that dynamically updates risk models using recent feedback. This approach utilizes online gradient descent and FIFO memory buffers to enhance stability and robustness. Our empirical study shows the proposed framework outperforms state-of-the-art static methods, offering more accurate and resilient SLA decomposition under varying conditions and sparse data.
- [644] arXiv:2408.09130 (replaced) [pdf, html, other]
-
Title: Gaussian in the Dark: Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian SplattingComments: accepted by PG 2024Subjects: Computer Vision and Pattern Recognition (cs.CV)
3D Gaussian Splatting has recently emerged as a powerful representation that can synthesize remarkable novel views using consistent multi-view images as input. However, we notice that images captured in dark environments where the scenes are not fully illuminated can exhibit considerable brightness variations and multi-view inconsistency, which poses great challenges to 3D Gaussian Splatting and severely degrades its performance. To tackle this problem, we propose Gaussian-DK. Observing that inconsistencies are mainly caused by camera imaging, we represent a consistent radiance field of the physical world using a set of anisotropic 3D Gaussians, and design a camera response module to compensate for multi-view inconsistencies. We also introduce a step-based gradient scaling strategy to constrain Gaussians near the camera, which turn out to be floaters, from splitting and cloning. Experiments on our proposed benchmark dataset demonstrate that Gaussian-DK produces high-quality renderings without ghosting and floater artifacts and significantly outperforms existing methods. Furthermore, we can also synthesize light-up images by controlling exposure levels that clearly show details in shadow areas.
- [645] arXiv:2408.09172 (replaced) [pdf, html, other]
-
Title: Unc-TTP: A Method for Classifying LLM Uncertainty to Improve In-Context Example SelectionComments: 9 pages, long paperSubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Nowadays, Large Language Models (LLMs) have demonstrated exceptional performance across various downstream tasks. However, it is challenging for users to discern whether the responses are generated with certainty or are fabricated to meet user expectations. Estimating the uncertainty of LLMs is particularly challenging due to their vast scale and the lack of white-box access. In this work, we propose a novel Uncertainty Tripartite Testing Paradigm (Unc-TTP) to classify LLM uncertainty, via evaluating the consistency of LLM outputs when incorporating label interference into the sampling-based approach. Based on Unc-TTP outputs, we aggregate instances into certain and uncertain categories. Further, we conduct a detailed analysis of the uncertainty properties of LLMs and show Unc-TTP's superiority over the existing sampling-based methods. In addition, we leverage the obtained uncertainty information to guide in-context example selection, demonstrating that Unc-TTP obviously outperforms retrieval-based and sampling-based approaches in selecting more informative examples. Our work paves a new way to classify the uncertainty of both open- and closed-source LLMs, and introduces a practical approach to exploit this uncertainty to improve LLMs performance.
- [646] arXiv:2408.09235 (replaced) [pdf, html, other]
-
Title: Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form TextSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
The emergence of Large Language Models (LLMs) as chat assistants capable of generating human-like conversations has amplified the need for robust evaluation methods, particularly for open-ended tasks. Conventional metrics like BLEU and ROUGE, while useful, are increasingly inadequate for capturing the subtle semantics and contextual richness of such generative outputs. We propose a reference-guided verdict method that automates the evaluation process by leveraging multiple LLMs-as-judges. Through experiments on three open-ended question-answering tasks, we demonstrate that combining multiple LLMs-as-judges significantly improves the reliability and accuracy of evaluations, particularly in complex tasks where a single model might struggle. Our findings reveal a strong correlation with human evaluations, establishing our method as a viable and effective alternative to traditional metrics and human judgments, particularly in the context of LLM-based chat assistants where the complexity and diversity of responses challenge existing benchmarks.
- [647] arXiv:2408.09236 (replaced) [pdf, other]
-
Title: Hybrid Semantic Search: Unveiling User Intent Beyond KeywordsSubjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)
This paper addresses the limitations of traditional keyword-based search in understanding user intent and introduces a novel hybrid search approach that leverages the strengths of non-semantic search engines, Large Language Models (LLMs), and embedding models. The proposed system integrates keyword matching, semantic vector embeddings, and LLM-generated structured queries to deliver highly relevant and contextually appropriate search results. By combining these complementary methods, the hybrid approach effectively captures both explicit and implicit user intent.The paper further explores techniques to optimize query execution for faster response times and demonstrates the effectiveness of this hybrid search model in producing comprehensive and accurate search outcomes.
- [648] arXiv:2408.09248 (replaced) [pdf, html, other]
-
Title: MagicID: Flexible ID Fidelity Generation SystemSubjects: Computer Vision and Pattern Recognition (cs.CV)
Portrait Fidelity Generation is a prominent research area in generative models, with a primary focus on enhancing both controllability and fidelity. Current methods face challenges in generating high-fidelity portrait results when faces occupy a small portion of the image with a low resolution, especially in multi-person group photo settings. To tackle these issues, we propose a systematic solution called MagicID, based on a self-constructed million-level multi-modal dataset named IDZoom. MagicID consists of Multi-Mode Fusion training strategy (MMF) and DDIM Inversion based ID Restoration inference framework (DIIR). During training, MMF iteratively uses the skeleton and landmark modalities from IDZoom as conditional guidance. By introducing the Clone Face Tuning in training stage and Mask Guided Multi-ID Cross Attention (MGMICA) in inference stage, explicit constraints on face positional features are achieved for multi-ID group photo generation. The DIIR aims to address the issue of artifacts. The DDIM Inversion is used in conjunction with face landmarks, global and local face features to achieve face restoration while keeping the background unchanged. Additionally, DIIR is plug-and-play and can be applied to any diffusion-based portrait generation method. To validate the effectiveness of MagicID, we conducted extensive comparative and ablation experiments. The experimental results demonstrate that MagicID has significant advantages in both subjective and objective metrics, and achieves controllable generation in multi-person scenarios.
- [649] arXiv:2408.09358 (replaced) [pdf, html, other]
-
Title: Panorama Tomosynthesis from Head CBCT with Simulated Projection GeometryComments: 12 pages, 6 figures, 1 table, Journal submission plannedSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cone Beam Computed Tomography (CBCT) and Panoramic X-rays are the most commonly used imaging modalities in dental health care. CBCT can produce three-dimensional views of a patient's head, providing clinicians with better diagnostic capability, whereas Panoramic X-ray can capture the entire maxillofacial region in a single image. If the CBCT is already available, it can be beneficial to synthesize a Panoramic X-ray, thereby avoiding an immediate additional scan and extra radiation exposure. Existing methods focus on delineating an approximate dental arch and creating orthogonal projections along this arch. However, no golden standard is available for such dental arch extractions, and this choice can affect the quality of synthesized X-rays. To avoid such issues, we propose a novel method for synthesizing Panoramic X-rays from diverse head CBCTs, employing a simulated projection geometry and dynamic rotation centers. Our method effectively synthesized panoramic views from CBCT, even for patients with missing or nonexistent teeth and in the presence of severe metal implants. Our results demonstrate that this method can generate high-quality panoramic images irrespective of the CBCT scanner geometry.
- [650] arXiv:2408.09380 (replaced) [pdf, other]
-
Title: ELASTIC: Efficient Linear Attention for Sequential Interest CompressionComments: We hereby withdraw this paper from arXiv due to incomplete experiments. Upon further review, we have determined that additional experimental work is necessary to fully validate our findings and conclusionsSubjects: Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
State-of-the-art sequential recommendation models heavily rely on transformer's attention mechanism. However, the quadratic computational and memory complexities of self attention have limited its scalability for modeling users' long range behaviour sequences. To address this problem, we propose ELASTIC, an Efficient Linear Attention for SequenTial Interest Compression, requiring only linear time complexity and decoupling model capacity from computational cost. Specifically, ELASTIC introduces a fixed length interest experts with linear dispatcher attention mechanism which compresses the long-term behaviour sequences to a significantly more compact representation which reduces up to 90% GPU memory usage with x2.7 inference speed up. The proposed linear dispatcher attention mechanism significantly reduces the quadratic complexity and makes the model feasible for adequately modeling extremely long sequences. Moreover, in order to retain the capacity for modeling various user interests, ELASTIC initializes a vast learnable interest memory bank and sparsely retrieves compressed user's interests from the memory with a negligible computational overhead. The proposed interest memory retrieval technique significantly expands the cardinality of available interest space while keeping the same computational cost, thereby striking a trade-off between recommendation accuracy and efficiency. To validate the effectiveness of our proposed ELASTIC, we conduct extensive experiments on various public datasets and compare it with several strong sequential recommenders. Experimental results demonstrate that ELASTIC consistently outperforms baselines by a significant margin and also highlight the computational efficiency of ELASTIC when modeling long sequences. We will make our implementation code publicly available.
- [651] arXiv:2408.09469 (replaced) [pdf, html, other]
-
Title: Enhancing Adversarial Transferability with Adversarial Weight TuningJiahao Chen, Zhou Feng, Rui Zeng, Yuwen Pu, Chunyi Zhou, Yi Jiang, Yuyou Gan, Jinbao Li, Shouling JiComments: 13 pagesSubjects: Cryptography and Security (cs.CR)
Deep neural networks (DNNs) are vulnerable to adversarial examples (AEs) that mislead the model while appearing benign to human observers. A critical concern is the transferability of AEs, which enables black-box attacks without direct access to the target model. However, many previous attacks have failed to explain the intrinsic mechanism of adversarial transferability. In this paper, we rethink the property of transferable AEs and reformalize the formulation of transferability. Building on insights from this mechanism, we analyze the generalization of AEs across models with different architectures and prove that we can find a local perturbation to mitigate the gap between surrogate and target models. We further establish the inner connections between model smoothness and flat local maxima, both of which contribute to the transferability of AEs. Further, we propose a new adversarial attack algorithm, \textbf{A}dversarial \textbf{W}eight \textbf{T}uning (AWT), which adaptively adjusts the parameters of the surrogate model using generated AEs to optimize the flat local maxima and model smoothness simultaneously, without the need for extra data. AWT is a data-free tuning method that combines gradient-based and model-based attack methods to enhance the transferability of AEs. Extensive experiments on a variety of models with different architectures on ImageNet demonstrate that AWT yields superior performance over other attacks, with an average increase of nearly 5\% and 10\% attack success rates on CNN-based and Transformer-based models, respectively, compared to state-of-the-art attacks.
- [652] arXiv:2408.09484 (replaced) [pdf, html, other]
-
Title: Fredholm Neural NetworksComments: 22 pages, 11 figures Replacement reason: corrected licence to arXiv.org perpetual, non-exclusive licenseSubjects: Numerical Analysis (math.NA); Dynamical Systems (math.DS)
Within the family of explainable machine-learning, we present Fredholm neural networks (Fredholm NNs), deep neural networks (DNNs) which replicate fixed point iterations for the solution of linear and nonlinear Fredholm Integral Equations (FIE) of the second kind. Applications of FIEs include the solution of ordinary, as well as partial differential equations (ODEs, PDEs) and many more. We first prove that Fredholm NNs provide accurate solutions. We then provide insight into the values of the hyperparameters and trainable/explainable weights and biases of the DNN, by directly connecting their values to the underlying mathematical theory. For our illustrations, we use Fredholm NNs to solve both linear and nonlinear problems, including elliptic PDEs and boundary value problems. We show that the proposed scheme achieves significant numerical approximation accuracy across both the domain and boundary. The proposed methodology provides insight into the connection between neural networks and classical numerical methods, and we posit that it can have applications in fields such as Uncertainty Quantification (UQ) and explainable artificial intelligence (XAI). Thus, we believe that it will trigger further advances in the intersection between scientific machine learning and numerical analysis.
- [653] arXiv:2408.09545 (replaced) [pdf, html, other]
-
Title: Seamless Integration: Sampling Strategies in Federated Learning SystemsComments: The 2nd IEEE International Conference on Federated Learning Technologies and Applications (FLTA24)Subjects: Machine Learning (cs.LG)
Federated Learning (FL) represents a paradigm shift in the field of machine learning, offering an approach for a decentralized training of models across a multitude of devices while maintaining the privacy of local data. However, the dynamic nature of FL systems, characterized by the ongoing incorporation of new clients with potentially diverse data distributions and computational capabilities, poses a significant challenge to the stability and efficiency of these distributed learning networks. The seamless integration of new clients is imperative to sustain and enhance the performance and robustness of FL systems. This paper looks into the complexities of integrating new clients into existing FL systems and explores how data heterogeneity and varying data distribution (not independent and identically distributed) among them can affect model training, system efficiency, scalability and stability. Despite these challenges, the integration of new clients into FL systems presents opportunities to enhance data diversity, improve learning performance, and leverage distributed computational power. In contrast to other fields of application such as the distributed optimization of word predictions on Gboard (where federated learning once originated), there are usually only a few clients in the production environment, which is why information from each new client becomes all the more valuable. This paper outlines strategies for effective client selection strategies and solutions for ensuring system scalability and stability. Using the example of images from optical quality inspection, it offers insights into practical approaches. In conclusion, this paper proposes that addressing the challenges presented by new client integration is crucial to the advancement and efficiency of distributed learning networks, thus paving the way for the adoption of Federated Learning in production environments.
- [654] arXiv:2408.09632 (replaced) [pdf, html, other]
-
Title: MoDeGPT: Modular Decomposition for Large Language Model CompressionChi-Heng Lin, Shangqian Gao, James Seale Smith, Abhishek Patel, Shikhar Tuli, Yilin Shen, Hongxia Jin, Yen-Chang HsuComments: 31 pages, 9 figuresSubjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Machine Learning (stat.ML)
Large Language Models (LLMs) have reshaped the landscape of artificial intelligence by demonstrating exceptional performance across various tasks. However, substantial computational requirements make their deployment challenging on devices with limited resources. Recently, compression methods using low-rank matrix techniques have shown promise, yet these often lead to degraded accuracy or introduce significant overhead in parameters and inference latency. This paper introduces \textbf{Mo}dular \textbf{De}composition (MoDeGPT), a novel structured compression framework that does not need recovery fine-tuning while resolving the above drawbacks. MoDeGPT partitions the Transformer block into modules comprised of matrix pairs and reduces the hidden dimensions via reconstructing the module-level outputs. MoDeGPT is developed based on a theoretical framework that utilizes three well-established matrix decomposition algorithms -- Nyström approximation, CR decomposition, and SVD -- and applies them to our redefined transformer modules. Our comprehensive experiments show MoDeGPT, without backward propagation, matches or surpasses previous structured compression methods that rely on gradient information, and saves 98% of compute costs on compressing a 13B model. On \textsc{Llama}-2/3 and OPT models, MoDeGPT maintains 90-95% zero-shot performance with 25-30% compression rates. Moreover, the compression can be done on a single GPU within a few hours and increases the inference throughput by up to 46%.
- [655] arXiv:2408.09656 (replaced) [pdf, html, other]
-
Title: A Comparison of Large Language Model and Human Performance on Random Number Generation TasksSubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Neurons and Cognition (q-bio.NC)
Random Number Generation Tasks (RNGTs) are used in psychology for examining how humans generate sequences devoid of predictable patterns. By adapting an existing human RNGT for an LLM-compatible environment, this preliminary study tests whether ChatGPT-3.5, a large language model (LLM) trained on human-generated text, exhibits human-like cognitive biases when generating random number sequences. Initial findings indicate that ChatGPT-3.5 more effectively avoids repetitive and sequential patterns compared to humans, with notably lower repeat frequencies and adjacent number frequencies. Continued research into different models, parameters, and prompting methodologies will deepen our understanding of how LLMs can more closely mimic human random generation behaviors, while also broadening their applications in cognitive and behavioral science research.
- [656] arXiv:2408.09663 (replaced) [pdf, html, other]
-
Title: CHASE: 3D-Consistent Human Avatars with Sparse Inputs via Gaussian Splatting and Contrastive LearningComments: 13 pages, 6 figuresSubjects: Computer Vision and Pattern Recognition (cs.CV)
Recent advancements in human avatar synthesis have utilized radiance fields to reconstruct photo-realistic animatable human avatars. However, both NeRFs-based and 3DGS-based methods struggle with maintaining 3D consistency and exhibit suboptimal detail reconstruction, especially with sparse inputs. To address this challenge, we propose CHASE, which introduces supervision from intrinsic 3D consistency across poses and 3D geometry contrastive learning, achieving performance comparable with sparse inputs to that with full inputs. Following previous work, we first integrate a skeleton-driven rigid deformation and a non-rigid cloth dynamics deformation to coordinate the movements of individual Gaussians during animation, reconstructing basic avatar with coarse 3D consistency. To improve 3D consistency under sparse inputs, we design Dynamic Avatar Adjustment(DAA) to adjust deformed Gaussians based on a selected similar pose/image from the dataset. Minimizing the difference between the image rendered by adjusted Gaussians and the image with the similar pose serves as an additional form of supervision for avatar. Furthermore, we propose a 3D geometry contrastive learning strategy to maintain the 3D global consistency of generated avatars. Though CHASE is designed for sparse inputs, it surprisingly outperforms current SOTA methods \textbf{in both full and sparse settings} on the ZJU-MoCap and H36M datasets, demonstrating that our CHASE successfully maintains avatar's 3D consistency, hence improving rendering quality.
- [657] arXiv:2408.09680 (replaced) [pdf, html, other]
-
Title: MambaLoc: Efficient Camera Localisation via State Space ModelSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Location information is pivotal for the automation and intelligence of terminal devices and edge-cloud IoT systems, such as autonomous vehicles and augmented reality. However, achieving reliable positioning across diverse IoT applications remains challenging due to significant training costs and the necessity of densely collected data. To tackle these issues, we have innovatively applied the selective state space (SSM) model to visual localization, introducing a new model named MambaLoc. The proposed model demonstrates exceptional training efficiency by capitalizing on the SSM model's strengths in efficient feature extraction, rapid computation, and memory optimization, and it further ensures robustness in sparse data environments due to its parameter sparsity. Additionally, we propose the Global Information Selector (GIS), which leverages selective SSM to implicitly achieve the efficient global feature extraction capabilities of Non-local Neural Networks. This design leverages the computational efficiency of the SSM model alongside the Non-local Neural Networks' capacity to capture long-range dependencies with minimal layers. Consequently, the GIS enables effective global information capture while significantly accelerating convergence. Our extensive experimental validation using public indoor and outdoor datasets first demonstrates our model's effectiveness, followed by evidence of its versatility with various existing localization models. Our code and models are publicly available to support further research and development in this area.
- [658] arXiv:2408.09697 (replaced) [pdf, html, other]
-
Title: Heta: Distributed Training of Heterogeneous Graph Neural NetworksSubjects: Distributed, Parallel, and Cluster Computing (cs.DC)
Heterogeneous Graph Neural Networks (HGNNs) leverage diverse semantic relationships in Heterogeneous Graphs (HetGs) and have demonstrated remarkable learning performance in various applications. However, current distributed GNN training systems often overlook unique characteristics of HetGs, such as varying feature dimensions and the prevalence of missing features among nodes, leading to suboptimal performance or even incompatibility with distributed HGNN training. We introduce Heta, a framework designed to address the communication bottleneck in distributed HGNN training. Heta leverages the inherent structure of HGNNs - independent relation-specific aggregations for each relation, followed by a cross-relation aggregation - and advocates for a novel Relation-Aggregation-First computation paradigm. It performs relation-specific aggregations within graph partitions and then exchanges partial aggregations. This design, coupled with a new graph partitioning method that divides a HetG based on its graph schema and HGNN computation dependency, substantially reduces communication overhead. Heta further incorporates an innovative GPU feature caching strategy that accounts for the different cache miss-penalties associated with diverse node types. Comprehensive evaluations of various HGNN models and large heterogeneous graph datasets demonstrate that Heta outperforms state-of-the-art systems like DGL and GraphLearn by up to 5.8x and 2.3x in end-to-end epoch time, respectively.
- [659] arXiv:2408.09698 (replaced) [pdf, html, other]
-
Title: Harnessing Multimodal Large Language Models for Multimodal Sequential RecommendationYuyang Ye, Zhi Zheng, Yishan Shen, Tianshu Wang, Hengruo Zhang, Peijun Zhu, Runlong Yu, Kai Zhang, Hui XiongSubjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)
Recent advances in Large Language Models (LLMs) have demonstrated significant potential in the field of Recommendation Systems (RSs). Most existing studies have focused on converting user behavior logs into textual prompts and leveraging techniques such as prompt tuning to enable LLMs for recommendation tasks. Meanwhile, research interest has recently grown in multimodal recommendation systems that integrate data from images, text, and other sources using modality fusion techniques. This introduces new challenges to the existing LLM-based recommendation paradigm which relies solely on text modality information. Moreover, although Multimodal Large Language Models (MLLMs) capable of processing multi-modal inputs have emerged, how to equip MLLMs with multi-modal recommendation capabilities remains largely unexplored. To this end, in this paper, we propose the Multimodal Large Language Model-enhanced Multimodaln Sequential Recommendation (MLLM-MSR) model. To capture the dynamic user preference, we design a two-stage user preference summarization method. Specifically, we first utilize an MLLM-based item-summarizer to extract image feature given an item and convert the image into text. Then, we employ a recurrent user preference summarization generation paradigm to capture the dynamic changes in user preferences based on an LLM-based user-summarizer. Finally, to enable the MLLM for multi-modal recommendation task, we propose to fine-tune a MLLM-based recommender using Supervised Fine-Tuning (SFT) techniques. Extensive evaluations across various datasets validate the effectiveness of MLLM-MSR, showcasing its superior ability to capture and adapt to the evolving dynamics of user preferences.
- [660] arXiv:2408.09713 (replaced) [pdf, other]
-
Title: Carbon Footprint Accounting Driven by Large Language Models and Retrieval-augmented GenerationSubjects: Information Retrieval (cs.IR)
Carbon footprint accounting is crucial for quantifying greenhouse gas emissions and achieving carbon neutrality.The dynamic nature of processes, accounting rules, carbon-related policies, and energy supply structures necessitates real-time updates of CFA. Traditional life cycle assessment methods rely heavily on human expertise, making near-real-time updates challenging. This paper introduces a novel approach integrating large language models (LLMs) with retrieval-augmented generation technology to enhance the real-time, professional, and economical aspects of carbon footprint information retrieval and analysis. By leveraging LLMs' logical and language understanding abilities and RAG's efficient retrieval capabilities, the proposed method LLMs-RAG-CFA can retrieve more relevant professional information to assist LLMs, enhancing the model's generative abilities. This method offers broad professional coverage, efficient real-time carbon footprint information acquisition and accounting, and cost-effective automation without frequent LLMs' parameter updates. Experimental results across five industries(primary aluminum, lithium battery, photovoltaic, new energy vehicles, and transformers)demonstrate that the LLMs-RAG-CFA method outperforms traditional methods and other LLMs, achieving higher information retrieval rates and significantly lower information deviations and carbon footprint accounting deviations. The economically viable design utilizes RAG technology to balance real-time updates with cost-effectiveness, providing an efficient, reliable, and cost-saving solution for real-time carbon emission management, thereby enhancing environmental sustainability practices.
- [661] arXiv:2408.09715 (replaced) [pdf, html, other]
-
Title: HYDEN: Hyperbolic Density Representations for Medical Images and ReportsSubjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV)
In light of the inherent entailment relations between images and text, hyperbolic point vector embeddings, leveraging the hierarchical modeling advantages of hyperbolic space, have been utilized for visual semantic representation learning. However, point vector embedding approaches fail to address the issue of semantic uncertainty, where an image may have multiple interpretations, and text may refer to different images, a phenomenon particularly prevalent in the medical domain. Therefor, we propose \textbf{HYDEN}, a novel hyperbolic density embedding based image-text representation learning approach tailored for specific medical domain data. This method integrates text-aware local features alongside global features from images, mapping image-text features to density features in hyperbolic space via using hyperbolic pseudo-Gaussian distributions. An encapsulation loss function is employed to model the partial order relations between image-text density distributions. Experimental results demonstrate the interpretability of our approach and its superior performance compared to the baseline methods across various zero-shot tasks and different datasets.
- [662] arXiv:2408.09716 (replaced) [pdf, other]
-
Title: RENAS: Prioritizing Co-Renaming Opportunities of IdentifiersComments: ICSME 2024. (C) 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksSubjects: Software Engineering (cs.SE)
Renaming identifiers in source code is a common refactoring task in software development. When renaming an identifier, other identifiers containing words with the same naming intention related to the renaming should be renamed simultaneously. However, identifying these related identifiers can be challenging. This study introduces a technique called RENAS, which identifies and recommends related identifiers that should be renamed simultaneously in Java applications. RENAS determines priority scores for renaming candidates based on the relationships and similarities among identifiers. Since identifiers that have a relationship and/or have similar vocabulary in the source code are often renamed together, their priority scores are determined based on these factors. Identifiers with higher priority are recommended to be renamed together. Through an evaluation involving real renaming instances extracted from change histories and validated manually, RENAS demonstrated an improvement in the F1-measure by more than 0.11 compared with existing renaming recommendation approaches.
- [663] arXiv:2408.09768 (replaced) [pdf, html, other]
-
Title: MalLight: Influence-Aware Coordinated Traffic Signal Control for Traffic Signal MalfunctionsComments: Paper accepted to CIKM24 Full Research trackSubjects: Artificial Intelligence (cs.AI)
Urban traffic is subject to disruptions that cause extended waiting time and safety issues at signalized intersections. While numerous studies have addressed the issue of intelligent traffic systems in the context of various disturbances, traffic signal malfunction, a common real-world occurrence with significant repercussions, has received comparatively limited attention. The primary objective of this research is to mitigate the adverse effects of traffic signal malfunction, such as traffic congestion and collision, by optimizing the control of neighboring functioning signals. To achieve this goal, this paper presents a novel traffic signal control framework (MalLight), which leverages an Influence-aware State Aggregation Module (ISAM) and an Influence-aware Reward Aggregation Module (IRAM) to achieve coordinated control of surrounding traffic signals. To the best of our knowledge, this study pioneers the application of a Reinforcement Learning(RL)-based approach to address the challenges posed by traffic signal malfunction. Empirical investigations conducted on real-world datasets substantiate the superior performance of our proposed methodology over conventional and deep learning-based alternatives in the presence of signal malfunction, with reduction of throughput alleviated by as much as 48.6$\%$.
- [664] arXiv:2408.09807 (replaced) [pdf, html, other]
-
Title: World Models Increase Autonomy in Reinforcement LearningSubjects: Artificial Intelligence (cs.AI)
Reinforcement learning (RL) is an appealing paradigm for training intelligent agents, enabling policy acquisition from the agent's own autonomously acquired experience. However, the training process of RL is far from automatic, requiring extensive human effort to reset the agent and environments. To tackle the challenging reset-free setting, we first demonstrate the superiority of model-based (MB) RL methods in such setting, showing that a straightforward adaptation of MBRL can outperform all the prior state-of-the-art methods while requiring less supervision. We then identify limitations inherent to this direct extension and propose a solution called model-based reset-free (MoReFree) agent, which further enhances the performance. MoReFree adapts two key mechanisms, exploration and policy learning, to handle reset-free tasks by prioritizing task-relevant states. It exhibits superior data-efficiency across various reset-free tasks without access to environmental reward or demonstrations while significantly outperforming privileged baselines that require supervision. Our findings suggest model-based methods hold significant promise for reducing human effort in RL. Website: this https URL
- [665] arXiv:2408.09908 (replaced) [pdf, html, other]
-
Title: $p$SVM: Soft-margin SVMs with $p$-norm Hinge LossSubjects: Machine Learning (cs.LG)
Support Vector Machines (SVMs) based on hinge loss have been extensively discussed and applied to various binary classification tasks. These SVMs achieve a balance between margin maximization and the minimization of slack due to outliers. Although many efforts have been dedicated to enhancing the performance of SVMs with hinge loss, studies on $p$SVMs, soft-margin SVMs with $p$-norm hinge loss, remain relatively scarce. In this paper, we explore the properties, performance, and training algorithms of $p$SVMs. We first derive the generalization bound of $p$SVMs, then formulate the dual optimization problem, comparing it with the traditional approach. Furthermore, we discuss a generalized version of the Sequential Minimal Optimization (SMO) algorithm, $p$SMO, to train our $p$SVM model. Comparative experiments on various datasets, including binary and multi-class classification tasks, demonstrate the effectiveness and advantages of our $p$SVM model and the $p$SMO method. Code is available at this https URL.
- [666] arXiv:2408.09939 (replaced) [pdf, html, other]
-
Title: "Image, Tell me your story!" Predicting the original meta-context of visual misinformationComments: Preprint. Code available at this https URLSubjects: Computation and Language (cs.CL)
To assist human fact-checkers, researchers have developed automated approaches for visual misinformation detection. These methods assign veracity scores by identifying inconsistencies between the image and its caption, or by detecting forgeries in the image. However, they neglect a crucial point of the human fact-checking process: identifying the original meta-context of the image. By explaining what is actually true about the image, fact-checkers can better detect misinformation, focus their efforts on check-worthy visual content, engage in counter-messaging before misinformation spreads widely, and make their explanation more convincing. Here, we fill this gap by introducing the task of automated image contextualization. We create 5Pils, a dataset of 1,676 fact-checked images with question-answer pairs about their original meta-context. Annotations are based on the 5 Pillars fact-checking framework. We implement a first baseline that grounds the image in its original meta-context using the content of the image and textual evidence retrieved from the open web. Our experiments show promising results while highlighting several open challenges in retrieval and reasoning. We make our code and data publicly available.
- [667] arXiv:2408.09955 (replaced) [pdf, html, other]
-
Title: MegaAgent: A Practical Framework for Autonomous Cooperation in Large-Scale LLM Agent SystemsSubjects: Multiagent Systems (cs.MA)
With the emergence of large language models (LLMs), LLM-powered multi-agent systems (LLM-MA systems) have been proposed to tackle real-world tasks. However, their agents mostly follow predefined Standard Operating Procedures (SOPs) that remain unchanged across the whole interaction, lacking autonomy and scalability. Additionally, current solutions often overlook the necessity for effective agent cooperation. To address the above limitations, we propose MegaAgent, a practical framework designed for autonomous cooperation in large-scale LLM Agent systems. MegaAgent leverages the autonomy of agents to dynamically generate agents based on task requirements, incorporating features such as automatically dividing tasks, systematic planning and monitoring of agent activities, and managing concurrent operations. In addition, MegaAgent is designed with a hierarchical structure and employs system-level parallelism to enhance performance and boost communication. We demonstrate the effectiveness of MegaAgent through Gobang game development, showing that it outperforms popular LLM-MA systems; and national policy simulation, demonstrating its high autonomy and potential to rapidly scale up to 590 agents while ensuring effective cooperation among them. Our results indicate that MegaAgent is the first autonomous large-scale LLM-MA system with no pre-defined SOPs, high effectiveness and scalability, paving the way for further research in this field. Our code is at https://anonymous.4open.science/r/MegaAgent-81F3.
- [668] arXiv:2408.09982 (replaced) [pdf, other]
-
Title: Application of Large Language Models in Automated Question Generation: A Case Study on ChatGLM's Structured Questions for National Teacher Certification ExamsSubjects: Computers and Society (cs.CY)
This study delves into the application potential of the large language models (LLMs) ChatGLM in the automatic generation of structured questions for National Teacher Certification Exams (NTCE). Through meticulously designed prompt engineering, we guided ChatGLM to generate a series of simulated questions and conducted a comprehensive comparison with questions recollected from past examinees. To ensure the objectivity and professionalism of the evaluation, we invited experts in the field of education to assess these questions and their scoring criteria. The research results indicate that the questions generated by ChatGLM exhibit a high level of rationality, scientificity, and practicality similar to those of the real exam questions across most evaluation criteria, demonstrating the model's accuracy and reliability in question generation. Nevertheless, the study also reveals limitations in the model's consideration of various rating criteria when generating questions, suggesting the need for further optimization and adjustment. This research not only validates the application potential of ChatGLM in the field of educational assessment but also provides crucial empirical support for the development of more efficient and intelligent educational automated generation systems in the future.
- [669] arXiv:2408.10014 (replaced) [pdf, html, other]
-
Title: Improved Distance (Sensitivity) Oracles with Subquadratic SpaceComments: An extended abstract of this work appeared at FOCS 2024Subjects: Data Structures and Algorithms (cs.DS)
A distance oracle (DO) with stretch $(\alpha, \beta)$ for a graph $G$ is a data structure that, when queried with vertices $s$ and $t$, returns a value $\widehat{d}(s,t)$ such that $d(s,t) \le \widehat{d}(s,t) \le \alpha \cdot d(s,t) + \beta$. An $f$-edge fault-tolerant distance sensitivity oracle ($f$-DSO) additionally receives a set $F$ of up to $f$ edges and estimates the $s$-$t$-distance in $G{-}F$. Our first contribution is a new distance oracle with subquadratic space for undirected graphs. Introducing a small additive stretch $\beta > 0$ allows us to make the multiplicative stretch $\alpha$ arbitrarily small. This sidesteps a known lower bound of $\alpha \ge 3$ (for $\beta = 0$ and subquadratic space) [Thorup & Zwick, JACM 2005]. We present a DO for graphs with edge weights in $[0,W]$ that, for any positive integer $t$ and any $c \in (0, \ell/2]$, has stretch $(1{+}\frac{1}{\ell}, 2W)$, space $\widetilde{O}(n^{2-\frac{c}{t}})$, and query time $O(n^c)$. These are the first subquadratic-space DOs with $(1+\epsilon, O(1))$-stretch generalizing Agarwal and Godfrey's results for sparse graphs [SODA 2013] to general undirected graphs. Our second contribution is a framework that turns a $(\alpha,\beta)$-stretch DO for unweighted graphs into an $(\alpha (1{+}\varepsilon),\beta)$-stretch $f$-DSO with sensitivity $f = o(\log(n)/\log\log n)$ and retains subquadratic space. This generalizes a result by Bilò, Chechik, Choudhary, Cohen, Friedrich, Krogmann, and Schirneck [STOC 2023, TheoretiCS 2024] for the special case of stretch $(3,0)$ and $f = O(1)$. By combining the framework with our new distance oracle, we obtain an $f$-DSO that, for any $\gamma \in (0, (\ell{+}1)/2]$, has stretch $((1{+}\frac{1}{\ell}) (1{+}\varepsilon), 2)$, space $n^{ 2- \frac{\gamma}{(\ell+1)(f+1)} + o(1)}/\varepsilon^{f+2}$, and query time $\widetilde{O}(n^{\gamma} /{\varepsilon}^2)$.
- [670] arXiv:2408.10111 (replaced) [pdf, html, other]
-
Title: PLUTUS: A Well Pre-trained Large Unified Transformer can Unveil Financial Time Series RegularitiesSubjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Financial time series modeling is crucial for understanding and predicting market behaviors but faces challenges such as non-linearity, non-stationarity, and high noise levels. Traditional models struggle to capture complex patterns due to these issues, compounded by limitations in computational resources and model capacity. Inspired by the success of large language models in NLP, we introduce $\textbf{PLUTUS}$, a $\textbf{P}$re-trained $\textbf{L}$arge $\textbf{U}$nified $\textbf{T}$ransformer-based model that $\textbf{U}$nveils regularities in financial time $\textbf{S}$eries. PLUTUS uses an invertible embedding module with contrastive learning and autoencoder techniques to create an approximate one-to-one mapping between raw data and patch embeddings. TimeFormer, an attention based architecture, forms the core of PLUTUS, effectively modeling high-noise time series. We incorporate a novel attention mechanisms to capture features across both variable and temporal dimensions. PLUTUS is pre-trained on an unprecedented dataset of 100 billion observations, designed to thrive in noisy financial environments. To our knowledge, PLUTUS is the first open-source, large-scale, pre-trained financial time series model with over one billion parameters. It achieves state-of-the-art performance in various tasks, demonstrating strong transferability and establishing a robust foundational model for finance. Our research provides technical guidance for pre-training financial time series data, setting a new standard in the field.
- [671] arXiv:2408.10154 (replaced) [pdf, html, other]
-
Title: LoopSplat: Loop Closure by Registering 3D Gaussian SplatsComments: Project page: this https URLSubjects: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
Simultaneous Localization and Mapping (SLAM) based on 3D Gaussian Splats (3DGS) has recently shown promise towards more accurate, dense 3D scene maps. However, existing 3DGS-based methods fail to address the global consistency of the scene via loop closure and/or global bundle adjustment. To this end, we propose LoopSplat, which takes RGB-D images as input and performs dense mapping with 3DGS submaps and frame-to-model tracking. LoopSplat triggers loop closure online and computes relative loop edge constraints between submaps directly via 3DGS registration, leading to improvements in efficiency and accuracy over traditional global-to-local point cloud registration. It uses a robust pose graph optimization formulation and rigidly aligns the submaps to achieve global consistency. Evaluation on the synthetic Replica and real-world TUM-RGBD, ScanNet, and ScanNet++ datasets demonstrates competitive or superior tracking, mapping, and rendering compared to existing methods for dense RGB-D SLAM. Code is available at this http URL.
- [672] arXiv:2408.10188 (replaced) [pdf, html, other]
-
Title: LongVILA: Scaling Long-Context Visual Language Models for Long VideosFuzhao Xue, Yukang Chen, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Ethan He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu, Song HanComments: Code and models are available at this https URLSubjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
Long-context capability is critical for multi-modal foundation models. We introduce LongVILA, a full-stack solution for long-context vision-language models, including system, model training, and dataset development. On the system side, we introduce the first long-context Multi-Modal Sequence Parallelism (MM-SP) system that enables long training and inference, enabling 2M context length training on 256 GPUs without any gradient checkpointing. MM-SP is 2.1x - 5.7x faster than ring sequence parallelism and 1.1x - 1.4x faster than Megatron context parallelism + tensor parallelism in text-only settings. Moreover, it seamlessly integrates with Hugging Face Transformers. For model training, we propose a five-stage pipeline comprising alignment, pre-training, short supervised fine-tuning, context extension, and long supervised fine-tuning. On datasets, we construct large-scale visual language pre-training datasets and long video instruction-following datasets to support our multi-stage training process. LongVILA extends the number of frames of VILA from 8 to 1024, and improves the long video captioning score from 2.00 to 3.26 (1.6x), achieving 99.5% accuracy in 1400-frames video (274k context length) needle-in-a-haystack. LongVILA-8B demonstrates consistent accuracy improvements on long videos in the VideoMME benchmark as the number of frames increases.
- [673] arXiv:2204.05796 (replaced) [pdf, html, other]
-
Title: A deep learning method for solving stochastic optimal control problems driven by fully-coupled FBSDEsSubjects: Optimization and Control (math.OC); Artificial Intelligence (cs.AI)
In this paper,we mainly focus on the numerical solution of high-dimensional stochastic optimal control problem driven by fully-coupled forward-backward stochastic differential equations (FBSDEs in short) through deep learning. We first transform the problem into a stochastic Stackelberg differential game problem (leader-follower problem), then a bi-level optimization method is developed where the leader's cost functional and the follower's cost functional are optimized alternatively via deep neural networks. As for the numerical results, we compute two examples of the investment-consumption problem solved through stochastic recursive utility models, and the results of both examples demonstrate the effectiveness of our proposed algorithm.
- [674] arXiv:2206.00794 (replaced) [pdf, html, other]
-
Title: Sequential Bayesian Neural Subnetwork EnsemblesSanket Jantre, Shrijita Bhattacharya, Nathan M. Urban, Byung-Jun Yoon, Tapabrata Maiti, Prasanna Balaprakash, Sandeep MadireddySubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
Deep ensembles have emerged as a powerful technique for improving predictive performance and enhancing model robustness across various applications by leveraging model diversity. However, traditional deep ensemble methods are often computationally expensive and rely on deterministic models, which may limit their flexibility. Additionally, while sparse subnetworks of dense models have shown promise in matching the performance of their dense counterparts and even enhancing robustness, existing methods for inducing sparsity typically incur training costs comparable to those of training a single dense model, as they either gradually prune the network during training or apply thresholding post-training. In light of these challenges, we propose an approach for sequential ensembling of dynamic Bayesian neural subnetworks that consistently maintains reduced model complexity throughout the training process while generating diverse ensembles in a single forward pass. Our approach involves an initial exploration phase to identify high-performing regions within the parameter space, followed by multiple exploitation phases that take advantage of the compactness of the sparse model. These exploitation phases quickly converge to different minima in the energy landscape, corresponding to high-performing subnetworks that together form a diverse and robust ensemble. We empirically demonstrate that our proposed approach outperforms traditional dense and sparse deterministic and Bayesian ensemble models in terms of prediction accuracy, uncertainty estimation, out-of-distribution detection, and adversarial robustness.
- [675] arXiv:2208.05949 (replaced) [pdf, html, other]
-
Title: Valid Inference After Causal DiscoverySubjects: Methodology (stat.ME); Machine Learning (cs.LG); Machine Learning (stat.ML)
Causal discovery and causal effect estimation are two fundamental tasks in causal inference. While many methods have been developed for each task individually, statistical challenges arise when applying these methods jointly: estimating causal effects after running causal discovery algorithms on the same data leads to "double dipping," invalidating the coverage guarantees of classical confidence intervals. To this end, we develop tools for valid post-causal-discovery inference. Across empirical studies, we show that a naive combination of causal discovery and subsequent inference algorithms leads to highly inflated miscoverage rates; on the other hand, applying our method provides reliable coverage while achieving more accurate causal discovery than data splitting.
- [676] arXiv:2301.00434 (replaced) [pdf, html, other]
-
Title: Cops and robbers pebbling in graphsSubjects: Combinatorics (math.CO); Discrete Mathematics (cs.DM)
Here we merge the two fields of Cops and Robbers and Graph Pebbling to introduce the new topic of Cops and Robbers Pebbling. Both paradigms can be described by moving tokens (the cops) along the edges of a graph to capture a special token (the robber). In Cops and Robbers, all tokens move freely, whereas, in Graph Pebbling, some of the chasing tokens disappear with movement while the robber is stationary. In Cops and Robbers Pebbling, some of the chasing tokens (cops) disappear with movement, while the robber moves freely. We define the cop pebbling number of a graph to be the minimum number of cops necessary to capture the robber in this context, and present upper and lower bounds and exact values, some involving various domination parameters, for an array of graph classes, including paths, cycles, trees, chordal graphs, high girth graphs, and cop-win graphs, as well as graph products. Furthermore we show that the analogous inequality for Graham's Pebbling Conjecture fails for cop pebbling and posit a conjecture along the lines of Meyniel's Cops and Robbers Conjecture that may hold for cop pebbling. We also offer several new problems.
- [677] arXiv:2303.14353 (replaced) [pdf, html, other]
-
Title: DiracDiffusion: Denoising and Incremental Reconstruction with Assured Data-ConsistencyComments: 30 pages, 15 figures, published at the 41st International Conference on Machine Learning, Vienna, Austria, 2024Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Diffusion models have established new state of the art in a multitude of computer vision tasks, including image restoration. Diffusion-based inverse problem solvers generate reconstructions of exceptional visual quality from heavily corrupted measurements. However, in what is widely known as the perception-distortion trade-off, the price of perceptually appealing reconstructions is often paid in declined distortion metrics, such as PSNR. Distortion metrics measure faithfulness to the observation, a crucial requirement in inverse problems. In this work, we propose a novel framework for inverse problem solving, namely we assume that the observation comes from a stochastic degradation process that gradually degrades and noises the original clean image. We learn to reverse the degradation process in order to recover the clean image. Our technique maintains consistency with the original measurement throughout the reverse process, and allows for great flexibility in trading off perceptual quality for improved distortion metrics and sampling speedup via early-stopping. We demonstrate the efficiency of our method on different high-resolution datasets and inverse problems, achieving great improvements over other state-of-the-art diffusion-based methods with respect to both perceptual and distortion metrics.
- [678] arXiv:2304.10659 (replaced) [pdf, html, other]
-
Title: Isolation of regular graphs and $k$-chromatic graphsComments: 12 pages, minor corrections made. arXiv admin note: text overlap with arXiv:2303.13709Journal-ref: Mediterranean Journal of Mathematics 21 (2024), article number 148Subjects: Combinatorics (math.CO); Discrete Mathematics (cs.DM)
Given a set $\mathcal{F}$ of graphs, we call a copy of a graph in $\mathcal{F}$ an $\mathcal{F}$-graph. The $\mathcal{F}$-isolation number of a graph $G$, denoted by $\iota(G,\mathcal{F})$, is the size of a smallest set $D$ of vertices of $G$ such that the closed neighbourhood of $D$ intersects the vertex sets of the $\mathcal{F}$-graphs contained by $G$ (equivalently, $G - N[D]$ contains no $\mathcal{F}$-graph). Thus, $\iota(G,\{K_1\})$ is the domination number of $G$. For any integer $k \geq 1$, let $\mathcal{F}_{1,k}$ be the set of regular graphs of degree at least $k-1$, let $\mathcal{F}_{2,k}$ be the set of graphs whose chromatic number is at least $k$, and let $\mathcal{F}_{3,k}$ be the union of $\mathcal{F}_{1,k}$ and $\mathcal{F}_{2,k}$. Thus, $k$-cliques are members of both $\mathcal{F}_{1,k}$ and $\mathcal{F}_{2,k}$. We prove that for each $i \in \{1, 2, 3\}$, $\frac{m+1}{{k \choose 2} + 2}$ is a best possible upper bound on $\iota(G, \mathcal{F}_{i,k})$ for connected $m$-edge graphs $G$ that are not $k$-cliques. The bound is attained by infinitely many (non-isomorphic) graphs. The proof of the bound depends on determining the graphs attaining the bound. This appears to be a new feature in the literature on isolation. Among the result's consequences are a sharp bound of Fenech, Kaemawichanurat and the present author on the $k$-clique isolation number and a sharp bound on the cycle isolation number.
- [679] arXiv:2305.04844 (replaced) [pdf, html, other]
-
Title: SR+Codec: a Benchmark of Super-Resolution for Video Compression Bitrate ReductionSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
In recent years, there has been significant interest in Super-Resolution (SR), which focuses on generating a high-resolution image from a low-resolution input. Deep learning-based methods for super-resolution have been particularly popular and have shown impressive results on various benchmarks. However, research indicates that these methods may not perform as well on strongly compressed videos. We developed a super-resolution benchmark to analyze SR's capacity to upscale compressed videos. Our dataset employed video codecs based on five widely-used compression standards: H.264, H.265, H.266, AV1, and AVS3. We assessed 19 popular SR models using our benchmark and evaluated their ability to restore details and their susceptibility to compression artifacts. To get an accurate perceptual ranking of SR models, we conducted a crowd-sourced side-by-side comparison of their outputs. We found that some SR models, combined with compression, allow us to reduce the video bitrate without significant loss of quality. We also compared a range of image and video quality metrics with subjective scores to evaluate their accuracy on super-resolved compressed videos. The benchmark is publicly available at this https URL
- [680] arXiv:2307.07439 (replaced) [pdf, html, other]
-
Title: Atlas-Based Interpretable Age Prediction In Whole-Body MR ImagesSophie Starck, Yadunandan Vivekanand Kini, Jessica Johanna Maria Ritter, Rickmer Braren, Daniel Rueckert, Tamara MuellerSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Age prediction is an important part of medical assessments and research. It can aid in detecting diseases as well as abnormal ageing by highlighting potential discrepancies between chronological and biological age. To improve understanding of age-related changes in various body parts, we investigate the ageing of the human body on a large scale by using whole-body 3D images. We utilise the Grad-CAM method to determine the body areas most predictive of a person's age. In order to expand our analysis beyond individual subjects, we employ registration techniques to generate population-wide importance maps that show the most predictive areas in the body for a whole cohort of subjects. We show that the investigation of the full 3D volume of the whole body and the population-wide analysis can give important insights into which body parts play the most important roles in predicting a person's age. Our findings reveal three primary areas of interest: the spine, the autochthonous back muscles, and the cardiac region, which exhibits the highest importance. Finally, we investigate differences between subjects that show accelerated and decelerated ageing.
- [681] arXiv:2307.11609 (replaced) [pdf, html, other]
-
Title: Persistent Ballistic Entanglement Spreading with Optimal Control in Quantum Spin ChainsComments: Main text: 6 pages, 5 figures + Supplemental materialJournal-ref: Phys. Rev. Lett. 133, 070402 (2024)Subjects: Quantum Physics (quant-ph); Strongly Correlated Electrons (cond-mat.str-el); Machine Learning (cs.LG)
Entanglement propagation provides a key routine to understand quantum many-body dynamics in and out of equilibrium. The entanglement entropy (EE) usually approaches to a sub-saturation known as the Page value $\tilde{S}_{P} =\tilde{S} - dS$ (with $\tilde{S}$ the maximum of EE and $dS$ the Page correction) in, e.g., the random unitary evolutions. The ballistic spreading of EE usually appears in the early time and will be deviated far before the Page value is reached. In this work, we uncover that the magnetic field that maximizes the EE robustly induces persistent ballistic spreading of entanglement in quantum spin chains. The linear growth of EE is demonstrated to persist till the maximal $\tilde{S}$ (along with a flat entanglement spectrum) is reached. The robustness of ballistic spreading and the enhancement of EE under such an optimal control are demonstrated, considering particularly perturbing the initial state by random pure states (RPS's). These are argued as the results from the endomorphism of the time evolution under such an entanglement-enhancing optimal control for the RPS's.
- [682] arXiv:2309.00223 (replaced) [pdf, html, other]
-
Title: The FruitShell French synthesis system at the Blizzard 2023 ChallengeSubjects: Audio and Speech Processing (eess.AS); Computation and Language (cs.CL); Sound (cs.SD)
This paper presents a French text-to-speech synthesis system for the Blizzard Challenge 2023. The challenge consists of two tasks: generating high-quality speech from female speakers and generating speech that closely resembles specific individuals. Regarding the competition data, we conducted a screening process to remove missing or erroneous text data. We organized all symbols except for phonemes and eliminated symbols that had no pronunciation or zero duration. Additionally, we added word boundary and start/end symbols to the text, which we have found to improve speech quality based on our previous experience. For the Spoke task, we performed data augmentation according to the competition rules. We used an open-source G2P model to transcribe the French texts into phonemes. As the G2P model uses the International Phonetic Alphabet (IPA), we applied the same transcription process to the provided competition data for standardization. However, due to compiler limitations in recognizing special symbols from the IPA chart, we followed the rules to convert all phonemes into the phonetic scheme used in the competition data. Finally, we resampled all competition audio to a uniform sampling rate of 16 kHz. We employed a VITS-based acoustic model with the hifigan vocoder. For the Spoke task, we trained a multi-speaker model and incorporated speaker information into the duration predictor, vocoder, and flow layers of the model. The evaluation results of our system showed a quality MOS score of 3.6 for the Hub task and 3.4 for the Spoke task, placing our system at an average level among all participating teams.
- [683] arXiv:2309.06642 (replaced) [pdf, html, other]
-
Title: Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion ModelsComments: 30 pages, 21 figures, published at the 41st International Conference on Machine Learning, Vienna, Austria, 2024Subjects: Image and Video Processing (eess.IV); Machine Learning (cs.LG)
Inverse problems arise in a multitude of applications, where the goal is to recover a clean signal from noisy and possibly (non)linear observations. The difficulty of a reconstruction problem depends on multiple factors, such as the ground truth signal structure, the severity of the degradation and the complex interactions between the above. This results in natural sample-by-sample variation in the difficulty of a reconstruction problem. Our key observation is that most existing inverse problem solvers lack the ability to adapt their compute power to the difficulty of the reconstruction task, resulting in subpar performance and wasteful resource allocation. We propose a novel method, $\textit{severity encoding}$, to estimate the degradation severity of corrupted signals in the latent space of an autoencoder. We show that the estimated severity has strong correlation with the true corruption level and can provide useful hints on the difficulty of reconstruction problems on a sample-by-sample basis. Furthermore, we propose a reconstruction method based on latent diffusion models that leverages the predicted degradation severities to fine-tune the reverse diffusion sampling trajectory and thus achieve sample-adaptive inference times. Our framework, Flash-Diffusion, acts as a wrapper that can be combined with any latent diffusion-based baseline solver, imbuing it with sample-adaptivity and acceleration. We perform experiments on both linear and nonlinear inverse problems and demonstrate that our technique greatly improves the performance of the baseline solver and achieves up to $10\times$ acceleration in mean sampling speed.
- [684] arXiv:2309.14353 (replaced) [pdf, html, other]
-
Title: Limited Communications Distributed Optimization via Deep Unfolded Distributed ADMMSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Signal Processing (eess.SP)
Distributed optimization is a fundamental framework for collaborative inference and decision making in decentralized multi-agent systems. The operation is modeled as the joint minimization of a shared objective which typically depends on observations gathered locally by each agent. Distributed optimization algorithms, such as the common D-ADMM, tackle this task by iteratively combining local computations and message exchanges. One of the main challenges associated with distributed optimization, and particularly with D-ADMM, is that it requires a large number of communications, i.e., messages exchanged between the agents, to reach consensus. This can make D-ADMM costly in power, latency, and channel resources. In this work we propose unfolded D-ADMM, which follows the emerging deep unfolding methodology to enable D-ADMM to operate reliably with a predefined and small number of messages exchanged by each agent. Unfolded D-ADMM fully preserves the operation of D-ADMM, while leveraging data to tune the hyperparameters of each iteration of the algorithm. These hyperparameters can either be agent-specific, aiming at achieving the best performance within a fixed number of iterations over a given network, or shared among the agents, allowing to learn to distributedly optimize over different networks. For both settings, our unfolded D-ADMM operates with limited communications, while preserving the interpretability and flexibility of the original D-ADMM algorithm. We specialize unfolded D-ADMM for two representative settings: a distributed estimation task, considering a sparse recovery setup, and a distributed learning scenario, where multiple agents collaborate in learning a machine learning model. Our numerical results demonstrate that the proposed approach dramatically reduces the number of communications utilized by D-ADMM, without compromising on its performance.
- [685] arXiv:2310.09128 (replaced) [pdf, other]
-
Title: Isolation of squares in graphsComments: 16 pages, 1 figure, minor corrections madeJournal-ref: Discrete Mathematics 347 (2024), article number 114161Subjects: Combinatorics (math.CO); Discrete Mathematics (cs.DM)
Given a set $\mathcal{F}$ of graphs, we call a copy of a graph in $\mathcal{F}$ an $\mathcal{F}$-graph. The $\mathcal{F}$-isolation number of a graph $G$, denoted by $\iota(G,\mathcal{F})$, is the size of a smallest subset $D$ of the vertex set $V(G)$ such that the closed neighbourhood of $D$ intersects the vertex sets of the $\mathcal{F}$-graphs contained by $G$ (equivalently, $G - N[D]$ contains no $\mathcal{F}$-graph). Thus, $\iota(G,\{K_1\})$ is the domination number of $G$. The second author showed that if $\mathcal{F}$ is the set of cycles and $G$ is a connected $n$-vertex graph that is not a triangle, then $\iota(G,\mathcal{F}) \leq \left \lfloor \frac{n}{4} \right \rfloor$. This bound is attainable for every $n$ and solved a problem of Caro and Hansberg. A question that arises immediately is how much smaller an upper bound can be if $\mathcal{F} = \{C_k\}$ for some $k \geq 3$, where $C_k$ is a cycle of length $k$. The problem is to determine the smallest real number $c_k$ (if it exists) such that for some finite set $\mathcal{E}_k$ of graphs, $\iota(G, \{C_k\}) \leq c_k |V(G)|$ for every connected graph $G$ that is not an $\mathcal{E}_k$-graph. The above-mentioned result yields $c_3 = \frac{1}{4}$ and $\mathcal{E}_3 = \{C_3\}$. The second author also showed that if $k \geq 5$ and $c_k$ exists, then $c_k \geq \frac{2}{2k + 1}$. We prove that $c_4 = \frac{1}{5}$ and determine $\mathcal{E}_4$, which consists of three $4$-vertex graphs and six $9$-vertex graphs. The $9$-vertex graphs in $\mathcal{E}_4$ were fully determined by means of a computer program. A method that has the potential of yielding similar results is introduced.
- [686] arXiv:2312.07784 (replaced) [pdf, html, other]
-
Title: Robust MRI Reconstruction by Smoothed Unrolling (SMUG)Subjects: Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Signal Processing (eess.SP)
As the popularity of deep learning (DL) in the field of magnetic resonance imaging (MRI) continues to rise, recent research has indicated that DL-based MRI reconstruction models might be excessively sensitive to minor input disturbances, including worst-case additive perturbations. This sensitivity often leads to unstable, aliased images. This raises the question of how to devise DL techniques for MRI reconstruction that can be robust to train-test variations. To address this problem, we propose a novel image reconstruction framework, termed Smoothed Unrolling (SMUG), which advances a deep unrolling-based MRI reconstruction model using a randomized smoothing (RS)-based robust learning approach. RS, which improves the tolerance of a model against input noises, has been widely used in the design of adversarial defense approaches for image classification tasks. Yet, we find that the conventional design that applies RS to the entire DL-based MRI model is ineffective. In this paper, we show that SMUG and its variants address the above issue by customizing the RS process based on the unrolling architecture of a DL-based MRI reconstruction model. Compared to the vanilla RS approach, we show that SMUG improves the robustness of MRI reconstruction with respect to a diverse set of instability sources, including worst-case and random noise perturbations to input measurements, varying measurement sampling rates, and different numbers of unrolling steps. Furthermore, we theoretically analyze the robustness of our method in the presence of perturbations.
- [687] arXiv:2401.05648 (replaced) [pdf, html, other]
-
Title: On the on-line coloring of unit interval graphs with proper interval representationComments: Alternative title: On the on-line chain partitioning of semi-orders with proper interval representationSubjects: Combinatorics (math.CO); Data Structures and Algorithms (cs.DS)
We define the problem as a two-player game between Algorithm and Builder. The game is played in rounds. Each round, Builder presents an interval that is neither contained in nor contains any previously presented interval. Algorithm immediately and irrevocably assigns the interval a color that has not been assigned to any interval intersecting it. The set of intervals form an interval representation for a unit interval graph and the colors form a proper coloring of that graph. For every positive integer $\omega$, we define the value $R(\omega)$ as the maximum number of colors for which Builder has a strategy that forces Algorithm to use $R(\omega)$ colors with the restriction that the unit interval graph constructed cannot contain a clique of size $\omega$. In 1981, Chrobak and Ślusarek showed that $R(\omega)\leq2\omega -1$. In 2005, Epstein and Levy showed that $R(\omega)\geq\lfloor{3\omega/2\rfloor}$. This problem remained unsolved for $\omega\geq 3$. In 2022, Biró and Curbelo showed that $R(3)=5$. In this paper, we show that $R(4)=7$
- [688] arXiv:2401.16819 (replaced) [pdf, html, other]
-
Title: Localizing uniformly moving single-frequency sources using an inverse 2.5D approachComments: 35 pages, 17 figuresJournal-ref: J. Sound Vib., 593 (2024), 118653Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)
Localizing linearly moving sound sources using microphone arrays is challenging as the transient nature of the signal leads to relatively short observation periods. Commonly, a moving focus is used and most methods operate at least partially in the time domain. In contrast, this manuscript presents an inverse source localization algorithm for uniformly moving single-frequency sources that acts entirely in the frequency domain. For this, a 2.5D approach is utilized and a transfer function between sources and a microphone grid is derived. By solving a least squares problem using the data at the microphone grid, the unknown source distribution in the moving frame can be determined. First, the time signals need to be transformed from time into frequency domain using a windowed discrete Fourier transform (DFT), which leads to spectral leakage that depends on the length of the time interval and the analysis window used. To include spectral leakage in the numerical model, the calculation of the transfer matrix is modified using the Fourier transform of the analysis window in the DFT applied to the measurements. Currently, this approach is limited to single-frequency sources as this restriction allows for simplified calculations and reduces the computational effort. The least squares problem is solved using a Tikhonov regularization and an L-curve approach. As moving sources are considered, utilizing the Doppler effect enhances the stability of the system by combining the transfer functions for multiple frequencies in the measured signals. The performance is validated using simulated data of a moving point source with or without a reflecting ground. Numerical experiments are performed to show the effect of the choice of frequencies in the receiver spectrum, the effect of the DFT, the source frequency, the distance between source and receiver, and the robustness with respect to noise.
- [689] arXiv:2402.09076 (replaced) [pdf, html, other]
-
Title: Preserving system activity while controlling epidemic spreading in adaptive temporal networksComments: 16 pages, 8 figuresJournal-ref: Phys. Rev. Research 6, 033159 (2024)Subjects: Physics and Society (physics.soc-ph); Social and Information Networks (cs.SI); Populations and Evolution (q-bio.PE)
Human behaviour strongly influences the spread of infectious diseases: understanding the interplay between epidemic dynamics and adaptive behaviours is essential to improve response strategies to epidemics, with the goal of containing the epidemic while preserving a sufficient level of operativeness in the population. Through activity-driven temporal networks, we formulate a general framework which models a wide range of adaptive behaviours and mitigation strategies, observed in real populations. We analytically derive the conditions for a widespread diffusion of epidemics in the presence of arbitrary adaptive behaviours, highlighting the crucial role of correlations between agents behaviour in the infected and in the susceptible state. We focus on the effects of sick-leave, comparing the effectiveness of different strategies in reducing the impact of the epidemic and preserving the system operativeness. We show the critical relevance of heterogeneity in individual behavior: in homogeneous networks, all sick-leave strategies are equivalent and poorly effective, while in heterogeneous networks, strategies targeting the most vulnerable nodes are able to effectively mitigate the epidemic, also avoiding a deterioration in system activity and maintaining a low level of absenteeism. Interestingly, with targeted strategies both the minimum of population activity and the maximum of absenteeism anticipate the infection peak, which is effectively flattened and delayed, so that full operativeness is almost restored when the infection peak arrives. We also provide realistic estimates of the model parameters for influenza-like illness, thereby suggesting strategies for managing epidemics and absenteeism in realistic populations.
- [690] arXiv:2402.18697 (replaced) [pdf, html, other]
-
Title: Inferring Dynamic Networks from Marginals with Iterative Proportional FittingComments: Conference version available from https://proceedings.mlr.press/v235/chang24b.htmlJournal-ref: Proceedings of the 41st International Conference on Machine Learning, PMLR 235:6202-6252, 2024Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Social and Information Networks (cs.SI); Optimization and Control (math.OC); Statistics Theory (math.ST)
A common network inference problem, arising from real-world data constraints, is how to infer a dynamic network from its time-aggregated adjacency matrix and time-varying marginals (i.e., row and column sums). Prior approaches to this problem have repurposed the classic iterative proportional fitting (IPF) procedure, also known as Sinkhorn's algorithm, with promising empirical results. However, the statistical foundation for using IPF has not been well understood: under what settings does IPF provide principled estimation of a dynamic network from its marginals, and how well does it estimate the network? In this work, we establish such a setting, by identifying a generative network model whose maximum likelihood estimates are recovered by IPF. Our model both reveals implicit assumptions on the use of IPF in such settings and enables new analyses, such as structure-dependent error bounds on IPF's parameter estimates. When IPF fails to converge on sparse network data, we introduce a principled algorithm that guarantees IPF converges under minimal changes to the network structure. Finally, we conduct experiments with synthetic and real-world data, which demonstrate the practical value of our theoretical and algorithmic contributions.
- [691] arXiv:2403.00447 (replaced) [pdf, html, other]
-
Title: Continuous Approximations of Projected Dynamical Systems via Control Barrier FunctionsComments: Accepted to IEEE Transactions on Automatic Control (IEEE TAC). Compared to the accepted version, this version contains an additional numerical example on feedback optimizationSubjects: Optimization and Control (math.OC); Systems and Control (eess.SY)
Projected Dynamical Systems (PDSs) form a class of discontinuous constrained dynamical systems, and have been used widely to solve optimization problems and variational inequalities. Recently, they have also gained significant attention for control purposes, such as high-performance integrators, saturated control and feedback optimization. In this work, we establish that locally Lipschitz continuous dynamics, involving Control Barrier Functions (CBFs), namely CBF-based dynamics, approximate PDSs. Specifically, we prove that trajectories of CBF-based dynamics uniformly converge to trajectories of PDSs, as a CBF-parameter approaches infinity. Towards this, we also prove that CBF-based dynamics are perturbations of PDSs, with quantitative bounds on the perturbation. Our results pave the way to implement discontinuous PDS-based controllers in a continuous fashion, employing CBFs. We demonstrate this on numerical examples on feedback optimization and synchronverter control. Moreover, our results can be employed to numerically simulate PDSs, overcoming disadvantages of existing discretization schemes, such as computing projections to possibly non-convex sets. Finally, this bridge between CBFs and PDSs may yield other potential benefits, including novel insights on stability.
- [692] arXiv:2403.10136 (replaced) [pdf, other]
-
Title: Response Style Characterization for Repeated Measures Using the Visual Analogue ScaleComments: Accepted to IEEE Access. Accessible at this https URLSubjects: Methodology (stat.ME); Artificial Intelligence (cs.AI)
Self-report measures (e.g., Likert scales) are widely used to evaluate subjective health perceptions. Recently, the visual analog scale (VAS), a slider-based scale, has become popular owing to its ability to precisely and easily assess how people feel. These data can be influenced by the response style (RS), a user-dependent systematic tendency that occurs regardless of questionnaire instructions. Despite its importance, especially in between-individual analysis, little attention has been paid to handling the RS in the VAS (denoted as response profile (RP)), as it is mainly used for within-individual monitoring and is less affected by RP. However, VAS measurements often require repeated self-reports of the same questionnaire items, making it difficult to apply conventional methods on a Likert scale. In this study, we developed a novel RP characterization method for various types of repeatedly measured VAS data. This approach involves the modeling of RP as distributional parameters ${\theta}$ through a mixture of RS-like distributions, and addressing the issue of unbalanced data through bootstrap sampling for treating repeated measures. We assessed the effectiveness of the proposed method using simulated pseudo-data and an actual dataset from an empirical study. The assessment of parameter recovery showed that our method accurately estimated the RP parameter ${\theta}$, demonstrating its robustness. Moreover, applying our method to an actual VAS dataset revealed the presence of individual RP heterogeneity, even in repeated VAS measurements, similar to the findings of the Likert scale. Our proposed method enables RP heterogeneity-aware VAS data analysis, similar to Likert-scale data analysis.
- [693] arXiv:2403.18198 (replaced) [pdf, html, other]
-
Title: Generative Medical SegmentationSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Rapid advancements in medical image segmentation performance have been significantly driven by the development of Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). These models follow the discriminative pixel-wise classification learning paradigm and often have limited ability to generalize across diverse medical imaging datasets. In this manuscript, we introduce Generative Medical Segmentation (GMS), a novel approach leveraging a generative model to perform image segmentation. Concretely, GMS employs a robust pre-trained vision foundation model to extract latent representations for images and corresponding ground truth masks, followed by a model that learns a mapping function from the image to the mask in the latent space. Once trained, the model generates an estimated segmentation mask using the pre-trained vision foundation model to decode the predicted latent representation back into the image space. The design of GMS leads to fewer trainable parameters in the model which reduces the risk of overfitting and enhances its generalization capability. Our experimental analysis across five public datasets in different medical imaging domains demonstrates GMS outperforms existing discriminative and generative segmentation models. Furthermore, GMS is able to generalize well across datasets from different centers within the same imaging modality. Our experiments suggest GMS offers a scalable and effective solution for medical image segmentation. GMS implementation and trained model weights are available at this https URL.
- [694] arXiv:2405.05638 (replaced) [pdf, html, other]
-
Title: A Correlation-induced Finite Difference EstimatorSubjects: Methodology (stat.ME); Machine Learning (cs.LG); Numerical Analysis (math.NA); Optimization and Control (math.OC)
Finite difference (FD) approximation is a classic approach to stochastic gradient estimation when only noisy function realizations are available. In this paper, we first provide a sample-driven method via the bootstrap technique to estimate the optimal perturbation, and then propose an efficient FD estimator based on correlated samples at the estimated optimal perturbation. Furthermore, theoretical analyses of both the perturbation estimator and the FD estimator reveal that, {\it surprisingly}, the correlation enables the proposed FD estimator to achieve a reduction in variance and, in some cases, a decrease in bias compared to the traditional optimal FD estimator. Numerical results confirm the efficiency of our estimators and align well with the theory presented, especially in scenarios with small sample sizes. Finally, we apply the estimator to solve derivative-free optimization (DFO) problems, and numerical studies show that DFO problems with 100 dimensions can be effectively solved.
- [695] arXiv:2405.13073 (replaced) [pdf, other]
-
Title: A graph-structured distance for mixed-variable domains with meta variablesComments: 25 pages (without references), 5 figures, data and scripts available at this https URLSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Heterogeneous datasets emerge in various machine learning and optimization applications that feature different input sources, types or formats. Most models or methods do not natively tackle heterogeneity. Hence, such datasets are often partitioned into smaller and simpler ones, which may limit the generalizability or performance, especially if data is limited. The first main contribution of this work is a modeling framework that generalizes hierarchical, tree-structured, variable-size or conditional search frameworks. The framework models mixed-variable domains in which variables may be continuous, integer, or categorical, with some identified as meta when they influence the structure of the problem. The second main contribution is a novel distance that compares any pair of mixed-variable points that do not share the same variables, allowing to use whole heterogeneous datasets that reside in mixed-variable domains with meta variables. The contributions are illustrated on several regression experiments, in which the performance of a multilayer perceptron with respect to its hyperparameters is modeled.
- [696] arXiv:2405.17172 (replaced) [pdf, html, other]
-
Title: Partitioning Complete Geometric Graphs on Dense Point Sets into Plane SubgraphsComments: 10 pages, 5 figuresSubjects: Combinatorics (math.CO); Discrete Mathematics (cs.DM)
A \emph{complete geometric graph} consists of a set $P$ of $n$ points in the plane, in general position, and all segments (edges) connecting them. It is a well known question of Bose, Hurtado, Rivera-Campo, and Wood, whether there exists a positive constant $c<1$, such that every complete geometric graph on $n$ points can be partitioned into at most $cn$ plane graphs (that is, noncrossing subgraphs). We answer this question in the affirmative in the special case where the underlying point set $P$ is \emph{dense}, which means that the ratio between the maximum and the minimum distances in $P$ is of the order of $\Theta(\sqrt{n})$.
- [697] arXiv:2405.19224 (replaced) [pdf, html, other]
-
Title: A study on the adequacy of common IQA measures for medical imagesAnna Breger, Clemens Karner, Ian Selby, Janek Gröhl, Sören Dittmer, Edward Lilley, Judith Babar, Jake Beckford, Thomas R Else, Timothy J Sadler, Shahab Shahipasand, Arthikkaa Thavakumar, Michael Roberts, Carola-Bibiane SchönliebSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Image quality assessment (IQA) is standard practice in the development stage of novel machine learning algorithms that operate on images. The most commonly used IQA measures have been developed and tested for natural images, but not in the medical setting. Reported inconsistencies arising in medical images are not surprising, as they have different properties than natural images. In this study, we test the applicability of common IQA measures for medical image data by comparing their assessment to manually rated chest X-ray (5 experts) and photoacoustic image data (2 experts). Moreover, we include supplementary studies on grayscale natural images and accelerated brain MRI data. The results of all experiments show a similar outcome in line with previous findings for medical imaging: PSNR and SSIM in the default setting are in the lower range of the result list and HaarPSI outperforms the other tested measures in the overall performance. Also among the top performers in our medical experiments are the full reference measures FSIM, GMSD, LPIPS and MS-SSIM. Generally, the results on natural images yield considerably higher correlations, suggesting that the additional employment of tailored IQA measures for medical imaging algorithms is needed.
- [698] arXiv:2405.19725 (replaced) [pdf, html, other]
-
Title: Quantum Visual Feature Encoding RevisitedComments: Accepted to Quantum Machine IntelligenceSubjects: Quantum Physics (quant-ph); Computer Vision and Pattern Recognition (cs.CV)
Although quantum machine learning has been introduced for a while, its applications in computer vision are still limited. This paper, therefore, revisits the quantum visual encoding strategies, the initial step in quantum machine learning. Investigating the root cause, we uncover that the existing quantum encoding design fails to ensure information preservation of the visual features after the encoding process, thus complicating the learning process of the quantum machine learning models. In particular, the problem, termed "Quantum Information Gap" (QIG), leads to a gap of information between classical and corresponding quantum features. We provide theoretical proof and practical demonstrations of that found and underscore the significance of QIG, as it directly impacts the performance of quantum machine learning algorithms. To tackle this challenge, we introduce a simple but efficient new loss function named Quantum Information Preserving (QIP) to minimize this gap, resulting in enhanced performance of quantum machine learning algorithms. Extensive experiments validate the effectiveness of our approach, showcasing superior performance compared to current methodologies and consistently achieving state-of-the-art results in quantum modeling.
- [699] arXiv:2406.05074 (replaced) [pdf, html, other]
-
Title: Hibou: A Family of Foundational Vision Transformers for PathologySubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
Pathology, the microscopic examination of diseased tissue, is critical for diagnosing various medical conditions, particularly cancers. Traditional methods are labor-intensive and prone to human error. Digital pathology, which converts glass slides into high-resolution digital images for analysis by computer algorithms, revolutionizes the field by enhancing diagnostic accuracy, consistency, and efficiency through automated image analysis and large-scale data processing. Foundational transformer pretraining is crucial for developing robust, generalizable models as it enables learning from vast amounts of unannotated data.
This paper introduces the Hibou family of foundational vision transformers for pathology, leveraging the DINOv2 framework to pretrain two model variants, Hibou-B and Hibou-L, on a proprietary dataset of over 1 million whole slide images (WSIs) representing diverse tissue types and staining techniques. Our pretrained models demonstrate superior performance on both patch-level and slide-level benchmarks, surpassing existing state-of-the-art methods. Notably, Hibou-L achieves the highest average accuracy across multiple benchmark datasets. To support further research and application in the field, we have open-sourced the Hibou models, which can be accessed at this https URL. - [700] arXiv:2406.05175 (replaced) [pdf, html, other]
-
Title: Robust quantum dots charge autotuning using neural networks uncertaintyVictor Yon, Bastien Galaup, Claude Rohrbacher, Joffrey Rivard, Clément Godfrin, Ruoyu Li, Stefan Kubicek, Kristiaan De Greve, Louis Gaudreau, Eva Dupont-Ferrier, Yann Beilliard, Roger G. Melko, Dominique DrouinComments: 12 pages (main) + 13 pages (supplementary)Subjects: Quantum Physics (quant-ph); Machine Learning (cs.LG)
This study presents a machine-learning-based procedure to automate the charge tuning of semiconductor spin qubits with minimal human intervention, addressing one of the significant challenges in scaling up quantum dot technologies. This method exploits artificial neural networks to identify noisy transition lines in stability diagrams, guiding a robust exploration strategy leveraging neural networks' uncertainty estimations. Tested across three distinct offline experimental datasets representing different single quantum dot technologies, the approach achieves over 99% tuning success rate in optimal cases, where more than 10% of the success is directly attributable to uncertainty exploitation. The challenging constraints of small training sets containing high diagram-to-diagram variability allowed us to evaluate the capabilities and limits of the proposed procedure.
- [701] arXiv:2406.11168 (replaced) [pdf, html, other]
-
Title: Two-Timescale Optimization Framework for Decentralized Linear-Quadratic Optimal ControlSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG)
This study investigates a decentralized linear-quadratic optimal control problem, and several approximate separable constrained optimization problems are formulated for the first time based on the selection of sparsity promoting functions. First, for the optimization problem with weighted $\ell_1$ sparsity promoting function, a two-timescale algorithm is adopted that is based on the BSUM (Block Successive Upper-bound Minimization) framework and a differential equation solver. Second, a piecewise quadratic sparsity promoting function is introduced, and the induced optimization problem demonstrates an accelerated convergence rate by performing the same two-timescale algorithm. Finally, the optimization problem with $\ell_0$ sparsity promoting function is considered that is nonconvex and discontinuous, and can be approximated by successive coordinatewise convex optimization problems.
- [702] arXiv:2407.12288 (replaced) [pdf, other]
-
Title: Information-Theoretic Foundations for Machine LearningSubjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
The staggering progress of machine learning in the past decade has been a sight to behold. In retrospect, it is both remarkable and unsettling that these milestones were achievable with little to no rigorous theory to guide experimentation. Despite this fact, practitioners have been able to guide their future experimentation via observations from previous large-scale empirical investigations. However, alluding to Plato's Allegory of the cave, it is likely that the observations which form the field's notion of reality are but shadows representing fragments of that reality. In this work, we propose a theoretical framework which attempts to answer what exists outside of the cave. To the theorist, we provide a framework which is mathematically rigorous and leaves open many interesting ideas for future exploration. To the practitioner, we provide a framework whose results are very intuitive, general, and which will help form principles to guide future investigations. Concretely, we provide a theoretical framework rooted in Bayesian statistics and Shannon's information theory which is general enough to unify the analysis of many phenomena in machine learning. Our framework characterizes the performance of an optimal Bayesian learner, which considers the fundamental limits of information. Throughout this work, we derive very general theoretical results and apply them to derive insights specific to settings ranging from data which is independently and identically distributed under an unknown distribution, to data which is sequential, to data which exhibits hierarchical structure amenable to meta-learning. We conclude with a section dedicated to characterizing the performance of misspecified algorithms. These results are exciting and particularly relevant as we strive to overcome increasingly difficult machine learning challenges in this endlessly complex world.
- [703] arXiv:2407.13220 (replaced) [pdf, html, other]
-
Title: MEDIC: Zero-shot Music Editing with Disentangled Inversion ControlSubjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)
Text-guided diffusion models catalyze a paradigm shift in audio generation, facilitating the adaptability of source audio to conform to specific textual prompts. Recent advancements introduce inversion techniques, like DDIM inversion, to zero-shot editing, exploiting pre-trained diffusion models for audio modification. Nonetheless, our investigation exposes that DDIM inversion suffers from an accumulation of errors across each diffusion step, undermining its efficacy. And the lack of attention control hinders the fine-grained manipulations of music. To counteract these limitations, we introduce the \textit{Disentangled Inversion} technique, which is designed to disentangle the diffusion process into triple branches, thereby magnifying their individual capabilities for both precise editing and preservation. Furthermore, we propose the \textit{Harmonized Attention Control} framework, which unifies the mutual self-attention and cross-attention with an additional Harmonic Branch to achieve the desired composition and structural information in the target music. Collectively, these innovations comprise the \textit{Disentangled Inversion Control (DIC)} framework, enabling accurate music editing whilst safeguarding structural integrity. To benchmark audio editing efficacy, we introduce \textit{ZoME-Bench}, a comprehensive music editing benchmark hosting 1,100 samples spread across 10 distinct editing categories, which facilitates both zero-shot and instruction-based music editing tasks. Our method demonstrates unparalleled performance in edit fidelity and essential content preservation, outperforming contemporary state-of-the-art inversion techniques.
- [704] arXiv:2408.02299 (replaced) [pdf, other]
-
Title: Various Properties of Various Ultrafilters, Various Graph Width Parameters, and Various Connectivity SystemsComments: 64 pages, 3 tables, 3 figuresSubjects: Combinatorics (math.CO); Discrete Mathematics (cs.DM); Logic (math.LO)
Filters are collections of sets that are closed under supersets and finite intersections, serving as fundamental tools in topology and set theory. An ultrafilter, a maximal filter on a set, plays a crucial role in these fields by rigorously handling limits, convergence, and compactness. A connectivity system is defined as a pair (X,f) , where X is a finite set and f is a symmetric submodular function. Understanding the duality in these parameters elucidates the relationship between different decompositions and measures of a graph's complexity. In this paper, we delve into ultrafilters on connectivity systems, applying Tukey's Lemma to these systems. Additionally, we explore prefilters, ultra-prefilters, and subbases within the context of connectivity systems. Furthermore, we introduce and investigate new parameters related to width, length, and depth, enhancing our understanding of these mathematical structures.
- [705] arXiv:2408.02693 (replaced) [pdf, html, other]
-
Title: Diff-PIC: Revolutionizing Particle-In-Cell Simulation for Advancing Nuclear Fusion with Diffusion ModelsChuan Liu, Chunshu Wu, Shihui Cao, Mingkai Chen, James Chenhao Liang, Ang Li, Michael Huang, Chuang Ren, Dongfang Liu, Ying Nian Wu, Tong GengSubjects: Computational Physics (physics.comp-ph); Artificial Intelligence (cs.AI)
Sustainable energy is a crucial global challenge, and recent breakthroughs in nuclear fusion ignition underscore the potential of harnessing energy extracted from nuclear fusion in everyday life, thereby drawing significant attention to fusion ignition research, especially Laser-Plasma Interaction (LPI). Unfortunately, the complexity of LPI at ignition scale renders theory-based analysis nearly impossible -- instead, it has to rely heavily on Particle-in-Cell (PIC) simulations, which is extremely computationally intensive, making it a major bottleneck in advancing fusion ignition. In response, this work introduces Diff-PIC, a novel paradigm that leverages conditional diffusion models as a computationally efficient alternative to PIC simulations for generating high-fidelity scientific data. Specifically, we design a distillation paradigm to distill the physical patterns captured by PIC simulations into diffusion models, demonstrating both theoretical and practical feasibility. Moreover, to ensure practical effectiveness, we provide solutions for two critical challenges: (1) We develop a physically-informed conditional diffusion model that can learn and generate meaningful embeddings for mathematically continuous physical conditions. This model offers algorithmic generalization and adaptable transferability, effectively capturing the complex relationships between physical conditions and simulation outcomes; and (2) We employ the rectified flow technique to make our model a one-step conditional diffusion model, enhancing its efficiency further while maintaining high fidelity and physical validity. Diff-PIC establishes a new paradigm for using diffusion models to overcome the computational barriers in nuclear fusion research, setting a benchmark for future innovations and advancements in this field.
- [706] arXiv:2408.07947 (replaced) [pdf, html, other]
-
Title: Conditional Brownian Bridge Diffusion Model for VHR SAR to Optical Image TranslationComments: 5 pages, 2 figures, 1 tableSubjects: Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
Synthetic Aperture Radar (SAR) imaging technology provides the unique advantage of being able to collect data regardless of weather conditions and time. However, SAR images exhibit complex backscatter patterns and speckle noise, which necessitate expertise for interpretation. Research on translating SAR images into optical-like representations has been conducted to aid the interpretation of SAR data. Nevertheless, existing studies have predominantly utilized low-resolution satellite imagery datasets and have largely been based on Generative Adversarial Network (GAN) which are known for their training instability and low fidelity. To overcome these limitations of low-resolution data usage and GAN-based approaches, this paper introduces a conditional image-to-image translation approach based on Brownian Bridge Diffusion Model (BBDM). We conducted comprehensive experiments on the MSAW dataset, a paired SAR and optical images collection of 0.5m Very-High-Resolution (VHR). The experimental results indicate that our method surpasses both the Conditional Diffusion Models (CDMs) and the GAN-based models in diverse perceptual quality metrics.
- [707] arXiv:2408.07997 (replaced) [pdf, html, other]
-
Title: Enhanced Quantum Energy Teleportation using a 3-Qubit SystemComments: 10 pages, 6 figures, 1 table, 44 equationsSubjects: Quantum Physics (quant-ph); Emerging Technologies (cs.ET)
Quantum Energy Teleportation (QET) is a novel method that leverages quantum entanglement to transfer energy between two distant locations without any physical movement of the energy. The first realization of QET on superconducting hardware, utilizing a 2-qubit system, demonstrated an average energy retrieval efficiency of 11.4% by the receiver, Bob. In this paper, we present a new approach using a 3-qubit system to enhance the energy efficiency of QET. We have incorporated a novel 3-qubit ground state hamiltonian H to achieve this, that conforms the constraints of Zero mean energy. Our experimental results show a significant improvement in energy retrieval, achieving an average efficiency of 46.4%, which is significantly higher than that of the 2-qubit system. This advancement not only marks a step forward in practical quantum energy applications but also provides a new framework for future research in quantum energy teleportation and related quantum technologies.
- [708] arXiv:2408.08065 (replaced) [pdf, html, other]
-
Title: SPEED: Scalable Preprocessing of EEG Data for Self-Supervised LearningComments: To appear in proceedings of 2024 IEEE International workshop on Machine Learning for Signal ProcessingSubjects: Signal Processing (eess.SP); Artificial Intelligence (cs.AI)
Electroencephalography (EEG) research typically focuses on tasks with narrowly defined objectives, but recent studies are expanding into the use of unlabeled data within larger models, aiming for a broader range of applications. This addresses a critical challenge in EEG research. For example, Kostas et al. (2021) show that self-supervised learning (SSL) outperforms traditional supervised methods. Given the high noise levels in EEG data, we argue that further improvements are possible with additional preprocessing. Current preprocessing methods often fail to efficiently manage the large data volumes required for SSL, due to their lack of optimization, reliance on subjective manual corrections, and validation processes or inflexible protocols that limit SSL. We propose a Python-based EEG preprocessing pipeline optimized for self-supervised learning, designed to efficiently process large-scale data. This optimization not only stabilizes self-supervised training but also enhances performance on downstream tasks compared to training with raw data.
- [709] arXiv:2408.09218 (replaced) [pdf, other]
-
Title: A Fast and Computationally Inexpensive Method For Image Translation of 3D Volume Patient DataSubjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
CycleGAN was trained on SynthRAD Grand Challenge Dataset using the single-epoch modification (SEM) method proposed in this paper which is referred to as (CycleGAN-single) compared to the usual method of training CycleGAN on around 200 epochs (CycleGAN-multi). Model performance were evaluated qualitatively and quantitatively with quantitative performance metrics like PSNR, SSIM, MAE and MSE. The consideration of both quantitative and qualitative performance when evaluating a model is unique to certain image-translation tasks like medical imaging as detailed in this paper. Also, this paper shows that good quantitative performance does not always imply good qualitative performance and the converse is also not always True (i.e. good qualitative performance does not always imply good quantitative performance). This paper also proposes FQGA (Fast Paired Image-to-Image Translation Quarter-Generator Adversary) Model which has 1/4 the number of parameters compared to CycleGAN (when comparing their Generator Models). FQGA outperforms CycleGAN qualitatively and quantitatively even only after training on 20 epochs. Finally, using SEM method on FQGA allowed it to again outperform CycleGAN both quantitatively and qualitatively. These performance gains with fewer model parameters and time savings from running fewer epochs may also be applicable to other image-to-image translation tasks in Machine Learning apart from the Medical image-translation task discussed in this paper between Cone Beam Computed Tomography (CBCT) and Computed Tomography (CT) images.
- [710] arXiv:2408.09420 (replaced) [pdf, html, other]
-
Title: Enhancing Startup Success Predictions in Venture Capital: A GraphRAG Augmented Multivariate Time Series MethodComments: arXiv admin note: text overlap with arXiv:2312.13936, arXiv:2312.04876, arXiv:2402.11454 by other authorsSubjects: Computational Finance (q-fin.CP); Computation and Language (cs.CL); Machine Learning (cs.LG)
In the Venture Capital(VC) industry, predicting the success of startups is challenging due to limited financial data and the need for subjective revenue forecasts. Previous methods based on time series analysis or deep learning often fall short as they fail to incorporate crucial inter-company relationships such as competition and collaboration. Regarding the issues, we propose a novel approach using GrahphRAG augmented time series model. With GraphRAG, time series predictive methods are enhanced by integrating these vital relationships into the analysis framework, allowing for a more dynamic understanding of the startup ecosystem in venture capital. Our experimental results demonstrate that our model significantly outperforms previous models in startup success predictions. To the best of our knowledge, our work is the first application work of GraphRAG.
- [711] arXiv:2408.09554 (replaced) [pdf, html, other]
-
Title: Screen Them All: High-Throughput Pan-Cancer Genetic and Phenotypic Biomarker Screening from H&E Whole Slide ImagesYi Kan Wang, Ludmila Tydlitatova, Jeremy D. Kunz, Gerard Oakley, Ran A. Godrich, Matthew C. H. Lee, Chad Vanderbilt, Razik Yousfi, Thomas Fuchs, David S. Klimstra, Siqi LiuSubjects: Quantitative Methods (q-bio.QM); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)
Many molecular alterations serve as clinically prognostic or therapy-predictive biomarkers, typically detected using single or multi-gene molecular assays. However, these assays are expensive, tissue destructive and often take weeks to complete. Using AI on routine H&E WSIs offers a fast and economical approach to screen for multiple molecular biomarkers. We present a high-throughput AI-based system leveraging Virchow2, a foundation model pre-trained on 3 million slides, to interrogate genomic features previously determined by an next-generation sequencing (NGS) assay, using 47,960 scanned hematoxylin and eosin (H&E) whole slide images (WSIs) from 38,984 cancer patients. Unlike traditional methods that train individual models for each biomarker or cancer type, our system employs a unified model to simultaneously predict a wide range of clinically relevant molecular biomarkers across cancer types. By training the network to replicate the MSK-IMPACT targeted biomarker panel of 505 genes, it identified 80 high performing biomarkers with a mean AU-ROC of 0.89 in 15 most common cancer types. In addition, 40 biomarkers demonstrated strong associations with specific cancer histologic subtypes. Furthermore, 58 biomarkers were associated with targets frequently assayed clinically for therapy selection and response prediction. The model can also predict the activity of five canonical signaling pathways, identify defects in DNA repair mechanisms, and predict genomic instability measured by tumor mutation burden, microsatellite instability (MSI), and chromosomal instability (CIN). The proposed model can offer potential to guide therapy selection, improve treatment efficacy, accelerate patient screening for clinical trials and provoke the interrogation of new therapeutic targets.
- [712] arXiv:2408.09641 (replaced) [pdf, html, other]
-
Title: Character Complexity: A Novel Measure for Quantum Circuit AnalysisSubjects: Quantum Physics (quant-ph); Computational Complexity (cs.CC); Mathematical Physics (math-ph)
In the rapidly evolving field of quantum computing, quantifying circuit complexity remains a critical challenge. This paper introduces Character Complexity, a novel measure that bridges Group-theoretic concepts with practical quantum computing concerns. By leveraging tools from representation theory, I prove several key properties of character complexity and establish a surprising connection to the classical simulability of quantum circuits. This new measure offers a fresh perspective on the complexity landscape of quantum algorithms, potentially reshaping our understanding of quantum-classical computational boundaries. I present innovative visualization methods for character complexity, providing intuitive insights into the structure of quantum circuits. The empirical results reveal intriguing scaling behaviors with respect to qubit and gate counts, opening new avenues for quantum algorithm design and optimization. This work not only contributes to the theoretical foundations of quantum complexity but also offers practical tools for the quantum computing community. As quantum hardware continues to advance, character complexity could play a crucial role in developing more efficient quantum algorithms and in exploring the fundamental limits of quantum computation.
- [713] arXiv:2408.10114 (replaced) [pdf, html, other]
-
Title: Topics in Algebra of Synchronous Games, Algebraic Graph Identities and Quantum NP-hardness ReductionsComments: Mitacs research internship report. 15.5 pages + 1 page citationSubjects: Quantum Physics (quant-ph); Computational Complexity (cs.CC); Operator Algebras (math.OA)
We review the correspondence between a synchronous game and its associated game algebra. We slightly develop the work of Helton et al.[HMPS17] by proposing results on algebraic and locally commuting graph identities. Based on the theoretical works on noncommutative Nullstellensätze [BWHK23], we build computational tools involving Gröbner basis method and semidefinite programming to check the existence of perfect strategies with specific models. We prove the equivalence between the hereditary and $C^*$ models proposed in [HMPS17]. We also extend Ji's reduction $\texttt{3-Coloring}^* \leq_p \texttt{3-SAT}^*$ [Ji13] and exhibit another instance of quantum-version NP-hardness reduction $\texttt{Clique}^* \leq_p \texttt{3-SAT}^*$.