Showing 1–50 of 121 results for author: Petrov, S

Gemma 2: Improving Open Language Models at a Practical Size

Authors: Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stanczyk, Sertan Girgin, Nikola Momchev, Matt Hoffman , et al. (172 additional authors not shown)

Abstract: In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In this new version, we apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions (Beltagy et al., 2020a) and group-query attention (Ainslie et al., 2023). We al… ▽ More In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In this new version, we apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions (Beltagy et al., 2020a) and group-query attention (Ainslie et al., 2023). We also train the 2B and 9B models with knowledge distillation (Hinton et al., 2015) instead of next token prediction. The resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger. We release all our models to the community. △ Less

Submitted 2 August, 2024; v1 submitted 31 July, 2024; originally announced August 2024.

Reinforcement Learning to Disentangle Multiqubit Quantum States from Partial Observations

Authors: Pavel Tashev, Stefan Petrov, Friederike Metz, Marin Bukov

Abstract: Using partial knowledge of a quantum state to control multiqubit entanglement is a largely unexplored paradigm in the emerging field of quantum interactive dynamics with the potential to address outstanding challenges in quantum state preparation and compression, quantum control, and quantum complexity. We present a deep reinforcement learning (RL) approach to constructing short disentangling circ… ▽ More Using partial knowledge of a quantum state to control multiqubit entanglement is a largely unexplored paradigm in the emerging field of quantum interactive dynamics with the potential to address outstanding challenges in quantum state preparation and compression, quantum control, and quantum complexity. We present a deep reinforcement learning (RL) approach to constructing short disentangling circuits for arbitrary 4-, 5-, and 6-qubit states using an actor-critic algorithm. With access to only two-qubit reduced density matrices, our agent decides which pairs of qubits to apply two-qubit gates on; requiring only local information makes it directly applicable on modern NISQ devices. Utilizing a permutation-equivariant transformer architecture, the agent can autonomously identify qubit permutations within the state, and adjusts the disentangling protocol accordingly. Once trained, it provides circuits from different initial states without further optimization. We demonstrate the agent's ability to identify and exploit the entanglement structure of multiqubit states. For 4-, 5-, and 6-qubit Haar-random states, the agent learns to construct disentangling circuits that exhibit strong correlations both between consecutive gates and among the qubits involved. Through extensive benchmarking, we show the efficacy of the RL approach to find disentangling protocols with minimal gate resources. We explore the resilience of our trained agents to noise, highlighting their potential for real-world quantum computing applications. Analyzing optimal disentangling protocols, we report a general circuit to prepare an arbitrary 4-qubit state using at most 5 two-qubit (10 CNOT) gates. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: The source code as well as a demo in the form of an interactive Jupyter notebook are available on Github: https://github.com/mgbukov/RL_disentangle

Absence of binding of heavy fermions by two light atoms in two dimensions

Authors: J. Givois, D. S. Petrov

Abstract: By developing the mean-field theory valid for large $N$, we investigate the problem of two light fermions interacting via a zero-range potential with $N$ heavy fermions in two dimensions. We obtain numerical evidence that this system is never fully bound. It always splits into droplets containing a single light atom. This is in contrast to the one-dimensional case where any number of heavy and lig… ▽ More By developing the mean-field theory valid for large $N$, we investigate the problem of two light fermions interacting via a zero-range potential with $N$ heavy fermions in two dimensions. We obtain numerical evidence that this system is never fully bound. It always splits into droplets containing a single light atom. This is in contrast to the one-dimensional case where any number of heavy and light fermions can be bound together. △ Less

Submitted 7 May, 2024; originally announced May 2024.

Comments: 11 pages, 1 figure

Journal ref: Few-Body Syst. 65, 80 (2024)

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

Authors: Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, Soroosh Mariooryad, Yifan Ding, Xinyang Geng, Fred Alcober, Roy Frostig, Mark Omernick, Lexi Walker, Cosmin Paduraru, Christina Sorokin, Andrea Tacchetti, Colin Gaffney, Samira Daruki, Olcan Sercinoglu, Zach Gleicher, Juliette Love , et al. (1110 additional authors not shown)

Abstract: In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February… ▽ More In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content. △ Less

Submitted 8 August, 2024; v1 submitted 8 March, 2024; originally announced March 2024.

Gemini: A Family of Highly Capable Multimodal Models

Authors: Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul R. Barham, Tom Hennigan, Benjamin Lee , et al. (1325 additional authors not shown)

Abstract: This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultr… ▽ More This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI. △ Less

Submitted 17 June, 2024; v1 submitted 18 December, 2023; originally announced December 2023.

On the rank of the communication matrix for deterministic two-way finite automata

Authors: Semyon Petrov, Fedor Petrov, Alexander Okhotin

Abstract: The communication matrix for two-way deterministic finite automata (2DFA) with $n$ states is defined for an automaton over a full alphabet of all $(2n+1)^n$ possible symbols: its rows and columns are indexed by strings, and the entry $(u, v)$ is $1$ if $uv$ is accepted by the automaton, and $0$ otherwise. With duplicate rows and columns removed, this is a square matrix of order $n(n^n-(n-1)^n)+1$,… ▽ More The communication matrix for two-way deterministic finite automata (2DFA) with $n$ states is defined for an automaton over a full alphabet of all $(2n+1)^n$ possible symbols: its rows and columns are indexed by strings, and the entry $(u, v)$ is $1$ if $uv$ is accepted by the automaton, and $0$ otherwise. With duplicate rows and columns removed, this is a square matrix of order $n(n^n-(n-1)^n)+1$, and its rank is known to be a lower bound on the number of states necessary to transform an $n$-state 2DFA to a one-way unambiguous finite automaton (UFA). This paper determines this rank, showing that it is exactly $f(n)=\sum_{k=1}^n \binom{n}{k-1} \binom{n}{k} \binom{2k-2}{k-1} =(1+o(1)) \frac{3\sqrt{3}}{8πn} 9^n$, and this function becomes the new lower bound on the state complexity of the 2DFA to UFA transformation, thus improving a recent lower bound by S. Petrov and Okhotin (``On the transformation of two-way deterministic finite automata to unambiguous finite automata'', Inf. Comput., 2023). The key element of the proof is determining the rank of a $k! \times k!$ submatrix, with its rows and columns indexed by permutations, where the entry $(π, σ)$ is $1$ if $σ\circ π$ is a cycle of length $k$, and 0 otherwise; using the methods of group representation theory it is shown that its rank is exactly $\binom{2k-2}{k-1}$, and this implies the above formula for $f(n)$. △ Less

Submitted 10 December, 2023; originally announced December 2023.

Comments: 17 pages, 3 figures

MSC Class: 68Q45 (primary); 20C30

Beyond-mean-field effects in mixtures: few-body and many-body aspects

Authors: D. S. Petrov

Abstract: The discovery of ultracold dilute liquids has significantly elevated our interest in various phenomena which go under the name of beyond-mean-field (BMF) physics. In these lecture notes we give an elementary introduction to the quantum stabilization and liquefaction of a collapsing weakly interacting Bose-Bose mixture. A detailed derivation of the leading BMF correction, also known as the Lee-Huan… ▽ More The discovery of ultracold dilute liquids has significantly elevated our interest in various phenomena which go under the name of beyond-mean-field (BMF) physics. In these lecture notes we give an elementary introduction to the quantum stabilization and liquefaction of a collapsing weakly interacting Bose-Bose mixture. A detailed derivation of the leading BMF correction, also known as the Lee-Huang-Yang (LHY) term, in this system is presented in a manner suitable for further generalizations and extensions. Although the LHY term is a nonanalytic function of the density $n$, under certain conditions the leading BMF correction becomes analytic and can be expanded in integer powers of $n$, effectively introducing three-body and higher-order interactions. We discuss why and how well the Bogoliubov approach can predict these few-body observables. △ Less

Submitted 8 December, 2023; originally announced December 2023.

Comments: 36 pages, Proceedings of the International School of Physics "Enrico Fermi", Course 211 - Quantum Mixtures with Ultra-cold Atoms, July 2022

Selective damping of plasmons in coupled two-dimensional systems by Coulomb drag

Authors: Ilya Safonov, Aleksandr S. Petrov, Dmitry Svintsov

Abstract: The Coulomb drag is a many-body effect observed in proximized low-dimensional systems. It appears as emergence of voltage in one of them upon passage of bias current in another. The magnitude of drag voltage can be strongly affected by exchange of plasmonic excitations between the layers; however, the reverse effect of Coulomb drag on properties of plasmons has not been studied. Here, we study the… ▽ More The Coulomb drag is a many-body effect observed in proximized low-dimensional systems. It appears as emergence of voltage in one of them upon passage of bias current in another. The magnitude of drag voltage can be strongly affected by exchange of plasmonic excitations between the layers; however, the reverse effect of Coulomb drag on properties of plasmons has not been studied. Here, we study the plasmon spectra and damping in parallel two-dimensional systems in the presence of Coulomb drag. We find that Coulomb drag leads to selective damping of one of the two fundamental plasma modes of a coupled bilayer. For identical electron doping of both layers, the drag suppresses the acoustic plasma mode; while for symmetric electron-hole doping of the coupled pair, the drag suppresses the optical plasma mode. The selective damping can be observed both for propagating modes in extended bilayers and for localized plasmons in bilayers confined by source and drain contacts. The discussed effect may provide access to the strength of Coulomb interaction in 2d electron systems from various optical and microwave scattering experiments. △ Less

Submitted 8 December, 2023; originally announced December 2023.

Gas-to-soliton transition of attractive bosons on a spherical surface

Authors: A. Tononi, G. E. Astrakharchik, D. S. Petrov

Abstract: We investigate the ground state properties of $N$ bosons with attractive zero-range interactions characterized by the scattering length $a>0$ and confined to the surface of a sphere of radius $R$. We present the analytic solution of the problem for $N=2$, mean-field analysis for $N\rightarrow \infty$, and exact diffusion Monte-Carlo results for intermediate $N$. For finite $N$, we observe a smooth… ▽ More We investigate the ground state properties of $N$ bosons with attractive zero-range interactions characterized by the scattering length $a>0$ and confined to the surface of a sphere of radius $R$. We present the analytic solution of the problem for $N=2$, mean-field analysis for $N\rightarrow \infty$, and exact diffusion Monte-Carlo results for intermediate $N$. For finite $N$, we observe a smooth crossover from the uniform state in the limit $a/R\gg 1$ (weak attraction) to a localized state at small $a/R$ (strong attraction). With increasing $N$ this crossover narrows down to a discontinuous transition from the uniform state to a soliton of size $\sim R/\sqrt{N}$. The two states are separated by an energy barrier, tunneling under which is exponentially suppressed at large $N$. The system behavior is marked by a peculiar competition between space-curvature effects and beyond-mean-field terms, both breaking the scaling invariance of a two-dimensional mean-field theory. △ Less

Submitted 13 May, 2024; v1 submitted 8 December, 2023; originally announced December 2023.

Comments: 10 pages, 3 figures

Journal ref: AVS Quantum Sci. 6, 023201 (2024)

Theoretical Bounds for Noise Filtration using Low-Rank Tensor Approximations

Authors: Sergey Petrov, Nikolai Zamarashkin

Abstract: Low-rank tensor approximation error bounds are proposed for the case of noisy input data that depend on low-rank representation type, rank and the dimensionality of the tensor. The bounds show that high-dimensional low-rank structured approximations provide superior noise-filtering properties compared to matrices with the same rank and total element count. Low-rank tensor approximation error bounds are proposed for the case of noisy input data that depend on low-rank representation type, rank and the dimensionality of the tensor. The bounds show that high-dimensional low-rank structured approximations provide superior noise-filtering properties compared to matrices with the same rank and total element count. △ Less

Submitted 4 December, 2023; originally announced December 2023.

Heavy-light $N+1$ clusters of two-dimensional fermions

Authors: J. Givois, A. Tononi, D. S. Petrov

Abstract: We study binding of $N$ identical heavy fermions by a light atom in two dimensions assuming zero-range attractive heavy-light interactions. By using the mean-field theory valid for large $N$ we show that the $N+1$ cluster is bound when the mass ratio exceeds $1.074N^2$. The mean-field theory, being scale invariant in two dimensions, predicts only the shapes of the clusters leaving their sizes and… ▽ More We study binding of $N$ identical heavy fermions by a light atom in two dimensions assuming zero-range attractive heavy-light interactions. By using the mean-field theory valid for large $N$ we show that the $N+1$ cluster is bound when the mass ratio exceeds $1.074N^2$. The mean-field theory, being scale invariant in two dimensions, predicts only the shapes of the clusters leaving their sizes and energies undefined. By taking into account beyond-mean-field effects we find closed-form expressions for these quantities. We also discuss differences between the Thomas-Fermi and Hartree-Fock approaches for treating the heavy fermions. △ Less

Submitted 29 August, 2024; v1 submitted 17 October, 2023; originally announced October 2023.

Comments: 11 pages, 3 figures

Journal ref: SciPost Phys. 17, 050 (2024)

PaLM 2 Technical Report

Authors: Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez Abrego , et al. (103 additional authors not shown)

Abstract: We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on… ▽ More We introduce PaLM 2, a new state-of-the-art language model that has better multilingual and reasoning capabilities and is more compute-efficient than its predecessor PaLM. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Through extensive evaluations on English and multilingual language, and reasoning tasks, we demonstrate that PaLM 2 has significantly improved quality on downstream tasks across different model sizes, while simultaneously exhibiting faster and more efficient inference compared to PaLM. This improved efficiency enables broader deployment while also allowing the model to respond faster, for a more natural pace of interaction. PaLM 2 demonstrates robust reasoning capabilities exemplified by large improvements over PaLM on BIG-Bench and other reasoning tasks. PaLM 2 exhibits stable performance on a suite of responsible AI evaluations, and enables inference-time control over toxicity without additional overhead or impact on other capabilities. Overall, PaLM 2 achieves state-of-the-art performance across a diverse set of tasks and capabilities. When discussing the PaLM 2 family, it is important to distinguish between pre-trained models (of various sizes), fine-tuned variants of these models, and the user-facing products that use these models. In particular, user-facing products typically include additional pre- and post-processing steps. Additionally, the underlying models may evolve over time. Therefore, one should not expect the performance of user-facing products to exactly match the results reported in this report. △ Less

Submitted 13 September, 2023; v1 submitted 17 May, 2023; originally announced May 2023.

Plasmons in a Square of Two-Dimensional Electrons

Authors: A. M. Zarezin, D. Mylnikov, A. S. Petrov, D. Svintsov, P. A. Gusikhin, I. V. Kukushkin, V. M. Muravev

Abstract: Microwave absorption spectra of a single square of two-dimensional electrons (2DES) have been investigated using an optical detection technique. Fundamental dipole and harmonic quadrupole plasmon modes have been identified and compared to those in the disk geometry. In the square-shaped 2DES, a strong interaction is discovered between the neighboring plasmon modes, whereas no such hybridization is… ▽ More Microwave absorption spectra of a single square of two-dimensional electrons (2DES) have been investigated using an optical detection technique. Fundamental dipole and harmonic quadrupole plasmon modes have been identified and compared to those in the disk geometry. In the square-shaped 2DES, a strong interaction is discovered between the neighboring plasmon modes, whereas no such hybridization is observed in the disk-shaped geometry. We establish a rigid theoretical platform to analytically describe the magneto-optical response of confined two-dimensional systems. The developed theory provides a proper description of the obtained experimental results. △ Less

Submitted 20 December, 2022; originally announced December 2022.

Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models

Authors: Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Massimiliano Ciaramita, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, Tom Kwiatkowski, Ji Ma, Jianmo Ni, Lierni Sestorain Saralegui, Tal Schuster, William W. Cohen, Michael Collins, Dipanjan Das, Donald Metzler, Slav Petrov, Kellie Webster

Abstract: Large language models (LLMs) have shown impressive results while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM to attribute the text that it generates is likely to be crucial in this setting. We formulate and study Attributed QA as a key first step in the development of… ▽ More Large language models (LLMs) have shown impressive results while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM to attribute the text that it generates is likely to be crucial in this setting. We formulate and study Attributed QA as a key first step in the development of attributed LLMs. We propose a reproducible evaluation framework for the task and benchmark a broad set of architectures. We take human annotations as a gold standard and show that a correlated automatic metric is suitable for development. Our experimental work gives concrete answers to two key questions (How to measure attribution?, and How well do current state-of-the-art methods perform on attribution?), and give some hints as to how to address a third (How to build LLMs with attribution?). △ Less

Submitted 10 February, 2023; v1 submitted 15 December, 2022; originally announced December 2022.

Scaling Instruction-Finetuned Language Models

Authors: Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang , et al. (10 additional authors not shown)

Abstract: Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects d… ▽ More Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation). For instance, Flan-PaLM 540B instruction-finetuned on 1.8K tasks outperforms PALM 540B by a large margin (+9.4% on average). Flan-PaLM 540B achieves state-of-the-art performance on several benchmarks, such as 75.2% on five-shot MMLU. We also publicly release Flan-T5 checkpoints, which achieve strong few-shot performance even compared to much larger models, such as PaLM 62B. Overall, instruction finetuning is a general method for improving the performance and usability of pretrained language models. △ Less

Submitted 6 December, 2022; v1 submitted 20 October, 2022; originally announced October 2022.

Comments: Public checkpoints: https://huggingface.co/docs/transformers/model_doc/flan-t5

Transcending Scaling Laws with 0.1% Extra Compute

Authors: Yi Tay, Jason Wei, Hyung Won Chung, Vinh Q. Tran, David R. So, Siamak Shakeri, Xavier Garcia, Huaixiu Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc V. Le, Mostafa Dehghani

Abstract: Scaling language models improves performance but comes with significant computational costs. This paper proposes UL2R, a method that substantially improves existing language models and their scaling curves with a relatively tiny amount of extra compute. The key idea is to continue training a state-of-the-art large language model (e.g., PaLM) on a few more steps with UL2's mixture-of-denoiser objec… ▽ More Scaling language models improves performance but comes with significant computational costs. This paper proposes UL2R, a method that substantially improves existing language models and their scaling curves with a relatively tiny amount of extra compute. The key idea is to continue training a state-of-the-art large language model (e.g., PaLM) on a few more steps with UL2's mixture-of-denoiser objective. We show that, with almost negligible extra computational costs and no new sources of data, we are able to substantially improve the scaling properties of large language models on downstream metrics. In this paper, we continue training PaLM with UL2R, introducing a new set of models at 8B, 62B, and 540B scale which we call U-PaLM. Impressively, at 540B scale, we show an approximately 2x computational savings rate where U-PaLM achieves the same performance as the final PaLM 540B model at around half its computational budget (i.e., saving $\sim$4.4 million TPUv4 hours). We further show that this improved scaling curve leads to 'emergent abilities' on challenging BIG-Bench tasks -- for instance, U-PaLM does much better than PaLM on some tasks or demonstrates better quality at much smaller scale (62B as opposed to 540B). Overall, we show that U-PaLM outperforms PaLM on many few-shot setups, i.e., English NLP tasks (e.g., commonsense reasoning, question answering), reasoning tasks with chain-of-thought (e.g., GSM8K), multilingual tasks (MGSM, TydiQA), MMLU and challenging BIG-Bench tasks. Finally, we provide qualitative examples showing the new capabilities of U-PaLM for single and multi-span infilling. △ Less

Submitted 16 November, 2022; v1 submitted 20 October, 2022; originally announced October 2022.

Comments: V2 has updated references/related work

Self-binding of one-dimensional fermionic mixtures with zero-range interspecies attraction

Authors: J. Givois, A. Tononi, D. S. Petrov

Abstract: For sufficiently large mass ratios the attractive exchange force caused by a single light atom interacting with a few heavy identical fermions can overcome their Fermi degeneracy pressure and bind them into an $N+1$ cluster. Here, by using a mean-field approach valid for large $N$, we find that $N+1$ clusters can attract each other and form a self-bound charge density wave, the properties of which… ▽ More For sufficiently large mass ratios the attractive exchange force caused by a single light atom interacting with a few heavy identical fermions can overcome their Fermi degeneracy pressure and bind them into an $N+1$ cluster. Here, by using a mean-field approach valid for large $N$, we find that $N+1$ clusters can attract each other and form a self-bound charge density wave, the properties of which we fully characterize. Our work shows that there are no fundamental obstacles for having self-bound states in fermionic mixtures with zero-range interactions. △ Less

Submitted 8 June, 2023; v1 submitted 11 July, 2022; originally announced July 2022.

Comments: 11 pages, 4 figures

Journal ref: SciPost Phys. 14, 091 (2023)

Binding of heavy fermions by a single light atom in one dimension

Authors: A. Tononi, J. Givois, D. S. Petrov

Abstract: We consider the problem of $N$ identical fermions interacting via a zero-range attractive potential with a lighter atom in one dimension. Using the few-body approach based on the Skorniakov and Ter-Martirosian equation, we determine the energies and the critical mass ratios for the emergence of the tetramer, pentamer, and hexamer. For large $N$, we solve the problem analytically by using the mean-… ▽ More We consider the problem of $N$ identical fermions interacting via a zero-range attractive potential with a lighter atom in one dimension. Using the few-body approach based on the Skorniakov and Ter-Martirosian equation, we determine the energies and the critical mass ratios for the emergence of the tetramer, pentamer, and hexamer. For large $N$, we solve the problem analytically by using the mean-field theory based on the Thomas-Fermi approximation. The system becomes bound when the heavy-to-light mass ratio exceeds a critical value which grows as $N^3$ at large $N$. We also employ a more sophisticated Hartree-Fock approach, which turns out to be equivalent to the Thomas-Fermi approximation for determining the energies, but provides a better description of the microscopic structure of the clusters. △ Less

Submitted 12 July, 2022; v1 submitted 2 May, 2022; originally announced May 2022.

Comments: 6 pages, 2 figures

Journal ref: Phys. Rev. A 106, L011302 (2022)

PaLM: Scaling Language Modeling with Pathways

Authors: Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin , et al. (42 additional authors not shown)

Abstract: Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular application. To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Tran… ▽ More Large language models have been shown to achieve remarkable performance across a variety of natural language tasks using few-shot learning, which drastically reduces the number of task-specific training examples needed to adapt the model to a particular application. To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM. We trained PaLM on 6144 TPU v4 chips using Pathways, a new ML system which enables highly efficient training across multiple TPU Pods. We demonstrate continued benefits of scaling by achieving state-of-the-art few-shot learning results on hundreds of language understanding and generation benchmarks. On a number of these tasks, PaLM 540B achieves breakthrough performance, outperforming the finetuned state-of-the-art on a suite of multi-step reasoning tasks, and outperforming average human performance on the recently released BIG-bench benchmark. A significant number of BIG-bench tasks showed discontinuous improvements from model scale, meaning that performance steeply increased as we scaled to our largest model. PaLM also has strong capabilities in multilingual tasks and source code generation, which we demonstrate on a wide array of benchmarks. We additionally provide a comprehensive analysis on bias and toxicity, and study the extent of training data memorization with respect to model scale. Finally, we discuss the ethical considerations related to large language models and discuss potential mitigation strategies. △ Less

Submitted 5 October, 2022; v1 submitted 5 April, 2022; originally announced April 2022.

Measuring Attribution in Natural Language Generation Models

Authors: Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, David Reitter

Abstract: With recent improvements in natural language generation (NLG) models for various applications, it has become imperative to have the means to identify and evaluate whether NLG output is only sharing verifiable information about the external world. In this work, we present a new evaluation framework entitled Attributable to Identified Sources (AIS) for assessing the output of natural language genera… ▽ More With recent improvements in natural language generation (NLG) models for various applications, it has become imperative to have the means to identify and evaluate whether NLG output is only sharing verifiable information about the external world. In this work, we present a new evaluation framework entitled Attributable to Identified Sources (AIS) for assessing the output of natural language generation models, when such output pertains to the external world. We first define AIS and introduce a two-stage annotation pipeline for allowing annotators to appropriately evaluate model output according to AIS guidelines. We empirically validate this approach on generation datasets spanning three tasks (two conversational QA datasets, a summarization dataset, and a table-to-text dataset) via human evaluation studies that suggest that AIS could serve as a common framework for measuring whether model-generated statements are supported by underlying sources. We release guidelines for the human evaluation studies. △ Less

Submitted 2 August, 2022; v1 submitted 23 December, 2021; originally announced December 2021.

Soft-Sensing ConFormer: A Curriculum Learning-based Convolutional Transformer

Authors: Jaswanth Yella, Chao Zhang, Sergei Petrov, Yu Huang, Xiaoye Qian, Ali A. Minai, Sthitie Bom

Abstract: Over the last few decades, modern industrial processes have investigated several cost-effective methodologies to improve the productivity and yield of semiconductor manufacturing. While playing an essential role in facilitating real-time monitoring and control, the data-driven soft-sensors in industries have provided a competitive edge when augmented with deep learning approaches for wafer fault-d… ▽ More Over the last few decades, modern industrial processes have investigated several cost-effective methodologies to improve the productivity and yield of semiconductor manufacturing. While playing an essential role in facilitating real-time monitoring and control, the data-driven soft-sensors in industries have provided a competitive edge when augmented with deep learning approaches for wafer fault-diagnostics. Despite the success of deep learning methods across various domains, they tend to suffer from bad performance on multi-variate soft-sensing data domains. To mitigate this, we propose a soft-sensing ConFormer (CONvolutional transFORMER) for wafer fault-diagnostic classification task which primarily consists of multi-head convolution modules that reap the benefits of fast and light-weight operations of convolutions, and also the ability to learn the robust representations through multi-head design alike transformers. Another key issue is that traditional learning paradigms tend to suffer from low performance on noisy and highly-imbalanced soft-sensing data. To address this, we augment our soft-sensing ConFormer model with a curriculum learning-based loss function, which effectively learns easy samples in the early phase of training and difficult ones later. To further demonstrate the utility of our proposed architecture, we performed extensive experiments on various toolsets of Seagate Technology's wafer manufacturing process which are shared openly along with this work. To the best of our knowledge, this is the first time that curriculum learning-based soft-sensing ConFormer architecture has been proposed for soft-sensing data and our results show strong promise for future use in soft-sensing research domain. △ Less

Submitted 12 November, 2021; originally announced November 2021.

GraSSNet: Graph Soft Sensing Neural Networks

Authors: Yu Huang, Chao Zhang, Jaswanth Yella, Sergei Petrov, Xiaoye Qian, Yufei Tang, Xingquan Zhu, Sthitie Bom

Abstract: In the era of big data, data-driven based classification has become an essential method in smart manufacturing to guide production and optimize inspection. The industrial data obtained in practice is usually time-series data collected by soft sensors, which are highly nonlinear, nonstationary, imbalanced, and noisy. Most existing soft-sensing machine learning models focus on capturing either intra… ▽ More In the era of big data, data-driven based classification has become an essential method in smart manufacturing to guide production and optimize inspection. The industrial data obtained in practice is usually time-series data collected by soft sensors, which are highly nonlinear, nonstationary, imbalanced, and noisy. Most existing soft-sensing machine learning models focus on capturing either intra-series temporal dependencies or pre-defined inter-series correlations, while ignoring the correlation between labels as each instance is associated with multiple labels simultaneously. In this paper, we propose a novel graph based soft-sensing neural network (GraSSNet) for multivariate time-series classification of noisy and highly-imbalanced soft-sensing data. The proposed GraSSNet is able to 1) capture the inter-series and intra-series dependencies jointly in the spectral domain; 2) exploit the label correlations by superimposing label graph that built from statistical co-occurrence information; 3) learn features with attention mechanism from both textual and numerical domain; and 4) leverage unlabeled data and mitigate data imbalance by semi-supervised learning. Comparative studies with other commonly used classifiers are carried out on Seagate soft sensing data, and the experimental results validate the competitive performance of our proposed method. △ Less

Submitted 12 November, 2021; originally announced November 2021.

Soft Sensing Transformer: Hundreds of Sensors are Worth a Single Word

Authors: Chao Zhang, Jaswanth Yella, Yu Huang, Xiaoye Qian, Sergei Petrov, Andrey Rzhetsky, Sthitie Bom

Abstract: With the rapid development of AI technology in recent years, there have been many studies with deep learning models in soft sensing area. However, the models have become more complex, yet, the data sets remain limited: researchers are fitting million-parameter models with hundreds of data samples, which is insufficient to exercise the effectiveness of their models and thus often fail to perform wh… ▽ More With the rapid development of AI technology in recent years, there have been many studies with deep learning models in soft sensing area. However, the models have become more complex, yet, the data sets remain limited: researchers are fitting million-parameter models with hundreds of data samples, which is insufficient to exercise the effectiveness of their models and thus often fail to perform when implemented in industrial applications. To solve this long-lasting problem, we are providing large scale, high dimensional time series manufacturing sensor data from Seagate Technology to the public. We demonstrate the challenges and effectiveness of modeling industrial big data by a Soft Sensing Transformer model on these data sets. Transformer is used because, it has outperformed state-of-the-art techniques in Natural Language Processing, and since then has also performed well in the direct application to computer vision without introduction of image-specific inductive biases. We observe the similarity of a sentence structure to the sensor readings and process the multi-variable sensor readings in a time series in a similar manner of sentences in natural language. The high-dimensional time-series data is formatted into the same shape of embedded sentences and fed into the transformer model. The results show that transformer model outperforms the benchmark models in soft sensing field based on auto-encoder and long short-term memory (LSTM) models. To the best of our knowledge, we are the first team in academia or industry to benchmark the performance of original transformer model with large-scale numerical soft sensing data. △ Less

Submitted 10 November, 2021; originally announced November 2021.

Trajectory-based simulation of far-infrared CIA profiles of CH$_4-$N$_2$ for modeling Titan's atmosphere

Authors: Artem Finenko, Bruno Bézard, Iouli Gordon, Daniil Chistikov, Sergei Lokshtanov, Sergey Petrov, Andrey Vigasin

Abstract: We report the results of the trajectory-based simulation of far-infrared collision-induced absorption (CIA) due to CH$_4-$N$_2$ pairs at temperatures between 70 and 400 K. Our analysis utilizes recently calculated high-level potential energy (PES) and induced dipole surfaces (IDS) [Finenko, A. A., Chistikov, D. N., Kalugina, Y. N., Conway E. K., Gordon, I. E., Phys. Chem. Chem. Phys., 2021, doi: 1… ▽ More We report the results of the trajectory-based simulation of far-infrared collision-induced absorption (CIA) due to CH$_4-$N$_2$ pairs at temperatures between 70 and 400 K. Our analysis utilizes recently calculated high-level potential energy (PES) and induced dipole surfaces (IDS) [Finenko, A. A., Chistikov, D. N., Kalugina, Y. N., Conway E. K., Gordon, I. E., Phys. Chem. Chem. Phys., 2021, doi: 10.1039/d1cp02161c]. Treating collision partners as rigid rotors, the time evolution of interaction-induced dipole is accumulated over a vast ensemble of classical trajectories and subsequently transformed into CIA spectrum via Fourier transform. In our calculations, both bound and unbound states are properly accounted for, and the rigorous theory of lower-order spectral moments is addressed to check the accuracy of simulated profiles. Classically derived trajectory-based profiles are subject to two approximate desymmetrization procedures so that resulting profiles conform to the quantum principle of detailed balance. The simulated profiles are compared to laboratory measurements and employed for modeling Titan's spectra in the 50-500 cm$^{-1}$ range. Based on the desymmetrized simulated profiles, a new semi-empirical model for CH$_4-$N$_2$ CIA is proposed for modeling Titan's infrared spectra. Synthetic spectra derived using this model yield an excellent agreement with the data recorded by the Composite Infrared Spectrometer (CIRS) aboard the Cassini spacecraft at low and high emission angles. △ Less

Submitted 3 November, 2021; originally announced November 2021.

Comments: 18 pages, 11 figures, accepted for publication in ApJS . The semiempirical CH4-N2 profiles tabulated in the HITRAN CIA format are available on the ApJS website or upon reasonable request

Atomic structure of the single step and dynamics of Sn adatoms on the Si(111)-\sqrt{3}\times\sqrt{3}-Sn surface

Authors: R. A. Zhachuk, D. I. Rogilo, A. S. Petrov, D. V. Sheglov, A. V. Latyshev, S. Colonna, F. Ronci

Abstract: The atomic structure of well-ordered single steps on the Si(111)-\sqrt{3}\times\sqrt{3}-Sn surface and the dynamics of Sn adatoms in the vicinity of these steps was studied. The work was performed using low-temperature scanning tunneling microscopy (STM) and ab initio calculations based on the density-functional theory. The STM tip was used to record the tunneling current versus time on top of osc… ▽ More The atomic structure of well-ordered single steps on the Si(111)-\sqrt{3}\times\sqrt{3}-Sn surface and the dynamics of Sn adatoms in the vicinity of these steps was studied. The work was performed using low-temperature scanning tunneling microscopy (STM) and ab initio calculations based on the density-functional theory. The STM tip was used to record the tunneling current versus time on top of oscillating adatoms, keeping the feedback loop turned off. The dynamics of adatoms, detected as the telegraph noise present in the tunneling current, was registered near the steps at 80 K. The atomic structure model of the single steps consisting of Sn atomic chains along the steps was developed. This structure leads to the formation of potential double wells near the steps acting as traps for Sn atoms and explains the fluctuating current recorded in these areas. △ Less

Submitted 28 September, 2021; originally announced September 2021.

Comments: 9 pages, 8 figures

Journal ref: Physical Review B 104, 125437 (2021)

IEEE BigData 2021 Cup: Soft Sensing at Scale

Authors: Sergei Petrov, Chao Zhang, Jaswanth Yella, Yu Huang, Xiaoye Qian, Sthitie Bom

Abstract: IEEE BigData 2021 Cup: Soft Sensing at Scale is a data mining competition organized by Seagate Technology, in association with the IEEE BigData 2021 conference. The scope of this challenge is to tackle the task of classifying soft sensing data with machine learning techniques. In this paper we go into the details of the challenge and describe the data set provided to participants. We define the me… ▽ More IEEE BigData 2021 Cup: Soft Sensing at Scale is a data mining competition organized by Seagate Technology, in association with the IEEE BigData 2021 conference. The scope of this challenge is to tackle the task of classifying soft sensing data with machine learning techniques. In this paper we go into the details of the challenge and describe the data set provided to participants. We define the metrics of interest, baseline models, and describe approaches we found meaningful which may be a good starting point for further analysis. We discuss the results obtained with our approaches and give insights on what potential challenges participants may run into. Students, researchers, and anyone interested in working on a major industrial problem are welcome to participate in the challenge! △ Less

Submitted 7 September, 2021; originally announced September 2021.

Comments: 4 pages, 4 figures, for IEEE Big Data Cup challenge 2021

MSC Class: 68T01

Status of the Yakutsk air shower array and future plans

Authors: A. K. Alekseev, E. A. Atlasov, N. G. Bolotnikov, A. V. Bosikov, N. A. Dyachkovskiy, N. S. Gerasimova, A. V. Glushkov, A. A. Ivanov, O. N. Ivanov, D. N. Kardashevsky, I. A. Kellarev, S. P. Knurenko, A. D. Krasilnikov, A. N. Krivenkov, I. V. Ksenofontov, L. T. Ksenofontov, K. G. Lebedev, S. V. Matarkin, V. P. Mokhnachevskaya, E. V. Nikolaeva, N. I. Neustroev, I. S. Petrov, N. D. Platonov, A. S. Proshutinsky, A. V. Sabourov , et al. (4 additional authors not shown)

Abstract: The Yakutsk Extensive Air Shower Array has been continuously operating for more than 50 years (since 1970) and up until recently it has been one of world's largest ground-based instruments aimed at studying the properties of cosmic rays in the ultra-high energy domain. In this report we discuss results recently obtained at the array - on cosmic rays energy spectrum, mass composition and directiona… ▽ More The Yakutsk Extensive Air Shower Array has been continuously operating for more than 50 years (since 1970) and up until recently it has been one of world's largest ground-based instruments aimed at studying the properties of cosmic rays in the ultra-high energy domain. In this report we discuss results recently obtained at the array - on cosmic rays energy spectrum, mass composition and directional anisotropy - and how they fit into the world data. Special attention is paid to the measurements of muonic component of extensive air showers. Theoretical results of particle acceleration at shocks are also briefly reviewed. Future scientific and engineering plans on the array modernization are discussed. △ Less

Submitted 15 July, 2021; originally announced July 2021.

Comments: 24 pages, 7 figures. Accepted for publication in Physics of Atomic Nuclei. For color figures see ISCRA-2021 contribution: https://indico.nevod.mephi.ru/event/6/contributions/161/

Journal ref: Physics of Atomic Nuclei 2021 V.84 P.893

Full reconstruction of acoustic wavefields by means of pointwise measurements

Authors: Denis V. Makarov, Pavel S. Petrov

Abstract: Sound propagation in the ocean is considered. We demonstrate a novel algorithm for full wavefield reconstruction using pointwise measurements by means of a vertical array. The algorithm is based on the so-called discrete variable representation and can be implemented both for tonal and pulse signals. It is shown that the algorithm is robust against array distortions and ambient noise of moderate a… ▽ More Sound propagation in the ocean is considered. We demonstrate a novel algorithm for full wavefield reconstruction using pointwise measurements by means of a vertical array. The algorithm is based on the so-called discrete variable representation and can be implemented both for tonal and pulse signals. It is shown that the algorithm is robust against array distortions and ambient noise of moderate amplitude. Efficiency of reconstruction is verified by means of numerical simulation with a model of a shallow-sea waveguide. It is found that the effect of bottom sound attenuation enables accurate reconstruction with arrays having relatively low density of hydrophones. △ Less

Submitted 10 March, 2021; originally announced March 2021.

Comments: 20 pages, 10 figures

Enabling four-dimensional conformal hybrid meshing with cubic pyramids

Authors: Miroslav S. Petrov, Todor D. Todorov, Gage S. Walters, David M. Williams, Freddie D. Witherden

Abstract: The main purpose of this article is to develop a novel refinement strategy for four-dimensional hybrid meshes based on cubic pyramids. This optimal refinement strategy subdivides a given cubic pyramid into a conforming set of congruent cubic pyramids and invariant bipentatopes. The theoretical properties of the refinement strategy are rigorously analyzed and evaluated. In addition, a new class of… ▽ More The main purpose of this article is to develop a novel refinement strategy for four-dimensional hybrid meshes based on cubic pyramids. This optimal refinement strategy subdivides a given cubic pyramid into a conforming set of congruent cubic pyramids and invariant bipentatopes. The theoretical properties of the refinement strategy are rigorously analyzed and evaluated. In addition, a new class of fully symmetric quadrature rules with positive weights are generated for the cubic pyramid. These rules are capable of exactly integrating polynomials with degrees up to 12. Their effectiveness is successfully demonstrated on polynomial and transcendental functions. Broadly speaking, the refinement strategy and quadrature rules in this paper open new avenues for four-dimensional hybrid meshing, and space-time finite element methods. △ Less

Submitted 14 January, 2021; originally announced January 2021.

Comments: 30 pages, 15 figures

MSC Class: 52B11; 65N30; 65N50; 65D30; 65D32

Higher-order effective interactions for bosons near a two-body zero crossing

Authors: A. Pricoupenko, D. S. Petrov

Abstract: We develop the perturbation theory for bosons interacting via a two-body potential $V$ of vanishing mean value. We find that the leading nonpairwise contribution to the energy emerges in the third order in $V$ and represents an effective three-body interaction, the sign of which in most cases (although not in general) is anticorrelated with the sign of the long-range tail of $V$. Explicit results… ▽ More We develop the perturbation theory for bosons interacting via a two-body potential $V$ of vanishing mean value. We find that the leading nonpairwise contribution to the energy emerges in the third order in $V$ and represents an effective three-body interaction, the sign of which in most cases (although not in general) is anticorrelated with the sign of the long-range tail of $V$. Explicit results are obtained for a few particular two-body interaction potentials and we perform a detailed perturbative analysis of tilted dipoles in quasi-low-dimensional geometries. △ Less

Submitted 18 December, 2020; originally announced December 2020.

Comments: 11 pages, 1 figure

Journal ref: Phys. Rev. A 103, 033326 (2021)

Measuring and Reducing Gendered Correlations in Pre-trained Models

Authors: Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, Slav Petrov

Abstract: Pre-trained models have revolutionized natural language understanding. However, researchers have found they can encode artifacts undesired in many applications, such as professions correlating with one gender more than another. We explore such gendered correlations as a case study for how to address unintended correlations in pre-trained models. We define metrics and reveal that it is possible for… ▽ More Pre-trained models have revolutionized natural language understanding. However, researchers have found they can encode artifacts undesired in many applications, such as professions correlating with one gender more than another. We explore such gendered correlations as a case study for how to address unintended correlations in pre-trained models. We define metrics and reveal that it is possible for models with similar accuracy to encode correlations at very different rates. We show how measured correlations can be reduced with general-purpose techniques, and highlight the trade offs different strategies have. With these results, we make recommendations for training robust models: (1) carefully evaluate unintended correlations, (2) be mindful of seemingly innocuous configuration differences, and (3) focus on general mitigations. △ Less

Submitted 2 March, 2021; v1 submitted 12 October, 2020; originally announced October 2020.

Mixed bubbles in Bose-Bose mixtures

Authors: P. Naidon, D. S. Petrov

Abstract: Repulsive Bose-Bose mixtures are known to either mix or phase-separate into pure components. Here we predict a mixed-bubble regime in which bubbles of the mixed phase coexist with a pure phase of one of the components. This is a beyond-mean-field effect which occurs for unequal masses or unequal intraspecies coupling constants and is due to a competition between the mean-field term, quadratic in d… ▽ More Repulsive Bose-Bose mixtures are known to either mix or phase-separate into pure components. Here we predict a mixed-bubble regime in which bubbles of the mixed phase coexist with a pure phase of one of the components. This is a beyond-mean-field effect which occurs for unequal masses or unequal intraspecies coupling constants and is due to a competition between the mean-field term, quadratic in densities, and a nonquadratic beyond-mean-field correction. We find parameters of the mixed-bubble regime in all dimensions and discuss implications for current experiments. △ Less

Submitted 13 August, 2020; originally announced August 2020.

Comments: 2 figures; 9 pages

Journal ref: Phys. Rev. Lett. 126, 115301 (2021)

Singularity-enhanced terahertz detection in high-mobility field-effect transistors

Authors: Michail Khavronin, Aleksandr S. Petrov, Aleksandr E. Kazantsev, Egor I. Nikulin, Denis A. Bandurin

Abstract: Detectors of high-frequency radiation based on high-electron-mobility transistors benefit from low noise, room-temperature operation, and the possibility to perform radiation spectroscopy using gate-tunable plasmon resonance. Despite successful proof-of-concept demonstrations, the responsivity of transistor-based detectors of THz radiation, at present, remains fairly poor. To resolve this problem,… ▽ More Detectors of high-frequency radiation based on high-electron-mobility transistors benefit from low noise, room-temperature operation, and the possibility to perform radiation spectroscopy using gate-tunable plasmon resonance. Despite successful proof-of-concept demonstrations, the responsivity of transistor-based detectors of THz radiation, at present, remains fairly poor. To resolve this problem, we propose a class of devices supporting singular plasmon modes, i.e. modes with strong electric fields near keen electrodes. A large plasmon-enhanced electric field results in amplified non-linearities, and thus efficient ac-to-dc conversion. We analyze sub-terahertz detectors based on a two-dimensional electron system (2DES) in the Corbino geometry as a prototypical and exactly solvable model and show that the responsivity scales as $1/r_0^{2}$ with the radius of the inner contact $r_0$. This enables responsivities exceeding 10 kV/W at sub-THz frequencies for nanometer-scale contacts readily accessible by modern nanofabrication techniques. △ Less

Submitted 10 May, 2020; originally announced May 2020.

Comments: 6 paged, 5 figures

Journal ref: Phys. Rev. Applied 13, 064072 (2020)

Thresholdless excitation of edge plasmons by transverse current

Authors: Aleksandr S. Petrov, Dmitry Svintsov

Abstract: We theoretically demonstrate that dc electron flow across the junction of two-dimensional electron systems leads to excitation of edge magnetoplasmons. The threshold current for such plasmon excitation does not depend on contact effects and approaches zero for ballistic electron systems, which makes a strong distinction from the well-known Dyakonov-Shur and Cerenkov-type instabilities. We estimate… ▽ More We theoretically demonstrate that dc electron flow across the junction of two-dimensional electron systems leads to excitation of edge magnetoplasmons. The threshold current for such plasmon excitation does not depend on contact effects and approaches zero for ballistic electron systems, which makes a strong distinction from the well-known Dyakonov-Shur and Cerenkov-type instabilities. We estimate the competing plasmon energy gain from dc current and loss due to electron scattering. We show that plasmon self excitation is feasible in GaAs-based heterostructures at $T\lesssim 200$ K and magnetic fields $B \lesssim 10$ T. △ Less

Submitted 8 May, 2020; originally announced May 2020.

Comments: 6 pages, 3 figures

Journal ref: Phys. Rev. B 102, 121402 (2020)

Collective Excitations of a One-Dimensional Quantum Droplet

Authors: Marek Tylutki, Grigori E. Astrakharchik, Boris A. Malomed, Dmitry S. Petrov

Abstract: We calculate the excitation spectrum of a one-dimensional self-bound quantum droplet in a two-component bosonic mixture described by the Gross-Pitaevskii equation (GPE) with cubic and quadratic nonlinearities. The cubic term originates from the mean-field energy of the mixture proportional to the effective coupling constant $δg$, whereas the quadratic nonlinearity corresponds to the attractive bey… ▽ More We calculate the excitation spectrum of a one-dimensional self-bound quantum droplet in a two-component bosonic mixture described by the Gross-Pitaevskii equation (GPE) with cubic and quadratic nonlinearities. The cubic term originates from the mean-field energy of the mixture proportional to the effective coupling constant $δg$, whereas the quadratic nonlinearity corresponds to the attractive beyond-mean-field contribution. The droplet properties are governed by a control parameter $γ\propto δg N^{2/3}$, where $N$ is the particle number. For large $γ>0$ the droplet features the flat-top shape with the discrete part of its spectrum consisting of plane-wave Bogoliubov phonons propagating through the flat-density bulk and reflected by edges of the droplet. With decreasing $γ$ these modes cross into the continuum, sequentially crossing the particle-emission threshold at specific critical values. A notable exception is the breathing mode which we find to be always bound. The balance point $γ= 0$ provides implementation of a system governed by the GPE with an unusual quadratic nonlinearity. This case is characterized by the ratio of the breathing-mode frequency to the particle-emission threshold equal to 0.8904. As $γ$ tends to $-\infty$ this ratio tends to 1 and the droplet transforms into the soliton solution of the integrable cubic GPE. △ Less

Submitted 22 May, 2020; v1 submitted 12 March, 2020; originally announced March 2020.

Comments: 5 pages, 1 figure

Journal ref: Phys. Rev. A 101, 051601(R) (2020)

Few-body bound states of two-dimensional bosons

Authors: G. Guijarro, G. E. Astrakharchik, J. Boronat, B. Bazak, D. S. Petrov

Abstract: We study clusters of the type A$_N$B$_M$ with $N\leq M\leq 3$ in a two-dimensional mixture of A and B bosons, with attractive AB and equally repulsive AA and BB interactions. In order to check universal aspects of the problem, we choose two very different models: dipolar bosons in a bilayer geometry and particles interacting via separable Gaussian potentials. We find that all the considered cluste… ▽ More We study clusters of the type A$_N$B$_M$ with $N\leq M\leq 3$ in a two-dimensional mixture of A and B bosons, with attractive AB and equally repulsive AA and BB interactions. In order to check universal aspects of the problem, we choose two very different models: dipolar bosons in a bilayer geometry and particles interacting via separable Gaussian potentials. We find that all the considered clusters are bound and that their energies are universal functions of the scattering lengths $a_{AB}$ and $a_{AA}=a_{BB}$, for sufficiently large attraction-to-repulsion ratios $a_{AB}/a_{BB}$. When $a_{AB}/a_{BB}$ decreases below $\approx 10$, the dimer-dimer interaction changes from attractive to repulsive and the population-balanced AABB and AAABBB clusters break into AB dimers. Calculating the AAABBB hexamer energy just below this threshold, we find an effective three-dimer repulsion which may have important implications for the many-body problem, particularly for observing liquid and supersolid states of dipolar dimers in the bilayer geometry. The population-imbalanced ABB trimer, ABBB tetramer, and AABBB pentamer remain bound beyond the dimer-dimer threshold. In the dipolar model, they break up at $a_{AB}\approx 2 a_{BB}$ where the atom-dimer interaction switches to repulsion. △ Less

Submitted 7 April, 2020; v1 submitted 5 November, 2019; originally announced November 2019.

Comments: 3 figures

Journal ref: Phys. Rev. A 101, 041602 (2020)

Three-body interaction near a narrow two-body zero crossing

Authors: A. Pricoupenko, D. S. Petrov

Abstract: We calculate the effective three-body force for bosons interacting with each other by a two-body potential tuned to a narrow zero crossing in any dimension. We use the standard two-channel model parametrized by the background atom-atom interaction strength, the amplitude of the open-channel to closed-channel coupling, and the atom-dimer interaction strength. The three-body force originates from th… ▽ More We calculate the effective three-body force for bosons interacting with each other by a two-body potential tuned to a narrow zero crossing in any dimension. We use the standard two-channel model parametrized by the background atom-atom interaction strength, the amplitude of the open-channel to closed-channel coupling, and the atom-dimer interaction strength. The three-body force originates from the atom-dimer interaction, but it can be dramatically enhanced for narrow crossings, i.e., for small atom-dimer conversion amplitudes. This effect can be used to stabilize quasi-two-dimensional dipolar atoms and molecules. △ Less

Submitted 14 October, 2019; v1 submitted 4 July, 2019; originally announced July 2019.

Comments: published version

Journal ref: Phys. Rev. A 100, 042707 (2019)

Perturbation theory for two-dimensional hydrodynamic plasmons

Authors: Aleksandr S. Petrov, Dmitry Svintsov

Abstract: Perturbation theory is an indispensable tool in quantum mechanics and electrodynamics that handles weak effects on particle motion or fields. However, its extension to plasmons involving complex motion of {\it both} particles and fields remained challenging. We show that this challenge can be mastered if electron motion obeys the laws of hydrodynamics, as recently confirmed in experiments with ult… ▽ More Perturbation theory is an indispensable tool in quantum mechanics and electrodynamics that handles weak effects on particle motion or fields. However, its extension to plasmons involving complex motion of {\it both} particles and fields remained challenging. We show that this challenge can be mastered if electron motion obeys the laws of hydrodynamics, as recently confirmed in experiments with ultra-clean heterostructures. We present a unified approach to evaluate corrections to plasmon spectra induced by carrier drift, magnetic field, Berry curvature, scattering, and viscosity. As a first application, we study the stability of direct current in confined two-dimensional electron systems against self-excitation of plasmons. We show that arbitrarily weak current in the absence of dissipation is unstable provided the structure lacks mirror symmetry. As a second application, we indicate that in extended periodic systems -- plasmonic crystals -- carrier drift induces anomalous Doppler shift, which can be both below and higher than its value in uniform systems. Finally, we exactly evaluate the effect of Berry curvature on spectra of edge plasmons and demonstrate the non-reciprocity induced by anomalous velocity. △ Less

Submitted 10 March, 2019; originally announced March 2019.

Journal ref: Phys. Rev. B 99 195437 (2019)

Stable p-wave resonant two-dimensional Fermi-Bose dimers

Authors: Betzalel Bazak, Dmitry S. Petrov

Abstract: We consider two-dimensional weakly-bound heterospecies molecules formed in a Fermi-Bose mixture with attractive Fermi-Bose and repulsive Bose-Bose interactions. Bosonic exchanges lead to an intermolecular attraction, which can be controlled and tuned to a p-wave resonance. Such attractive fermionic molecules can be realized in quasi-two-dimensional ultracold isotopic or heteronuclear mixtures. We… ▽ More We consider two-dimensional weakly-bound heterospecies molecules formed in a Fermi-Bose mixture with attractive Fermi-Bose and repulsive Bose-Bose interactions. Bosonic exchanges lead to an intermolecular attraction, which can be controlled and tuned to a p-wave resonance. Such attractive fermionic molecules can be realized in quasi-two-dimensional ultracold isotopic or heteronuclear mixtures. We show that they are stable with respect to the recombination to deeply-bound molecular states and with respect to the formation of higher-order clusters (trimers, tetramers, etc.) △ Less

Submitted 29 December, 2018; v1 submitted 18 July, 2018; originally announced July 2018.

Journal ref: Phys. Rev. Lett. 121, 263001 (2018)

Dimensional crossover for the beyond-mean-field correction in Bose gases

Authors: Tobias Ilg, Jan Kumlin, Luis Santos, Dmitry S. Petrov, Hans Peter Büchler

Abstract: We present a detailed beyond-mean-field analysis of a weakly interacting Bose gas in the crossover from three to low dimensions. We find an analytical solution for the energy and provide a clear qualitative picture of the crossover in the case of a box potential with periodic boundary conditions. We show that the leading contribution of the confinement-induced resonance is of beyond-mean-field ord… ▽ More We present a detailed beyond-mean-field analysis of a weakly interacting Bose gas in the crossover from three to low dimensions. We find an analytical solution for the energy and provide a clear qualitative picture of the crossover in the case of a box potential with periodic boundary conditions. We show that the leading contribution of the confinement-induced resonance is of beyond-mean-field order and calculate the leading corrections in the three- and low-dimensional limits. We also characterize the crossover for harmonic potentials in a model system with particularly chosen short- and long-range interactions and show the limitations of the local-density approximation. Our analysis is applicable to Bose-Bose mixtures and gives a starting point for developing the beyond-mean-field theory in inhomogeneous systems with long-range interactions such as dipolar particles or Rydberg-dressed atoms. △ Less

Submitted 28 November, 2018; v1 submitted 5 June, 2018; originally announced June 2018.

Comments: 5 pages and supplemental material, revised treatment of the harmonic confinement

Journal ref: Phys. Rev. A 98, 051604 (2018)

Reaction kinetics of ultracold molecule-molecule collisions

Authors: Daniel K. Hoffmann, Thomas Paintner, Wolfgang Limmer, Dmitry S. Petrov, Johannes Hecker Denschlag

Abstract: Studying chemical reactions on a state-to-state level tests and improves our fundamental understanding of chemical processes. For such investigations it is convenient to make use of ultracold atomic and molecular reactants as they can be prepared in well defined internal and external quantum states$^{1-4}$. In general, even cold reactions have many possible final product states$^{5-15}$ and reacti… ▽ More Studying chemical reactions on a state-to-state level tests and improves our fundamental understanding of chemical processes. For such investigations it is convenient to make use of ultracold atomic and molecular reactants as they can be prepared in well defined internal and external quantum states$^{1-4}$. In general, even cold reactions have many possible final product states$^{5-15}$ and reaction channels are therefore hard to track individually$^{16}$. In special cases, however, only a single reaction channel is essentially participating, as observed e.g. in the recombination of two atoms forming a Feshbach molecule$^{17-19}$ or in atom-Feshbach molecule exchange reactions$^{20,21}$. Here, we investigate a single-channel reaction of two Li$_2$-Feshbach molecules where one of the molecules dissociates into two atoms $2\mathrm{AB}\Rightarrow \mathrm{AB}+\mathrm{A}+\mathrm{B}$. The process is a prototype for a class of four-body collisions where two reactants produce three product particles. We measure the collisional dissociation rate constant of this process as a function of collision energy/ temperature and scattering length. We confirm an Arrhenius-law dependence on the collision energy, an $a^4$ power-law dependence on the scattering length $a$ and determine a universal four body reaction constant. △ Less

Submitted 21 May, 2018; originally announced May 2018.

Comments: 17 pages, 5 figures

One-dimensional three-boson problem with two- and three-body interactions

Authors: G. Guijarro, A. Pricoupenko, G. E. Astrakharchik, J. Boronat, D. S. Petrov

Abstract: We solve the three-boson problem with contact two- and three-body interactions in one dimension and analytically calculate the ground and excited trimer-state energies. Then, by using the diffusion Monte Carlo technique we calculate the binding energy of three dimers formed in a one-dimensional Bose-Bose or Fermi-Bose mixture with attractive interspecies and repulsive intraspecies interactions. Co… ▽ More We solve the three-boson problem with contact two- and three-body interactions in one dimension and analytically calculate the ground and excited trimer-state energies. Then, by using the diffusion Monte Carlo technique we calculate the binding energy of three dimers formed in a one-dimensional Bose-Bose or Fermi-Bose mixture with attractive interspecies and repulsive intraspecies interactions. Combining these results with our three-body analytics we extract the three-dimer scattering length close to the dimer-dimer zero crossing. In both considered cases the three-dimer interaction turns out to be repulsive. Our results constitute a concrete proposal for obtaining a one-dimensional gas with a pure three-body repulsion. △ Less

Submitted 18 June, 2018; v1 submitted 22 March, 2018; originally announced March 2018.

Comments: published version

Journal ref: Phys. Rev. A 97, 061605 (2018)

Asymmetry-driven plasmon instabilities in confined hydrodynamic electron flows

Authors: Aleksandr S. Petrov, Dmitry Svintsov

Abstract: Direct current in confined two-dimensional (2d) electron systems can become unstable with respect to the excitation of plasmons. Numerous experiments and simulations hint that structural asymmetry somehow promotes plasmon generation, but a constitutive relation between asymmetry and instability has been missing. We provide such relation in the present paper and show that bounded perfect 2d electro… ▽ More Direct current in confined two-dimensional (2d) electron systems can become unstable with respect to the excitation of plasmons. Numerous experiments and simulations hint that structural asymmetry somehow promotes plasmon generation, but a constitutive relation between asymmetry and instability has been missing. We provide such relation in the present paper and show that bounded perfect 2d electron fluids in asymmetric structures are unstable under arbitrarily weak drive currents. To this end, we develop a perturbation theory for hydrodynamic plasmons and evaluate corrections to their eigenfrequency induced by carrier drift, scattering, and viscosity. We show that plasmon gain continuously increases with degree of plasmon mode asymmetry until it surrenders to viscous dissipation that also benefits from asymmetry. The developed formalism allows us to put a lower bound on the instability threshold current, which corresponds to the Reynolds number $R_{\min} = 2\sqrt{3}$ for one-dimensional plasmons in 2d channel under constant voltage bias. △ Less

Submitted 12 February, 2018; originally announced February 2018.

Comments: 9 pages, 4 figures

Journal ref: Phys. Rev. B 99, 195437 (2019)

Dimer-dimer zero crossing and dilute dimerized liquid in a one-dimensional mixture

Authors: A. Pricoupenko, D. S. Petrov

Abstract: We consider the system of dimers formed in a one-dimensional mass-balanced Bose-Bose mixture of species $σ=\uparrow$, $\downarrow$ with attractive interspecies and repulsive intraspecies contact interactions. In the plane parametrized by the ratios of the coupling constants $g_{\uparrow\uparrow}/|g_{\uparrow\downarrow}|$ and $g_{\downarrow\downarrow}/|g_{\uparrow\downarrow}|$ we trace out the curv… ▽ More We consider the system of dimers formed in a one-dimensional mass-balanced Bose-Bose mixture of species $σ=\uparrow$, $\downarrow$ with attractive interspecies and repulsive intraspecies contact interactions. In the plane parametrized by the ratios of the coupling constants $g_{\uparrow\uparrow}/|g_{\uparrow\downarrow}|$ and $g_{\downarrow\downarrow}/|g_{\uparrow\downarrow}|$ we trace out the curve where the dimer-dimer interaction switches from attractive to repulsive. We find this curve to be significantly (by more than a factor of 2) shifted towards larger $g_{σσ}$ (or smaller $|g_{\uparrow\downarrow}|$) compared to the mean-field stability boundary $g_{\uparrow\uparrow}g_{\downarrow\downarrow}=g_{\uparrow\downarrow}^2$. For a weak dimer-dimer attraction we predict a dilute dimerized liquid phase stabilized against collapse by a repulsive three-dimer force. △ Less

Submitted 18 June, 2018; v1 submitted 6 February, 2018; originally announced February 2018.

Comments: published version

Journal ref: Phys. Rev. A 97, 063616 (2018)

Energy of N two-dimensional bosons with zero-range interactions

Authors: Betzalel Bazak, Dmitry S. Petrov

Abstract: We derive an integral equation describing $N$ two-dimensional bosons with zero-range interactions and solve it for the ground state energy $B_N$ by applying a stochastic diffusion Monte Carlo scheme for up to 26 particles. We confirm and go beyond the scaling $B_N\propto 8.567^N$ predicted by Hammer and Son [Phys. Rev. Lett. {\bf 93}, 250408 (2004)] in the large-$N$ limit. We derive an integral equation describing $N$ two-dimensional bosons with zero-range interactions and solve it for the ground state energy $B_N$ by applying a stochastic diffusion Monte Carlo scheme for up to 26 particles. We confirm and go beyond the scaling $B_N\propto 8.567^N$ predicted by Hammer and Son [Phys. Rev. Lett. {\bf 93}, 250408 (2004)] in the large-$N$ limit. △ Less

Submitted 20 February, 2018; v1 submitted 7 November, 2017; originally announced November 2017.

Journal ref: New J. Phys. 20, 023045 (2018)

Natural Language Processing with Small Feed-Forward Networks

Authors: Jan A. Botha, Emily Pitler, Ji Ma, Anton Bakalov, Alex Salcianu, David Weiss, Ryan McDonald, Slav Petrov

Abstract: We show that small and shallow feed-forward neural networks can achieve near state-of-the-art results on a range of unstructured and structured language processing tasks while being considerably cheaper in memory and computational requirements than deep recurrent models. Motivated by resource-constrained environments like mobile phones, we showcase simple techniques for obtaining such small neural… ▽ More We show that small and shallow feed-forward neural networks can achieve near state-of-the-art results on a range of unstructured and structured language processing tasks while being considerably cheaper in memory and computational requirements than deep recurrent models. Motivated by resource-constrained environments like mobile phones, we showcase simple techniques for obtaining such small neural network models, and investigate different tradeoffs when deciding how to allocate a small memory budget. △ Less

Submitted 1 August, 2017; originally announced August 2017.

Comments: EMNLP 2017 short paper

MSC Class: 68T50 ACM Class: I.2.7

Radio emission of air showers with extremely high energy measured by the Yakutsk radio array

Authors: S. P. Knurenko, Z. E. Petrov, I. S. Petrov

Abstract: The Yakutsk Array is designed to study cosmic rays at energy 10$^{15}$-10$^{20}$ eV. It consists several independent arrays that register charged particles, muons with energy E$\geq$1 GeV, Cherenkov light and radio emission. The paper presents a technical description of the Yakutsk Radio Array and some preliminary results obtained from measurements of radio emission at 30-35 MHz frequency induced… ▽ More The Yakutsk Array is designed to study cosmic rays at energy 10$^{15}$-10$^{20}$ eV. It consists several independent arrays that register charged particles, muons with energy E$\geq$1 GeV, Cherenkov light and radio emission. The paper presents a technical description of the Yakutsk Radio Array and some preliminary results obtained from measurements of radio emission at 30-35 MHz frequency induced by air shower particles with energy $\varepsilon$ $\geq$ 1$\cdot$10$^{17}$ eV. The data obtained at the Yakutsk array in 1986-1989 (first set of measurements) and 2009-2014 (new set of measurements). Based on the obtained results we determined: Lateral distribution function (LDF) of air showers radio emission with energy $\geq$ 10$^{17}$ eV. Radio emission amplitude empirical connection with air shower energy. Determination of depth of maximum by the ratio of amplitude at different distances from the shower axis. For the first time, at the Yakutsk array, radio emission from the air shower with energy $>$ 10$^{19}$ eV was registered including the shower with the highest energy ever registered at the Yakutsk array with energy $\sim$2 $\cdot$ 10$^{20}$ eV. △ Less

Submitted 3 May, 2017; originally announced May 2017.

Comments: 11 pages, 20 figures, 2 tables, accepted for publication in NIM A. arXiv admin note: text overlap with arXiv:1609.03273

Journal ref: NIMA, vol. 866, p. 230-241 (2017)

Two- and three-body problem with Floquet-driven zero-range interactions

Authors: A. G. Sykes, H. Landa, D. S. Petrov

Abstract: We study the two-body scattering problem in the zero-range approximation with a sinusoidally driven scattering length and calculate the relation between the mean value and amplitude of the drive for which the effective scattering amplitude is resonantly enhanced. In this manner we arrive at a family of curves along which the effective scattering length diverges but the nature of the corresponding… ▽ More We study the two-body scattering problem in the zero-range approximation with a sinusoidally driven scattering length and calculate the relation between the mean value and amplitude of the drive for which the effective scattering amplitude is resonantly enhanced. In this manner we arrive at a family of curves along which the effective scattering length diverges but the nature of the corresponding Floquet-induced resonance changes from narrow to wide. Remarkably, on these curves the driving does not induce heating. In order to study the effect of these resonances on the three-body problem we consider one light and two heavy particles with driven heavy-light interaction in the Born-Oppenheimer approximation and find that the Floquet driving can be used to tune the three-body and inelasticity parameters. △ Less

Submitted 4 April, 2017; originally announced April 2017.

Comments: 14 pages, 7 figures

Journal ref: Phys. Rev. A 95, 062705 (2017)

SyntaxNet Models for the CoNLL 2017 Shared Task

Authors: Chris Alberti, Daniel Andor, Ivan Bogatyy, Michael Collins, Dan Gillick, Lingpeng Kong, Terry Koo, Ji Ma, Mark Omernick, Slav Petrov, Chayut Thanapirom, Zora Tung, David Weiss

Abstract: We describe a baseline dependency parsing system for the CoNLL2017 Shared Task. This system, which we call "ParseySaurus," uses the DRAGNN framework [Kong et al, 2017] to combine transition-based recurrent parsing and tagging with character-based word representations. On the v1.3 Universal Dependencies Treebanks, the new system outpeforms the publicly available, state-of-the-art "Parsey's Cousins"… ▽ More We describe a baseline dependency parsing system for the CoNLL2017 Shared Task. This system, which we call "ParseySaurus," uses the DRAGNN framework [Kong et al, 2017] to combine transition-based recurrent parsing and tagging with character-based word representations. On the v1.3 Universal Dependencies Treebanks, the new system outpeforms the publicly available, state-of-the-art "Parsey's Cousins" models by 3.47% absolute Labeled Accuracy Score (LAS) across 52 treebanks. △ Less

Submitted 15 March, 2017; originally announced March 2017.

Comments: Tech report

Masses of optical components and black holes in x-ray novae: the effects of components proximity

Authors: V. S. Petrov, E. A. Antokhina, A. M. Cherepashchuk

Abstract: An exact calculation of CaI $λ6439.075$A absorption profiles in the spectra of optical stars in low-mass X-ray binary systems with black holes (BH LMXBs) is carried out. We show that the approximation of a real Roche lobe filling star as disk with uniform local line profile and linear limb darkening law leads to overestimation of projected equatorial rotational velocity $V_{rot} \sin i$ and accord… ▽ More An exact calculation of CaI $λ6439.075$A absorption profiles in the spectra of optical stars in low-mass X-ray binary systems with black holes (BH LMXBs) is carried out. We show that the approximation of a real Roche lobe filling star as disk with uniform local line profile and linear limb darkening law leads to overestimation of projected equatorial rotational velocity $V_{rot} \sin i$ and accordingly, underestimation of mass ratio $q=M_x/M_v$. Refined value of $q$ does not affect the mass of a black hole, but the mass of an optical star has shrunk $\sim 1.5$ times. We present refined components masses in low-mass X-ray binaries with black holes. Companion masses in BH LMXBs are found in the mass range 0.1 - 1.6 $M_{\odot}$ with the peak at $M_v \simeq 0.35 M_{\odot}$. This finding poses additional problem for the standard evolutionary scenarios of BH LMXBs formation with a common envelope phase (CE). We also discuss the implications of these masses into the evolutionary history of the binary. △ Less

Submitted 15 February, 2017; originally announced February 2017.

Comments: 12 pages, 2 figures, 2 tables