Rogier Landman

Vocal communication in animals often involves taking turns vocalizing. In humans, turn-taking is a fundamental rule in conversation. Among non-human primates, the common marmoset is known to engage in antiphonal calling using phee calls and trill calls. Calls of the trill type are the most common, yet difficult to study, because they are not very loud and uttered in conditions when animals are in close proximity to one another. Here we recorded trill calls in captive pair-housed marmosets using… Show more

Vocal communication in animals often involves taking turns vocalizing. In humans, turn-taking is a fundamental rule in conversation. Among non-human primates, the common marmoset is known to engage in antiphonal calling using phee calls and trill calls. Calls of the trill type are the most common, yet difficult to study, because they are not very loud and uttered in conditions when animals are in close proximity to one another. Here we recorded trill calls in captive pair-housed marmosets using wearable microphones, while the animals were together with their partner or separated, but within trill call range. Trills were exchanged mainly with the partner and not with other animals in the room. Animals placed outside the home cage increased their trill call rate and uttered more trills in response to their partner compared to strangers. The fundamental frequency, F0, of trills increased when animals were placed outside the cage. Our results indicate that trill calls can be monitored using wearable audio equipment and that minor changes in social context affect trill call interactions and spectral properties of trill calls.

Turn-taking is a fundamental feature of human conversation. Vocal exchanges develop in infancy and universally converge towards a general rule of minimizing overlap and minimizing the time between turns. Temporal regulation of vocal interactions of contact calls can be observed in non-human primates as well. These include loud calls exchanged periodically between widely separated individuals, and more quiet, frequently uttered calls while in dense vegetation where there is risk of becoming separated. In the common marmoset (Callithrix jacchus), turn-taking is observed in phee and trill calls. It is not known if marmoset trill calls are primarily exchanged with specific individuals, and whether separation affects trill call rate, exchange and spectral properties. Show less

Mutation or disruption of the SH3 and ankyrin repeat domains 3 (SHANK3) gene represents a highly penetrant, monogenic risk factor for autism spectrum disorder, and is a cause of Phelan–McDermid syndrome. Recent advances in gene editing have enabled the creation of genetically engineered non-human-primate models, which might better approximate the behavioural and neural phenotypes of autism spectrum disorder than do rodent models, and may lead to more effective treatments. Here we report… Show more

Mutation or disruption of the SH3 and ankyrin repeat domains 3 (SHANK3) gene represents a highly penetrant, monogenic risk factor for autism spectrum disorder, and is a cause of Phelan–McDermid syndrome. Recent advances in gene editing have enabled the creation of genetically engineered non-human-primate models, which might better approximate the behavioural and neural phenotypes of autism spectrum disorder than do rodent models, and may lead to more effective treatments. Here we report CRISPR–Cas9-mediated generation of germline-transmissible mutations of SHANK3 in cynomolgus macaques (Macaca fascicularis) and their F1 offspring. Genotyping of somatic cells as well as brain biopsies confirmed mutations in the SHANK3 gene and reduced levels of SHANK3 protein in these macaques. Analysis of data from functional magnetic resonance imaging revealed altered local and global connectivity patterns that were indicative of circuit abnormalities. The founder mutants exhibited sleep disturbances, motor deficits and increased repetitive behaviours, as well as social and learning impairments. Together, these results parallel some aspects of the dysfunctions in the SHANK3 gene and circuits, as well as the behavioural phenotypes, that characterize autism spectrum disorder and Phelan–McDermid syndrome. SHANK3 encodes major scaffolding proteins at excitatory synapses, coordinates the recruitment of signalling molecules and creates scaffolds for appropriate alignment of glutamatergic neurotransmitter receptors, which promotes the development and maturation of excitatory synapses. Mutation of SHANK3 accounts for about 1% of idiopathic forms of autism spectrum disorder, and disruption of SHANK3 is a major cause of neurodevelopmental deficits in Phelan–McDermid syndrome. Patients with a SHANK3 gene mutation often exhibit a variety of comorbid traits, which include
global developmental delay, severe sleep disturbances, severe language delay and autism spectrum disorder. Show less

This paper introduces an end-to-end feedforward convolutional neural network that is able to reliably classify the source and type of animal calls in a noisy environment using two streams of audio data after being trained on a dataset of modest size and imperfect labels. The data consists of audio recordings from captive marmoset monkeys housed in pairs, with several other cages nearby. The network in this paper can classify both the call type and which animal made it with a single pass through… Show more

This paper introduces an end-to-end feedforward convolutional neural network that is able to reliably classify the source and type of animal calls in a noisy environment using two streams of audio data after being trained on a dataset of modest size and imperfect labels. The data consists of audio recordings from captive marmoset monkeys housed in pairs, with several other cages nearby. The network in this paper can classify both the call type and which animal made it with a single pass through a single network using raw spectrogram images as input. The network vastly increases data analysis capacity for researchers interested in studying marmoset vocalizations, and allows data collection in the home cage, in group housed animals. Show less

Various studies suggest that marmosets ( Callithrix jacchus) show behavior similar to that of humans in many aspects. Analyzing their calls would not only enable us to better understand these species but would also give insights into the evolution of human languages and vocal tract. This paper describes a technique to discover the patterns in marmoset vocalization in an unsupervised fashion. The proposed unsupervised clustering approach operates in two stages. Initially, voice activity… Show more

Various studies suggest that marmosets ( Callithrix jacchus) show behavior similar to that of humans in many aspects. Analyzing their calls would not only enable us to better understand these species but would also give insights into the evolution of human languages and vocal tract. This paper describes a technique to discover the patterns in marmoset vocalization in an unsupervised fashion. The proposed unsupervised clustering approach operates in two stages. Initially, voice activity detection (VAD) is applied to remove silences and non-voiced regions from the audio. This is followed by a group-delay based segmentation on the voiced regions to obtain smaller segments. In the second stage, a two-tier clustering is performed on the segments obtained. Individual hidden Markov models (HMMs) are built for each of the segments using a multiple frame size and multiple frame rate. The HMMs are then clustered until each cluster is made up of a large number of segments. Once all the clusters get enough number of segments, one Gaussian mixture model (GMM) is built for each of the clusters. These clusters are then merged using Kullback-Leibler (KL) divergence. The algorithm converges to the total number of distinct sounds in the audio, as evidenced by listening tests. Show less

Molecular genetic tools have had a profound impact on neuroscience, but until recently their application has largely been confined to a few model species, most notably mouse, zebrafish, Drosophila melanogaster and Caenorhabditis elegans. With the development of new genome engineering technologies such as CRISPR, it is becoming increasingly feasible to apply these molecular tools in a wider range of species, including nonhuman primates. This will lead to many opportunities for brain research… Show more

Molecular genetic tools have had a profound impact on neuroscience, but until recently their application has largely been confined to a few model species, most notably mouse, zebrafish, Drosophila melanogaster and Caenorhabditis elegans. With the development of new genome engineering technologies such as CRISPR, it is becoming increasingly feasible to apply these molecular tools in a wider range of species, including nonhuman primates. This will lead to many opportunities for brain research, but it will also pose challenges. Here we identify some of these opportunities and challenges in light of recent and foreseeable technological advances and offer some suggestions. Our main focus is on the creation of new primate disease models for understanding the pathological mechanisms of brain disorders and for developing new approaches to effective treatment. However, we also emphasize that primate genetic models have great potential to address many fundamental questions about brain function, providing an essential foundation for future progress in disease research.

Brain disorders are among the largest causes of disease burden worldwide, affecting millions of people and imposing enormous societal and economic costs. Many of these disorders are chronic and incurable conditions, for which existing treatment options are inadequate and in some cases almost completely ineffective. Yet despite the urgent clinical need, there has been little recent progress in the development of new treatments for most common brain disorders, and many currently prescribed drugs are based on decades-old science. This may seem surprising given the rapid rate of progress in fundamental neuroscience, but it has proven extraordinarily difficult to translate advances in basic science into the development of new and better clinical treatments. Show less

This paper describes a novel framework for automated marmoset vocalization detection and classification from within long audio streams recorded in a noisy animal room, where multiple marmosets are housed. To overcome the challenge of limited manually annotated data, we implemented a data augmentation method using only a small number of labeled vocalizations. The feature sets chosen have the desirable property of capturing characteristics of the signals that are useful in both identifying and… Show more

This paper describes a novel framework for automated marmoset vocalization detection and classification from within long audio streams recorded in a noisy animal room, where multiple marmosets are housed. To overcome the challenge of limited manually annotated data, we implemented a data augmentation method using only a small number of labeled vocalizations. The feature sets chosen have the desirable property of capturing characteristics of the signals that are useful in both identifying and distinguishing marmoset vocalizations. Unlike many previous methods, feature extraction, call detection, and call classification in our system are completely automated. The system maintains a good performance of 80% detection rate in data with high number of noise events and is able to obtain a classification error of 15%. Performance can be further improved with additional labeled training data. Because this extensible system is capable of identifying both positive and negative welfare indicators, it provides a powerful framework for non-human primate welfare monitoring as well as behavior assessment. Index Terms: Automated detection and classification, marmoset vocalization, primate behavioral analysis, primate clinical health and welfare monitoring, Teager energy operator. Show less

The common marmoset is emerging as an important transgenic model for improving the understanding of the underlying neurological basis of many brain disorders. Automated systems for quantitative monitoring of marmoset behaviors in naturalist settings over long period of time are needed to facilitate this process. This paper presents the preliminary work toward building a novel multimodal acquisition system for the automated marmoset behavior analysis in home cage. In addition to integrating… Show more

The common marmoset is emerging as an important transgenic model for improving the understanding of the underlying neurological basis of many brain disorders. Automated systems for quantitative monitoring of marmoset behaviors in naturalist settings over long period of time are needed to facilitate this process. This paper presents the preliminary work toward building a novel multimodal acquisition system for the automated marmoset behavior analysis in home cage. In addition to integrating commercial available devices such as Microsoft Kinect sensors and microphones of different characteristics, we also developed a wireless flexible neck collar with acoustic and non-acoustic sensors onboard for marmoset vocalization recording and caller identification. Our initial effort has been focused on the real-time synchronization of multiple sensor outputs, the engineering design of the wireless collar, and algorithms for global 3D position and local head movement from a Microsoft Kinect sensor. With limited preliminary data, we are able to estimate 3D trajectories of two marmosets with a RMSE of ~3.2 mm and track colored ear tufts with an accuracy of RMSE ~1.8 mm. A larger dataset is needed for a complete assessment and validation. Our system architecture is modular and flexible, and can be extended to include more sensors and devices if needed. Show less

In primates, visual stimuli with social and emotional content tend to attract attention. Attention might be captured through rapid, automatic, subcortical processing or guided by slower, more voluntary cortical processing. Here we examined whether irrelevant faces with varied emotional expressions interfere with a covert attention task in macaque monkeys. In the task, the monkeys monitored a target grating in the periphery for a subtle color change while ignoring distracters that included faces… Show more

In primates, visual stimuli with social and emotional content tend to attract attention. Attention might be captured through rapid, automatic, subcortical processing or guided by slower, more voluntary cortical processing. Here we examined whether irrelevant faces with varied emotional expressions interfere with a covert attention task in macaque monkeys. In the task, the monkeys monitored a target grating in the periphery for a subtle color change while ignoring distracters that included faces appearing elsewhere on the screen. The onset time of distracter faces before the target change, as well as their spatial proximity to the target, was varied from trial to trial. The presence of faces, especially faces with emotional expressions interfered with the task, indicating a competition for attentional resources between the task and the face stimuli. However, this interference was significant only when faces were presented for greater than 200 ms. Emotional faces also affected saccade velocity and reduced pupillary reflex. Our results indicate that the attraction of attention by emotional faces in the monkey takes a considerable amount of processing time, possibly involving cortical-subcortical interactions. Intranasal application of the hormone oxytocin ameliorated the interfering effects of faces. Together these results provide evidence for slow modulation of attention by emotional distracters, which likely involves oxytocinergic brain circuits.

The viewing of faces with an emotional expression affects emotion circuits in the brain, even when they are not directly attended. This has led to a debate about whether faces attract attention automatically. We tested the influence of emotional faces as irrelevant distracters in an attention task in monkeys. We conclude that, in monkeys, emotional distractors do attract attention away from other tasks but not instantly. Administration of the hormone oxytocin reduced the effect. Show less

Attention to a stimulus enhances both neuronal responses and gamma frequency synchrony in visual area V4, both of which should increase the impact of attended information on downstream neurons. To determine whether gamma synchrony is common throughout the ventral stream, we recorded from neurons in the superficial and deep layers of V1, V2, and V4 in two rhesus monkeys. We found an unexpected striking difference in gamma synchrony in the superficial vs. deep layers. In all three areas… Show more

Attention to a stimulus enhances both neuronal responses and gamma frequency synchrony in visual area V4, both of which should increase the impact of attended information on downstream neurons. To determine whether gamma synchrony is common throughout the ventral stream, we recorded from neurons in the superficial and deep layers of V1, V2, and V4 in two rhesus monkeys. We found an unexpected striking difference in gamma synchrony in the superficial vs. deep layers. In all three areas, spike-field coherence in the gamma (40–60 Hz) frequency range was largely confined to the superficial layers, whereas the deep layers showed maximal coherence at low frequencies (6–16 Hz), which included the alpha range. In the superficial layers of V2 and V4, gamma synchrony was enhanced by attention, whereas in the deep layers, alpha synchrony was reduced by attention. Unlike these major differences in synchrony, attentional effects on firing rates and noise correlation did not differ substantially between the superficial and deep layers. The results suggest that synchrony plays very different roles in feedback and feedforward projections.

Anatomical and physiological studies have characterized the afferent inputs to and efferent inputs from neurons in different layers of visual cortical areas. However, physiological distinctions across layers, such as synchronous interactions, have not been fully identified. We first came across laminar differences in synchrony serendipitously. Gamma-band synchrony, measured either by spike-field or spike-spike interactions across multiple electrodes, is a prominent feature in visual cortex, and several studies have shown that attention enhances gamma-band synchrony in area V4. Show less

The visual processing of behaviorally relevant stimuli is enhanced through top-down attentional feedback. One possibility is that feedback targets early visual areas first and the attentional enhancement builds up at progressively later stages of the visual hierarchy. An alternative possibility is that the feedback targets the higher-order areas first and the attentional effects are communicated “backward” to early visual areas. Here, we compared the magnitude and latency of attentional… Show more

The visual processing of behaviorally relevant stimuli is enhanced through top-down attentional feedback. One possibility is that feedback targets early visual areas first and the attentional enhancement builds up at progressively later stages of the visual hierarchy. An alternative possibility is that the feedback targets the higher-order areas first and the attentional effects are communicated “backward” to early visual areas. Here, we compared the magnitude and latency of attentional enhancement of firing rates in V1, V2, and V4 in the same animals performing the same task. We found a reverse order of attentional effects, such that attentional enhancement was larger and earlier in V4 and smaller and later in V1, with intermediate results in V2. These results suggest that attentional mechanisms operate via feedback from higher-order areas to lower-order ones.

Neurophysiologic and brain imaging studies in monkeys and humans have shown that attended stimuli evoke larger responses in visual cortex than unattended distracters (1–6), giving attended stimuli a competitive advantage for representation in the cortex (7). These top-down attentional effects are thought to be mediated in part by feedback from prefrontal and posterior parietal cortex (8–12) acting directly or indirectly on all visual areas in the dorsal and ventral stream, including V1. However, the mechanism of this feedback is unclear. In particular, a first-order question is whether the top-down feedback targets V1 [or even the lateral geniculate nucleus (LGN)] first and then is passed on successively to later areas, or whether it targets higher-order areas first and then is fed back to successively lower areas. Without an understanding of the basic functional anatomy of the attentional feedback, it will be difficult to make progress in unraveling the circuitry for attention. Show less

Every time we look around we can see a rich and detailed world surrounding us. Nevertheless, the majority of visual information seems to slip out of our thoughts instantly. Can we still say that this fleeting percept of the entire world was a conscious percept in the first place, as Block proposes?

Humans are poor at detecting changes to visual scenes occurring during brief disruptions. It is unclear whether this 'change blindness' results from failure to process the relevant item before the change, or failure to compare/recall the item after the change. We recorded pre-change multi-unit activity in area V1 of monkeys performing a change detection task. The animals were rewarded for making a saccade to the changing figure. Figure-ground related activity was observed, even when no correct… Show more

Humans are poor at detecting changes to visual scenes occurring during brief disruptions. It is unclear whether this 'change blindness' results from failure to process the relevant item before the change, or failure to compare/recall the item after the change. We recorded pre-change multi-unit activity in area V1 of monkeys performing a change detection task. The animals were rewarded for making a saccade to the changing figure. Figure-ground related activity was observed, even when no correct saccade was made. However, for the changing figure, pre-change activity was stronger in correct trials than in incorrect trials. We conclude that small differences in pre-change figure-ground segregation have predictive value in whether the change will be successfully detected. Show less

Partial report methods have shown that a large-capacityrepresentation exists for a few hundred milliseconds after a picture has disappeared. However, change blindness studies indicate that very limited information remains available when a changed version of the image is presented subsequently.
What happens to the large-capacity representation? New input after the first image may interfere, but
this is likely to depend on the characteristicsof the new input. In our first experiment, we… Show more

Partial report methods have shown that a large-capacityrepresentation exists for a few hundred milliseconds after a picture has disappeared. However, change blindness studies indicate that very limited information remains available when a changed version of the image is presented subsequently.
What happens to the large-capacity representation? New input after the first image may interfere, but
this is likely to depend on the characteristicsof the new input. In our first experiment, we show that a
display containing homogeneous image elements between changing images does not render the large capacity representation unavailable. Interference occurs when these new elements define objects. On that basis we introduce a new method to produce change blindness: The second experiment shows that
change blindness can be induced by redefining figure and background, without an interval between the
displays. The local features (line segments) that defined figures and background were swapped, while
the contours of the figures remained where they were. Normally, changes are easily detected when
there is no interval. However, our paradigm results in massive change blindness. We propose that in a
change blindness experiment, there is a large-capacity representation of the original image when it is
followed by a homogeneous interval display, but that change blindness occurs whenever the changed
image forces resegregation of figures from the background. Show less

Normal people have a strikingly low ability to detect changes in a visual scene. This has been taken as evidence that the brain represents only a few objects at a time, namely those currently in the focus of attention. In the present study, subjects were asked to detect changes in the orientation of rectangular figures in a textured display across a 1600 ms gray interval. In the first experiment, change detection improved when the location of a possible change was cued during the interval. The… Show more

Normal people have a strikingly low ability to detect changes in a visual scene. This has been taken as evidence that the brain represents only a few objects at a time, namely those currently in the focus of attention. In the present study, subjects were asked to detect changes in the orientation of rectangular figures in a textured display across a 1600 ms gray interval. In the first experiment, change detection improved when the location of a possible change was cued during the interval. The cue remained effective during the entire interval, but after the interval, it was ineffective, suggesting that an initially large representation was overwritten by the post-change display. To control for an effect of light intensity during the interval on the decay of the representation, we compared performance with a gray or a white interval screen in a second experiment. We found no difference between these conditions. In the third experiment, attention was occasionally misdirected during the interval by first cueing the wrong figure, before cueing the correct figure. This did not compromise performance compared to a single cue, indicating that when an item is attentionally selected, the representation of yet unchosen items remains available. In the fourth experiment, the cue was shown to be effective when changes in figure size and orientation were randomly mixed. At the time the cue appeared, subjects could not know whether size or orientation would change, therefore these results suggest that the representation contains features in their ‘bound’ state. Together, these findings indicate that change blindness involves overwriting of a large capacity representation by the post-change display. Show less

Attentive processing is often described as a competition for resources among stimuli by mutual suppression. This is supported by findings that activity in extrastriate cortex is suppressed when several stimuli are presented simultaneously, compared to a single stimulus. In this study, we randomly varied the number of simultaneously presented figures (set size) in an attention-demanding change detection task, while we recorded multiunit activity in striate cortex (V1) in monkeys. After… Show more

Attentive processing is often described as a competition for resources among stimuli by mutual suppression. This is supported by findings that activity in extrastriate cortex is suppressed when several stimuli are presented simultaneously, compared to a single stimulus. In this study, we randomly varied the number of simultaneously presented figures (set size) in an attention-demanding change detection task, while we recorded multiunit activity in striate cortex (V1) in monkeys. After figure–background segregation, activity was suppressed as set size increased. This effect was stronger and started earlier among cells stimulated by the background than those stimulated by the figures themselves. As a consequence, contextual modulation, a correlate of figure–background segregation, increased with set size, approximately 100 msec after its initial generation. The results indicate that suppression of responses under increasing attentional demands differentially affects figure and background responses in area V1. Show less

Sudden changes are often not noticed when coupled with a brief disruption (‘change blindness’). Change detection under these circumstances appears to require focal attention. Little is known about what happens in the brain when a change is detected/missed. To explore the relation between brain activity and change detection, we trained monkeys to detect which of a varied number of rectangles changed its orientation across two subsequent presentations. We recorded multi-unit activity in the… Show more

Sudden changes are often not noticed when coupled with a brief disruption (‘change blindness’). Change detection under these circumstances appears to require focal attention. Little is known about what happens in the brain when a change is detected/missed. To explore the relation between brain activity and change detection, we trained monkeys to detect which of a varied number of rectangles changed its orientation across two subsequent presentations. We recorded multi-unit activity in the primary visual cortex while the animals performed the task. The percentage of correct responses indicated that change detection accuracy depended on the number of rectangles. With 4 rectangles on screen, about 50% of the changes were missed. Here we compared neuronal activity before the change occurred, between correct and incorrect trials. In correct trials, cells with their receptive field inside the rectangle that was going to change had a firing rate, higher then in incorrect trials, starting 95 ms after stimulus onset. In the background, activity was lower in correct than incorrect trials, starting 235 ms after stimulus onset. At non-changing rectangles there was no difference. Thus, figure/background segregation was greater in correct trials than incorrect trials, particularly at the location of the change. These results show that the magnitude of a V1 response related to pre-attentive figure-ground segregation predicts whether a change is going to be detected or not. This suggests that early, perhaps pre-attentive, processing is very important in change detection. Show less

In O'Regan & Noë's (O&N's) account for the phenomenal experience of seeing, awareness is equated to what is within the current focus of attention. They find no place for a distinction between phenomenal and access awareness. In doing so, they essentially present a dualistic solution to the mind-brain problem, and ignore that we do have phenomenal experience of what is outside the focus of attention.

In the search for the neural correlate of visual awareness, much controversy exists about the role of primary visual cortex. Here, the neurophysiological data from V1 recordings in awake monkeys are examined in light of two general classes of models of visual awareness. In the first model type, visual awareness is seen as being mediated either by a particular set of areas or pathways, or alternatively by a specific set of neurons. In these models, the role of V1 seems rather limited, as the… Show more

In the search for the neural correlate of visual awareness, much controversy exists about the role of primary visual cortex. Here, the neurophysiological data from V1 recordings in awake monkeys are examined in light of two general classes of models of visual awareness. In the first model type, visual awareness is seen as being mediated either by a particular set of areas or pathways, or alternatively by a specific set of neurons. In these models, the role of V1 seems rather limited, as the mere activity of V1 cells seems insufficient to mediate awareness. In the second model type, awareness is hypothesized to be mediated by a global mechanism, i.e. a specific kind of activity not linked to a particular area or cell type. Two separate versions of global models are discussed, synchronous oscillations and spike rate modulations. It is shown that V1 synchrony does not reflect perception but rather the horizontal connections between neurons, indicating that V1 synchrony cannot be a direct neural correlate of conscious percepts. However, the rate of spike discharges of V1 neurons is strongly modulated by perceptual context, and these modulations correlate very well with aspects of perceptual organization, visual awareness, and attention. If these modulations serve as a neural correlate of visual awareness, then V1 contributes to that neural correlate. Whether V1 plays a role in the neural correlate of visual awareness thus strongly depends on the way visual awareness is hypothesized to be implemented in the brain. Show less