LLLaura Leal-Taixé

Laura Leal-Taixé

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Laura Leal-Taixé has a total of 109 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Human neuron activity during an 83-minute movie from 2,286 neurons and 29 patients

    by Alana Darcher, Franziska Gerken, Johannes Niediek, Marcel S. Kehl, Thomas P. Reber, Stefanie Liebe, Laura Nett, Attila Racz, Lukas Kunz, Bernhard Staresina, et al.

    Despite a growing trend towards more naturalistic experiments, few single-unit datasets collected during dynamic and naturalistic stimuli have been released, and none using a full-length movie. Here, we present SUMMER (Single Unit activity during a Movie in the human Medial temporal lobe via Electrophysiological Recordings), a dataset containing recordings from 2,286 neurons from the human amygdala, hippocampus, entorhinal cortex, parahippocampal cortex, and neighboring structures during the complete presentation of the commercial film 500 Days of Summer to 29 intracranially implanted patients. We provide a rich set of frame-wise annotations spanning the entirety of the movie’s 83-minute runtime, which systematically label the most salient narrative and visual elements, from characters and locations to camera cuts and main character speech. This Neurodata Without Borders-formatted dataset contains the spike times and corresponding mean waveforms from all recorded neurons, along with demographic information and electrode localizations. For technical validation, we provide spike-sorting metrics and demonstrate the tuning of individual neurons to movie features. We additionally provide decoding results from a machine learning-based pipeline for predicting movie features from neuronal population activity. To facilitate immediate use, we offer the dataset in a machine learning-ready format alongside a codebase demonstrating both neural feature and movie feature prediction tasks. This dataset offers a valuable foundation for exploring how the human brain processes semantic content, particularly in real-world contexts involving dynamic and naturalistic stimuli.


  • Open Access

    A Guide to Structureless Visual Localization

    by Vojtech Panek, Qunjie Zhou, Yaqing Ding, Sérgio Agostinho, Zuzana Kukelova, Torsten Sattler, Laura Leal-Taixé

    Visual localization algorithms, i.e., methods that estimate the camera pose of a query image in a known scene, are core components of many applications, including self-driving cars and augmented / mixed reality systems. State-of-the-art visual localization algorithms are structure-based, i.e., they store a 3D model of the scene and use 2D-3D correspondences between the query image and 3D points in the model for camera pose estimation. While such approaches are highly accurate, they are also rather inflexible when it comes to adjusting the underlying 3D model after changes in the scene. Structureless localization approaches represent the scene as a database of images with known poses and thus offer a much more flexible representation that can be easily updated by adding or removing images. Although there is a large amount of literature on structure-based approaches, there is significantly less work on structureless methods. Hence, this paper is dedicated to providing the, to the best of our knowledge, first comprehensive discussion and comparison of structureless methods. Extensive experiments show that approaches that use a higher degree of classical geometric reasoning generally achieve higher pose accuracy. In particular, approaches based on classical absolute or semi-generalized relative pose estimation outperform very recent methods based on pose regression by a wide margin. Compared with state-of-the-art structure-based approaches, the flexibility of structureless methods comes at the cost of (slightly) lower pose accuracy, indicating an interesting direction for future work.


  • Open Access

    Decoding movie content from neuronal population activity in the human medial temporal lobe

    by Franziska Gerken, Alana Darcher, Pedro J Gonçalves, Rachel Rapp, Ismail Elezi, Johannes Niediek, Marcel S Kehl, Thomas P Reber, Stefanie Liebe, Jakob H Macke, et al.

    The human medial temporal lobe (MTL), a region implicated in memory and high-level cognition, contains neurons that respond selectively to stimuli belonging to specific categories, such as individual people, landmarks, or objects. However, these neurons have been largely studied via static, isolated presentations of stimuli. Therefore, it is unclear how neurons in the MTL respond to rich stimuli such as movies, and which dynamical stimulus features can be retrieved from neuronal population spiking activity. We studied single-unit responses from 2286 neurons recorded from the amygdala, hippocampus, entorhinal cortex, and parahippocampal cortex of 29 intracranially implanted patients during the presentation of an 83-minute movie. We found only a few individual neurons that exhibited a classic selective response to semantic features. However, we successfully decoded the presence of characters, settings, and visual transitions from neuronal population activity. The information relevant for decoding varies across regions depending on the feature category, as visual transitions could be decoded from subsets of neurons with selective responses, whereas character and location features relied on distributed representations. Our results demonstrate an approach for reliably decoding movie features in the human MTL, and suggest that the brain uses a population code when representing character and location features.


  • Open Access

    A Guide to Structureless Visual Localization

    by Vojtech Panek, Qunjie Zhou, Yaqing Ding, Sérgio Agostinho, Zuzana Kukelova, Torsten Sattler, Laura Leal-Taixé

    Visual localization algorithms, i.e., methods that estimate the camera pose of a query image in a known scene, are core components of many applications, including self-driving cars and augmented / mixed reality systems. State-of-the-art visual localization algorithms are structure-based, i.e., they store a 3D model of the scene and use 2D-3D correspondences between the query image and 3D points in the model for camera pose estimation. While such approaches are highly accurate, they are also rather inflexible when it comes to adjusting the underlying 3D model after changes in the scene. Structureless localization approaches represent the scene as a database of images with known poses and thus offer a much more flexible representation that can be easily updated by adding or removing images. Although there is a large amount of literature on structure-based approaches, there is significantly less work on structureless methods. Hence, this paper is dedicated to providing the, to the best of our knowledge, first comprehensive discussion and comparison of structureless methods. Extensive experiments show that approaches that use a higher degree of classical geometric reasoning generally achieve higher pose accuracy. In particular, approaches based on classical absolute or semi-generalized relative pose estimation outperform very recent methods based on pose regression by a wide margin. Compared with state-of-the-art structure-based approaches, the flexibility of structureless methods comes at the cost of (slightly) lower pose accuracy, indicating an interesting direction for future work.

  • The NeRFect Match: Exploring NeRF Features for Visual Localization

    by Qunjie Zhou, Maxim Maximov, Or Litany, Laura Leal-Taixé


  • Open Access

    Lidar Panoptic Segmentation in an Open World

    by Anirudh S Chakravarthy, Meghana Reddy Ganesina, Peiyun Hu, Laura Leal-Taixe, Shu Kong, Deva Ramanan, Aljosa Osep

    Addressing Lidar Panoptic Segmentation (LPS ) is crucial for safe deployment of autonomous vehicles. LPS aims to recognize and segment lidar points w.r.t. a pre-defined vocabulary of semantic classes, including thing classes of countable objects (e.g., pedestrians and vehicles) and stuff classes of amorphous regions (e.g., vegetation and road). Importantly, LPS requires segmenting individual thing instances (e.g., every single vehicle). Current LPS methods make an unrealistic assumption that the semantic class vocabulary is fixed in the real open world, but in fact, class ontologies usually evolve over time as robots encounter instances of novel classes that are considered to be unknowns w.r.t. the pre-defined class vocabulary. To address this unrealistic assumption, we study LPS in the Open World (LiPSOW): we train models on a dataset with a pre-defined semantic class vocabulary and study their generalization to a larger dataset where novel instances of thing and stuff classes can appear. This experimental setting leads to interesting conclusions. While prior art train class-specific instance segmentation methods and obtain state-of-the-art results on known classes, methods based on class-agnostic bottom-up grouping perform favorably on classes outside of the initial class vocabulary (i.e., unknown classes). Unfortunately, these methods do not perform on-par with fully data-driven methods on known classes. Our work suggests a middle ground: we perform class-agnostic point clustering and over-segment the input cloud in a hierarchical fashion, followed by binary point segment classification, akin to Region Proposal Network [1]. We obtain the final point cloud segmentation by computing a cut in the weighted hierarchical tree of point segments, independently of semantic classification. Remarkably, this unified approach leads to strong performance on both known and unknown classes.

  • Lidar Panoptic Segmentation in an Open World

    by Anirudh S. Chakravarthy, Meghana Reddy Ganesina, Peiyun Hu, Laura Leal-Taixé, Shu Kong, Deva Ramanan, Aljosa Osep


  • Open Access

    Decoding movie content from neuronal population activity in the human medial temporal lobe

    by Franziska Gerken, Alana Darcher, Pedro J Gonçalves, Rachel Rapp, Ismail Elezi, Johannes Niediek, Marcel S Kehl, Thomas P Reber, Stefanie Liebe, Jakob H Macke, et al.

    The human medial temporal lobe (MTL), a region implicated in memory and high-level cognition, contains neurons that respond selectively to stimuli belonging to specific categories, such as individual people, landmarks, or objects. However, these neurons have been largely studied via static, isolated presentations of stimuli. Therefore, it is unclear how neurons in the MTL respond to rich stimuli such as movies, and which dynamical stimulus features can be retrieved from neuronal population spiking activity. We studied single-unit responses from 2286 neurons recorded from the amygdala, hippocampus, entorhinal cortex, and parahippocampal cortex of 29 intracranially implanted patients during the presentation of an 83-minute movie. We found only a few individual neurons that exhibited a classic selective response to semantic features. However, we successfully decoded the presence of characters, settings, and visual transitions from neuronal population activity. The information relevant for decoding varies across regions depending on the feature category, as visual transitions could be decoded from subsets of neurons with selective responses, whereas character and location features relied on distributed representations. Our results demonstrate an approach for reliably decoding movie features in the human MTL, and suggest that the brain uses a population code when representing character and location features.


  • Open Access

    Soft Augmentation for Image Classification

    by Yang Liu, Shen Yan, Laura Leal-Taixé, James Hays, Deva Ramanan

    Modern neural networks are over-parameterized and thus rely on strong regularization such as data augmentation and weight decay to reduce overfitting and improve generalization. The dominant form of data augmentation applies invariant transforms, where the learning target of a sample is invariant to the transform applied to that sample. We draw inspiration from human visual classification studies and propose generalizing augmentation with invariant transforms to soft augmentation where the learning target softens non-linearly as a function of the degree of the transform applied to the sample: e.g., more aggressive image crop augmentations produce less confident learning targets. We demonstrate that soft targets allow for more aggressive data augmentation, offer more robust performance boosts, work with other augmentation policies, and interestingly, produce better calibrated models (since they are trained to be less confident on aggressively cropped/occluded examples). Combined with existing aggressive augmentation strategies, soft target 1) doubles the top-1 accuracy boost across Cifar-10, Cifar-100, ImageNet-1K, and ImageNet-V2, 2) improves model occlusion performance by up to $4\times$, and 3) halves the expected calibration error (ECE). Finally, we show that soft augmentation generalizes to self-supervised classification tasks. Code available at https://github.com/youngleox/soft_augmentation

  • Open Access

    Staged Contact-Aware Global Human Motion Forecasting

    by Luca Scofano, Alessio Sampieri, Elisabeth Schiele, Edoardo De Matteis, Laura Leal-Taixé, Fabio Galasso

    Scene-aware global human motion forecasting is critical for manifold applications, including virtual reality, robotics, and sports. The task combines human trajectory and pose forecasting within the provided scene context, which represents a significant challenge. So far, only Mao et al. NeurIPS'22 have addressed scene-aware global motion, cascading the prediction of future scene contact points and the global motion estimation. They perform the latter as the end-to-end forecasting of future trajectories and poses. However, end-to-end contrasts with the coarse-to-fine nature of the task and it results in lower performance, as we demonstrate here empirically. We propose a STAGed contact-aware global human motion forecasting STAG, a novel three-stage pipeline for predicting global human motion in a 3D environment. We first consider the scene and the respective human interaction as contact points. Secondly, we model the human trajectory forecasting within the scene, predicting the coarse motion of the human body as a whole. The third and last stage matches a plausible fine human joint motion to complement the trajectory considering the estimated contacts. Compared to the state-of-the-art (SoA), STAG achieves a 1.8% and 16.2% overall improvement in pose and trajectory prediction, respectively, on the scene-aware GTA-IM dataset. A comprehensive ablation study confirms the advantages of staged modeling over end-to-end approaches. Furthermore, we establish the significance of a newly proposed temporal counter called the "time-to-go", which tells how long it is before reaching scene contact and endpoints. Notably, STAG showcases its ability to generalize to datasets lacking a scene and achieves a new state-of-the-art performance on CMU-Mocap, without leveraging any social cues. Our code is released at: https://github.com/L-Scofano/STAG