LJLaszlo Attila Jeni

Laszlo Attila Jeni

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Laszlo Attila Jeni has a total of 73 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    REVIT: A Dataset and Benchmark for Remote Vital Sign Estimation in Real-World Settings

    by Ananyananda Dasari, Ambareesh Revanur, Sai Shweta Gadepally, Nathaniel Houston-Suluku, Sheku Kamara, Laszlo A. Jeni, Conrad Tucker

    Video-based physiological signal estimation is crucial for non-invasive and contactless patient monitoring in scenarios where traditional sensor-based methods are impractical, such as telemedicine and large-scale population health studies. The development and efficacy of such approaches is critically influenced by the availability of representative human subject datasets that reflect real-world scenarios. However, existing datasets predominantly consist of recordings collected in controlled laboratory settings with limited diversity in environmental conditions and demographic representation. Such constraints hinder the generalizability and robustness of video-based physiological signal estimation approaches in real-world scenarios (due to real-world scenarios demonstrating a challenging distribution of environmental diversity as compared to laboratory conditions). To this end, we present the REmote VITals (REVIT) dataset, comprising of 1553 human subject videos and associated ground truth physiological sign measurements (such as heart rate) collected in multiple outside-the-lab settings for the remote estimation of human physiological signs. The dataset is uniquely designed to capture a wide range of environmental diversity (such as variable lighting and background) encountered in outside-the-lab settings, thereby enabling the development of more robust physiological signal estimation approaches. Through statistical hypothesis testing, we demonstrate that utilizing the REVIT dataset results in a significant improvement in the robustness of existing video-based physiological signal estimation approaches. Furthermore, this work highlights the impact of existing data distributions toward controlled laboratory conditions, underscoring their limitations in real world applicability.

  • Open Access

    Exploring image and skeleton-based action recognition approaches for clinical in-bed classification of simulated epileptic seizure movements

    by Tamás Karácsony, Nicholas Fearns, Denise Birk, Selina Denise Trapp, Katharina Ernst, Christian Vollmar, Jan Rémi, László Attila Jeni, Fernando De la Torre, João Paulo Silva Cunha


  • Open Access

    Through the Curved Cover: Synthesizing Cover Aberrated Scenes with Refractive Field

    by Liuyue Xie, Jiancong Guo, Laszlo A. Jeni, Zhiheng Jia, Mingyang Li, Yunwen Zhou, Chao Guo

    Recent extended reality headsets and field robots have adopted covers to protect the front-facing cameras from environmental hazards and falls. The surface irregularities on the cover can lead to optical aberrations like blurring and non-parametric distortions. Novel view synthesis methods like NeRF and 3D Gaussian Splatting are ill-equipped to synthesize from sequences with optical aberrations. To address this challenge, we introduce SynthCover to enable novel view synthesis through protective covers for downstream extended reality applications. SynthCover employs a Refractive Field that estimates the cover's geometry, enabling precise analytical calculation of refracted rays. Experiments on synthetic and real-world scenes demonstrate our method's ability to accurately model scenes viewed through protective covers, achieving a significant improvement in rendering quality compared to prior methods. We also show that the model can adjust well to various cover geometries with synthetic sequences captured with covers of different surface curvatures. To motivate further studies on this problem, we provide the benchmarked dataset containing real and synthetic walkable scenes captured with protective cover optical aberrations.

  • Video Question Answering with Procedural Programs

    by Rohan Choudhury, Koichiro Niinuma, Kris M. Kitani, László A. Jeni


  • Open Access

    CoGS: Controllable Gaussian Splatting

    by Heng Yu, Joel Julin, Zoltán Á. Milacski, Koichiro Niinuma, László A. Jeni

    Capturing and re-animating the 3D structure of articulated objects present significant barriers. On one hand, methods requiring extensively calibrated multi-view setups are prohibitively complex and resource-intensive, limiting their practical applicability. On the other hand, while single-camera Neural Radiance Fields (NeRFs) offer a more streamlined approach, they have excessive training and rendering costs. 3D Gaussian Splatting would be a suitable alternative but for two reasons. Firstly, existing methods for 3D dynamic Gaussians require synchronized multi-view cameras, and secondly, the lack of controllability in dynamic scenarios. We present CoGS, a method for Controllable Gaussian Splatting, that enables the direct manipulation of scene elements, offering real-time control of dynamic scenes without the prerequisite of pre-computing control signals. We evaluated CoGS using both synthetic and real-world datasets that include dynamic objects that differ in degree of difficulty. In our evaluations, CoGS consistently outperformed existing dynamic and controllable neural representations in terms of visual fidelity.


  • Open Access

    Zero-Shot Video Question Answering with Procedural Programs

    by Rohan Choudhury, Koichiro Niinuma, Kris M. Kitani, László A. Jeni

    We propose to answer zero-shot questions about videos by generating short procedural programs that derive a final answer from solving a sequence of visual subtasks. We present Procedural Video Querying (ProViQ), which uses a large language model to generate such programs from an input question and an API of visual modules in the prompt, then executes them to obtain the output. Recent similar procedural approaches have proven successful for image question answering, but videos remain challenging: we provide ProViQ with modules intended for video understanding, allowing it to generalize to a wide variety of videos. This code generation framework additionally enables ProViQ to perform other video tasks in addition to question answering, such as multi-object tracking or basic video editing. ProViQ achieves state-of-the-art results on a diverse range of benchmarks, with improvements of up to 25% on short, long, open-ended, and multimodal video question-answering datasets. Our project page is at https://rccchoudhury.github.io/proviq2023.

  • Open Access

    Video-based Estimation of Blood Pressure

    by Conrad Tucker, Ananyananda Dasari, Laszlo Jeni

    In this work, we propose a non-contact video-based approach that estimates an individual’s blood pressure. The estimation of blood pressure is critical for monitoring hypertension and cardiovascular diseases such as coronary artery disease or stroke. Estimation of blood pressure is typically achieved using contact-based devices which apply pressure on the arm through a cuff. Such contact-based devices are cost-prohibitive as well as limited in their scalability due to the requirement of specialized equipment. The ubiquity of mobile phones and video-based capturing devices motivates the development of a non-contact blood pressure estimation method - Video-based Blood Pressure Estimation (V-BPE). We leverage the time difference of the blood pulse arrival at two different locations in the body (Pulse Transit Time) and the inverse relation between the blood pressure and the velocity of blood in the artery to analytically estimate the blood pressure. Through statistical hypothesis testing, we demonstrate that Pulse Transit Time-based approaches to estimate blood pressure require knowledge of subject specific blood vessel parameters, such as the length of the blood vessel. We utilize a combination of computer vision techniques and demographic information (such as the height and the weight of the subject) to capture and incorporate the aforementioned subject specific blood vessel parameters into our estimation of blood pressure. We demonstrate the robustness of V-BPEby evaluating the efficacy of blood pressure estimation in demographically diverse, outside-the-lab conditions. V-BPE is advantageous in two ways; 1) it is non-contact-based, reducing the possibility of infection due to contact and 2) it is scalable, given the ubiquity of video recording devices.

  • Open Access

    TEMPO: Efficient Multi-View Pose Estimation, Tracking, and Forecasting

    by Rohan Choudhury, Kris Kitani, Laszlo A. Jeni

    Existing volumetric methods for predicting 3D human pose estimation are accurate, but computationally expensive and optimized for single time-step prediction. We present TEMPO, an efficient multi-view pose estimation model that learns a robust spatiotemporal representation, improving pose accuracy while also tracking and forecasting human pose. We significantly reduce computation compared to the state-of-the-art by recurrently computing per-person 2D pose features, fusing both spatial and temporal information into a single representation. In doing so, our model is able to use spatiotemporal context to predict more accurate human poses without sacrificing efficiency. We further use this representation to track human poses over time as well as predict future poses. Finally, we demonstrate that our model is able to generalize across datasets without scene-specific fine-tuning. TEMPO achieves 10$\%$ better MPJPE with a 33$\times$ improvement in FPS compared to TesseTrack on the challenging CMU Panoptic Studio dataset.


  • Open Access

    Deep Learning Methods for Single Camera Based Clinical In-bed Movement Action Recognition

    by Tamas Karacsony, Laszlo A. Jeni, Fernando De La Torre Frade, João Paulo Silva Cunha

    Many clinical applications involve in-bed patient activity monitoring, from intensive care and neuro-critical infirmary, to semiology-based epileptic seizure diagnosis support or sleep monitoring at home, which require accurate recognition of in-bed movement actions from video streams. The major challenges of clinical application arise from the domain gap between common in-the-lab and clinical scenery (e.g. viewpoint, occlusions, out-of-domain actions), the requirement of minimally intrusive monitoring to already existing clinical practices (e.g. non-contact monitoring), and the significantly limited amount of labeled clinical action data available. Focusing on one of the most demanding in-bed clinical scenarios - semiology-based epileptic seizure classification – this review explores the challenges of video-based clinical in-bed monitoring, reviews video-based action recognition trends, monocular 3D MoCap, and semiology-based automated seizure classification approaches. Moreover, provides a guideline to take full advantage of transfer learning for in-bed action recognition for quantified, evidence-based clinical diagnosis support. The review suggests that an approach based on 3D MoCap and skeleton-based action recognition, strongly relying on transfer learning, could be advantageous for these clinical in-bed action recognition problems. However, these still face several challenges, such as spatio-temporal stability, occlusion handling, and robustness before realizing the full potential of this technology for routine clinical usage.

  • Open Access

    Unsupervised Style-based Explicit 3D Face Reconstruction from Single Image

    by Heng Yu, Zoltan A. Milacski, Laszlo A. Jeni

    Inferring 3D object structures from a single image is an ill-posed task due to depth ambiguity and occlusion. Typical resolutions in the literature include leveraging 2D or 3D ground truth for supervised learning, as well as imposing hand-crafted symmetry priors or using an implicit representation to hallucinate novel viewpoints for unsupervised methods. In this work, we propose a general adversarial learning framework for solving Unsupervised 2D to Explicit 3D Style Transfer (UE3DST). Specifically, we merge two architectures: the unsupervised explicit 3D reconstruction network of Wu et al.\ and the Generative Adversarial Network (GAN) named StarGAN-v2. We experiment across three facial datasets (Basel Face Model, 3DFAW and CelebA-HQ) and show that our solution is able to outperform well established solutions such as DepthNet in 3D reconstruction and Pix2NeRF in conditional style transfer, while we also justify the individual contributions of our model components via ablation. In contrast to the aforementioned baselines, our scheme produces features for explicit 3D rendering, which can be manipulated and utilized in downstream tasks.