Jochen Lang

Objective. Ventilation–perfusion ( V / Q ) planar scintigraphy remains widely used for diagnosing pulmonary embolism, despite the reported superior diagnostic performance of V / Q SPECT. In part, SPECT adoption is limited by the differences in clinical interpretation. Pseudo-planar from SPECT data can facilitate transition to SPECT, but existing methods, such as summed angular projections or forward reprojection, fail to reliably reproduce true planar image characteristics. Approach. We propose a novel artificial intelligence–based count enhancement framework to generate high-fidelity pseudo-planar perfusion images ( pQ ) from low-count inputs. Two conditional generative adversarial networks (cGANs) were trained using (i) 10% Poisson-resampled planar perfusion images and (ii) matched single-projection SPECT views as inputs, with corresponding full-count planar perfusion images as ground truth. The models were trained and evaluated on 829 paired planar and SPECT studies from a multi-scanner clinical cohort. Performance was assessed using quantitative image similarity metrics (SSIM, PSNR, MSE) and a clinically meaningful task-based evaluation using a previously validated deep learning computer-assisted diagnostic model for vascular defect detection via free-response receiver operating characteristic analysis. Main Results. The cGAN trained on resampled planar images achieved the highest image fidelity (test SSIM: 0.70 ± 0.05; PSNR: 22.3 ± 1.2 dB), significantly outperforming SPECT-trained models and the conventional summed angular method ( p < 0.001). In downstream clinical evaluation, this model also yielded superior defect detection performance (sensitivity: 0.800 ± 0.035 at 1.0 false positives per projection), surpassing current clinical practice of the summed angular method. Significance. This study introduces the first AI-based framework for generating pseudo-planar lung scintigraphy from low-count data. The proposed method demonstrates improved image realism and clinically relevant defect preservation, supporting its potential for hybrid planar–SPECT workflows in V / Q imaging and for accelerating acquisition of planar images.

Dynamic object modeling is a critical challenge in 3D scene reconstruction. Previous methods typically maintain a canonical space to represent the object model, and a deformation field to express the object motion. However, this approach fails when the object undergoes large motions. The position variation caused by significant motion not only complicates the establishment of a canonical space, but also misleads the interpretation of the deformation field. To overcome the above weaknesses, we propose Motion Decoupled Dynamic 3D Gaussian Splatting (M5D-GS), the first 3D-GS model that separates motion and deformation modeling for dynamic object representation with large motion from a monocular camera. M5D-GS increases the practicality of 3D-GS, as it is common for objects to move, rotate, and deform simultaneously. Current datasets only contain object deformations with slight motions. We introduce a pipeline to reuse current benchmarks by adding large motions into the scene. We also introduce a new benchmark featuring several new scenes with complex motions, scenes augmented from previous datasets, and some real world recorded testcases, to fully demonstrate our improvements. Our M5D-GS significantly increases the accuracy under large motion scenarios while maintaining high rendering speed, which makes it suitable for dynamic object representation tasks including 4D novel view synthesis and real-time rendering.

Unrealistic parser-based dialogue systems limit player agency. LLM characters can enhance agency but lack structure and measurable objectives. We propose a framework for structured interactions that tracks player progress through specific objectives, while also improving character LLM responses. This approach frames interactions as puzzles with states representing goal-based milestones. We employ an LLM to analyze dialogue history and enforce state transitions for state awareness and to enable specific actions like tailored LLM prompts and multi-modal content changes. This results in a robust dialogue state tracking system for goal-based interactions. Using our method a designer can craft transition rules as abstract goals that allows players to invent their own solutions rather than discovering the designer's intent. We demonstrate this with a hostage scenario game, where the player negotiates with a hostage-taker adversary. The game's effectiveness is assessed through qualitative gameplay analysis and a quantitative evaluation of our state tracking method.

Video frame interpolation based on optical flow has made great progress in recent years. Most of the previous studies have focused on improving the quality of clean videos. However, many real-world videos contain large obstructions making the video discontinuous. To address this challenge, we propose our Obstruction Robustness Framework (ORF) that enhances the robustness of existing VFI networks in the face of large obstructions. The ORF contains two components: (1) A feature repair module that first captures ambiguous pixels in the synthetic frame by a region similarity map, then repairs them with a cross-overlap attention module. (2) A data augmentation strategy that enables the network to handle dynamic obstructions without extra data. To the best of our knowledge, this is the first work that explicitly addresses the error caused by large obstructions in video frame interpolation. By using previous state-of-the-art methods as backbones, our method does not only improve the results in original benchmarks but also significantly enhances the interpolation quality for videos with obstructions.

The goal of Referring Video Object Segmentation is to extract an object from a video clip based on a given expression. While previous methods have utilized the transformer’s multi-modal learning capabilities to aggregate information from different modalities, they have mainly focused on spatial information and paid less attention to temporal information. To enhance the learning of temporal information, we propose TCE-RVOS with a novel frame token fusion (FTF) structure and a novel instance query transformer (IQT). Our technical innovations maximize the potential information gain of videos over single images. Our contributions also include a new classification of two widely used validation datasets for investigation of challenging cases. Our experimental results demonstrate that TCERVOS effectively captures temporal information and outperforms the previous state-of-the-art methods by increasing the J&F score by 4.0 and 1.9 points using ResNet-50 and VSwin-Tiny as the backbone on Ref-Youtube-VOS, respectively, and +2.0 mAP on A2D-Sentences dataset by using VSwin-Tiny backbone. The code is available at https://github.com/haliphinx/TCE-RVOS