Junseok Kwon

We propose WildTalker$^{\infty }$, a novel framework for synthesizing high-quality talking portraits capable of effectively addressing challenges encountered in real-world scenarios. Conventional techniques typically struggle to manage transient visual artifacts and auditory disturbances, thereby compromising the realism and synchronization of synthesized talking portraits. To overcome these limitations, WildTalker$^{\infty }$ introduces flow-guided temporal masking, a strategy specifically designed to accurately identify and mitigate transient movements within dynamic scenes, thereby enhancing visual coherence. Furthermore, WildTalker$^{\infty }$ integrates a multi-scale spectral subtraction method for robust audio denoising, significantly improving lip synchronization and ensuring natural audio-visual alignment even under challenging acoustic conditions. Our comprehensive experiments validate that WildTalker$^{\infty }$ considerably advances the synthesis quality of audio-driven 3D talking portraits, demonstrating superior lip synchronization performance in dynamic environments characterized by auditory and visual complexity. Comparative results confirm that WildTalker$^{\infty }$ consistently outperforms state-of-the-art methods across both controlled and uncontrolled scenarios, underscoring its practical efficacy and broad applicability.

In this paper, we present DiffMusic, a novel methodology for generating high-quality music from a single image. Existing methodologies have achieved multi-modality by integrating data from various domains or employing high-cost approaches, such as Large Language Models (LLMs), to generate high-quality music. The proposed DiffMusic adopts a diffusion-based approach to generate music descriptions from a single image, aiming to solve the issues encountered in traditional music generation methods. Unlike traditional image captioning, this approach does not simply describe the scene within the image. Instead, it generates descriptions that capture genre-related, melodic, and rhythmic elements essential for music generation. This approach enables the direct conversion of an image into music with a single inference process while allowing seamless integration into various existing music generators. Experimental results demonstrate that our method outperforms existing approaches by 29.9% in terms of Frechet Audio Distance, while remaining more cost-effective.

Deep Metric Learning (DML) is critical for constructing feature spaces where similar samples cluster closely, yet conventional cross-entropy (CE)–based methods often struggle to generalize to novel, unseen classes. The primary challenge arises from train–test scale misalignment, which disproportionately affects ambiguous (low-confidence) samples at test time and reduces separability. This paper presents a robust CE-based DML framework specifically designed to enhance generalization for unseen categories. We introduce Embedding Space Augmentation (ESA), a plug-in training component that (i) deliberately lowers the confidence of hard samples to amplify corrective gradients and (ii) guides updates along each class’s principal eigen-direction, thereby reducing intra-class variance and restoring train–test norm alignment. We quantify alignment with scale-invariant diagnostics—the Relative Radius Ratio and mean-radius gaps—and observe that ESA maintains small gaps throughout training, whereas CE drifts over time. Comprehensive experiments on standard image retrieval benchmarks, including CUB200, CARS196, SOP, and In-Shop, validate the effectiveness of the proposed framework. Our method addresses the generalization gap, achieving superior retrieval performance and consistently yielding substantial Recall@1 gains over multiple baseline networks. The technique is modular and can be seamlessly integrated into existing CE-based DML methods, offering significant and consistent accuracy improvements.

Deepfake generators evolve quickly, making exhaustive data collection and repeated retraining impractical. We argue that model merging is a natural fit for deepfake detection: unlike generic multi-task settings with disjoint labels, deepfake specialists share the same binary decision and differ in generator-specific artifacts. Empirically, we show that simple weight averaging preserves Real representations while attenuating Fake-specific cues. Building upon these findings, we propose Real-aware Residual Model Merging (R$^2$M), a training-free parameter-space merging framework. R$^2$M estimates a shared Real component via a low-rank factorization of task vectors, decomposes each specialist into a Real-aligned part and a Fake residual, denoises residuals with layerwise rank truncation, and aggregates them with per-task norm matching to prevent any single generator from dominating. A concise rationale explains why a simple head suffices: the Real component induces a common separation direction in feature space, while truncated residuals contribute only minor off-axis variations. Across in-distribution, cross-dataset, and unseen-dataset, R$^2$M outperforms joint training and other merging baselines. Importantly, R$^2$M is also composable: when a new forgery family appears, we fine-tune one specialist and re-merge, eliminating the need for retraining.

Multivariate time-series forecasting (MTSF) has become crucial across various domains, with transformer-based architectures emerging as the primary choice because of their excellent ability to capture long-term dependencies. However, these models often fail to represent fine-grained local patterns, which are critical for accurate forecasting. To address this limitation, we propose a novel MTSF method based on the diffusion process, utilizing the transformer as prior knowledge. By conditioning the denoising process on global embeddings derived from the transformer, our approach effectively captures both global and local patterns. However, when transformers are used as conditional priors in the diffusion process, errors in their predictions can propagate, adversely affecting overall performance. Thus, we also introduce a mechanism that mitigates the impact of such errors by enabling effective interactions between transformers and diffusion models. Furthermore, the proposed multi-step denoising process progressively refines temporal patterns, preserving both global structures and local variations to enhance robustness and generalization. Experimental results demonstrate that the proposed approach overcomes the limitations of existing models, consistently outperforming previous state-of-the-art models (TMDM), achieving up to 97.9% lower mean squared error (MSE) on Weather and 96.3% on ETTm2.