Dongsan Jun

Multi-scale rendering exposes scale-dependent reconstruction errors in 3D Gaussian splatting (3DGS) when the rendering resolution differs from the training resolution. Existing anti-aliasing and multi-scale optimization methods reduce these artifacts, but each Gaussian usually keeps a fixed color representation and a fixed spatial allocation after densification. This fixed representation is restrictive at coarse scales because one rendered pixel combines multiple Gaussian contributions into an average color that can differ from the color assigned to each Gaussian. In addition, Gaussians allocated by fine-scale reconstruction criteria may provide insufficient coverage for regions with persistent coarse-scale errors. This paper proposes adaptive-scale color offset and error-guided Gaussian reallocation for multi-scale 3DGS. The adaptive-scale color offset applies a scale-specific correction to the shared base color of each Gaussian while keeping its geometry unchanged. Error-guided Gaussian reallocation shifts low-contribution Gaussians toward high-error regions under a fixed primitive budget and preserves Gaussians that support coarse-scale rendering. Experimental results show that the proposed method improves multi-scale rendering quality across rendering scales on Mip-NeRF 360, Tanks and Temples, and Deep Blending, with larger gains at coarse scales. These results demonstrate that adapting Gaussian color and primitive allocation together improves scale-consistent 3DGS rendering without modifying the inference-stage rendering process.

The Moving Picture Experts Group (MPEG) is developing the Video Coding for Machines (VCM) standard to support efficient video compression for machine vision tasks. The VCM standard primarily targets object detection, tracking, and semantic segmentation. Since VCM mainly focuses on object-centric tasks such as detection and tracking, it employs Region-of-Interest (ROI) coding to allocate more bits to foregrounds, while suppressing background regions. This suppression reduces segmentation accuracy by degrading contextual background information. To address this limitation, we propose a luma background restoration method that reconstructs degraded background regions by exploiting the structural correlation between decoded luma and chroma components without relying on complex chroma modeling. The proposed method integrates multi-channel linear modeling with context-based arithmetic coding to efficiently transmit grouped Linear Model (LM) indices for luma restoration. Under VCM test conditions, experimental results show that the proposed method achieves an average Bjøntegaard Delta mean Intersection-over-Union (BD-mIoU) of 7.70, compared with 7.41 achieved by the latest background preservation method. These results demonstrate that the proposed method effectively restores structural background details in luma regions essential for semantic segmentation in VCM frameworks.

Versatile Video Coding (VVC) achieves higher compression efficiency than the previous High Efficiency Video Coding (HEVC) standard by employing advanced coding tools, including Quad Tree (QT) and Multi-Type Tree (MTT) block partitioning, extended intra prediction modes, and affine motion compensation. Among these tools, the QT-MTT hierarchical partitioning structure significantly increases encoder complexity, since Rate-Distortion Optimization (RDO) must be performed over an exponentially growing number of partition candidates. To mitigate this complexity, a quantization-adaptive early termination method is proposed that combines neural network-based and rule-based partitioning strategies. The proposed decision mechanism significantly reduces the number of Coding Unit (CU) partition candidates, which directly lowers the number of required RDO evaluations and overall encoder complexity. Experimental results demonstrate that the proposed method achieves a 38.28% reduction in encoding time with only a 0.85% increase in Bjøntegaard Delta Bitrate (BD-BR) under the VVC common test conditions. These results indicate that the proposed method effectively balances computational complexity and rate-distortion performance.

Video super-resolution (VSR) aims to reconstruct high-resolution (HR) videos from low-resolution (LR) inputs by utilizing spatio-temporal correlations across consecutive LR frames. While recent advances in deep learning, particularly transformer-based architecture, have substantially improved VSR performance, maintaining spatio-temporal coherence remains a critical challenge. To address this issue, we propose a novel U-Net based spatial transformer module (USTM) that can be seamlessly integrated into the reconstruction stage of existing VSR frameworks. The proposed USTM combines a spatial transformer with a U-Net structure to extract fine-grained spatio-temporal features to enhance the reconstruction of complex motions and texture patterns. Extensive ablation studies were conducted to identify the optimal configuration and verify the contribution of each USTM component. For performance evaluations, USTM was incorporated into multiple representative VSR methods. Experimental results demonstrate that integrating USTM consistently enhances PSNR and SSIM scores on benchmark datasets, including REDS, Vimeo-90K, and Vid4. Furthermore, visual comparisons highlight the superiority of the proposed method, particularly in high-frequency regions, compared to baseline VSR methods without USTM.

Disentangling complex causal relationships is important for accurate detection of anomalies. In multivariate time series analysis, dynamic interactions among data variables over time complicate the interpretation of causal relationships. Traditional approaches assume statistical independence between variables in unsupervised settings, whereas recent methods capture feature correlations through graph representation learning. However, their representations fail to explicitly infer the causal relationships over different time periods. To solve the problem, we propose Causally Disentangled Representation Learning for Anomaly Detection (CDRL4AD) to detect anomalies and identify their causal relationships in multivariate time series. First, we design the causal process as model input, the temporal heterogeneous graph, and causal relationships. Second, our representation identifies causal relationships over different time periods and disentangles latent variables to infer the corresponding causal factors. Third, our experiments on real-world datasets demonstrate that CDRL4AD outperforms state-of-the-art methods in terms of accuracy and root cause analysis. Fourth, our model analysis validates hyperparameter sensitivity and the time complexity of CDRL4AD. Last, we conduct a case study to show how our approach assists human experts in diagnosing the root causes of anomalies.

The increasing demand for high-quality audio content in applications such as media streaming and immersive audio services has driven the development of efficient audio coding techniques. Among these techniques, entropy coding is an essential component for the lossless compression of quantized spectral coefficients, which constitute the largest portion of the compressed audio bitstream. As the latest audio coding standard, Unified Speech and Audio Coding (USAC) has adopted ContextAdaptive Arithmetic Coding (CAAC) to employ its entropy coding method. Although CAAC can provide efficient lossless coding, it has several limitations, including bit overhead caused by variations in bit lengths between coefficients and considerable memory requirements for storing multiple context tables. To reduce excessive memory consumption, this study proposes Dynamic Bit-plane Arithmetic Coding (DBAC) as a new entropy coding method for quantized spectral coefficients. DBAC mitigates the bit overhead by using escape symbols for binarized coefficients and reduces memory usage while maintaining coding performance, even with marginal coding gains. In addition, the generation and selection methods of newly designed multiple contexts are also introduced to optimize arithmetic coding on the proposed dynamic bit-plane of DBAC. Experimental results demonstrate that the proposed method achieves 30% memory reduction with comparable coding performance compared to USAC.

Recently, the demand for immersive videos has surged with the expansion of virtual reality, augmented reality, and metaverse technologies. As an international standard, moving picture experts group (MPEG) has developed MPEG immersive video (MIV) to efficiently transmit large-volume immersive videos. The MIV encoder generates atlas videos to convert extensive multi-view videos into low-bitrate formats. When these atlas videos are compressed using conventional video codecs, compression artifacts often appear in the reconstructed atlas videos. To address this issue, this study proposes a feature-extraction-based convolutional neural network (FECNN) to reduce the compression artifacts during MIV atlas video transmission. The proposed FECNN uses quantization parameter (QP) maps and depth information as inputs and consists of shallow feature extraction (SFE) blocks and deep feature extraction (DFE) blocks to utilize layered feature characteristics. Compared to the existing MIV, the proposed method improves the Bjontegaard delta bit-rate (BDBR) by −4.12% and −6.96% in the basic and additional views, respectively.

Versatile Video Coding (VVC), the state-of-the-art video coding standard, was developed by the Joint Video Experts Team (JVET) of ISO/IEC Moving Picture Experts Group (MPEG) and ITU-T Video Coding Experts Group (VCEG) in 2020. Although VVC can provide powerful coding performance, it requires tremendous computational complexity to determine the optimal mode decision during the encoding process. In particular, VVC adopted the bi-prediction with CU-level weight (BCW) as one of the new tools, which enhanced the coding efficiency of conventional bi-prediction by assigning different weights to the two prediction blocks in the process of inter prediction. In this study, we investigate the statistical characteristics of input features that exhibit a correlation with the BCW and define four useful types of categories to facilitate the inter prediction of VVC. With the investigated input features, a lightweight neural network with multilayer perceptron (MLP) architecture is designed to provide high accuracy and low complexity. We propose a fast BCW mode decision method with a lightweight MLP to reduce the computational complexity of the weighted multiple bi-prediction in the VVC encoder. The experimental results show that the proposed method significantly reduced the BCW encoding complexity by up to 33% with unnoticeable coding loss, compared to the VVC test model (VTM) under the random-access (RA) configuration.

With the advancement of sensors, image and video processing have developed for use in the visual sensing area. Among them, video super-resolution (VSR) aims to reconstruct high-resolution sequences from low-resolution sequences. To use consecutive contexts within a low-resolution sequence, VSR learns the spatial and temporal characteristics of multiple frames of the low-resolution sequence. As one of the convolutional neural network-based VSR methods, we propose a deformable convolution-based alignment network (DCAN) to generate scaled high-resolution sequences with quadruple the size of the low-resolution sequences. The proposed method consists of a feature extraction block, two different alignment blocks that use deformable convolution, and an up-sampling block. Experimental results show that the proposed DCAN achieved better performances in both the peak signal-to-noise ratio and structural similarity index measure than the compared methods. The proposed DCAN significantly reduces the network complexities, such as the number of network parameters, the total memory, and the inference speed, compared with the latest method.

In this paper, we propose an object-cooperated decision method for efficient ternary tree (TT) partitioning that reduces the encoding complexity of versatile video coding (VVC). In most previous studies, the VVC complexity was reduced using decision schemes based on the encoding context, which do not apply object detecion models. We assume that high-level objects are important for deciding whether complex TT partitioning is required because they can provide hints on the characteristics of a video. Herein, we apply an object detection model that discovers and extracts the high-level object features—the number and ratio of objects from frames in a video sequence. Using the extracted features, we propose machine learning (ML)-based classifiers for each TT-split direction to efficiently reduce the encoding complexity of VVC and decide whether the TT-split process can be skipped in the vertical or horizontal direction. The TT-split decision of classifiers is formulated as a binary classification problem. Experimental results show that the proposed method more effectively decreases the encoding complexity of VVC than a state-of-the-art model based on ML.