Joungil Yun
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Joungil Yun has a total of 25 co-authors. Here are the top co-authors by number of shared publications:
- Geuntaek Lim
Sejong University
2 shared publications - Joonsoo Kim
Electronics and Telecommunications Research Institute
2 shared publications - Won Jo
Department of Artificial Intelligence, Sejong University, Gwangjin-gu, Seoul, Republic of Korea
2 shared publications - Yukyung Choi
Sejong University
2 shared publications Byungjun Bae
Digital Broadcasting Res. Div., Electron. & Telecommun. Res. Inst., Daejeon
1 shared publicationChunhyun Ahn
Digital Broadcasting Res. Div., Electron. & Telecommun. Res. Inst., Daejeon
1 shared publication
Latest Publications
- Open Access
- 13 January 2024
- Published by Springer Nature in Virtual Reality
- Vol. 28 (1) , 1-17
Ray tracing-based construction of 3D background model for real-time stereoscopic rendering of live immersive video
Immersive video stored in multiview video-plus-depth format can provide viewers with vivid immersive experiences. However, rendering such video in real time in immersive environments remains a challenging task due to the high resolution and refresh rate demanded by recent extended reality displays. An essential issue in this immersive rendering is the disocclusion problem that inevitably occurs when virtual views are synthesized via the de facto standard 3D warping technique. In this paper, we present a novel virtual view synthesis framework that, from a live immersive video stream, renders stereoscopic images in real time for a freely moving virtual viewer. The main difference from previous approaches is that the surrounding background environment of the immersive video’s virtual scene is progressively reproduced on the fly directly in the 3D space while the input stream is being rendered. To allow this, we propose a new 3D background modeling scheme that, based on GPU-accelerated real-time ray tracing, efficiently and incrementally builds the background model in compact 3D triangular mesh. Then, we demonstrate that the 3D background environment can effectively alleviate the critical disocclusion problem in the immersive rendering, eventually reducing spatial and temporal aliasing artifacts. It is also suggested that the 3D representation of background environment enables extension of the virtual environment of immersive video by interactively adding 3D visual effects during rendering.
- 13 January 2024
- Published by Springer Nature in Virtual Reality
- Vol. 28 (1) , 1-17
Open AccessRay tracing-based construction of 3D background model for real-time stereoscopic rendering of live immersive video
Immersive video stored in multiview video-plus-depth format can provide viewers with vivid immersive experiences. However, rendering such video in real time in immersive environments remains a challenging task due to the high resolution and refresh rate demanded by recent extended reality displays. An essential issue in this immersive rendering is the disocclusion problem that inevitably occurs when virtual views are synthesized via the de facto standard 3D warping technique. In this paper, we present a novel virtual view synthesis framework that, from a live immersive video stream, renders stereoscopic images in real time for a freely moving virtual viewer. The main difference from previous approaches is that the surrounding background environment of the immersive video’s virtual scene is progressively reproduced on the fly directly in the 3D space while the input stream is being rendered. To allow this, we propose a new 3D background modeling scheme that, based on GPU-accelerated real-time ray tracing, efficiently and incrementally builds the background model in compact 3D triangular mesh. Then, we demonstrate that the 3D background environment can effectively alleviate the critical disocclusion problem in the immersive rendering, eventually reducing spatial and temporal aliasing artifacts. It is also suggested that the 3D representation of background environment enables extension of the virtual environment of immersive video by interactively adding 3D visual effects during rendering.
- Open Access
- 20 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 28466-28478
Simultaneous Video Retrieval and Alignment
With the growth of the video streaming industry, video retrieval and video alignment are facing high levels of demand. Several studies have demonstrated the feasibility of these methods for various problems related to video retrieval and alignment independently, but test in a unified framework has never been done. However, in real-world applications, it is also concurrently necessary not only to find which video pairs are similar (video retrieval) but also to align the positions of the pair that are related (video alignment). In this paper, we present a new task that simultaneously retrieves and aligns videos. As a solution to this task, a simultaneous retrieval and alignment framework abbreviated as SRA is proposed, which is a two-stage approach consisting of a foreground proposal stage and a downstream stage to efficiently process untrimmed videos. Furthermore, two criteria are suggested to support the new task: a metric mAP@J assessing how highly the related videos are ranked and how well relevant positions are assigned in that video, and a dataset FIVR+A that includes video-level relationships and hierarchical segment-level annotations. Finally, we conduct multi-pronged analyses to assess how our approach handles the new task in various experiments.
- 20 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 28466-28478
Open AccessSimultaneous Video Retrieval and Alignment
With the growth of the video streaming industry, video retrieval and video alignment are facing high levels of demand. Several studies have demonstrated the feasibility of these methods for various problems related to video retrieval and alignment independently, but test in a unified framework has never been done. However, in real-world applications, it is also concurrently necessary not only to find which video pairs are similar (video retrieval) but also to align the positions of the pair that are related (video alignment). In this paper, we present a new task that simultaneously retrieves and aligns videos. As a solution to this task, a simultaneous retrieval and alignment framework abbreviated as SRA is proposed, which is a two-stage approach consisting of a foreground proposal stage and a downstream stage to efficiently process untrimmed videos. Furthermore, two criteria are suggested to support the new task: a metric mAP@J assessing how highly the related videos are ranked and how well relevant positions are assigned in that video, and a dataset FIVR+A that includes video-level relationships and hierarchical segment-level annotations. Finally, we conduct multi-pronged analyses to assess how our approach handles the new task in various experiments.
- Open Access
- 11 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 38973-38981
Exploring the Temporal Cues to Enhance Video Retrieval on Standardized CDVA
As the demand for large-scale video analysis increases, video retrieval research is also becoming more active. In 2014, ISO/IEC MPEG began standardizing compact descriptors for video analysis, known as CDVA, and it is now adopted as a standard. However, the standardized CDVA is not easily compared to other methods because the MPEG-CDVA dataset used for performance verification is not disclosed, despite the fact that follow-up studies are underway with multiple versions of the CDVA experimental model. In addition, analyses of modules constituting the CDVA framework are insufficient in previous studies. Therefore, we conduct self-evaluations of CDVA to analyze the impact of each module on the retrieval task. Furthermore, to overcome the obstacles identified through these self-evaluations, we suggest temporal nested invariance pooling, abbreviated as TNIP, which implies temporal robustness realized by improving nested invariance pooling, abbreviated as NIP, one of the features in CDVA. Finally, benchmarks of the existing CDVA and the proposed approach are provided on several public datasets. Through this, we show that the CDVA framework is capable of boosting the retrieval performance if utilizing the proposed approach.
- 11 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 38973-38981
Open AccessExploring the Temporal Cues to Enhance Video Retrieval on Standardized CDVA
As the demand for large-scale video analysis increases, video retrieval research is also becoming more active. In 2014, ISO/IEC MPEG began standardizing compact descriptors for video analysis, known as CDVA, and it is now adopted as a standard. However, the standardized CDVA is not easily compared to other methods because the MPEG-CDVA dataset used for performance verification is not disclosed, despite the fact that follow-up studies are underway with multiple versions of the CDVA experimental model. In addition, analyses of modules constituting the CDVA framework are insufficient in previous studies. Therefore, we conduct self-evaluations of CDVA to analyze the impact of each module on the retrieval task. Furthermore, to overcome the obstacles identified through these self-evaluations, we suggest temporal nested invariance pooling, abbreviated as TNIP, which implies temporal robustness realized by improving nested invariance pooling, abbreviated as NIP, one of the features in CDVA. Finally, benchmarks of the existing CDVA and the proposed approach are provided on several public datasets. Through this, we show that the CDVA framework is capable of boosting the retrieval performance if utilizing the proposed approach.
- Open Access
- 24 February 2022
- Published by Wiley in ETRI Journal
- Vol. 44 (1) , 24-37
View synthesis with sparse light field for 6DoF immersive video
Virtual view synthesis, which generates novel views similar to the characteristics of actually acquired images, is an essential technical component for delivering an immersive video with realistic binocular disparity and smooth motion parallax. This is typically achieved in sequence by warping the given images to the designated viewing position, blending warped images, and filling the remaining holes. When considering 6DoF use cases with huge motion, the warping method in patch unit is more preferable than other conventional methods running in pixel unit. Regarding the prior case, the quality of synthesized image is highly relevant to the means of blending. Based on such aspect, we proposed a novel blending architecture that exploits the similarity of the directions of rays and the distribution of depth values. By further employing the proposed method, results showed that more enhanced view was synthesized compared with the well‐designed synthesizers used within moving picture expert group (MPEG‐I). Moreover, we explained the GPU‐based implementation synthesizing and rendering views in the level of real time by considering the applicability for immersive video service.
- 24 February 2022
- Published by Wiley in ETRI Journal
- Vol. 44 (1) , 24-37
Open AccessView synthesis with sparse light field for 6DoF immersive video
Virtual view synthesis, which generates novel views similar to the characteristics of actually acquired images, is an essential technical component for delivering an immersive video with realistic binocular disparity and smooth motion parallax. This is typically achieved in sequence by warping the given images to the designated viewing position, blending warped images, and filling the remaining holes. When considering 6DoF use cases with huge motion, the warping method in patch unit is more preferable than other conventional methods running in pixel unit. Regarding the prior case, the quality of synthesized image is highly relevant to the means of blending. Based on such aspect, we proposed a novel blending architecture that exploits the similarity of the directions of rays and the distribution of depth values. By further employing the proposed method, results showed that more enhanced view was synthesized compared with the well‐designed synthesizers used within moving picture expert group (MPEG‐I). Moreover, we explained the GPU‐based implementation synthesizing and rendering views in the level of real time by considering the applicability for immersive video service.
- Open Access
Enhanced Soft 3D Reconstruction Method with an Iterative Matching Cost Update Using Object Surface Consensus
In this paper, we propose a multi-view stereo matching method, EnSoft3D (Enhanced Soft 3D Reconstruction) to obtain dense and high-quality depth images. Multi-view stereo is one of the high-interest research areas and has wide applications. Motivated by the Soft3D reconstruction method, we introduce a new multi-view stereo matching scheme. The original Soft3D method is introduced for novel view synthesis, while occlusion-aware depth is also reconstructed by integrating the matching costs of the Plane Sweep Stereo (PSS) and soft visibility volumes. However, the Soft3D method has an inherent limitation because the erroneous PSS matching costs are not updated. To overcome this limitation, the proposed scheme introduces an update process of the PSS matching costs. From the object surface consensus volume, an inverse consensus kernel is derived, and the PSS matching costs are iteratively updated using the kernel. The proposed EnSoft3D method reconstructs a highly accurate 3D depth image because both the multi-view matching cost and soft visibility are updated simultaneously. The performance of the proposed method is evaluated by using structured and unstructured benchmark datasets. Disparity error is measured to verify 3D reconstruction accuracy, and both PSNR and SSIM are measured to verify the simultaneous enhancement of view synthesis.
Open AccessEnhanced Soft 3D Reconstruction Method with an Iterative Matching Cost Update Using Object Surface Consensus
In this paper, we propose a multi-view stereo matching method, EnSoft3D (Enhanced Soft 3D Reconstruction) to obtain dense and high-quality depth images. Multi-view stereo is one of the high-interest research areas and has wide applications. Motivated by the Soft3D reconstruction method, we introduce a new multi-view stereo matching scheme. The original Soft3D method is introduced for novel view synthesis, while occlusion-aware depth is also reconstructed by integrating the matching costs of the Plane Sweep Stereo (PSS) and soft visibility volumes. However, the Soft3D method has an inherent limitation because the erroneous PSS matching costs are not updated. To overcome this limitation, the proposed scheme introduces an update process of the PSS matching costs. From the object surface consensus volume, an inverse consensus kernel is derived, and the PSS matching costs are iteratively updated using the kernel. The proposed EnSoft3D method reconstructs a highly accurate 3D depth image because both the multi-view matching cost and soft visibility are updated simultaneously. The performance of the proposed method is evaluated by using structured and unstructured benchmark datasets. Disparity error is measured to verify 3D reconstruction accuracy, and both PSNR and SSIM are measured to verify the simultaneous enhancement of view synthesis.
- 22 September 2006
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 0747668X,p. 1-4
Development of a Novel Ensemble Analysis System for T-DMB Services
This paper introduces the architecture of the terrestrial digital multimedia broadcasting (T-DMB) transmission system. The layered structure of T-DMB ensemble for video service is studied to develop a novel ensemble analysis system which consists of a hardware module for signal processing and a software module for analysis processing
- 22 September 2006
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 0747668X,p. 1-4
Development of a Novel Ensemble Analysis System for T-DMB Services
This paper introduces the architecture of the terrestrial digital multimedia broadcasting (T-DMB) transmission system. The layered structure of T-DMB ensemble for video service is studied to develop a novel ensemble analysis system which consists of a hardware module for signal processing and a software module for analysis processing