Byeoli Choi
Research Subjects
Byeoli Choi has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Deok-Kyeong Jang
MOVIN Inc
4 shared publications - Dongseok Yang
Korea Advanced Institute of Science and Technology (KAIST), Daejeon, SOUTH KOREA
4 shared publications - Sung-Hee Lee
Korea Advanced Institute of Science and Technology
4 shared publications - Taeil Jin
Korea Institute of Science and Technology Republic of Korea
2 shared publications - Deok-Yun Jang
Gwangju Institute of Science and Technology
1 shared publication - Donghoon Shin
MOVIN Inc., Seoul, South Korea
1 shared publication
Latest Publications
- Open Access
- 19 November 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 43 (6) , 1-14
ELMO: Enhanced Real-time LiDAR Motion Capture through Upsampling
This paper introduces ELMO, a real-time upsampling motion capture framework designed for a single LiDAR sensor. Modeled as a conditional autoregressive transformer-based upsampling motion generator, ELMO achieves 60 fps motion capture from a 20 fps LiDAR point cloud sequence. The key feature of ELMO is the coupling of the self-attention mechanism with thoughtfully designed embedding modules for motion and point clouds, significantly elevating the motion quality. To facilitate accurate motion capture, we develop a one-time skeleton calibration model capable of predicting user skeleton off-sets from a single-frame point cloud. Additionally, we introduce a novel data augmentation technique utilizing a LiDAR simulator, which enhances global root tracking to improve environmental understanding. To demonstrate the effectiveness of our method, we compare ELMO with state-of-the-art methods in both image-based and point cloud-based motion capture. We further conduct an ablation study to validate our design principles. ELMO's fast inference time makes it well-suited for real-time applications, exemplified in our demo video featuring live streaming and interactive gaming scenarios. Furthermore, we contribute a high-quality LiDAR-mocap synchronized dataset comprising 20 different subjects performing a range of motions, which can serve as a valuable resource for future research. The dataset and evaluation code are available at https://movin3d.github.io/ELMO_SIGASIA2024/
- 19 November 2024
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Graphics
- Vol. 43 (6) , 1-14
Open AccessELMO: Enhanced Real-time LiDAR Motion Capture through Upsampling
This paper introduces ELMO, a real-time upsampling motion capture framework designed for a single LiDAR sensor. Modeled as a conditional autoregressive transformer-based upsampling motion generator, ELMO achieves 60 fps motion capture from a 20 fps LiDAR point cloud sequence. The key feature of ELMO is the coupling of the self-attention mechanism with thoughtfully designed embedding modules for motion and point clouds, significantly elevating the motion quality. To facilitate accurate motion capture, we develop a one-time skeleton calibration model capable of predicting user skeleton off-sets from a single-frame point cloud. Additionally, we introduce a novel data augmentation technique utilizing a LiDAR simulator, which enhances global root tracking to improve environmental understanding. To demonstrate the effectiveness of our method, we compare ELMO with state-of-the-art methods in both image-based and point cloud-based motion capture. We further conduct an ablation study to validate our design principles. ELMO's fast inference time makes it well-suited for real-time applications, exemplified in our demo video featuring live streaming and interactive gaming scenarios. Furthermore, we contribute a high-quality LiDAR-mocap synchronized dataset comprising 20 different subjects performing a range of motions, which can serve as a valuable resource for future research. The dataset and evaluation code are available at https://movin3d.github.io/ELMO_SIGASIA2024/
- Open Access
- 25 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-2
MOVIN TRACIN' : Move Outside the Box
We introduce MOVIN TRACIN’, an advanced AI-driven motion capture system that leverages a single LiDAR sensor for precise, real-time, full-body tracking. TRACIN’ streamlines the user experience by eliminating the need for traditional suit markers or body-mounted sensors, while delivering motion capture accuracy comparable to conventional systems. The system captures full-body motion at 60 frames per second by upscaling from a 20 fps LiDAR point cloud sequence, making it particularly effective for applications demanding high responsiveness, such as live streaming, metaverse platforms, gaming, and interactive media art. Unlike traditional motion capture systems that often require multiple sensors or high-quality cameras with extensive calibration, our approach uses a single LiDAR sensor with instant calibration. This significantly reduces setup costs, time, and effort, making the system accessible to general users without the need for expert mocap engineers. To address the limitations of a single LiDAR setup, such as self-occlusion, we utilize a generative AI model that generates the most plausible output pose from the learned motion distribution, even with sparse or incomplete input point clouds. Our model maintains real-time performance with a fast inference time of around 50 ms, ensuring suitability for real-time applications. MOVIN TRACIN’ also supports an API-SDK for developers, enabling easy integration and streaming to game engines such as Unity and Unreal. In summary, MOVIN TRACIN’ represents a significant advancement in motion capture technology, offering:
- 25 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-2
Open AccessMOVIN TRACIN' : Move Outside the Box
We introduce MOVIN TRACIN’, an advanced AI-driven motion capture system that leverages a single LiDAR sensor for precise, real-time, full-body tracking. TRACIN’ streamlines the user experience by eliminating the need for traditional suit markers or body-mounted sensors, while delivering motion capture accuracy comparable to conventional systems. The system captures full-body motion at 60 frames per second by upscaling from a 20 fps LiDAR point cloud sequence, making it particularly effective for applications demanding high responsiveness, such as live streaming, metaverse platforms, gaming, and interactive media art. Unlike traditional motion capture systems that often require multiple sensors or high-quality cameras with extensive calibration, our approach uses a single LiDAR sensor with instant calibration. This significantly reduces setup costs, time, and effort, making the system accessible to general users without the need for expert mocap engineers. To address the limitations of a single LiDAR setup, such as self-occlusion, we utilize a generative AI model that generates the most plausible output pose from the learned motion distribution, even with sparse or incomplete input point clouds. Our model maintains real-time performance with a fast inference time of around 50 ms, ensuring suitability for real-time applications. MOVIN TRACIN’ also supports an API-SDK for developers, enabling easy integration and streaming to game engines such as Unity and Unreal. In summary, MOVIN TRACIN’ represents a significant advancement in motion capture technology, offering:
- 1 October 2023
- Published by Wiley in Computer Graphics Forum
- Vol. 42 (7)
MOVIN: Real‐time Motion Capture using a Single LiDAR
Recent advancements in technology have brought forth new forms of interactive applications, such as the social metaverse, where end users interact with each other through their virtual avatars. In such applications, precise full‐body tracking is essential for an immersive experience and a sense of embodiment with the virtual avatar. However, current motion capture systems are not easily accessible to end users due to their high cost, the requirement for special skills to operate them, or the discomfort associated with wearable devices. In this paper, we present MOVIN, the data‐driven generative method for real‐time motion capture with global tracking, using a single LiDAR sensor. Our autoregressive conditional variational autoencoder (CVAE) model learns the distribution of pose variations conditioned on the given 3D point cloud from LiDAR. As a central factor for high‐accuracy motion capture, we propose a novel feature encoder to learn the correlation between the historical 3D point cloud data and global, local pose features, resulting in effective learning of the pose prior. Global pose features include root translation, rotation, and foot contacts, while local features comprise joint positions and rotations. Subsequently, a pose generator takes into account the sampled latent variable along with the features from the previous frame to generate a plausible current pose. Our framework accurately predicts the performer's 3D global information and local joint details while effectively considering temporally coherent movements across frames. We demonstrate the effectiveness of our architecture through quantitative and qualitative evaluations, comparing it against state‐of‐the‐art methods. Additionally, we implement a real‐time application to showcase our method in real‐world scenarios. MOVIN dataset is available at https://movin3d.github.io/movin_pg2023/https://movin3d.github.io/movin_pg2023/">https://movin3d.github.io/movin_pg2023/.
- 1 October 2023
- Published by Wiley in Computer Graphics Forum
- Vol. 42 (7)
MOVIN: Real‐time Motion Capture using a Single LiDAR
Recent advancements in technology have brought forth new forms of interactive applications, such as the social metaverse, where end users interact with each other through their virtual avatars. In such applications, precise full‐body tracking is essential for an immersive experience and a sense of embodiment with the virtual avatar. However, current motion capture systems are not easily accessible to end users due to their high cost, the requirement for special skills to operate them, or the discomfort associated with wearable devices. In this paper, we present MOVIN, the data‐driven generative method for real‐time motion capture with global tracking, using a single LiDAR sensor. Our autoregressive conditional variational autoencoder (CVAE) model learns the distribution of pose variations conditioned on the given 3D point cloud from LiDAR. As a central factor for high‐accuracy motion capture, we propose a novel feature encoder to learn the correlation between the historical 3D point cloud data and global, local pose features, resulting in effective learning of the pose prior. Global pose features include root translation, rotation, and foot contacts, while local features comprise joint positions and rotations. Subsequently, a pose generator takes into account the sampled latent variable along with the features from the previous frame to generate a plausible current pose. Our framework accurately predicts the performer's 3D global information and local joint details while effectively considering temporally coherent movements across frames. We demonstrate the effectiveness of our architecture through quantitative and qualitative evaluations, comparing it against state‐of‐the‐art methods. Additionally, we implement a real‐time application to showcase our method in real‐world scenarios. MOVIN dataset is available at https://movin3d.github.io/movin_pg2023/https://movin3d.github.io/movin_pg2023/">https://movin3d.github.io/movin_pg2023/.
- Open Access
MOVIN: Real-time Motion Capture using a Single LiDAR
Recent advancements in technology have brought forth new forms of interactive applications, such as the social metaverse, where end users interact with each other through their virtual avatars. In such applications, precise full-body tracking is essential for an immersive experience and a sense of embodiment with the virtual avatar. However, current motion capture systems are not easily accessible to end users due to their high cost, the requirement for special skills to operate them, or the discomfort associated with wearable devices. In this paper, we present MOVIN, the data-driven generative method for real-time motion capture with global tracking, using a single LiDAR sensor. Our autoregressive conditional variational autoencoder (CVAE) model learns the distribution of pose variations conditioned on the given 3D point cloud from LiDAR.As a central factor for high-accuracy motion capture, we propose a novel feature encoder to learn the correlation between the historical 3D point cloud data and global, local pose features, resulting in effective learning of the pose prior. Global pose features include root translation, rotation, and foot contacts, while local features comprise joint positions and rotations. Subsequently, a pose generator takes into account the sampled latent variable along with the features from the previous frame to generate a plausible current pose. Our framework accurately predicts the performer's 3D global information and local joint details while effectively considering temporally coherent movements across frames. We demonstrate the effectiveness of our architecture through quantitative and qualitative evaluations, comparing it against state-of-the-art methods. Additionally, we implement a real-time application to showcase our method in real-world scenarios. MOVIN dataset is available at \url{https://movin3d.github.io/movin_pg2023/}.
Open AccessMOVIN: Real-time Motion Capture using a Single LiDAR
Recent advancements in technology have brought forth new forms of interactive applications, such as the social metaverse, where end users interact with each other through their virtual avatars. In such applications, precise full-body tracking is essential for an immersive experience and a sense of embodiment with the virtual avatar. However, current motion capture systems are not easily accessible to end users due to their high cost, the requirement for special skills to operate them, or the discomfort associated with wearable devices. In this paper, we present MOVIN, the data-driven generative method for real-time motion capture with global tracking, using a single LiDAR sensor. Our autoregressive conditional variational autoencoder (CVAE) model learns the distribution of pose variations conditioned on the given 3D point cloud from LiDAR.As a central factor for high-accuracy motion capture, we propose a novel feature encoder to learn the correlation between the historical 3D point cloud data and global, local pose features, resulting in effective learning of the pose prior. Global pose features include root translation, rotation, and foot contacts, while local features comprise joint positions and rotations. Subsequently, a pose generator takes into account the sampled latent variable along with the features from the previous frame to generate a plausible current pose. Our framework accurately predicts the performer's 3D global information and local joint details while effectively considering temporally coherent movements across frames. We demonstrate the effectiveness of our architecture through quantitative and qualitative evaluations, comparing it against state-of-the-art methods. Additionally, we implement a real-time application to showcase our method in real-world scenarios. MOVIN dataset is available at \url{https://movin3d.github.io/movin_pg2023/}.
- Open Access
- 1 September 2022
- Published by Korea Computer Graphics Society in Journal of the Korea Computer Graphics Society
- Vol. 28 (4) , 1-11
K-SMPL: Korean Body Measurement Data Based Parametric Human Model
사단법인 한국컴퓨터그래픽스학회의 논문지로서 그래픽스 분야와 그에 연관된 연구영역의 새로운 발견과 최첨단 연구결과를 발표하는 토론의 장입니다. 학회를 대표하는 학술지인 논문지는 1995년에 논문지 발간을 시작으로 2000년부터 3월, 6월, 9월, 12월 총 4회 발간되었고, 2015년부터는 KCGS 학술대회 특별호를 7월에 발간하면서 총 5회 발간되고 있습니다.
- 1 September 2022
- Published by Korea Computer Graphics Society in Journal of the Korea Computer Graphics Society
- Vol. 28 (4) , 1-11
Open AccessK-SMPL: Korean Body Measurement Data Based Parametric Human Model
사단법인 한국컴퓨터그래픽스학회의 논문지로서 그래픽스 분야와 그에 연관된 연구영역의 새로운 발견과 최첨단 연구결과를 발표하는 토론의 장입니다. 학회를 대표하는 학술지인 논문지는 1995년에 논문지 발간을 시작으로 2000년부터 3월, 6월, 9월, 12월 총 4회 발간되었고, 2015년부터는 KCGS 학술대회 특별호를 7월에 발간하면서 총 5회 발간되고 있습니다.