Junsoo Lee
Junsoo Lee has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Jaegul Choo
KAIST
2 shared publications - Jeongho Kim
KAIST
2 shared publications - Min-Jung Kim
KAIST
2 shared publications - Chunggi Lee
Visual Computing Group, Harvard University, Cambridge, Massachusetts, USA
1 shared publication - Daesik Kim
NAVER WEBTOON AI, Gyeonggi-do, Republic of Korea
1 shared publication - Dohyun Kim
NAVER WEBTOON AI, Gyeonggi-do, Republic of Korea
1 shared publication
Latest Publications
- 21 November 2024
- Published by Springer Nature
- p. 326-342
TCAN: Animating Human Images with Temporally Consistent Pose Guidance Using Diffusion Models
- 21 November 2024
- Published by Springer Nature
- p. 326-342
TCAN: Animating Human Images with Temporally Consistent Pose Guidance Using Diffusion Models
- 26 October 2024
- Published by Springer Nature
- p. 41-57
SAVE: Protagonist Diversification with Structure Agnostic Video Editing
- 26 October 2024
- Published by Springer Nature
- p. 41-57
SAVE: Protagonist Diversification with Structure Agnostic Video Editing
- Open Access
TCAN: Animating Human Images with Temporally Consistent Pose Guidance using Diffusion Models
Pose-driven human-image animation diffusion models have shown remarkable capabilities in realistic human video synthesis. Despite the promising results achieved by previous approaches, challenges persist in achieving temporally consistent animation and ensuring robustness with off-the-shelf pose detectors. In this paper, we present TCAN, a pose-driven human image animation method that is robust to erroneous poses and consistent over time. In contrast to previous methods, we utilize the pre-trained ControlNet without fine-tuning to leverage its extensive pre-acquired knowledge from numerous pose-image-caption pairs. To keep the ControlNet frozen, we adapt LoRA to the UNet layers, enabling the network to align the latent space between the pose and appearance features. Additionally, by introducing an additional temporal layer to the ControlNet, we enhance robustness against outliers of the pose detector. Through the analysis of attention maps over the temporal axis, we also designed a novel temperature map leveraging pose information, allowing for a more static background. Extensive experiments demonstrate that the proposed method can achieve promising results in video synthesis tasks encompassing various poses, like chibi. Project Page: https://eccv2024tcan.github.io/
Open AccessTCAN: Animating Human Images with Temporally Consistent Pose Guidance using Diffusion Models
Pose-driven human-image animation diffusion models have shown remarkable capabilities in realistic human video synthesis. Despite the promising results achieved by previous approaches, challenges persist in achieving temporally consistent animation and ensuring robustness with off-the-shelf pose detectors. In this paper, we present TCAN, a pose-driven human image animation method that is robust to erroneous poses and consistent over time. In contrast to previous methods, we utilize the pre-trained ControlNet without fine-tuning to leverage its extensive pre-acquired knowledge from numerous pose-image-caption pairs. To keep the ControlNet frozen, we adapt LoRA to the UNet layers, enabling the network to align the latent space between the pose and appearance features. Additionally, by introducing an additional temporal layer to the ControlNet, we enhance robustness against outliers of the pose detector. Through the analysis of attention maps over the temporal axis, we also designed a novel temperature map leveraging pose information, allowing for a more static background. Extensive experiments demonstrate that the proposed method can achieve promising results in video synthesis tasks encompassing various poses, like chibi. Project Page: https://eccv2024tcan.github.io/
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 8242-8250
FlatGAN: A Holistic Approach for Robust Flat-Coloring in High-Definition with Understanding Line Discontinuity
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 8242-8250
Open AccessFlatGAN: A Holistic Approach for Robust Flat-Coloring in High-Definition with Understanding Line Discontinuity