Chang-Hun Kim

Internet memes have become an essential tool for online communication and interaction on social media platforms. They reflect a wide range of emotional and cultural elements, thereby serving as effective vectors for rapid information exchange. In this digital age, the popularity and influence of memes highlight the need for innovative communication methods. This need has resulted in the exploration of technologies that are capable of capturing and expressing this unique cultural phenomenon. This research presents an innovative Chinese meme generation system that is capable of generating contextually relevant Chinese meme-style text using input images and dynamically predicting the best location of the text within the meme images. Our approach combines the tasks of object detection and image captioning, thereby drawing on the strengths of the fully convolutional one-stage object detection (FCOS) and multimodal pretrained models, ChineseCLIP and GPT-2, respectively. To facilitate this task, we constructed a dataset comprising meme images, textual content, and location information of the text within the images. Through comprehensive experiments, we validated the proficiency of our model by not only generating contextually relevant text that corresponds with the image but also locating it to improve the aesthetic allure and communicative efficacy of the memes. This study propels the automation of Chinese meme generation and provides new perspectives on the interplay between image understanding and multimodal learning.

Conducting realistic interactions while communicating efficiently in online conferences is highly desired but challenging. In this work, we propose a novel pixel-style virtual avatar for interacting with virtual objects in virtual conferences that can be generated in real-time. It consists of a 2D segmented head video stream for real-time facial expressions and a 3D point cloud body for realistic interactions, both of which are generated from RGB video input of a monocular webcam. We obtain a human-only video stream with a human matting method and generate the 3D avatar’s arms with a 3D pose estimation method, which improves the stereoscopic realism and sense of interaction of conference participants while interacting with virtual objects. Our approach fills the gap between 2D video conferences and 3D virtual avatars and combines the advantages of both. We evaluated our pixel-style avatar by conducting a user study; the result proved that the efficiency of our method is superior to other various existing avatar types.

사단법인 한국컴퓨터그래픽스학회의 논문지로서 그래픽스 분야와 그에 연관된 연구영역의 새로운 발견과 최첨단 연구결과를 발표하는 토론의 장입니다. 학회를 대표하는 학술지인 논문지는 1995년에 논문지 발간을 시작으로 2000년부터 3월, 6월, 9월, 12월 총 4회 발간되었고, 2015년부터는 KCGS 학술대회 특별호를 7월에 발간하면서 총 5회 발간되고 있습니다.

In this paper, we propose an angular momentum-based advection technique that can express the turbulent foam effect. The motion of foam particles, which are strongly bound to the motion of the underlying fluid, is viscous, and sometimes clumping problems occur. This problem is a decisive factor that makes it difficult to express realistic foam effects. Since foam particles, which are secondary effects, depend on the motion of the underlying water, in order to exaggerate the foam effects or express more lively foam effects, it is inevitable to tune the motion of the underlying water and then readjust the foam particles. Because of such a cumbersome process, the readjustment of the foam effects requires a change in the motion of the underlying water, and it is not easy to produce such a scene because the water and foam effects must change at the same time. In this paper, we present a method to maintain angular momentum-based force from water particles without tuning the motion of the underlying water. We can restore the lost turbulent flow by additional advection of foam particles based on this force. In addition, our method can be integrated with screen-space projection frameworks, allowing us to fully embrace all the advantages of this approach. In this paper, the turbulence of the foam particles was improved by minimizing the viscous motion of the foam particles without tuning the motion of the underlying water, and as a result, lively foam effects can be expressed.

Internet memes have become widely used by people for online communication and interaction, particularly through social media. Interest in meme-generation research has been increasing rapidly. In this study, we address the problem of meme generation as an image captioning task, which uses an encoder–decoder architecture to generate Chinese meme texts that match image content. First, to train the model on the characteristics of Chinese memes, we collected a dataset of 3,000 meme images with 30,000 corresponding humorous Chinese meme texts. Second, we introduced a Chinese meme generation system that can generate humorous and relevant texts from any given image. Our system used a pre-trained ResNet-50 for image feature extraction and a state-of-the-art transformer-based GPT-2 model to generate Chinese meme texts. Finally, we combined the generated text and images to form common image memes. We performed qualitative evaluations of the generated Chinese meme texts through different user studies. The evaluation results revealed that the Chinese memes generated by our model were indistinguishable from real ones.

We propose a novel method for addressing the problem of efficiently generating a highly refined normal map for screen-space fluid rendering. Because the process of filtering the normal map is crucially important to ensure the quality of the final screen-space fluid rendering, we employ a conditional generative adversarial network (cGAN) as a filter that learns a deep normal map representation, thereby refining the low-quality normal map. In particular, we have designed a novel loss function dedicated to refining the normal map information, and we use a specific set of auxiliary features to train the cGAN generator to learn features that are more robust with respect to edge details. Additionally, we constructed a dataset of six different typical scenes to enable effective demonstrations of multitype fluid simulation. Experiments indicated that our generator was able to infer clearer and more detailed features for this dataset than a basic screen-space fluid rendering method. Moreover, in some cases, the results generated by our method were even smoother than those generated by the conventional surface reconstruction method. Our method improves the fluid rendering results via the high-quality normal map while preserving the advantages of the screen-space fluid rendering methods and the traditional surface reconstruction methods, including that of the computation time being independent of the number of simulation particles and the spatial resolution being related only to image resolution.

Surface tension has a great influence on the shape of the fluid interface, and is an important physical characteristic in expressing not only liquids but also liquid metals such as mercury and gallium. In the field of physics-based particle fluid simulations, it is a challenging problem to express the high surface tension generated by fluid-air or fluid-solid interaction in real time. The main reasons for this are (1) The magnitude of the force that can be stably expressed in real-time fluid simulation is limited, so when the magnitude of the surface tension increases at a large time-step, the simulation stability decreases, and (2) If we use a small time-step, a stronger force can be expressed. However, it becomes difficult to operate in real time because the computational cost increases. Techniques were proposed to solve this problem for a few specific scenes, but there has not yet been a general approach that can reliably express high surface tension in various scenarios. In this paper, we propose a real-time particle-based fluid simulation framework that can efficiently and stably express high surface tension. Unlike the previous methods, we newly model the surface tension so that the strong surface tension force generated in the droplet area with a large curvature is applied evenly in the normal and tangent directions regardless of the size of the droplet. We also propose new pressure constraints that converge quickly and accurately using this force. Our method can be effectively used in various physics-based simulation scenarios because it can easily express and control surface tension effects that appear in materials such as liquid metal as well as water.

In this paper, we propose a highly efficient method for synthesizing high-resolution(HR) smoke simulations based on deep learning. A major issue for physics-based HR fluid simulations is that they require large amounts of physical memory and long execution times. In recent years, this issue has been addressed by developing deep-learning-based super-resolution(SR) methods that convert low-resolution(LR) fluid simulation results to HR(High-resolution) versions. However, these methods were not very efficient because they performed operations even in areas with low density or no density. In this paper, we propose a method that can maximize its efficiency by introducing a downscaled and binarized adaptive octree. However, even if it is divided by octree, because the number of nodes increases when the resolution of the simulation space is large, we reduce the size of the space by multiscaling and at the same time perform binarization to preserve the density that may be lost in this process. The octree calculated in this process has a structure similar to that of a multigrid solver, and the octree calculated at coarse resolution is restored to its original size and used for HR expression. Finally, we apply the SR process only to those areas having significant density values. Using the proposed method, the SR process is significantly faster and the memory efficiency is improved. The performance of our method is compared with that of an existing SR method to demonstrate its efficiency.

This paper proposes a new data-driven neural network-based fire-flake simulation model. Our model trains a neural network using precomputed fire simulation data. The trained neural network model generates fire flakes in appropriate locations and infers their velocity to make them appear natural to their surroundings. The neural network model consists of a fire-flake generator and a velocity modifier. The fire-flake generator uses the velocity, temperature, and density fields of the precomputed fire simulation as inputs to determine the locations at which natural fire flakes would be generated. The velocity modifier takes the velocity field of the precomputed fire simulation as input and infers the velocity of the generated fire flakes so that they appear natural relative to the flame motions and surroundings. Our method adopts a neural network to efficiently improve the fire-flake simulation, enhancing the performance while maintaining the visual quality. Our model is approximately three times faster than the traditional fire-flake model. In particular, our model is 30 times faster in the velocity modification step. Our method is also easier to implement than the existing physically based fire-flake simulation method and can reduce the time spent by artists and developers on their applications.