Junhee Seok

Learning expressive and multimodal policies is essential for solving complex continuous control tasks. However, most reinforcement learning (RL) algorithms rely on unimodal or factorized Gaussian policies, limiting their representational flexibility. While soft actor-critic (SAC) encourages stochasticity through the maximum entropy RL framework, its reverse KL-based update biases the policy toward dominant modes, often resulting in mode collapse. In this work, we propose Fokker-Planck SAC (FP-SAC), a principled policy optimization algorithm that formulates soft policy improvement through stochastic differential equations and derives a distribution-level objective from the corresponding Fokker-Planck (FP) equation. By minimizing the residual of this partial differential equationusing physics-informed learning and leveraging the expressiveness of normalizing flows, FP-SAC enables stable and accurate learning of multimodal policies. Experiments on multigoal environments, MuJoCo and Meta-World benchmarks, demonstrate that FP-SAC captures multimodal behaviors more effectively and achieves higher task performance with greater stability compared to existing approaches.

In deep reinforcement learning (RL), learning stochastic and multimodal policies are crucial for various tasks, but most continuous control algorithms model the policy using a deterministic or unimodal Gaussian distribution. Despite being designed to inherently learn a stochastic policy under the maximum entropy RL framework, soft actor–critic (SAC) also uses a factorized Gaussian policy for tractable optimization, which not only restricts the expressiveness of the policy but also ignores correlations among the components of the action vector. In this article, we revisit the approach of employing normalizing flow for SAC policy, justified by the change of variable theorem, and then propose a state-dependent nonvolume preserving (SD-NVP) architecture suitable for SAC learning. In addition, we introduce a generative adversarial SAC (GASAC) that implicitly defines and optimizes various divergences without calculating the normalization constant through a generative adversarial loss. Experimental results on multigoal environment and MuJoCo continuous control tasks suite demonstrate that GASAC model multimodal policy and learn policy more stably in terms of cumulative return.

The advancement of multimedia and artificial intelligence (AI) technologies has dismantled the barriers of information sharing, yet it has also ushered in a double-edged sword: a surge in the spread of fake information. In this context, there is a growing need for research on the detection of 'cheapfakes,' which are low-cost fake media, known for their ease of creation. This paper proposes a multi-stage deep learning process designed to effectively detect the diverse and rapidly evolving nature of cheapfakes. A single-step deep learning model faces limitations in distinguishing various types of cheapfakes, necessitating the application of a complex deep learning model approach to detect subtle Out-of-Context (OOC) phenomena. This study employs models based on Bidirectional Encoder Representations from Transformers (BERT) and stable diffusion technologies to approach cheapfake detection. Through the ACM ICMR 2024 challenge, the performance of this model was evaluated on a real dataset, achieving an accuracy of 71.9% in Task 1, an improvement of 7% over previous methods, and an accuracy of 55.7% in Task 2. These results are expected to make a significant contribution to the development of strategies for creating and countering cheapfakes. Additionally, this research aims to contribute to the detection of OOC media misuse through this challenge.

Recent studies have extensively used deep learning algorithms to analyze gene expression to predict disease diagnosis, treatment effectiveness, and survival outcomes. Survival analysis studies on diseases with high mortality rates, such as cancer, are indispensable. However, deep learning models are plagued by overfitting owing to the limited sample size relative to the large number of genes. Consequently, the latest style-transfer deep generative models have been implemented to generate gene expression data. However, these models are limited in their applicability for clinical purposes because they generate only transcriptomic data. Therefore, this study proposes ctGAN, which enables the combined transformation of gene expression and survival data using a generative adversarial network (GAN). ctGAN improves survival analysis by augmenting data through style transformations between breast cancer and 11 other cancer types. We evaluated the concordance index (C-index) enhancements compared with previous models to demonstrate its superiority. Performance improvements were observed in nine of the 11 cancer types. Moreover, ctGAN outperformed previous models in seven out of the 11 cancer types, with colon adenocarcinoma (COAD) exhibiting the most significant improvement (median C-index increase of ~15.70%). Furthermore, integrating the generated COAD enhanced the log-rank p-value (0.041) compared with using only the real COAD (p-value = 0.797). Based on the data distribution, we demonstrated that the model generated highly plausible data. In clustering evaluation, ctGAN exhibited the highest performance in most cases (89.62%). These findings suggest that ctGAN can be meaningfully utilized to predict disease progression and select personalized treatments in the medical field.

Understanding the intricate relationships between diseases is critical for both prevention and recovery. However, there is a lack of suitable methodologies for exploring the precedence relationships within multiple censored time-to-event data, resulting in decreased analytical accuracy. This study introduces the Censored Event Precedence Analysis (CEPA), which is a nonparametric Bayesian approach suitable for understanding the precedence relationships in censored multivariate events. CEPA aims to analyze the precedence relationships between events to predict subsequent occurrences effectively. We applied CEPA to neonatal data from the National Health Insurance Service, identifying the precedence relationships among the seven most commonly diagnosed diseases categorized by the International Classification of Diseases. This analysis revealed a typical diagnostic sequence, starting with respiratory diseases, followed by skin, infectious, digestive, ear, eye, and injury-related diseases. Furthermore, simulation studies were conducted to demonstrate CEPA suitability for censored multivariate datasets compared to traditional models. The performance accuracy reached 76% for uniform distribution and 65% for exponential distribution, showing superior performance in all four tested environments. Therefore, the statistical approach based on CEPA enhances our understanding of disease interrelationships beyond competitive methodologies. By identifying disease precedence with CEPA, we can preempt subsequent disease occurrences and propose a healthcare system based on these relationships.

The simplified molecular-input line-entry system (SMILES) has been utilized in a variety of artificial intelligence analyses owing to its capability of representing chemical structures using line notation. However, its ease of representation is limited, which has led to the proposal of BigSMILES as an alternative method suitable for the representation of macromolecules. Nevertheless, research on BigSMILES remains limited due to its preprocessing requirements. Thus, this study proposes a conversion workflow of BigSMILES, focusing on its automated generation from SMILES representations of homopolymers. BigSMILES representations for 4,927,181 records are provided, thereby enabling its immediate use for various research and development applications. Our study presents detailed descriptions on a validation process to ensure the accuracy, interchangeability, and robustness of the conversion. Additionally, a systematic overview of utilized codes and functions that emphasizes their relevance in the context of BigSMILES generation are produced. This advancement is anticipated to significantly aid researchers and facilitate further studies in BigSMILES representation, including potential applications in deep learning and further extension to complex structures such as copolymers.

Breast cancer has the highest incidence and death rate among women; moreover, its metastasis to other organs increases the mortality rate. Since several studies have reported gene expression and cancer prognosis to be related, the study of breast cancer metastasis using gene expression is crucial. To this end, a novel deep neural network architecture, deep learning-based cancer metastasis estimator (DeepCME), is proposed in this paper for predicting breast cancer metastasis. However, the problem of overfitting occurs frequently while training deep learning models using gene expression data because they contain a large number of genes and the sample size is rather small. To address overfitting, several regularization methods are implemented, such as L1 penalty, batch normalization, and dropout. To demonstrate the superior performance of our model, area under curve (AUC) scores are evaluated and then compared with five baseline models: logistic regression, support vector classifier (SVC), random forest, decision tree, and k-nearest neighbor. Considering results, DeepCME demonstrates the highest average AUC scores in most cross-validation cases, and the average AUC score of DeepCME is 0.754, which is approximately 12.9% higher than SVC, the second-best model. In addition, the 30 most significant genes related to breast cancer metastasis are identified based on DeepCME results and some are discussed in further detail considering the reports from some previous medical studies. Considering the high expense involved in measuring the expression of a single gene, the ability to develop the cost-effective and time-efficient tests using only a few key genes is valuable. Based on this study, we expect DeepCME to be utilized clinically for predicting breast cancer metastasis and be applied to other types of cancer as well after further research.

Owing to the recent explosive expansion of deep learning, several challenging problems in a variety of fields have been handled by deep learning, yet deep learning methods have been limited in their application to the network estimation problem. While network estimation has a possibility to be a useful method in various domains, deep learning-based network estimation has a limitation in that the number of variables must be fixed and the estimation cannot be performed by convolutional layers. In this study, we propose a Generative Adversarial Network (GAN) based method, called Inverse Covariance Estimating GAN (ICEGAN), which can alleviate these limitations. In ICEGAN, the concepts in Cycle-Consistent Adversarial Networks are modified for the problem and employed to adopt gene expression data. Additionally, the Monte Carlo approach is used to address the fixed size in the network estimation process. Thus, sub-networks are sampled from the entire network and estimated by ICEGAN; then, the Monte Carlo approach reconstructs the entire network with the estimations. In the simulation study, ICEGAN demonstrated superior performances compared to conventional models and the ordinary GAN model in estimating networks. Specifically, ICEGAN outperformed an ordinary GAN by 85.9% on average when the models were evaluated using the area under curve. In addition, ICEGAN performed gene network estimation of breast cancer using a gene expression dataset. Consequently, ICEGAN demonstrated promising results, considering the deep learning-based network estimation and the proposed Monte Carlo approach for GAN models, both of which can be expanded to other domains.

The significance of simulation has been increasing in device design due to the cost of real test. The accuracy of the simulation increases as the resolution of the simulation increases. However, the high-resolution simulation is not suited for actual device design because the amount of computing exponentially increases as the resolution increases. In this study, we introduce a model that predicts high-resolution outcomes using low-resolution calculated values which successfully achieves high simulation accuracy with low computational cost. The fast residual learning super-resolution (FRSR) convolutional network model is a model that we introduced that can simulate electromagnetic fields of optical. Our model achieved high accuracy when using the super-resolution technique on a 2D slit array under specific circumstances and achieved an approximately 18 times faster execution time than the simulator. To reduce the model training time and enhance performance, the proposed model shows the best accuracy (R2: 0.9941) by restoring high-resolution images using residual learning and a post-upsampling method to reduce computation. It has the shortest training time among the models that use super-resolution (7000 s). This model addresses the issue of temporal limitations of high-resolution simulations of device module characteristics.