Bonggyun Ko

Prompt escalation can increase the resource requirements of lightweight large language models (LLMs) without improving predictive performance. We evaluated four instruction-tuned 2–4B models on Grade School Math 8K (GSM8K), CommonsenseQA (CSQA), Recognizing Textual Entailment (RTE), and the binary Stanford Sentiment Treebank (SST-2). Zero-shot (ZS), few-shot (FS), chain-of-thought (CoT), and few-shot CoT (FS+CoT) were compared across 64 conditions using predictive performance, tokens, latency, and stochastic response consistency. Demonstrations came from training splits; FS+CoT used worked rationales with automatic screening and a partial manual audit. Latency was measured separately with synchronization, warm-up exclusion, and counterbalanced prompt order. After Holm adjustment, 12.5% of predictive-performance contrasts were significant, and ZS was significantly outperformed in 1 of 48 contrasts, compared with significant differences in 100% of total-token and 75.0% of latency contrasts. Strategy rankings varied by model and task. A single auxiliary 7B model showed no significant accuracy gain over ZS but did not establish a general scale effect. Scenario-weight sensitivity frequently favored ZS, with model–task exceptions. These results support ZS as a low-overhead reference within the four evaluated benchmarks and the stated model, quantization, prompt, and decoding settings. The rankings and guidelines have not been validated for summarization, code generation, or multi-turn dialogue.

Kim J, et al. J Korean Med Sci. 2024 Jan;39:e56. https://doi.org/10.3346/jkms.2024.39.e56

Keyword extraction is an effective way to quickly identify key elements in text. It can accelerate the identification of key factors that play a role in accidents when applied to incident report analysis. Our research presents an innovative process for extracting keywords from incident reports with the pre-trained natural language processing models. We utilized fine-tuning techniques to integrate a BiLSTM-CRF with a fully-connected layer and pre-trained natural language models. The process of extracting keyphrases is approached as a task of labeling sequences. To analyze incident reports from Korea, we employ pre-trained models customized for the Korean context, such as KoBERT and KoELECTRA. Our approach is assessed using a range of metrics, including accuracy, area under the curve (AUC), F1-score, slot error rate (SER), and simple matching coefficient (SMC). In contrast to traditional approaches which mainly concentrate on document summarization, our research provides a distinct method tailored to identifying falling objects as the main cause of accidents. Our findings demonstrate that the ELECTRA-based model with a BiLSTM-CRF outperforms other models, achieving an accuracy of 0.943, an AUC of 0.991, and a low SER of 0.075. The F1-score and SMC closely resemble the BERT-based model with a BiLSTM-CRF, with no significant differences observed within the 95% confidence interval. These results underscore the potential of fine-tuning pre-trained models for post-hoc traffic accident analysis. This method offers a swift preliminary step to identify the key factors before human analysis, presenting a multifaceted strategy to enhance road safety and prevent accidents.

Pneumonia is an inflammation of the lungs caused by pathogens or autoimmune diseases, with about 450 million patients worldwide each year. Chest X–ray analysis is the most common radiographic method used to diagnose pneumonia, and advances in deep learning have led to the availability of high-dimensional image, audio, and video data. Deep learning is being applied in many fields, including the medical field, where numerous researchers have attempted to use it for computer-aided diagnosis. Recently, with the appearance of generative adversarial networks, it is possible to generate plausible and realistic images. In this paper, we combined cycle Generative Adversarial Networks (GANs) and conditional GANs, which are extensions of GANs, to convert the domains between images and generate images of the intermediate domains. We conducted the domain change between pneumonia images and normal images by applying our framework to a Chest X–ray image dataset. We evaluated the domain change by redefining the ResNet152-based classifier, and we generated the pneumonia progression images by inputting a value between two domains in the conditional vector of the generator. We then evaluated the ability of the trained GANs by comparing the original dataset with the generated dataset, and generated plausible progression images of pneumonia.

We have investigated the effects of the secondary defects caused by ion implantation on wafer strength. The change in wafer strength with the ion dose has been examined after implanting phosphorus or (BF₂)⁺ ions into wafers with and without heat treatment. Ion implantation defects have been observed using transmission electron microscopy after ion implantation with and without subsequent annealing. The three-point bending tests carried out with the ion implanted samples show that the upper yield stress, which represents wafer strength, gradually decreases with the increasing dose in the case of phosphorus ions and rapidly decreases in the case of (BF₂)⁺ ions due to the formation of secondary defects. In addition, the strength of the wafers with a low oxygen concentration is lower than that of the wafers with a high oxygen concentration. However, the rate of decrease in wafer strength with increasing ion dose is not affected by the oxygen concentration; it only depended on the implanted element.

Cardiovascular diseases are the leading cause of death globally. The ECG is the most commonly used tool for diagnosing cardiovascular diseases, and, recently, there are a number of attempts to use deep learning to analyze ECG. In this study, we propose a method for performing multi-label classification on standard ECG (12-lead with duration of 10 s) data. We used the ResNet model that can perform residual learning as a base model for classification in this work, and we tried to improve performance through SE-ResNet, which added squeeze and excitation blocks on the plain ResNet. As a result of the experiment, it was possible to induce overall performance improvement through squeeze and excitation blocks. In addition, the random k-labelsets (RAKEL) algorithm was applied to improve the performance in multi-label classification problems. As a result, the model that applied soft voting through the RAKEL algorithm to SE-ResNet-34 represented the best performance, and the average performances according to the number of label divisions k were achieved 0.99%, 88.49%, 92.43%, 90.54%, and 93.40% in exact match, accuracy, F1-score, precision, and recall, respectively.

Pneumonia is a form of acute respiratory infection commonly caused by germs, viruses, and fungi, and can prove fatal at any age. Chest X-rays is the most common technique for diagnosing pneumonia. There have been several attempts to apply transfer learning based on a Convolutional Neural Network to build a stable model in computer-aided diagnosis. Recently, with the appearance of an attention mechanism that automatically focuses on the critical part of the image that is crucial for the diagnosis of disease, it is possible to increase the performance of previous models. The goal of this study is to improve the accuracy of a computer-aided diagnostic approach that medical professionals can easily use as an auxiliary tool. In this paper, we proposed the attention-based transfer learning framework for efficient pneumonia detection in chest X-ray images. We collected features from three-types of pre-trained models, ResNet152, DenseNet121, ResNet18 as a role of feature extractor. We redefined the classifier for a new task and applied the attention mechanism as a feature selector. As a result, the proposed approach achieved accuracy, F-score, Area Under the Curve(AUC), precision and recall of 96.63%, 0.973, 96.03%, 96.23% and 98.46%, respectively.

Rapid and massive advances in deep learning have made it possible to address with issues with computer vision. In recent years, one type of generative model has emerged, generative adversarial networks (GAN), that enables creating realistic and plausible images. GAN allows for building competition models based on game theory that allows for modeling data probability distributions. Since the introduction of GAN, researchers have conducted many follow–up studies to apply and improve these models. In this article, using a global optimization technique called a genetic algorithm, we suggest the methodology restoring latent vector of pre–trained GAN and measure its performance as hyper–parameters and fitness functions; specifically, we utilize the mating and mutation rate as hyper–parameters of the genetic algorithms and use the mean squared error and the structural similarity as the fitness functions and evaluate their impact. We obtain image reconstruction results through experiment using the MNIST, Fashion–MNIST, Cifar10, and CelebA dataset, and we compare our method with the gradient descent method. We discuss limitations of these experiments and future research topics.

The relationship between slip dislocations generation due to scratches on silicon wafers and oxygen concentration has been investigated by three-point bending tests at high temperatures from 973 to 1123 K. We have measured the critical shear stresses for slip initiation caused by surface scratches as the onset of dislocation motions. In addition to the locking effect by oxygen atoms of dislocations, it was found that strain rate also has an important role on the value of the critical stress for slip generation caused by scratch. In order to take into account the effect of the strain rate, a new empirical equation as a function of strain rate for the estimation of critical shear stress was developed using the results with the three-point bending in this study. The equation obtained in this work indicates that ramp rates in the thermal process of wafer have a crucial effect on the slip dislocation generation. The estimation of critical shear stress of slip dislocation using this equation shows a good agreement with the experimental results of a heat treatment process.