Kyungseop Shin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Kyungseop Shin has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
- Ohyun Jo
Chungbuk National University
4 shared publications - Mincheol Seong2 shared publications
Ohyun Jo
2 shared publicationsDong Hyun Kim
Energy Storage Research Center Korea Institute of Science and Technology (KIST) 5 Hwarang-ro 14-gil Seongbuk-gu 02792 Seoul Republic of Korea
1 shared publicationDowon Kim
Department of Computer Science Department of Game Design and Development Sangmyung University Department of Computer Science Department of Computer Science Department of Computer Science Department of Computer Science Department of Computer Science Sangmyung University Sangmyung University Sangmyung University Sangmyung University Sangmyung University Sangmyung University
1 shared publication- Dowon Kim
Department of Computer Science, Sangmyung University, Seoul, South Korea
1 shared publication
Latest Publications
- Open Access
Optimal Power Allocation and Sub-Optimal Channel Assignment for Downlink NOMA Systems Using Deep Reinforcement Learning
In recent years, Non-Orthogonal Multiple Access (NOMA) system has emerged as a promising candidate for multiple access frameworks due to the evolution of deep machine learning, trying to incorporate deep machine learning into the NOMA system. The main motivation for such active studies is the growing need to optimize the utilization of network resources as the expansion of the internet of things (IoT) caused a scarcity of network resources. The NOMA addresses this need by power multiplexing, allowing multiple users to access the network simultaneously. Nevertheless, the NOMA system has few limitations. Several works have proposed to mitigate this, including the optimization of power allocation known as joint resource allocation(JRA) method, and integration of the JRA method and deep reinforcement learning (JRA-DRL). Despite this, the channel assignment problem remains unclear and requires further investigation. In this paper, we propose a deep reinforcement learning framework incorporating replay memory with an on-policy algorithm, allocating network resources in a NOMA system to generalize the learning. Also, we provide extensive simulations to evaluate the effects of varying the learning rate, batch size, type of model, and the number of features in the state.
Open AccessOptimal Power Allocation and Sub-Optimal Channel Assignment for Downlink NOMA Systems Using Deep Reinforcement Learning
In recent years, Non-Orthogonal Multiple Access (NOMA) system has emerged as a promising candidate for multiple access frameworks due to the evolution of deep machine learning, trying to incorporate deep machine learning into the NOMA system. The main motivation for such active studies is the growing need to optimize the utilization of network resources as the expansion of the internet of things (IoT) caused a scarcity of network resources. The NOMA addresses this need by power multiplexing, allowing multiple users to access the network simultaneously. Nevertheless, the NOMA system has few limitations. Several works have proposed to mitigate this, including the optimization of power allocation known as joint resource allocation(JRA) method, and integration of the JRA method and deep reinforcement learning (JRA-DRL). Despite this, the channel assignment problem remains unclear and requires further investigation. In this paper, we propose a deep reinforcement learning framework incorporating replay memory with an on-policy algorithm, allocating network resources in a NOMA system to generalize the learning. Also, we provide extensive simulations to evaluate the effects of varying the learning rate, batch size, type of model, and the number of features in the state.
- Open Access
- 4 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 183024-183037
Q-Learning-Based Medium Access Technology for Minimizing AoI in LoRa Wireless Relay Networks
LoRaWAN has emerged as a leading communication protocol for Low Power Wide Area Networks (LPWANs), gaining widespread adoption across diverse Internet of Things (IoT) deployments. Our approach integrates LoRa relay devices and Age of Information (AoI) metrics to enhance network performance. The algorithm dynamically adjusts Spreading Factors (SFs) based on network conditions, utilizing reinforcement learning techniques for optimal SF selection. Key innovations in this paper include the strategic use of LoRa relays, which is particularly effective for mitigating signal attenuation in long-distance communication scenarios. Another significant advancement is the utilization of AoI in two crucial aspects: as a component of the reinforcement learning algorithm and as an evaluation metric. This novel approach prioritizes data freshness in transmission decisions, enabling the algorithm to optimize communication based on the timeliness of information. Simulations demonstrate significant performance improvements over baseline algorithms, achieving average AoI reductions of 23% in high-density scenarios and 31% in high data transfer environments. These results highlight the effectiveness of combining AoI metrics and intelligent relay selection in improving LoRaWAN performance for IoT applications.
- 4 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 183024-183037
Open AccessQ-Learning-Based Medium Access Technology for Minimizing AoI in LoRa Wireless Relay Networks
LoRaWAN has emerged as a leading communication protocol for Low Power Wide Area Networks (LPWANs), gaining widespread adoption across diverse Internet of Things (IoT) deployments. Our approach integrates LoRa relay devices and Age of Information (AoI) metrics to enhance network performance. The algorithm dynamically adjusts Spreading Factors (SFs) based on network conditions, utilizing reinforcement learning techniques for optimal SF selection. Key innovations in this paper include the strategic use of LoRa relays, which is particularly effective for mitigating signal attenuation in long-distance communication scenarios. Another significant advancement is the utilization of AoI in two crucial aspects: as a component of the reinforcement learning algorithm and as an evaluation metric. This novel approach prioritizes data freshness in transmission decisions, enabling the algorithm to optimize communication based on the timeliness of information. Simulations demonstrate significant performance improvements over baseline algorithms, achieving average AoI reductions of 23% in high-density scenarios and 31% in high data transfer environments. These results highlight the effectiveness of combining AoI metrics and intelligent relay selection in improving LoRaWAN performance for IoT applications.
- Open Access
- 31 August 2024
- Published by The Korean Electrochemical Society - English Journal in Journal of Electrochemical Science and Technology
- Vol. 15 (3) , 414-420
Empirical Capacity Degradation Model for a Lithium-Ion Battery Based on Various C-Rate Charging Conditions
Lithium-ion batteries are widely used in many applications due to their high energy density, high efficiency, and excellent cycle ability. Once an unknown Li-ion battery is reusable, it is important to measure its lifetime and state of health. The most favorable measurement method is the cycle test, which is accurate but time- and capacity-consuming. In this study, instead of a cycle test, we present an empirical model based on the C-rate test to understand the state of health of the battery in a short time. As a result, we show that the partially accelerated charge/discharge condition of the Li-ion battery is highly effective for the degradation of battery capacity, even when half of the charge/discharge conditions are the same. This observation provides a measurable method for predicting battery reuse and future capacity degradation.
- 31 August 2024
- Published by The Korean Electrochemical Society - English Journal in Journal of Electrochemical Science and Technology
- Vol. 15 (3) , 414-420
Open AccessEmpirical Capacity Degradation Model for a Lithium-Ion Battery Based on Various C-Rate Charging Conditions
Lithium-ion batteries are widely used in many applications due to their high energy density, high efficiency, and excellent cycle ability. Once an unknown Li-ion battery is reusable, it is important to measure its lifetime and state of health. The most favorable measurement method is the cycle test, which is accurate but time- and capacity-consuming. In this study, instead of a cycle test, we present an empirical model based on the C-rate test to understand the state of health of the battery in a short time. As a result, we show that the partially accelerated charge/discharge condition of the Li-ion battery is highly effective for the degradation of battery capacity, even when half of the charge/discharge conditions are the same. This observation provides a measurable method for predicting battery reuse and future capacity degradation.
- 19 September 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Wireless Communications Letters
- Vol. 11 (12) , 2542-2545
Knowledge Transfer-Based Multiagent Q-Learning for Medium Access in Dense Cellular Networks
Inter-cell interference in dense cellular networks causes severe network performance degradation because the base stations (BSs) and devices cannot fully utilize all the network status information in practice. Reinforcement learning enables the wireless network to find an effective transmission strategy with limited information and the network entities to operate in a distributed manner. We herein propose a medium access algorithm based on multi-agent reinforcement learning by exchanging knowledge of transmissions from other cells. Specifically, a state mapping function between the device and the BS and a transfer value iteration are proposed. The BS transfers reward processes derived from inter-cell interference so that the partially observed medium of devices is expanded to monitor entire networks. Extensive simulation results show that simple knowledge transfer can help performance improvement in terms of network stability and throughput.
- 19 September 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Wireless Communications Letters
- Vol. 11 (12) , 2542-2545
Knowledge Transfer-Based Multiagent Q-Learning for Medium Access in Dense Cellular Networks
Inter-cell interference in dense cellular networks causes severe network performance degradation because the base stations (BSs) and devices cannot fully utilize all the network status information in practice. Reinforcement learning enables the wireless network to find an effective transmission strategy with limited information and the network entities to operate in a distributed manner. We herein propose a medium access algorithm based on multi-agent reinforcement learning by exchanging knowledge of transmissions from other cells. Specifically, a state mapping function between the device and the BS and a transfer value iteration are proposed. The BS transfers reward processes derived from inter-cell interference so that the partially observed medium of devices is expanded to monitor entire networks. Extensive simulation results show that simple knowledge transfer can help performance improvement in terms of network stability and throughput.
- Open Access
- 12 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 9, 30408-30416
Iterative Learning for Reliable Link Adaptation in the Internet of Underwater Things
Given the ever-increasing interest in the Internet of Underwater Things (IoUT), various studies are ongoing to solve some of the practical problems affecting the development of underwater wireless communication. The main problems are related to the use of acoustic waves in a water medium, in which extremely high propagation loss and drastic channel fluctuation are common. On the basis of hands-on experience and measurements made in real underwater environments, the conventional Adaptive Modulation and Coding (AMC), which uses the high correlation between SNR (Signal to Noise Ratio) and BER (Bit Error Rate), might not be affordable in underwater environments because the normal correlation between SNR and BER almost disappears altogether. This work therefore collectively takes into account multiple quality factors of communication at the same time by creating, analysing and validating the machine learning model to predict the most adequate communication parameters to solve the problem. The dataset of underwater wireless communication used in the learning models was obtained from measurements made in a real underwater environment near the Gulf of Incheon, South Korea, using a practical testbed designed and implemented by the authors. The estimated network throughput based on the communications parameters predicted using the machine learning models was enhanced by up to 25% compared with the conventional handcraft method.
- 12 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 9, 30408-30416
Open AccessIterative Learning for Reliable Link Adaptation in the Internet of Underwater Things
Given the ever-increasing interest in the Internet of Underwater Things (IoUT), various studies are ongoing to solve some of the practical problems affecting the development of underwater wireless communication. The main problems are related to the use of acoustic waves in a water medium, in which extremely high propagation loss and drastic channel fluctuation are common. On the basis of hands-on experience and measurements made in real underwater environments, the conventional Adaptive Modulation and Coding (AMC), which uses the high correlation between SNR (Signal to Noise Ratio) and BER (Bit Error Rate), might not be affordable in underwater environments because the normal correlation between SNR and BER almost disappears altogether. This work therefore collectively takes into account multiple quality factors of communication at the same time by creating, analysing and validating the machine learning model to predict the most adequate communication parameters to solve the problem. The dataset of underwater wireless communication used in the learning models was obtained from measurements made in a real underwater environment near the Gulf of Incheon, South Korea, using a practical testbed designed and implemented by the authors. The estimated network throughput based on the communications parameters predicted using the machine learning models was enhanced by up to 25% compared with the conventional handcraft method.
- Open Access
- 28 May 2020
- Published by MDPI AG in Electronics
- Vol. 9 (6) , 896
Joint Scheduling and Power Allocation Using Non-Orthogonal Multiple Access in Multi-Cell Beamforming Networks
The proliferation of smart devices has boosted the improvement of wireless network technologies. Herein, networking functions should be properly guaranteed even in highly dense environments in terms of service quality and data rate. In this paper, we present an efficient power allocation algorithm using non-orthogonal multiple access and smart array antennas to increase the capacity in highly overlapped multi-cell environments. We evaluate the proposed algorithm and compare with the conventional orthogonal multiple access scheme with smart antennas. Through intensive simulations and experiments at the system level for performance evaluations, it is confirmed that the proposed scheme obtains a drastic throughput gain up to 50% in the overlapped region of highly dense networks.
- 28 May 2020
- Published by MDPI AG in Electronics
- Vol. 9 (6) , 896
Open AccessJoint Scheduling and Power Allocation Using Non-Orthogonal Multiple Access in Multi-Cell Beamforming Networks
The proliferation of smart devices has boosted the improvement of wireless network technologies. Herein, networking functions should be properly guaranteed even in highly dense environments in terms of service quality and data rate. In this paper, we present an efficient power allocation algorithm using non-orthogonal multiple access and smart array antennas to increase the capacity in highly overlapped multi-cell environments. We evaluate the proposed algorithm and compare with the conventional orthogonal multiple access scheme with smart antennas. Through intensive simulations and experiments at the system level for performance evaluations, it is confirmed that the proposed scheme obtains a drastic throughput gain up to 50% in the overlapped region of highly dense networks.
- Open Access
- 11 January 2020
- Published by MDPI AG in Electronics
- Vol. 9 (1) , 143
CoRL: Collaborative Reinforcement Learning-Based MAC Protocol for IoT Networks
Devices used in Internet of Things (IoT) networks continue to perform sensing, gathering, modifying, and forwarding data. Since IoT networks have a lot of participants, mitigating and reducing collisions among the participants becomes an essential requirement for the Medium Access Control (MAC) protocols to increase system performance. A collision occurs in wireless channel when two or more nodes try to access the channel at the same time. In this paper, a reinforcement learning-based MAC protocol was proposed to provide high throughput and alleviate the collision problem. A collaboratively predicted Q-value was proposed for nodes to update their value functions by using communications trial information of other nodes. Our proposed protocol was confirmed by intensive system level simulations that it can reduce convergence time in 34.1% compared to the conventional Q-learning-based MAC protocol.
- 11 January 2020
- Published by MDPI AG in Electronics
- Vol. 9 (1) , 143
Open AccessCoRL: Collaborative Reinforcement Learning-Based MAC Protocol for IoT Networks
Devices used in Internet of Things (IoT) networks continue to perform sensing, gathering, modifying, and forwarding data. Since IoT networks have a lot of participants, mitigating and reducing collisions among the participants becomes an essential requirement for the Medium Access Control (MAC) protocols to increase system performance. A collision occurs in wireless channel when two or more nodes try to access the channel at the same time. In this paper, a reinforcement learning-based MAC protocol was proposed to provide high throughput and alleviate the collision problem. A collaboratively predicted Q-value was proposed for nodes to update their value functions by using communications trial information of other nodes. Our proposed protocol was confirmed by intensive system level simulations that it can reduce convergence time in 34.1% compared to the conventional Q-learning-based MAC protocol.