Siming Liu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Siming Liu has a total of 21 co-authors. Here are the top co-authors by number of shared publications:
- Sushil J. Louis
University of Nevada
3 shared publications - Yan Liu
Key Laboratory for Embedded and Network Computing of Hunan Province, College of Computer Science and Electronic Engineering, Hunan University, Changsha, China
3 shared publications Anthony Harris
Department of Computer Science, Missouri State University, Springfield, MO, USA
1 shared publicationAyesha Siddika Nipu
Department of Computer Science and Software Engineering, University of Wisconsin-Platteville, Platteville, WI, USA
1 shared publication- Ayesha Siddiqua
Department of Computer Science, Missouri State University, Springfield, MO, USA
1 shared publication - Christopher Ballinger
Soar Technology, Inc
1 shared publication
Latest Publications
- Open Access
- 17 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 99439-99451
Co-Evolving Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representation
Multi-Agent Reinforcement Learning (MARL) is extensively utilized for addressing intricate tasks that involve cooperation and competition among agents in Multi-Agent Systems (MAS). However, learning such tasks from scratch is challenging and often unfeasible, especially for MASs with a large number of agents. Hence, leveraging knowledge from prior experiences can effectively expedite the MARL learning process. Prior work has shown that we successfully facilitated transfer learning for MARL by consolidating various state spaces into fixed-size inputs, enabling a single unified deep-learning policy applicable to several scenarios within the StarCraft Multi-Agent Challenge (SMAC) environment. In this study, we expand SMAC to Multi-Player enabled SMAC (MP-SMAC) by enabling the dynamic selection of training opponents and introducing a co-evolving MARL framework, which creates a co-evolutionary arena where multiple policies learn simultaneously. Our arena comprised the simultaneous training of multiple policies in diverse scenarios, pitting them against both static AI opponents and their peers within MP-SMAC. Furthermore, we integrate co-evolution with curriculum transfer learning into Co-MACTRL framework, enabling our MARL policies to systematically acquire knowledge and skills across predetermined scenarios organized by varying difficulty levels, including evolving opponents. The results revealed significant enhancements in MARL learning performance, demonstrating the advantage of leveraging the co-evolving opponents and maneuvering skills obtained from different scenarios. Additionally, the Co-MACTRL learners consistently attained high performance across a range of SMAC scenarios, showcasing the robustness and generalizability of Co-MACTRL.
- 17 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 99439-99451
Open AccessCo-Evolving Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representation
Multi-Agent Reinforcement Learning (MARL) is extensively utilized for addressing intricate tasks that involve cooperation and competition among agents in Multi-Agent Systems (MAS). However, learning such tasks from scratch is challenging and often unfeasible, especially for MASs with a large number of agents. Hence, leveraging knowledge from prior experiences can effectively expedite the MARL learning process. Prior work has shown that we successfully facilitated transfer learning for MARL by consolidating various state spaces into fixed-size inputs, enabling a single unified deep-learning policy applicable to several scenarios within the StarCraft Multi-Agent Challenge (SMAC) environment. In this study, we expand SMAC to Multi-Player enabled SMAC (MP-SMAC) by enabling the dynamic selection of training opponents and introducing a co-evolving MARL framework, which creates a co-evolutionary arena where multiple policies learn simultaneously. Our arena comprised the simultaneous training of multiple policies in diverse scenarios, pitting them against both static AI opponents and their peers within MP-SMAC. Furthermore, we integrate co-evolution with curriculum transfer learning into Co-MACTRL framework, enabling our MARL policies to systematically acquire knowledge and skills across predetermined scenarios organized by varying difficulty levels, including evolving opponents. The results revealed significant enhancements in MARL learning performance, demonstrating the advantage of leveraging the co-evolving opponents and maneuvering skills obtained from different scenarios. Additionally, the Co-MACTRL learners consistently attained high performance across a range of SMAC scenarios, showcasing the robustness and generalizability of Co-MACTRL.
- Open Access
- 1 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 57965-57976
SC-MAIRL: Semi-Centralized Multi-Agent Imitation Reinforcement Learning
Multi-agent reinforcement learning (MARL) is a challenging branch of reinforcement learning that requires cooperation of interactive learning agents to achieve individual objectives as well as shared team objectives. Existing MARL algorithms generally use either centralized global state representation or decentralized local observation to perform training and execution. In this paper, we introduce a novel MARL learning paradigm, centralized training with semi-centralized execution (CTSCE), and present a new MARL algorithm for addressing multi-agent problems: Semi-Centralized Multi-Agent Imitation Reinforcement Learning (SC-MAIRL). The semi-centralized approach aggregated with agents’ spatial and temporal information serves as a joint knowledge base to facilitate a learning agent to discover team objectives and make fine-grained decisions. We also utilize a pre-trained performant teacher policy to guide an untrained model towards positive game states as a form of imitation learning, significantly increasing the agent’s learning speed. In addition, for encouraging agents to learn both offensive and defensive behaviors and smoothing the high-dimensional learning curve, we present a new set of reward-shaping functions to further improve SC-MAIRL’s learning performance. Our approach is evaluated using one of the most challenging scenarios within the StarCraft Multi-Agent Challenge environment, and the results show that SC-MAIRL outperforms the state-of-the-art MARL algorithm MAPPO in several metrics and allows our agents to learn and employ novel, complex macro strategies more effectively.
- 1 June 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 57965-57976
Open AccessSC-MAIRL: Semi-Centralized Multi-Agent Imitation Reinforcement Learning
Multi-agent reinforcement learning (MARL) is a challenging branch of reinforcement learning that requires cooperation of interactive learning agents to achieve individual objectives as well as shared team objectives. Existing MARL algorithms generally use either centralized global state representation or decentralized local observation to perform training and execution. In this paper, we introduce a novel MARL learning paradigm, centralized training with semi-centralized execution (CTSCE), and present a new MARL algorithm for addressing multi-agent problems: Semi-Centralized Multi-Agent Imitation Reinforcement Learning (SC-MAIRL). The semi-centralized approach aggregated with agents’ spatial and temporal information serves as a joint knowledge base to facilitate a learning agent to discover team objectives and make fine-grained decisions. We also utilize a pre-trained performant teacher policy to guide an untrained model towards positive game states as a form of imitation learning, significantly increasing the agent’s learning speed. In addition, for encouraging agents to learn both offensive and defensive behaviors and smoothing the high-dimensional learning curve, we present a new set of reward-shaping functions to further improve SC-MAIRL’s learning performance. Our approach is evaluated using one of the most challenging scenarios within the StarCraft Multi-Agent Challenge environment, and the results show that SC-MAIRL outperforms the state-of-the-art MARL algorithm MAPPO in several metrics and allows our agents to learn and employ novel, complex macro strategies more effectively.
- 29 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 19 (2) , 1185-1195
TCE-IDS: Time Interval Conditional Entropy- Based Intrusion Detection System for Automotive Controller Area Networks
Intelligent connected vehicle is rapidly growing with the 5-G technology; the diversity of functional interfaces has significantly expanded the avenues of attack, making automotive controller area network (CAN) more vulnerable to cyberthreats. Automotive CAN network attacks are a direct threat to traffic safety, and in this study, we explore the use of intrusion detection techniques for mitigating cyberattacks. However, most automotive CAN network intrusion detection technologies are not capable of defending against sophisticated attacks, making it extremely challenging for detecting intrusions in practice. In this article, we propose a novel time interval conditional entropy method for detecting intrusions in automotive CAN networks. The time interval conditional entropy intrusion detection method is not susceptible to interference and is capable of detecting a variety of attacks. In our experiments, the conditional entropy values of regular communication messages are collected and utilized to distinguish and detect the attacks. The time interval conditional entropy detection method is implemented and evaluated in our controller area net-work bus (CAN-BUS) network platform. The experiments show that our method has higher detection accuracy and is easier to deploy compared to existing automotive CAN network intrusion detection methods.
- 29 August 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Industrial Informatics
- Vol. 19 (2) , 1185-1195
TCE-IDS: Time Interval Conditional Entropy- Based Intrusion Detection System for Automotive Controller Area Networks
Intelligent connected vehicle is rapidly growing with the 5-G technology; the diversity of functional interfaces has significantly expanded the avenues of attack, making automotive controller area network (CAN) more vulnerable to cyberthreats. Automotive CAN network attacks are a direct threat to traffic safety, and in this study, we explore the use of intrusion detection techniques for mitigating cyberattacks. However, most automotive CAN network intrusion detection technologies are not capable of defending against sophisticated attacks, making it extremely challenging for detecting intrusions in practice. In this article, we propose a novel time interval conditional entropy method for detecting intrusions in automotive CAN networks. The time interval conditional entropy intrusion detection method is not susceptible to interference and is capable of detecting a variety of attacks. In our experiments, the conditional entropy values of regular communication messages are collected and utilized to distinguish and detect the attacks. The time interval conditional entropy detection method is implemented and evaluated in our controller area net-work bus (CAN-BUS) network platform. The experiments show that our method has higher detection accuracy and is easier to deploy compared to existing automotive CAN network intrusion detection methods.
- Open Access
- 13 July 2019
- Published by Association for Computing Machinery (ACM)
- p. 369-370
A genetic algorithm for multi-robot routing in automated bridge inspection
- 13 July 2019
- Published by Association for Computing Machinery (ACM)
- p. 369-370
Open AccessA genetic algorithm for multi-robot routing in automated bridge inspection
- 1 June 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 482-489
Multi-objective cooperative co-evolution of micro for RTS games
We investigate a multi-objective, cooperative, co-evolutionary algorithm to evolve control tactics for groups composed from multiple types of units in real-time strategy games. Such tactical control or micromanagement of units is called micro. Building on prior work, we cooperatively co-evolve micro for a ranged unit using a parameterized control algorithm along with micro for a melee unit using a pure potential fields approach and show that we can simultaneously co-evolve micro for melee and ranged units. These cooperatively co-evolved control algorithms for melee and ranged units evolve to work well together to defeat the default Starcraft II AI, even when outnumbered. We are also able to generate manually co-evolved AI that is significantly better than the default Starcraft II AI and defeat it to generate human competitive micro for controlling multiple types of units. Furthermore, using a multi-objective fitness function leads to a pareto front of near-optimal micro behaviors that range from fleeing while sustaining minimal damage to fighting and maximizing damage to opponents. Such a pareto front naturally provides a user or AI player a variety of micro behaviors suitable for the different types of situations encountered in real-time strategy games. We believe these results indicate the potential of our method for generating effective micro for multiple types of units in real-time strategy games with application in multi-agent control, robotics, and other heterogeneous system control problems.
- 1 June 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 482-489
Multi-objective cooperative co-evolution of micro for RTS games
We investigate a multi-objective, cooperative, co-evolutionary algorithm to evolve control tactics for groups composed from multiple types of units in real-time strategy games. Such tactical control or micromanagement of units is called micro. Building on prior work, we cooperatively co-evolve micro for a ranged unit using a parameterized control algorithm along with micro for a melee unit using a pure potential fields approach and show that we can simultaneously co-evolve micro for melee and ranged units. These cooperatively co-evolved control algorithms for melee and ranged units evolve to work well together to defeat the default Starcraft II AI, even when outnumbered. We are also able to generate manually co-evolved AI that is significantly better than the default Starcraft II AI and defeat it to generate human competitive micro for controlling multiple types of units. Furthermore, using a multi-objective fitness function leads to a pareto front of near-optimal micro behaviors that range from fleeing while sustaining minimal damage to fighting and maximizing damage to opponents. Such a pareto front naturally provides a user or AI player a variety of micro behaviors suitable for the different types of situations encountered in real-time strategy games. We believe these results indicate the potential of our method for generating effective micro for multiple types of units in real-time strategy games with application in multi-agent control, robotics, and other heterogeneous system control problems.
- Open Access
- 6 July 2018
- Published by Association for Computing Machinery (ACM)
- p. 1656-1663
Real-time strategy game micro for tactical training simulations
- 6 July 2018
- Published by Association for Computing Machinery (ACM)
- p. 1656-1663
Open AccessReal-time strategy game micro for tactical training simulations
- 1 January 2018
- Published by Optica Publishing Group
- p. Th2A.48
Real-Time Demonstration of Adaptive Functional Split in 5G Flexible Mobile Fronthaul Networks
We experimentally demonstrate a flexible and reconfigurable Fronthaul-I with analog RoF integration that aims to optimally serve different 5G applications. RoF integration can reduce the latency by more than %15 to support URLLC applications.
- 1 January 2018
- Published by Optica Publishing Group
- p. Th2A.48
Real-Time Demonstration of Adaptive Functional Split in 5G Flexible Mobile Fronthaul Networks
We experimentally demonstrate a flexible and reconfigurable Fronthaul-I with analog RoF integration that aims to optimally serve different 5G applications. RoF integration can reduce the latency by more than %15 to support URLLC applications.
- Open Access
- 22 March 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Intelligence and AI in Games
- Vol. 8 (4) , 351-362
Evolving Effective Microbehaviors in Real-Time Strategy Games
We investigate heuristic search algorithms to generate high-quality micromanagement in combat scenarios for real-time strategy (RTS) games. Macro- and micromanagement are two key aspects of RTS games. While good macro helps a player collect more resources and build more units, good micro helps a player win skirmishes and battles against equal numbers and types of opponent units or win even when outnumbered. In this paper, we use influence maps and potential fields as a basis representation to evolve short-term positioning and movement tactics. Unit microbehaviors in combat are compactly encoded into 14 parameters. A genetic algorithm evolves good microbehaviors by manipulating these 14 parameters. We compared the performance of our evolved ECSLBot with two other state-of-the-art bots, UAlbertaBot and Nova, on several skirmish scenarios in a popular RTS game StarCraft. The results show that the ECSLBot tuned by genetic algorithms outperforms UAlbertaBot and Nova in kiting efficiency, target selection, and fleeing. Further experiments show that the parameter values evolved in one scenario work well in other scenarios and that we can switch between preevolved parameter sets to perform well in unseen scenarios containing more than one type of opponent unit. We believe our representation and approach applied to each unit type of interest can result in effective microperformance against melee and ranged opponents and provides a viable approach toward complete RTS bots.
- 22 March 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Intelligence and AI in Games
- Vol. 8 (4) , 351-362
Open AccessEvolving Effective Microbehaviors in Real-Time Strategy Games
We investigate heuristic search algorithms to generate high-quality micromanagement in combat scenarios for real-time strategy (RTS) games. Macro- and micromanagement are two key aspects of RTS games. While good macro helps a player collect more resources and build more units, good micro helps a player win skirmishes and battles against equal numbers and types of opponent units or win even when outnumbered. In this paper, we use influence maps and potential fields as a basis representation to evolve short-term positioning and movement tactics. Unit microbehaviors in combat are compactly encoded into 14 parameters. A genetic algorithm evolves good microbehaviors by manipulating these 14 parameters. We compared the performance of our evolved ECSLBot with two other state-of-the-art bots, UAlbertaBot and Nova, on several skirmish scenarios in a popular RTS game StarCraft. The results show that the ECSLBot tuned by genetic algorithms outperforms UAlbertaBot and Nova in kiting efficiency, target selection, and fleeing. Further experiments show that the parameter values evolved in one scenario work well in other scenarios and that we can switch between preevolved parameter sets to perform well in unseen scenarios containing more than one type of opponent unit. We believe our representation and approach applied to each unit type of interest can result in effective microperformance against melee and ranged opponents and provides a viable approach toward complete RTS bots.