Sushil J. Louis
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Sushil J. Louis has a total of 24 co-authors. Here are the top co-authors by number of shared publications:
- Chris Miles
Evolutionary Computing Systems Lab, Dept. of Computer Science and Engineering, University of Nevada, Reno
3 shared publications - Monica Nicolescu
University of Nevada, Reno
3 shared publications Aavaas Gajurel
Department of Computer Science and Engineering University of Nevada Reno Reno Nevada
2 shared publications- Christopher Ballinger
Soar Technology, Inc
2 shared publications - Mircea Nicolescu
University of Nevada
2 shared publications - Ryan Leigh
Evolutionary Computing Systems Lab, Dept. of Computer Science and Engineering, University of Nevada, Reno
2 shared publications
Latest Publications
- Open Access
Automatic Parameter Optimization Using Genetic Algorithm in Deep Reinforcement Learning for Robotic Manipulation Tasks
Learning agents can make use of Reinforcement Learning (RL) to decide their actions by using a reward function. However, the learning process is greatly influenced by the elect of values of the hyperparameters used in the learning algorithm. This work proposed a Deep Deterministic Policy Gradient (DDPG) and Hindsight Experience Replay (HER) based method, which makes use of the Genetic Algorithm (GA) to fine-tune the hyperparameters' values. This method (GA+DDPG+HER) experimented on six robotic manipulation tasks: FetchReach; FetchSlide; FetchPush; FetchPickAndPlace; DoorOpening; and AuboReach. Analysis of these results demonstrated a significant increase in performance and a decrease in learning time. Also, we compare and provide evidence that GA+DDPG+HER is better than the existing methods.
Open AccessAutomatic Parameter Optimization Using Genetic Algorithm in Deep Reinforcement Learning for Robotic Manipulation Tasks
Learning agents can make use of Reinforcement Learning (RL) to decide their actions by using a reward function. However, the learning process is greatly influenced by the elect of values of the hyperparameters used in the learning algorithm. This work proposed a Deep Deterministic Policy Gradient (DDPG) and Hindsight Experience Replay (HER) based method, which makes use of the Genetic Algorithm (GA) to fine-tune the hyperparameters' values. This method (GA+DDPG+HER) experimented on six robotic manipulation tasks: FetchReach; FetchSlide; FetchPush; FetchPickAndPlace; DoorOpening; and AuboReach. Analysis of these results demonstrated a significant increase in performance and a decrease in learning time. Also, we compare and provide evidence that GA+DDPG+HER is better than the existing methods.
- Open Access
GPU Acceleration of Sparse Neural Networks
In this paper, we use graphics processing units(GPU) to accelerate sparse and arbitrary structured neural networks. Sparse networks have nodes in the network that are not fully connected with nodes in preceding and following layers, and arbitrary structure neural networks have different number of nodes in each layers. Sparse Neural networks with arbitrary structures are generally created in the processes like neural network pruning and evolutionary machine learning strategies. We show that we can gain significant speedup for full activation of such neural networks using graphical processing units. We do a prepossessing step to determine dependency groups for all the nodes in a network, and use that information to guide the progression of activation in the neural network. Then we compute activation for each nodes in its own separate thread in the GPU, which allows for massive parallelization. We use CUDA framework to implement our approach and compare the results of sequential and GPU implementations. Our results show that the activation of sparse neural networks lends very well to GPU acceleration and can help speed up machine learning strategies which generate such networks or other processes that have similar structure.
Open AccessGPU Acceleration of Sparse Neural Networks
In this paper, we use graphics processing units(GPU) to accelerate sparse and arbitrary structured neural networks. Sparse networks have nodes in the network that are not fully connected with nodes in preceding and following layers, and arbitrary structure neural networks have different number of nodes in each layers. Sparse Neural networks with arbitrary structures are generally created in the processes like neural network pruning and evolutionary machine learning strategies. We show that we can gain significant speedup for full activation of such neural networks using graphical processing units. We do a prepossessing step to determine dependency groups for all the nodes in a network, and use that information to guide the progression of activation in the neural network. Then we compute activation for each nodes in its own separate thread in the GPU, which allows for massive parallelization. We use CUDA framework to implement our approach and compare the results of sequential and GPU implementations. Our results show that the activation of sparse neural networks lends very well to GPU acceleration and can help speed up machine learning strategies which generate such networks or other processes that have similar structure.
- Open Access
Evolutionary Multi-objective Optimization of Real-Time Strategy Micro
We investigate an evolutionary multi-objective approach to good micro for real-time strategy games. Good micro helps a player win skirmishes and is one of the keys to developing better real-time strategy game play. In prior work, the same multi-objective approach of maximizing damage done while minimizing damage received was used to evolve micro for a group of ranged units versus a group of melee units. We extend this work to consider groups composed from two types of units. Specifically, this paper uses evolutionary multi-objective optimization to generate micro for one group composed from both ranged and melee units versus another group of ranged and melee units. Our micro behavior representation uses influence maps to represent enemy spatial information and potential fields generated from distance, health, and weapons cool down to guide unit movement. Experimental results indicate that our multi-objective approach leads to a Pareto front of diverse high-quality micro encapsulating multiple possible tactics. This range of micro provided by the Pareto front enables a human or AI player to trade-off among short term tactics that better suit the player's longer term strategy - for example, choosing to minimize friendly unit damage at the cost of only lightly damaging the enemy versus maximizing damage to the enemy units at the cost of increased damage to friendly units. We believe that our results indicate the usefulness of potential fields as a representation, and of evolutionary multi-objective optimization as an approach, for generating good micro.
Open AccessEvolutionary Multi-objective Optimization of Real-Time Strategy Micro
We investigate an evolutionary multi-objective approach to good micro for real-time strategy games. Good micro helps a player win skirmishes and is one of the keys to developing better real-time strategy game play. In prior work, the same multi-objective approach of maximizing damage done while minimizing damage received was used to evolve micro for a group of ranged units versus a group of melee units. We extend this work to consider groups composed from two types of units. Specifically, this paper uses evolutionary multi-objective optimization to generate micro for one group composed from both ranged and melee units versus another group of ranged and melee units. Our micro behavior representation uses influence maps to represent enemy spatial information and potential fields generated from distance, health, and weapons cool down to guide unit movement. Experimental results indicate that our multi-objective approach leads to a Pareto front of diverse high-quality micro encapsulating multiple possible tactics. This range of micro provided by the Pareto front enables a human or AI player to trade-off among short term tactics that better suit the player's longer term strategy - for example, choosing to minimize friendly unit damage at the cost of only lightly damaging the enemy versus maximizing damage to the enemy units at the cost of increased damage to friendly units. We believe that our results indicate the usefulness of potential fields as a representation, and of evolutionary multi-objective optimization as an approach, for generating good micro.
- Open Access
Co-evolving Real-Time Strategy Game Micro
We investigate competitive co-evolution of unit micromanagement in real-time strategy games. Although good long-term macro-strategy and good short-term unit micromanagement both impact real-time strategy games performance, this paper focuses on generating quality micro. Better micro, for example, can help players win skirmishes and battles even when outnumbered. Prior work has shown that we can evolve micro to beat a given opponent. We remove the need for a good opponent to evolve against by using competitive co-evolution to evolve high-quality micro for both sides from scratch. We first co-evolve micro to control a group of ranged units versus a group of melee units. We then move to co-evolve micro for a group of ranged and melee units versus a group of ranged and melee units. Results show that competitive co-evolution produces good quality micro and when combined with the well-known techniques of fitness sharing, shared sampling, and a hall of fame takes less time to produce better quality micro than simple co-evolution. We believe these results indicate the viability of co-evolutionary approaches for generating good unit micro-management.
Open AccessCo-evolving Real-Time Strategy Game Micro
We investigate competitive co-evolution of unit micromanagement in real-time strategy games. Although good long-term macro-strategy and good short-term unit micromanagement both impact real-time strategy games performance, this paper focuses on generating quality micro. Better micro, for example, can help players win skirmishes and battles even when outnumbered. Prior work has shown that we can evolve micro to beat a given opponent. We remove the need for a good opponent to evolve against by using competitive co-evolution to evolve high-quality micro for both sides from scratch. We first co-evolve micro to control a group of ranged units versus a group of melee units. We then move to co-evolve micro for a group of ranged and melee units versus a group of ranged and melee units. Results show that competitive co-evolution produces good quality micro and when combined with the well-known techniques of fitness sharing, shared sampling, and a hall of fame takes less time to produce better quality micro than simple co-evolution. We believe these results indicate the viability of co-evolutionary approaches for generating good unit micro-management.
- Open Access
Neuroevolution for RTS Micro
This paper uses neuroevolution of augmenting topologies to evolve control tactics for groups of units in real-time strategy games. In such games, players build economies to generate armies composed of multiple types of units with different attack and movement characteristics to combat each other. This paper evolves neural networks to control movement and attack commands, also called micro, for a group of ranged units skirmishing with a group of melee units. Our results show that neuroevolution of augmenting topologies can effectively generate neural networks capable of good micro for our ranged units against a group of hand-coded melee units. The evolved neural networks lead to kiting behavior for the ranged units which is a common tactic used by professional players in ranged versus melee skirmishes in popular real-time strategy games like Starcraft. The evolved neural networks also generalized well to other starting positions and numbers of units. We believe these results indicate the potential of neuroevolution for generating effective micro in real-time strategy games.
Open AccessNeuroevolution for RTS Micro
This paper uses neuroevolution of augmenting topologies to evolve control tactics for groups of units in real-time strategy games. In such games, players build economies to generate armies composed of multiple types of units with different attack and movement characteristics to combat each other. This paper evolves neural networks to control movement and attack commands, also called micro, for a group of ranged units skirmishing with a group of melee units. Our results show that neuroevolution of augmenting topologies can effectively generate neural networks capable of good micro for our ranged units against a group of hand-coded melee units. The evolved neural networks lead to kiting behavior for the ranged units which is a common tactic used by professional players in ranged versus melee skirmishes in popular real-time strategy games like Starcraft. The evolved neural networks also generalized well to other starting positions and numbers of units. We believe these results indicate the potential of neuroevolution for generating effective micro in real-time strategy games.
- 11 April 2016
- Published by Springer Nature in Neural Computing and Applications
- Vol. 28 (5) , 881-897
Using patterns of firing neurons in spiking neural networks for learning and early recognition of spatio-temporal patterns
- 11 April 2016
- Published by Springer Nature in Neural Computing and Applications
- Vol. 28 (5) , 881-897
Using patterns of firing neurons in spiking neural networks for learning and early recognition of spatio-temporal patterns
- Open Access
- 22 March 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Intelligence and AI in Games
- Vol. 8 (4) , 351-362
Evolving Effective Microbehaviors in Real-Time Strategy Games
We investigate heuristic search algorithms to generate high-quality micromanagement in combat scenarios for real-time strategy (RTS) games. Macro- and micromanagement are two key aspects of RTS games. While good macro helps a player collect more resources and build more units, good micro helps a player win skirmishes and battles against equal numbers and types of opponent units or win even when outnumbered. In this paper, we use influence maps and potential fields as a basis representation to evolve short-term positioning and movement tactics. Unit microbehaviors in combat are compactly encoded into 14 parameters. A genetic algorithm evolves good microbehaviors by manipulating these 14 parameters. We compared the performance of our evolved ECSLBot with two other state-of-the-art bots, UAlbertaBot and Nova, on several skirmish scenarios in a popular RTS game StarCraft. The results show that the ECSLBot tuned by genetic algorithms outperforms UAlbertaBot and Nova in kiting efficiency, target selection, and fleeing. Further experiments show that the parameter values evolved in one scenario work well in other scenarios and that we can switch between preevolved parameter sets to perform well in unseen scenarios containing more than one type of opponent unit. We believe our representation and approach applied to each unit type of interest can result in effective microperformance against melee and ranged opponents and provides a viable approach toward complete RTS bots.
- 22 March 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computational Intelligence and AI in Games
- Vol. 8 (4) , 351-362
Open AccessEvolving Effective Microbehaviors in Real-Time Strategy Games
We investigate heuristic search algorithms to generate high-quality micromanagement in combat scenarios for real-time strategy (RTS) games. Macro- and micromanagement are two key aspects of RTS games. While good macro helps a player collect more resources and build more units, good micro helps a player win skirmishes and battles against equal numbers and types of opponent units or win even when outnumbered. In this paper, we use influence maps and potential fields as a basis representation to evolve short-term positioning and movement tactics. Unit microbehaviors in combat are compactly encoded into 14 parameters. A genetic algorithm evolves good microbehaviors by manipulating these 14 parameters. We compared the performance of our evolved ECSLBot with two other state-of-the-art bots, UAlbertaBot and Nova, on several skirmish scenarios in a popular RTS game StarCraft. The results show that the ECSLBot tuned by genetic algorithms outperforms UAlbertaBot and Nova in kiting efficiency, target selection, and fleeing. Further experiments show that the parameter values evolved in one scenario work well in other scenarios and that we can switch between preevolved parameter sets to perform well in unseen scenarios containing more than one type of opponent unit. We believe our representation and approach applied to each unit type of interest can result in effective microperformance against melee and ranged opponents and provides a viable approach toward complete RTS bots.
- 1 January 2015
- Published by Springer Nature
- p. 158-170
Intent Recognition in a Simulated Maritime Multi-agent Domain
- 1 January 2015
- Published by Springer Nature
- p. 158-170
Intent Recognition in a Simulated Maritime Multi-agent Domain
- 8 March 2010
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Evolutionary Computation
- Vol. 14 (4) , 547-560
XCS for Personalizing Desktop Interfaces
We investigate whether XCS, a genetic algorithm based learning classifier system, can harness information from a user's environment to help desktop applications better personalize themselves to individual users. Specifically, we evaluate XCSs ability to predict user-preferred actions for a calendar and a media player. Results from three real-world user studies indicate that XCS significantly outperforms a decision-tree learner to successfully predict user preferences for these two desktop interfaces. Our results also show that removing external user-related contextual information degrades XCSs performance. This performance degradation emphasizes the need for desktop applications to access external contextual information to better learn user preferences. Our results highlight the potential for a learning classifier systems based approach for personalizing desktop applications to improve the quality of human-computer interaction.
- 8 March 2010
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Evolutionary Computation
- Vol. 14 (4) , 547-560
XCS for Personalizing Desktop Interfaces
We investigate whether XCS, a genetic algorithm based learning classifier system, can harness information from a user's environment to help desktop applications better personalize themselves to individual users. Specifically, we evaluate XCSs ability to predict user-preferred actions for a calendar and a media player. Results from three real-world user studies indicate that XCS significantly outperforms a decision-tree learner to successfully predict user preferences for these two desktop interfaces. Our results also show that removing external user-related contextual information degrades XCSs performance. This performance degradation emphasizes the need for desktop applications to access external contextual information to better learn user preferences. Our results highlight the potential for a learning classifier systems based approach for personalizing desktop applications to improve the quality of human-computer interaction.
- 1 January 2007
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 88-95
Co-Evolving Influence Map Tree Based Strategy Game Players
We investigate the use of genetic algorithms to evolve AI players for real-time strategy games. To overcome the knowledge acquisition bottleneck found in using traditional expert systems, scripts, or decision trees we evolve players through co-evolution. Our game players are implemented as resource allocation systems. Influence map trees are used to analyze the game-state and determine promising places to attack, defend, etc. These spatial objectives are chained to non-spatial objectives (train units, build buildings, gather resources) in a dependency graph. Players are encoded within the individuals of a genetic algorithm and co-evolved against each other, with results showing the production of strategies that are innovative, robust, and capable of defeating a suite of hand-coded opponents
- 1 January 2007
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 88-95
Co-Evolving Influence Map Tree Based Strategy Game Players
We investigate the use of genetic algorithms to evolve AI players for real-time strategy games. To overcome the knowledge acquisition bottleneck found in using traditional expert systems, scripts, or decision trees we evolve players through co-evolution. Our game players are implemented as resource allocation systems. Influence map trees are used to analyze the game-state and determine promising places to attack, defend, etc. These spatial objectives are chained to non-spatial objectives (train units, build buildings, gather resources) in a dependency graph. Players are encoded within the individuals of a genetic algorithm and co-evolved against each other, with results showing the production of strategies that are innovative, robust, and capable of defeating a suite of hand-coded opponents