JLJunior Samuel Lopez-Yepez

Junior Samuel Lopez-Yepez

*This is an algorithmically generated author record based on publicly available information.

Junior Samuel Lopez-Yepez has a total of 4 co-authors. Here are the top co-authors by number of shared publications:

  • Antoine Fagette

    Thales Digital Solutions Thales Group,Montreal,Canada

    1 shared publication
  • Duda Kvitsiani

    Aarhus University/DANDRITE

    1 shared publication
  • Juliane Martin

    Affiliation Aarhus University, Danish Research Institute of Translational Neuroscience (DANDRITE), Aarhus, Denmark

    1 shared publication
  • Oliver James Hulme

    Affiliations London Mathematical Laboratory, London, United Kingdom, Danish Research Centre for Magnetic Resonance, Department of Radiology and Nuclear Medicine, Copenhagen University Hospital - Amager and Hvidovre, Copenhagen, Denmark, Department of Psychology, University of Copenhagen, Copenhagen, Denmark

    1 shared publication

Latest Publications

  • Increasing attacker engagement on SSH honeypots using semantic embeddings of cyber-attack patterns and deep reinforcement learning

    by Junior Samuel Lopez-Yepez, Antoine Fagette

    In this article, we demonstrate how combining semantic embeddings of cyber-attack patterns and deep reinforcement learning (DRL) can increase the engagement of attackers in interactive honeypots. We use a dataset of real-world cyber-attacks to pre-train, train and test our system. Our results show that our implementation is effective at increasing attackers' engagement compared to non-interactive honeypots or tabular reinforcement learning algorithms. In addition, we present how the inclusion of pattern data, as opposed to only current data increases significantly the engagement of artificial attackers in a simulated environment.


  • Open Access

    Choice history effects in mice and humans improve reward harvesting efficiency

    by Junior Samuel López-Yépez, Juliane Martin, Oliver Hulme, Duda Kvitsiani

    Choice history effects describe how future choices depend on the history of past choices. In experimental tasks this is typically framed as a bias because it often diminishes the experienced reward rates. However, in natural habitats, choices made in the past constrain choices that can be made in the future. For foraging animals, the probability of earning a reward in a given patch depends on the degree to which the animals have exploited the patch in the past. One problem with many experimental tasks that show choice history effects is that such tasks artificially decouple choice history from its consequences on reward availability over time. To circumvent this, we use a variable interval (VI) reward schedule that reinstates a more natural contingency between past choices and future reward availability. By examining the behavior of optimal agents in the VI task we discover that choice history effects observed in animals serve to maximize reward harvesting efficiency. We further distil the function of choice history effects by manipulating first- and second-order statistics of the environment. We find that choice history effects primarily reflect the growth rate of the reward probability of the unchosen option, whereas reward history effects primarily reflect environmental volatility. Based on observed choice history effects in animals, we develop a reinforcement learning model that explicitly incorporates choice history over multiple time scales into the decision process, and we assess its predictive adequacy in accounting for the associated behavior. We show that this new variant, known as the double trace model, has a higher performance in predicting choice data, and shows near optimal reward harvesting efficiency in simulated environments. These results suggests that choice history effects may be adaptive for natural contingencies between consumption and reward availability. This concept lends credence to a normative account of choice history effects that extends beyond its description as a bias.