HSHrvoje Stojić

Hrvoje Stojić

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Hrvoje Stojić has a total of 51 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Ranking-based rich-get-richer processes

    by Pantelis P. Analytis, Alexandros Gelastopoulos, Hrvoje Stojic

    We study a discrete-time Markov process Xn∈Rd for which the distribution of the future increments depends only on the relative ranking of its components (descending order by value). We endow the process with a rich-get-richer assumption and show that, together with a finite second moments assumption, it is enough to guarantee almost sure convergence of Xn/n. We characterize the possible limits if one is free to choose the initial state and we give a condition under which the initial state is irrelevant. Finally, we show how our framework can account for ranking-based Pólya urns and can be used to study ranking algorithms for web interfaces.


  • Open Access

    An empirical evaluation of active inference in multi-armed bandits

    by Dimitrije Marković, Hrvoje Stojić, Sarah Schwöbel, Stefan J. Kiebel


  • Open Access

    Bellman: A Toolbox for Model-Based Reinforcement Learning in TensorFlow

    by John McLeod, Hrvoje Stojic, Vincent Adam, Dongho Kim, Jordi Grau-Moya, Peter Vrancx, Felix Leibfried

    In the past decade, model-free reinforcement learning (RL) has provided solutions to challenging domains such as robotics. Model-based RL shows the prospect of being more sample-efficient than model-free methods in terms of agent-environment interactions, because the model enables to extrapolate to unseen situations. In the more recent past, model-based methods have shown superior results compared to model-free methods in some challenging domains with non-linear state transitions. At the same time, it has become apparent that RL is not market-ready yet and that many real-world applications are going to require model-based approaches, because model-free methods are too sample-inefficient and show poor performance in early stages of training. The latter is particularly important in industry, e.g. in production systems that directly impact a company's revenue. This demonstrates the necessity for a toolbox to push the boundaries for model-based RL. While there is a plethora of toolboxes for model-free RL, model-based RL has received little attention in terms of toolbox development. Bellman aims to fill this gap and introduces the first thoroughly designed and tested model-based RL toolbox using state-of-the-art software engineering practices. Our modular approach enables to combine a wide range of environment models with generic model-based agent classes that recover state-of-the-art algorithms. We also provide an experiment harness to compare both model-free and model-based agents in a systematic fashion w.r.t. user-defined evaluation metrics (e.g. cumulative reward). This paves the way for new research directions, e.g. investigating uncertainty-aware environment models that are not necessarily neural-network-based, or developing algorithms to solve industrially-motivated benchmarks that share characteristics with real-world problems.

  • Linear integration and lexicographic models of choice: A cue weight learning perspective.

    by Hrvoje Stojić, Henrik Olsson, Pantelis P. Analytis

    Choosing between options characterized by multiple cues can be a daunting task. People may integrate all information at hand or just use lexicographic strategies that ignore most of it. Notably, integrative strategies require knowing exact cue weights, whereas lexicographic heuristics can operate by merely knowing the importance order of cues. Here we study how using integrative or lexicographic strategies interacts with learning about cues. In our choice-learning-estimation paradigm people first make choices, learning about cues from the experienced qualities of chosen options, and then estimate qualities of new options. We developed delta-elimination (DE), a new lexicographic strategy that generalizes previous heuristics to any type of environment, and compared it to the integrative weighted-additive (WADD) strategy. Our results show that participants learned cue weights, regardless of whether the DE strategy or the WADD strategy described their choices the best. Still, there was an interaction between the adopted strategy and the cue weight learning process: the DE users learned cue weights slower than the WADD users. This work advances the study of lexicographic choice strategies, both empirically and theoretically, and deepens our understanding of strategy selection, in particular the interaction between the strategy used and learning the structure of the environment.

  • It’s new, but is it good? How generalization and uncertainty guide the exploration of novel options.

    by Hrvoje Stojić, Eric Schulz, Pantelis P. Analytis, Maarten Speekenbrink

    How do people decide whether to try out novel options as opposed to tried-and-tested ones? We argue that they infer a novel option's reward from contextual information learned from functional relations and take uncertainty into account when making a decision. We propose a Bayesian optimization model to describe their learning and decision making. This model relies on similarity-based learning of functional relationships between features and rewards, and a choice rule that balances exploration and exploitation by combining predicted rewards and the uncertainty of these predictions. Our model makes 2 main predictions. First, decision makers who learn functional relationships will generalize based on the learned reward function, choosing novel options only if their predicted reward is high. Second, they will take uncertainty about the function into account, and prefer novel options that can reduce this uncertainty. We test these predictions in 3 preregistered experiments in which we examine participants' preferences for novel options using a feature-based multiarmed bandit task in which rewards are a noisy function of observable features. Our results reveal strong evidence for functional exploration and moderate evidence for uncertainty-guided exploration. However, whether or not participants chose a novel option also depended on their attention, as well as reflecting on the value of the options. These results advance our understanding of people's reactions in the face of novelty. (PsycInfo Database Record (c) 2020 APA, all rights reserved).


  • Open Access

    Uncertainty in learning, choice, and visual fixation

    by Hrvoje Stojić, Jacob L. Orquin, Peter Dayan, Raymond J. Dolan, Maarten Speekenbrink

    Uncertainty plays a critical role in reinforcement learning and decision making. However, exactly how it influences behavior remains unclear. Multiarmed-bandit tasks offer an ideal test bed, since computational tools such as approximate Kalman filters can closely characterize the interplay between trial-by-trial values, uncertainty, learning, and choice. To gain additional insight into learning and choice processes, we obtained data from subjects’ overt allocation of gaze. The estimated value and estimation uncertainty of options influenced what subjects looked at before choosing; these same quantities also influenced choice, as additionally did fixation itself. A momentary measure of uncertainty in the form of absolute prediction errors determined how long participants looked at the obtained outcomes. These findings affirm the importance of uncertainty in multiple facets of behavior and help delineate its effects on decision making.


  • Open Access

    Gender effects for loss aversion: Yes, no, maybe?

    by Ranoua Bouchouicha, Lachlan Deer, Ashraf Galal Eid, Peter McGee, Daniel Schoch, Hrvoje Stojic, Jolanda Ygosse-Battisti, Ferdinand M. Vieider

    Gender effects in risk taking have attracted much attention by economists, and remain debated. Loss aversion—the stylized finding that a given loss carries substantially greater weight than a monetarily equivalent gain—is a fundamental driver of risk aversion. We deploy four definitions of loss aversion commonly used in the literature to investigate gender effects. Even though the definitions only differ in subtle ways, we find women to be more loss averse than men according to one definition, while another definition results in no gender differences, and the remaining two definitions point to women beinglessloss averse than men. Conceptually, these contradictory effects can be organized by systematic measurement error resulting from model mis-specifications relative to the true underlying decision process.


  • Open Access

    Analysis of task-based functional MRI data preprocessed with fMRIPrep

    by Oscar Esteban, Rastko Ciric, Karolina Finc, Ross Blair, Christopher J. Markiewicz, Craig A. Moodie, James D. Kent, Mathias Goncalves, Elizabeth DuPre, Daniel E. P. Gomez, et al.

    Functional magnetic resonance imaging (fMRI) is a standard tool to investigate the neural correlates of cognition. fMRI noninvasively measures brain activity, allowing identification of patterns evoked by tasks performed during scanning. Despite the long history of this technique, the idiosyncrasies of each dataset have led to the use of ad-hoc preprocessing protocols customized for nearly every different study. This approach is time-consuming, error-prone, and unsuitable for combining datasets from many sources. Here we showcase fMRIPrep ( http://fmriprep.org ), a robust tool to prepare human fMRI data for statistical analysis. This software instrument addresses the reproducibility concerns of the established protocols for fMRI preprocessing. By leveraging the Brain Imaging Data Structure (BIDS) to standardize both the input datasets —MRI data as stored by the scanner— and the outputs —data ready for modeling and analysis—, fMRIPrep is capable of preprocessing a diversity of datasets without manual intervention. In support of the growing popularity of fMRIPrep , this protocol describes how to integrate the tool in a task-based fMRI investigation workflow.