MSMuhammed Omer Sayin

Muhammed Omer Sayin

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Muhammed Omer Sayin has a total of 41 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Decentralized Optimal Equilibrium Learning in Stochastic Games via Single-bit Feedback

    by Seref Taha Kiremitci, Ahmed Said Donmez, Muhammed O. Sayin

    We study decentralized equilibrium selection in stochastic games under severe information and communication constraints. In such settings, convergence to equilibrium alone is insufficient, as stochastic games typically admit many equilibria with markedly different welfare properties. We address decentralized optimal equilibrium selection, where agents coordinate on equilibria that optimize a designer-specified social welfare objective while allowing heterogeneous tolerance to deviations from strict best responses. Agents observe only the global state trajectory and their realized rewards, and exchange a single randomized bit of feedback per agent per round. This semantic content/discontent signaling mechanism implicitly aligns decentralized learning dynamics with the global welfare objective. We develop explore-and-commit and online variants applicable to general stochastic games, accommodating heterogeneous model-based or model-free methods for solving the induced Markov decision processes, and establish explicit finite-time regret guarantees, showing logarithmic expected regret under mild conditions.

  • Open Access

    Actor-Dual-Critic Dynamics for Zero-sum and Identical-Interest Stochastic Games

    by Ahmed Said Donmez, Yuksel Arslantas, Muhammed O. Sayin

    We propose a novel independent and payoff-based learning framework for stochastic games that is model-free, game-agnostic, and gradient-free. The learning dynamics follow a best-response-type actor-critic architecture, where agents update their strategies (actors) using feedback from two distinct critics: a fast critic that intuitively responds to observed payoffs under limited information, and a slow critic that deliberatively approximates the solution to the underlying dynamic programming problem. Crucially, the learning process relies on non-equilibrium adaptation through smoothed best responses to observed payoffs. We establish convergence to (approximate) equilibria in two-agent zero-sum and multi-agent identical-interest stochastic games over an infinite horizon. This provides one of the first payoff-based and fully decentralized learning algorithms with theoretical guarantees in both settings. Empirical results further validate the robustness and effectiveness of the proposed approach across both classes of games.

  • Open Access

    Distributionally Robust Multi-Agent Reinforcement Learning for Intelligent Traffic Control

    by Shuwei Pei, Joran Borger, Arda Kosay, Muhammed O. Sayin, Saeed Ahmed

    Learning-based traffic signal control is typically optimized for average performance under a few nominal demand patterns, which can result in poor behavior under atypical traffic conditions. To address this, we develop a distributionally robust multi-agent reinforcement learning framework for signal control on a 3x3 urban grid calibrated from a contiguous 3x3 subarea of central Athens covered by the pNEUMA trajectory dataset (Barmpounakis and Geroliminis, 2020). Our approach proceeds in three stages. First, we train a baseline multi-agent RL controller in which each intersection is governed by a proximal policy optimization agent with discrete signal phases, using a centralized training, decentralized execution paradigm. Second, to capture demand uncertainty, we construct eight heterogeneous origin-destination-based traffic scenarios-one directly derived from pNEUMA and seven synthetically generated-to span a wide range of spatial and temporal demand patterns. Over this scenario set, we train a contextual-bandit worst-case estimator that assigns mixture weights to estimate adversarial demand distributions conditioned on context. Finally, without modifying the controller architecture, we fine-tune the baseline multi-agent reinforcement learning agents under these estimated worst-case mixtures to obtain a distributionally robust multi-agent reinforcement learning controller. Across all eight scenarios, as well as on an unseen validation network based on the Sioux Falls configuration, the distributionally robust multi-agent reinforcement learning controller consistently reduces horizon-averaged queues and increases average speeds relative to the baseline, achieving up to 51% shorter queues and 38% higher speeds on the worst-performing scenarios.

  • Open Access

    Omniscient Attacker in Stochastic Security Games with Interdependent Nodes

    by Yuksel Arslantas, Ahmed Said Donmez, Ege Yuceel, Muhammed O. Sayin

    The adoption of reinforcement learning for critical infrastructure defense introduces a vulnerability where sophisticated attackers can strategically exploit the defense algorithm's learning dynamics. While prior work addresses this vulnerability in the context of repeated normal-form games, its extension to the stochastic games remains an open research gap. We close this gap by examining stochastic security games between an RL defender and an omniscient attacker, utilizing a tractable linear influence network model. To overcome the structural limitations of prior methods, we propose and apply neuro-dynamic programming. Our experimental results demonstrate that the omniscient attacker can significantly outperform a naive defender, highlighting the critical vulnerability introduced by the learning dynamics and the effectiveness of the proposed strategy.

  • Logit-Q Dynamics for Efficient Learning in Stochastic Teams

    by Ahmed Said Donmez, Onur Unlu, Muhammed O. Sayin

    We present a new family of logit-Q dynamics for efficient learning in stochastic games by combining the log-linear learning (also known as logit dynamics) for the repeated play of normal-form games with Q-learning for unknown Markov decision processes within the auxiliary stage-game framework. In this framework, we view stochastic games as agents repeatedly playing some stage game associated with the current state of the underlying game while the agents’ Q-functions determine the payoffs of these stage games. We show that the logit-Q dynamics presented reach (near) efficient equilibrium in stochastic teams with unknown dynamics and quantify the approximation error. We also show the rationality of the logit-Q dynamics against agents following pure stationary strategies. The key idea is to approximate the dynamics with a fictional scenario where the Q-function estimates are stationary over epochs whose lengths grow at a sufficiently slow rate. We then couple the dynamics in the main and fictional scenarios to show that these two scenarios become more and more similar across epochs due to the vanishing step size and growing epoch lengths.


  • Open Access

    Dynamic Feedback Strategies for Duopolies Over Partially Observed Consumer Networks

    by Saeed Ahmed, Muhammed O. Sayin

    This paper investigates dynamic advertising competition in duopoly markets operating over partially observed consumer networks. Traditional models often adopt impulse-like interventions that induce abrupt opinion changes; in contrast, we develop a continuous control framework motivated by the modern role of influencer marketing. By leveraging a linear time-invariant (LTI) system, we model consumer opinions as evolving through local interactions, where each firm allocates an advertising budget to a predetermined set of influencers. Recognizing that advertising efforts typically evolve more slowly than consumer opinions, our approach incorporates a two-timescale framework, enabling firms to steer the steady-state opinion profile in the presence of time-varying, unknown disturbances such as economic fluctuations. To address the challenge of incomplete information-arising from limited surveys—we propose a feedback strategy based on decentralized observers, ensuring robust estimation of the network state. Our analysis derives equilibrium strategies for the competing firms by extending time-varying optimization techniques to zero-sum games. This work not only advances theoretical understanding of dynamic competition over consumer networks but also offers practical guidelines for firms seeking to optimize advertising strategies under uncertainty and partial observability.


  • Convergence of Heterogeneous Learning Dynamics in Zero-Sum Stochastic Games

    by Yuksel Arslantas, Ege Yuceel, Yigit Yalin, Muhammed O. Sayin

    In this article, we present new families of algorithms for the repeated play of two-agent (near) zero-sum games and two-agent zero-sum stochastic games. For example, the family includes fictitious play and its variants as members. Commonly, the algorithms in this family are all uncoupled, rational, and convergent even in heterogeneous cases, e.g., where the dynamics may differ in terms of learning rates, full, none or temporal access to opponent actions, and model-based versus model-free learning. The convergence of heterogeneous dynamics is of practical interest especially in competitive environments since agents may have no means or interests in following the same dynamic with the same parameters. We prove that any mixture of such asymmetries does not impact the algorithms’ convergence to equilibrium (or near equilibrium if there is experimentation) in zero-sum games with repeated play and in zero-sum (irreducible) stochastic games with sufficiently small discount factors.


  • Open Access

    Balancing anarchy and efficiency: partial team formations and learning in potential games

    by Muhammed Sayin

    Non-cooperative multi-agent learning, focusing on individual rationality (anarchy), often falls short in achieving system-wide efficiency in potential games, a class of games with applications in decentralized control and optimization. On the other hand, cooperative approaches prioritize system efficiency but often via global coordination, which could be impractical, e.g., for large-scale and less controlled environments. To address this dilemma, we propose a novel framework that introduces partial team formations, allowing team members with shared objectives to coordinate their actions while maintaining team-wise rationality for improved system-wide efficiency without the burden of global coordination. We model such interactions as a multi-team game and analyze team equilibrium, where no team has an incentive to deviate unilaterally. We show that team formations preserve the potential game structure and guarantee improved or maintained worst-case equilibrium values. To learn effective team coordination, we leverage the Team-Fictitious Play (Team-FP) dynamics, allowing agents to adapt their strategies based on past interactions with other teams and teammates. However, learning to coordinate in the best team response against adapting opponents poses a challenge for the convergence analyses. We leverage a divide-and-conquer approach by dividing the horizon into epochs whose lengths grow sufficiently slowly to couple the evolution of the dynamics with a reference (stationary) scenario and address the non-stationarity challenge gradually across these epochs. We prove the almost sure convergence of Team-FP to equilibrium under standard conditions on the step sizes, ensuring the long-term stability and efficiency of the proposed framework. Our approach has significant implications for various decentralized control and optimization applications, including distributed resource allocation, traffic management, and power grid control. For example, we provide numerical experiments in the context of wireless network optimization, corroborating the theoretical convergence guarantees and demonstrating the effectiveness of Team-FP in achieving efficient and stable outcomes in practice.

  • Open Access

    Generalized Individual Q-learning for Polymatrix Games with Partial Observations

    by Ahmed Said Donmez, Muhammed O. Sayin

    This paper addresses the challenge of limited observations in non-cooperative multi-agent systems where agents can have partial access to other agents' actions. We present the generalized individual Q-learning dynamics that combine belief-based and payoff-based learning for the networked interconnections of more than two self-interested agents. This approach leverages access to opponents' actions whenever possible, demonstrably achieving a faster (guaranteed) convergence to quantal response equilibrium in multi-agent zero-sum and potential polymatrix games. Notably, the dynamics reduce to the well-studied smoothed fictitious play and individual Q-learning under full and no access to opponent actions, respectively. We further quantify the improvement in convergence rate due to observing opponents' actions through numerical simulations.

  • Strategizing Against Q-Learners: A Control-Theoretical Approach

    by Yuksel Arslantas, Ege Yuceel, Muhammed O. Sayin

    In this paper, we explore the susceptibility of the independent Q-learning algorithms (a classical and widely used multi-agent reinforcement learning method) to strategic manipulation of sophisticated opponents in normal-form games played repeatedly. We quantify how much strategically sophisticated agents can exploit naive Q-learners if they know the opponents’ Q-learning algorithm. To this end, we formulate the strategic actors’ interactions as a stochastic game (whose state encompasses Q-function estimates of the Q-learners) as if the Q-learning algorithms are the underlying dynamical system. We also present a quantization-based approximation scheme to tackle the continuum state space and analyze its performance for two competing strategic actors and a single strategic actor both analytically and numerically.