AMAbdullah Makkeh

Abdullah Makkeh

*This is an algorithmically generated author record based on publicly available information.

Abdullah Makkeh has a total of 26 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Redundancy maximization as a principle of associative memory learning in Hopfield networks

    by Mark Blümel, Andreas C. Schneider, Valentin Neuhaus, David A. Ehrlich, Marcel Graetz, Michael Wibral, Abdullah Makkeh, Viola Priesemann

    Associative memory, traditionally modeled by Hopfield networks, enables the retrieval of previously stored patterns from partial or noisy cues. Yet, the local computational principles which are required to enable this function remain incompletely understood. To formally characterize the local information processing in such systems, we employ a recent extension of information theory—Partial Information Decomposition. It decomposes the contribution of different inputs to an output into unique information from each input, redundant information across inputs, and synergistic information that emerges from combining different inputs. Applying this framework to individual neurons in classical Hopfield networks we find that below the memory capacity, the information in a neuron’s activity is characterized by high redundancy between the external pattern input and the internal recurrent input, while synergy and unique information are close to zero until the memory capacity is surpassed and performance drops steeply. Inspired by this observation, we use redundancy maximization at each neuron as an information-theoretic learning goal. This increases the network’s memory capacity to 1.59, a more than tenfold improvement over the 0.14 capacity of classical Hopfield networks, and also outperforming recent state-of-the-art implementations of Hopfield networks. Overall, this work establishes redundancy maximization as a design principle for associative memories and opens pathways for new associative memory models based on information-theoretic goals.

  • Open Access

    Higher-order interactions in neuronal function: From genes to ionic currents in biophysical models

    by Maria Reva, Alexis Arnaudon, Mickael Zbili, Abdullah Makkeh, Henry Markram, Jean-Marc Goaillard, Werner Van Geit

    Neuronal firing patterns are the consequence of precise variations in neuronal membrane potential, which are themselves shaped by multiple ionic currents. In this study, we use biophysical models, statistical methods, and information theory to explore the interaction between these ionic currents and neuron electrophysiological phenotype. We created numerous electrical models with diverse firing patterns. By analyzing these models, we identified intricate relationships between model parameters and electrical features. Our findings show that neuronal activity is often influenced by multiple biophysical model parameters, in a nonadditive (i.e. synergistic) fashion. When comparing this with single-cell RNAseq data, we found a contrasting structure: gene expression profiles were dominated by redundancy, reflecting differences in regulatory constraints and sampling diversity. This research sheds light on the complex links between biophysical parameters and neuronal phenotypes.


  • Open Access

    From Babel to Boole: the logical organization of information decompositions

    by A. J. Gutknecht, A. Makkeh, M. Wibral

    The field of partial information decomposition (PID) has become increasingly fragmented, with various proposed solutions approaching the problem from distinct perspectives. At the heart of each proposal lies what we call a PID-inducing concept —a type of information that, once formally specified, determines the PID components. Traditionally, redundancy has been the PID-inducing concept of choice, but other concepts, such as synergy, unique information and union information, have also been used. However, no cohesive logical framework exists to unify these approaches and clarify their relationships, particularly in the general case of n information sources. In this work, we introduce a unifying framework that connects PID components to Boolean functions and identifies PID-inducing concepts with logical conditions on these functions. We demonstrate that all existing PID-inducing concepts are special cases of this general construction. We also introduce novel concepts, including vulnerable information and partner concepts , and provide a comprehensive analysis of their properties, lattice structures and interrelationships.


  • Open Access

    A general framework for interpretable neural learning based on local information-theoretic goal functions

    by Abdullah Makkeh, Marcel Graetz, Andreas C. Schneider, David A. Ehrlich, Viola Priesemann, Michael Wibral

    Despite the impressive performance of biological and artificial networks, an intuitive understanding of how their local learning dynamics contribute to network-level task solutions remains a challenge to this date. Efforts to bring learning to a more local scale indeed lead to valuable insights, however, a general constructive approach to describe local learning goals that is both interpretable and adaptable across diverse tasks is still missing. We have previously formulated a local information processing goal that is highly adaptable and interpretable for a model neuron with compartmental structure. Building on recent advances in Partial Information Decomposition (PID), we here derive a corresponding parametric local learning rule, which allows us to introduce “infomorphic” neural networks. We demonstrate the versatility of these networks to perform tasks from supervised, unsupervised, and memory learning. By leveraging the interpretable nature of the PID framework, infomorphic networks represent a valuable tool to advance our understanding of the intricate structure of local learning.


  • Partial information decomposition for continuous variables based on shared exclusions: Analytical formulation and estimation

    by David A. Ehrlich, Kyle Schick-Poland, Abdullah Makkeh, Felix Lanfermann, Patricia Wollstadt, Michael Wibral

    Describing statistical dependencies is foundational to empirical scientific research. For uncovering intricate and possibly nonlinear dependencies between a single target variable and several source variables within a system, a principled and versatile framework can be found in the theory of partial information decomposition (PID). Nevertheless, the majority of existing PID measures are restricted to categorical variables, while many systems of interest in science are continuous. In this paper, we present a novel analytic formulation for continuous redundancy—a generalization of mutual information—drawing inspiration from the concept of shared exclusions in probability space as in the discrete PID definition of I∩sx. Furthermore, we introduce a nearest-neighbor-based estimator for continuous PID and showcase its effectiveness by applying it to a simulated energy management system provided by the Honda Research Institute Europe GmbH. This work bridges the gap between the measure-theoretically postulated existence proofs for a continuous I∩sx and its practical application to real-world scientific problems.


  • Open Access

    Quantifying Reinforcement-Learning Agent’s Autonomy, Reliance on Memory and Internalisation of the Environment

    by Anti Ingel, Abdullah Makkeh, Oriol Corcoll, Raul Vicente

    Intuitively, the level of autonomy of an agent is related to the degree to which the agent’s goals and behaviour are decoupled from the immediate control by the environment. Here, we capitalise on a recent information-theoretic formulation of autonomy and introduce an algorithm for calculating autonomy in a limiting process of time step approaching infinity. We tackle the question of how the autonomy level of an agent changes during training. In particular, in this work, we use the partial information decomposition (PID) framework to monitor the levels of autonomy and environment internalisation of reinforcement-learning (RL) agents. We performed experiments on two environments: a grid world, in which the agent has to collect food, and a repeating-pattern environment, in which the agent has to learn to imitate a sequence of actions by memorising the sequence. PID also allows us to answer how much the agent relies on its internal memory (versus how much it relies on the observations) when transitioning to its next internal state. The experiments show that specific terms of PID strongly correlate with the obtained reward and with the agent’s behaviour against perturbations in the observations.


  • Top Cited Papers
    Open Access

    Bits and pieces: understanding information decomposition from part-whole relationships and formal logic

    by A. J. Gutknecht, M. Wibral, A. Makkeh

    Partial information decomposition (PID) seeks to decompose the multivariate mutual information that a set of source variables contains about a target variable into basic pieces, the so-called ‘atoms of information’. Each atom describes a distinct way in which the sources may contain information about the target. For instance, some information may be contained uniquely in a particular source, some information may be shared by multiple sources and some information may only become accessible synergistically if multiple sources are combined. In this paper, we show that the entire theory of PID can be derived, firstly, from considerations of part-whole relationships between information atoms and mutual information terms, and secondly, based on a hierarchy of logical constraints describing how a given information atom can be accessed. In this way, the idea of a PID is developed on the basis of two of the most elementary relationships in nature: the part-whole relationship and the relation of logical implication. This unifying perspective provides insights into pressing questions in the field such as the possibility of constructing a PID based on concepts other than redundant information in the general n-sources case. Additionally, it admits of a particularly accessible exposition of PID theory.


  • Open Access

    Introducing a differentiable measure of pointwise shared information

    by Abdullah Makkeh, Aaron J. Gutknecht, Michael Wibral

    Partial information decomposition (PID) of the multivariate mutual information describes the distinct ways in which a set of source variables contains information about a target variable. The groundbreaking work of Williams and Beer has shown that this decomposition cannot be determined from classic information theory without making additional assumptions, and several candidate measures have been proposed, often drawing on principles from related fields such as decision theory. None of these measures is differentiable with respect to the underlying probability mass function. We here present a novel measure that satisfies this property, emerges solely from information-theoretic principles, and has the form of a local mutual information. We show how the measure can be understood from the perspective of exclusions of probability mass, a principle that is foundational to the original definition of the mutual information by Fano. Since our measure is well-defined for individual realizations of the random variables it lends itself for example to local learning in artificial neural networks. We also show that it has a meaningful M\"{o}bius inversion on a redundancy lattice and obeys a target chain rule. We give an operational interpretation of the measure based on the decisions that an agent should take if given only the shared information.

  • Introducing a differentiable measure of pointwise shared information

    by Abdullah Makkeh, Aaron J. Gutknecht, Michael Wibral

    Partial information decomposition of the multivariate mutual information describes the distinct ways in which a set of source variables contains information about a target variable. The groundbreaking work of Williams and Beer has shown that this decomposition cannot be determined from classic information theory without making additional assumptions, and several candidate measures have been proposed, often drawing on principles from related fields such as decision theory. None of these measures is differentiable with respect to the underlying probability mass function. We here present a measure that satisfies this property, emerges solely from information-theoretic principles, and has the form of a local mutual information. We show how the measure can be understood from the perspective of exclusions of probability mass, a principle that is foundational to the original definition of mutual information by Fano. Since our measure is well defined for individual realizations of random variables it lends itself, for example, to local learning in artificial neural networks. We also show that it has a meaningful Möbius inversion on a redundancy lattice and obeys a target chain rule. We give an operational interpretation of the measure based on the decisions that an agent should take if given only the shared information.


  • Open Access

    MAXENT3D_PID: An Estimator for the Maximum-Entropy Trivariate Partial Information Decomposition

    by Abdullah Makkeh, Daniel Chicharro, Dirk Oliver Theis, Raul Vicente

    Partial information decomposition (PID) separates the contributions of sources about a target into unique, redundant, and synergistic components of information. In essence, PID answers the question of “who knows what” of a system of random variables and hence has applications to a wide spectrum of fields ranging from social to biological sciences. The paper presents MaxEnt3D_Pid, an algorithm that computes the PID of three sources, based on a recently-proposed maximum entropy measure, using convex optimization (cone programming). We describe the algorithm and its associated software utilization and report the results of various experiments assessing its accuracy. Moreover, the paper shows that a hierarchy of bivariate and trivariate PID allows obtaining the finer quantities of the trivariate partial information measure.