Mario Martin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Mario Martin has a total of 23 co-authors. Here are the top co-authors by number of shared publications:
Enrique Contreras-Hernández
Department of Physiology, Biophysics and Neurosciences, Center for Research and Advanced Studies, National Polytechnic Institute, Mexico City, Mexico
3 shared publications- Gennaro Esposito
Computer Science Department, Universitat Politècnica de Catalunya, Barcelona, Spain
3 shared publications Silvio Glusman
Department of Physiology, Biophysics and Neurosciences, Center for Research and Advanced Studies, National Polytechnic Institute, Mexico City, Mexico
3 shared publications- Ulises Cortés
Universitat Politecnica de Catalunya
3 shared publications Antonio Agudo
Institut de Robòtica i Informàtica Industrial CSIC -UPC
2 shared publicationsDiógenes Chávez
Department of Physiology, Biophysics and Neurosciences, Center for Research and Advanced Studies, National Polytechnic Institute, Mexico City, Mexico
2 shared publications
Latest Publications
- Open Access
On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis
We present a comprehensive study on discrete morphological symmetries of dynamical systems, which are commonly observed in biological and artificial locomoting systems, such as legged, swimming, and flying animals/robots/virtual characters. These symmetries arise from the presence of one or more planes/axis of symmetry in the system's morphology, resulting in harmonious duplication and distribution of body parts. Significantly, we characterize how morphological symmetries extend to symmetries in the system's dynamics, optimal control policies, and in all proprioceptive and exteroceptive measurements related to the system's dynamics evolution. In the context of data-driven methods, symmetry represents an inductive bias that justifies the use of data augmentation or symmetric function approximators. To tackle this, we present a theoretical and practical framework for identifying the system's morphological symmetry group $\G$ and characterizing the symmetries in proprioceptive and exteroceptive data measurements. We then exploit these symmetries using data augmentation and $\G$-equivariant neural networks. Our experiments on both synthetic and real-world applications provide empirical evidence of the advantageous outcomes resulting from the exploitation of these symmetries, including improved sample efficiency, enhanced generalization, and reduction of trainable parameters.Open AccessOn discrete symmetries of robotics systems: A group-theoretic and data-driven analysis
We present a comprehensive study on discrete morphological symmetries of dynamical systems, which are commonly observed in biological and artificial locomoting systems, such as legged, swimming, and flying animals/robots/virtual characters. These symmetries arise from the presence of one or more planes/axis of symmetry in the system's morphology, resulting in harmonious duplication and distribution of body parts. Significantly, we characterize how morphological symmetries extend to symmetries in the system's dynamics, optimal control policies, and in all proprioceptive and exteroceptive measurements related to the system's dynamics evolution. In the context of data-driven methods, symmetry represents an inductive bias that justifies the use of data augmentation or symmetric function approximators. To tackle this, we present a theoretical and practical framework for identifying the system's morphological symmetry group $\G$ and characterizing the symmetries in proprioceptive and exteroceptive data measurements. We then exploit these symmetries using data augmentation and $\G$-equivariant neural networks. Our experiments on both synthetic and real-world applications provide empirical evidence of the advantageous outcomes resulting from the exploitation of these symmetries, including improved sample efficiency, enhanced generalization, and reduction of trainable parameters. - Open Access
A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles
Underwater target localization using range-only and single-beacon (ROSB) techniques with autonomous vehicles has been used recently to improve the limitations of more complex methods, such as long baseline and ultra-short baseline systems. Nonetheless, in ROSB target localization methods, the trajectory of the tracking vehicle near the localized target plays an important role in obtaining the best accuracy of the predicted target position. Here, we investigate a Reinforcement Learning (RL) approach to find the optimal path that an autonomous vehicle should follow in order to increase and optimize the overall accuracy of the predicted target localization, while reducing time and power consumption. To accomplish this objective, different experimental tests have been designed using state-of-the-art deep RL algorithms. Our study also compares the results obtained with the analytical Fisher information matrix approach used in previous studies. The results revealed that the policy learned by the RL agent outperforms trajectories based on these analytical solutions, e.g. the median predicted error at the beginning of the target's localisation is 17% less. These findings suggest that using deep RL for localizing acoustic targets could be successfully applied to in-water applications that include tracking of acoustically tagged marine animals by autonomous underwater vehicles. This is envisioned as a first necessary step to validate the use of RL to tackle such problems, which could be used later on in a more complex scenarios
Open AccessA reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles
Underwater target localization using range-only and single-beacon (ROSB) techniques with autonomous vehicles has been used recently to improve the limitations of more complex methods, such as long baseline and ultra-short baseline systems. Nonetheless, in ROSB target localization methods, the trajectory of the tracking vehicle near the localized target plays an important role in obtaining the best accuracy of the predicted target position. Here, we investigate a Reinforcement Learning (RL) approach to find the optimal path that an autonomous vehicle should follow in order to increase and optimize the overall accuracy of the predicted target localization, while reducing time and power consumption. To accomplish this objective, different experimental tests have been designed using state-of-the-art deep RL algorithms. Our study also compares the results obtained with the analytical Fisher information matrix approach used in previous studies. The results revealed that the policy learned by the RL agent outperforms trajectories based on these analytical solutions, e.g. the median predicted error at the beginning of the target's localisation is 17% less. These findings suggest that using deep RL for localizing acoustic targets could be successfully applied to in-water applications that include tracking of acoustically tagged marine animals by autonomous underwater vehicles. This is envisioned as a first necessary step to validate the use of RL to tackle such problems, which could be used later on in a more complex scenarios
- Open Access
TransfQMix: Transformers for Leveraging the Graph Structure of Multi-Agent Reinforcement Learning Problems
Coordination is one of the most difficult aspects of multi-agent reinforcement learning (MARL). One reason is that agents normally choose their actions independently of one another. In order to see coordination strategies emerging from the combination of independent policies, the recent research has focused on the use of a centralized function (CF) that learns each agent's contribution to the team reward. However, the structure in which the environment is presented to the agents and to the CF is typically overlooked. We have observed that the features used to describe the coordination problem can be represented as vertex features of a latent graph structure. Here, we present TransfQMix, a new approach that uses transformers to leverage this latent structure and learn better coordination policies. Our transformer agents perform a graph reasoning over the state of the observable entities. Our transformer Q-mixer learns a monotonic mixing-function from a larger graph that includes the internal and external states of the agents. TransfQMix is designed to be entirely transferable, meaning that same parameters can be used to control and train larger or smaller teams of agents. This enables to deploy promising approaches to save training time and derive general policies in MARL, such as transfer learning, zero-shot transfer, and curriculum learning. We report TransfQMix's performances in the Spread and StarCraft II environments. In both settings, it outperforms state-of-the-art Q-Learning models, and it demonstrates effectiveness in solving problems that other methods can not solve.
Open AccessTransfQMix: Transformers for Leveraging the Graph Structure of Multi-Agent Reinforcement Learning Problems
Coordination is one of the most difficult aspects of multi-agent reinforcement learning (MARL). One reason is that agents normally choose their actions independently of one another. In order to see coordination strategies emerging from the combination of independent policies, the recent research has focused on the use of a centralized function (CF) that learns each agent's contribution to the team reward. However, the structure in which the environment is presented to the agents and to the CF is typically overlooked. We have observed that the features used to describe the coordination problem can be represented as vertex features of a latent graph structure. Here, we present TransfQMix, a new approach that uses transformers to leverage this latent structure and learn better coordination policies. Our transformer agents perform a graph reasoning over the state of the observable entities. Our transformer Q-mixer learns a monotonic mixing-function from a larger graph that includes the internal and external states of the agents. TransfQMix is designed to be entirely transferable, meaning that same parameters can be used to control and train larger or smaller teams of agents. This enables to deploy promising approaches to save training time and derive general policies in MARL, such as transfer learning, zero-shot transfer, and curriculum learning. We report TransfQMix's performances in the Spread and StarCraft II environments. In both settings, it outperforms state-of-the-art Q-Learning models, and it demonstrates effectiveness in solving problems that other methods can not solve.
- Open Access
An Adaptable Approach to Learn Realistic Legged Locomotion without Examples
Learning controllers that reproduce legged locomotion in nature has been a long-time goal in robotics and computer graphics. While yielding promising results, recent approaches are not yet flexible enough to be applicable to legged systems of different morphologies. This is partly because they often rely on precise motion capture references or elaborate learning environments that ensure the naturality of the emergent locomotion gaits but prevent generalization. This work proposes a generic approach for ensuring realism in locomotion by guiding the learning process with the spring-loaded inverted pendulum model as a reference. Leveraging on the exploration capacities of Reinforcement Learning (RL), we learn a control policy that fills in the information gap between the template model and full-body dynamics required to maintain stable and periodic locomotion. The proposed approach can be applied to robots of different sizes and morphologies and adapted to any RL technique and control architecture. We present experimental results showing that even in a model-free setup and with a simple reactive control architecture, the learned policies can generate realistic and energy-efficient locomotion gaits for a bipedal and a quadrupedal robot. And most importantly, this is achieved without using motion capture, strong constraints in the dynamics or kinematics of the robot, nor prescribing limb coordination. We provide supplemental videos for qualitative analysis of the naturality of the learned gaits.
Open AccessAn Adaptable Approach to Learn Realistic Legged Locomotion without Examples
Learning controllers that reproduce legged locomotion in nature has been a long-time goal in robotics and computer graphics. While yielding promising results, recent approaches are not yet flexible enough to be applicable to legged systems of different morphologies. This is partly because they often rely on precise motion capture references or elaborate learning environments that ensure the naturality of the emergent locomotion gaits but prevent generalization. This work proposes a generic approach for ensuring realism in locomotion by guiding the learning process with the spring-loaded inverted pendulum model as a reference. Leveraging on the exploration capacities of Reinforcement Learning (RL), we learn a control policy that fills in the information gap between the template model and full-body dynamics required to maintain stable and periodic locomotion. The proposed approach can be applied to robots of different sizes and morphologies and adapted to any RL technique and control architecture. We present experimental results showing that even in a model-free setup and with a simple reactive control architecture, the learned policies can generate realistic and energy-efficient locomotion gaits for a bipedal and a quadrupedal robot. And most importantly, this is achieved without using motion capture, strong constraints in the dynamics or kinematics of the robot, nor prescribing limb coordination. We provide supplemental videos for qualitative analysis of the naturality of the learned gaits.
- Open Access
Coin_flipper at eHealth-KD Challenge 2019: Voting LSTMs for Key Phrases and Semantic Relation Identification Applied to Spanish eHealth Texts
This paper describes our approach presented for the eHealth-KD 2019 challenge. Our participation was aimed at testing how far we could go using generic tools for Text-Processing but, at the same time, using common optimization techniques in the field of Data Mining. The architecture proposed for both tasks of the challenge is a standard stacked 2-layer bi-LSTM. The main particularities of our approach are: (a) The use of a surrogate function of F1 as loss function to close the gap between the minimization function and the evaluation metric, and (b) The generation of an ensemble of models for generating predictions by majority vote. Our system ranked second with an F1 score of 62.18% in the main task by a narrow margin with the winner that scored 63.94%.
Open AccessCoin_flipper at eHealth-KD Challenge 2019: Voting LSTMs for Key Phrases and Semantic Relation Identification Applied to Spanish eHealth Texts
This paper describes our approach presented for the eHealth-KD 2019 challenge. Our participation was aimed at testing how far we could go using generic tools for Text-Processing but, at the same time, using common optimization techniques in the field of Data Mining. The architecture proposed for both tasks of the challenge is a standard stacked 2-layer bi-LSTM. The main particularities of our approach are: (a) The use of a surrogate function of F1 as loss function to close the gap between the minimization function and the evaluation metric, and (b) The generation of an ensemble of models for generating predictions by majority vote. Our system ranked second with an F1 score of 62.18% in the main task by a narrow margin with the winner that scored 63.94%.
- Open Access
- 26 September 2019
- Published by Frontiers Media SA in Frontiers in Systems Neuroscience
- Vol. 13, 47
Supraspinal Shaping of Adaptive Transitions in the State of Functional Connectivity Between Segmentally Distributed Dorsal Horn Neuronal Populations in Response to Nociception and Antinociception
In the anesthetized cat the correlation between the ongoing cord dorsum potentials (CDPs) recorded from different lumbar spinal segments has a non-random structure, suggesting relatively stable patterns of functional connectivity between the dorsal horn neuronal ensembles involved in the generation of these potentials. During the nociception induced by the intradermic injection of capsaicin, the patterns of segmental correlation between the spontaneous CDPs acquire other non-random configurations that are temporarily reversed to their pre-capsaicin state by the systemic injection of lidocaine, a procedure known to decrease the manifestation of neuropathic pain in both animals and humans. We have now extended these studies and utilized machine learning for the automatic extraction and selection of particular classes of CDPs according to their shapes and amplitudes. By using a Markovian analysis, we disclosed the transitions between the different kinds of CDPs induced by capsaicin and lidocaine and constructed a global model based on the changes in the behavior of the CDPs generated along the whole set of lumbar segments. This allowed the identification of the different states of functional connectivity within the whole ensemble of dorsal horn neurones attained during nociception and their transitory reversal by systemic administration of lidocaine in preparations with the intact neuroaxis and after spinalization. The present observations provide additional information on the state of self-organized criticality that leads to the adaptive behavior of the dorsal horn neuronal networks during nociception and antinociception both shaped by supraspinal descending influences.
- 26 September 2019
- Published by Frontiers Media SA in Frontiers in Systems Neuroscience
- Vol. 13, 47
Open AccessSupraspinal Shaping of Adaptive Transitions in the State of Functional Connectivity Between Segmentally Distributed Dorsal Horn Neuronal Populations in Response to Nociception and Antinociception
In the anesthetized cat the correlation between the ongoing cord dorsum potentials (CDPs) recorded from different lumbar spinal segments has a non-random structure, suggesting relatively stable patterns of functional connectivity between the dorsal horn neuronal ensembles involved in the generation of these potentials. During the nociception induced by the intradermic injection of capsaicin, the patterns of segmental correlation between the spontaneous CDPs acquire other non-random configurations that are temporarily reversed to their pre-capsaicin state by the systemic injection of lidocaine, a procedure known to decrease the manifestation of neuropathic pain in both animals and humans. We have now extended these studies and utilized machine learning for the automatic extraction and selection of particular classes of CDPs according to their shapes and amplitudes. By using a Markovian analysis, we disclosed the transitions between the different kinds of CDPs induced by capsaicin and lidocaine and constructed a global model based on the changes in the behavior of the CDPs generated along the whole set of lumbar segments. This allowed the identification of the different states of functional connectivity within the whole ensemble of dorsal horn neurones attained during nociception and their transitory reversal by systemic administration of lidocaine in preparations with the intact neuroaxis and after spinalization. The present observations provide additional information on the state of self-organized criticality that leads to the adaptive behavior of the dorsal horn neuronal networks during nociception and antinociception both shaped by supraspinal descending influences.
- Open Access
- 1 May 2017
- Published by Frontiers Media SA in Frontiers in Computational Neuroscience
- Vol. 11, 32
Markovian Analysis of the Sequential Behavior of the Spontaneous Spinal Cord Dorsum Potentials Induced by Acute Nociceptive Stimulation in the Anesthetized Cat
In a previous study we developed a Machine Learning procedure for the automatic identification and classification of spontaneous cord dorsum potentials (CDP). This study further supported the proposal that in the anesthetized cat, the spontaneous CDPs recorded from different lumbar spinal segments are generated by a distributed network of dorsal horn neurons with structured (non-random) patterns of functional connectivity and that these configurations can be changed to other non-random and stable configurations after the noceptive stimulation produced by the intradermic injection of capsaicin in the anesthetized cat. Here we present a study showing that the sequence of identified forms of the spontaneous CDPs follows a Markov chain of at least order one. That is, the system has memory in the sense that the spontaneous activation of dorsal horn neuronal ensembles producing the CDPs is not independent of the most recent activity. We used this markovian property to build a procedure to identify portions of signals as belonging to a specific functional state of connectivity among the neuronal networks involved in the generation of the CDP. We have tested this procedure during acute nociceptive stimulation produced by the intradermic injection of capsaicin in intact as well as spinalized preparations. Altogether, our results indicate that CDP sequences cannot be generated by a renewal stochastic process. Moreover, it is possible to describe some functional features of activity in the cord dorsum by modeling the \CDP\ sequences as generated by a Markov order one stochastic process. Finally, these Markov models make possible to determine the functional state which produced a CDP sequence. The proposed identification procedures appear to be useful for the analysis of the sequential behavior of the ongoing CDPs recorded from different spinal segments in response to a variety of experimental procedures including the changes produced by acute nociceptive stimulation. They are envisaged as a useful tool to examine alterations of the patterns of functional connectivity between dorsal horn neurons under normal and different pathological conditions, an issue of potential clinical concern.
- 1 May 2017
- Published by Frontiers Media SA in Frontiers in Computational Neuroscience
- Vol. 11, 32
Open AccessMarkovian Analysis of the Sequential Behavior of the Spontaneous Spinal Cord Dorsum Potentials Induced by Acute Nociceptive Stimulation in the Anesthetized Cat
In a previous study we developed a Machine Learning procedure for the automatic identification and classification of spontaneous cord dorsum potentials (CDP). This study further supported the proposal that in the anesthetized cat, the spontaneous CDPs recorded from different lumbar spinal segments are generated by a distributed network of dorsal horn neurons with structured (non-random) patterns of functional connectivity and that these configurations can be changed to other non-random and stable configurations after the noceptive stimulation produced by the intradermic injection of capsaicin in the anesthetized cat. Here we present a study showing that the sequence of identified forms of the spontaneous CDPs follows a Markov chain of at least order one. That is, the system has memory in the sense that the spontaneous activation of dorsal horn neuronal ensembles producing the CDPs is not independent of the most recent activity. We used this markovian property to build a procedure to identify portions of signals as belonging to a specific functional state of connectivity among the neuronal networks involved in the generation of the CDP. We have tested this procedure during acute nociceptive stimulation produced by the intradermic injection of capsaicin in intact as well as spinalized preparations. Altogether, our results indicate that CDP sequences cannot be generated by a renewal stochastic process. Moreover, it is possible to describe some functional features of activity in the cord dorsum by modeling the \CDP\ sequences as generated by a Markov order one stochastic process. Finally, these Markov models make possible to determine the functional state which produced a CDP sequence. The proposed identification procedures appear to be useful for the analysis of the sequential behavior of the ongoing CDPs recorded from different spinal segments in response to a variety of experimental procedures including the changes produced by acute nociceptive stimulation. They are envisaged as a useful tool to examine alterations of the patterns of functional connectivity between dorsal horn neurons under normal and different pathological conditions, an issue of potential clinical concern.
- 18 April 2017
- Published by Springer Nature in Applied Intelligence
- Vol. 47 (3) , 670-704
Bellman residuals minimization using online support vector machines
- 18 April 2017
- Published by Springer Nature in Applied Intelligence
- Vol. 47 (3) , 670-704
Bellman residuals minimization using online support vector machines
- Open Access
- 26 August 2015
- Published by Frontiers Media SA in Frontiers in Neuroinformatics
- Vol. 9, 21
A machine learning methodology for the selection and classification of spontaneous spinal cord dorsum potentials allows disclosure of structured (non-random) changes in neuronal connectivity induced by nociceptive stimulation
Fractal analysis of spontaneous cord dorsum potentials (CDPs) generated in the lumbosacral spinal segments has revealed that these potentials are generated by ongoing structured (non-random) neuronal activity. Studies aimed to disclose the changes produced by nociceptive stimulation on the functional organization of the neuronal networks generating these potentials used predetermined templates to select specific classes of spontaneous CDPs. Since this procedure was time consuming and required continuous supervision, it was limited to the analysis of two types of CDPs (negative CDPs and negative positive CDPs), thus excluding potentials that may reflect activation of other neuronal networks of presumed functional relevance. We now present a novel procedure based in machine learning that allows the efficient and unbiased selection of a variety of spontaneous CDPs with different shapes and amplitudes. The reliability and performance of the method is evaluated by analyzing the effects on the probabilities of generation of different types of spontaneous CDPs induced by the intradermic injection of small amounts of capsaicin in the anesthetized cat. The results obtained with the selection method presently described allowed detection of spontaneous CDPs with specific shapes and amplitudes that are assumed to represent the activation of functionally coupled sets of dorsal horn neurones that acquire different, structured configurations in response to nociceptive stimuli.
- 26 August 2015
- Published by Frontiers Media SA in Frontiers in Neuroinformatics
- Vol. 9, 21
Open AccessA machine learning methodology for the selection and classification of spontaneous spinal cord dorsum potentials allows disclosure of structured (non-random) changes in neuronal connectivity induced by nociceptive stimulation
Fractal analysis of spontaneous cord dorsum potentials (CDPs) generated in the lumbosacral spinal segments has revealed that these potentials are generated by ongoing structured (non-random) neuronal activity. Studies aimed to disclose the changes produced by nociceptive stimulation on the functional organization of the neuronal networks generating these potentials used predetermined templates to select specific classes of spontaneous CDPs. Since this procedure was time consuming and required continuous supervision, it was limited to the analysis of two types of CDPs (negative CDPs and negative positive CDPs), thus excluding potentials that may reflect activation of other neuronal networks of presumed functional relevance. We now present a novel procedure based in machine learning that allows the efficient and unbiased selection of a variety of spontaneous CDPs with different shapes and amplitudes. The reliability and performance of the method is evaluated by analyzing the effects on the probabilities of generation of different types of spontaneous CDPs induced by the intradermic injection of small amounts of capsaicin in the anesthetized cat. The results obtained with the selection method presently described allowed detection of spontaneous CDPs with specific shapes and amplitudes that are assumed to represent the activation of functionally coupled sets of dorsal horn neurones that acquire different, structured configurations in response to nociceptive stimuli.