Todd P. Coleman
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Todd P. Coleman has a total of 11 co-authors. Here are the top co-authors by number of shared publications:
- Negar Kiyavash
University of Illinois at Urbana-Champaign Urbana USA
3 shared publications - Christopher J. Quinn
University of Illinois 61801 Urbana Illinois
2 shared publications - Andrea Trevino
Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
1 shared publication - Ira S. Moskowitz
Naval Research Laboratory
1 shared publication - Jont B. Allen
Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA
1 shared publication - Myong Kang
US Naval Research Laboratory
1 shared publication
Latest Publications
- 1 August 2011
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 7 (1) , 64-75
Characterizing the Efficacy of the NRL Network Pump in Mitigating Covert Timing Channels
The Naval Research Laboratory (NRL) Network Pump, or Pump, is a standard for mitigating covert channels that arise in a multilevel secure (MLS) system when a high user (HU) sends acknowledgements to a low user (LU). The issue here is that HU can encode information in the "timings" of the acknowledgements. The Pump aims at mitigating the covert timing channel by introducing buffering between HU and LU, as well as adding noise to the acknowledgment timings. We model the working of the Pump in certain situations, as a communication system with feedback and use then this perspective to derive an upper bound on the capacity of the covert channel between HU and LU in the Pump. This upper bound is presented in terms of a directed information flow over the dynamics of the system. We also present an achievable scheme that can transmit information over this channel. When the support of the noise added by Pump to acknowledgment timings is finite, the achievable rate is nonzero, i.e., infinite number of bits can be reliably communicated. If the support of the noise is infinite, the achievable rate is zero and hence a finite number of bits can be communicated.
- 1 August 2011
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 7 (1) , 64-75
Characterizing the Efficacy of the NRL Network Pump in Mitigating Covert Timing Channels
The Naval Research Laboratory (NRL) Network Pump, or Pump, is a standard for mitigating covert channels that arise in a multilevel secure (MLS) system when a high user (HU) sends acknowledgements to a low user (LU). The issue here is that HU can encode information in the "timings" of the acknowledgements. The Pump aims at mitigating the covert timing channel by introducing buffering between HU and LU, as well as adding noise to the acknowledgment timings. We model the working of the Pump in certain situations, as a communication system with feedback and use then this perspective to derive an upper bound on the capacity of the covert channel between HU and LU in the Pump. This upper bound is presented in terms of a directed information flow over the dynamics of the system. We also present an achievable scheme that can transmit information over this channel. When the support of the noise added by Pump to acknowledgment timings is finite, the achievable rate is nonzero, i.e., infinite number of bits can be reliably communicated. If the support of the noise is infinite, the achievable rate is zero and hence a finite number of bits can be communicated.
- Open Access
Information-Theoretic Viewpoints on Optimal Causal Coding-Decoding Problems
In this paper we consider an interacting two-agent sequential decision-making problem consisting of a Markov source process, a causal encoder with feedback, and a causal decoder. Motivated by a desire to foster links between control and information theory, we augment the standard formulation by considering general alphabets and a cost function operating on current and previous symbols. Using dynamic programming, we provide a structural result whereby an optimal scheme exists that operates on appropriate sufficient statistics. We emphasize an example where the decoder alphabet lies in a space of beliefs on the source alphabet, and the additive cost function is a log likelihood ratio pertaining to sequential information gain. We also consider the inverse optimal control problem, where a fixed encoder/decoder pair satisfying statistical conditions is shown to be optimal for some cost function, using probabilistic matching. We provide examples of the applicability of this framework to communication with feedback, hidden Markov models and the nonlinear filter, decentralized control, brain-machine interfaces, and queuing theory.
Open AccessInformation-Theoretic Viewpoints on Optimal Causal Coding-Decoding Problems
In this paper we consider an interacting two-agent sequential decision-making problem consisting of a Markov source process, a causal encoder with feedback, and a causal decoder. Motivated by a desire to foster links between control and information theory, we augment the standard formulation by considering general alphabets and a cost function operating on current and previous symbols. Using dynamic programming, we provide a structural result whereby an optimal scheme exists that operates on appropriate sufficient statistics. We emphasize an example where the decoder alphabet lies in a space of beliefs on the source alphabet, and the additive cost function is a log likelihood ratio pertaining to sequential information gain. We also consider the inverse optimal control problem, where a fixed encoder/decoder pair satisfying statistical conditions is shown to be optimal for some cost function, using probabilistic matching. We provide examples of the applicability of this framework to communication with feedback, hidden Markov models and the nonlinear filter, decentralized control, brain-machine interfaces, and queuing theory.
- Open Access
Causal Dependence Tree Approximations of Joint Distributions for Multiple Random Processes
We investigate approximating joint distributions of random processes with causal dependence tree distributions. Such distributions are particularly useful in providing parsimonious representation when there exists causal dynamics among processes. By extending the results by Chow and Liu on dependence tree approximations, we show that the best causal dependence tree approximation is the one which maximizes the sum of directed informations on its edges, where best is defined in terms of minimizing the KL-divergence between the original and the approximate distribution. Moreover, we describe a low-complexity algorithm to efficiently pick this approximate distribution.
Open AccessCausal Dependence Tree Approximations of Joint Distributions for Multiple Random Processes
We investigate approximating joint distributions of random processes with causal dependence tree distributions. Such distributions are particularly useful in providing parsimonious representation when there exists causal dynamics among processes. By extending the results by Chow and Liu on dependence tree approximations, we show that the best causal dependence tree approximation is the one which maximizes the sum of directed informations on its edges, where best is defined in terms of minimizing the KL-divergence between the original and the approximate distribution. Moreover, we describe a low-complexity algorithm to efficiently pick this approximate distribution.
- 1 August 2010
- Published by MIT Press in Neural Computation
- Vol. 22 (8) , 2002-2030
A Computationally Efficient Method for Nonparametric Modeling of Neural Spiking Activity with Point Processes
Point-process models have been shown to be useful in characterizing neural spiking activity as a function of extrinsic and intrinsic factors. Most point-process models of neural activity are parametric, as they are often efficiently computable. However, if the actual point process does not lie in the assumed parametric class of functions, misleading inferences can arise. Nonparametric methods are attractive due to fewer assumptions, but computation in general grows with the size of the data. We propose a computationally efficient method for nonparametric maximum likelihood estimation when the conditional intensity function, which characterizes the point process in its entirety, is assumed to be a Lipschitz continuous function but otherwise arbitrary. We show that by exploiting much structure, the problem becomes efficiently solvable. We next demonstrate a model selection procedure to estimate the Lipshitz parameter from data, akin to the minimum description length principle and demonstrate consistency of our estimator under appropriate assumptions. Finally, we illustrate the effectiveness of our method with simulated neural spiking data, goldfish retinal ganglion neural data, and activity recorded in CA1 hippocampal neurons from an awake behaving rat. For the simulated data set, our method uncovers a more compact representation of the conditional intensity function when it exists. For the goldfish and rat neural data sets, we show that our nonparametric method gives a superior absolute goodness-of-fit measure used for point processes than the most common parametric and splines-based approaches.
- 1 August 2010
- Published by MIT Press in Neural Computation
- Vol. 22 (8) , 2002-2030
A Computationally Efficient Method for Nonparametric Modeling of Neural Spiking Activity with Point Processes
Point-process models have been shown to be useful in characterizing neural spiking activity as a function of extrinsic and intrinsic factors. Most point-process models of neural activity are parametric, as they are often efficiently computable. However, if the actual point process does not lie in the assumed parametric class of functions, misleading inferences can arise. Nonparametric methods are attractive due to fewer assumptions, but computation in general grows with the size of the data. We propose a computationally efficient method for nonparametric maximum likelihood estimation when the conditional intensity function, which characterizes the point process in its entirety, is assumed to be a Lipschitz continuous function but otherwise arbitrary. We show that by exploiting much structure, the problem becomes efficiently solvable. We next demonstrate a model selection procedure to estimate the Lipshitz parameter from data, akin to the minimum description length principle and demonstrate consistency of our estimator under appropriate assumptions. Finally, we illustrate the effectiveness of our method with simulated neural spiking data, goldfish retinal ganglion neural data, and activity recorded in CA1 hippocampal neurons from an awake behaving rat. For the simulated data set, our method uncovers a more compact representation of the conditional intensity function when it exists. For the goldfish and rat neural data sets, we show that our nonparametric method gives a superior absolute goodness-of-fit measure used for point processes than the most common parametric and splines-based approaches.
- 26 June 2010
- Published by Springer Nature in Journal of Computational Neuroscience
- Vol. 30 (1) , 17-44
Estimating the directed information to infer causal relationships in ensemble neural spike train recordings
Advances in recording technologies have given neuroscience researchers access to large amounts of data, in particular, simultaneous, individual recordings of large groups of neurons in different parts of the brain. A variety of quantitative techniques have been utilized to analyze the spiking activities of the neurons to elucidate the functional connectivity of the recorded neurons. In the past, researchers have used correlative measures. More recently, to better capture the dynamic, complex relationships present in the data, neuroscientists have employed causal measures—most of which are variants of Granger causality—with limited success. This paper motivates the directed information, an information and control theoretic concept, as a modality-independent embodiment of Granger’s original notion of causality. Key properties include: (a) it is nonzero if and only if one process causally influences another, and (b) its specific value can be interpreted as the strength of a causal relationship. We next describe how the causally conditioned directed information between two processes given knowledge of others provides a network version of causality: it is nonzero if and only if, in the presence of the present and past of other processes, one process causally influences another. This notion is shown to be able to differentiate between true direct causal influences, common inputs, and cascade effects in more two processes. We next describe a procedure to estimate the directed information on neural spike trains using point process generalized linear models, maximum likelihood estimation and information-theoretic model order selection. We demonstrate that on a simulated network of neurons, it (a) correctly identifies all pairwise causal relationships and (b) correctly identifies network causal relationships. This procedure is then used to analyze ensemble spike train recordings in primary motor cortex of an awake monkey while performing target reaching tasks, uncovering causal relationships whose directionality are consistent with predictions made from the wave propagation of simultaneously recorded local field potentials.
- 26 June 2010
- Published by Springer Nature in Journal of Computational Neuroscience
- Vol. 30 (1) , 17-44
Estimating the directed information to infer causal relationships in ensemble neural spike train recordings
Advances in recording technologies have given neuroscience researchers access to large amounts of data, in particular, simultaneous, individual recordings of large groups of neurons in different parts of the brain. A variety of quantitative techniques have been utilized to analyze the spiking activities of the neurons to elucidate the functional connectivity of the recorded neurons. In the past, researchers have used correlative measures. More recently, to better capture the dynamic, complex relationships present in the data, neuroscientists have employed causal measures—most of which are variants of Granger causality—with limited success. This paper motivates the directed information, an information and control theoretic concept, as a modality-independent embodiment of Granger’s original notion of causality. Key properties include: (a) it is nonzero if and only if one process causally influences another, and (b) its specific value can be interpreted as the strength of a causal relationship. We next describe how the causally conditioned directed information between two processes given knowledge of others provides a network version of causality: it is nonzero if and only if, in the presence of the present and past of other processes, one process causally influences another. This notion is shown to be able to differentiate between true direct causal influences, common inputs, and cascade effects in more two processes. We next describe a procedure to estimate the directed information on neural spike trains using point process generalized linear models, maximum likelihood estimation and information-theoretic model order selection. We demonstrate that on a simulated network of neurons, it (a) correctly identifies all pairwise causal relationships and (b) correctly identifies network causal relationships. This procedure is then used to analyze ensemble spike train recordings in primary motor cortex of an awake monkey while performing target reaching tasks, uncovering causal relationships whose directionality are consistent with predictions made from the wave propagation of simultaneously recorded local field potentials.
- 8 April 2009
- Published by Springer Nature in Journal of Computational Neuroscience
- Vol. 29 (1-2) , 193-201
A dynamical point process model of auditory nerve spiking in response to complex sounds
In this paper, we develop a dynamical point process model for how complex sounds are represented by neural spiking in auditory nerve fibers. Although many models have been proposed, our point process model is the first to capture elements of spontaneous rate, refractory effects, frequency selectivity, phase locking at low frequencies, and short-term adaptation, all within a compact parametric approach. Using a generalized linear model for the point process conditional intensity, driven by extrinsic covariates, previous spiking, and an input-dependent charging/discharging capacitor model, our approach robustly captures the aforementioned features on datasets taken at the auditory nerve of chinchilla in response to speech inputs. We confirm the goodness of fit of our approach using the Time-Rescaling Theorem for point processes.
- 8 April 2009
- Published by Springer Nature in Journal of Computational Neuroscience
- Vol. 29 (1-2) , 193-201
A dynamical point process model of auditory nerve spiking in response to complex sounds
In this paper, we develop a dynamical point process model for how complex sounds are represented by neural spiking in auditory nerve fibers. Although many models have been proposed, our point process model is the first to capture elements of spontaneous rate, refractory effects, frequency selectivity, phase locking at low frequencies, and short-term adaptation, all within a compact parametric approach. Using a generalized linear model for the point process conditional intensity, driven by extrinsic covariates, previous spiking, and an input-dependent charging/discharging capacitor model, our approach robustly captures the aforementioned features on datasets taken at the auditory nerve of chinchilla in response to speech inputs. We confirm the goodness of fit of our approach using the Time-Rescaling Theorem for point processes.