Sina Molavipour
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Sina Molavipour has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
Mikael Skoglund
School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm Sweden
5 shared publications- Germán Bassi
Division of Information Science and Engineering (ISE KTH Royal Institute of Technology
4 shared publications - Foroogh Sadat Tabataba
Department of Electrical and Computer Engineering, Isfahan University of Technology, Isfahan, Iran
2 shared publications Germán Bassi
School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm Sweden
2 shared publications- Mikael Skoglund
Department of Information Science and Engineering, KTH Royal Institute of Technology, Stockholm, Sweden
2 shared publications Amin Gohari
1 shared publication
Latest Publications
- Open Access
Neural Estimator of Information for Time-Series Data with Dependency
Novel approaches to estimate information measures using neural networks are well-celebrated in recent years both in the information theory and machine learning communities. These neural-based estimators are shown to converge to the true values when estimating mutual information and conditional mutual information using independent samples. However, if the samples in the dataset are not independent, the consistency of these estimators requires further investigation. This is of particular interest for a more complex measure such as the directed information, which is pivotal in characterizing causality and is meaningful over time-dependent variables. The extension of the convergence proof for such cases is not trivial and demands further assumptions on the data. In this paper, we show that our neural estimator for conditional mutual information is consistent when the dataset is generated with samples of a stationary and ergodic source. In other words, we show that our information estimator using neural networks converges asymptotically to the true value with probability one. Besides universal functional approximation of neural networks, a core lemma to show the convergence is Birkhoff’s ergodic theorem. Additionally, we use the technique to estimate directed information and demonstrate the effectiveness of our approach in simulations.
Open AccessNeural Estimator of Information for Time-Series Data with Dependency
Novel approaches to estimate information measures using neural networks are well-celebrated in recent years both in the information theory and machine learning communities. These neural-based estimators are shown to converge to the true values when estimating mutual information and conditional mutual information using independent samples. However, if the samples in the dataset are not independent, the consistency of these estimators requires further investigation. This is of particular interest for a more complex measure such as the directed information, which is pivotal in characterizing causality and is meaningful over time-dependent variables. The extension of the convergence proof for such cases is not trivial and demands further assumptions on the data. In this paper, we show that our neural estimator for conditional mutual information is consistent when the dataset is generated with samples of a stationary and ergodic source. In other words, we show that our information estimator using neural networks converges asymptotically to the true value with probability one. Besides universal functional approximation of neural networks, a core lemma to show the convergence is Birkhoff’s ergodic theorem. Additionally, we use the technique to estimate directed information and demonstrate the effectiveness of our approach in simulations.
- 11 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 69 (1053587X) , 766-780
Neural Estimators for Conditional Mutual Information Using Nearest Neighbors Sampling
The estimation of mutual information (MI) or conditional mutual information (CMI) from a set of samples is a longstanding problem. A recent line of work in this area has leveraged the approximation power of artificial neural networks and has shown improvements over conventional methods. One important challenge in this new approach is the need to obtain, given the original dataset, a different set where the samples are distributed according to a specific product density function. This is particularly challenging when estimating CMI. In this paper, we introduce a new technique, based on k nearest neighbors (k-NN), to perform the resampling and derive high-confidence concentration bounds for the sample average. Then the technique is employed to train a neural network classifier and the CMI is estimated accordingly. We propose three estimators using this technique and prove their consistency, make a comparison between them and similar approaches in the literature, and experimentally show improvements in estimating the CMI in terms of accuracy and variance of the estimators.
- 11 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 69 (1053587X) , 766-780
Neural Estimators for Conditional Mutual Information Using Nearest Neighbors Sampling
The estimation of mutual information (MI) or conditional mutual information (CMI) from a set of samples is a longstanding problem. A recent line of work in this area has leveraged the approximation power of artificial neural networks and has shown improvements over conventional methods. One important challenge in this new approach is the need to obtain, given the original dataset, a different set where the samples are distributed according to a specific product density function. This is particularly challenging when estimating CMI. In this paper, we introduce a new technique, based on k nearest neighbors (k-NN), to perform the resampling and derive high-confidence concentration bounds for the sample average. Then the technique is employed to train a neural network classifier and the CMI is estimated accordingly. We propose three estimators using this technique and prove their consistency, make a comparison between them and similar approaches in the literature, and experimentally show improvements in estimating the CMI in terms of accuracy and variance of the estimators.
- Open Access
Causality Graph of Vehicular Traffic Flow
In an intelligent transportation system, the effects and relations of traffic flow at different points in a network are valuable features which can be exploited for control system design and traffic forecasting. In this paper, we define the notion of causality based on the directed information, a well-established data-driven measure, to represent the effective connectivity among nodes of a vehicular traffic network. This notion indicates whether the traffic flow at any given point affects another point's flow in the future and, more importantly, reveals the extent of this effect. In contrast with conventional methods to express connections in a network, it is not limited to linear models and normality conditions. In this work, directed information is used to determine the underlying graph structure of a network, denoted directed information graph, which expresses the causal relations among nodes in the network. We devise an algorithm to estimate the extent of the effects in each link and build the graph. The performance of the algorithm is then analyzed with synthetic data and real aggregated data of vehicular traffic.
Open AccessCausality Graph of Vehicular Traffic Flow
In an intelligent transportation system, the effects and relations of traffic flow at different points in a network are valuable features which can be exploited for control system design and traffic forecasting. In this paper, we define the notion of causality based on the directed information, a well-established data-driven measure, to represent the effective connectivity among nodes of a vehicular traffic network. This notion indicates whether the traffic flow at any given point affects another point's flow in the future and, more importantly, reveals the extent of this effect. In contrast with conventional methods to express connections in a network, it is not limited to linear models and normality conditions. In this work, directed information is used to determine the underlying graph structure of a network, denoted directed information graph, which expresses the causal relations among nodes in the network. We devise an algorithm to estimate the extent of the effects in each link and build the graph. The performance of the algorithm is then analyzed with synthetic data and real aggregated data of vehicular traffic.
- 1 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5025-5029
Conditional Mutual Information Neural Estimator
Several recent works in communication systems have proposed to leverage the power of neural networks in the design of encoders and decoders. In this approach, these blocks can be tailored to maximize the transmission rate based on aggregated samples from the channel. Motivated by the fact that, in many communication schemes, the achievable transmission rate is determined by a conditional mutual information term, this paper focuses on neural-based estimators for this information-theoretic quantity. Our results are based on variational bounds for the KL-divergence and, in contrast to some previous works, we provide a mathematically rigorous lower bound. However, additional challenges with respect to the un-conditional mutual information emerge due to the presence of a conditional density function which we address here.
- 1 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 5025-5029
Conditional Mutual Information Neural Estimator
Several recent works in communication systems have proposed to leverage the power of neural networks in the design of encoders and decoders. In this approach, these blocks can be tailored to maximize the transmission rate based on aggregated samples from the channel. Motivated by the fact that, in many communication schemes, the achievable transmission rate is determined by a conditional mutual information term, this paper focuses on neural-based estimators for this information-theoretic quantity. Our results are based on variational bounds for the KL-divergence and, in contrast to some previous works, we provide a mathematically rigorous lower bound. However, additional challenges with respect to the un-conditional mutual information emerge due to the presence of a conditional density function which we address here.
- 12 August 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 68 (10) , 10134-10144
Multirate Packet Delivery in Heterogeneous Broadcast Networks
In this paper, we study the problem of multirate packet delivery in heterogeneous packet erasure broadcast networks. The technical challenge is to enable users receive packets at different rates, as dictated by the quality of their individual channel. We present a new analytical framework for characterizing the delivery rate and delivery delay performance of a previously proposed non-block-based network coding scheme in the literature. This scheme was studied in homogeneous network settings. We show for the first time, via new theoretical analysis and simulations that it can actually achieve multirate packet delivery. Using acknowledgments from each user, we show that the user with the highest link capacity achieves the maximum possible throughput. Also, a non-zero packet delivery rate is possible for other users, and the delivery rate depends on the difference between the packet arrival rate at the sender and the link capacity of each user. The accuracy of our analytical framework is confirmed by comparing the results with simulations for different settings of packet arrival rate at the sender and link capacities.
- 12 August 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Vehicular Technology
- Vol. 68 (10) , 10134-10144
Multirate Packet Delivery in Heterogeneous Broadcast Networks
In this paper, we study the problem of multirate packet delivery in heterogeneous packet erasure broadcast networks. The technical challenge is to enable users receive packets at different rates, as dictated by the quality of their individual channel. We present a new analytical framework for characterizing the delivery rate and delivery delay performance of a previously proposed non-block-based network coding scheme in the literature. This scheme was studied in homogeneous network settings. We show for the first time, via new theoretical analysis and simulations that it can actually achieve multirate packet delivery. Using acknowledgments from each user, we show that the user with the highest link capacity achieves the maximum possible throughput. Also, a non-zero packet delivery rate is possible for other users, and the delivery rate depends on the difference between the packet arrival rate at the sender and the link capacity of each user. The accuracy of our analytical framework is confirmed by comparing the results with simulations for different settings of packet arrival rate at the sender and link capacities.
- Open Access
Multirate Packet Delivery In Heterogeneous Broadcast Networks
In this paper, we study the problem of multirate packet delivery in heterogeneous packet erasure broadcast networks. The technical challenge is to enable users receive packets at different rates, as dictated by the quality of their individual channel. We present a new analytical framework for characterizing the delivery rate and delivery delay performance of a previously proposed non-block-based network coding scheme in the literature. This scheme was studied in homogeneous network settings. We show for the first time, via new theoretical analysis and simulations that it can actually achieve multirate packet delivery. Using acknowledgments from each user, we show that the user with the highest link capacity achieves the maximum possible throughput. Also, a non-zero packet delivery rate is possible for other users, and the delivery rate depends on the difference between the packet arrival rate at the sender and the link capacity of each user. The accuracy of our analytical framework is confirmed by comparing the results with simulations for different settings of packet arrival rate at the sender and link capacities.
Open AccessMultirate Packet Delivery In Heterogeneous Broadcast Networks
In this paper, we study the problem of multirate packet delivery in heterogeneous packet erasure broadcast networks. The technical challenge is to enable users receive packets at different rates, as dictated by the quality of their individual channel. We present a new analytical framework for characterizing the delivery rate and delivery delay performance of a previously proposed non-block-based network coding scheme in the literature. This scheme was studied in homogeneous network settings. We show for the first time, via new theoretical analysis and simulations that it can actually achieve multirate packet delivery. Using acknowledgments from each user, we show that the user with the highest link capacity achieves the maximum possible throughput. Also, a non-zero packet delivery rate is possible for other users, and the delivery rate depends on the difference between the packet arrival rate at the sender and the link capacity of each user. The accuracy of our analytical framework is confirmed by comparing the results with simulations for different settings of packet arrival rate at the sender and link capacities.
- Open Access
Non-Asymptotic Behavior of the Maximum Likelihood Estimate of a Discrete Distribution
In this paper, we study the maximum likelihood estimate of the probability mass function (pmf) of $n$ independent and identically distributed (i.i.d.) random variables, in the non-asymptotic regime. We are interested in characterizing the Neyman--Pearson criterion, i.e., the log-likelihood ratio for testing a true hypothesis within a larger hypothesis. Wilks' theorem states that this ratio behaves like a $\chi^2$ random variable in the asymptotic case; however, less is known about the precise behavior of the ratio when the number of samples is finite. In this work, we find an explicit bound for the difference between the cumulative distribution function (cdf) of the log-likelihood ratio and the cdf of a $\chi^2$ random variable. Furthermore, we show that this difference vanishes with a rate of order $1/\sqrt{n}$ in accordance with Wilks' theorem.Open AccessNon-Asymptotic Behavior of the Maximum Likelihood Estimate of a Discrete Distribution
In this paper, we study the maximum likelihood estimate of the probability mass function (pmf) of $n$ independent and identically distributed (i.i.d.) random variables, in the non-asymptotic regime. We are interested in characterizing the Neyman--Pearson criterion, i.e., the log-likelihood ratio for testing a true hypothesis within a larger hypothesis. Wilks' theorem states that this ratio behaves like a $\chi^2$ random variable in the asymptotic case; however, less is known about the precise behavior of the ratio when the number of samples is finite. In this work, we find an explicit bound for the difference between the cumulative distribution function (cdf) of the log-likelihood ratio and the cdf of a $\chi^2$ random variable. Furthermore, we show that this difference vanishes with a rate of order $1/\sqrt{n}$ in accordance with Wilks' theorem. - 1 October 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 212-219
Testing for directed information graphs
In this paper, we study a hypothesis test to determine the underlying directed graph structure of nodes in a network, where the nodes represent random processes and the direction of the links indicate a causal relationship between said processes. Specifically, a k-th order Markov structure is considered for them, and the chosen metric to determine a connection between nodes is the directed information. The hypothesis test is based on the empirically calculated transition probabilities which are used to estimate the directed information. For a single edge, it is proven that the detection probability can be chosen arbitrarily close to one, while the false alarm probability remains negligible. When the test is performed on the whole graph, we derive bounds for the false alarm and detection probabilities, which show that the test is asymptotically optimal by properly setting the threshold test and using a large number of samples. Furthermore, we study how the convergence of the measures relies on the existence of links in the true graph.
- 1 October 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 212-219
Testing for directed information graphs
In this paper, we study a hypothesis test to determine the underlying directed graph structure of nodes in a network, where the nodes represent random processes and the direction of the links indicate a causal relationship between said processes. Specifically, a k-th order Markov structure is considered for them, and the chosen metric to determine a connection between nodes is the directed information. The hypothesis test is based on the empirically calculated transition probabilities which are used to estimate the directed information. For a single edge, it is proven that the detection probability can be chosen arbitrarily close to one, while the false alarm probability remains negligible. When the test is performed on the whole graph, we derive bounds for the false alarm and detection probabilities, which show that the test is asymptotically optimal by properly setting the threshold test and using a large number of samples. Furthermore, we study how the convergence of the measures relies on the existence of links in the true graph.
- 1 May 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1067-1072
Improved performance bounds for the infinite-dimensional Witsenhausen problem
- 1 May 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1067-1072
Improved performance bounds for the infinite-dimensional Witsenhausen problem
- 15 September 2015
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Communications Letters
- Vol. 19 (11) , 1929-1932
Recovery From Random Samples in a Big Data Set
Consider a collection of files, each of which is a sequence of letters. One of these files is randomly chosen and a random subsequence of the file is revealed. This random subsequence can be the result of a random sampling of the file. The goal is to recover the identity of the file, assuming a simple greedy matching algorithm to search the file collection. We study the fundamental limits on the maximum size of the file collection for reliable recovery in terms of the length of the random subsequence. The sequence of each file is assumed to follow a hidden Markov model (HMM), which is a common model for many data structures such as voice or DNA sequences. The connection between this problem and coding over a deletion channel with greedy decoders is discussed.
- 15 September 2015
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Communications Letters
- Vol. 19 (11) , 1929-1932
Recovery From Random Samples in a Big Data Set
Consider a collection of files, each of which is a sequence of letters. One of these files is randomly chosen and a random subsequence of the file is revealed. This random subsequence can be the result of a random sampling of the file. The goal is to recover the identity of the file, assuming a simple greedy matching algorithm to search the file collection. We study the fundamental limits on the maximum size of the file collection for reliable recovery in terms of the length of the random subsequence. The sequence of each file is assumed to follow a hidden Markov model (HMM), which is a common model for many data structures such as voice or DNA sequences. The connection between this problem and coding over a deletion channel with greedy decoders is discussed.