Hamid Ghourchian
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Hamid Ghourchian has a total of 12 co-authors. Here are the top co-authors by number of shared publications:
- Mikael Skoglund
Department of Information Science and Engineering, KTH Royal Institute of Technology, Stockholm, Sweden
3 shared publications - Amin Gohari
The Chinese University of Hong Kong
2 shared publications - Arash Amini
Electrical Engineering Department, Sharif University of Technology, Tehran, Iran
2 shared publications Amin Gohari
Department of Electrical Engineering, Sharif University of Technology, Tehran, Iran
1 shared publication- Germán Bassi
Division of Information Science and Engineering (ISE KTH Royal Institute of Technology
1 shared publication - Gholamali Aminian
Department of Electrical Engineering and Advanced Communication Research Institute (ACRI), Sharif University of Technology, Tehran 11155, Iran
1 shared publication
Latest Publications
- Open Access
Secure Block Joint Source-Channel Coding with Sequential Encoding
We extend the results of Ghourchian et al. [IEEE JSAIT-2021], to joint source-channel coding with eavesdropping. Our work characterizes the sequential encoding process using the cumulative rate distribution functions (CRDF) and includes a security constraint using the cumulative leakage distribution functions (CLF). The information leakage is defined based on the mutual information between the source and the output of the wiretap channel to the eavesdropper. We derive inner and outer bounds on the achievable CRDF for a given source and CLF, and show that the bounds are tight when the distribution achieving the capacity of the wiretap channel is the same as the one achieving the capacity of the channel.
Open AccessSecure Block Joint Source-Channel Coding with Sequential Encoding
We extend the results of Ghourchian et al. [IEEE JSAIT-2021], to joint source-channel coding with eavesdropping. Our work characterizes the sequential encoding process using the cumulative rate distribution functions (CRDF) and includes a security constraint using the cumulative leakage distribution functions (CLF). The information leakage is defined based on the mutual information between the source and the output of the wiretap channel to the eavesdropper. We derive inner and outer bounds on the achievable CRDF for a given source and CLF, and show that the bounds are tight when the distribution achieving the capacity of the wiretap channel is the same as the one achieving the capacity of the channel.
- Open Access
Neural Estimator of Information for Time-Series Data with Dependency
Novel approaches to estimate information measures using neural networks are well-celebrated in recent years both in the information theory and machine learning communities. These neural-based estimators are shown to converge to the true values when estimating mutual information and conditional mutual information using independent samples. However, if the samples in the dataset are not independent, the consistency of these estimators requires further investigation. This is of particular interest for a more complex measure such as the directed information, which is pivotal in characterizing causality and is meaningful over time-dependent variables. The extension of the convergence proof for such cases is not trivial and demands further assumptions on the data. In this paper, we show that our neural estimator for conditional mutual information is consistent when the dataset is generated with samples of a stationary and ergodic source. In other words, we show that our information estimator using neural networks converges asymptotically to the true value with probability one. Besides universal functional approximation of neural networks, a core lemma to show the convergence is Birkhoff’s ergodic theorem. Additionally, we use the technique to estimate directed information and demonstrate the effectiveness of our approach in simulations.
Open AccessNeural Estimator of Information for Time-Series Data with Dependency
Novel approaches to estimate information measures using neural networks are well-celebrated in recent years both in the information theory and machine learning communities. These neural-based estimators are shown to converge to the true values when estimating mutual information and conditional mutual information using independent samples. However, if the samples in the dataset are not independent, the consistency of these estimators requires further investigation. This is of particular interest for a more complex measure such as the directed information, which is pivotal in characterizing causality and is meaningful over time-dependent variables. The extension of the convergence proof for such cases is not trivial and demands further assumptions on the data. In this paper, we show that our neural estimator for conditional mutual information is consistent when the dataset is generated with samples of a stationary and ergodic source. In other words, we show that our information estimator using neural networks converges asymptotically to the true value with probability one. Besides universal functional approximation of neural networks, a core lemma to show the convergence is Birkhoff’s ergodic theorem. Additionally, we use the technique to estimate directed information and demonstrate the effectiveness of our approach in simulations.
- 22 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Information Theory
- Vol. 2 (1) , 32-48
Secure Block Source Coding With Sequential Encoding
We introduce fundamental bounds on achievable cumulative rate distribution functions (CRDF) to characterize a sequential encoding process that ensures lossless or lossy reconstruction subject to an average distortion criterion using a non-causal decoder. The CRDF describes the rate resources spent sequentially to compress the sequence. We also include a security constraint that affects the set of achievable CRDF. The information leakage is defined sequentially based on the mutual information between the source and its compressed representation, as it evolves. To characterize the security constraints, we introduce the concept of cumulative leakage distribution functions (CLF), which determines the allowed information leakage as distributed over encoded sub-blocks. Utilizing tools from majorization theory, we derive necessary and sufficient conditions on the achievable CRDF for a given independent and identically distributed (IID) source and CLF. One primary result of this paper is that the concave-hull of the CRDF characterizes the optimal achievable rate distribution.
- 22 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal on Selected Areas in Information Theory
- Vol. 2 (1) , 32-48
Secure Block Source Coding With Sequential Encoding
We introduce fundamental bounds on achievable cumulative rate distribution functions (CRDF) to characterize a sequential encoding process that ensures lossless or lossy reconstruction subject to an average distortion criterion using a non-causal decoder. The CRDF describes the rate resources spent sequentially to compress the sequence. We also include a security constraint that affects the set of achievable CRDF. The information leakage is defined sequentially based on the mutual information between the source and its compressed representation, as it evolves. To characterize the security constraints, we introduce the concept of cumulative leakage distribution functions (CLF), which determines the allowed information leakage as distributed over encoded sub-blocks. Utilizing tools from majorization theory, we derive necessary and sufficient conditions on the achievable CRDF for a given independent and identically distributed (IID) source and CLF. One primary result of this paper is that the concave-hull of the CRDF characterizes the optimal achievable rate distribution.
- 29 August 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 64 (12) , 7828-7846
On the Capacity of a Class of Signal-Dependent Noise Channels
In some applications, the variance of additive measurement noise depends on the signal that we aim to measure. For instance, additive signal-dependent Gaussian noise (ASDGN) channel models are used in molecular and optical communication. Herein we provide lower and upper bounds on the capacity of additive signal-dependent noise (ASDN) channels. The first lower bound is based on an extension of majorization inequalities, and the second lower bound utilizes properties of the differential entropy. The lower bounds are valid for arbitrary additive signaldependent noise (ASDN) channels. The upper bound is based on a previous idea of the authors (“symmetric relative entropy”) and is applied to the additive signal-dependent Gaussian noise (ASDGN) channels. These bounds indicate that in the ASDN channels (unlike the classical AWGN channels), the capacity does not necessarily become larger by reducing the noise variance function. We also provide sufficient conditions under which the capacity becomes infinite. This is complemented by some conditions implying that the capacity is finite, and a unique capacity achieving measure exists (in the sense of the output measure).
- 29 August 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 64 (12) , 7828-7846
On the Capacity of a Class of Signal-Dependent Noise Channels
In some applications, the variance of additive measurement noise depends on the signal that we aim to measure. For instance, additive signal-dependent Gaussian noise (ASDGN) channel models are used in molecular and optical communication. Herein we provide lower and upper bounds on the capacity of additive signal-dependent noise (ASDN) channels. The first lower bound is based on an extension of majorization inequalities, and the second lower bound utilizes properties of the differential entropy. The lower bounds are valid for arbitrary additive signaldependent noise (ASDN) channels. The upper bound is based on a previous idea of the authors (“symmetric relative entropy”) and is applied to the additive signal-dependent Gaussian noise (ASDGN) channels. These bounds indicate that in the ASDN channels (unlike the classical AWGN channels), the capacity does not necessarily become larger by reducing the noise variance function. We also provide sufficient conditions under which the capacity becomes infinite. This is complemented by some conditions implying that the capacity is finite, and a unique capacity achieving measure exists (in the sense of the output measure).
- 4 April 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 64 (7) , 4843-4871
How Compressible Are Innovation Processes?
The sparsity and compressibility of finite-dimensional signals are of great interest in fields, such as compressed sensing. The notion of compressibility is also extended to infinite sequences of independent identically distributed or ergodic random variables based on the observed error in their nonlinear k-term approximation. In this paper, we use the entropy measure to study the compressibility of continuous-domain innovation processes (alternatively known as white noise). Specifically, we define such a measure as the entropy limit of the doubly quantized (time and amplitude) process. This provides a tool to compare the compressibility of various innovation processes. It also allows us to identify an analogue of the concept of “entropy dimension" which was originally defined by Rényi for random variables. Particular attention is given to stable and impulsive Poisson innovation processes. Here, our results recognize Poisson innovations as the more compressible ones with an entropy measure far below that of stable innovations. While this result departs from the previous knowledge regarding the compressibility of impulsive Poisson laws compared with continuous fat-tailed distributions, our entropy measure ranks α-stable innovations according to their tail.
- 4 April 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 64 (7) , 4843-4871
How Compressible Are Innovation Processes?
The sparsity and compressibility of finite-dimensional signals are of great interest in fields, such as compressed sensing. The notion of compressibility is also extended to infinite sequences of independent identically distributed or ergodic random variables based on the observed error in their nonlinear k-term approximation. In this paper, we use the entropy measure to study the compressibility of continuous-domain innovation processes (alternatively known as white noise). Specifically, we define such a measure as the entropy limit of the doubly quantized (time and amplitude) process. This provides a tool to compare the compressibility of various innovation processes. It also allows us to identify an analogue of the concept of “entropy dimension" which was originally defined by Rényi for random variables. Particular attention is given to stable and impulsive Poisson innovation processes. Here, our results recognize Poisson innovations as the more compressible ones with an entropy measure far below that of stable innovations. While this result departs from the previous knowledge regarding the compressibility of impulsive Poisson laws compared with continuous fat-tailed distributions, our entropy measure ranks α-stable innovations according to their tail.
- 30 March 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Communications Letters
- Vol. 21 (7) , 1469-1472
Existence and Continuity of Differential Entropy for a Class of Distributions
In this letter, we identify a class of absolutely continuous probability distributions, and show that the differential entropy is uniformly convergent over this space under the metric of total variation distance. One of the advantages of this class is that the requirements could be readily verified for a given distribution.
- 30 March 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Communications Letters
- Vol. 21 (7) , 1469-1472
Existence and Continuity of Differential Entropy for a Class of Distributions
In this letter, we identify a class of absolutely continuous probability distributions, and show that the differential entropy is uniformly convergent over this space under the metric of total variation distance. One of the advantages of this class is that the requirements could be readily verified for a given distribution.