Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Charalambos Demetriou Charalambous has a total of 49 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    The Capacity Gains of Gaussian Channels with Unstable Versus Stable Autoregressive Noise

    by Charalambos D. Charalambous, Christos Kourtellaris, Stelios Louka, Sergey Loyka

    In this paper, we consider Cover’s and Pombra’s formulation of feedback capacity of additive Gaussian noise (AGN) channels, with jointly Gaussian nonstationary and nonergodic noise. We derive closed-form feedback capacity formulas, using Karush–Kuhn–Tucker (KKT) conditions and convergence properties of difference Riccati equations to limiting algebraic Riccati equations of filtering theory, for unstable and stable autoregressive (AR) noise. Surprisingly, the capacity formulas depend on the parameters of the AR noise, its pole c∈(−∞,∞) and noise variance KW∈(0,∞), and the total transmit power κ∈[0,∞), indicating substantial gains for the unstable noise region ∀c2∈(1,∞),∀κ>κmin=▵KW1+4c2−32c2−12 compared to its complement region. In particular, feedback capacity is distinguished by three regimes, as follows. Regime 1, ∀c2∈(1,∞),∀κ>κmin: the optimal channel input includes an innovations part, the capacity increases as |c|>1 increases, while κmin and the allocated transmit power decrease. Regime 2, ∀c2∈(1,∞),∀κ≤κmin, Regime 3, ∀c∈[−1,1],∀κ∈[0,∞) (complement of Regime 1): the innovations part of the optimal channel is asymptotically zero and the capacity is fundamentally different compared to Regime 1. The differences of capacity formulas for Regimes 1, 2 and 3 are directly related to their operational meaning: (i) Regime 1 is an ergodic capacity while Regimes 2 and 3 are nonergodic capacities; (ii) Regime 1 is achieved by an asymptotically stationary channel input with a non-zero innovations part, while Regimes 2 and 3 are achieved by an asymptotically zero innovations part. The gains of capacity for Regime 1 are attributed to the high correlation of noise samples compared to stable noise and the use of an informative innovations part by the optimal channel input, which make possible the prediction of future noise samples from past samples, unlike memoryless noise. Our results provide answers to certain open questions regarding the validity of capacity formulas of stable noise that appeared in the literature.

  • Open Access

    General Decentralized Stochastic Optimal Control via Change of Measure: Applications to the Witsenhausen Counterexample

    by Bhagyashri Telsang, Seddik Djouadi, Charalambos D. Charalambous

    In this paper we present global and person-by-person (PbP) optimality conditions for general decentralized stochastic dynamic optimal control problems, using a discrete-time version of Girsanov's change of measure. The PbP optimality conditions are applied to the Witsenhausen counterexample to show that the two strategies satisfy two coupled nonlinear integral equations. Further, we prove a fixed point theorem in a function space, establishing existence and uniqueness of solutions to the integral equations. We also provide numerical solutions of the two integral equations using the Gauss Hermite Quadrature scheme, and include a detail comparison to other numerical methods of the literature. The numerical solutions confirm Witsehausen's observation that, for certain choices of parameters, linear or affine strategies are optimal, while for other choices of parameters nonlinear strategies outperformed affine strategies.

  • Open Access

    Rollout-Based Approximate Dynamic Programming for MDPs with Information-Theoretic Constraints

    by Zixuan He, Charalambos D. Charalambous, Photios A. Stavrou

    This paper studies a finite-horizon Markov decision problem with information-theoretic constraints, where the goal is to minimize directed information from the controlled source process to the control process, subject to stage-wise cost constraints, aiming for an optimal control policy. We propose a new way of approximating a solution for this problem, which is known to be formulated as an unconstrained MDP with a continuous information-state using Q-factors. To avoid the computational complexity of discretizing the continuous information-state space, we propose a truncated rollout-based backward-forward approximate dynamic programming (ADP) framework. Our approach consists of two phases: an offline base policy approximation over a shorter time horizon, followed by an online rollout lookahead minimization, both supported by provable convergence guarantees. We supplement our theoretical results with a numerical example where we demonstrate the cost improvement of the rollout method compared to a previously proposed policy approximation method, and the computational complexity observed in executing the offline and online phases for the two methods.

  • Open Access

    New Formulas of Feedback Capacity for AGN Channels with Memory: A Time-Domain Sufficient Statistic Approach

    by Charalambos D. Charalambous, Christos Kourtellaris, Stelios Louka

    Recently, several papers identified technical issues related to equivalent time-domain and frequency-domain “characterization of the n–block or transmission” feedback capacity formula and its asymptotic limit, the feedback capacity, of additive Gaussian noise (AGN) channels, first introduce by Cover and Pombra in 1989 (IEEE Transactions on Information Theory). The main objective of this paper is to derive new results on the Cover and Pombra characterization of the n–block feedback capacity formula, and to clarify the main points of confusion regarding the time-domain results that appeared in the literature. The first part of this paper derives new equivalent time-domain sequential characterizations of feedback capacity of AGN channels driven by non-stationary and non-ergodic Gaussian noise. It is shown that the optimal channel input processes of the new equivalent sequential characterizations are expressed as functionals of a sufficient statistic and a Gaussian orthogonal innovations process. Further, the Cover and Pombra n–block capacity formula is expressed as a functional of two generalized matrix difference Riccati equations (DREs) of the filtering theory of Gaussian systems, contrary to results that appeared in the literature and involve only one DRE. It is clarified that prior literature deals with a simpler problem that presupposes the state of the noise is known to the encoder and the decoder. In the second part of this paper, the existence of the asymptotic limit of the n–block feedback capacity formula is shown to be equivalent to the convergence properties of solutions of the two generalized DREs. Further, necessary and or sufficient conditions are identified for the existence of asymptotic limits, for stable and unstable Gaussian noise, when the optimal input distributions are asymptotically time-invariant but not necessarily stationary. This paper contains an in-depth analysis, with various examples, and identifies the technical conditions on the feedback code and state space noise realization, so that the time-domain capacity formulas that appeared in the literature, for AGN channels with stationary noises, are indeed correct.


  • Open Access

    A New Finite-Horizon Dynamic Programming Analysis of Nonanticipative Rate-Distortion Function for Markov Sources

    by Zixuan He, Charalambos D. Charalambous, Photios A. Stavrou

    This paper deals with the computation of a non-asymptotic lower bound by means of the nonanticipative rate-distortion function (NRDF) on the discrete-time zero-delay variable-rate lossy compression problem for discrete Markov sources with per-stage, single-letter distortion. First, we derive a new information structure of the NRDF for Markov sources and single-letter distortions. Second, we derive new convexity results on the NRDF, which facilitate the use of Lagrange duality theorem to cast the problem as an unconstrained partially observable finite-time horizon stochastic dynamic programming (DP) algorithm subject to a probabilistic state (belief state) that summarizes the past information about the reproduction symbols and takes values in a continuous state space. Instead of approximating the DP algorithm directly, we use Karush-Kuhn-Tucker (KKT) conditions to find an implicit closed-form expression of the optimal control policy of the stochastic DP (i.e., the minimizing distribution of the NRDF) and approximate the control policy and the cost-to-go function (a function of the rate) stage-wise, via a novel dynamic alternating minimization (AM) approach, that is realized by an offline algorithm operating using backward recursions, with provable convergence guarantees. We obtain the clean values of the aforementioned quantities using an online (forward) algorithm operating for any finite-time horizon. Our methodology provides an approximate solution to the exact NRDF solution, which becomes near-optimal as the search space of the belief state becomes sufficiently large at each time stage. We corroborate our theoretical findings with simulation studies where we apply our algorithms assuming time-varying and time-invariant binary Markov processes.

  • Open Access

    Signalling and Control in Nonlinear Stochastic Systems: An Information State Approach with Applications

    by Charalambos D. Charalambous, Stelios Louka

    We consider optimal signalling and control of discrete-time nonlinear partially observable stochastic systems in state space form. In the first part of the paper, we characterize the operational {\it control-coding capacity}, $C_{FB}$ in bits/second, by an information theoretic optimization problem of encoding signals or messages into randomized controller-encoder strategies, and reproducing the messages at the output of the system using a decoder or estimator with arbitrary small asymptotic error probability. Our analysis of $C_{FB}$ is based on realizations of randomized strategies (controller-encoders), in terms of information states of nonlinear filtering theory, and either uniform or arbitrary distributed random variables (RVs). In the second part of the paper, we analyze the linear-quadratic Gaussian partially observable stochastic system (LQG-POSS). We show that simultaneous signalling and control leads to randomized strategies described by finite-dimensional sufficient statistics, that involve two Kalman-filters, and consist of control, estimation and signalling strategies. We apply decentralized optimization techniques to prove a separation principle, and to derive the optimal control part of randomized strategies explicitly in terms of a control matrix difference Riccati equation (DRE).

  • Open Access

    Structural Properties of the Wyner–Ziv Rate Distortion Function: Applications for Multivariate Gaussian Sources

    by Michail Gkagkos, Charalambos D. Charalambous

    The main focus of this paper is the derivation of the structural properties of the test channels of Wyner’s operational information rate distortion function (RDF), R¯(ΔX), for arbitrary abstract sources and, subsequently, the derivation of additional properties for a tuple of multivariate correlated, jointly independent, and identically distributed Gaussian random variables, {Xt,Yt}t=1∞, Xt:Ω→Rnx, Yt:Ω→Rny, with average mean-square error at the decoder and the side information, {Yt}t=1∞, available only at the decoder. For the tuple of multivariate correlated Gaussian sources, we construct optimal test channel realizations which achieve the informational RDF, R¯(ΔX)=▵infM(ΔX)I(X;Z|Y), where M(ΔX) is the set of auxiliary RVs Z such that PZ|X,Y=PZ|X, X^=f(Y,Z), and E{||X−X^||2}≤ΔX. We show the following fundamental structural properties: (1) Optimal test channel realizations that achieve the RDF and satisfy conditional independence, PX|X^,Y,Z=PX|X^,Y=PX|X^,EX|X^,Y,Z=EX|X^=X^. (2) Similarly, for the conditional RDF, RX|Y(ΔX), when the side information is available to both the encoder and the decoder, we show the equality R¯(ΔX)=RX|Y(ΔX). (3) We derive the water-filling solution for RX|Y(ΔX).


  • Open Access

    A Realization Approach to Lossy Network Compression of a Tuple of Correlated Multivariate Gaussian RVs

    by Charalambos D. Charalambous, Jan H. van Schuppen

    Examined in this paper is the Gray and Wyner source coding for a simple network of correlated multivariate Gaussian random variables, Y1:Ω→Rp1 and Y2:Ω→Rp2. The network consists of an encoder that produces two private rates R1 and R2, and a common rate R0, and two decoders, where decoder 1 receives rates (R1,R0) and reproduces Y1 by Y^1, and decoder 2 receives rates (R2,R0) and reproduces Y2 by Y^2, with mean-square error distortions E||Yi−Y^i||Rpi2≤Δi∈[0,∞],i=1,2. Use is made of the weak stochastic realization and the geometric approach of such random variables to derive test channel distributions, which characterize the rates that lie on the Gray and Wyner rate region. Specific new results include: (1) A proof that, among all continuous or finite-valued random variables, W:Ω→W, Wyner’s common information, C(Y1,Y2)=infPY1,Y2,W:PY1,Y2|W=PY1|WPY2|WI(Y1,Y2;W), is achieved by a Gaussian random variable, W:Ω→Rn of minimum dimension n, which makes the two components of the tuple (Y1,Y2) conditionally independent according to the weak stochastic realization of (Y1,Y2), and a the formula C(Y1,Y2)=12∑j=1nln1+dj1−dj, where di∈(0,1),i=1,…,n are the canonical correlation coefficients of the correlated parts of Y1 and Y2, and a realization of (Y1,Y2,W) which achieves this. (2) The parameterization of rates that lie on the Gray and Wyner rate region, and several of its subsets. The discussion is largely self-contained and proceeds from first principles, while connections to prior literature is discussed.


  • On the Capacity of Gaussian MIMO Channels With Memory

    by Sergey Loyka, Charalambos D. Charalambous

    The operational capacity of Gaussian MIMO channels with memory was obtained by Brandenburg and Wyner in [9] under certain mild assumptions on the channel impulse response and its noise covariance matrix, which essentuially require channel memory to be not too strong. This channel was also considered by Tsybakov in [10] and its information capacity was obtained in some cases. It was further conjectured, based on numerical evidence, that these capacities are the same in all cases. This conjecture is proved here. An explicit closed-form expression for the optimal input power spectral density matrix is also given. The obtained result is further extended to the case of joint constraints, including per-antenna and interference power constraints as well as energy harvesting constraints. These results imply the information-theoretic optimality of OFDM-type transmission systems for such channels with memory.


  • Open Access

    Complete Characterization of Gorbunov and Pinsker Nonanticipatory Epsilon Entropy of Multivariate Gaussian Sources: Structural Properties

    by Charalambos D. Charalambous, Themistoklis Charalambous, Christos Kourtellaris, Jan H. van Schuppen

    This paper derives the optimal test channel distribution and the complete characterization of the classical Gorbunov and Pinsker (1973), Gorbunov and Pinsker (1974) nonanticipatory epsilon entropy of multivariate Gaussian Markov sources with square-error fidelity, which remained an open problem since 1974. The paper also formulates a state dependent nonanticipatory epsilon entropy, in which past reproductions are available to the decoder and not to the encoder, the test channel is specified with respect to an auxiliary (state) random process, and the reproduction process is a causal function of past reproduction and the auxiliary random process. This variation is analogous to the Wyner and Ziv (1976) and Wyner (1978) rate distortion function (RDF), of memoryless sources. It is shown that the operational rate of zero-delay codes, with past reproductions available to the decoder but not to the encoder is bounded below by the state dependent nonanticipatory epsilon entropy rate. For the case of multivariate Gaussian Markov sources with square-error fidelity, the optimal test channel distribution and the complete characterization of the state dependent of nonanticipatory epsilon entropy are derived, and also shown that that the two nonanticipatory epsilon entropies coincide. The derivations are new; they are based on structural properties of the stochastic realizations of the reproduction process that induce the optimal test channel distributions. They are derived using, achievable lower bounds on information theoretic measures, properties of mean-square estimation theory, Hadamard’s inequality, and canonical correlation coefficients of a tuple of multivariate jointly Gaussian random processes. Applications of the nonanticipatory epsilon entropy and its state dependent variation are discussed to the areas of control of unstable Gaussian systems over limited memory channels, design of causal estimators for Gaussian Markov sources with a fidelity criterion, computation of the rate loss of causal and zero-delay codes of Gaussian Markov sources with respect to non-causal codes.