Hamdi Joudeh

We study the information bottleneck (IB) source coding problem, also known as remote lossy source coding under logarithmic loss. Based on a rate-limited description of noisy observations, the receiver produces a soft estimate for the remote source, i.e., a probability distribution, evaluated under the logarithmic loss. We focus on the excess distortion probability of IB source coding and investigate how fast it converges to 0 or 1, depending on whether the rate is above or below the rate-distortion function. The latter case is also known as the exponential strong converse. We establish both the exact error exponent and the exact strong converse exponent for IB source coding by deriving matching upper and lower exponential bounds. The obtained exponents involve optimizations over auxiliary random variables. The matching converse bounds are derived through non-trivial extensions of existing sphere packing and single-letterization techniques, which we adapt to incorporate auxiliary random variables. In the second part of this paper, we establish a code-level connection between IB source coding and source coding with a helper, also known as the Wyner-Ahlswede-Körner (WAK) problem. We show that every code for the WAK problem is a code for IB source coding. This requires noticing that IB source coding, under the excess distortion criterion, is equivalent to source coding with a helper available at both the transmitter and the receiver; the latter in turn relates to the WAK problem. Through this connection, we re-derive the best known sphere packing exponent of the WAK problem, and provide it with an operational interpretation.

We study a joint communication and sensing setting comprising a transmitter, a receiver, and a sensor, all equipped with multiple antennas. The transmitter sends an encoded signal over the channel with the dual purpose of communicating an information message to the receiver, and enabling the sensor to estimate a target parameter vector by generating back-scattered signals.We assume that the transmitter and sensor are co-located, or fully connected, giving the latter access to the transmitted signal. The target parameter vector is randomly drawn from a continuous distribution, yet remains fixed throughout the transmission block. We establish the fundamental performance trade-off between the communication and sensing tasks, captured in terms of a capacity-MSE function. In doing so, we identify optimal coding schemes for this multi-antenna joint communication and sensing setting. Moreover, we particularize our result to two practically-inspired scenarios where we showcase optimal schemes and trade-offs.

We consider a multiple-access channel with singl-eantenna users and a multi-antenna access point under imperfect channel state information. We derive an achievable rate region under Gaussian codebooks and a mismatched decoder that treats the signal-dependent noise from channel imperfection as additional Gaussian noise. We further show that the rate tuples in this region are also achieved using a lower-complexity scheme based on linear combining and mismatched single-user decoding with successive interference cancellation. We then specialize the results to a block-fading setting with pilot-assisted channel estimation and derive analytical bounds on the sum-rate gap between matched and mismatched decoding. We show that this gap is typically small in practically relevant scenarios.

We introduce an expurgation method for source coding with side information that enables direct dual-domain derivations of expurgated error exponents. Dual-domain methods yield optimization problems over few parameters, with any sub-optimal choice resulting in an achievable exponent, as opposed to primal-domain optimization over distributions. In addition, dual-domain methods naturally allow for general alphabets and/or memory. We derive two such expurgated error exponents for different random-coding ensembles in the case where the decoder is possibly mismatched with respect to the source and side information joint distribution. We show the better of the exponents coincides with the Csiszár-Körner exponent obtained via a graph decomposition lemma. We show some numerical examples that illustrate the differences between the two exponents and show that in the case of source coding without side information, the expurgated exponent coincides with the error exponent of the source optimal code.

The information bottleneck channel, also known as oblivious relaying, is a two-hop channel where a transmitter sends messages to a remote receiver via an intermediate relay node. A codeword sent by the transmitter passes through a discrete memoryless channel to reach the relay, which then processes the noisy channel output and forwards it to the receiver through a noiseless rate-limited link. The relay is oblivious, in the sense that it has no knowledge of the channel codebook used in transmission. Previous works on oblivious relaying focus on characterizing achievable rates. In this work, we study error exponents and explore connections to lossless source coding with a helper, also known as the Wyner-Ahlswede-Körner (WAK) problem. We first establish an achievable error exponent for oblivious relaying under constant compositions codes. A key feature of our analysis is the use of the type covering lemma to design the relay’s compress-forward scheme. We then show that employing constant composition code ensembles does not improve the rates achieved with their IID counterparts. We also derive a sphere packing upper bound for the error exponent. In the second part of this paper, we establish a connection between the information bottleneck channel and the WAK problem. We show that good codes for the latter can be produced through permuting codes designed for the former. This is accomplished by revisiting Ahlswede’s covering lemma, and extending it to achieve simultaneous covering of a type class by several distinct sets using the same sequence of permutations. We then apply our approach to attain the best known achievable error exponent for the WAK problem, previously established by Kelly and Wagner. As a byproduct of our derivations, we also establish error exponents and achievable rates under mismatched decoding rules.

This paper considers the problem of soft guessing under a logarithmic loss distortion measure while allowing errors. We find an optimal guessing strategy, and derive single-shot upper and lower bounds for the minimal guessing moments as well as an asymptotic expansion for i.i.d. sources. These results are extended to the case where side information is available to the guesser. Furthermore, a connection between soft guessing allowing errors and variable-length lossy source coding under logarithmic loss is demonstrated. The Rényi entropy, the smooth Rényi entropy, and their conditional versions play an important role.

This paper considers guessing-based decoders with abandonment for discrete memoryless channels in which all codewords have the same composition. This class of decoders rank-orders all input sequences in the codebook’s composition class from “closest” to “farthest” from the channel output and then queries them sequentially in that order for codebook membership. Decoding terminates when a codeword is encountered or when a predetermined number of guesses is reached, and decoding is abandoned. We derive ensemble-tight first-order asymptotics for the code rate and abandonment rate, which shows that guessing-based decoding is more efficient than conventional testing-based decoding whenever the capacity of the channel exceeds half the entropy of the capacity-achieving input distribution. The main focus of this paper is on refined asymptotics, specifically, second-order asymptotics, error exponents, and strong converse exponents. The optimal second-order region is characterized in terms of the minimum of the second-order code and abandonment rates. The error (resp. strong converse) exponent is characterized in terms of the minimum (resp. maximum) of the usual channel coding exponent and an abandonment exponent, which turns out to be a special case of the exponent of conditional almost-lossless source coding.

In medical imaging, unsupervised out-of-distribution (OOD) detection offers an attractive approach for identifying pathological cases with extremely low incidence rates. In contrast to supervised methods, OOD-based approaches function without labels and are inherently robust to data imbalances. Current generative approaches often rely on likelihood estimation or reconstruction error, but these methods can be computationally expensive, unreliable, and require retraining if the inlier data changes. These limitations hinder their ability to distinguish nominal from anomalous inputs efficiently, consistently, and robustly. We propose a reconstruction-free OOD detection method that leverages the forward diffusion trajectories of a Stein score-based denoising diffusion model (SBDDM). By capturing trajectory curvature via the estimated Stein score, our approach enables accurate anomaly scoring with only five diffusion steps. A single SBDDM pre-trained on a large, semantically aligned medical dataset generalizes effectively across multiple Near-OOD and Far-OOD benchmarks, achieving state-of-the-art performance while drastically reducing computational cost during inference. Compared to existing methods, SBDDM achieves a relative improvement of up to 10.43% and 18.10% for Near-OOD and Far-OOD detection, making it a practical building block for real-time, reliable computer-aided diagnosis.

The capacity of a discrete-time channel with correlated phase noises is investigated. In particular, the electro-optic frequency comb system is considered, where the phase noise of each subchannel is a combination of two independent Wiener phase-noise sources. Capacity upper and lower bounds are derived for this channel and are compared with lower bounds obtained by numerically evaluating the achievable information rates using quadrature amplitude modulation constellations. Capacity upper and lower bounds are provided for the high signal-to-noise ratio (SNR) regime. The multiplexing gain (pre-log) is shown to be $M-1$ , where M represents the number of subchannels. A constant gap between the asymptotic upper and lower bounds is observed, which depends on the number of subchannels M. For the specific case of $M=2$ , capacity is characterized up to a term that vanishes as the SNR grows large.