Submit to Entropy Review for Entropy Propose a Special Issue

Journal Browser

► Journal Browser

Information Theory for Data Communications and Processing

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Entropy (ISSN 1099-4300).

Deadline for manuscript submissions: closed (30 September 2019) | Viewed by 45349

Printed Edition Available!
A printed edition of this Special Issue is available here.

Special issue Information Theory for Data Communications and Processing book cover image

Share This Special Issue

Special Issue Editors

Prof. Dr. Shlomo Shamai (Shitz)

E-Mail Website
Guest Editor

Faculty of Electrical Engineering, Technion—Israel Institute of Technology, Haifa 3200003, Israel
Interests: multi-user information theory; modern communication networks (cloud and fog radio networks); information and signal processing (information–estimation); information bottleneck problems in communications and learning; sparse communications models and non-orthogonal (NOMA) systems
Special Issues, Collections and Topics in MDPI journals

Prof. Dr. Abdellatif Zaidi

E-Mail Website
Guest Editor

Institut Gaspard Monge, Université Paris-Est, 05 Boulevard Descartes, Cité Descartes, 77454 Champs sur Marne, France
Interests: Network information theory, statistical decision theory, data compression, security and privacy

Special Issue Information

Dear Colleagues,

This Special Issue focuses on fundamental information-theoretic aspects of remote processing in networks. We welcome unpublished contributions related to advanced distributed data processing techniques distributively in networks. Examples include signal processing solutions based on communication and information-theoretic considerations, for Cloud and Fog Radio Access Networks (RAN), remote source coding, as well as interdisciplinary connections with problems such as information bottleneck, information-theoretic learning and prediction, distributed estimation and decision making, secrecy/privacy and identification in communication systems.

Prof. Dr. Shlomo Shamai (Shitz)
Prof. Dr. Abdellatif Zaidi
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

Information theory, communications, signal processing. In particular, topics of interest include, but are not restricted to, the following:

Fog and Cloud RANs
Remote source coding and indirect rate distortion theory
Chief Executive Officer (CEO) source coding problems
Noisy network coding
Distributed estimation
Information Bottleneck
Information theoretic aspects of prediction and Deep learning
Universal compression
Hypothesis testing and statistics
Caching
Network information theoretic frameworks, including: multiple access, broadcast, relay, wiretap and X channels

Published Papers (12 papers)

Download All Papers

Editorial

Jump to: Research, Other

4 pages, 172 KiB

Open AccessEditorial

Information Theory for Data Communications and Processing

by Shlomo Shamai (Shitz) and Abdellatif Zaidi

Entropy 2020, 22(11), 1250; https://0-doi-org.brum.beds.ac.uk/10.3390/e22111250 - 03 Nov 2020

Viewed by 2259

Abstract

This book, composed of the collection of papers that have appeared in the Special Issue of the
Entropy journal dedicated to “Information Theory for Data Communications and Processing”,
reflects, in its eleven chapters, novel contributions based on the firm basic grounds of information
theory. The book chapters [1–11] address timely theoretical and practical aspects that carry both
interesting and relevant theoretical contributions, as well as direct implications for modern current
and future communications systems. [...] Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

Research

Jump to: Editorial, Other

16 pages, 2057 KiB

Open AccessArticle

Variational Information Bottleneck for Unsupervised Clustering: Deep Gaussian Mixture Embedding

by Yiğit Uğur, George Arvanitakis and Abdellatif Zaidi

Entropy 2020, 22(2), 213; https://0-doi-org.brum.beds.ac.uk/10.3390/e22020213 - 13 Feb 2020

Cited by 12 | Viewed by 3680

Abstract

In this paper, we develop an unsupervised generative clustering framework that combines the variational information bottleneck and the Gaussian mixture model. Specifically, in our approach, we use the variational information bottleneck method and model the latent space as a mixture of Gaussians. We derive a bound on the cost function of our model that generalizes the Evidence Lower Bound (ELBO) and provide a variational inference type algorithm that allows computing it. In the algorithm, the coders’ mappings are parametrized using neural networks, and the bound is approximated by Markov sampling and optimized with stochastic gradient descent. Numerical results on real datasets are provided to support the efficiency of our method. Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

► Show Figures

Figure 1

17 pages, 437 KiB

Open AccessArticle

Cross-Entropy Method for Content Placement and User Association in Cache-Enabled Coordinated Ultra-Dense Networks

by Jia Yu, Ye Wang, Shushi Gu, Qinyu Zhang, Siyun Chen and Yalin Zhang

Entropy 2019, 21(6), 576; https://0-doi-org.brum.beds.ac.uk/10.3390/e21060576 - 08 Jun 2019

Cited by 6 | Viewed by 2706

Abstract

Due to the high splitting-gain of dense small cells, Ultra-Dense Network (UDN) is regarded as a promising networking technology to achieve high data rate and low latency in 5G mobile communications. In UDNs, each User Equipment (UE) may receive signals from multiple Base Stations (BSs), which impose severe interference in the networks and in turn motivates the possibility of using Coordinated Multi-Point (CoMP) transmissions to further enhance network capacity. In CoMP-based Ultra-Dense Networks, a great challenge is to tradeoff between the gain of network throughput and the worsening backhaul latency. Caching popular files on BSs has been identified as a promising method to reduce the backhaul traffic load. In this paper, we investigated content placement strategies and user association algorithms for the proactive caching ultra dense networks. The problem has been formulated to maximize network throughput of cell edge UEs under the consideration of backhaul load, which is a constrained non-convex combinatorial optimization problem. To decrease the complexity, the problem is decomposed into two suboptimal problems. We first solved the content placement algorithm based on the cross-entropy (CE) method to minimize the backhaul load of the network. Then, a user association algorithm based on the CE method was employed to pursue larger network throughput of cell edge UEs. Simulation were conducted to validate the performance of the proposed cross-entropy based schemes in terms of network throughput and backhaul load. The simulation results show that the proposed cross-entropy based content placement scheme significantly outperform the conventional random and Most Popular Content placement schemes, with with 50% and 20% backhaul load decrease respectively. Furthermore, the proposed cross-entropy based user association scheme can achieve 30% and 23% throughput gain, compared with the conventional N-best, No-CoMP, and Threshold based user association schemes. Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

► Show Figures

Figure 1

15 pages, 574 KiB

Open AccessArticle

Robust Baseband Compression Against Congestion in Packet-Based Fronthaul Networks Using Multiple Description Coding

by Seok-Hwan Park, Osvaldo Simeone and Shlomo Shamai (Shitz)

Entropy 2019, 21(4), 433; https://0-doi-org.brum.beds.ac.uk/10.3390/e21040433 - 24 Apr 2019

Cited by 6 | Viewed by 3093

Abstract

In modern implementations of Cloud Radio Access Network (C-RAN), the fronthaul transport network will often be packet-based and it will have a multi-hop architecture built with general-purpose switches using network function virtualization (NFV) and software-defined networking (SDN). This paper studies the joint design of uplink radio and fronthaul transmission strategies for a C-RAN with a packet-based fronthaul network. To make an efficient use of multiple routes that carry fronthaul packets from remote radio heads (RRHs) to cloud, as an alternative to more conventional packet-based multi-route reception or coding, a multiple description coding (MDC) strategy is introduced that operates directly at the level of baseband signals. MDC ensures an improved quality of the signal received at the cloud in conditions of low network congestion, i.e., when more fronthaul packets are received within a tolerated deadline. The advantages of the proposed MDC approach as compared to the traditional path diversity scheme are validated via extensive numerical results. Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

► Show Figures

Figure 1

32 pages, 1139 KiB

Open AccessArticle

Efficient Algorithms for Coded Multicasting in Heterogeneous Caching Networks

by Giuseppe Vettigli, Mingyue Ji, Karthikeyan Shanmugam, Jaime Llorca, Antonia M. Tulino and Giuseppe Caire

Entropy 2019, 21(3), 324; https://0-doi-org.brum.beds.ac.uk/10.3390/e21030324 - 25 Mar 2019

Cited by 6 | Viewed by 3699

Abstract

Coded multicasting has been shown to be a promising approach to significantly improve the performance of content delivery networks with multiple caches downstream of a common multicast link. However, the schemes that have been shown to achieve order-optimal performance require content items to be partitioned into several packets that grows exponentially with the number of caches, leading to codes of exponential complexity that jeopardize their promising performance benefits. In this paper, we address this crucial performance-complexity tradeoff in a heterogeneous caching network setting, where edge caches with possibly different storage capacity collect multiple content requests that may follow distinct demand distributions. We extend the asymptotic (in the number of packets per file) analysis of shared link caching networks to heterogeneous network settings, and present novel coded multicast schemes, based on local graph coloring, that exhibit polynomial-time complexity in all the system parameters, while preserving the asymptotically proven multiplicative caching gain even for finite file packetization. We further demonstrate that the packetization order (the number of packets each file is split into) can be traded-off with the number of requests collected by each cache, while preserving the same multiplicative caching gain. Simulation results confirm the superiority of the proposed schemes and illustrate the interesting request aggregation vs. packetization order tradeoff within several practical settings. Our results provide a compelling step towards the practical achievability of the promising multiplicative caching gain in next generation access networks. Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

► Show Figures

Figure 1

14 pages, 327 KiB

Open AccessArticle

Asymptotic Rate-Distortion Analysis of Symmetric Remote Gaussian Source Coding: Centralized Encoding vs. Distributed Encoding

by Yizhong Wang, Li Xie, Siyao Zhou, Mengzhen Wang and Jun Chen

Entropy 2019, 21(2), 213; https://0-doi-org.brum.beds.ac.uk/10.3390/e21020213 - 23 Feb 2019

Cited by 4 | Viewed by 2893

Abstract

Consider a symmetric multivariate Gaussian source with ℓ components, which are corrupted by independent and identically distributed Gaussian noises; these noisy components are compressed at a certain rate, and the compressed version is leveraged to reconstruct the source subject to a mean squared error distortion constraint. The rate-distortion analysis is performed for two scenarios: centralized encoding (where the noisy source components are jointly compressed) and distributed encoding (where the noisy source components are separately compressed). It is shown, among other things, that the gap between the rate-distortion functions associated with these two scenarios admits a simple characterization in the large ℓ limit. Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

► Show Figures

Figure 1

33 pages, 486 KiB

Open AccessArticle

Amplitude Constrained MIMO Channels: Properties of Optimal Input Distributions and Bounds on the Capacity

by Alex Dytso, Mario Goldenbaum, H. Vincent Poor and Shlomo Shamai (Shitz)

Entropy 2019, 21(2), 200; https://0-doi-org.brum.beds.ac.uk/10.3390/e21020200 - 19 Feb 2019

Cited by 8 | Viewed by 3720

Abstract

In this work, the capacity of multiple-input multiple-output channels that are subject to constraints on the support of the input is studied. The paper consists of two parts. The first part focuses on the general structure of capacity-achieving input distributions. Known results are surveyed and several new results are provided. With regard to the latter, it is shown that the support of a capacity-achieving input distribution is a small set in both a topological and a measure theoretical sense. Moreover, explicit conditions on the channel input space and the channel matrix are found such that the support of a capacity-achieving input distribution is concentrated on the boundary of the input space only. The second part of this paper surveys known bounds on the capacity and provides several novel upper and lower bounds for channels with arbitrary constraints on the support of the channel input symbols. As an immediate practical application, the special case of multiple-input multiple-output channels with amplitude constraints is considered. The bounds are shown to be within a constant gap to the capacity if the channel matrix is invertible and are tight in the high amplitude regime for arbitrary channel matrices. Moreover, in the regime of high amplitudes, it is shown that the capacity scales linearly with the minimum between the number of transmit and receive antennas, similar to the case of average power-constrained inputs. Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

► Show Figures

Figure 1

18 pages, 1119 KiB

Open AccessArticle

Quasi-Concavity for Gaussian Multicast Relay Channels

by Mohit Thakur and Gerhard Kramer

Entropy 2019, 21(2), 109; https://0-doi-org.brum.beds.ac.uk/10.3390/e21020109 - 24 Jan 2019

Cited by 1 | Viewed by 3116

Abstract

Standard upper and lower bounds on the capacity of relay channels are cut-set (CS), decode-forward (DF), and quantize-forward (QF) rates. For real additive white Gaussian noise (AWGN) multicast relay channels with one source node and one relay node, these bounds are shown to be quasi-concave in the receiver signal-to-noise ratios and the squared source-relay correlation coefficient. Furthermore, the CS rates are shown to be quasi-concave in the relay position for a fixed correlation coefficient, and the DF rates are shown to be quasi-concave in the relay position. The latter property characterizes the optimal relay position when using DF. The results extend to complex AWGN channels with random phase variations. Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

► Show Figures

Figure 1

23 pages, 458 KiB

Open AccessArticle

Gaussian Multiple Access Channels with One-Bit Quantizer at the Receiver †^,^‡

by Borzoo Rassouli, Morteza Varasteh and Deniz Gündüz

Entropy 2018, 20(9), 686; https://0-doi-org.brum.beds.ac.uk/10.3390/e20090686 - 07 Sep 2018

Cited by 11 | Viewed by 2936

Abstract

The capacity region of a two-transmitter Gaussian multiple access channel (MAC) under average input power constraints is studied, when the receiver employs a zero-threshold one-bit analogue-to-digital converter (ADC). It is proven that the input distributions of the two transmitters that achieve the boundary points of the capacity region are discrete. Based on the position of a boundary point, upper bounds on the number of the mass points of the corresponding distributions are derived. Furthermore, a lower bound on the sum capacity is proposed that can be achieved by time division with power control. Finally, inspired by the numerical results, the proposed lower bound is conjectured to be tight. Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

► Show Figures

Figure 1

27 pages, 4528 KiB

Open AccessArticle

Non-Orthogonal eMBB-URLLC Radio Access for Cloud Radio Access Networks with Analog Fronthauling

by Andrea Matera, Rahif Kassab, Osvaldo Simeone and Umberto Spagnolini

Entropy 2018, 20(9), 661; https://0-doi-org.brum.beds.ac.uk/10.3390/e20090661 - 02 Sep 2018

Cited by 24 | Viewed by 5259

Abstract

This paper considers the coexistence of Ultra Reliable Low Latency Communications (URLLC) and enhanced Mobile BroadBand (eMBB) services in the uplink of Cloud Radio Access Network (C-RAN) architecture based on the relaying of radio signals over analog fronthaul links. While Orthogonal Multiple Access (OMA) to the radio resources enables the isolation and the separate design of different 5G services, Non-Orthogonal Multiple Access (NOMA) can enhance the system performance by sharing wireless and fronthaul resources. This paper provides an information-theoretic perspective in the performance of URLLC and eMBB traffic under both OMA and NOMA. The analysis focuses on standard cellular models with additive Gaussian noise links and a finite inter-cell interference span, and it accounts for different decoding strategies such as puncturing, Treating Interference as Noise (TIN) and Successive Interference Cancellation (SIC). Numerical results demonstrate that, for the considered analog fronthauling C-RAN architecture, NOMA achieves higher eMBB rates with respect to OMA, while guaranteeing reliable low-rate URLLC communication with minimal access latency. Moreover, NOMA under SIC is seen to achieve the best performance, while, unlike the case with digital capacity-constrained fronthaul links, TIN always outperforms puncturing. Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

► Show Figures

Figure 1

43 pages, 543 KiB

Open AccessEditor’s ChoiceArticle

Symmetry, Outer Bounds, and Code Constructions: A Computer-Aided Investigation on the Fundamental Limits of Caching

by Chao Tian

Entropy 2018, 20(8), 603; https://0-doi-org.brum.beds.ac.uk/10.3390/e20080603 - 13 Aug 2018

Cited by 41 | Viewed by 4312

Abstract

We illustrate how computer-aided methods can be used to investigate the fundamental limits of the caching systems, which are significantly different from the conventional analytical approach usually seen in the information theory literature. The linear programming (LP) outer bound of the entropy space serves as the starting point of this approach; however, our effort goes significantly beyond using it to prove information inequalities. We first identify and formalize the symmetry structure in the problem, which enables us to show the existence of optimal symmetric solutions. A symmetry-reduced linear program is then used to identify the boundary of the memory-transmission-rate tradeoff for several small cases, for which we obtain a set of tight outer bounds. General hypotheses on the optimal tradeoff region are formed from these computed data, which are then analytically proven. This leads to a complete characterization of the optimal tradeoff for systems with only two users, and certain partial characterization for systems with only two files. Next, we show that by carefully analyzing the joint entropy structure of the outer bounds for certain cases, a novel code construction can be reverse-engineered, which eventually leads to a general class of codes. Finally, we show that outer bounds can be computed through strategically relaxing the LP in different ways, which can be used to explore the problem computationally. This allows us firstly to deduce generic characteristic of the converse proof, and secondly to compute outer bounds for larger problem cases, despite the seemingly impossible computation scale. Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

► Show Figures

Figure 1

Other

Jump to: Editorial, Research

36 pages, 889 KiB

Open AccessTutorial

On the Information Bottleneck Problems: Models, Connections, Applications and Information Theoretic Views

by Abdellatif Zaidi, Iñaki Estella-Aguerri and Shlomo Shamai (Shitz)

Entropy 2020, 22(2), 151; https://0-doi-org.brum.beds.ac.uk/10.3390/e22020151 - 27 Jan 2020

Cited by 55 | Viewed by 6142

Abstract

This tutorial paper focuses on the variants of the bottleneck problem taking an information theoretic perspective and discusses practical methods to solve it, as well as its connection to coding and learning aspects. The intimate connections of this setting to remote source-coding under logarithmic loss distortion measure, information combining, common reconstruction, the Wyner–Ahlswede–Korner problem, the efficiency of investment information, as well as, generalization, variational inference, representation learning, autoencoders, and others are highlighted. We discuss its extension to the distributed information bottleneck problem with emphasis on the Gaussian model and highlight the basic connections to the uplink Cloud Radio Access Networks (CRAN) with oblivious processing. For this model, the optimal trade-offs between relevance (i.e., information) and complexity (i.e., rates) in the discrete and vector Gaussian frameworks is determined. In the concluding outlook, some interesting problems are mentioned such as the characterization of the optimal inputs (“features”) distributions under power limitations maximizing the “relevance” for the Gaussian information bottleneck, under “complexity” constraints. Full article

(This article belongs to the Special Issue Information Theory for Data Communications and Processing)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Information Theory for Data Communications and Processing

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (12 papers)

Editorial

Research

Other

Further Information

Guidelines

MDPI Initiatives

Follow MDPI