Christian Herglotz

The demand for high-quality, immersive video experiences has necessitated adopting higher frame rates for more realistic scene portrayal. However, there is a growing demand for energy-saving video applications, where temporal downsampling is a crucial energy-saving factor. To this end, the main contributions of this paper are twofold: First, we present an in-depth analysis of the impact of frame rate reductions on the visual quality of the video and the encoding as well as decoding energy. Second, we propose a lightweight frame rate selection method for energy- and quality-aware encoding. Concerning the first contribution, this paper performs extensive encoding and decoding measurements, followed by an investigation of the impact of temporal downsampling on the energy demand of encoding and decoding at different frame rates. Furthermore, we determine the objective visual quality of the downsampled videos. As a result of this investigation, we identify content- and quantization-setting-dependent energy-aware frame rates, i.e., the temporal downsampling factors that lead to Pareto-optimality in terms of energy and quality. We demonstrate that significant energy savings are achieved while maintaining constant visual quality. Subsequently, a subjective experiment is conducted to verify this observation regarding perceptual quality using mean opinion scores. As the second contribution, we propose an energy-aware frame rate selection method that extracts spatio-temporal features from the video sequences. Based on these features, the proposed method employs a feature-based supervised machine learning approach to predict energy-aware frame rates for a given quantization parameter and video sequence, aiming to reduce energy consumption during encoding and decoding. The experimental results demonstrate that the proposed method offers significant energy savings, with an average of 17.46% and 17.60% of encoding and decoding energy demand reduction, respectively, alongside 3.38% average bitrate savings at a constant quality.

In today's society, live video streaming and user generated content streamed from battery powered devices are ubiquitous. Live streaming requires real-time video encoding, and hardware video encoders are well suited for such an encoding task. In this paper, we introduce a high-level feature model using Gaussian process regression that can predict the encoding energy of a hardware video encoder. In an evaluation setup restricted to only P-frames and a single keyframe, the model can predict the encoding energy with a mean absolute percentage error of approximately 9%. Further, we demonstrate with an ablation study that spatial resolution is a key high-level feature for encoding energy prediction of a hardware encoder. A practical application of our model is that it can be used to perform a prior estimation of the energy required to encode a video at various spatial resolutions, with different coding standards and codec presets.

With the advent of advanced transmission technologies, powerful neural-network architectures, and the availability of cheap sensing solutions, new paradigms for monitoring, controlling, and optimizing any automated system are appearing. While traditionally, sensors simply provided visual or acoustic feedback to human operators of such systems, nowadays, autonomous systems more and more rely on the automatic sensing and processing of such data to react and adapt their control mechanisms. In such complex control systems, sensor networks play a vital role. In this paper, we assess the state of the art in the three main components used within sensor networks: smart sensing elements, transmission technologies, and processing technologies. Concerning the latter, more and more artificial intelligence-based methods are developed and applied turning smart sensor networks into intelligent or even cognitive sensor networks. For the development of such sensing systems, a high variety of performance metrics could be important to the designer. This paper discusses the vast design space for intelligent sensor networks and provides guidance and directions on how to construct future intelligent sensor networks efficiently.

Given the growing environmental concerns and significant resource consumption associated with video streaming on electronic devices, measuring the energy consumption is important to guide optimisation and to assess its relative environmental impact. In this paper, we provide comprehensive guidance to accurately measure the energy and power consumption in video communication technologies. We address the complexities inherent in measuring energy consumption across diverse software and hardware setups, with a focus on video communication tasks. We review current measurement techniques, identify limitations in existing practices, and propose a structured methodology that incorporates considerations for static and dynamic power consumption, appropriate sampling frequencies, and statistical rigor. Additionally, we introduce a reference workflow that is adaptable to various multimedia applications and demonstrate its applicability through a case study. By offering clear guidance and practical tools, this work aims to improve the reliability, reproducibility, and comparability of energy consumption measurements in video technologies, providing a strong foundation for the multimedia community to base decisions on.

The past years have shown that due to the global success of video communication technology, the corresponding hardware systems nowadays contribute significantly to pollution and resource consumption on a global scale, accounting for 1% of global green house gas emissions in 2018. This aspect of sustainability has thus reached increasing attention in academia and industry. In this paper, we present different aspects of sustainability including resource consumption and greenhouse gas emissions, while putting a major focus on the energy consumption during the use of video systems. Finally, we provide an overview on recent research in the domain of green video communications showing promising results and highlighting areas where more research should be performed.

High-dynamic range (HDR) video content has gained popularity due to its enhanced color depth and luminance range, but it also presents new challenges in terms of compression efficiency and energy consumption. In this paper, we present an in-depth study of the compression performance and energy efficiency of HDR video encoding using High-Efficiency Video Coding (HEVC). In addition to using a native 10-bit HDR encoding configuration as a reference, we explore whether applying tone mapping to an 8-bit representation before encoding can result in additional bitrate and energy savings without compromising visual quality. The main contributions of this work are as follows: 1) a detailed evaluation of four HDR video encoding configurations, three of which leverage tone mapping techniques, 2) a comprehensive experimental setup involving over 15,000 individual encodings across three open-source HEVC encoders (Kvazaar, x265, and SVT-HEVC) and multiple presets, 3) the use of two advanced perception-based metrics for BD-rate calculations, one of which is specifically tailored to capture colour distortions and 4) an open-source dataset consisting of all experimental results for further research. Among the three tone-mapping configurations tested, our findings show that a simple bit-shifting approach can achieves significant reductions in both bitrate and energy consumption compared to the native 10-bit HDR encoding configuration. This research aims to lay an initial foundation for understanding the balance between coding efficiency and energy consumption in HDR video encoding, offering valuable insights to guide future advancements in the field.