Liviu-Daniel Stefan

One of the powers of visual media lies in its ability to create a lasting impression on the viewer’s memory. In this digital age, where media is abundant and attention spans are fleeting, the task of predicting which content will stick in the viewer’s mind has become a critical challenge in computer vision. Computational memorability seeks to address this by developing models that estimate how memorable a piece of media is likely to be. In this review we focus on the MediaEval Predicting Video Memorability benchmark, a recurring evaluation task that has run annually since 2018. This benchmark provides a unique and consistent framework for researchers to compare and refine their memorability prediction techniques using standardised datasets and metrics. Its reproducible framework has proven invaluable for tracking progress and fostering innovation in this rapidly evolving domain. We analyse the evolution of the benchmark across its 2018–2023 editions, discussing the challenges that still remain, such as the need for more interpretability in models and the difficulty of predicting subjective and context-dependent memorability. By analysing and synthesising the collective insights gained from this task, we endeavour to inspire new avenues of inquiry and drive progress towards a more comprehensive understanding of this topic.

The advent of generative networks and their adoption in numerous domains and communities have led to a wave of innovation and breakthroughs in AI and machine learning. Generative Adversarial Networks (GANs) have expanded the scope of what is possible with machine learning, allowing for new applications in areas such as computer vision, natural language processing, and creative AI. GANs, in particular, have been used for a wide range of tasks, including image and video generation, data augmentation, style transfer, and anomaly detection. They have also been used for medical imaging and drug discovery, where they can generate synthetic data to augment small datasets, reduce the need for expensive experiments, and lower the number of real patients that must be included in medical trials. Given these developments, we propose using the power of GANs to create and augment flow-based network traffic datasets. We evaluate a series of GAN architectures, including Wasserstein, conditional, energy-based, gradient penalty, and LSTM-GANs. We evaluate their performance on a set of flow-based network traffic data collected from 16 subjects who used their computers for home, work, and study purposes. The performance of these GAN architectures is described according to metrics that involve networking principles, data distribution among a collection of flows, and temporal data distribution. Given the tendency of network intrusion detection datasets to have a very imbalanced data distribution, i.e., a large number of samples in the “normal traffic” category and a comparatively low number of samples assigned to the “intrusion” categories, we test our GANs by augmenting the intrusion data and checking whether this helps intrusion detection neural networks in their task. We publish the resulting UPBFlow dataset and code on GitHub. ¹

Automated lip-reading, i.e., translating lip movements into text, has received growing interest in recent years with the success of deep learning across a wide variety of tasks. One major obstacle to progress in this field has been the lack of suitable training resources, with the vast majority being limited to a selective set of languages. In this paper, we study the effectiveness of transfer learning to address the lack of massive amounts of labeled data for building a language-independent lip-reading system. Towards this target, we exploit existing knowledge and generalize to new languages via deep neural networks. Experimental validation is carried out on several publicly available data, i.e., LRW for English, LRRo for Romanian, and LRW-1000 for Mandarin showing promising results with significant performance improvements of the multilingual models in contrast to the monolingual ones.