Descriptor: Benchmark Dataset for Animated Multimedia Quality Assessment (BeDAMQA)
Open Access
- 5 November 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Data Descriptions
- Vol. 2, 441-451
Abstract
The tremendous development of animated multimedia (AM) utilization is attributed to major advancements in computer graphics, education, gaming, and immersive perceptual technologies like augmented and virtual realities. However, neither the perceptual quality nor mathematical assessments of AM content have kept pace with the rapid growth of the content, and standard AM datasets are also quite limited. This work addresses the aforementioned shortcomings by developing a new AM video dataset with a large subjective study. The proposed study is comprehensive in terms of a variety of video content, distortion types, test stimuli count, and human rating computation. Specifically, the dataset consists of 15 pristine videos that cover a wide range of spatial and motion information, and these videos are subjected to five commonly occurring distortions, including H.264 and H.265 compressions, blur, frame freeze, and frame hanging distortions. We generate a total of 375 distorted test stimuli by applying five perceptual levels of each distortion type to the reference videos. Further, 30 participants were involved in our human-assessed study, and finally, mean opinion scores were calculated based on the subjective scores. As a part of mathematical analysis, several well-known image and video quality assessment algorithms were evaluated on the dataset, and their performances are reported in terms of correlation coefficient and error values. Based on the proposed human-assessed study and mathematical analysis, we conclude that the existing objective quality assessment models do not perform well on the AM.Keywords
This publication has 21 references indexed in Scilit:
- A content-oriented no-reference perceptual video quality assessment method for computer graphics animation videosInformation Sciences, 2022
- Blind image quality evaluation using perception based features2015
- Making a “Completely Blind” Image Quality AnalyzerIEEE Signal Processing Letters, 2012
- Video Quality Assessment on Mobile Devices: Subjective, Behavioral and Objective StudiesIEEE Journal of Selected Topics in Signal Processing, 2012
- Effect of brightness on the quality of visual 3D perception2011
- A spatiotemporal most-apparent-distortion model for video quality assessment2011
- SOS: The MOS is not enough!2011
- Image information and visual qualityIEEE Transactions on Image Processing, 2006
- Multiscale structural similarity for image quality assessment2004
- Image quality assessment: from error visibility to structural similarityIEEE Transactions on Image Processing, 2004