Qiang Li

Integral imaging is a promising three-dimensional (3D) display technology as it can reconstruct full-parallax light fields without causing vergence-accommodation conflicts. However, generating high frame rate element image arrays (EIAs) effectively for 3D videos remains a challenging task. In this study, we propose a novel intermediate frame prediction method for integral imaging using a network called Triple I-3D Net. The proposed method constructs adaptive receptive fields and patches to estimate each pixel's values, ensuring the generated intermediate EIA frames preserve a correct internal optical logic. Additionally, the proposed method incorporates both motion estimation and pixel synthesis within a unified loss function, thereby achieving balanced optimization between the two tasks. By exploiting the intrinsic properties of EIA, our method enables the efficient synthesis of high frame rate sequences without modifying the optical system, thereby significantly enhancing the dynamic performance of 3D display. The experimental results indicate that Triple I-3D Net significantly outperforms other methods, with a maximum improvement of 12.2% in peak signal-to-noise ratio, a 37.8% enhancement in structural similarity index measure, and a 95% boost in learned perceptual image patch similarity. Furthermore, the corrected sample variance across evaluation metrics is reduced by up to 93.5% in the best cases. These results demonstrate that the proposed method realizes accurate frame interpolation for integral imaging and confirms the stability of the interpolated frames.

To address voxel diffusion in integral imaging caused by multiple rotation errors in the lens array, a global optimization method based on homologous pixel-voxel distribution learning of a generative adversarial network (PVGAN) is proposed. By establishing a 3D resolution degradation model under multiple rotation errors, this paper breaks the bottlenecks by proposing the quantitative characterization of multi-error coupling mechanisms for what we believe to be the first time, overcoming the inherent limitations of conventional methods in holistic error-system modeling. The PVGAN framework consists of pixel rearrangement and pixel-to-pixel conditional generative adversarial network modules. The former module establishes the geometric correspondence between preprocessed voxels and the homologous pixels on the element image arrays, while the latter enables end-to-end voxel diffusion correction. Notably, the trained model is capable of batch-processing the voxel diffusion without requiring iterative preprocessing stages. Optical experiments reveal that the method achieves high-quality displays with a horizontal and vertical resolution of 16 lp/mm over a 40° field of view, which is 3.174 and 4 times better than the uncorrected system, respectively. Compared with the existing single-error correction methods, the proposed method significantly suppresses voxel diffusion while improving edge texture feature clarity, which provides stronger support for accurate battlefield simulation and instructional demonstration.

Under the framework of integral imaging, a high-precision 3D salient object detection and high-quality texture features reconstruction method is proposed by using the element to element-transformer and generative adversarial network (E2E-TransGAN). The proposed method fully exploits the global salient clues of the element image array to establish cross-regional element image dependency, thereby the accuracy of the salient prediction improved. Meanwhile, our proposed E2E-TransGAN algorithm stratifies the binary salient object area and the grayscale salient object edge, effectively addressing salient region prediction errors, obvious background noise, and color loss. The experimental results show that the detection accuracy of the proposed method improves to an average of 90% across various evaluation metrics. Compared with other methods, the proposed method can reconstruct 3D images with texture features, which have high values of PSNR and SSIM, at 30 and 0.95 on average, respectively. We have experimentally verified that the proposed method achieves precise 3D object salient detection and restores the detailed textures and depth information of 3D salient objects. The proposed method provides strong support for efficient and accurate medical image analysis and simplifies subsequent computer vision tasks.

Augmented reality overlays computer-generated virtual information onto real-world scenes, enhancing user interaction and perception. However, traditional augmented reality optical systems are usually large, bulky, and have limited optical performance. In this paper, we propose a novel compact monochrome reflective dielectric metalens visor with see-through properties, engineered using a periodic structure of spin-dependent supercells. The supercell, which is composed of staggered twin nanofins, provides spin-dependent destructive or constructive interference with different circularly polarized incidences. The design combines the principles of interference with the Pancharatnam–Berry phase to enhance reflection at a working wavelength of 650 nm while maintaining good transmission. Right circularly polarized light incident from the substrate side causes destructive interference, enabling the supercell to work in reflection mode, while left circularly polarized light causes constructive interference, enabling the supercell to work in transmission mode. Furthermore, the supercell-constructed metalens can achieve near-diffraction-limited reflective focusing and a broad diagonal field of view of approximately 96°. In addition, compared to transmissive metalens visors, the reflective design eliminates the need for a beam splitter, significantly reducing the size and weight of the system. Our work could facilitate the development of compact and lightweight imaging systems and provide valuable insights for augmented reality near-eye display applications.

Despite the promising performance of existing visual models on public benchmarks, the critical assessment of their robustness for real-world applications remains an ongoing challenge. To bridge this gap, we propose an explainable visual dataset, XIMAGENET-12, to evaluate the robustness of visual models. XIMAGENET-12 consists of over 200K images with 15,410 manual semantic annotations. Specifically, we deliberately selected 12 categories from ImageNet, representing objects commonly encountered in practical life. To simulate real-world situations, we incorporated six diverse scenarios, such as overexposure, blurring, and color changes, etc. We further develop a quantitative criterion for robustness assessment, allowing for a nuanced understanding of how visual models perform under varying conditions, notably in relation to the background. We make the XIMAGENET-12 dataset and its corresponding code openly accessible at \url{https://sites.google.com/view/ximagenet-12/home}. We expect the introduction of the XIMAGENET-12 dataset will empower researchers to thoroughly evaluate the robustness of their visual models under challenging conditions.

The advent of deep learning has brought a revolutionary transformation to image denoising techniques. However, the persistent challenge of acquiring noise-clean pairs for supervised methods in real-world scenarios remains formidable, necessitating the exploration of more practical self-supervised image denoising. This paper focuses on self-supervised image denoising methods that offer effective solutions to address this challenge. Our comprehensive review thoroughly analyzes the latest advancements in self-supervised image denoising approaches, categorizing them into three distinct classes: General methods, Blind Spot Network (BSN)-based methods, and Transformer-based methods. For each class, we provide a concise theoretical analysis along with their practical applications. To assess the effectiveness of these methods, we present both quantitative and qualitative experimental results on various datasets, utilizing classical algorithms as benchmarks. Additionally, we critically discuss the current limitations of these methods and propose promising directions for future research. By offering a detailed overview of recent developments in self-supervised image denoising, this review serves as an invaluable resource for researchers and practitioners in the field, facilitating a deeper understanding of this emerging domain and inspiring further advancements.