Zhenyu Bao

Despite much progress, achieving real-time high-fidelity head avatar animation is still difficult and existing methods have to trade-off between speed and quality. 3DMM based methods often fail to model non-facial structures such as eyeglasses and hairstyles, while neural implicit models suffer from deformation inflexibility and rendering inefficiency. Although 3D Gaussian has been demonstrated to possess promising capability for geometry representation and radiance field reconstruction, applying 3D Gaussian in head avatar creation remains a major challenge since it is difficult for 3D Gaussian to model the head shape variations caused by changing poses and expressions. In this paper, we introduce PSAvatar,¹ a novel framework for animatable head avatar creation that utilizes discrete geometric primitive to create a parametric shape model and employs 3D Gaussian for fine detail representation and high fidelity rendering. The parametric shape model is a Point-based Shape Model (PSM) which uses points instead of meshes for 3D representation to achieve enhanced representation flexibility. Specifically, PSM first converts the FLAME mesh to points by sampling on the surfaces as well as off the meshes to enable the reconstruction of not only surface-like structures but also complex geometries such as eyeglasses and hairstyles. By aligning these points with the head shape in an analysis-by-synthesis manner, the PSM makes it possible to utilize 3D Gaussian for fine detail representation and appearance modeling, thus enabling the creation of high-fidelity avatars. We show that PSAvatar can reconstruct high-fidelity head avatars of varieties of subjects and the avatars can be animated in real-time.².

Exploiting 3D Gaussian Splatting (3DGS) with Contrastive Language-Image Pre-Training (CLIP) models for open-vocabulary 3D semantic understanding of indoor scenes has emerged as an attractive research focus. Existing methods typically attach high-dimensional CLIP semantic embeddings to 3D Gaussians and leverage view-inconsistent 2D CLIP semantics as Gaussian supervision, resulting in efficiency bottlenecks and deficient 3D semantic consistency. To address these challenges, we present CLIP-GS, efficiently achieving a coherent semantic understanding of 3D indoor scenes via the proposed Semantic Attribute Compactness (SAC) and 3D Coherent Regularization (3DCR). SAC approach exploits the naturally unified semantics within objects to learn compact, yet effective, semantic Gaussian representations, enabling highly efficient rendering (>100 FPS). 3DCR enforces semantic consistency in 2D and 3D domains: In 2D, 3DCR utilizes refined view-consistent semantic outcomes derived from 3DGS to establish cross-view coherence constraints; in 3D, 3DCR encourages features similar among 3D Gaussian primitives associated with the same object, leading to more precise and coherent segmentation results. Extensive experimental results demonstrate that our method remarkably suppresses existing state-of-the-art approaches, achieving mIoU improvements of 21.20% and 13.05% on ScanNet and Replica datasets, respectively, while maintaining real-time rendering speed. Furthermore, our approach exhibits superior performance even with sparse input data, substantiating its robustness.

Despite the photorealistic novel view synthesis (NVS) performance achieved by the original 3D Gaussian splatting (3DGS), its rendering quality significantly degrades with sparse input views. This performance drop is mainly caused by the limited number of initial points generated from the sparse input, lacking reliable geometric supervision during the training process, and inadequate regularization of the oversized Gaussian ellipsoids. To handle these issues, we propose the LoopSparseGS, a loop-based 3DGS framework for the sparse novel view synthesis task. In specific, we propose a loop-based Progressive Gaussian Initialization (PGI) strategy that could iteratively densify the initialized point cloud using the rendered pseudo images during the training process. Then, the sparse and reliable depth from the Structure from Motion, and the window-based dense monocular depth are leveraged to provide precise geometric supervision via the proposed Depth-alignment Regularization (DAR). Additionally, we introduce a novel Sparse-friendly Sampling (SFS) strategy to handle oversized Gaussian ellipsoids leading to large pixel errors. Comprehensive experiments on four datasets demonstrate that LoopSparseGS outperforms existing state-of-the-art methods for sparse-input novel view synthesis, across indoor, outdoor, and object-level scenes with various image resolutions. Code is available at: https://github.com/pcl3dv/LoopSparseGS.

Metal defect detection is significant for smart manufacturing, such as improving product quality, reducing production costs, and ensuring production safety. Compared to defect detection, defect imaging can obtain information on the spatial location, orientation and distribution of defects, so as to make a quantitative judgment on the degree of damage of defects. The electromagnetic properties of metal samples are closely related to their microstructures, and the reconstruction of electromagnetic distribution can image defects. In this paper, a PID controlled L₁ regularized tomography method is investigated to achieve high-precision 3D imaging of defects. Since the coupling between the conductivity and magnetic of the magnetic sample will affect the reconstruction results, the relationship between the excitation frequency and the electromagnetic properties is first analyzed in accordance with theoretical model to select a suitable excitation frequency. Moreover, the idea of closed-loop control is applied to the L₁ regularization algorithm. Specifically, the L₁ regularization algorithm is used as the controlled object of the PID controller. In the feedback loop, the reconstructed image is mapped to the computed projected values via Dirichlet-to-Neumann. The deviation between the computed projection value and the measured value is used as an input to the controller, thus constituting the PID controlled iterative L₁ regularization method. The DP steels with different defects distribution are used to verify the proposed method, and the numerical and experimental results indicate the proposed method can eliminate artifacts and obtain high-quality defect reconstruction results. Compared with classical imaging method, the correlation coefficients of the reconstruction errors using the proposed method are improved by 18.0% and 23.6% for complex defect distribution, respectively.

The parameters coupling affects the measurement accuracy in eddy current testing. In this paper, we have developed a decoupling method to simultaneously measure conductivity and radius for non-magnetic metallic rod using sweep frequency strategy. First of all, the crossover frequency (i.e. the frequency corresponding to the intersection of real and imaginary parts of mutual inductance) is derived to proportional to the reciprocal of conductivity in line with the proposed simplified Dodd-Deeds analytical solution. Meanwhile, the slope of linear fitting function can be expressed as a power function of metallic rod diameter, and the corresponding coefficient of power function is associated with the coil size. Hence, a sensor composed of two absolute coils with different sizes is designed to simultaneously measure the conductivity and diameter. According to the relationship between the crossover frequency and parameters of metallic rod, the conductivity and diameter can be reconstructed from response signals of two absolute coils with different sizes. Finally, the proposed rod conductivity and diameter measurement method has been validated by experiments, which demonstrates the proposed methodology is feasible and effectiveness.

Simultaneously achieving 3D reconstruction and novel view synthesis for indoor environments has widespread applications but is technically very challenging. State-of-the-art methods based on implicit neural functions can achieve excellent 3D reconstruction results, but their performances on new view synthesis can be unsatisfactory. The exciting development of neural radiance field (NeRF) has revolutionized novel view synthesis, however, NeRF-based models can fail to reconstruct clean geometric surfaces. We have developed a dual neural radiance field (Du-NeRF) to simultaneously achieve high-quality geometry reconstruction and view rendering. Du-NeRF contains two geometric fields, one derived from the SDF field to facilitate geometric reconstruction and the other derived from the density field to boost new view synthesis. One of the innovative features of Du-NeRF is that it decouples a view-independent component from the density field and uses it as a label to supervise the learning process of the SDF field. This reduces shape-radiance ambiguity and enables geometry and color to benefit from each other during the learning process. Extensive experiments demonstrate that Du-NeRF can significantly improve the performance of novel view synthesis and 3D reconstruction for indoor environments and it is particularly effective in constructing areas containing fine geometries that do not obey multi-view color consistency. Our code is available at: https://github.com/pcl3dv/DuNeRF.

The development of Neural Radiance Fields (NeRFs) has provided a potent representation for encapsulating the geometric and appearance characteristics of 3D scenes. Enhancing the capabilities of NeRFs in open-vocabulary 3D semantic perception tasks has been a recent focus. However, current methods that extract semantics directly from Contrastive Language-Image Pretraining (CLIP) for semantic field learning encounter difficulties due to noisy and view-inconsistent semantics provided by CLIP. To tackle these limitations, we propose OV-NeRF, which exploits the potential of pre-trained vision and language foundation models to enhance semantic field learning through proposed single-view and cross-view strategies. First, from the single-view perspective, we introduce Region Semantic Ranking (RSR) regularization by leveraging 2D mask proposals derived from Segment Anything (SAM) to rectify the noisy semantics of each training view, facilitating accurate semantic field learning. Second, from the cross-view perspective, we propose a Cross-view Self-enhancement (CSE) strategy to address the challenge raised by view-inconsistent semantics. Rather than invariably utilizing the 2D inconsistent semantics from CLIP, CSE leverages the 3D consistent semantics generated from the well-trained semantic field itself for semantic field training, aiming to reduce ambiguity and enhance overall semantic consistency across different views. Extensive experiments validate our OV-NeRF outperforms current state-of-the-art methods, achieving a significant improvement of 20.31% and 18.42% in mIoU metric on Replica and ScanNet, respectively. Furthermore, our approach exhibits consistent superior results across various CLIP configurations, further verifying its robustness. Codes are available at: https://github.com/pcl3dv/OV-NeRF.

This paper proposed a method to simultaneously estimate the permeability, conductivity, thickness and lift-off of metallic plates using sweep frequency eddy current measurement combined with machine learning. The whole processing mainly divided into characteristic extractions and data driven machine learning model. According to the classical Dodd and Deeds analytical solution, the characteristic value of mutual inductance signal related with metallic plates parameters are extracted based on theoretical derivation. The parameters of metallic plates are inherently coupled together, thus the parameters of metallic plates cannot be directly and accurately obtained only using characteristic information. In this paper, the machine learning modelling combining the extracted features is established to achieve decoupling and high precision inversion. The data required for model training comes from the customized fast FEM solver and analytical solution. While, the experiments data acts as the test set to verify the performance of the proposed method. The results show the proposed method can be applied to accurately estimate the parameters of metallic plates, and the relative errors of inverted parameters are within 3.5%.

With the continuous development of human society, earthquakes are becoming more and more dangerous to the production and life of human society. Researchers continue to try to predict earthquakes, but the results are still not significant. With the development of data science, sensing and communication technologies, there are increasing efforts to use machine learning methods to predict earthquakes. Our work raises a method that applies big data analysis and machine learning algorithms to earthquakes prediction. All data are accumulated by the Acoustic and Electromagnetic Testing All in One System (AETA). We propose the multi-station Principal Component Analysis (PCA) algorithm and extract features based on this method. At last, we propose a weekly-scale earthquake prediction model, which has a 60% accuracy using LightGBM (LGB).

The influence of earthquake disasters on human social life is positively related to the magnitude and intensity of the earthquake, and effectively avoiding casualties and property losses can be attributed to the accurate prediction of earthquakes. In this study, an electromagnetic sensor is investigated to assess earthquakes in advance by collecting earthquake signals. At present, the mainstream earthquake magnitude prediction comprises two methods. On the one hand, most geophysicists or data analysis experts extract a series of basic features from earthquake precursor signals for seismic classification. On the other hand, the obtained data related to earth activities by seismograph or space satellite are directly used in classification networks. This article proposes a CNN and designs a 3D feature-map which can be used to solve the problem of earthquake magnitude classification by combining the advantages of shallow features and high-dimensional information. In addition, noise simulation technology and SMOTE oversampling technology are applied to overcome the problem of seismic data imbalance. The signals collected by electromagnetic sensors are used to evaluate the method proposed in this article. The results show that the method proposed in this paper can classify earthquake magnitudes well.