Tianxing Zhang

As a self-supervised learning paradigm, contrastive learning has been widely used to pre-train a powerful encoder as an effective feature extractor for various downstream tasks. This process requires numerous unlabeled training data and computational resources, which makes the pre-trained encoder become the valuable intellectual property of the owner. However, the lack of a priori knowledge of downstream tasks makes it non-trivial to protect the intellectual property of the pre-trained encoder by applying conventional watermarking methods. To deal with this problem, in this paper, we introduce AWEncoder, an adversarial method for watermarking the pre-trained encoder in contrastive learning. First, as an adversarial perturbation, the watermark is generated by enforcing the training samples to be marked to deviate respective location and surround a randomly selected key image in the embedding space. Then, the watermark is embedded into the pre-trained encoder by further optimizing a joint loss function. As a result, the watermarked encoder not only performs very well for downstream tasks, but also enables us to verify its ownership by analyzing the discrepancy of output provided using the encoder as the backbone under both white-box and black-box conditions. Extensive experiments demonstrate that the proposed work enjoys quite good effectiveness and robustness on different contrastive learning algorithms and downstream tasks, which has verified the superiority and applicability of the proposed work.

In the field of three-dimensional (3D) reconstruction, learning-based neural radiation field (NeRF) and 3D-aware generative model (3D-aware GAN) set the foundation for implicit reconstruction of a 3D object. However, this method has problems of slow reconstruction speed and uncontrollable reconstruction results. For this reason, we proposed Info-Giraffe, a virtual neural feature field generation method for object in limited views condition. Info-Giraffe is divided into two phases: pretraining and retraining. First, we generate the basic neural feature field of the object using the simplified Giraffe model in the pretraining phase. Then, during the retraining phase, we extract the 3D information of the object in limited views using a set of latent code encoders designed based on InfoGAN, which is then reparameterized into the latent space. Finally, we continue migration learning by introducing 3D information on the foundation of the object’s basic neural feature field and generate a controllable virtual neural feature field of the object in limited views, which enables fast 3D reconstruction of the object and control of the reconstruction results. Experimental results on Chairs 642 and CompCars 2562 natural image datasets reveal the effectiveness of the proposed method.

Deep neural networks are vulnerable to backdoor attacks, in which the adversary injects a trigger embedded set into the training process. Inputs marked with the trigger provide incorrect predictions, whereas clean inputs remain unaffected. To erase the latent triggers in models, increasing depth, distribution distillation, and model soup (ID3MS), a defensive solution that operates without prior knowledge of triggers and relies on a small clean set is introduced. The depth of the backdoor model is increased by adding fully connected layer(s) at the penultimate layer. Without a classification layer, the original backdoor and increased depth models are considered as teacher and student, respectively. The student model applies distribution distillation to refit the distribution of the clean set and erase the backdoor triggers. The distilled student model is then recovered with the classification layer and model soup is used to ensemble a collection of models generated by various fine-tuning hyperparameters. The experimental results validate the superior performance of the ID3MS compared with existing defensive techniques against several attacks across datasets.