Haiyan Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Haiyan Zhang has a total of 26 co-authors. Here are the top co-authors by number of shared publications:
- Xinghua Li
Xidian University
10 shared publications - Robert Huijie Deng
Singapore Management University, Singapore, Singapore
9 shared publications - Shunjie Yuan
State Key Laboratory of Integrated Services Networks, and the School of Cyber Engineering, Xidian University, Xian, China
6 shared publications - Yinbin Miao
Xidian University
6 shared publications - Ximeng Liu
College of Computer and Data Science, Fuzhou University, Fuzhou, China
5 shared publications Mengfan Xu
School of Computer Science, Shaanxi Normal University, Xian, China
2 shared publications
Latest Publications
- 16 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 25 (8) , 13166-13182
Defend Against Label Inference Attacks in Vertical Federated Learning via Label Compression
Vertical federated learning (VFL) has been widely adopted in various domains for collaborative decision-making. However, recent studies have revealed critical privacy vulnerabilities in VFL, particularly label inference attacks, which significantly undermine label confidentiality and limit the applicability of VFL in privacy-sensitive scenarios. To mitigate such threats, several defense methods have been proposed by incorporating diverse privacy-preserving techniques. Nevertheless, existing defenses fail to effectively prevent the recently proposed model completion-based label inference attacks. To address this limitation, we propose a novel defense method, termed Label Compression-Based Defense (LCD), to defend against this class of attacks. The core idea of LCD is to train the VFL model using fake labels, thereby decoupling the ground-truth labels from the outputs of the malicious bottom model, which constitute the critical component exploited in the model completion-based attacks. Specifically, we introduce a multi-stage training strategy that decomposes the training process into different stages to deceive the malicious bottom model without affecting the original task. In addition, we design a deep feature-based label compression mechanism to generate fake labels for misleading the attacker. To further enhance the defense effectiveness, we propose an embedding compaction strategy based on center loss, which substantially increases the difficulty of label inference. Moreover, we theoretically prove the effectiveness of LCD from an information-theoretic perspective. Extensive experiments on both tabular and image datasets demonstrate that LCD can effectively defend against label inference attacks. The source code of LCD is publicly available at GitHub: https://github.com/YuanShunJie1/LCD.
- 16 March 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Mobile Computing
- Vol. 25 (8) , 13166-13182
Defend Against Label Inference Attacks in Vertical Federated Learning via Label Compression
Vertical federated learning (VFL) has been widely adopted in various domains for collaborative decision-making. However, recent studies have revealed critical privacy vulnerabilities in VFL, particularly label inference attacks, which significantly undermine label confidentiality and limit the applicability of VFL in privacy-sensitive scenarios. To mitigate such threats, several defense methods have been proposed by incorporating diverse privacy-preserving techniques. Nevertheless, existing defenses fail to effectively prevent the recently proposed model completion-based label inference attacks. To address this limitation, we propose a novel defense method, termed Label Compression-Based Defense (LCD), to defend against this class of attacks. The core idea of LCD is to train the VFL model using fake labels, thereby decoupling the ground-truth labels from the outputs of the malicious bottom model, which constitute the critical component exploited in the model completion-based attacks. Specifically, we introduce a multi-stage training strategy that decomposes the training process into different stages to deceive the malicious bottom model without affecting the original task. In addition, we design a deep feature-based label compression mechanism to generate fake labels for misleading the attacker. To further enhance the defense effectiveness, we propose an embedding compaction strategy based on center loss, which substantially increases the difficulty of label inference. Moreover, we theoretically prove the effectiveness of LCD from an information-theoretic perspective. Extensive experiments on both tabular and image datasets demonstrate that LCD can effectively defend against label inference attacks. The source code of LCD is publicly available at GitHub: https://github.com/YuanShunJie1/LCD.
- 4 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 22 (6) , 7157-7171
General Test-Time Backdoor Detection in Split Neural Network-Based Vertical Federated Learning
As a new distributed machine learning framework, vertical federated learning (VFL) has been widely applied in the industry. However, recent studies have demonstrated that VFL faces serious challenges from backdoor attacks, which significantly hinder its further development. Although a few studies have focused on defending against VFL backdoor attacks, these defenses either do not consider the latest attack methods or show limited effectiveness. Moreover, most existing backdoor defense efforts primarily focus on backdoor attacks in horizontal federated learning (HFL) and centralized learning. Due to the unique architecture of VFL models, these methods cannot be directly applied to backdoor defense in VFL. To mitigate the threat of backdoor attacks in VFL, we propose a general backdoor detection (GBD) scheme for backdoor defense, which detects backdoor samples by analyzing the correlation between backdoor samples and the target label, as well as by leveraging the response differences between clean and backdoor samples. Specifically, we propose two backdoor detection metrics: Class Activation Probability (CAP) and Class Activation Contribution (CAC), which are used to calculate the likelihood of a sample being a backdoor sample. We leverage these two metrics to identify backdoor samples during the inference stage. Evaluation results on both tabular and image datasets show that GBD can detect backdoor samples with high accuracy, demonstrating its effectiveness in backdoor defense. The source code of GBD is available at GitHub: https://github.com/YuanShunJie1/GBD.
- 4 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 22 (6) , 7157-7171
General Test-Time Backdoor Detection in Split Neural Network-Based Vertical Federated Learning
As a new distributed machine learning framework, vertical federated learning (VFL) has been widely applied in the industry. However, recent studies have demonstrated that VFL faces serious challenges from backdoor attacks, which significantly hinder its further development. Although a few studies have focused on defending against VFL backdoor attacks, these defenses either do not consider the latest attack methods or show limited effectiveness. Moreover, most existing backdoor defense efforts primarily focus on backdoor attacks in horizontal federated learning (HFL) and centralized learning. Due to the unique architecture of VFL models, these methods cannot be directly applied to backdoor defense in VFL. To mitigate the threat of backdoor attacks in VFL, we propose a general backdoor detection (GBD) scheme for backdoor defense, which detects backdoor samples by analyzing the correlation between backdoor samples and the target label, as well as by leveraging the response differences between clean and backdoor samples. Specifically, we propose two backdoor detection metrics: Class Activation Probability (CAP) and Class Activation Contribution (CAC), which are used to calculate the likelihood of a sample being a backdoor sample. We leverage these two metrics to identify backdoor samples during the inference stage. Evaluation results on both tabular and image datasets show that GBD can detect backdoor samples with high accuracy, demonstrating its effectiveness in backdoor defense. The source code of GBD is available at GitHub: https://github.com/YuanShunJie1/GBD.
- 17 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 22 (6) , 6732-6747
FL-CDF: Collaborative Defense Framework for Backdoor Mitigation in Federated Learning
Federated learning (FL) is vulnerable to backdoor attacks due to its distributed nature. Existing unilateral defense mechanisms often fail against persistent attack strategies, primarily due to their limited perspectives. To address the challenge of model misclassification on the server side caused by overlooked model similarity drift, and gradient misjudgment on the client side caused by semantic learning imbalances across classes, this paper proposes a collaborative defense framework for federated learning, termed FL-CDF. FL-CDF establishes an end-to-end defense through a bidirectional client-server collaboration mechanism. Specifically: (1) On the client side, an adversarial perturbation-based malicious neuron detection module is introduced. This module measures neuron activation sensitivity by generating adversarial perturbations, and adaptively prunes backdoor neurons exhibiting high sensitivity. (2) On the server side, a multi-dimensional detection scheme is designed, which integrates neuron localization, adversarial sensitivity, and model parameters. By incorporating client-side feedback on malicious neurons, the server performs robust model aggregation. Theoretical analysis verifies the robustness of FL-CDF, and extensive experiments on public benchmarks demonstrate its effectiveness. In the best-case scenario, FL-CDF improves defense performance by 42.5% compared to current state-of-the-art (SOTA) defense.
- 17 July 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Dependable and Secure Computing
- Vol. 22 (6) , 6732-6747
FL-CDF: Collaborative Defense Framework for Backdoor Mitigation in Federated Learning
Federated learning (FL) is vulnerable to backdoor attacks due to its distributed nature. Existing unilateral defense mechanisms often fail against persistent attack strategies, primarily due to their limited perspectives. To address the challenge of model misclassification on the server side caused by overlooked model similarity drift, and gradient misjudgment on the client side caused by semantic learning imbalances across classes, this paper proposes a collaborative defense framework for federated learning, termed FL-CDF. FL-CDF establishes an end-to-end defense through a bidirectional client-server collaboration mechanism. Specifically: (1) On the client side, an adversarial perturbation-based malicious neuron detection module is introduced. This module measures neuron activation sensitivity by generating adversarial perturbations, and adaptively prunes backdoor neurons exhibiting high sensitivity. (2) On the server side, a multi-dimensional detection scheme is designed, which integrates neuron localization, adversarial sensitivity, and model parameters. By incorporating client-side feedback on malicious neurons, the server performs robust model aggregation. Theoretical analysis verifies the robustness of FL-CDF, and extensive experiments on public benchmarks demonstrate its effectiveness. In the best-case scenario, FL-CDF improves defense performance by 42.5% compared to current state-of-the-art (SOTA) defense.
- 12 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 20 (15566013) , 6086-6100
Robust Federated Learning Client Selection With Combinatorial Class Representations and Data Augmentation
The federated learning (FL) client selection scheme can effectively mitigate global model performance degradation caused by the random aggregation of clients with heterogeneous data. Simultaneously, research has exposed FL’s susceptibility to backdoor attacks. However herein lies the dilemma, traditional client selection methods and backdoor defenses stand at odds, so their integration is an elusive goal. To resolve this, we introduce Grace, a resilient client selection framework blending combinational class sampling with data augmentation. On the client side, Grace first proposes a local model purification method, fortifying the model’s defenses by bolstering its innate robustness. After, local class representations are extracted for server-side client selection. This approach not only shields benign models from backdoor tampering but also allows the server to glean insights into local class representations without infringing upon the client’s privacy. On the server side, Grace introduces a novel representation combination sampling method. Clients are selected based on the interplay of their class representations, a strategy that simultaneously weeds out malicious actors and draws in clients whose data holds unique value. Our extensive experiments highlight Grace’s capabilities. The results are compelling: Grace enhances defense performance by over 50% compared to state-of-the-art (SOTA) backdoor defenses, and, in the best case, improves accuracy by 3.19% compared to SOTA client selection schemes. Consequently, Grace achieves substantial advancements in both security and accuracy.
- 12 June 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 20 (15566013) , 6086-6100
Robust Federated Learning Client Selection With Combinatorial Class Representations and Data Augmentation
The federated learning (FL) client selection scheme can effectively mitigate global model performance degradation caused by the random aggregation of clients with heterogeneous data. Simultaneously, research has exposed FL’s susceptibility to backdoor attacks. However herein lies the dilemma, traditional client selection methods and backdoor defenses stand at odds, so their integration is an elusive goal. To resolve this, we introduce Grace, a resilient client selection framework blending combinational class sampling with data augmentation. On the client side, Grace first proposes a local model purification method, fortifying the model’s defenses by bolstering its innate robustness. After, local class representations are extracted for server-side client selection. This approach not only shields benign models from backdoor tampering but also allows the server to glean insights into local class representations without infringing upon the client’s privacy. On the server side, Grace introduces a novel representation combination sampling method. Clients are selected based on the interplay of their class representations, a strategy that simultaneously weeds out malicious actors and draws in clients whose data holds unique value. Our extensive experiments highlight Grace’s capabilities. The results are compelling: Grace enhances defense performance by over 50% compared to state-of-the-art (SOTA) backdoor defenses, and, in the best case, improves accuracy by 3.19% compared to SOTA client selection schemes. Consequently, Grace achieves substantial advancements in both security and accuracy.
- 24 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 597-609
Combating Noisy Labels by Alleviating the Memorization of DNNs to Noisy Labels
Data is the essential fuel for deep neural networks (DNNs), and its quality affects the practical performance of DNNs. In real-world training scenarios, the successful generalization performance of DNNs is severely challenged by noisy samples with incorrect labels. To combat noisy samples in image classification, numerous methods based on sample selection and semi-supervised learning (SSL) have been developed, where sample selection is used to provide the supervision signal for SSL, achieving great success in resisting noisy samples. Due to the necessary warm-up training on noisy datasets and the basic sample selection mechanism, DNNs are still confronted with the challenge of memorizing noisy samples. However, existing methods do not address the memorization of noisy samples by DNNs explicitly, which hinders the generalization performance of DNNs. To alleviate this issue, we present a new approach to combat noisy samples. First, we propose a memorized noise detection method to detect noisy samples that DNNs have already memorized during the training process. Next, we design a noise-excluded sample selection method and a noise-alleviated MixMatch to alleviate the memorization of DNNs to noisy samples. Finally, we integrate our approach with the established method DivideMix, proposing Modified-DivideMix. The experimental results on CIFAR-10, CIFAR-100, and Clothing1M demonstrate the effectiveness of our approach
- 24 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 597-609
Combating Noisy Labels by Alleviating the Memorization of DNNs to Noisy Labels
Data is the essential fuel for deep neural networks (DNNs), and its quality affects the practical performance of DNNs. In real-world training scenarios, the successful generalization performance of DNNs is severely challenged by noisy samples with incorrect labels. To combat noisy samples in image classification, numerous methods based on sample selection and semi-supervised learning (SSL) have been developed, where sample selection is used to provide the supervision signal for SSL, achieving great success in resisting noisy samples. Due to the necessary warm-up training on noisy datasets and the basic sample selection mechanism, DNNs are still confronted with the challenge of memorizing noisy samples. However, existing methods do not address the memorization of noisy samples by DNNs explicitly, which hinders the generalization performance of DNNs. To alleviate this issue, we present a new approach to combat noisy samples. First, we propose a memorized noise detection method to detect noisy samples that DNNs have already memorized during the training process. Next, we design a noise-excluded sample selection method and a noise-alleviated MixMatch to alleviate the memorization of DNNs to noisy samples. Finally, we integrate our approach with the established method DivideMix, proposing Modified-DivideMix. The experimental results on CIFAR-10, CIFAR-100, and Clothing1M demonstrate the effectiveness of our approach
- 1 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (5) , 9098-9111
DefendFL: A Privacy-Preserving Federated Learning Scheme Against Poisoning Attacks
Federated learning (FL) has become a popular mode of learning, allowing model training without the need to share data. Unfortunately, it remains vulnerable to privacy leakage and poisoning attacks, which compromise user data security and degrade model quality. Therefore, numerous privacy-preserving frameworks have been proposed, among which mask-based framework has certain advantages in terms of efficiency and functionality. However, it is more susceptible to poisoning attacks from malicious users, and current works lack practical means to detect such attacks within this framework. To overcome this challenge, we present DefendFL, an efficient, privacy-preserving, and poisoning-detectable mask-based FL scheme. We first leverage collinearity mask to protect users’ gradient privacy. Then, cosine similarity is utilized to detect masked gradients to identify poisonous gradients. Meanwhile, a verification mechanism is designed to detect the mask, ensuring the mask’s validity in aggregation and preventing poisoning attacks by intentionally changing the mask. Finally, we resist poisoning attacks by removing malicious gradients or lowering their weights in aggregation. Through security analysis and experimental evaluation, DefendFL can effectively detect and mitigate poisoning attacks while outperforming existing privacy-preserving detection works in efficiency.
- 1 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (5) , 9098-9111
DefendFL: A Privacy-Preserving Federated Learning Scheme Against Poisoning Attacks
Federated learning (FL) has become a popular mode of learning, allowing model training without the need to share data. Unfortunately, it remains vulnerable to privacy leakage and poisoning attacks, which compromise user data security and degrade model quality. Therefore, numerous privacy-preserving frameworks have been proposed, among which mask-based framework has certain advantages in terms of efficiency and functionality. However, it is more susceptible to poisoning attacks from malicious users, and current works lack practical means to detect such attacks within this framework. To overcome this challenge, we present DefendFL, an efficient, privacy-preserving, and poisoning-detectable mask-based FL scheme. We first leverage collinearity mask to protect users’ gradient privacy. Then, cosine similarity is utilized to detect masked gradients to identify poisonous gradients. Meanwhile, a verification mechanism is designed to detect the mask, ensuring the mask’s validity in aggregation and preventing poisoning attacks by intentionally changing the mask. Finally, we resist poisoning attacks by removing malicious gradients or lowering their weights in aggregation. Through security analysis and experimental evaluation, DefendFL can effectively detect and mitigate poisoning attacks while outperforming existing privacy-preserving detection works in efficiency.
- 25 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 19 (15566013) , 8140-8154
FDFL: Fair and Discrepancy-Aware Incentive Mechanism for Federated Learning
Federated Learning (FL) is an emerging distributed machine learning paradigm crucial for ensuring privacy-preserving learning. In FL, a fair incentive mechanism is indispensable for inspiring more clients to participate in FL training. Nevertheless, achieving a fair incentive mechanism in FL is an arduous endeavor, underscored by two significant challenges that persistently elude resolution within existing methodologies. Firstly, existing works overlook the issue of category distribution heterogeneity in contribution evaluation, leading to incomplete contribution evaluations. Secondly, the fact that malicious servers will dishonestly allocate rewards to save costs is not considered in existing work, which can be a barrier to client participation in FL. This paper introduces FDFL (Fair and Discrepancy-aware incentive mechanism for Federated Learning), a novel system addressing these concerns. FDFL encompasses two key elements: 1) Discrepancy-aware contribution evaluation approach; 2) Provable reward allocation approach. Extensive experiments on four model-dataset combinations demonstrate that, under the heterogeneous setting, our scheme improves accuracy by an average of 9.85% and 11.97% compared to FedAvg and FAIR, respectively.
- 25 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 19 (15566013) , 8140-8154
FDFL: Fair and Discrepancy-Aware Incentive Mechanism for Federated Learning
Federated Learning (FL) is an emerging distributed machine learning paradigm crucial for ensuring privacy-preserving learning. In FL, a fair incentive mechanism is indispensable for inspiring more clients to participate in FL training. Nevertheless, achieving a fair incentive mechanism in FL is an arduous endeavor, underscored by two significant challenges that persistently elude resolution within existing methodologies. Firstly, existing works overlook the issue of category distribution heterogeneity in contribution evaluation, leading to incomplete contribution evaluations. Secondly, the fact that malicious servers will dishonestly allocate rewards to save costs is not considered in existing work, which can be a barrier to client participation in FL. This paper introduces FDFL (Fair and Discrepancy-aware incentive mechanism for Federated Learning), a novel system addressing these concerns. FDFL encompasses two key elements: 1) Discrepancy-aware contribution evaluation approach; 2) Provable reward allocation approach. Extensive experiments on four model-dataset combinations demonstrate that, under the heterogeneous setting, our scheme improves accuracy by an average of 9.85% and 11.97% compared to FedAvg and FAIR, respectively.
- 28 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 36 (11) , 5661-5674
BADFL: Backdoor Attack Defense in Federated Learning From Local Model Perspective
There is substantial attention to federated learning with its ability to train a powerful global model collaboratively while protecting data privacy. Despite its many advantages, federated learning is vulnerable to backdoor attacks, where an adversary injects malicious weights into the global model, making the global model's targeted predictions incorrect. Existing defenses based on identifying and eliminating malicious weights ignore the similarity variation of the local weights during iterations in the malicious model detection and the presence of benign weights in the malicious model during the malicious local weight elimination, resulting in a poor defense and a degradation of global model accuracy. In this paper, we defend against backdoor attacks from the perspective of local models. First, a malicious model detection method based on interpretability techniques is proposed. The method appends a sampling check after clustering to identify malicious models accurately. We further design a malicious local weight elimination method based on local weight contributions. This method preserves the benign weights in the malicious model to maintain their contributions to the global model. Finally, we analyze the security of the proposed method in terms of model closeness and then verify the effectiveness of the proposed method through experiments. In comparison with existing defenses, the results show that BADFL improves the global model accuracy by 23.14% while reducing the attack success rate to 0.04% in the best case.
- 28 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 36 (11) , 5661-5674
BADFL: Backdoor Attack Defense in Federated Learning From Local Model Perspective
There is substantial attention to federated learning with its ability to train a powerful global model collaboratively while protecting data privacy. Despite its many advantages, federated learning is vulnerable to backdoor attacks, where an adversary injects malicious weights into the global model, making the global model's targeted predictions incorrect. Existing defenses based on identifying and eliminating malicious weights ignore the similarity variation of the local weights during iterations in the malicious model detection and the presence of benign weights in the malicious model during the malicious local weight elimination, resulting in a poor defense and a degradation of global model accuracy. In this paper, we defend against backdoor attacks from the perspective of local models. First, a malicious model detection method based on interpretability techniques is proposed. The method appends a sampling check after clustering to identify malicious models accurately. We further design a malicious local weight elimination method based on local weight contributions. This method preserves the benign weights in the malicious model to maintain their contributions to the global model. Finally, we analyze the security of the proposed method in terms of model closeness and then verify the effectiveness of the proposed method through experiments. In comparison with existing defenses, the results show that BADFL improves the global model accuracy by 23.14% while reducing the attack success rate to 0.04% in the best case.
- 19 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (10) , 2405-2419
Hiding in Plain Sight: Adversarial Attack via Style Transfer on Image Borders
Deep Convolution Neural Networks (CNNs) have become the cornerstone of image classification, but the emergence of adversarial image attacks brings serious security risks to CNN-based applications. As a local perturbation attack, the border attack can achieve high success rates by only modifying the pixels around the border of an image, which is a novel attack perspective. However, existing border attacks have shortcomings in stealthiness and are easily detected. In this paper, we propose a novel stealthy border attack method based on deep feature alignment. Specifically, we propose a deep feature alignment algorithm based on style transfer to guarantee the stealthiness of adversarial borders. The algorithm takes the deep feature difference between the adversarial and the original borders as the stealthiness loss and thus ensures good stealthiness of the generated adversarial images. To ensure high attack success rates simultaneously, we apply cross entropy to design the targeted attack loss and use margin loss as well as Leaky ReLU to design the untargeted attack loss. Experiments show that the structural similarity between the generated adversarial images and the original images is 8.8% higher than the state-of-art border attack method, indicating that our proposed adversarial images have better stealthiness. At the same time, the success rate of our attack in the face of defense methods is much higher, which is about four times that of the state-of-art border attack under the adversarial training defense.
- 19 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computers
- Vol. 73 (10) , 2405-2419
Hiding in Plain Sight: Adversarial Attack via Style Transfer on Image Borders
Deep Convolution Neural Networks (CNNs) have become the cornerstone of image classification, but the emergence of adversarial image attacks brings serious security risks to CNN-based applications. As a local perturbation attack, the border attack can achieve high success rates by only modifying the pixels around the border of an image, which is a novel attack perspective. However, existing border attacks have shortcomings in stealthiness and are easily detected. In this paper, we propose a novel stealthy border attack method based on deep feature alignment. Specifically, we propose a deep feature alignment algorithm based on style transfer to guarantee the stealthiness of adversarial borders. The algorithm takes the deep feature difference between the adversarial and the original borders as the stealthiness loss and thus ensures good stealthiness of the generated adversarial images. To ensure high attack success rates simultaneously, we apply cross entropy to design the targeted attack loss and use margin loss as well as Leaky ReLU to design the untargeted attack loss. Experiments show that the structural similarity between the generated adversarial images and the original images is 8.8% higher than the state-of-art border attack method, indicating that our proposed adversarial images have better stealthiness. At the same time, the success rate of our attack in the face of defense methods is much higher, which is about four times that of the state-of-art border attack under the adversarial training defense.