Haoyun Sun

Effective defect recognition is essential for improving quality and stability in smart manufacturing, often necessitating collaborative learning across multiple organizations. Federated learning (FL) is promising in this regard. However, FL struggles with client-drift problem in nonindependent and identically distributed (non-IID) data. To this end, this article introduces a novel federated learning with diffusion generation (FL-DiG), which tackles the problem from the perspective of data generation. Specifically, a distribution conditional diffusion model (DCDM) is proposed to learn distribution characteristics of the raw data and generate virtual data with this distribution. Then, a data generation-based alignment fusion (DGAF) strategy is designed to use DCDMs to generate a batch of virtual data that conforms to the distribution of all raw data. This virtual data is then used to train both client models and the global FL model, thereby alleviating the client-drift problem for non-IID data. Experimental validation across two defect datasets demonstrates that our method outperforms other state-of-the-art approaches such as FedAVG, FedProx, FedBN, FedNova, SCAFFOLD, and FedDisco, achieving up to a 16.79% accuracy improvement over the second-best method in severely non-IID scenarios. Codes are available at https://github.com/hysshy/FL-DiG.

IoT devices have been widely deployed in medical industry, in the objective of improving diagnostic accuracy and increasing the efficiency of healthcare systems. However, traditional centralized learning approaches often fall short in meeting strict privacy requirements and adapting to emerging diseases in clinical environment. To address this, we propose a novel federated continual learning (CL) framework for disease diagnosis (FCL4DD), designed to enable distributed and incremental learning of new disease classes while safeguarding data privacy. To combat catastrophic forgetting in CL, FCL4DD integrates a replay strategy powered by a weakly supervised diffusion model (WSDM) to generate historical data for diagnosis model training. The WSDM leverages weak supervision into diffusion model to capture the diverse characteristics of the real data, enabling the generation of high-quality synthetic samples that maintain the data’s inherent variability. To overcome the challenges of nonindependent and identically distributed (non-IID) data in federated learning, WSDM is deployed at the central server to generate synthetic disease data that conforms to the global distribution. This synthetic data is then used to retrain client models, reducing discrepancies and enhancing performance consistency across clients. Evaluations on various datasets demonstrates that our method outperforms other state-of-the-art approaches, such as FedEWC, FedLwF, FedWeIT, TARGET, and DDDR, achieving up to a 4.85% accuracy improvement over the second-best method. Code are available at https://github.com/hysshy/FCL4DD.

Multiple organizations in social manufacturing can collaborate on high-quality product defect detection with social networks. Federated learning (FL) is an emerging paradigm where multiple clients can collaboratively train a defect detection model in a privacy-preserving manner. A prevalent issue in FL, concept drift, is discussed in this article. Feature representations of the same label may vary at different clients which affects the performance of FL. To address this issue, a novel community aware personalized federated learning (CA-PFL) is proposed in this article. A graph structured federation social network is constructed with local model updates. Communities in federation network are discovered with community detection to ensure that the same label at different clients have similar representations in each community. Shared layers of local models are aggregated in each community and each local client keeps their personalized layers. Furthermore, a federation community contrastive loss (FedCCL) is proposed to accelerate training convergence by constraining the direction of local model updating. Experimental results on nine datasets demonstrate that CA-PFL achieves higher accuracy and faster convergence than state-of- the-art personalized federated learning methods in concept drifts scenarios.

Multi-pedestrian attribute recognition (Multi-PAR) is a vital task for smart city surveillance applications, which requires identifying various attributes of multiple pedestrians in a single image. However, most existing methods are limited by the complex backgrounds and the time-consuming pedestrian detection preprocessing work in real-world scenarios, and cannot achieve satisfactory accuracy and efficiency. In this paper, we present a novel end-to-end solution, named Adaptive Multi-Task Network (AMTN), which jointly performs multiple tasks and leverages an adaptive feature re-extraction (AFRE) module to optimize them. Specially, We integrate pedestrian detection into AMTN to perform PAR preprocessing, and incorporate a person re-identification (ReID) task branch to track pedestrians in video streams, thereby selecting the clearest video frames for analysis instead of every video frame to improve analysis efficiency and recognition accuracy. Moreover, we design a dynamic weight fitting loss (DWFL) function to prevent gradient explosions and balance tasks during training. We conduct extensive experiments to evaluate the accuracy and efficiency of our approach, and compare it with the state-of-the-art methods. The experimental results demonstrate that our method outperforms other state-of-the-art algorithms, achieving 1.5%-4.9% improvement in accuracy on Multi-PAR. The experiments also show that the AMTN can greatly improve the efficiency of preprocessing by saving the computation of feature extraction through basic features sharing. Compared with the state-of-the-art detection algorithm Yolov5s, it can improve the efficiency by 42%.

The proliferation of privacy-sensitive data has spurred the development of federated learning (FL), which is an important technology for state-of-the-art machine learning and responsible AI. However, most existing FL methods are constrained in their applicability and generalizability due to their narrow focus on specific tasks. This paper presents a novel federated multi-task learning (FMTL) framework that is capable of acquiring knowledge across multiple tasks. To address the challenges posed by Non-IID data and task imbalance in FMTL, this study proposes a federated fusion strategy based on dynamic guided attention (FedDGA), which adaptively fine-tunes local models for multiple tasks with personalized attention. In addition, this paper designed dynamic batch weight (DBW) to balance the task losses and improve the convergence speed. Extensive experiments were conducted on various datasets, tasks and settings, and the proposed method was compared with state-of-the-art methods such as FedAvg, FedProx and SCAFFOLD. The results show that our method achieves significant performance gains, with up to 11.1% increase in accuracy over the baselines.

Video data is the biggest IoT data which is challenging for effective analysis with good performance. Object misdetection is usually inevitable in edge-based distributed cross-scene video analysis. Traditional centralized model training can potentially result in edge data leakage. Even though joint model can be trained with federated learning while maintaining data privacy, the size of gradient data transmitted is large for computer vision models used. To address these problems, this article proposed an accurate and efficient federated learning-based edge intelligence for effective video analysis method called EIEVA-AEFL. In EIEVA-AEFL, a federation misdetection reinforcement network (FMRN) is designed to alleviate the misdetection problem. FMRN contains a vanilla object detection network and a misdetection reinforcement branch, which finetunes object detection via feature re-extraction to reduce object misdetection. To reduce the communication cost in training, an efficient federated learning strategy is designed. In this strategy, an oscillation suppression loss function is proposed to suppress the loss fluctuation resulting from data on edge clients. Average accuracy and recall increase 0.5 and 0.7 with FMRN on the Microsoft common objects in context (MS COCO) data set, respectively, and with improvements of 4.5 and 5.5 with FMRN on our self-made mis-detection data set, respectively. EIEVA-AEFL can reduce the training speed on the premise of ensuring the accuracy of the model. The model parameters, data amount, transmission delay, and convergence epochs on EIEVA-AEFL model training are reduced by 78%, 89%, 84%, and 36%, respectively.

Ship detection and recognition are important for smart monitoring of ships in order to manage port resources effectively. However, this is challenging due to complex ship profiles, ship background, object occlusion, variations of weather and light conditions, and other issues. It is also expensive to transmit monitoring video in a whole, especially if the port is not in a rural area. In this paper, we propose an on-site processing approach, which is called Embedded Ship Detection and Recognition using Deep Learning (ESDR-DL). In ESDR-DL, the video stream is processed using embedded devices, and we design a two-stage neural network named DCNet, which is composed of a DNet for ship detection and a CNet for ship recognition, running on embedded devices. We have extensively evaluated ESDR-DL, including performance of accuracy and efficiency. The ESDR-DL is deployed at the Dongying port of China, which has been running for over a year and demonstrates that it can work reliably for practical usage.