Jiwei Shen
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jiwei Shen has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Yue Lu
East China Normal University
8 shared publications - Shujing Lyu
Shanghai Key Laboratory of Multidimensional Information Processing, School of Communication and Electronic Engineering, East China Normal University, Shanghai, China
7 shared publications - Liang Yuan
USC-SJTU Institute of Cultural and Creative Industry, Shanghai Jiao Tong University, Shanghai, China
2 shared publications - Botong Zhao
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China
1 shared publication - Fang Yu1 shared publication
- Hongjian Zhan
East China Normal University
1 shared publication
Latest Publications
- 2 January 2025
- Published by Elsevier in Expert Systems with Applications
- Vol. 268
MAED: Mask assignment encoder decoder solver for multiple patterning layout decomposition
- 2 January 2025
- Published by Elsevier in Expert Systems with Applications
- Vol. 268
MAED: Mask assignment encoder decoder solver for multiple patterning layout decomposition
- 30 November 2024
- Published by Springer Nature
- p. 94-109
Learning to Detect Lithography Defects in SEM Images
- 30 November 2024
- Published by Springer Nature
- p. 94-109
Learning to Detect Lithography Defects in SEM Images
- 4 November 2024
- Published by Elsevier in Expert Systems with Applications
- Vol. 262
GCENet: A geometric correspondence estimation network for tracking and loop detection in visual–inertial SLAM
- 4 November 2024
- Published by Elsevier in Expert Systems with Applications
- Vol. 262
GCENet: A geometric correspondence estimation network for tracking and loop detection in visual–inertial SLAM
- 30 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (10) , 9298-9310
LithoPW: Leveraging Visual Memory Encoding and Defect-Aware Optimization for Precise Determination of the Lithography Process Windows
Lithography stands as a critical step in the manufacturing of integrated circuits, where the precise control of focus and exposure dose parameters is vital for optimal results. The conventional methodologies for defining lithography process windows often face difficulties with managing measurement errors, detecting printed defects, and exploiting visual features from Scanning Electron Microscope (SEM) images. This paper proposes LithoPW, a novel framework that utilizes visual features of SEM images for the determination of process windows. This approach is comprised of a denoising module, a Transformer-based visual memory encoder, and a defect-aware process window optimization module. The denoising module incorporates a Transformer architecture to mitigate the impact of noise, thereby enhancing the efficiency of downstream tasks in leveraging information embedded within SEM images. The transformer-based visual memory encoder discerns each SEM image as a Query, maintaining neighbouring SEM images in memory as Key and Value elements, thereby facilitating precise lithography quality classification associated with the query image. The defect-aware process window optimization module heightens the reliability of the results by adjusting the process window according to the defects identified within the SEM images. Experimental results confirm the efficacy of our framework, highlighting its promising application in lithography production for accurate process window determination.
- 30 April 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems for Video Technology
- Vol. 34 (10) , 9298-9310
LithoPW: Leveraging Visual Memory Encoding and Defect-Aware Optimization for Precise Determination of the Lithography Process Windows
Lithography stands as a critical step in the manufacturing of integrated circuits, where the precise control of focus and exposure dose parameters is vital for optimal results. The conventional methodologies for defining lithography process windows often face difficulties with managing measurement errors, detecting printed defects, and exploiting visual features from Scanning Electron Microscope (SEM) images. This paper proposes LithoPW, a novel framework that utilizes visual features of SEM images for the determination of process windows. This approach is comprised of a denoising module, a Transformer-based visual memory encoder, and a defect-aware process window optimization module. The denoising module incorporates a Transformer architecture to mitigate the impact of noise, thereby enhancing the efficiency of downstream tasks in leveraging information embedded within SEM images. The transformer-based visual memory encoder discerns each SEM image as a Query, maintaining neighbouring SEM images in memory as Key and Value elements, thereby facilitating precise lithography quality classification associated with the query image. The defect-aware process window optimization module heightens the reliability of the results by adjusting the process window according to the defects identified within the SEM images. Experimental results confirm the efficacy of our framework, highlighting its promising application in lithography production for accurate process window determination.
- 2 February 2024
- Published by Elsevier in Expert Systems with Applications
- Vol. 248
Enhancing parcel singulation efficiency through transformer-based position attention and state space augmentation
- 2 February 2024
- Published by Elsevier in Expert Systems with Applications
- Vol. 248
Enhancing parcel singulation efficiency through transformer-based position attention and state space augmentation
- 1 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (1) , 704-717
Leveraging Predictions of Task-Related Latents for Interactive Visual Navigation
Interactive visual navigation (IVN) involves tasks where embodied agents learn to interact with the objects in the environment to reach the goals. Current approaches exploit visual features to train a reinforcement learning (RL) navigation control policy network. However, RL-based methods continue to struggle at the IVN tasks as they are inefficient in learning a good representation of the unknown environment in partially observable settings. In this work, we introduce predictions of task-related latents (PTRLs), a flexible self-supervised RL framework for IVN tasks. PTRL learns the latent structured information about environment dynamics and leverages multistep representations of the sequential observations. Specifically, PTRL trains its representation by explicitly predicting the next pose of the agent conditioned on the actions. Moreover, an attention and memory module is employed to associate the learned representation to each action and exploit spatiotemporal dependencies. Furthermore, a state value boost module is introduced to adapt the model to previously unseen environments by leveraging input perturbations and regularizing the value function. Sample efficiency in the training of RL networks is enhanced by modular training and hierarchical decomposition. Extensive evaluations have proved the superiority of the proposed method in increasing the accuracy and generalization capacity.
- 1 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (1) , 704-717
Leveraging Predictions of Task-Related Latents for Interactive Visual Navigation
Interactive visual navigation (IVN) involves tasks where embodied agents learn to interact with the objects in the environment to reach the goals. Current approaches exploit visual features to train a reinforcement learning (RL) navigation control policy network. However, RL-based methods continue to struggle at the IVN tasks as they are inefficient in learning a good representation of the unknown environment in partially observable settings. In this work, we introduce predictions of task-related latents (PTRLs), a flexible self-supervised RL framework for IVN tasks. PTRL learns the latent structured information about environment dynamics and leverages multistep representations of the sequential observations. Specifically, PTRL trains its representation by explicitly predicting the next pose of the agent conditioned on the actions. Moreover, an attention and memory module is employed to associate the learned representation to each action and exploit spatiotemporal dependencies. Furthermore, a state value boost module is introduced to adapt the model to previously unseen environments by leveraging input perturbations and regularizing the value function. Sample efficiency in the training of RL networks is enhanced by modular training and hierarchical decomposition. Extensive evaluations have proved the superiority of the proposed method in increasing the accuracy and generalization capacity.
- 15 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (1) , 643-650
VME-Transformer: Enhancing Visual Memory Encoding for Navigation in Interactive Environments
The efficiency of a robotic system is primarily determined by its ability to navigate complex and interactive environments. In real-world scenarios, cluttered surroundings are common, requiring a robot to navigate diverse spaces and displace objects to pave a path towards its objective. Consequently, “Visual Interactive Navigation” presents several challenges, including how to retain historical exploration information from partially observable visual signals, and how to utilize sparse rewards in reinforcement learning to simultaneously learn a latent representation and a control policy. Addressing these challenges, we introduce a Transformer-based Visual Memory Encoder (VME-Transformer), capable of embedding both recent and long-term exploration information into memory. Additionally, we explicitly estimate the robot's next pose, conditioned on the impending action, to bootstrap the learning process of the high-capacity VME-Transformer. We further regularize the value function by introducing input perturbations, thereby enhancing its generalization capabilities in previously unseen environments. In the Visual Interactive Navigation tasks within the iGibson environment, the VME-Transformer demonstrates superior performance compared to state-of-the-art methods, underlining its effectiveness.
- 15 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (1) , 643-650
VME-Transformer: Enhancing Visual Memory Encoding for Navigation in Interactive Environments
The efficiency of a robotic system is primarily determined by its ability to navigate complex and interactive environments. In real-world scenarios, cluttered surroundings are common, requiring a robot to navigate diverse spaces and displace objects to pave a path towards its objective. Consequently, “Visual Interactive Navigation” presents several challenges, including how to retain historical exploration information from partially observable visual signals, and how to utilize sparse rewards in reinforcement learning to simultaneously learn a latent representation and a control policy. Addressing these challenges, we introduce a Transformer-based Visual Memory Encoder (VME-Transformer), capable of embedding both recent and long-term exploration information into memory. Additionally, we explicitly estimate the robot's next pose, conditioned on the impending action, to bootstrap the learning process of the high-capacity VME-Transformer. We further regularize the value function by introducing input perturbations, thereby enhancing its generalization capabilities in previously unseen environments. In the Visual Interactive Navigation tasks within the iGibson environment, the VME-Transformer demonstrates superior performance compared to state-of-the-art methods, underlining its effectiveness.
- 2 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (3) , 1731-1738
Transformer Memory for Interactive Visual Navigation in Cluttered Environments
Substantial progress has been achieved in embodied visual navigation based on reinforcement learning (RL). These studies presume that the environment is stationary where all the obstacles are static. However, in real cluttered scenes, interactable objects (e.g. shoes and boxes) blocking the way of robots makes the environment non-stationary. Accordingly, the ego-centric visual agent will easily get stuck in the dilemma of finding the next waypoint as it struggles to decide whether to push the obstacles ahead. To handle the predicament, we formulate this interactive visual navigation as Partial Observed Markov Decision Process (POMDP). As the transformer encoder has demonstrated its superior ability to capture the spatial-temporal dependencies in natural language processing. We propose a transformer-based memory to empower the agents utilizing the historical interactive information. However, directly leveraging the transformer architecture in the RL settings is highly unstable. We further propose a surrogate objective to predict the next waypoint as the auxiliary task, which facilitates the representation learning and bootstraps the RL. We demonstrate our method in the iGibson environment and experimental results show a significant improvement over the interactive Gibson benchmark and the related recurrent RL policy both in the validation seen scenes and the test unseen scenes.
- 2 February 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (3) , 1731-1738
Transformer Memory for Interactive Visual Navigation in Cluttered Environments
Substantial progress has been achieved in embodied visual navigation based on reinforcement learning (RL). These studies presume that the environment is stationary where all the obstacles are static. However, in real cluttered scenes, interactable objects (e.g. shoes and boxes) blocking the way of robots makes the environment non-stationary. Accordingly, the ego-centric visual agent will easily get stuck in the dilemma of finding the next waypoint as it struggles to decide whether to push the obstacles ahead. To handle the predicament, we formulate this interactive visual navigation as Partial Observed Markov Decision Process (POMDP). As the transformer encoder has demonstrated its superior ability to capture the spatial-temporal dependencies in natural language processing. We propose a transformer-based memory to empower the agents utilizing the historical interactive information. However, directly leveraging the transformer architecture in the RL settings is highly unstable. We further propose a surrogate objective to predict the next waypoint as the auxiliary task, which facilitates the representation learning and bootstraps the RL. We demonstrate our method in the iGibson environment and experimental results show a significant improvement over the interactive Gibson benchmark and the related recurrent RL policy both in the validation seen scenes and the test unseen scenes.