Mingjun Jiang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Mingjun Jiang has a total of 30 co-authors. Here are the top co-authors by number of shared publications:
- Idaku Ishii
Hiroshima University
12 shared publications - Takeshi Takaki
Hiroshima University
7 shared publications - Kohei Shimasaki
Hiroshima University
6 shared publications - Shaopeng Hu
Hiroshima University
5 shared publications - Taku Senoo
Graduate School of Information Science and Technology, Hokkaido University, Kita 14, Nishi 9, Kita-ku, Sapporo, Hokkaido 060-0814, Japan
4 shared publications - Chikako Umemoto
Keisoku Res Consultant Co, Higashi Ku, 1-665-1 Fukuda, Hiroshima, Hiroshima 7320029, Japan
2 shared publications
Latest Publications
- 11 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 4086-4093
Omnidirectional Panoramic Video System With Frame-by-Frame Ultrafast Viewpoint Control
This study develops an omnidirectional panoramic video system that can simultaneously observe zoom-in images at dozens of different viewpoints in a 360-degree field of view by synchronizing hundreds of frames per second video shooting, with a rotation of tens-of-rotation-per-second mirror-rotator. When video shooting a scene with a certain exposure utilizing a mirror-rotator, apparent motion in the captured images becomes too large to observe non-blurred images, owing to its high-speed rotation. To reduce motion blurs caused by the rotation of a mirror-rotator, our system introduces an ultrafast pan-tilt mirror mechanism that can cancel apparent motion in images by executing frame-by-frame viewpoint control, so that the apparent speed in images becomes zero when the camera shutter is open, and it returns to its home position when the shutter is closed. Our system can capture 40 omnidirectionally different view images at 10 fps in real time by synchronizing 400-fps video shooting with frame-by-frame ultrafast pan-tilt mirror control for motion blur reduction when a mirror-rotator rotates at 10 rps. Its performance is verified by experimental results for several omnidirectional real scenes, and our system can capture clear and bright panoramic zoom-in images without decreasing the camera exposure when omnidirectionally observing dozens-of-meters-distance outdoor scenes.
- 11 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 4086-4093
Omnidirectional Panoramic Video System With Frame-by-Frame Ultrafast Viewpoint Control
This study develops an omnidirectional panoramic video system that can simultaneously observe zoom-in images at dozens of different viewpoints in a 360-degree field of view by synchronizing hundreds of frames per second video shooting, with a rotation of tens-of-rotation-per-second mirror-rotator. When video shooting a scene with a certain exposure utilizing a mirror-rotator, apparent motion in the captured images becomes too large to observe non-blurred images, owing to its high-speed rotation. To reduce motion blurs caused by the rotation of a mirror-rotator, our system introduces an ultrafast pan-tilt mirror mechanism that can cancel apparent motion in images by executing frame-by-frame viewpoint control, so that the apparent speed in images becomes zero when the camera shutter is open, and it returns to its home position when the shutter is closed. Our system can capture 40 omnidirectionally different view images at 10 fps in real time by synchronizing 400-fps video shooting with frame-by-frame ultrafast pan-tilt mirror control for motion blur reduction when a mirror-rotator rotates at 10 rps. Its performance is verified by experimental results for several omnidirectional real scenes, and our system can capture clear and bright panoramic zoom-in images without decreasing the camera exposure when omnidirectionally observing dozens-of-meters-distance outdoor scenes.
- 8 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 71 (00189456) , 1-13
View and Scanning-Depth Expansion Photographic Microscope Using Ultrafast Switching Mirrors
This paper presents a novel view and scanning-depth expansion photographic microscope using ultrafast switching mirrors and high-speed vision that can simultaneously extend both the field of view and the scanning-depth range of a typical optical microscope in real time. Two ultrafast switching mirrors that can switch at several hundred hertz are used for micro-gaze control via synchronization through a high-speed vision system. The field of view can be changed while maintaining a constant length of the optical path by adjusting the angles of the two mirrors, which ensures that the extended field of view is within the focal range. We evaluated and designed mirror angle combinations for view expansion at multiple depths, making it possible to extend view and depth in real time without considering the complex design and computing required by computational imaging micro view expansion approaches. The effectiveness of our system was demonstrated by several experiments, including micro view expansion at a single depth and multiple depths in real time. In addition, by evaluating the image quality of the view expansion image, our proposed system can automatically output always-focused view expansion images in real time even when samples move at different depths.
- 8 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 71 (00189456) , 1-13
View and Scanning-Depth Expansion Photographic Microscope Using Ultrafast Switching Mirrors
This paper presents a novel view and scanning-depth expansion photographic microscope using ultrafast switching mirrors and high-speed vision that can simultaneously extend both the field of view and the scanning-depth range of a typical optical microscope in real time. Two ultrafast switching mirrors that can switch at several hundred hertz are used for micro-gaze control via synchronization through a high-speed vision system. The field of view can be changed while maintaining a constant length of the optical path by adjusting the angles of the two mirrors, which ensures that the extended field of view is within the focal range. We evaluated and designed mirror angle combinations for view expansion at multiple depths, making it possible to extend view and depth in real time without considering the complex design and computing required by computational imaging micro view expansion approaches. The effectiveness of our system was demonstrated by several experiments, including micro view expansion at a single depth and multiple depths in real time. In addition, by evaluating the image quality of the view expansion image, our proposed system can automatically output always-focused view expansion images in real time even when samples move at different depths.
- 25 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 21 (7) , 9436-9448
A Simultaneous Multi-Object Zooming System Using an Ultrafast Pan-Tilt Camera
This paper presents a novel dual-camera system that can simultaneously capture zoomed-in images using an ultrafast pan-tilt camera and a fixed wide-view camera using deep learning methods. An ultrafast pan-tilt camera can function as multiple virtual pan-tilt cameras by synchronizing a high-frame-rate zooming-view camera and an ultrafast pan-tilt mirror device that can switch over 500 different views in a second. A wide-view camera can obtain images in a fixed view in which multiple targets to be tracked are captured in low resolution and then recognized by processing the images using deep learning methods at a rate of dozens of frames per second. Based on the positions of all targets, recognized by the wide-view camera, the pan and tilt angles of multiple pan-tilt cameras are virtually controlled using an ultrafast pan-tilt camera through multithread viewpoint control to simultaneously capture the zoomed-in images of all targets. Our developed system can operate 10 virtual pan-tilt cameras (25 fps) with multithread viewpoint control and 4 ms time granularity in synchronization with convolutional neural-network-based recognition model operating at 25 fps, which is accelerated by a general-purpose computing on graphics processing units. The effectiveness of our system was demonstrated by the results of several experiments conducted on simultaneous zoom shooting of multiple running objects (persons and cars) in the range of approximately 80 m or higher in a natural outdoor scene, which was formerly too wide for a single fixed camera to capture clearly and simultaneously.
- 25 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 21 (7) , 9436-9448
A Simultaneous Multi-Object Zooming System Using an Ultrafast Pan-Tilt Camera
This paper presents a novel dual-camera system that can simultaneously capture zoomed-in images using an ultrafast pan-tilt camera and a fixed wide-view camera using deep learning methods. An ultrafast pan-tilt camera can function as multiple virtual pan-tilt cameras by synchronizing a high-frame-rate zooming-view camera and an ultrafast pan-tilt mirror device that can switch over 500 different views in a second. A wide-view camera can obtain images in a fixed view in which multiple targets to be tracked are captured in low resolution and then recognized by processing the images using deep learning methods at a rate of dozens of frames per second. Based on the positions of all targets, recognized by the wide-view camera, the pan and tilt angles of multiple pan-tilt cameras are virtually controlled using an ultrafast pan-tilt camera through multithread viewpoint control to simultaneously capture the zoomed-in images of all targets. Our developed system can operate 10 virtual pan-tilt cameras (25 fps) with multithread viewpoint control and 4 ms time granularity in synchronization with convolutional neural-network-based recognition model operating at 25 fps, which is accelerated by a general-purpose computing on graphics processing units. The effectiveness of our system was demonstrated by the results of several experiments conducted on simultaneous zoom shooting of multiple running objects (persons and cars) in the range of approximately 80 m or higher in a natural outdoor scene, which was formerly too wide for a single fixed camera to capture clearly and simultaneously.
- 25 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 70 (00189456) , 1-11
500-Fps Omnidirectional Visual Tracking Using Three-Axis Active Vision System
This study proposes a visual tracking system with 500-fps image processing that can detect and track a fast-moving appearance-varying target in a 360-degree-field of view (FOV) from high-spatial-resolution images. The system consists of a high-speed camera, a pan-tilt Galvano-mirror system, and a mirror-drive high-speed pan control system with 360-degree-rotational capacity. Omnidirectional visual tracking of a remote object using complex visual features becomes possible due to the acquisition of high-spatial-resolution images of the target object via the system. We introduce an intelligent high frame rate (HFR) target-tracking algorithm that can simultaneously detect and track an appearance-varying object against a complex background for high-speed visual feedback control. Furthermore, the proposed surveillance system enables robust visual tracking under low-light conditions using a 3-D light detection and ranging (LiDAR) sensor system and a simultaneously controlled active spotlight projection system. The effectiveness of the proposed omnidirectional visual tracking system was demonstrated by conducting several omnidirectional detecting and tracking experiments on persons dozens of meters away using a 25-mm focal length lens under both daytime and night-time conditions.
- 25 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Instrumentation and Measurement
- Vol. 70 (00189456) , 1-11
500-Fps Omnidirectional Visual Tracking Using Three-Axis Active Vision System
This study proposes a visual tracking system with 500-fps image processing that can detect and track a fast-moving appearance-varying target in a 360-degree-field of view (FOV) from high-spatial-resolution images. The system consists of a high-speed camera, a pan-tilt Galvano-mirror system, and a mirror-drive high-speed pan control system with 360-degree-rotational capacity. Omnidirectional visual tracking of a remote object using complex visual features becomes possible due to the acquisition of high-spatial-resolution images of the target object via the system. We introduce an intelligent high frame rate (HFR) target-tracking algorithm that can simultaneously detect and track an appearance-varying object against a complex background for high-speed visual feedback control. Furthermore, the proposed surveillance system enables robust visual tracking under low-light conditions using a 3-D light detection and ranging (LiDAR) sensor system and a simultaneously controlled active spotlight projection system. The effectiveness of the proposed omnidirectional visual tracking system was demonstrated by conducting several omnidirectional detecting and tracking experiments on persons dozens of meters away using a 25-mm focal length lens under both daytime and night-time conditions.
- 1 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (2) , 691-698
A 500-Fps Pan-Tilt Tracking System With Deep-Learning-Based Object Detection
In this letter, we propose a fast mirror-drive pan-tilt target tracking system that can robustly track an object whose appearance varies in a complex background at 500 fps. By assuming a small image displacement between frames, which is a property of high-frame rate vision, we develop an fast object tracking algorithm by hybridizing the convolutional-neural-network (CNN) based object detection with template-matching (TM) based tracking operating at hundreds of frames per second (fps). For object tracking with high-speed visual feedback, the proposed tracking algorithm can remarkably reduce dozens-of-milliseconds-latency in the CNN-based object detection by simultaneously executing TM-based tracking for several images at consecutive frames within a few milliseconds. In the proposed pan-tilt tracking system, when the current tracked objects are occluded or out of the camera view, it can recognize objects to be newly tracked with CNN-based object detection at the rate of 33 fps with acceleration using graphic processing units (GPUs). Controlling the pan-tilt tracking system via visual feedback at 500 Hz, fast moving objects can be robustly tracked at the center of the camera view. The effectiveness of our method was experimentally demonstrated via several results when fast-moving pre-learned objects, such as toy cars were tracked in complex backgrounds.
- 1 January 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (2) , 691-698
A 500-Fps Pan-Tilt Tracking System With Deep-Learning-Based Object Detection
In this letter, we propose a fast mirror-drive pan-tilt target tracking system that can robustly track an object whose appearance varies in a complex background at 500 fps. By assuming a small image displacement between frames, which is a property of high-frame rate vision, we develop an fast object tracking algorithm by hybridizing the convolutional-neural-network (CNN) based object detection with template-matching (TM) based tracking operating at hundreds of frames per second (fps). For object tracking with high-speed visual feedback, the proposed tracking algorithm can remarkably reduce dozens-of-milliseconds-latency in the CNN-based object detection by simultaneously executing TM-based tracking for several images at consecutive frames within a few milliseconds. In the proposed pan-tilt tracking system, when the current tracked objects are occluded or out of the camera view, it can recognize objects to be newly tracked with CNN-based object detection at the rate of 33 fps with acceleration using graphic processing units (GPUs). Controlling the pan-tilt tracking system via visual feedback at 500 Hz, fast moving objects can be robustly tracked at the center of the camera view. The effectiveness of our method was experimentally demonstrated via several results when fast-moving pre-learned objects, such as toy cars were tracked in complex backgrounds.
- 1 October 2019
- Published by Fuji Technology Press Ltd. in Journal of Robotics and Mechatronics
- Vol. 31 (5) , 671-685
A Tandem Marker-Based Motion Capture Method for Dynamic Small Displacement Distribution Analysis
This study proposes a novel vision-based measurement method to capture small dynamic displacements at many points on a large-scale structure. The measurement points are aligned in the depth direction so that all points are observable in a single field of view with a high power zoom lens. To cope with insufficient incident light and lens blur when capturing video in a limited depth of field with large magnification, our method used highly retroreflective cubes as markers, combined with a strong coaxial lighting device for measuring image displacements with a tandem-layout in images. We conducted experiments to measure dynamic displacements of a 4 m long truss bridge model, and 18 corner cubes were attached as retroreflective markers. 752 x 2048 images were captured with a coaxial lighting device at 240 fps. The experimental results show that the deformation of the bridge model, its resonant frequencies, and mode shapes at a frequency of dozens of Hz can be determined by analyzing images captured from a single camera view.
- 1 October 2019
- Published by Fuji Technology Press Ltd. in Journal of Robotics and Mechatronics
- Vol. 31 (5) , 671-685
A Tandem Marker-Based Motion Capture Method for Dynamic Small Displacement Distribution Analysis
This study proposes a novel vision-based measurement method to capture small dynamic displacements at many points on a large-scale structure. The measurement points are aligned in the depth direction so that all points are observable in a single field of view with a high power zoom lens. To cope with insufficient incident light and lens blur when capturing video in a limited depth of field with large magnification, our method used highly retroreflective cubes as markers, combined with a strong coaxial lighting device for measuring image displacements with a tandem-layout in images. We conducted experiments to measure dynamic displacements of a 4 m long truss bridge model, and 18 corner cubes were attached as retroreflective markers. 752 x 2048 images were captured with a coaxial lighting device at 240 fps. The experimental results show that the deformation of the bridge model, its resonant frequencies, and mode shapes at a frequency of dozens of Hz can be determined by analyzing images captured from a single camera view.
- 3 September 2019
- Published by SPIE-Intl Soc Optical Eng
- Vol. 11102, 111020A-9
Precise microscopic profilometry using diffractive image correlation and theoretical model simulation
This work presents a novel microscopic profilometry using diffraction image correlation, which is appropriate for in-situ automated optical inspection (AOI). In contrast to the traditional confocal microscopy, the developed technique replaces the detector pinhole by observing and matching the diffraction patterns. Thus accurate surface depth detection can be achieved by eliminating time-consuming vertical scanning operation. The development is the first attempt in optics to use image correlation between the pre-calibrated database of diffraction images and the measured one of the detecting object surface. The feasibility of the method has been theoretically verified by scalar diffraction theory, then verified by experimental testing. Also, a depth response curve with the physical meaning of similarity is introduced and interpreted. Meanwhile, multi-point lateral scanning in one field of view (FOV) is achieved by quickly switching micromirrors on the digital micromirror device (DMD), thus the quasi full-field 3-D reconstruction can be acquired by combining tens of captured images. Verified experimentally, a 3-D reconstruction with sub-micrometer vertical resolution can be realized, with a tunable lateral resolution.
- 3 September 2019
- Published by SPIE-Intl Soc Optical Eng
- Vol. 11102, 111020A-9
Precise microscopic profilometry using diffractive image correlation and theoretical model simulation
This work presents a novel microscopic profilometry using diffraction image correlation, which is appropriate for in-situ automated optical inspection (AOI). In contrast to the traditional confocal microscopy, the developed technique replaces the detector pinhole by observing and matching the diffraction patterns. Thus accurate surface depth detection can be achieved by eliminating time-consuming vertical scanning operation. The development is the first attempt in optics to use image correlation between the pre-calibrated database of diffraction images and the measured one of the detecting object surface. The feasibility of the method has been theoretically verified by scalar diffraction theory, then verified by experimental testing. Also, a depth response curve with the physical meaning of similarity is introduced and interpreted. Meanwhile, multi-point lateral scanning in one field of view (FOV) is achieved by quickly switching micromirrors on the digital micromirror device (DMD), thus the quasi full-field 3-D reconstruction can be acquired by combining tens of captured images. Verified experimentally, a 3-D reconstruction with sub-micrometer vertical resolution can be realized, with a tunable lateral resolution.
- 1 January 2019
- Published by Japan Society of Mechanical Engineers in The Proceedings of JSME annual Conference on Robotics and Mechatronics (Robomec)
- Vol. 2019, 2P1-I08
Remote Person Authentication using a Super-telephoto Video Tracking System
Walking persons at a distance of 300 m are authenticated in video-shooting through a f = 1600 mm lens on a super-telephoto tracking system, which can control a mirror-drive pan-tilt system with high-speed visual feedback. Experimental results show the effectiveness of supertelephoto tracking in remote authentication for very distant persons in real scenarios.
- 1 January 2019
- Published by Japan Society of Mechanical Engineers in The Proceedings of JSME annual Conference on Robotics and Mechatronics (Robomec)
- Vol. 2019, 2P1-I08
Remote Person Authentication using a Super-telephoto Video Tracking System
Walking persons at a distance of 300 m are authenticated in video-shooting through a f = 1600 mm lens on a super-telephoto tracking system, which can control a mirror-drive pan-tilt system with high-speed visual feedback. Experimental results show the effectiveness of supertelephoto tracking in remote authentication for very distant persons in real scenarios.
- 8 November 2018
- Published by ASME International in Journal of Dynamic Systems, Measurement, and Control
- Vol. 141 (3)
Vision-Based Modal Analysis Using Multiple Vibration Distribution Synthesis to Inspect Large-Scale Structures
Previously, we proposed a multithread active vision system with virtual multiple pan-tilt tracking cameras by rapidly switching the viewpoints for the vibration sensing of large-scale structures. We also developed a system using a galvanometer mirror that can switch 500 different viewpoints in 1 s. However, the measurement rate of each observation point is low, and the time density is not always sufficient. In addition, strong multiple illuminations are required for the system owing to the retro reflective markers attached to the object being observed. In this study, we propose a multiple vibration distribution synthesis method for vibration analysis that increases the sampling rate of each observation point in the multi-thread active vision system, which is subsequently modified to a system that requires only one illumination by using corner cubes as markers. Several dynamics-based inspection experiments are conducted for a 4 m long truss-structure bridge model. The proposed method and system are verified via a high-order modal analysis, which was impossible to perform in the previous method and system.
- 8 November 2018
- Published by ASME International in Journal of Dynamic Systems, Measurement, and Control
- Vol. 141 (3)
Vision-Based Modal Analysis Using Multiple Vibration Distribution Synthesis to Inspect Large-Scale Structures
Previously, we proposed a multithread active vision system with virtual multiple pan-tilt tracking cameras by rapidly switching the viewpoints for the vibration sensing of large-scale structures. We also developed a system using a galvanometer mirror that can switch 500 different viewpoints in 1 s. However, the measurement rate of each observation point is low, and the time density is not always sufficient. In addition, strong multiple illuminations are required for the system owing to the retro reflective markers attached to the object being observed. In this study, we propose a multiple vibration distribution synthesis method for vibration analysis that increases the sampling rate of each observation point in the multi-thread active vision system, which is subsequently modified to a system that requires only one illumination by using corner cubes as markers. Several dynamics-based inspection experiments are conducted for a 4 m long truss-structure bridge model. The proposed method and system are verified via a high-order modal analysis, which was impossible to perform in the previous method and system.
- Open Access
- 20 June 2018
- Published by Fuji Technology Press Ltd. in Journal of Robotics and Mechatronics
- Vol. 30 (3) , 453-466
Real-Time Monocular Three-Dimensional Motion Tracking Using a Multithread Active Vision System
In this study, we developed a monocular stereo tracking system to be used as a marker-based, three-dimensional (3-D) motion capture system. This system aims to localize dozens of markers on multiple moving objects in real time by switching five hundred different views in 1 s. The ultrafast mirror-drive active vision used in our catadioptric stereo tracking system can accelerate a series of operations for multithread gaze control with video shooting, computation, and actuation within 2 ms. By switching between five hundred different views in 1 s, with real-time video processing for marker extraction, our system can function asJvirtual left and right pan-tilt tracking cameras, operating at 250/Jfps to simultaneously capture and processJpairs of 512 × 512 stereo images with different views via the catadioptric mirror system. We conducted several real-time 3-D motion experiments to capture multiple fast-moving objects with markers. The results demonstrated the effectiveness of our monocular 3-D motion tracking system.
- 20 June 2018
- Published by Fuji Technology Press Ltd. in Journal of Robotics and Mechatronics
- Vol. 30 (3) , 453-466
Open AccessReal-Time Monocular Three-Dimensional Motion Tracking Using a Multithread Active Vision System
In this study, we developed a monocular stereo tracking system to be used as a marker-based, three-dimensional (3-D) motion capture system. This system aims to localize dozens of markers on multiple moving objects in real time by switching five hundred different views in 1 s. The ultrafast mirror-drive active vision used in our catadioptric stereo tracking system can accelerate a series of operations for multithread gaze control with video shooting, computation, and actuation within 2 ms. By switching between five hundred different views in 1 s, with real-time video processing for marker extraction, our system can function asJvirtual left and right pan-tilt tracking cameras, operating at 250/Jfps to simultaneously capture and processJpairs of 512 × 512 stereo images with different views via the catadioptric mirror system. We conducted several real-time 3-D motion experiments to capture multiple fast-moving objects with markers. The results demonstrated the effectiveness of our monocular 3-D motion tracking system.