Shotaro Kojima
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Shotaro Kojima has a total of 58 co-authors. Here are the top co-authors by number of shared publications:
- Kazunori Ohno
Graduate School of Information Science, Tohoku University, Sendai, Miyagi, Japan
12 shared publications - Satoshi Tadokoro
Tohoku University
10 shared publications - Ranulfo Bezerra
Graduate School of Information Science, Tohoku University, Sendai, Miyagi, Japan
5 shared publications - Ryunosuke Hamada
Tohoku University
5 shared publications - Takahiro Suzuki
New Industry Creation Hatchery Center, Tohoku University, Sendai, Japan
5 shared publications - Thomas Westfechtel
Tadokoro Lab, Tohoku University, Sendai Japan 980-0815
4 shared publications
Latest Publications
- Open Access
- 11 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 142030-142044
Mapping the Carrier Phase of a Propagating Wave by Wireless Two-Way Interferometry Scanned by a Mobile Robot
This study proposes a wireless high-precision carrier phase mapping system that removes the need for wired synchronization, which has been a significant limitation of conventional methods. The system achieves a root mean square error (RMSE) of $\leq 0.2\,\mathrm {rad}$ across five trials. It employs wireless two-way interferometry (Wi-Wi) with 920 MHz modules, a mobile robot, and a high-precision positioning device to enable real-time and wide-area spatial sensing. To validate its effectiveness, mapping experiments were conducted over a $3\,\mathrm {m} \times 2\,\mathrm {m}$ area in an anechoic chamber. The system is particularly suitable for challenging environments such as large indoor facilities or elevated outdoor locations. This technology contributes to applications including wireless communication optimization and environmental sensing, especially in visualizing multipath, diffraction, and interference effects.
- 11 August 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 142030-142044
Open AccessMapping the Carrier Phase of a Propagating Wave by Wireless Two-Way Interferometry Scanned by a Mobile Robot
This study proposes a wireless high-precision carrier phase mapping system that removes the need for wired synchronization, which has been a significant limitation of conventional methods. The system achieves a root mean square error (RMSE) of $\leq 0.2\,\mathrm {rad}$ across five trials. It employs wireless two-way interferometry (Wi-Wi) with 920 MHz modules, a mobile robot, and a high-precision positioning device to enable real-time and wide-area spatial sensing. To validate its effectiveness, mapping experiments were conducted over a $3\,\mathrm {m} \times 2\,\mathrm {m}$ area in an anechoic chamber. The system is particularly suitable for challenging environments such as large indoor facilities or elevated outdoor locations. This technology contributes to applications including wireless communication optimization and environmental sensing, especially in visualizing multipath, diffraction, and interference effects.
- Open Access
- 9 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 186573-186585
Validation of LLM-Generated Object Co-Occurrence Information for Understanding Three-Dimensional Scenes
This study delves into verifying the applicability of object co-occurrence information generated by a large-scale language model (LLM) to enhance a robot’s spatial ability to understand objects in the real world. Co-occurrence information is crucial in enabling robots to perceive and navigate their surroundings. LLM can generate object co-occurrence information based on the learned representations acquired from the learning process. However, the challenge lies in determining whether the co-occurrence gleaned from linguistic data can effectively translate to real-world object relationships, a concept yet to be thoroughly examined. After providing category information about a specific situation, this paper compares and evaluates the co-occurrence degree (co-occurrence coefficient) output by gpt-4-turbo-2024-04-09 (GPT-4) against the object pair data from the ScanNet v2 dataset. The results revealed that GPT-4 achieved a high recall of 0.78 across various situation categories annotated by ScanNet v2, although its precision was relatively low at an average of 0.29. The root mean square error of the co-occurrence coefficient was 0.31. While GPT-4 tends to output slightly higher co-occurrence coefficients, it effectively captures the overall co-occurrence patterns observed in the ScanNet v2 dataset. GPT-4 produced co-occurrence information for more objects than those available in ScanNet v2 while covering co-occurrences among objects within ScanNet v2. These results demonstrate that integrating co-occurrence data from different sources could enhance the ability to recognize real-world objects and potentially strengthen robot intelligence.
- 9 December 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 186573-186585
Open AccessValidation of LLM-Generated Object Co-Occurrence Information for Understanding Three-Dimensional Scenes
This study delves into verifying the applicability of object co-occurrence information generated by a large-scale language model (LLM) to enhance a robot’s spatial ability to understand objects in the real world. Co-occurrence information is crucial in enabling robots to perceive and navigate their surroundings. LLM can generate object co-occurrence information based on the learned representations acquired from the learning process. However, the challenge lies in determining whether the co-occurrence gleaned from linguistic data can effectively translate to real-world object relationships, a concept yet to be thoroughly examined. After providing category information about a specific situation, this paper compares and evaluates the co-occurrence degree (co-occurrence coefficient) output by gpt-4-turbo-2024-04-09 (GPT-4) against the object pair data from the ScanNet v2 dataset. The results revealed that GPT-4 achieved a high recall of 0.78 across various situation categories annotated by ScanNet v2, although its precision was relatively low at an average of 0.29. The root mean square error of the co-occurrence coefficient was 0.31. While GPT-4 tends to output slightly higher co-occurrence coefficients, it effectively captures the overall co-occurrence patterns observed in the ScanNet v2 dataset. GPT-4 produced co-occurrence information for more objects than those available in ScanNet v2 while covering co-occurrences among objects within ScanNet v2. These results demonstrate that integrating co-occurrence data from different sources could enhance the ability to recognize real-world objects and potentially strengthen robot intelligence.
- Open Access
- 11 July 2024
- Published by Taylor & Francis in Advanced Robotics
- Vol. 38 (14) , 947-957
Effectiveness of canine training using suit-mounted feeder
- 11 July 2024
- Published by Taylor & Francis in Advanced Robotics
- Vol. 38 (14) , 947-957
Open AccessEffectiveness of canine training using suit-mounted feeder
- Open Access
- 26 August 2022
- Published by Taylor & Francis in Advanced Robotics
- Vol. 36 (21) , 1102-1119
Heterogeneous robots coordination for industrial plant inspection and evaluation at World Robot Summit 2020
This paper describes a plant inspection robot team consisting of three types of robots that won the World Robot Summit 2020 plant disaster prevention challenge. There is a social demand to use robots for industrial plant inspection tasks to decrease the workload and risks of human operators. In this study, the authors developed a plant inspection system using a tracked vehicle, a mecanum-wheeled vehicle, and an unmanned aerial vehicle (UAV). The roles between work-related tasks and visual inspection tasks are divided by two operators, resulting in efficient operation. Experimental results show that the tracked vehicle can conduct challenging tasks such as climbing up the stairs and large-valve operations. The results also showed that the visual inspection task was performed 58% faster by the mecanum vehicle in a narrow passage and 48% faster by the UAV on the stairs, as compared with the tracked vehicle in the same situations. The analysis of the competition results showed that we obtained points in 57% (160/280) of the work-related tasks and 85% (170/200) of the visual inspection tasks as well as 92% (110/120) of the emergency response tasks in the final round. GRAPHICAL ABSTRACT
- 26 August 2022
- Published by Taylor & Francis in Advanced Robotics
- Vol. 36 (21) , 1102-1119
Open AccessHeterogeneous robots coordination for industrial plant inspection and evaluation at World Robot Summit 2020
This paper describes a plant inspection robot team consisting of three types of robots that won the World Robot Summit 2020 plant disaster prevention challenge. There is a social demand to use robots for industrial plant inspection tasks to decrease the workload and risks of human operators. In this study, the authors developed a plant inspection system using a tracked vehicle, a mecanum-wheeled vehicle, and an unmanned aerial vehicle (UAV). The roles between work-related tasks and visual inspection tasks are divided by two operators, resulting in efficient operation. Experimental results show that the tracked vehicle can conduct challenging tasks such as climbing up the stairs and large-valve operations. The results also showed that the visual inspection task was performed 58% faster by the mecanum vehicle in a narrow passage and 48% faster by the UAV on the stairs, as compared with the tracked vehicle in the same situations. The analysis of the competition results showed that we obtained points in 57% (160/280) of the work-related tasks and 85% (170/200) of the visual inspection tasks as well as 92% (110/120) of the emergency response tasks in the final round. GRAPHICAL ABSTRACT
- Open Access
- 25 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 4047-4054
Electrocardiogram Measurement and Emotion Estimation of Working Dogs
Physiological measurements of dogs' emotional states during human-animal interactions are essential for understanding the underlying biological relationship. Heart rate measured by electrocardiogram (ECG) can be used for the physiological measurement of emotional state. Soft disposable electrodes, which can be purchased commercially and reduce the risk of an injury on hitting objects, are used to measure the heart rate during working dogs' exercise. However, the heart rate measurements using soft disposable electrodes in M-X lead layout are unstable due to the detachment of electrodes during intensive movement. In addition, the collection of accurately labeled ECG data, which reflects the emotion of the working dog during exercise, is a key issue to conduct reliable estimation and visualization of the working dog's emotional states. In this paper, we propose an innerwear that fixes soft disposable electrodes in M-X lead layout on a dog's body for measuring the heart rates of exercising dogs, even during intense movement. We experimentally tested the proposed method using seven dogs. Our proposed system reduced the time required to attach soft disposable electrodes by more than two-thirds and enabled the measurement of >97% of the heartbeats in exercising dogs. We also proposed a method for collecting ECG data with reliable positive/negative emotional labels of search and rescue (SAR) dogs. A combination of the collected data with the emotion estimation algorithm proposed in previous studies estimated SAR dogs' positive/negative emotions at 97.9% accuracy. The transition of the two SAR dogs' emotions during drills was visualized using the collected data.
- 25 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 4047-4054
Open AccessElectrocardiogram Measurement and Emotion Estimation of Working Dogs
Physiological measurements of dogs' emotional states during human-animal interactions are essential for understanding the underlying biological relationship. Heart rate measured by electrocardiogram (ECG) can be used for the physiological measurement of emotional state. Soft disposable electrodes, which can be purchased commercially and reduce the risk of an injury on hitting objects, are used to measure the heart rate during working dogs' exercise. However, the heart rate measurements using soft disposable electrodes in M-X lead layout are unstable due to the detachment of electrodes during intensive movement. In addition, the collection of accurately labeled ECG data, which reflects the emotion of the working dog during exercise, is a key issue to conduct reliable estimation and visualization of the working dog's emotional states. In this paper, we propose an innerwear that fixes soft disposable electrodes in M-X lead layout on a dog's body for measuring the heart rates of exercising dogs, even during intense movement. We experimentally tested the proposed method using seven dogs. Our proposed system reduced the time required to attach soft disposable electrodes by more than two-thirds and enabled the measurement of >97% of the heartbeats in exercising dogs. We also proposed a method for collecting ECG data with reliable positive/negative emotional labels of search and rescue (SAR) dogs. A combination of the collected data with the emotion estimation algorithm proposed in previous studies estimated SAR dogs' positive/negative emotions at 97.9% accuracy. The transition of the two SAR dogs' emotions during drills was visualized using the collected data.
- Open Access
- 13 October 2021
- Published by Taylor & Francis in Advanced Robotics
- Vol. 35 (23) , 1388-1403
An individual prediction model of the pre-loading motion for operator and backhoe pairs
The autonomous dump truck must judge whether the backhoe is ready for loading the sediment for smooth and safe cooperation with the human-operated backhoes. Analyzing time-series data of the human-operated backhoe loading motion is an effective method to build a prediction model. The transition of several primitive motions enables us to predict the timing when the backhoe starts loading. However, in transition modeling, manually selecting the appropriate primitive motions in the pre-loading motions requires considerable effort. In addition, a robust loading prediction is required for sensor layout changes in the installations of them. Here, we propose a BP-HMM-based prediction method of the pre-loading motion. The algorithm automatically finds the transition of several primitive motions from time-series data and its annotation labels. The selection of three angular velocities as features of the BP-HMM increases the robustness of the prediction method. The proposed method built a suitable prediction model for three different combinations of operators and backhoes. The prediction method was robust for sensor layout changes, and showed an accuracy of 100%. The proposed prediction method contributes to the automation of earthmoving work by enabling smooth cooperation between autonomous dump trucks and human-operated backhoe. GRAPHICAL ABSTRACT
- 13 October 2021
- Published by Taylor & Francis in Advanced Robotics
- Vol. 35 (23) , 1388-1403
Open AccessAn individual prediction model of the pre-loading motion for operator and backhoe pairs
The autonomous dump truck must judge whether the backhoe is ready for loading the sediment for smooth and safe cooperation with the human-operated backhoes. Analyzing time-series data of the human-operated backhoe loading motion is an effective method to build a prediction model. The transition of several primitive motions enables us to predict the timing when the backhoe starts loading. However, in transition modeling, manually selecting the appropriate primitive motions in the pre-loading motions requires considerable effort. In addition, a robust loading prediction is required for sensor layout changes in the installations of them. Here, we propose a BP-HMM-based prediction method of the pre-loading motion. The algorithm automatically finds the transition of several primitive motions from time-series data and its annotation labels. The selection of three angular velocities as features of the BP-HMM increases the robustness of the prediction method. The proposed method built a suitable prediction model for three different combinations of operators and backhoes. The prediction method was robust for sensor layout changes, and showed an accuracy of 100%. The proposed prediction method contributes to the automation of earthmoving work by enabling smooth cooperation between autonomous dump trucks and human-operated backhoe. GRAPHICAL ABSTRACT
- 26 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (2) , 3073-3080
Semantic Mapping of Construction Site From Multiple Daily Airborne LiDAR Data
Semantic maps are an important tool to provide robots with high-level knowledge about the environment, enabling them to better react to and interact with their surroundings. However, as a single measurement of the environment is solely a snapshot of a specific time, it does not necessarily reflect the underlying semantics. In this work, we propose a method to create a semantic map of a construction site by fusing multiple daily data. The construction site is measured by an unmanned aerial vehicle (UAV) equipped with a LiDAR. We extract clusters above ground level from the measurements and classify them using either a random forest or a deep learning based classifier. Furthermore, we combine the classification results of several measurements to generalize the classification of the single measurements and create a general semantic map of the working site. We measured two construction fields for our evaluation. The classification models can achieve an average intersection over union (IoU) score of 69.2% during classification on the Sanbongi field, which is used for training, validation and testing and an IoU score of 49.16% on a hold-out testing field. In a final step, we show how the semantic map can be employed to suggest a parking spot for a dump truck, and in addition, show that the semantic map can be utilized to improve path planning inside the construction site.
- 26 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (2) , 3073-3080
Semantic Mapping of Construction Site From Multiple Daily Airborne LiDAR Data
Semantic maps are an important tool to provide robots with high-level knowledge about the environment, enabling them to better react to and interact with their surroundings. However, as a single measurement of the environment is solely a snapshot of a specific time, it does not necessarily reflect the underlying semantics. In this work, we propose a method to create a semantic map of a construction site by fusing multiple daily data. The construction site is measured by an unmanned aerial vehicle (UAV) equipped with a LiDAR. We extract clusters above ground level from the measurements and classify them using either a random forest or a deep learning based classifier. Furthermore, we combine the classification results of several measurements to generalize the classification of the single measurements and create a general semantic map of the working site. We measured two construction fields for our evaluation. The classification models can achieve an average intersection over union (IoU) score of 69.2% during classification on the Sanbongi field, which is used for training, validation and testing and an IoU score of 49.16% on a hold-out testing field. In a final step, we show how the semantic map can be employed to suggest a parking spot for a dump truck, and in addition, show that the semantic map can be utilized to improve path planning inside the construction site.
- 11 August 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 5 (4) , 6575-6582
Stable Autonomous Spiral Stair Climbing of Tracked Vehicles Using Wall Reaction Force
In this paper, an autonomous spiral stair climbing method for tracked vehicles using the reaction force from side walls has been proposed. Spiral stairs are one of the most difficult terrains for tracked vehicles because of their asymmetrical ground shape and small turning radius. Tracked vehicles are expected to be used in industrial plant inspection tasks, where robots should navigate on multiple floors by ascending the stairs. Spiral or curved stairs are installed as part of inspection passages for cylindrical facilities, such as boilers, chimneys, or large tanks. Previously, the authors have experimentally demonstrated that the wall-following motion is effective for stabilizing and accelerating spiral stair climbing. However, the complete automation of climbing motion or the analysis of why the same motion is generated even if a disturbance exists in the initial entry angle to the wall should be investigated. In this study, the authors developed an autonomous spiral stair climbing method using the wall reaction force and clarified the applicable limitations of this method using a geometrical model. Autonomous spiral stair climbing is realized by attaching passive wheels on its collision point and automating the motions of main-tracks and sub-tracks. The geometrical model shows the expected trajectory of the robot on the spiral stairs, which suggests that the robot 's rotation radius converges to a specific value; this is experimentally confirmed by measuring the robot 's motion. The wall-following motion of robots is equivalent to human inspectors grasping handrails while climbing stairs. Through collisions with surrounding objects, motion is stabilized and certainty is guaranteed.
- 11 August 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 5 (4) , 6575-6582
Stable Autonomous Spiral Stair Climbing of Tracked Vehicles Using Wall Reaction Force
In this paper, an autonomous spiral stair climbing method for tracked vehicles using the reaction force from side walls has been proposed. Spiral stairs are one of the most difficult terrains for tracked vehicles because of their asymmetrical ground shape and small turning radius. Tracked vehicles are expected to be used in industrial plant inspection tasks, where robots should navigate on multiple floors by ascending the stairs. Spiral or curved stairs are installed as part of inspection passages for cylindrical facilities, such as boilers, chimneys, or large tanks. Previously, the authors have experimentally demonstrated that the wall-following motion is effective for stabilizing and accelerating spiral stair climbing. However, the complete automation of climbing motion or the analysis of why the same motion is generated even if a disturbance exists in the initial entry angle to the wall should be investigated. In this study, the authors developed an autonomous spiral stair climbing method using the wall reaction force and clarified the applicable limitations of this method using a geometrical model. Autonomous spiral stair climbing is realized by attaching passive wheels on its collision point and automating the motions of main-tracks and sub-tracks. The geometrical model shows the expected trajectory of the robot on the spiral stairs, which suggests that the robot 's rotation radius converges to a specific value; this is experimentally confirmed by measuring the robot 's motion. The wall-following motion of robots is equivalent to human inspectors grasping handrails while climbing stairs. Through collisions with surrounding objects, motion is stabilized and certainty is guaranteed.
- Open Access
- 17 January 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 5 (2) , 1358-1365
Wall Deadlock Evasion Control Based on Rotation Radius Adjustment
This paper describes a wall deadlock evasion method for tracked vehicles. Wall deadlock is a phenomenon where the robot cannot rotate to the commanded direction when it collides with a wall, because the motion is restricted by the wall. The key idea behind solving this problem involves an adjustment of the rotation radius to generate sufficient rotational moment. There are several approaches to generate a rotational moment; however, no previous solution has been established to address this problem by adjusting the rotation radius based on the dynamics of wall deadlock. In this paper, the authors propose a new wall deadlock evasion method based on the sufficient rotation radius estimation. Experimental results show that the robot can generate rotational motion that satisfies conditions expected by the model. The wall deadlock evasion method is implemented and shows improved performance in terms of reproducibility of motion compared with the different approach proposed in our previous work. Wall deadlock evasion provides more choices of motion such as being as close to the obstacles as possible and ensures that the robot can continue locomotion after such motion. By handling wall deadlock, the robots can utilize surrounding walls for motion in situations such as relative positioning or driving in fixed lanes.
- 17 January 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 5 (2) , 1358-1365
Open AccessWall Deadlock Evasion Control Based on Rotation Radius Adjustment
This paper describes a wall deadlock evasion method for tracked vehicles. Wall deadlock is a phenomenon where the robot cannot rotate to the commanded direction when it collides with a wall, because the motion is restricted by the wall. The key idea behind solving this problem involves an adjustment of the rotation radius to generate sufficient rotational moment. There are several approaches to generate a rotational moment; however, no previous solution has been established to address this problem by adjusting the rotation radius based on the dynamics of wall deadlock. In this paper, the authors propose a new wall deadlock evasion method based on the sufficient rotation radius estimation. Experimental results show that the robot can generate rotational motion that satisfies conditions expected by the model. The wall deadlock evasion method is implemented and shows improved performance in terms of reproducibility of motion compared with the different approach proposed in our previous work. Wall deadlock evasion provides more choices of motion such as being as close to the obstacles as possible and ensures that the robot can continue locomotion after such motion. By handling wall deadlock, the robots can utilize surrounding walls for motion in situations such as relative positioning or driving in fixed lanes.
- Open Access
- 22 June 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 3 (4) , 3371-3378
Parking Spot Estimation and Mapping Method for Mobile Robots
Self-driving vehicles rely on detailed semantic maps of the environment for operating. In this work, we propose a method to autonomously generate such a semantic map enriched with knowledge of parking spot locations. Our method detects and uses parked vehicles in the surroundings to estimate parking lot topology and infer vacant parking spots via a graph-based approach. We show that our method works for parking lot structures in different environments, such as structured parking lots, unstructured/unmarked parking lots, and typical suburban environments. Using the proposed graph-based approach to infer the parking lot structure, we can extend the estimated parking spots by 57%, averaged over six different areas with ten trials each. We also show that the accuracy of our algorithm increases when combining multiple trials over multiple days. With ten trials combined, we managed to estimate the whole parking lot structure and detected all parking spots in four out of the six evaluated areas.
- 22 June 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 3 (4) , 3371-3378
Open AccessParking Spot Estimation and Mapping Method for Mobile Robots
Self-driving vehicles rely on detailed semantic maps of the environment for operating. In this work, we propose a method to autonomously generate such a semantic map enriched with knowledge of parking spot locations. Our method detects and uses parked vehicles in the surroundings to estimate parking lot topology and infer vacant parking spots via a graph-based approach. We show that our method works for parking lot structures in different environments, such as structured parking lots, unstructured/unmarked parking lots, and typical suburban environments. Using the proposed graph-based approach to infer the parking lot structure, we can extend the estimated parking spots by 57%, averaged over six different areas with ten trials each. We also show that the accuracy of our algorithm increases when combining multiple trials over multiple days. With ten trials combined, we managed to estimate the whole parking lot structure and detected all parking spots in four out of the six evaluated areas.