Jan Weyler
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Jan Weyler has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Cyrill Stachniss
University of Bonn
6 shared publications - Jens Behley
University of Bonn
6 shared publications - Federico Magistri
Center for Robotics, University of Bonn, Bonn, Germany
2 shared publications - Philipp Lottes
University of Bonn, Bonn, Germany
2 shared publications - Thomas Läbe
Center for Robotics, University of Bonn, Bonn, Germany
2 shared publications - Yue Linn Chong
Center for Robotics, University of Bonn, Bonn, Germany
2 shared publications
Latest Publications
- Top Cited Papers
- 26 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (12) , 9583-9594
PhenoBench: A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain
The production of food, feed, fiber, and fuel is a key task of agriculture, which has to cope with many challenges in the upcoming decades, e.g., a higher demand, climate change, lack of workers, and the availability of arable land. Vision systems can support making better and more sustainable field management decisions, but also support the breeding of new crop varieties by allowing temporally dense and reproducible measurements. Recently, agricultural robotics got an increasing interest in the vision and robotics communities since it is a promising avenue for coping with the aforementioned lack of workers and enabling more sustainable production. While large datasets and benchmarks in other domains are readily available and enable significant progress, agricultural datasets and benchmarks are comparably rare. We present an annotated dataset and benchmarks for the semantic interpretation of real agricultural fields. Our dataset recorded with a UAV provides high-quality, pixel-wise annotations of crops and weeds, but also crop leaf instances at the same time. Furthermore, we provide benchmarks for various tasks on a hidden test set comprised of different fields: known fields covered by the training data and a completely unseen field.
- 26 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (12) , 9583-9594
Top Cited PapersPhenoBench: A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain
The production of food, feed, fiber, and fuel is a key task of agriculture, which has to cope with many challenges in the upcoming decades, e.g., a higher demand, climate change, lack of workers, and the availability of arable land. Vision systems can support making better and more sustainable field management decisions, but also support the breeding of new crop varieties by allowing temporally dense and reproducible measurements. Recently, agricultural robotics got an increasing interest in the vision and robotics communities since it is a promising avenue for coping with the aforementioned lack of workers and enabling more sustainable production. While large datasets and benchmarks in other domains are readily available and enable significant progress, agricultural datasets and benchmarks are comparably rare. We present an annotated dataset and benchmarks for the semantic interpretation of real agricultural fields. Our dataset recorded with a UAV provides high-quality, pixel-wise annotations of crops and weeds, but also crop leaf instances at the same time. Furthermore, we provide benchmarks for various tasks on a hidden test set comprised of different fields: known fields covered by the training data and a completely unseen field.
- 25 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (2) , 1660-1667
Panoptic Segmentation With Partial Annotations for Agricultural Robots
A detailed analysis of agricultural fields is key toward reducing the use of agrochemicals to achieve a more sustainable crop production. To this end, agricultural robots equipped with vision-based systems offer the potential to detect individual plants in the field automatically. This capability enables targeted management actions in the field, effectively reducing the amount of agrochemicals. A primary target of such vision systems is to perform a panoptic segmentation, combining the task of semantic and instance segmentation. Recent methods use neural networks for this task, which typically have to be trained on densely annotated images containing the required ground truth information for each pixel. Gathering these dense annotations is generally daunting and requires domain experts' knowledge in the agricultural domain. In this paper, we propose a method to effectively reduce the annotation bottleneck and yet achieve high performance using partial annotations. These partial annotations contain ground truth information only for a subset of pixels per image and are thus much faster to obtain than dense annotations. We propose a novel set of losses that exploit measures from vector fields used in physics, i.e. , divergence and curl, to effectively supervise predictions without ground truth annotations. The experimental evaluation shows that our approach outperforms several state-of-the-art methods targeting to reduce the amount of annotations.
- 25 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 9 (2) , 1660-1667
Panoptic Segmentation With Partial Annotations for Agricultural Robots
A detailed analysis of agricultural fields is key toward reducing the use of agrochemicals to achieve a more sustainable crop production. To this end, agricultural robots equipped with vision-based systems offer the potential to detect individual plants in the field automatically. This capability enables targeted management actions in the field, effectively reducing the amount of agrochemicals. A primary target of such vision systems is to perform a panoptic segmentation, combining the task of semantic and instance segmentation. Recent methods use neural networks for this task, which typically have to be trained on densely annotated images containing the required ground truth information for each pixel. Gathering these dense annotations is generally daunting and requires domain experts' knowledge in the agricultural domain. In this paper, we propose a method to effectively reduce the annotation bottleneck and yet achieve high performance using partial annotations. These partial annotations contain ground truth information only for a subset of pixels per image and are thus much faster to obtain than dense annotations. We propose a novel set of losses that exploit measures from vector fields used in physics, i.e. , divergence and curl, to effectively supervise predictions without ground truth annotations. The experimental evaluation shows that our approach outperforms several state-of-the-art methods targeting to reduce the amount of annotations.
- 7 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (8) , 5259-5266
Unsupervised Generation of Labeled Training Images for Crop-Weed Segmentation in New Fields and on Different Robotic Platforms
Agricultural robots have the potential to improve the efficiency and sustainability of existing agricultural practices. Most autonomous agricultural robots rely on machine vision systems. Such systems, however, often perform worse in new fields or when the robotic platforms change. While we can alleviate the performance degradation by manually labeling more data obtained in the new setup, this procedure is labor and cost-intensive. Therefore, we propose an approach to improve the performance of machine vision systems for new fields and different robotic platforms without additional manual labeling. In an unsupervised manner, our approach can generate images and corresponding labels to train machine vision systems. We use StyleGAN2 to generate images that appear like they are from desired new field or robotic platform. Additionally, we propose a label refinement method to generate labels corresponding to the generated images. We show that our approach can improve the performance of the crop-weed segmentation task in new fields and on different robotic platforms without additional manual labeling.
- 7 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (8) , 5259-5266
Unsupervised Generation of Labeled Training Images for Crop-Weed Segmentation in New Fields and on Different Robotic Platforms
Agricultural robots have the potential to improve the efficiency and sustainability of existing agricultural practices. Most autonomous agricultural robots rely on machine vision systems. Such systems, however, often perform worse in new fields or when the robotic platforms change. While we can alleviate the performance degradation by manually labeling more data obtained in the new setup, this procedure is labor and cost-intensive. Therefore, we propose an approach to improve the performance of machine vision systems for new fields and different robotic platforms without additional manual labeling. In an unsupervised manner, our approach can generate images and corresponding labels to train machine vision systems. We use StyleGAN2 to generate images that appear like they are from desired new field or robotic platform. Additionally, we propose a label refinement method to generate labels corresponding to the generated images. We show that our approach can improve the performance of the crop-weed segmentation task in new fields and on different robotic platforms without additional manual labeling.
- 28 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (6) , 3310-3317
Towards Domain Generalization in Crop and Weed Segmentation for Precision Farming Robots
Precision farming robots offer the potential to reduce the amount of used agrochemicals through targeted interventions and thus are a promising step towards sustainable agriculture. A prerequisite for such systems is a robust plant classification system that can identify crops and weeds in various agricultural fields. Most vision-based systems train convolutional neural networks (CNNs) on a given dataset, i.e., the source domain, to perform semantic segmentation of images. However, deploying these models on unseen fields, i.e., in the target domain, often shows a low generalization capability. Enhancing the generalization capability of CNNs is critical to increasing their performance on target domains with different operational conditions. In this letter, we present a domain generalized semantic segmentation approach for robust crop and weed detection by effectively extending and diversifying the source domain to achieve high performance across different agricultural field conditions. We propose to leverage unlabeled images captured from various agricultural fields during training in a two-step framework. First, we suggest a method to automatically compute sparse annotations and use them to present the model more plant varieties and growth stages to enhance its generalization capability. Among others, we exploit unlabeled images from fields containing crops sown in rows. Second, we propose a style transfer method that renders the source domain images in the style of images from various fields to achieve increased diversification. We conduct extensive experiments and show that we achieve superior performance in crop-weed segmentation across various fields compared to state-of-the-art methods.
- 28 March 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 8 (6) , 3310-3317
Towards Domain Generalization in Crop and Weed Segmentation for Precision Farming Robots
Precision farming robots offer the potential to reduce the amount of used agrochemicals through targeted interventions and thus are a promising step towards sustainable agriculture. A prerequisite for such systems is a robust plant classification system that can identify crops and weeds in various agricultural fields. Most vision-based systems train convolutional neural networks (CNNs) on a given dataset, i.e., the source domain, to perform semantic segmentation of images. However, deploying these models on unseen fields, i.e., in the target domain, often shows a low generalization capability. Enhancing the generalization capability of CNNs is critical to increasing their performance on target domains with different operational conditions. In this letter, we present a domain generalized semantic segmentation approach for robust crop and weed detection by effectively extending and diversifying the source domain to achieve high performance across different agricultural field conditions. We propose to leverage unlabeled images captured from various agricultural fields during training in a two-step framework. First, we suggest a method to automatically compute sparse annotations and use them to present the model more plant varieties and growth stages to enhance its generalization capability. Among others, we exploit unlabeled images from fields containing crops sown in rows. Second, we propose a style transfer method that renders the source domain images in the style of images from various fields to achieve increased diversification. We conduct extensive experiments and show that we achieve superior performance in crop-weed segmentation across various fields compared to state-of-the-art methods.
- 1 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 3787-3794
Joint Plant and Leaf Instance Segmentation on Field-Scale UAV Imagery
Monitoring of fields and breeding plots is critical for farmers, plant scientists, and breeders. In this process, a key objective is to assess and monitor the growth stages together with the number of individual plants on the field. Traditionally, this in-field assessment is performed manually and thus is limited in temporal and spatial throughput. In contrast, vision-based systems offer the potential to assess these traits frequently in an automated fashion on a large scale. The primary target of these systems is to detect and segment each plant and its leaves since this information directly correlates to the growth stage and allows for detailed monitoring. In this letter, we address the problem of automated, instance-level plant monitoring in agricultural fields and breeding plots. We propose a vision-based approach to perform a joint instance segmentation of crop plants and leaves in breeding plots. We develop a convolutional neural network to determine the position of specific plant keypoints and group pixels to detect individual leaf and plant instances. Finally, we provide a pixel-wise instance segmentation of each crop and its associated leaves based on orthorectified RGB images captured by UAVs. The experimental evaluation shows that our method outperforms state-of-the-art instance segmentation approaches such as Mask-RCNN on this task.
- 1 February 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 7 (2) , 3787-3794
Joint Plant and Leaf Instance Segmentation on Field-Scale UAV Imagery
Monitoring of fields and breeding plots is critical for farmers, plant scientists, and breeders. In this process, a key objective is to assess and monitor the growth stages together with the number of individual plants on the field. Traditionally, this in-field assessment is performed manually and thus is limited in temporal and spatial throughput. In contrast, vision-based systems offer the potential to assess these traits frequently in an automated fashion on a large scale. The primary target of these systems is to detect and segment each plant and its leaves since this information directly correlates to the growth stage and allows for detailed monitoring. In this letter, we address the problem of automated, instance-level plant monitoring in agricultural fields and breeding plots. We propose a vision-based approach to perform a joint instance segmentation of crop plants and leaves in breeding plots. We develop a convolutional neural network to determine the position of specific plant keypoints and group pixels to detect individual leaf and plant instances. Finally, we provide a pixel-wise instance segmentation of each crop and its associated leaves based on orthorectified RGB images captured by UAVs. The experimental evaluation shows that our method outperforms state-of-the-art instance segmentation approaches such as Mask-RCNN on this task.
- 19 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (2) , 3599-3606
Joint Plant Instance Detection and Leaf Count Estimation for In-Field Plant Phenotyping
Precision management of agricultural fields as well as plant breeding are central factors for keeping yields high and to provide food, feed, and fiber for our society. A key element in breeding trials but also for targeted management actions is to analyze the growth state of individual plants objectively and at a large scale. In this letter, we address the problem of analyzing crops in real agricultural fields based on camera data recorded with mobile robots and to derive information about the plant development, e.g., to monitor phenotypic traits such as growth stage. We propose a novel single-stage object detection approach that localizes crops and weeds in the field. At the same time, it detects plant-specific leaf keypoints intending to estimate leaf count at a plant level, which is a key trait for classifying the growth stage. We implemented and thoroughly tested our approach on real sugar beet fields. As our experiments show, it performs the required detections and shows superior performance with respect to a state-of-the-art two-stage approach based on Mask R-CNN.
- 19 February 2021
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 6 (2) , 3599-3606
Joint Plant Instance Detection and Leaf Count Estimation for In-Field Plant Phenotyping
Precision management of agricultural fields as well as plant breeding are central factors for keeping yields high and to provide food, feed, and fiber for our society. A key element in breeding trials but also for targeted management actions is to analyze the growth state of individual plants objectively and at a large scale. In this letter, we address the problem of analyzing crops in real agricultural fields based on camera data recorded with mobile robots and to derive information about the plant development, e.g., to monitor phenotypic traits such as growth stage. We propose a novel single-stage object detection approach that localizes crops and weeds in the field. At the same time, it detects plant-specific leaf keypoints intending to estimate leaf count at a plant level, which is a key trait for classifying the growth stage. We implemented and thoroughly tested our approach on real sugar beet fields. As our experiments show, it performs the required detections and shows superior performance with respect to a state-of-the-art two-stage approach based on Mask R-CNN.