Jeffrey Mason Earles
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Jeffrey Mason Earles has a total of 88 co-authors. Here are the top co-authors by number of shared publications:
- Craig Robert Brodersen
School of the Environment, Yale University, New Haven, Connecticut 06511, USA
9 shared publications - Andrew Joseph McElrone
Department of Viticulture and Enology, University of California, Davis, CA, USA
8 shared publications - Guillaume Théroux-Rancourt
Department of Ecosystem Management, Climate and Biodiversity Institute of Botany, BOKU University Vienna 1180 Austria
7 shared publications - Maciej Andrzej Zwieniecki
Plant Sciences, UC Davis, Davis, USA
7 shared publications - Matthew Edmund Gilbert
Department of Plant Sciences University of California Davis California USA
4 shared publications - Adam B. Roddy
Department of Environmental Studies New York University New York NY 10012 USA
3 shared publications
Latest Publications
- Open Access
Genetic mapping and genomic prediction for agronomic, grain compositional, and sensing-enabled traits in a cowpea MAGIC population along an environmental gradient
Summary: Cowpea ( Vigna unguiculata [L.] Walp.) is a resilient grain legume and an important global source of dietary protein, yet the genetic and environmental basis of phenological and canopy development, as well as grain composition, remains incompletely characterized across production environments. In this study, we evaluated a cowpea multi-parent advanced generation intercross (MAGIC) population along an environmental gradient in California (with contrasting daylengths, temperatures, and soil types) using agronomic, grain compositional, and uncrewed aerial vehicle (UAV) and rover-enabled phenotyping. Near-infrared spectroscopy (NIRS) enabled assessment of grain compositional traits, while sensing-enabled time-series imaging captured canopy and reproductive dynamics. Quantitative trait locus (QTL) mapping identified 267 QTL, and genome-wide association studies (GWAS) detected 1,973 marker-trait associations. Integrating QTL mapping and GWAS results identified two major genomic hotspots affecting multiple traits. A chromosome 9 hotspot (5.8–6.0 Mb) was associated with flowering time and co-localized with sensing-enabled measures of flower and pod counts, plant height, and vegetation fraction, indicating broad effects on phenological and canopy development. A chromosome 8 hotspot (37.3–37.9 Mb) contained co-localized signals for seed weight, protein, starch, phytate, and moisture. A total of 22 prioritized candidate genes were identified within these and other loci with multi-environment QTL and GWAS support. Genomic predictive abilities were moderate to high for most traits and scenarios, with multi-trait MegaLMM outperforming RR-BLUP. Together, these results define major genomic regions controlling cowpea phenology, canopy development, and grain composition, and provide targets and strategies for breeding cowpea cultivars with favorable and environmentally resilient productivity and grain composition. Significance Statement: To dissect the genetic basis of cowpea productivity, adaptation, and grain composition, and how performance for these traits varies and can be predicted across environments, we combined multi-environment phenotyping, including sensing of canopy and reproductive traits, with quantitative genetic analyses in a multi-parental population. We identified genomic hotspots for seed size/composition and reproductive phenology and an across-environment predictive advantage for multi-trait vs. single-trait genomic prediction. Overall, these findings support the comprehensive improvement of cowpea.
Open AccessGenetic mapping and genomic prediction for agronomic, grain compositional, and sensing-enabled traits in a cowpea MAGIC population along an environmental gradient
Summary: Cowpea ( Vigna unguiculata [L.] Walp.) is a resilient grain legume and an important global source of dietary protein, yet the genetic and environmental basis of phenological and canopy development, as well as grain composition, remains incompletely characterized across production environments. In this study, we evaluated a cowpea multi-parent advanced generation intercross (MAGIC) population along an environmental gradient in California (with contrasting daylengths, temperatures, and soil types) using agronomic, grain compositional, and uncrewed aerial vehicle (UAV) and rover-enabled phenotyping. Near-infrared spectroscopy (NIRS) enabled assessment of grain compositional traits, while sensing-enabled time-series imaging captured canopy and reproductive dynamics. Quantitative trait locus (QTL) mapping identified 267 QTL, and genome-wide association studies (GWAS) detected 1,973 marker-trait associations. Integrating QTL mapping and GWAS results identified two major genomic hotspots affecting multiple traits. A chromosome 9 hotspot (5.8–6.0 Mb) was associated with flowering time and co-localized with sensing-enabled measures of flower and pod counts, plant height, and vegetation fraction, indicating broad effects on phenological and canopy development. A chromosome 8 hotspot (37.3–37.9 Mb) contained co-localized signals for seed weight, protein, starch, phytate, and moisture. A total of 22 prioritized candidate genes were identified within these and other loci with multi-environment QTL and GWAS support. Genomic predictive abilities were moderate to high for most traits and scenarios, with multi-trait MegaLMM outperforming RR-BLUP. Together, these results define major genomic regions controlling cowpea phenology, canopy development, and grain composition, and provide targets and strategies for breeding cowpea cultivars with favorable and environmentally resilient productivity and grain composition. Significance Statement: To dissect the genetic basis of cowpea productivity, adaptation, and grain composition, and how performance for these traits varies and can be predicted across environments, we combined multi-environment phenotyping, including sensing of canopy and reproductive traits, with quantitative genetic analyses in a multi-parental population. We identified genomic hotspots for seed size/composition and reproductive phenology and an across-environment predictive advantage for multi-trait vs. single-trait genomic prediction. Overall, these findings support the comprehensive improvement of cowpea.
- Open Access
- 20 May 2026
- Published by Frontiers Media SA in Frontiers in Artificial Intelligence
- Vol. 9, 1844338
A vision language model for generating XML-based organ-level plant architecture representations of cowpea from simulated images
Three-dimensional (3D) procedural plant architecture models have emerged as an important tool for simulation-based studies of plant structure and function, extracting plant architectural parameters from field measurements, and for generating realistic plants in computer graphics. However, measuring the architectural parameters for these models at the field and population scales remains prohibitively labor-intensive. We present a novel algorithm that generates the 3D plant architecture from an image, to create a functional structural plant model from an image that reflects organ-level geometric and topological parameters, providing a more comprehensive representation of the plant’s architecture. Instead of using 3D sensors or processing multi-view images with computer vision to obtain the 3D structure of plants, we propose a method that generates token sequences containing a procedural definition of the plant architecture. This work uses only synthetic images for training and testing, where “exact” architectural parameters were known, which allowed for testing of the hypothesis that organ-level architectural parameters could be extracted from imagery data using a vision language model (VLM). A synthetic dataset of cowpea plant images was generated using the Helios 3D plant simulator, with the detailed plant architecture encoded in XML files. We developed a plant architecture tokenizer for the XML file defining plant architecture, converting it into a token sequence that a language model can predict. Then, a VLM was trained to predict plant architecture token sequences from images. Our results demonstrate that the model can predict plant architecture tokens with an F1 score of 0.73 in a teacher-forcing method. Evaluation of the model was performed through autoregressive generation, achieving a BLEU-4 score of 94.00% and a ROUGE-L score of 0.5182. Our model achieves lower MAPE than feature regression-based methods in estimating bulk plant-level traits that require understanding of the occluded 3D structure of the plant, such as leaf count and leaf area. We conclude that generating plant architecture and parameter extraction from synthetic imagery are feasible using a VLM approach, supporting future extension to real imagery.
- 20 May 2026
- Published by Frontiers Media SA in Frontiers in Artificial Intelligence
- Vol. 9, 1844338
Open AccessA vision language model for generating XML-based organ-level plant architecture representations of cowpea from simulated images
Three-dimensional (3D) procedural plant architecture models have emerged as an important tool for simulation-based studies of plant structure and function, extracting plant architectural parameters from field measurements, and for generating realistic plants in computer graphics. However, measuring the architectural parameters for these models at the field and population scales remains prohibitively labor-intensive. We present a novel algorithm that generates the 3D plant architecture from an image, to create a functional structural plant model from an image that reflects organ-level geometric and topological parameters, providing a more comprehensive representation of the plant’s architecture. Instead of using 3D sensors or processing multi-view images with computer vision to obtain the 3D structure of plants, we propose a method that generates token sequences containing a procedural definition of the plant architecture. This work uses only synthetic images for training and testing, where “exact” architectural parameters were known, which allowed for testing of the hypothesis that organ-level architectural parameters could be extracted from imagery data using a vision language model (VLM). A synthetic dataset of cowpea plant images was generated using the Helios 3D plant simulator, with the detailed plant architecture encoded in XML files. We developed a plant architecture tokenizer for the XML file defining plant architecture, converting it into a token sequence that a language model can predict. Then, a VLM was trained to predict plant architecture token sequences from images. Our results demonstrate that the model can predict plant architecture tokens with an F1 score of 0.73 in a teacher-forcing method. Evaluation of the model was performed through autoregressive generation, achieving a BLEU-4 score of 94.00% and a ROUGE-L score of 0.5182. Our model achieves lower MAPE than feature regression-based methods in estimating bulk plant-level traits that require understanding of the occluded 3D structure of the plant, such as leaf count and leaf area. We conclude that generating plant architecture and parameter extraction from synthetic imagery are feasible using a VLM approach, supporting future extension to real imagery.
- Open Access
- 1 May 2026
- Published by Elsevier in Plant Phenomics
- Vol. 8 (3)
Thermal image segmentation in weedy fields via synthetic RGB-trained models and GAN-based cross-modality alignment
- 1 May 2026
- Published by Elsevier in Plant Phenomics
- Vol. 8 (3)
Open AccessThermal image segmentation in weedy fields via synthetic RGB-trained models and GAN-based cross-modality alignment
- Open Access
- 31 October 2025
- Published by Wiley in The Plant Phenome Journal
- Vol. 8 (1)
Leaf hyperspectral reflectance detects pre‐visual stress to Fusarium wilt in strawberries
Fusarium wilt, caused by Fusarium oxysporum , threatens global food security and high‐value crops like strawberries ( Fragaria × ananassa ) in California. Traditional detection, reliant on visual symptoms, often comes too late for intervention. This study uses leaf‐level hyperspectral reflectance to detect physiological changes in resistant and susceptible strawberry cultivars. Weekly measurements of leaf reflectance, stomatal conductance, and chlorophyll fluorescence were collected across 14 cultivars inoculated with the pathogen. We examined both common spectral vegetation indices (SVIs) and patterns across the full hyperspectral range. In addition to SVIs, we assessed the full reflectance space (400–2515 nm) using principal coordinates analysis on Bray–Curtis dissimilarity and calculated coefficient of variation to evaluate spectral sensitivity to disease progression. Susceptible plants showed spectral shifts 3–5 weeks before visible symptoms. The normalized phaeophytinization index was most sensitive, indicating early chlorophyll degradation, while normalized difference vegetation index and photochemical reflectance index captured structural and physiological changes prior to visible infection. Spectral indices outperformed physiological traits, with stronger responses in stomatal conductance and leaf temperature than fluorescence. Hyperspectral disease associations highlight predictive power concentrated in the red and near‐infrared range, highlighting the potential of multispectral tools for field applications.
- 31 October 2025
- Published by Wiley in The Plant Phenome Journal
- Vol. 8 (1)
Open AccessLeaf hyperspectral reflectance detects pre‐visual stress to Fusarium wilt in strawberries
Fusarium wilt, caused by Fusarium oxysporum , threatens global food security and high‐value crops like strawberries ( Fragaria × ananassa ) in California. Traditional detection, reliant on visual symptoms, often comes too late for intervention. This study uses leaf‐level hyperspectral reflectance to detect physiological changes in resistant and susceptible strawberry cultivars. Weekly measurements of leaf reflectance, stomatal conductance, and chlorophyll fluorescence were collected across 14 cultivars inoculated with the pathogen. We examined both common spectral vegetation indices (SVIs) and patterns across the full hyperspectral range. In addition to SVIs, we assessed the full reflectance space (400–2515 nm) using principal coordinates analysis on Bray–Curtis dissimilarity and calculated coefficient of variation to evaluate spectral sensitivity to disease progression. Susceptible plants showed spectral shifts 3–5 weeks before visible symptoms. The normalized phaeophytinization index was most sensitive, indicating early chlorophyll degradation, while normalized difference vegetation index and photochemical reflectance index captured structural and physiological changes prior to visible infection. Spectral indices outperformed physiological traits, with stronger responses in stomatal conductance and leaf temperature than fluorescence. Hyperspectral disease associations highlight predictive power concentrated in the red and near‐infrared range, highlighting the potential of multispectral tools for field applications.
- Open Access
Comparison of Mask-R-CNN and Thresholding-Based Segmentation for High-Throughput Phenotyping of Walnut Kernel Color
High-throughput phenotyping has become essential for plant breeding programs, replacing traditional methods that rely on subjective scales influenced by human judgment. Machine learning (ML) computer vision systems have successfully used convolutional neural networks (CNNs) for image segmentation, providing greater flexibility than thresholding methods that may require carefully staged images. This study compares two quantitative image analysis methods, rule-based thresholding using the magick package in R and an instance-segmentation pipeline based on the widely used Mask-R-CNN architecture, and then compares the output of each to two different sets of human evaluations. Walnuts were collected over three years from over 3000 individual trees maintained by the UC Davis walnut breeding program. The resulting 90,961 kernels were placed into 100-cell trays and imaged using a 20-megapixel Basler camera with a Sony IMX183 sensor. Quantitative data from both image analysis methods were highly correlated for both lightness (L*; r2 = 0.997) and size (r2 = 0.984). The thresholding method required many manual adjustments to account for minor discrepancies in staging, while the CNN method was robust after a rapid initial training on only 13 images. The two human scoring methods were not highly correlated with the image analysis methods or with each other. Pixel classification provides data similar to human color assessments but offers greater consistency across different years. The thresholding approach offers flexibility and has been applied to other color-based phenotyping tasks, while the CNN approach can be adapted to images that are not perfectly staged and be retrained to quantify more subtle kernel characteristics such as spotting and shrivel.
Open AccessComparison of Mask-R-CNN and Thresholding-Based Segmentation for High-Throughput Phenotyping of Walnut Kernel Color
High-throughput phenotyping has become essential for plant breeding programs, replacing traditional methods that rely on subjective scales influenced by human judgment. Machine learning (ML) computer vision systems have successfully used convolutional neural networks (CNNs) for image segmentation, providing greater flexibility than thresholding methods that may require carefully staged images. This study compares two quantitative image analysis methods, rule-based thresholding using the magick package in R and an instance-segmentation pipeline based on the widely used Mask-R-CNN architecture, and then compares the output of each to two different sets of human evaluations. Walnuts were collected over three years from over 3000 individual trees maintained by the UC Davis walnut breeding program. The resulting 90,961 kernels were placed into 100-cell trays and imaged using a 20-megapixel Basler camera with a Sony IMX183 sensor. Quantitative data from both image analysis methods were highly correlated for both lightness (L*; r2 = 0.997) and size (r2 = 0.984). The thresholding method required many manual adjustments to account for minor discrepancies in staging, while the CNN method was robust after a rapid initial training on only 13 images. The two human scoring methods were not highly correlated with the image analysis methods or with each other. Pixel classification provides data similar to human color assessments but offers greater consistency across different years. The thresholding approach offers flexibility and has been applied to other color-based phenotyping tasks, while the CNN approach can be adapted to images that are not perfectly staged and be retrained to quantify more subtle kernel characteristics such as spotting and shrivel.
- Open Access
Enabling Plant Phenotyping in Weedy Environments using Multi-Modal Imagery via Synthetic and Generated Training Data
Accurate plant segmentation in thermal imagery remains a significant challenge for high throughput field phenotyping, particularly in outdoor environments where low contrast between plants and weeds and frequent occlusions hinder performance. To address this, we present a framework that leverages synthetic RGB imagery, a limited set of real annotations, and GAN-based cross-modality alignment to enhance semantic segmentation in thermal images. We trained models on 1,128 synthetic images containing complex mixtures of crop and weed plants in order to generate image segmentation masks for crop and weed plants. We additionally evaluated the benefit of integrating as few as five real, manually segmented field images within the training process using various sampling strategies. When combining all the synthetic images with a few labeled real images, we observed a maximum relative improvement of 22% for the weed class and 17% for the plant class compared to the full real-data baseline. Cross-modal alignment was enabled by translating RGB to thermal using CycleGAN-turbo, allowing robust template matching without calibration. Results demonstrated that combining synthetic data with limited manual annotations and cross-domain translation via generative models can significantly boost segmentation performance in complex field environments for multi-model imagery.
Open AccessEnabling Plant Phenotyping in Weedy Environments using Multi-Modal Imagery via Synthetic and Generated Training Data
Accurate plant segmentation in thermal imagery remains a significant challenge for high throughput field phenotyping, particularly in outdoor environments where low contrast between plants and weeds and frequent occlusions hinder performance. To address this, we present a framework that leverages synthetic RGB imagery, a limited set of real annotations, and GAN-based cross-modality alignment to enhance semantic segmentation in thermal images. We trained models on 1,128 synthetic images containing complex mixtures of crop and weed plants in order to generate image segmentation masks for crop and weed plants. We additionally evaluated the benefit of integrating as few as five real, manually segmented field images within the training process using various sampling strategies. When combining all the synthetic images with a few labeled real images, we observed a maximum relative improvement of 22% for the weed class and 17% for the plant class compared to the full real-data baseline. Cross-modal alignment was enabled by translating RGB to thermal using CycleGAN-turbo, allowing robust template matching without calibration. Results demonstrated that combining synthetic data with limited manual annotations and cross-domain translation via generative models can significantly boost segmentation performance in complex field environments for multi-model imagery.
- Open Access
California Crop Yield Benchmark: Combining Satellite Image, Climate, Evapotranspiration, and Soil Data Layers for County-Level Yield Forecasting of Over 70 Crops
California is a global leader in agricultural production, contributing 12.5% of the United States total output and ranking as the fifth-largest food and cotton supplier in the world. Despite the availability of extensive historical yield data from the USDA National Agricultural Statistics Service, accurate and timely crop yield forecasting remains a challenge due to the complex interplay of environmental, climatic, and soil-related factors. In this study, we introduce a comprehensive crop yield benchmark dataset covering over 70 crops across all California counties from 2008 to 2022. The benchmark integrates diverse data sources, including Landsat satellite imagery, daily climate records, monthly evapotranspiration, and high-resolution soil properties. To effectively learn from these heterogeneous inputs, we develop a multi-modal deep learning model tailored for county-level, crop-specific yield forecasting. The model employs stratified feature extraction and a timeseries encoder to capture spatial and temporal dynamics during the growing season. Static inputs such as soil characteristics and crop identity inform long-term variability. Our approach achieves an overall R2 score of 0.76 across all crops of unseen test dataset, highlighting strong predictive performance across California diverse agricultural regions. This benchmark and modeling framework offer a valuable foundation for advancing agricultural forecasting, climate adaptation, and precision farming. The full dataset and codebase are publicly available at our GitHub repository.
Open AccessCalifornia Crop Yield Benchmark: Combining Satellite Image, Climate, Evapotranspiration, and Soil Data Layers for County-Level Yield Forecasting of Over 70 Crops
California is a global leader in agricultural production, contributing 12.5% of the United States total output and ranking as the fifth-largest food and cotton supplier in the world. Despite the availability of extensive historical yield data from the USDA National Agricultural Statistics Service, accurate and timely crop yield forecasting remains a challenge due to the complex interplay of environmental, climatic, and soil-related factors. In this study, we introduce a comprehensive crop yield benchmark dataset covering over 70 crops across all California counties from 2008 to 2022. The benchmark integrates diverse data sources, including Landsat satellite imagery, daily climate records, monthly evapotranspiration, and high-resolution soil properties. To effectively learn from these heterogeneous inputs, we develop a multi-modal deep learning model tailored for county-level, crop-specific yield forecasting. The model employs stratified feature extraction and a timeseries encoder to capture spatial and temporal dynamics during the growing season. Static inputs such as soil characteristics and crop identity inform long-term variability. Our approach achieves an overall R2 score of 0.76 across all crops of unseen test dataset, highlighting strong predictive performance across California diverse agricultural regions. This benchmark and modeling framework offer a valuable foundation for advancing agricultural forecasting, climate adaptation, and precision farming. The full dataset and codebase are publicly available at our GitHub repository.
- Open Access
AgRowStitch: A High-fidelity Image Stitching Pipeline for Ground-based Agricultural Images
Agricultural imaging often requires individual images to be stitched together into a final mosaic for analysis. However, agricultural images can be particularly challenging to stitch because feature matching across images is difficult due to repeated textures, plants are non-planar, and mosaics built from many images can accumulate errors that cause drift. Although these issues can be mitigated by using georeferenced images or taking images at high altitude, there is no general solution for images taken close to the crop. To address this, we created a user-friendly and open source pipeline for stitching ground-based images of a linear row of crops that does not rely on additional data. First, we use SuperPoint and LightGlue to extract and match features within small batches of images. Then we stitch the images in each batch in series while imposing constraints on the camera movement. After straightening and rescaling each batch mosaic, all batch mosaics are stitched together in series and then straightened into a final mosaic. We tested the pipeline on images collected along 72 m long rows of crops using two different agricultural robots and a camera manually carried over the row. In all three cases, the pipeline produced high-quality mosaics that could be used to georeference real world positions with a mean absolute error of 20 cm. This approach provides accessible leaf-scale stitching to users who need to coarsely georeference positions within a row, but do not have access to accurate positional data or sophisticated imaging systems.
Open AccessAgRowStitch: A High-fidelity Image Stitching Pipeline for Ground-based Agricultural Images
Agricultural imaging often requires individual images to be stitched together into a final mosaic for analysis. However, agricultural images can be particularly challenging to stitch because feature matching across images is difficult due to repeated textures, plants are non-planar, and mosaics built from many images can accumulate errors that cause drift. Although these issues can be mitigated by using georeferenced images or taking images at high altitude, there is no general solution for images taken close to the crop. To address this, we created a user-friendly and open source pipeline for stitching ground-based images of a linear row of crops that does not rely on additional data. First, we use SuperPoint and LightGlue to extract and match features within small batches of images. Then we stitch the images in each batch in series while imposing constraints on the camera movement. After straightening and rescaling each batch mosaic, all batch mosaics are stitched together in series and then straightened into a final mosaic. We tested the pipeline on images collected along 72 m long rows of crops using two different agricultural robots and a camera manually carried over the row. In all three cases, the pipeline produced high-quality mosaics that could be used to georeference real world positions with a mean absolute error of 20 cm. This approach provides accessible leaf-scale stitching to users who need to coarsely georeference positions within a row, but do not have access to accurate positional data or sophisticated imaging systems.
- Open Access
iNatAg: Multi-Class Classification Models Enabled by a Large-Scale Benchmark Dataset with 4.7M Images of 2,959 Crop and Weed Species
Accurate identification of crop and weed species is critical for precision agriculture and sustainable farming. However, it remains a challenging task due to a variety of factors -- a high degree of visual similarity among species, environmental variability, and a continued lack of large, agriculture-specific image data. We introduce iNatAg, a large-scale image dataset which contains over 4.7 million images of 2,959 distinct crop and weed species, with precise annotations along the taxonomic hierarchy from binary crop/weed labels to specific species labels. Curated from the broader iNaturalist database, iNatAg contains data from every continent and accurately reflects the variability of natural image captures and environments. Enabled by this data, we train benchmark models built upon the Swin Transformer architecture and evaluate the impact of various modifications such as the incorporation of geospatial data and LoRA finetuning. Our best models achieve state-of-the-art performance across all taxonomic classification tasks, achieving 92.38\% on crop and weed classification. Furthermore, the scale of our dataset enables us to explore incorrect misclassifications and unlock new analytic possiblities for plant species. By combining large-scale species coverage, multi-task labels, and geographic diversity, iNatAg provides a new foundation for building robust, geolocation-aware agricultural classification systems. We release the iNatAg dataset publicly through AgML (https://github.com/Project-AgML/AgML), enabling direct access and integration into agricultural machine learning workflows.
Open AccessiNatAg: Multi-Class Classification Models Enabled by a Large-Scale Benchmark Dataset with 4.7M Images of 2,959 Crop and Weed Species
Accurate identification of crop and weed species is critical for precision agriculture and sustainable farming. However, it remains a challenging task due to a variety of factors -- a high degree of visual similarity among species, environmental variability, and a continued lack of large, agriculture-specific image data. We introduce iNatAg, a large-scale image dataset which contains over 4.7 million images of 2,959 distinct crop and weed species, with precise annotations along the taxonomic hierarchy from binary crop/weed labels to specific species labels. Curated from the broader iNaturalist database, iNatAg contains data from every continent and accurately reflects the variability of natural image captures and environments. Enabled by this data, we train benchmark models built upon the Swin Transformer architecture and evaluate the impact of various modifications such as the incorporation of geospatial data and LoRA finetuning. Our best models achieve state-of-the-art performance across all taxonomic classification tasks, achieving 92.38\% on crop and weed classification. Furthermore, the scale of our dataset enables us to explore incorrect misclassifications and unlock new analytic possiblities for plant species. By combining large-scale species coverage, multi-task labels, and geographic diversity, iNatAg provides a new foundation for building robust, geolocation-aware agricultural classification systems. We release the iNatAg dataset publicly through AgML (https://github.com/Project-AgML/AgML), enabling direct access and integration into agricultural machine learning workflows.
- Open Access
CMAViT: Integrating Climate, Managment, and Remote Sensing Data for Crop Yield Estimation with Multimodel Vision Transformers
Crop yield prediction is essential for agricultural planning but remains challenging due to the complex interactions between weather, climate, and management practices. To address these challenges, we introduce a deep learning-based multi-model called Climate-Management Aware Vision Transformer (CMAViT), designed for pixel-level vineyard yield predictions. CMAViT integrates both spatial and temporal data by leveraging remote sensing imagery and short-term meteorological data, capturing the effects of growing season variations. Additionally, it incorporates management practices, which are represented in text form, using a cross-attention encoder to model their interaction with time-series data. This innovative multi-modal transformer tested on a large dataset from 2016-2019 covering 2,200 hectares and eight grape cultivars including more than 5 million vines, outperforms traditional models like UNet-ConvLSTM, excelling in spatial variability capture and yield prediction, particularly for extreme values in vineyards. CMAViT achieved an R2 of 0.84 and a MAPE of 8.22% on an unseen test dataset. Masking specific modalities lowered performance: excluding management practices, climate data, and both reduced R2 to 0.73, 0.70, and 0.72, respectively, and raised MAPE to 11.92%, 12.66%, and 12.39%, highlighting each modality's importance for accurate yield prediction. Code is available at https://github.com/plant-ai-biophysics-lab/CMAViT.
Open AccessCMAViT: Integrating Climate, Managment, and Remote Sensing Data for Crop Yield Estimation with Multimodel Vision Transformers
Crop yield prediction is essential for agricultural planning but remains challenging due to the complex interactions between weather, climate, and management practices. To address these challenges, we introduce a deep learning-based multi-model called Climate-Management Aware Vision Transformer (CMAViT), designed for pixel-level vineyard yield predictions. CMAViT integrates both spatial and temporal data by leveraging remote sensing imagery and short-term meteorological data, capturing the effects of growing season variations. Additionally, it incorporates management practices, which are represented in text form, using a cross-attention encoder to model their interaction with time-series data. This innovative multi-modal transformer tested on a large dataset from 2016-2019 covering 2,200 hectares and eight grape cultivars including more than 5 million vines, outperforms traditional models like UNet-ConvLSTM, excelling in spatial variability capture and yield prediction, particularly for extreme values in vineyards. CMAViT achieved an R2 of 0.84 and a MAPE of 8.22% on an unseen test dataset. Masking specific modalities lowered performance: excluding management practices, climate data, and both reduced R2 to 0.73, 0.70, and 0.72, respectively, and raised MAPE to 11.92%, 12.66%, and 12.39%, highlighting each modality's importance for accurate yield prediction. Code is available at https://github.com/plant-ai-biophysics-lab/CMAViT.