A vision language model for generating XML-based organ-level plant architecture representations of cowpea from simulated images
Open Access
- 20 May 2026
- journal article
- research article
- Published by Frontiers Media SA in Frontiers in Artificial Intelligence
- Vol. 9, 1844338
Abstract
Three-dimensional (3D) procedural plant architecture models have emerged as an important tool for simulation-based studies of plant structure and function, extracting plant architectural parameters from field measurements, and for generating realistic plants in computer graphics. However, measuring the architectural parameters for these models at the field and population scales remains prohibitively labor-intensive. We present a novel algorithm that generates the 3D plant architecture from an image, to create a functional structural plant model from an image that reflects organ-level geometric and topological parameters, providing a more comprehensive representation of the plant’s architecture. Instead of using 3D sensors or processing multi-view images with computer vision to obtain the 3D structure of plants, we propose a method that generates token sequences containing a procedural definition of the plant architecture. This work uses only synthetic images for training and testing, where “exact” architectural parameters were known, which allowed for testing of the hypothesis that organ-level architectural parameters could be extracted from imagery data using a vision language model (VLM). A synthetic dataset of cowpea plant images was generated using the Helios 3D plant simulator, with the detailed plant architecture encoded in XML files. We developed a plant architecture tokenizer for the XML file defining plant architecture, converting it into a token sequence that a language model can predict. Then, a VLM was trained to predict plant architecture token sequences from images. Our results demonstrate that the model can predict plant architecture tokens with an F1 score of 0.73 in a teacher-forcing method. Evaluation of the model was performed through autoregressive generation, achieving a BLEU-4 score of 94.00% and a ROUGE-L score of 0.5182. Our model achieves lower MAPE than feature regression-based methods in estimating bulk plant-level traits that require understanding of the occluded 3D structure of the plant, such as leaf count and leaf area. We conclude that generating plant architecture and parameter extraction from synthetic imagery are feasible using a VLM approach, supporting future extension to real imagery.Keywords
All Related Versions
This publication has 36 references indexed in Scilit:
- Creating synthetic data sets for training of neural networks for automatic catch analysis in fisheriesComputers and Electronics in Agriculture, 2025
- Maize leaf angle genetic gain is slowing down in the last decadesCrop Science, 2023
- Screening of cowpea (Vigna unguiculata (L.) Walp.) genotypes for waterlogging tolerance using morpho-physiological traits at early growth stagePlant Science, 2022
- Deep learning and citizen science enable automated plant trait predictions from photographsScientific Reports, 2021
- Assessing diversity in canopy architecture, photosynthesis, and water‐use efficiency in a cowpea magic populationFood and Energy Security, 2020
- Rapid, high-resolution measurement of leaf area and leaf orientation using terrestrial LiDAR scanning dataMeasurement Science and Technology, 2017
- Functional–structural plant models: a growing paradigm for plant studiesAnnals of Botany, 2014
- Long‐Term Selection in a Commercial Hybrid Maize Breeding Program2003
- Mathematical models for cellular interactions in development II. Simple and branching filaments with two-sided inputsJournal of Theoretical Biology, 1968
- Mathematical models for cellular interactions in development I. Filaments with one-sided inputsJournal of Theoretical Biology, 1968