Alexandra Getmanskaya

This work is mostly devoted to the search for effective solutions to the problem of early diagnosis of plant stress (given an example of wheat and its drought stress), which would be based on explainable artificial intelligence (XAI). The main idea is to combine the benefits of two of the most popular agricultural data sources, hyperspectral images (HSI) and thermal infrared images (TIR), in a single XAI model. Our own dataset of a 25-day experiment was used, which was created via both (1) an HSI camera Specim IQ (400–1000 nm, 204, 512 × 512) and (2) a TIR camera Testo 885-2 (320 × 240, res. 0.1 °C). The HSI were a source of the k-dimensional high-level features of plants (k ≤ K, where K is the number of HSI channels) for the learning process. Such combination was implemented as a single-layer perceptron (SLP) regressor, which is the main feature of the XAI model and receives as input an HSI pixel-signature belonging to the plant mask, which then automatically through the mask receives a mark from the TIR. The correlation of HSI channels with the TIR image on the plant’s mask on the days of the experiment was studied. It was established that HSI channel 143 (820 nm) was the most correlated with TIR. The problem of training the HSI signatures of plants with their corresponding temperature value via the XAI model was solved. The RMSE of plant temperature prediction is 0.2–0.3 °C, which is acceptable for early diagnostics. Each HSI pixel was represented in training by a number (k) of channels (k ≤ K = 204 in our case). The number of channels used for training was minimized by a factor of 25–30, from 204 to eight or seven, while maintaining the RMSE value. The model is computationally efficient in training; the average training time was much less than one minute (Intel Core i3-8130U, 2.2 GHz, 4 cores, 4 GB). This XAI model can be considered a research-aimed model (R-XAI), which allows the transfer of knowledge about plants from the TIR domain to the HSI domain, with their contrasting onto only a few from hundreds of HSI channels.

The work is devoted to the search for effective solutions to the applied problem of early diagnostics of plant stress in the conditions of smart farming and based on modern explicable artificial intelligence (XAI). The study mostly oriented on the theory and practice of XAI, focused on the use of hyperspectral imagery (HSI) and Thermal Infra-Red (TIR) sensor data at the input of a neural network. The first our goal is to build an XAI neural network, explainable due to its structure, the input of which is a datascientist oriented HSI 'explanator', and the output is a biologist oriented TIR 'explanator'. In the middle is SLP-regressor which solves the universal problem of training HSI pixels to temperatures of plants, needed for early plant stress diagnostic. The result can be considered as prototype of a special XAI explanator which is assigned to transform explanator specialized on area 1 onto explanator specialized on area 2. Using this HSI-TIR explanator we ensured the follows: extend HSI data by TIR attribute; providing TIR data for early diagnostic of plant stress; reducing dimensionality HSI needed for TIR training 25 times (from 204 to 8) preserving the same accuracy of temperature prediction (RMSE=0.2-0.3C). This reducing was achieved without using PCA methods. The constructed model is computationally efficient in training: the average training time is significantly less then 1 min (Intel Core i3-8130U, 2.2 GHz, 4 cores, 4 GB). One of the 8 channels, 820 nm, is the leader in correlation with TIR, what allows building local linear temperature prediction functions.

This article is devoted to searching for high-level explainable features that can remain explainable for a wide class of objects or phenomena and become an integral part of explainable AI (XAI). The present study involved a 25-day experiment on early diagnosis of wheat stress using drought stress as an example. The state of the plants was periodically monitored via thermal infrared (TIR) and hyperspectral image (HSI) cameras. A single-layer perceptron (SLP)-based classifier was used as the main instrument in the XAI study. To provide explainability of the SLP input, the direct HSI was replaced by images of six popular vegetation indices and three HSI channels (R₆₃₀, G₅₅₀, and B₄₈₀; referred to as indices), along with the TIR image. Furthermore, in the explainability analysis, each of the 10 images was replaced by its 6 statistical features: min, max, mean, std, max–min, and the entropy. For the SLP output explainability, seven output neurons corresponding to the key states of the plants were chosen. The inner layer of the SLP was constructed using 15 neurons, including 10 corresponding to the indices and 5 reserved neurons. The classification possibilities of all 60 features and 10 indices of the SLP classifier were studied. Study result: Entropy is the earliest high-level stress feature for all indices; entropy and an entropy-like feature (max–min) paired with one of the other statistical features can provide, for most indices, 100% accuracy (or near 100%), serving as an integral part of XAI.

The article is devoted to solving the problem of searching for universal explainable features that can remain explainable for a wide class of objects and phenomena and become an integral part of Explainable AI (XAI). The study is implemented on the example of an applied problem of early diagnostics of plant stress, using Thermal IR (TIR) and HSI, presented by 8 vegetation indices/channels. Each such index was presented by 5 statistical values. A Single-Layer-Perceptron classifier was used as the main instrument. TIR turned out to be the best of the indices in terms of efficiency in the field and sufficient to detect all 7 key days with 100% accuracy. Our study shows also that there are a number of indices, inluding NDVI, and usual color channels Red, Green, Blue, which are close to TIR possibilities in early plant stress detection with 100% accurasy or near, and can be used for wide class of plants and in different conditions their treatment. The stability of the stress classification in our study was maintained when the training set was reduced up to 10% of the dataset volume. The entropy-like feature of (max-min) for any indices/channels have determined as a leadersheep universal high-level explainable feature for the plant stress detection, which used in interaction with some of other statistical features.