Thomas Martin Lange

Provides methods to analyse the stability of non-deterministic prediction models. Prediction stability is quantified either as data-based prediction stability (phi) or as model-based prediction stability (psi). The package implements measures for categorical, ordinal, and metric predictions based on repeated model fitting and corresponding predictions. Methods are based on Lange et al. (2025) .

Background: Genomic selection relies on a variety of statistical and machine learning methods to predict phenotypes from genomic data. Since no single method consistently outperforms others across datasets, evaluating and comparing model performance is essential. However, standard evaluation metrics such as Pearson’s correlation coefficient and mean squared error treat genomic prediction as a regression problem, assessing overall fit rather than the effectiveness of selecting top-performing individuals for breeding. This disconnect can lead to suboptimal model selection in practice. Results: To address this, we present the normalized cumulative gain (NCG) as an alternative evaluation measure that directly measures the phenotypic gain achieved from the individuals selected by the model. We applied this measure on four animal and plant datasets to compare nine commonly used methods for genomic prediction. Conclusions: NCG offers an intuitive and interpretable measure of selection efficiency, focusing solely on the individuals that would actually be chosen. We further demonstrate that calculating the performance under all possible selection thresholds provides more information than a single or few arbitrary thresholds. This more granular analysis shows that the performance of the methods may differ under varying selection intensities and can provide guidance for the choice of selection intensity. Our approach is implemented in R and is available at https://github.com/FelixHeinrich/GS_Comparison_with_NCG .

Machine learning is frequently used to make decisions based on big data. Among these techniques, random forest is particularly prominent. Although random forest is known to have many advantages, one aspect that is often overseen is that it is a non-deterministic method that can produce different models using the same input data. This can have severe consequences on decision-making processes. In this study, we introduce a method to quantify the impact of non-determinism on predictions, variable importance estimates, and decisions based on the predictions or variable importance estimates. Our findings demonstrate that increasing the number of trees in random forests enhances the stability in a non-linear way while computation time increases linearly. Consequently, we conclude that there exists an optimal number of trees for any given data set that maximises the stability without unnecessarily increasing the computation time. Based on these findings, we have developed the R package optRF which models the relationship between the number of trees and the stability of random forest, providing recommendations for the optimal number of trees for any given data set.

Background Rhizomania counts as the most important disease in sugar beet Beta vulgaris L. for which no plant protection is available, leaving plant breeding as the only defence strategy at the moment. Five resistance genes have been detected on the same chromosome and further studies suggested that these might be different alleles at two resistance clusters. Nevertheless, it was postulated that rhizomania resistance might be a quantitative trait with multiple unknown minor resistance genes. Here, we present a first attempt at genomic prediction of rhizomania resistance in a population that carries resistances at the two known resistance clusters. The sugar beet population was genotyped using single nucleotide polymorphism (SNP) markers. Methods First, genomic prediction was performed using all SNPs. Next, we calculated the variable importance for each SNP using machine learning and performed genomic prediction by including the SNPs incrementally in the prediction model based on their variable importance. Using this method, we selected the optimal number of SNPs that maximised the prediction accuracy. Furthermore, we performed genomic prediction with SNP pairs. We also performed feature selection with SNP pairs using the information about the variable importance of the single SNPs. Results From the four methods under investigation, the latter led to the highest prediction accuracy. These results lead to the conclusion that more than the two known resistance clusters are involved in rhizomania resistance and that genetic interactions affect rhizomania resistance. Finally, we have analysed which SNPs were repeatedly detected in the feature selection process and discovered four SNPs, two of which are located on chromosomes that were previously not associated with rhizomania resistance.

Background: Rhizomania counts as the most important disease in sugar beet Beta vulgaris L. for which no plant protection is available, leaving plant breeding as the only defence strategy at the moment. Five resistance genes have been detected on the same chromosome and further studies suggested that these might be different alleles at two resistance clusters. Nevertheless, it was postulated that rhizomania resistance might be a quantitative trait with multiple unknown minor resistance genes. Here, we present a first attempt at genomic prediction of rhizomania resistance in a population that was genotyped using single nucleotide polymorphism (SNP) markers. Methods: First, genomic prediction was performed using all SNPs. Next, we calculated the variable importance for each SNP using machine learning and performed genomic prediction by including the SNPs incrementally in the prediction model based on their variable importance. Using this method, we selected the optimal number of SNPs that maximised the prediction accuracy. Furthermore, we performed genomic prediction with SNP pairs. We also performed feature selection with SNP pairs using the information about the variable importance of the single SNPs. Results: From the four methods under investigation, the latter led to the highest prediction accuracy. These results lead to the following conclusions: (I) The genotypes that were resistant at all known resistance genes, provided the highest possible variation of virus concentrations that the machine can measure. Thus, it can be assumed that more genes must be involved in the resistance towards rhizomania. (II) We show that prediction models that include SNP interactions increased the prediction accuracy. Conclusions: Altogether, our findings suggest that rhizomania resistance is a complex quantitative trait that is affected by multiple genes as well as their interaction.

Background: In research questions such as in resistance breeding against theBeet necrotic yellow vein virusit is of interest to compare the virus concentrations of samples from different groups. The enzyme-linked immunosorbent assay (ELISA) counts as the standard tool to measure virus concentrations. Simple methods for data analysis such as analysis of variance (ANOVA), however, are impaired due to non-normality of the resulting optical density (OD) values as well as unequal variances in different groups.Methods: To understand the relationship between the OD values from an ELISA test and the virus concentration per sample, we used a large serial dilution and modelled its non-linear form using a five parameter logistic regression model. Furthermore, we examined if the quality of the model can be increased if one or several of the model parameters are defined beforehand. Subsequently, we used the inverse of the best model to estimate the virus concentration for every measured OD value.Results: We show that the transformed data are essentially normally distributed but provide unequal variances per group. Thus, we propose a generalised least squares model which allows for unequal variances of the groups to analyse the transformed data.Conclusions: ANOVA requires normally distributed data as well as equal variances. Both requirements are not met with raw OD values from an ELISA test. A transformation with an inverse logistic function, however, gives the possibility to use linear models for data analysis of virus concentrations. We conclude that this method can be applied in every trial where virus concentrations of samples from different groups are to be compared via OD values from an ELISA test. To encourage researchers to use this method in their studies, we provide an R script for data transformation as well as the data from our trial.

The Beet necrotic yellow vein virus (BNYVV) causes rhizomania in sugar beet (Beta vulgaris L.), which is one of the most destructive diseases in sugar beet worldwide. In breeding projects towards resistance against BNYVV, the enzyme-linked immunosorbent assay (ELISA) is used to determine the virus concentration in plant roots and, thus, the resistance levels of genotypes. Here, we present a simulation study to generate 10,000 small samples from the estimated density functions of ELISA values from susceptible and resistant sugar beet genotypes. We apply receiver operating characteristic (ROC) analysis to these samples to optimise the cutoff values for sample sizes from two to eight and determine the false positive rates (FPR), true positive rates (TPR), and area under the curve (AUC). We present, furthermore, an alternative approach based upon Bayes factors to improve the decision procedure. The Bayesian approach has proven to be superior to the simple cutoff approach. The presented results could help evaluate or improve existing breeding programs and help design future selection procedures based upon ELISA. An R-script for the classification of sample data based upon Bayes factors is provided.

Regulatory SNPs (rSNPs) are a special class of SNPs which have a high potential to affect the phenotype due to their impact on DNA-binding of transcription factors (TFs). Thus, the knowledge about such rSNPs and TFs could provide essential information regarding different genetic programs, such as tissue development or environmental stress responses. In this study, we use a multi-omics approach by combining genomics, transcriptomics, and proteomics data of two different Brassica napus L. cultivars, namely Zhongshuang11 (ZS11) and Zhongyou821 (ZY821), with high and low oil content, respectively, to monitor the regulatory interplay between rSNPs, TFs and their corresponding genes in the tissues flower, leaf, stem, and root. By predicting the effect of rSNPs on TF-binding and by measuring their association with the cultivars, we identified a total of 41,117 rSNPs, of which 1141 are significantly associated with oil content. We revealed several enriched members of the TF families DOF, MYB, NAC, or TCP, which are important for directing transcriptional programs regulating differential expression of genes within the tissues. In this work, we provide the first genome-wide collection of rSNPs for B. napus and their impact on the regulation of gene expression in vegetative and floral tissues, which will be highly valuable for future studies on rSNPs and gene regulation.