Andrew Siefert

The proliferation of high‐dimensional data in ecology and evolutionary biology raises the promise of statistical and machine learning models that are highly predictive and interpretable. However, high‐dimensional data are commonly burdened with an inherent trade‐off: in‐sample prediction of outcomes will improve as additional variables are included in the model, but this may come at the cost of poor predictive accuracy and limited generalizability for future or unsampled observations (out‐of‐sample prediction). To confront this problem of overfitting, sparse models can focus on key variables by correctly placing low weight on unimportant variables. We compared nine methods to quantify their performance in variable selection and prediction using simulated data with different sample sizes, numbers of variables, and strengths of effects. Overfitting was typical for many methods and simulation scenarios. Despite this, in‐sample and out‐of‐sample prediction converged on the true predictive target for simulations with more observations, larger causal effects, and fewer variables. Accurate variable selection to support process‐based understanding will be unattainable for many realistic sampling schemes in ecology and evolution. We use our analyses to characterize data attributes for which statistical learning is possible, and illustrate how some sparse methods can achieve predictive accuracy while mitigating and learning the extent of overfitting.

Summary: Evolutionary radiations of woody taxa within arid environments were made possible by multiple trait innovations including deep roots and embolism‐resistant xylem, but little is known about how these traits have coevolved across the phylogeny of woody plants or how they jointly influence the distribution of species. We synthesized global trait and vegetation plot datasets to examine how rooting depth and xylem vulnerability across 188 woody plant species interact with aridity, precipitation seasonality, and water table depth to influence species occurrence probabilities across all biomes. Xylem resistance to embolism and rooting depth are independent woody plant traits that do not exhibit an interspecific trade‐off. Resistant xylem and deep roots increase occurrence probabilities in arid, seasonal climates over deep water tables. Resistant xylem and shallow roots increase occurrence probabilities in arid, nonseasonal climates over deep water tables. Vulnerable xylem and deep roots increase occurrence probabilities in arid, nonseasonal climates over shallow water tables. Lastly, vulnerable xylem and shallow roots increase occurrence probabilities in humid climates. Each combination of trait values optimizes occurrence probabilities in unique environmental conditions. Responses of deeply rooted vegetation may be buffered if evaporative demand changes faster than water table depth under climate change.

Functional traits affect the demographic performance of individuals in their environment, leading to fitness differences that scale up to drive population dynamics and community assembly. Understanding the links between traits and fitness is, therefore, critical for predicting how populations and communities respond to environmental change. However, the net effects of traits on species fitness are largely unknown because we have lacked a framework for estimating fitness across multiple species and environments. We present a modelling framework that integrates trait effects on demographic performance over the life cycles of individuals to estimate the net effect of traits on species fitness. This approach involves (1) modelling trait effects on individual demographic rates (growth, survival and recruitment) as multidimensional performance surfaces that vary with individual size and environment and (2) integrating these effects into a population model to project population growth rates (i.e., fitness) as a function of traits and environment. We illustrate our approach by estimating performance surfaces and fitness landscapes for trees across a temperature gradient in the eastern United States. Functional traits (wood density, specific leaf area and maximum height) interacted with individual size and temperature to influence tree growth, survival and recruitment rates, generating demographic trade‐offs and shaping the contours of fitness landscapes. Tall tree species had high survival, growth and fitness across the temperature gradient. Wood density and specific leaf area had interactive effects on demographic performance, resulting in fitness landscapes with multiple peaks. With this approach it is now possible to empirically estimate the net effect of traits on fitness, leading to an improved understanding of the selective forces that drive community assembly and permitting generalizable predictions of population and community dynamics in changing environments.

Making predictions about species, including how they respond to environmental change, is a central challenge for ecologists. Due to the huge number of species, ecologists seek generalizations based on species’ traits and phylogenetic relationships, but the predictive power of trait-based and phylogenetic models is often low. Species co-occurrence patterns may contain additional information about species’ ecological attributes not captured by traits or phylogenies. We propose using ordination to encode the information contained in species co-occurrence data in low-dimensional vectors that can be used to represent species in ecological prediction. We present an efficient method to derive species vectors from co-occurrence data using GloVe, an unsupervised learning algorithm originally designed for language modeling. To demonstrate the method, we used GloVe to generate vectors for nearly 40,000 plant species using co-occurrence statistics derived from a global vegetation dataset and tested their ability to predict elevational range shifts in European montane plant species. Co-occurrence-based species vectors were weakly correlated with traits or phylogeny, indicating that they encode unique information about species. Models including co-occurrence-based vectors explained twice as much variation in species range shifts as models including only traits or phylogenetic information. Given the widespread availability of species occurrence data, species vectors learned from co-occurrence patterns are a widely applicable and powerful tool for encoding ecological information about species, with many potential applications for describing and predicting the ecology of species, communities, and ecosystems.

Ecological responses to environmental change are driven by population demography^1,2, yet trait-based population dynamics are rarely used to project community assemblages³. Fitness landscapes describe how selection influences the evolution of species^4,5 and can be extended to multispecies communities along environmental gradients based on the matching of traits to local environmental conditions⁶. Knowing the contours of fitness landscapes would improve our understanding of how selection shapes ecological communities and would permit predictions of community dynamics in changing environments. However, the shape of fitness landscapes for multispecies communities is unknown because we have lacked a framework for estimating fitness across individuals, phenotypes, and environments. Here we describe a new framework to estimate multispecies fitness landscapes and illustrate model results for trees in temperate forests across a climatic gradient in the eastern United States. Demographic models showed that multivariate phenotypes interact with climate to influence rates of tree growth, survival, and reproduction and generate demographic trade-offs that constrain life history strategies. By integrating trait effects on demographic performance across the life cycle, we estimated the net effect of traits on fitness and reveal the shapes of fitness landscapes driven by temperature. With increasing availability of trait⁷ and demographic data^8,9, our approach can be applied to any ecological system, providing a pathway to generate mechanistic, general predictions of population and community responses to environmental change.