Eduardo Soares

We present PDE-FM, a modular foundation model for physics-informed machine learning that unifies spatial, spectral, and temporal reasoning across heterogeneous partial differential equation (PDE) systems. PDE-FM combines spatial-spectral tokenization, physics-aware conditioning, and a Mamba-based state-space backbone with an operator-theoretic decoder, enabling scalable and data-efficient modeling of complex physical dynamics. In contrast to task-specific neural operators, PDE-FM is pretrained once on diverse PDE datasets and can be transferred to new physical regimes without architectural or data-specific modifications. Evaluated on twelve 2D and 3D datasets from The Well benchmark - spanning hydrodynamic, radiative, elastic, and astrophysical phenomena - PDE-FM achieves state-of-the-art accuracy in six domains, reducing mean VRMSE by 46% relative to prior operator-learning baselines. The model demonstrates robust cross-physics generalization, excelling in turbulent and radiative systems while maintaining strong performance in linear and steady-state regimes. These results suggest that large-scale pretraining across diverse physical processes can yield transferable representations of dynamics, marking a step toward unified, foundation-level surrogates for multi-physics simulation and scientific discovery.

The use of foundation models has extended from natural language processing to molecular modeling. In this context, large-scale pre-training strategies have been applied to chemical language models to enable representation learning across diverse tasks. Here we introduce a family of encoder-decoder chemical foundation models pre-trained on a curated dataset of 91 million molecular sequences from PubChem. These models support a range of applications, including property estimation and reaction outcome prediction. We evaluate two model variants across several benchmark datasets and show that they match or exceed existing approaches. We also assess the structure of the learned representations and find that the embedding space supports few-shot learning and separates molecules based on chemically relevant features. This structure appears to result from the decoder-based reconstruction objective used during pre-training. These findings suggest that the proposed models can serve as general-purpose tools for molecular analysis and reasoning with minimal supervision.

Chemical foundation models are generally based on a two-step approach-pre-training on unlabeled data followed by fine-tuning on specific tasks-to boost model capacity. With the increasing demand for training efficiency, mixture-of-experts (MoEs) has become essential for scaling large models by selectively activating sub-networks of experts through a gating network, thereby optimizing performance. This paper presents molecular MoL-MoE, a multi-view MoEs framework designed to predict molecular properties by integrating latent spaces derived from SMILES, SELFIES, and molecular graphs. Our approach leverages the complementary strengths of these representations to enhance predictive accuracy. Here, we evaluate the performance of MoL-MoE with a total of 12 experts, organized into 4 experts for each modality (SMILES, SELFIES, and molecular graphs). We evaluate MoL-MoE on a range of benchmark datasets from MoleculeNet, demonstrating its superior performance compared to state-of-the-art methods across all nine datasets considering two different routing activation settings: k = 4 and k = 6. The results underscore the model’s robustness and adaptability in handling various complex molecular prediction tasks. Our analysis of routing activation patterns reveals that MoL-MoE dynamically adjusts its use of different molecular representations based on task-specific requirements. This adaptability highlights the importance of representation choice in optimizing model performance. Source code for MoL-MoE is available at https://github.com/IBM/materials/tree/main/models/mol_moe.

Recent advancements in large foundation models have revealed impressive capabilities in mastering complex chemical language representations. These models undergo a task-agnostic learning phase, characterized by pre-training on extensive unlabeled corpora followed by fine-tuning on specific downstream tasks. This methodology reduces reliance on labeled data, facilitating data acquisition and broadening the scope of chemical language representation. However, real-world scenarios often pose challenges due to domain shift, a phenomenon where the data distribution in downstream tasks differs from that of the pre-training phase, potentially degrading model performance. To address this, we present a novel causal-based framework for feature selection and domain adaptation to enhance the performance of chemical foundation models on downstream tasks. Our approach employs a multi-stage feature selection method that identifies physico-chemical features based on their direct causal-effect over specific downstream properties. By employing Mordred descriptors and Markov blanket causal graphs, our approach provides insight into the causal relationships between features and target properties for prediction tasks. We evaluate our approach on various foundation model architectures and datasets, demonstrating performance improvements, which showcases the robustness and the agnostic nature of our approach.

Large-scale molecular representation methods have revolutionized applications in material science, such as drug discovery, chemical modeling, and material design. With the rise of transformers, models now learn representations directly from molecular structures. In this study, we develop an encoder-decoder model based on BART that is capable of leaning molecular representations and generate new molecules. Trained on SELFIES, a robust molecular string representation, our model outperforms existing baselines in downstream tasks, demonstrating its potential in efficient and effective molecular data analysis and manipulation.

The Covid-19 disease has spread widely over the whole world since the beginning of 2020. Following the epidemic which started in Wuhan, China on January 30, 2020 the World Health Organization (WHO) declared a global health emergency and a pandemic. Researchers of different disciplines work along with public health officials to understand the SARS-CoV-2 pathogenesis and jointly with the policymakers urgently develop strategies to control the spread of this new disease. Recent findings have observed specific image patterns from computed tomography (CT) for patients infected by SARS-CoV-2 which are distinct from the other pulmonary diseases. In this paper, we propose an explainable-by-design that has an integrated image segmentation mechanism based on SLIC that improves the algorithm performance and the interpretability of the resulting model. In order to evaluate the proposed approach, we used the SARS-CoV-2 CT scan dataset that we published recently and has been widely used in the literature. The proposed Super-xDNN could obtain statistically better results than traditional deep learning approaches as DenseNet-201 and Resnet-152. Furthermore, it also improved the explainability and interpretability of its decision mechanism when compared with the xDNN basis approach that uses the whole image as prototype. The segmentation mechanism of Super-xDNN favored a decision structure that is more close to the human logic. Moreover, it also allowed the provision of new insights as a heat-map which highlights the areas with highest similarities with Covid-19 prototypes, and an estimation of the area affected by the disease.

The Internet of Things (IoT) has made it possible to include everyday objects in a connected network, allowing them to intelligently process data and respond to their environment. Thus, it is expected that those objects will gain an intelligent understanding of their environment and be able to process data more efficiently than before. Particularly, such edge computing paradigm has allowed the execution of inference methods on resource-constrained devices such as microcontrollers, significantly changing the way IoT applications have evolved in recent years. However, although this scenario has supported the development of Tiny Machine Learning (TinyML) approaches on such devices, there are still some challenges that require further investigation when optimizing data streaming on the edge. Therefore, this article proposes a new unsupervised TinyML regression technique based on the typicality and eccentricity of the samples to be processed. Moreover, the proposed technique also exploits a Recursive Least Squares (RLS) filter approach. Combining all these features, the proposed method uses similarities between samples to identify patterns when processing data streams, predicting outcomes based on these patterns. The results obtained through the extensive experimentation utilizing vehicular data streams were highly encouraging. The proposed algorithm was meticulously compared with the RLS algorithm and Convolutional Neural Networks (CNN). It exhibited significantly superior performance, with mean squared errors that were 4.68 and 12.02 times lower, respectively, compared to the aforementioned techniques.

Revista acadêmica de divulgação acadêmico-científica de jovens pesquisadores