Nikola Simidjievski
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Nikola Simidjievski has a total of 69 co-authors. Here are the top co-authors by number of shared publications:
- Dragi Kocev
Jožef Stefan Institute, Ljubljana, Slovenia
5 shared publications Ljupčo Todorovski
Univerza V Ljubljani, Ljubljana, Slovenia
5 shared publicationsSašo Džeroski
Institut Jozef Stefan, Ljubljana, Slovenia
5 shared publications- Mateja Jamnik
University of Cambridge, Cambridge, United Kingdom
4 shared publications - Pietro Liò
University of Cambridge
4 shared publications - Blaž Škrlj
Jožef Stefan Institute, Ljubljana, Slovenia
2 shared publications
Latest Publications
- Open Access
Consensus Through Diversity: A Comprehensive Benchmark of Multi-Omic Approaches for Precision Breast Oncology
Emerging multi-omic profiling has made it feasible to subtype disease using multiple molecular layers. However, inconsistent preprocessing, heterogeneous implementations, variable evaluation, and limited reproducibility often constrain method selection. Here, we systematically benchmark 22 publicly available unsupervised approaches for bulk data on the TCGA-BRCA cohort across five modalities (RNA-seq, miRNA, DNA methylation, copy numbers, single nucleotide polymorphisms) and validate findings in two independent datasets, enabling a multi-layered comparison of performance, heterogeneous data support and interpretability. Most approaches fuse multi-omic data to produce a two-cluster solution largely aligned with ER status, with higher-resolution approaches further refining these into four coherent subclasses (angiogenic luminal, oxidative-phosphorylation/HER2-low luminal, immune-inflamed basal-like, and hyper-proliferative basal-like). Our benchmarking results indicate that methods based on similarity networks can efficiently produce stable, reliable partitions. Matrix factorisation and Bayesian factorisation algorithms produce rich latent representations, allowing quantification of feature and modality contributions, albeit at higher computational cost. Consensus clustering can be used on a case-by-case basis and refine partitions into more robust and generalisable findings. We aggregate our insights into a decision workflow that aligns with study goals, data characteristics, and computational resources, enabling optimal analytic strategies. This comprehensive assessment provides a practical roadmap for investigators seeking to extract reproducible, biologically meaningful subtypes from complex multi-omic datasets. We higlight the different technical and practical benefits and trade-offs that shape the selection and development of multi-omic approaches applied in precision oncology.
Open AccessConsensus Through Diversity: A Comprehensive Benchmark of Multi-Omic Approaches for Precision Breast Oncology
Emerging multi-omic profiling has made it feasible to subtype disease using multiple molecular layers. However, inconsistent preprocessing, heterogeneous implementations, variable evaluation, and limited reproducibility often constrain method selection. Here, we systematically benchmark 22 publicly available unsupervised approaches for bulk data on the TCGA-BRCA cohort across five modalities (RNA-seq, miRNA, DNA methylation, copy numbers, single nucleotide polymorphisms) and validate findings in two independent datasets, enabling a multi-layered comparison of performance, heterogeneous data support and interpretability. Most approaches fuse multi-omic data to produce a two-cluster solution largely aligned with ER status, with higher-resolution approaches further refining these into four coherent subclasses (angiogenic luminal, oxidative-phosphorylation/HER2-low luminal, immune-inflamed basal-like, and hyper-proliferative basal-like). Our benchmarking results indicate that methods based on similarity networks can efficiently produce stable, reliable partitions. Matrix factorisation and Bayesian factorisation algorithms produce rich latent representations, allowing quantification of feature and modality contributions, albeit at higher computational cost. Consensus clustering can be used on a case-by-case basis and refine partitions into more robust and generalisable findings. We aggregate our insights into a decision workflow that aligns with study goals, data characteristics, and computational resources, enabling optimal analytic strategies. This comprehensive assessment provides a practical roadmap for investigators seeking to extract reproducible, biologically meaningful subtypes from complex multi-omic datasets. We higlight the different technical and practical benefits and trade-offs that shape the selection and development of multi-omic approaches applied in precision oncology.
- Open Access
- 1 October 2024
- Published by Springer Nature in Neuroinformatics
- Vol. 22 (4) , 657-678
Interdisciplinary and Collaborative Training in Neuroscience: Insights from the Human Brain Project Education Programme
Neuroscience education is challenged by rapidly evolving technology and the development of interdisciplinary approaches for brain research. The Human Brain Project (HBP) Education Programme aimed to address the need for interdisciplinary expertise in brain research by equipping a new generation of researchers with skills across neuroscience, medicine, and information technology. Over its ten year duration, the programme engaged over 1,300 experts and attracted more than 5,500 participants from various scientific disciplines in its blended learning curriculum, specialised schools and workshops, and events fostering dialogue among early-career researchers. Key principles of the programme’s approach included fostering interdisciplinarity, adaptability to the evolving research landscape and infrastructure, and a collaborative environment with a focus on empowering early-career researchers. Following the programme’s conclusion, we provide here an analysis and in-depth view across a diverse range of educational formats and events. Our results show that the Education Programme achieved success in its wide geographic reach, the diversity of participants, and the establishment of transversal collaborations. Building on these experiences and achievements, we describe how leveraging digital tools and platforms provides accessible and highly specialised training, which can enhance existing education programmes for the next generation of brain researchers working in decentralised European collaborative spaces. Finally, we present the lessons learnt so that similar initiatives may improve upon our experience and incorporate our suggestions into their own programme.
- 1 October 2024
- Published by Springer Nature in Neuroinformatics
- Vol. 22 (4) , 657-678
Open AccessInterdisciplinary and Collaborative Training in Neuroscience: Insights from the Human Brain Project Education Programme
Neuroscience education is challenged by rapidly evolving technology and the development of interdisciplinary approaches for brain research. The Human Brain Project (HBP) Education Programme aimed to address the need for interdisciplinary expertise in brain research by equipping a new generation of researchers with skills across neuroscience, medicine, and information technology. Over its ten year duration, the programme engaged over 1,300 experts and attracted more than 5,500 participants from various scientific disciplines in its blended learning curriculum, specialised schools and workshops, and events fostering dialogue among early-career researchers. Key principles of the programme’s approach included fostering interdisciplinarity, adaptability to the evolving research landscape and infrastructure, and a collaborative environment with a focus on empowering early-career researchers. Following the programme’s conclusion, we provide here an analysis and in-depth view across a diverse range of educational formats and events. Our results show that the Education Programme achieved success in its wide geographic reach, the diversity of participants, and the establishment of transversal collaborations. Building on these experiences and achievements, we describe how leveraging digital tools and platforms provides accessible and highly specialised training, which can enhance existing education programmes for the next generation of brain researchers working in decentralised European collaborative spaces. Finally, we present the lessons learnt so that similar initiatives may improve upon our experience and incorporate our suggestions into their own programme.
- Open Access
In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification
We investigate the utility of in-domain self-supervised pre-training of vision models in the analysis of remote sensing imagery. Self-supervised learning (SSL) has emerged as a promising approach for remote sensing image classification due to its ability to exploit large amounts of unlabeled data. Unlike traditional supervised learning, SSL aims to learn representations of data without the need for explicit labels. This is achieved by formulating auxiliary tasks that can be used for pre-training models before fine-tuning them on a given downstream task. A common approach in practice to SSL pre-training is utilizing standard pre-training datasets, such as ImageNet. While relevant, such a general approach can have a sub-optimal influence on the downstream performance of models, especially on tasks from challenging domains such as remote sensing. In this paper, we analyze the effectiveness of SSL pre-training by employing the iBOT framework coupled with Vision transformers trained on Million-AID, a large and unlabeled remote sensing dataset. We present a comprehensive study of different self-supervised pre-training strategies and evaluate their effect across 14 downstream datasets with diverse properties. Our results demonstrate that leveraging large in-domain datasets for self-supervised pre-training consistently leads to improved predictive downstream performance, compared to the standard approaches found in practice.
Open AccessIn-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification
We investigate the utility of in-domain self-supervised pre-training of vision models in the analysis of remote sensing imagery. Self-supervised learning (SSL) has emerged as a promising approach for remote sensing image classification due to its ability to exploit large amounts of unlabeled data. Unlike traditional supervised learning, SSL aims to learn representations of data without the need for explicit labels. This is achieved by formulating auxiliary tasks that can be used for pre-training models before fine-tuning them on a given downstream task. A common approach in practice to SSL pre-training is utilizing standard pre-training datasets, such as ImageNet. While relevant, such a general approach can have a sub-optimal influence on the downstream performance of models, especially on tasks from challenging domains such as remote sensing. In this paper, we analyze the effectiveness of SSL pre-training by employing the iBOT framework coupled with Vision transformers trained on Million-AID, a large and unlabeled remote sensing dataset. We present a comprehensive study of different self-supervised pre-training strategies and evaluate their effect across 14 downstream datasets with diverse properties. Our results demonstrate that leveraging large in-domain datasets for self-supervised pre-training consistently leads to improved predictive downstream performance, compared to the standard approaches found in practice.
- 16 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 21 (1545598X) , 1-5
In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification
We investigate the utility of in-domain self-supervised pretraining of vision models in the analysis of remote sensing imagery. Self-supervised learning (SSL) has emerged as a promising approach for remote sensing image classification due to its ability to exploit large amounts of unlabeled data. Unlike traditional supervised learning, SSL aims to learn representations of data without the need for explicit labels. This is achieved by formulating auxiliary tasks that can be used for pretraining models before fine-tuning them on a given downstream task. A common approach in practice to SSL pretraining is utilizing standard pretraining datasets, such as ImageNet. While relevant, such a general approach can have a suboptimal influence on the downstream performance of models, especially on tasks from challenging domains such as remote sensing. In this letter, we analyze the effectiveness of SSL pretraining by employing the image bidirectional encoder representations from transformers (BERT) pretraining with online tokenizer (iBOT) framework coupled with Vision transformers (ViTs) trained on million aerial image dataset (Million-AID), a large and unlabeled remote sensing dataset. We present a comprehensive study of different self-supervised pretraining strategies and evaluate their effect across 14 downstream datasets with diverse properties. Our results demonstrate that leveraging large in-domain datasets for self-supervised pretraining consistently leads to improved predictive downstream performance, compared to the standard approaches found in practice.
- 16 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Geoscience and Remote Sensing Letters
- Vol. 21 (1545598X) , 1-5
In-Domain Self-Supervised Learning Improves Remote Sensing Image Scene Classification
We investigate the utility of in-domain self-supervised pretraining of vision models in the analysis of remote sensing imagery. Self-supervised learning (SSL) has emerged as a promising approach for remote sensing image classification due to its ability to exploit large amounts of unlabeled data. Unlike traditional supervised learning, SSL aims to learn representations of data without the need for explicit labels. This is achieved by formulating auxiliary tasks that can be used for pretraining models before fine-tuning them on a given downstream task. A common approach in practice to SSL pretraining is utilizing standard pretraining datasets, such as ImageNet. While relevant, such a general approach can have a suboptimal influence on the downstream performance of models, especially on tasks from challenging domains such as remote sensing. In this letter, we analyze the effectiveness of SSL pretraining by employing the image bidirectional encoder representations from transformers (BERT) pretraining with online tokenizer (iBOT) framework coupled with Vision transformers (ViTs) trained on million aerial image dataset (Million-AID), a large and unlabeled remote sensing dataset. We present a comprehensive study of different self-supervised pretraining strategies and evaluate their effect across 14 downstream datasets with diverse properties. Our results demonstrate that leveraging large in-domain datasets for self-supervised pretraining consistently leads to improved predictive downstream performance, compared to the standard approaches found in practice.
- Open Access
SHARCS: Shared Concept Space for Explainable Multimodal Learning
Multimodal learning is an essential paradigm for addressing complex real-world problems, where individual data modalities are typically insufficient to accurately solve a given modelling task. While various deep learning approaches have successfully addressed these challenges, their reasoning process is often opaque; limiting the capabilities for a principled explainable cross-modal analysis and any domain-expert intervention. In this paper, we introduce SHARCS (SHARed Concept Space) -- a novel concept-based approach for explainable multimodal learning. SHARCS learns and maps interpretable concepts from different heterogeneous modalities into a single unified concept-manifold, which leads to an intuitive projection of semantically similar cross-modal concepts. We demonstrate that such an approach can lead to inherently explainable task predictions while also improving downstream predictive performance. Moreover, we show that SHARCS can operate and significantly outperform other approaches in practically significant scenarios, such as retrieval of missing modalities and cross-modal explanations. Our approach is model-agnostic and easily applicable to different types (and number) of modalities, thus advancing the development of effective, interpretable, and trustworthy multimodal approaches.
Open AccessSHARCS: Shared Concept Space for Explainable Multimodal Learning
Multimodal learning is an essential paradigm for addressing complex real-world problems, where individual data modalities are typically insufficient to accurately solve a given modelling task. While various deep learning approaches have successfully addressed these challenges, their reasoning process is often opaque; limiting the capabilities for a principled explainable cross-modal analysis and any domain-expert intervention. In this paper, we introduce SHARCS (SHARed Concept Space) -- a novel concept-based approach for explainable multimodal learning. SHARCS learns and maps interpretable concepts from different heterogeneous modalities into a single unified concept-manifold, which leads to an intuitive projection of semantically similar cross-modal concepts. We demonstrate that such an approach can lead to inherently explainable task predictions while also improving downstream predictive performance. Moreover, we show that SHARCS can operate and significantly outperform other approaches in practically significant scenarios, such as retrieval of missing modalities and cross-modal explanations. Our approach is model-agnostic and easily applicable to different types (and number) of modalities, thus advancing the development of effective, interpretable, and trustworthy multimodal approaches.
- Open Access
Enhancing Representation Learning on High-Dimensional, Small-Size Tabular Data: A Divide and Conquer Method with Ensembled VAEs
Variational Autoencoders and their many variants have displayed impressive ability to perform dimensionality reduction, often achieving state-of-the-art performance. Many current methods however, struggle to learn good representations in High Dimensional, Low Sample Size (HDLSS) tasks, which is an inherently challenging setting. We address this challenge by using an ensemble of lightweight VAEs to learn posteriors over subsets of the feature-space, which get aggregated into a joint posterior in a novel divide-and-conquer approach. Specifically, we present an alternative factorisation of the joint posterior that induces a form of implicit data augmentation that yields greater sample efficiency. Through a series of experiments on eight real-world datasets, we show that our method learns better latent representations in HDLSS settings, which leads to higher accuracy in a downstream classification task. Furthermore, we verify that our approach has a positive effect on disentanglement and achieves a lower estimated Total Correlation on learnt representations. Finally, we show that our approach is robust to partial features at inference, exhibiting little performance degradation even with most features missing.
Open AccessEnhancing Representation Learning on High-Dimensional, Small-Size Tabular Data: A Divide and Conquer Method with Ensembled VAEs
Variational Autoencoders and their many variants have displayed impressive ability to perform dimensionality reduction, often achieving state-of-the-art performance. Many current methods however, struggle to learn good representations in High Dimensional, Low Sample Size (HDLSS) tasks, which is an inherently challenging setting. We address this challenge by using an ensemble of lightweight VAEs to learn posteriors over subsets of the feature-space, which get aggregated into a joint posterior in a novel divide-and-conquer approach. Specifically, we present an alternative factorisation of the joint posterior that induces a form of implicit data augmentation that yields greater sample efficiency. Through a series of experiments on eight real-world datasets, we show that our method learns better latent representations in HDLSS settings, which leads to higher accuracy in a downstream classification task. Furthermore, we verify that our approach has a positive effect on disentanglement and achieves a lower estimated Total Correlation on learnt representations. Finally, we show that our approach is robust to partial features at inference, exhibiting little performance degradation even with most features missing.
- Open Access
Spatio-relational inductive biases in spatial cell-type deconvolution
Spatial transcriptomic technologies profile gene expression in-situ , facilitating the spatial characterisation of molecular phenomena within tissues, yet often at multi-cellular resolution. Computational approaches have been developed to infer fine-grained cell-type compositions across locations, but they frequently treat neighbouring spots independently of each other. Here we present GNN-C2L, a flexible deconvolution approach that leverages proximal inductive biases to propagate information along adjacent spots. In performance comparison on simulated and semisimulated datasets, GNN-C2L achieves increased deconvolution performance over spatial-agnostic variants. We believe that accounting for spatial inductive biases can yield improved characterisation of cell-type heterogeneity in tissues.
Open AccessSpatio-relational inductive biases in spatial cell-type deconvolution
Spatial transcriptomic technologies profile gene expression in-situ , facilitating the spatial characterisation of molecular phenomena within tissues, yet often at multi-cellular resolution. Computational approaches have been developed to infer fine-grained cell-type compositions across locations, but they frequently treat neighbouring spots independently of each other. Here we present GNN-C2L, a flexible deconvolution approach that leverages proximal inductive biases to propagate information along adjacent spots. In performance comparison on simulated and semisimulated datasets, GNN-C2L achieves increased deconvolution performance over spatial-agnostic variants. We believe that accounting for spatial inductive biases can yield improved characterisation of cell-type heterogeneity in tissues.
- Open Access
- 28 April 2023
- Published by MDPI AG in Remote Sensing
- Vol. 15 (9) , 2343
AiTLAS: Artificial Intelligence Toolbox for Earth Observation
We propose AiTLAS—an open-source, state-of-the-art toolbox for exploratory and predictive analysis of satellite imagery. It implements a range of deep-learning architectures and models tailored for the EO tasks illustrated in this case. The versatility and applicability of the toolbox are showcased in a variety of EO tasks, including image scene classification, semantic image segmentation, object detection, and crop type prediction. These use cases demonstrate the potential of the toolbox to support the complete data analysis pipeline starting from data preparation and understanding, through learning novel models or fine-tuning existing ones, using models for making predictions on unseen images, and up to analysis and understanding of the predictions and the predictive performance yielded by the models. AiTLAS brings the AI and EO communities together by facilitating the use of EO data in the AI community and accelerating the uptake of (advanced) machine-learning methods and approaches by EO experts. It achieves this by providing: (1) user-friendly, accessible, and interoperable resources for data analysis through easily configurable and readily usable pipelines; (2) standardized, verifiable, and reusable data handling, wrangling, and pre-processing approaches for constructing AI-ready data; (3) modular and configurable modeling approaches and (pre-trained) models; and (4) standardized and reproducible benchmark protocols including data and models.
- 28 April 2023
- Published by MDPI AG in Remote Sensing
- Vol. 15 (9) , 2343
Open AccessAiTLAS: Artificial Intelligence Toolbox for Earth Observation
We propose AiTLAS—an open-source, state-of-the-art toolbox for exploratory and predictive analysis of satellite imagery. It implements a range of deep-learning architectures and models tailored for the EO tasks illustrated in this case. The versatility and applicability of the toolbox are showcased in a variety of EO tasks, including image scene classification, semantic image segmentation, object detection, and crop type prediction. These use cases demonstrate the potential of the toolbox to support the complete data analysis pipeline starting from data preparation and understanding, through learning novel models or fine-tuning existing ones, using models for making predictions on unseen images, and up to analysis and understanding of the predictions and the predictive performance yielded by the models. AiTLAS brings the AI and EO communities together by facilitating the use of EO data in the AI community and accelerating the uptake of (advanced) machine-learning methods and approaches by EO experts. It achieves this by providing: (1) user-friendly, accessible, and interoperable resources for data analysis through easily configurable and readily usable pipelines; (2) standardized, verifiable, and reusable data handling, wrangling, and pre-processing approaches for constructing AI-ready data; (3) modular and configurable modeling approaches and (pre-trained) models; and (4) standardized and reproducible benchmark protocols including data and models.
- Open Access
- 9 December 2021
- Published by Oxford University Press (OUP) in Bioinformatics
- Vol. 38 (5) , 1320-1327
Unsupervised construction of computational graphs for gene expression data with explicit structural inductive biases
Gene expression data are commonly used at the intersection of cancer research and machine learning for better understanding of the molecular status of tumour tissue. Deep learning predictive models have been employed for gene expression data due to their ability to scale and remove the need for manual feature engineering. However, gene expression data are often very high dimensional, noisy and presented with a low number of samples. This poses significant problems for learning algorithms: models often overfit, learn noise and struggle to capture biologically relevant information. In this article, we utilize external biological knowledge embedded within structures of gene interaction graphs such as protein–protein interaction (PPI) networks to guide the construction of predictive models. We present Gene Interaction Network Constrained Construction (GINCCo), an unsupervised method for automated construction of computational graph models for gene expression data that are structurally constrained by prior knowledge of gene interaction networks. We employ this methodology in a case study on incorporating a PPI network in cancer phenotype prediction tasks. Our computational graphs are structurally constructed using topological clustering algorithms on the PPI networks which incorporate inductive biases stemming from network biology research on protein complex discovery. Each of the entities in the GINCCo computational graph represents biological entities such as genes, candidate protein complexes and phenotypes instead of arbitrary hidden nodes of a neural network. This provides a biologically relevant mechanism for model regularization yielding strong predictive performance while drastically reducing the number of model parameters and enabling guided post-hoc enrichment analyses of influential gene sets with respect to target phenotypes. Our experiments analysing a variety of cancer phenotypes show that GINCCo often outperforms support vector machine, Fully Connected Multi-layer Perceptrons (MLP) and Randomly Connected MLPs despite greatly reduced model complexity. https://github.com/paulmorio/gincco contains the source code for our approach. We also release a library with algorithms for protein complex discovery within PPI networks at https://github.com/paulmorio/protclus. This repository contains implementations of the clustering algorithms used in this article. Supplementary data are available at Bioinformatics online.
- 9 December 2021
- Published by Oxford University Press (OUP) in Bioinformatics
- Vol. 38 (5) , 1320-1327
Open AccessUnsupervised construction of computational graphs for gene expression data with explicit structural inductive biases
Gene expression data are commonly used at the intersection of cancer research and machine learning for better understanding of the molecular status of tumour tissue. Deep learning predictive models have been employed for gene expression data due to their ability to scale and remove the need for manual feature engineering. However, gene expression data are often very high dimensional, noisy and presented with a low number of samples. This poses significant problems for learning algorithms: models often overfit, learn noise and struggle to capture biologically relevant information. In this article, we utilize external biological knowledge embedded within structures of gene interaction graphs such as protein–protein interaction (PPI) networks to guide the construction of predictive models. We present Gene Interaction Network Constrained Construction (GINCCo), an unsupervised method for automated construction of computational graph models for gene expression data that are structurally constrained by prior knowledge of gene interaction networks. We employ this methodology in a case study on incorporating a PPI network in cancer phenotype prediction tasks. Our computational graphs are structurally constructed using topological clustering algorithms on the PPI networks which incorporate inductive biases stemming from network biology research on protein complex discovery. Each of the entities in the GINCCo computational graph represents biological entities such as genes, candidate protein complexes and phenotypes instead of arbitrary hidden nodes of a neural network. This provides a biologically relevant mechanism for model regularization yielding strong predictive performance while drastically reducing the number of model parameters and enabling guided post-hoc enrichment analyses of influential gene sets with respect to target phenotypes. Our experiments analysing a variety of cancer phenotypes show that GINCCo often outperforms support vector machine, Fully Connected Multi-layer Perceptrons (MLP) and Randomly Connected MLPs despite greatly reduced model complexity. https://github.com/paulmorio/gincco contains the source code for our approach. We also release a library with algorithms for protein complex discovery within PPI networks at https://github.com/paulmorio/protclus. This repository contains implementations of the clustering algorithms used in this article. Supplementary data are available at Bioinformatics online.
- 1 December 2021
- Published by Elsevier in Applied Soft Computing
- Vol. 113
Fuzzy Jaccard Index: A robust comparison of ordered lists
- 1 December 2021
- Published by Elsevier in Applied Soft Computing
- Vol. 113
Fuzzy Jaccard Index: A robust comparison of ordered lists