SASelim Aksoy

Selim Aksoy

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Selim Aksoy has a total of 121 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    CODI: Contextual Object Detection via Image Inpainting

    by Sinan Çavdar, Selim Aksoy

    Object detection is a fundamental task in many applications of remote sensing image analysis. Problems such as inter-class similarity, intra-class variability, and existence of heterogeneous backgrounds can benefit from leveraging contextual information to improve the detection performance. Most context modeling architectures introduce additional modules that aim to extract contextual information from the main backbone that is also used for learning the object representations. We propose a new approach for contextual object detection via image inpainting named CODI where a custom module fuses object-specific features from an object detector and contextual representations from an inpainting-based generative model. The resulting multi-scale representation named the contextual pyramid network is injected into the feature extraction backbone so that both the oriented region proposal network and the classification and regression branches of the detector can benefit from the richer representations resulting from this fusion. In contrast to others, our architecture obtains semantic context independently from and without relying on the backbone of the object detection model to guide both localization and labeling. Extensive experiments with quantitative and qualitative evaluation are performed on the DOTA, HRSC2016, and DIOR-R datasets. With mean average precision scores of 76.61%, 90.57%, and 67.63% on these datasets respectively, the proposed model achieves higher performance compared to other models from the literature. Ablation experiments also show that CODI enables improvements in precision and recall with effective control of the score probability threshold during detection.

  • Open Access

    Patch relevance estimation and multilabel augmentation for weakly supervised histopathology image classification

    by Bulut Aygunes, Ramazan Gokberk Cinbis, Selim Aksoy

    PurposeWeakly supervised learning (WSL) is widely used for histopathological image analysis by modeling images as sets of fixed-size patches and utilizing image-level diagnoses as weak labels. However, in multiclass classification scenarios, patches corresponding to a wide spectrum of diagnostic categories can co-exist in a single image, complicating the learning process. We aim to address label uncertainty in such multiclass settings.ApproachWe propose a two-branch architecture and a complementary training strategy to improve patch-based WSL. One branch estimates patch-level class likelihoods, whereas the other predicts per-class patch relevance weights. These outputs are combined into image-level class predictions via a relevance-weighted sum of per-patch class likelihoods. To further improve performance, we introduce a multilabel augmentation strategy that forms new training samples by combining patch sets and labels from pairs of images, resulting in multilabel samples that enrich the training set by increasing the chance of having more patches that are relevant to the augmented label sets.ResultsWe evaluate our method on two challenging multiclass breast histopathology datasets for region of interest classification. The proposed architecture and training strategy outperform conventional weakly supervised methods, demonstrating improved classification accuracy and robustness, particularly in underrepresented classes.ConclusionsThe proposed architecture effectively models the complex relationship between image-level labels and patch-level content in multiclass histopathological image analysis. Combined with the image-level multilabel augmentation strategy, it improves learning under label uncertainty. These contributions hold potential for more accurate and scalable diagnostic support systems in digital pathology.


  • Open Access

    OT-Fusion: Cortex-conditioned Fusion of Matched PET and MRI Images Using Optimal Transport

    by Emir Türkölmez, Yasir Ali Khan, Caroline Bund, Izzie Jacques Namer, A. Ercüment Çiçek, Selim Aksoy

    Positron Emission Tomography (PET) and Magnetic Resonance Imaging (MRI) are crucial in diagnosing various medical conditions related to brain. FDG-PET provides essential functional information by capturing synaptic transmission activity, while MRI offers anatomical details. The integration of these modalities has become increasingly valuable, especially with the advent of hybrid PET/MRI systems that enable acquisition of both images in a single session. However, a notable challenge arises due to the inherent differences in spatial resolution between PET and MRI. PET images often have lower spatial resolution, leading to metabolic signals from active regions, such as the cerebral cortex and basal ganglia, appearing diffused and extended into adjacent areas like the white matter. This diffusion can complicate the accurate localization of metabolic activity, posing difficulties in clinical assessments, particularly in neurological pathologies such as epilepsy where precise mapping is crucial. To address this issue, we propose a novel approach that effectively maps the metabolic activity observed in PET onto the cortical structures delineated in MRI. We formulate this mapping as an optimal transport problem that corresponds to finding the optimum way to transform one probability distribution into another. Our method aims to enhance the structural quality of PET images, making them more detailed and informative for clinical decisions related to neurological disorders. The experiments using paired PET and MRI images indicate that our method performs better than the state-of-the-art image fusion methods on quantitative metrics and improves image quality based on visual qualitative evaluation by an expert physician.

  • Deep Feature Representations for Variable-Sized Regions of Interest in Breast Histopathology

    by Caner Mercan, Bulut Aygunes, Selim Aksoy, Ezgi Mercan, Linda G. Shapiro, Donald L. Weaver, Joann G. Elmore

    Objective: Modeling variable-sized regions of interest (ROIs) in whole slide images using deep convolutional networks is a challenging task, as these networks typically require fixed-sized inputs that should contain sufficient structural and contextual information for classification. We propose a deep feature extraction framework that builds an ROI-level feature representation via weighted aggregation of the representations of variable numbers of fixed-sized patches sampled from nuclei-dense regions in breast histopathology images. Methods: First, the initial patch-level feature representations are extracted from both fully-connected layer activations and pixel-level convolutional layer activations of a deep network, and the weights are obtained from the class predictions of the same network trained on patch samples. Then, the final patch-level feature representations are computed by concatenation of weighted instances of the extracted feature activations. Finally, the ROI-level representation is obtained by fusion of the patch-level representations by average pooling. Results: Experiments using a well-characterized data set of 240 slides containing 437 ROIs marked by experienced pathologists with variable sizes and shapes result in an accuracy score of 72.65% in classifying ROIs into four diagnostic categories that cover the whole histologic spectrum. Conclusion: The results show that the proposed feature representations are superior to existing approaches and provide accuracies that are higher than the average accuracy of another set of pathologists. Significance: The proposed generic representation that can be extracted from any type of deep convolutional architecture combines the patch appearance information captured by the network activations and the diagnostic relevance predicted by the class-specific scoring of patches for effective modeling of variable-sized ROIs.


  • Graph convolutional networks for region of interest classification in breast histopathology

    by Bulut Aygüneş, Selim Aksoy, Gökberk Cinbiş, Kemal Kösemehmetoglu, Sevgen Önder, Ayşegül Üner


  • Open Access

    Multisource Region Attention Network for Fine-Grained Object Recognition in Remote Sensing Imagery

    by Gencer Sumbul, Ramazan Gokberk Cinbis, Selim Aksoy

    Fine-grained object recognition concerns the identification of the type of an object among a large number of closely related sub-categories. Multisource data analysis, that aims to leverage the complementary spectral, spatial, and structural information embedded in different sources, is a promising direction towards solving the fine-grained recognition problem that involves low between-class variance, small training set sizes for rare classes, and class imbalance. However, the common assumption of co-registered sources may not hold at the pixel level for small objects of interest. We present a novel methodology that aims to simultaneously learn the alignment of multisource data and the classification model in a unified framework. The proposed method involves a multisource region attention network that computes per-source feature representations, assigns attention scores to candidate regions sampled around the expected object locations by using these representations, and classifies the objects by using an attention-driven multisource representation that combines the feature representations and the attention scores from all sources. All components of the model are realized using deep neural networks and are learned in an end-to-end fashion. Experiments using RGB, multispectral, and LiDAR elevation data for classification of street trees showed that our approach achieved 64.2% and 47.3% accuracies for the 18-class and 40-class settings, respectively, which correspond to 13% and 14.3% improvement relative to the commonly used feature concatenation approach from multiple sources.

  • Detection and classification of cancer in whole slide breast histopathology images using deep convolutional networks

    by Baris Gecer, Selim Aksoy, Ezgi Mercan, Linda G. Shapiro, Donald L. Weaver, Joann G. Elmore


  • Fine-grained object recognition and zero-shot learning in multispectral imagery

    by Gencer Sümbül, Selim Aksoy, Ramazan Gökberk Cinbiş

    We present a method for fine-grained object recognition problem, that aims to recognize the type of an object among a large number of sub-categories, and zero-shot learning scenario on multispectral images. In order to establish a relation between seen classes and new unseen classes, a compatibility function between image features extracted from a convolutional neural network and auxiliary information of classes is learnt. Knowledge transfer for unseen classes is carried out by maximizing this function. Performance of the model (15.2%) evaluated with manually annotated attributes, a natural language model, and a scientific taxonomy as auxiliary information is promisingly better than the other methods for 16 test classes.

  • Open Access

    Fine-Grained Object Recognition and Zero-Shot Learning in Remote Sensing Imagery

    by Gencer Sumbul, Ramazan Gokberk Cinbis, Selim Aksoy

    Fine-grained object recognition that aims to identify the type of an object among a large number of subcategories is an emerging application with the increasing resolution that exposes new details in image data. Traditional fully supervised algorithms fail to handle this problem where there is low between-class variance and high within-class variance for the classes of interest with small sample sizes. We study an even more extreme scenario named zero-shot learning (ZSL) in which no training example exists for some of the classes. ZSL aims to build a recognition model for new unseen categories by relating them to seen classes that were previously learned. We establish this relation by learning a compatibility function between image features extracted via a convolutional neural network and auxiliary information that describes the semantics of the classes of interest by using training samples from the seen classes. Then, we show how knowledge transfer can be performed for the unseen classes by maximizing this function during inference. We introduce a new data set that contains 40 different types of street trees in 1-ft spatial resolution aerial data, and evaluate the performance of this model with manually annotated attributes, a natural language model, and a scientific taxonomy as auxiliary information. The experiments show that the proposed model achieves 14.3% recognition accuracy for the classes with no training examples, which is significantly better than a random guess accuracy of 6.3% for 16 test classes, and three other ZSL algorithms.

  • Fine-Grained Object Recognition and Zero-Shot Learning in Remote Sensing Imagery

    by Gencer Sumbul, Ramazan Gokberk Cinbis, Selim Aksoy

    Fine-grained object recognition that aims to identify the type of an object among a large number of subcategories is an emerging application with the increasing resolution that exposes new details in image data. Traditional fully supervised algorithms fail to handle this problem where there is low between-class variance and high within-class variance for the classes of interest with small sample sizes. We study an even more extreme scenario named zero-shot learning (ZSL) in which no training example exists for some of the classes. ZSL aims to build a recognition model for new unseen categories by relating them to seen classes that were previously learned. We establish this relation by learning a compatibility function between image features extracted via a convolutional neural network and auxiliary information that describes the semantics of the classes of interest by using training samples from the seen classes. Then, we show how knowledge transfer can be performed for the unseen classes by maximizing this function during inference. We introduce a new data set that contains 40 different types of street trees in 1-ft spatial resolution aerial data, and evaluate the performance of this model with manually annotated attributes, a natural language model, and a scientific taxonomy as auxiliary information. The experiments show that the proposed model achieves 14.3% recognition accuracy for the classes with no training examples, which is significantly better than a random guess accuracy of 6.3% for 16 test classes, and three other ZSL algorithms.