Ilya Trofimov
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Ilya Trofimov has a total of 34 co-authors. Here are the top co-authors by number of shared publications:
- Evgeny Burnaev
Skolkovo Institute of Science and Technology
6 shared publications Dmitry Osin
Skolkovo Institute of Science and Technology, Moscow, Russia
2 shared publicationsE. Burnaev
Skolkovo Institute of Science and Technology
2 shared publications- Egor Shvetsov
Skolkovo Institute of Science and Technology, Moscow, Russia
2 shared publications N. Mazyavkina
Skolkovo Institute of Science and Technology
2 shared publicationsS. Moustafa
Skolkovo Institute of Science and Technology
2 shared publications
Latest Publications
- 8 November 2024
- Published by Springer Nature
- p. 274-290
Scalar Function Topology Divergence: Comparing Topology of 3D Objects
- 8 November 2024
- Published by Springer Nature
- p. 274-290
Scalar Function Topology Divergence: Comparing Topology of 3D Objects
- Open Access
- 19 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 117008-117025
QuantNAS for Super Resolution: Searching for Efficient Quantization-Friendly Architectures Against Quantization Noise
This work aims to develop an automated procedure for discovering new, efficient solutions that can be effectively quantized in mixed-precision mode with minimal degradation. While our primary focus is on Super-Resolution (SR), our proposed procedure is applicable beyond this domain. To achieve our goals, we first develop an efficient Neural Architecture Search (NAS) procedure for full-precision a models, surpassing existing NAS solutions for SR.We then adapt this procedure for quantization-aware search. By introducing Quantization Noise (QN) during the search phase, we approximate the model degradation after quantization. Additionally, we improve search performance by implementing entropy regularization, which prioritizes operations and its precision within each search space block. Our experiments confirm the superiority of quantization-aware NAS compared to the two-step process: NAS followed by quantization. Furthermore, approximating quantization with QN offers a 30% speed improvement over direct weight quantization. We validate our approach by developing and applying it to two search spaces inspired by state-of-the-art SR models. Our code is publicly available b .
- 19 August 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 117008-117025
Open AccessQuantNAS for Super Resolution: Searching for Efficient Quantization-Friendly Architectures Against Quantization Noise
This work aims to develop an automated procedure for discovering new, efficient solutions that can be effectively quantized in mixed-precision mode with minimal degradation. While our primary focus is on Super-Resolution (SR), our proposed procedure is applicable beyond this domain. To achieve our goals, we first develop an efficient Neural Architecture Search (NAS) procedure for full-precision a models, surpassing existing NAS solutions for SR.We then adapt this procedure for quantization-aware search. By introducing Quantization Noise (QN) during the search phase, we approximate the model degradation after quantization. Additionally, we improve search performance by implementing entropy regularization, which prioritizes operations and its precision within each search space block. Our experiments confirm the superiority of quantization-aware NAS compared to the two-step process: NAS followed by quantization. Furthermore, approximating quantization with QN offers a 30% speed improvement over direct weight quantization. We validate our approach by developing and applying it to two search spaces inspired by state-of-the-art SR models. Our code is publicly available b .
- Open Access
- 3 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 3898-3909
SeqNAS: Neural Architecture Search for Event Sequence Classification
Neural Architecture Search (NAS) methods are widely used in various industries to obtain high-quality, task-specific solutions with minimal human intervention. Event Sequences (EvS) find widespread use in various industrial applications, including churn prediction, customer segmentation, fraud detection, and fault diagnosis, among others. Such data consist of categorical and real-valued components with irregular timestamps. Despite the usefulness of NAS methods, previous approaches only have been applied to other domains: images, texts or time series. Our work addresses this limitation by introducing a novel NAS algorithm — SeqNAS, specifically designed for event sequence classification. We develop a simple yet expressive search space that leverages commonly used building blocks for event sequence classification, including multi-head self attention, convolutions, and recurrent cells. To perform the search, we adopt sequential Bayesian Optimization and utilize previously trained models as an ensemble of teachers to augment knowledge distillation. As a result of our work, we demonstrate that our method surpasses state-of-the-art NAS methods and popular architectures suitable for sequence classification and holds great potential for various industrial applications.
- 3 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 3898-3909
Open AccessSeqNAS: Neural Architecture Search for Event Sequence Classification
Neural Architecture Search (NAS) methods are widely used in various industries to obtain high-quality, task-specific solutions with minimal human intervention. Event Sequences (EvS) find widespread use in various industrial applications, including churn prediction, customer segmentation, fraud detection, and fault diagnosis, among others. Such data consist of categorical and real-valued components with irregular timestamps. Despite the usefulness of NAS methods, previous approaches only have been applied to other domains: images, texts or time series. Our work addresses this limitation by introducing a novel NAS algorithm — SeqNAS, specifically designed for event sequence classification. We develop a simple yet expressive search space that leverages commonly used building blocks for event sequence classification, including multi-head self attention, convolutions, and recurrent cells. To perform the search, we adopt sequential Bayesian Optimization and utilize previously trained models as an ensemble of teachers to augment knowledge distillation. As a result of our work, we demonstrate that our method surpasses state-of-the-art NAS methods and popular architectures suitable for sequence classification and holds great potential for various industrial applications.
- Open Access
- 5 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 59217-59225
Multi-Fidelity Neural Architecture Search With Knowledge Distillation
Neural architecture search (NAS) targets at finding the optimal architecture of a neural network for a problem or a family of problems. Evaluations of neural architectures are very time-consuming. One of the possible ways to mitigate this issue is to use low-fidelity evaluations, namely training on a part of a dataset, fewer epochs, with fewer channels, etc. In this paper, we propose a Bayesian multi-fidelity method for neural architecture search: MF-KD. The method relies on a new approach to low-fidelity evaluations of neural architectures by training for a few epochs using a knowledge distillation. Knowledge distillation adds to a loss function a term forcing a network to mimic some teacher network. We carry out experiments on CIFAR-10, CIFAR-100, and ImageNet-16-120. We show that training for a few epochs with such a modified loss function leads to a better selection of neural architectures than training for a few epochs with a logistic loss. The proposed method outperforms several state-of-the-art baselines.
- 5 January 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 59217-59225
Open AccessMulti-Fidelity Neural Architecture Search With Knowledge Distillation
Neural architecture search (NAS) targets at finding the optimal architecture of a neural network for a problem or a family of problems. Evaluations of neural architectures are very time-consuming. One of the possible ways to mitigate this issue is to use low-fidelity evaluations, namely training on a part of a dataset, fewer epochs, with fewer channels, etc. In this paper, we propose a Bayesian multi-fidelity method for neural architecture search: MF-KD. The method relies on a new approach to low-fidelity evaluations of neural architectures by training for a few epochs using a knowledge distillation. Knowledge distillation adds to a loss function a term forcing a network to mimic some teacher network. We carry out experiments on CIFAR-10, CIFAR-100, and ImageNet-16-120. We show that training for a few epochs with such a modified loss function leads to a better selection of neural architectures than training for a few epochs with a logistic loss. The proposed method outperforms several state-of-the-art baselines.
- Open Access
- 17 November 2022
- Published by MDPI AG in Remote Sensing
- Vol. 14 (22) , 5837
Data-Driven Short-Term Daily Operational Sea Ice Regional Forecasting
Global warming has made the Arctic increasingly available for marine operations and created a demand for reliable operational sea ice forecasts to increase safety. Because ocean-ice numerical models are highly computationally intensive, relatively lightweight ML-based methods may be more efficient for sea ice forecasting. Many studies have exploited different deep learning models alongside classical approaches for predicting sea ice concentration in the Arctic. However, only a few focus on daily operational forecasts and consider the real-time availability of data needed for marine operations. In this article, we aim to close this gap and investigate the performance of the U-Net model trained in two regimes for predicting sea ice for up to the next 10 days. We show that this deep learning model can outperform simple baselines by a significant margin, and we can improve the model’s quality by using additional weather data and training on multiple regions to ensure its generalization abilities. As a practical outcome, we build a fast and flexible tool that produces operational sea ice forecasts in the Barents Sea, the Labrador Sea, and the Laptev Sea regions.
- 17 November 2022
- Published by MDPI AG in Remote Sensing
- Vol. 14 (22) , 5837
Open AccessData-Driven Short-Term Daily Operational Sea Ice Regional Forecasting
Global warming has made the Arctic increasingly available for marine operations and created a demand for reliable operational sea ice forecasts to increase safety. Because ocean-ice numerical models are highly computationally intensive, relatively lightweight ML-based methods may be more efficient for sea ice forecasting. Many studies have exploited different deep learning models alongside classical approaches for predicting sea ice concentration in the Arctic. However, only a few focus on daily operational forecasts and consider the real-time availability of data needed for marine operations. In this article, we aim to close this gap and investigate the performance of the U-Net model trained in two regimes for predicting sea ice for up to the next 10 days. We show that this deep learning model can outperform simple baselines by a significant margin, and we can improve the model’s quality by using additional weather data and training on multiple regions to ensure its generalization abilities. As a practical outcome, we build a fast and flexible tool that produces operational sea ice forecasts in the Barents Sea, the Labrador Sea, and the Laptev Sea regions.
- Open Access
- 13 July 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 75205-75218
Learned Query Optimizers: Evaluation and Improvement
Query Optimization is considered to be one of the most important challenges in database management. Existing built-in query optimizers are very complex and rely on various approximations and hand-picked rules. The rise of deep learning and deep reinforcement learning has aided many scientific and industrial fields, providing an opportunity to develop a learnable query optimizer. In this paper, we analyse and improve the state-of-the-art learned query optimizer, Neo for the JOB benchmark on two database systems: PostgreSQL and Huawei GaussDB.We describe our methods, based on combination of Neo, Tree-Transformers, auxiliary tasks, reward weighting. Combinations of these methods improve latency of the found query execution plans. We also conduct a thorough analysis of the resulting execution plans and devise a set of decision-based rules to indicate the cases when the learned optimizer will outperform the built-in one. We also provide a source code for the proposed methods and experiments. Finally, we provide possible directions for further improvement in this field.
- 13 July 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 10, 75205-75218
Open AccessLearned Query Optimizers: Evaluation and Improvement
Query Optimization is considered to be one of the most important challenges in database management. Existing built-in query optimizers are very complex and rely on various approximations and hand-picked rules. The rise of deep learning and deep reinforcement learning has aided many scientific and industrial fields, providing an opportunity to develop a learnable query optimizer. In this paper, we analyse and improve the state-of-the-art learned query optimizer, Neo for the JOB benchmark on two database systems: PostgreSQL and Huawei GaussDB.We describe our methods, based on combination of Neo, Tree-Transformers, auxiliary tasks, reward weighting. Combinations of these methods improve latency of the found query execution plans. We also conduct a thorough analysis of the resulting execution plans and devise a set of decision-based rules to indicate the cases when the learned optimizer will outperform the built-in one. We also provide a source code for the proposed methods and experiments. Finally, we provide possible directions for further improvement in this field.
- 7 July 2021
- Published by Springer Nature
- p. 591-606
Optimizing the Neural Architecture of Reinforcement Learning Agents
- 7 July 2021
- Published by Springer Nature
- p. 591-606
Optimizing the Neural Architecture of Reinforcement Learning Agents
- Open Access
Optimizing the Neural Architecture of Reinforcement Learning Agents
Reinforcement learning (RL) enjoyed significant progress over the last years. One of the most important steps forward was the wide application of neural networks. However, architectures of these neural networks are typically constructed manually. In this work, we study recently proposed neural architecture search (NAS) methods for optimizing the architecture of RL agents. We carry out experiments on the Atari benchmark and conclude that modern NAS methods find architectures of RL agents outperforming a manually selected one.
Open AccessOptimizing the Neural Architecture of Reinforcement Learning Agents
Reinforcement learning (RL) enjoyed significant progress over the last years. One of the most important steps forward was the wide application of neural networks. However, architectures of these neural networks are typically constructed manually. In this work, we study recently proposed neural architecture search (NAS) methods for optimizing the architecture of RL agents. We carry out experiments on the Atari benchmark and conclude that modern NAS methods find architectures of RL agents outperforming a manually selected one.