Alexey Kornaev

Experience replay strongly shapes the learning dynamics of off-policy deep reinforcement learning, yet in engineering control it is often treated as passive memory. This paper proposes a cartography-guided adaptive replay framework for a deep Q-network controller in nonlinear dynamical systems. The implemented artificial intelligence method leaves the Bellman update, neural-network architecture, optimizer, exploration schedule, reward function, and environment dynamics unchanged, while modifying only replay prioritization and selective buffer admission. Replay priorities combine temporal-difference error with an entropy-based measure of action-preference ambiguity, and state-space cartography distinguishes stable, ambiguous, and value-unstable regions. Two replay mechanisms are introduced: entropy-selective replay, which separates numerical replay priority from stochastic admission of unstable transitions, and a stratified variant that distinguishes temporal-difference-dominant, entropy-dominant, and mixed instability. The artificial intelligence application is robust control of an active conical fluid-film bearing in a nonlinear rotor-bearing system. The method is evaluated on four benchmark control environments and on the rotor-bearing model. The benchmark results show improved late-stage stability or robustness in seed-sensitive tasks. In the rotor-bearing case, entropy-selective replay gives the strongest nominal performance, whereas the stratified variant provides the best robustness to observation noise and surrogate-model perturbations. These results show that state-aware replay management can improve robust artificial intelligence control without changing the underlying learning backbone.

This study evaluates the performance of classical and modern control methods for real-world Cable-Driven Parallel Robots (CDPRs), focusing on underconstrained systems with limited time discretization. A comparative analysis is conducted between classical PID controllers and modern reinforcement learning algorithms, including Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Trust Region Policy Optimization (TRPO). The results demonstrate that TRPO outperforms other methods, achieving the lowest root mean square (RMS) errors across various trajectories and exhibiting robustness to larger time intervals between control updates. TRPO's ability to balance exploration and exploitation enables stable control in noisy, real-world environments, reducing reliance on high-frequency sensor feedback and computational demands. These findings highlight TRPO's potential as a robust solution for complex robotic control tasks, with implications for dynamic environments and future applications in sensor fusion or hybrid control strategies.

Landmark detection using chest X-ray images is often a time-consuming task performed by radiologists, followed by the calculation of cardiometric indices and the diagnosis of heart diseases. The goal of this research is to estimate uncertainty in data and knowledge, which leads to increased accuracy and interpretability in models that predict landmark positions and classify human body states. To achieve this, data from about 800 X-ray images labeled by four practicing radiologists was utilized. An uncertaintyaware negative log-likelihood loss was proposed to address both regression and classification problems. The proposed models were trained to predict the mean and log variance of a normal distribution for each input image, with the variance value estimating the uncertainty of the prediction. Natural data uncertainty was also measured from labels provided by different radiologists and implemented in the form of a label transform for the coordinates of the landmarks. The classification models demonstrated good correspondence with methods such as Monte Carlo dropout and deep ensembling, which account for knowledge uncertainty. The proposed uncertainty-aware models achieved the best results in both landmark detection and heart disease diagnosis, with accuracy improvements of 3% and 4.61%, respectively. The proposed methods are effective and easy to use, making their application to other medical problems a subject of interest for further research.

Chest radiography is an indispensable diagnostic method for detecting a variety of medical conditions, such as infections, tumors, injuries, etc. Millions of chest X-ray examinations are conducted annually, providing crucial information about the functioning of the respiratory and circulatory systems. The conventional approach to quantifying cardiothoracic indices, such as the Lupi and Moore indices and the Cardiothoracic Index (CTI), requires considerable time and effort from radiologists. Consequently, it calls for the exploration of computational methods for improvement through deep learning. In this study, we addressed the challenge of automating the calculation of these cardiometric indices. We engaged four experienced radiologists to manually label 800 chest X-ray images each. Using these labeled images, we trained a deep learning model that achieved the level of performance comparable to that of a professional radiologist. Additionally, we have replaced the central points of the indices with landmarks based on the vertebrae, improving the accuracy. The use of AI led to improved accuracy of correct predictions, increasing it from 85.94% to 87.34% for the MOORE coefficient and from 87.55% to 90.67% for the LUPI coefficient.

Microscopic images of cells often contain multiple objects with similar properties. In this study, we collected videos of blood microcirculation in nailfold capillaries to identify correlations between video data and indicators of complete blood count tests. The specific features of these videos include a high number of capillaries and a high variance in their visual quality. Therefore, we proposed a novel loss function and a video processing framework that combines metric learning and classification with uncertainty estimation. We also applied test-time augmentation to enhance the confidence of the predictions. During simulation experiments, we used shallow and deep learning models to compare a set of classifiers for the collected metadata. Both types of models demonstrated good results, with an accuracy of up to 80% in gender classification. Deep learning models outperformed shallow models and also classified the age of the volunteers with an accuracy of 74%. Subsequently, we addressed a series of binary classification tasks to identify conditionally high or low levels of the main complete blood count indicators. The obtained accuracy of the models reached 63%. Although the accuracy of the blood test classifiers is not yet sufficient for clinical purposes, we have demonstrated that video data correlates with some of the indicators.

One of the ways to make artificial intelligence more natural is to give it some room for doubt. Two main questions should be resolved in that way. First, how to train a model to estimate uncertainties of its own predictions? And then, what to do with the uncertain predictions if they appear? First, we proposed an uncertainty-aware negative log-likelihood loss for the case of N-dimensional multivariate normal distribution with spherical variance matrix to the solution of N-classes classification tasks. The loss is similar to the heteroscedastic regression loss. The proposed model regularizes uncertain predictions, and trains to calculate both the predictions and their uncertainty estimations. The model fits well with the label smoothing technique. Second, we expanded the limits of data augmentation at the training and test stages, and made the trained model to give multiple predictions for a given number of augmented versions of each test sample. Given the multi-view predictions together with their uncertainties and confidences, we proposed several methods to calculate final predictions, including mode values and bin counts with soft and hard weights. For the latter method, we formalized the model tuning task in the form of multimodal optimization with non-differentiable criteria of maximum accuracy, and applied particle swarm optimization to solve the tuning task. The proposed methodology was tested using CIFAR-10 dataset with clean and noisy labels and demonstrated good results in comparison with other uncertainty estimation methods related to sample selection, co-teaching, and label smoothing.

Maxillary sinus pathologies remain among the most common ENT diseases requiring timely diagnosis for successful treatment. Standard ENT inspection approaches indicate low sensitivity in detecting maxillary sinus pathologies. In this paper, we report on capabilities of digital diaphanoscopy combined with machine learning tools in the detection of such pathologies. We provide a comparative analysis of two machine learning approaches applied to digital diapahnoscopy data, namely, convolutional neural networks and linear discriminant analysis. The sensitivity and specificity values obtained for both employed approaches exceed the reported accuracy indicators for traditional screening diagnosis methods (such as nasal endoscopy or ultrasound), suggesting the prospects of their usage for screening maxillary sinuses alterations. The analysis of the obtained values showed that the linear discriminant analysis, being a simpler approach as compared to neural networks, allows one to detect the maxillary sinus pathologies with the sensitivity and specificity of 0.88 and 0.98, respectively.