Andrey Zhdanov
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Andrey Zhdanov has a total of 41 co-authors. Here are the top co-authors by number of shared publications:
- Dmitri Dmitrievich Zhdanov
ITMO University
23 shared publications Dmitry Zhdanov
ITMO Univ. (Russian Federation)
13 shared publications- Igor Stanislavovich Potemin
ITMO University
8 shared publications - Maxim Igorevich Sorokin
ITMO University
8 shared publications Igor Potemin
ITMO Univ. (Russian Federation)
5 shared publicationsYan Wang
ITMO Univ. (Russian Federation)
5 shared publications
Latest Publications
- 27 November 2024
- Published by SPIE-Intl Soc Optical Eng
- Vol. 13237, 16-58
Constructing a metric for the selection of a starting optical design
In computer-aided design of optical systems, the choice of starting point is very important. Typically, the designer selects the initial optical design from any database available to him based on his own experience and technical specifications for designing the required optical system. To date, thousands of optical systems of various purposes and complexity have been designed. This diversity significantly complicates the process of selecting an initial circuit for its further optimization. The authors propose a method for constructing an evaluation function for the automated selection of a starting optical design. The proposed methodology combines the classification of optical systems by their type and the complexity of their structural design and includes requirements for image quality, size and weight characteristics, spectral range, manufacturing complexity, and estimation of manufacturing costs for mass production.
- 27 November 2024
- Published by SPIE-Intl Soc Optical Eng
- Vol. 13237, 16-58
Constructing a metric for the selection of a starting optical design
In computer-aided design of optical systems, the choice of starting point is very important. Typically, the designer selects the initial optical design from any database available to him based on his own experience and technical specifications for designing the required optical system. To date, thousands of optical systems of various purposes and complexity have been designed. This diversity significantly complicates the process of selecting an initial circuit for its further optimization. The authors propose a method for constructing an evaluation function for the automated selection of a starting optical design. The proposed methodology combines the classification of optical systems by their type and the complexity of their structural design and includes requirements for image quality, size and weight characteristics, spectral range, manufacturing complexity, and estimation of manufacturing costs for mass production.
- 23 November 2024
- Published by SPIE-Intl Soc Optical Eng
- Vol. 13239, 21-163
Design and virtual prototyping of a compact augmented reality system with 2D diffraction grating
This paper presents a computer model of an optically transparent augmented reality system built using a configuration with two diffraction elements. A distinctive feature of this device is that the second diffraction element is a two-dimensional diffraction grating that combines functions of beam expansion and image extraction from the light guide. The technology for manufacturing two-dimensional diffraction gratings is presented. Formulas for calculating the direction of beams diffracted by the diffraction gratings, including the two-dimensional grating, are proposed. Two ray-tracing algorithms for image formation by optical augmented reality systems containing diffraction gratings, including two-dimensional ones, are proposed. Based on these algorithms, virtual prototyping of several augmented reality systems was conducted. Design features, advantages, and disadvantages of these systems were identified. Recommendations for the design and optimization of augmented reality systems using a dual diffraction element configuration are provided.
- 23 November 2024
- Published by SPIE-Intl Soc Optical Eng
- Vol. 13239, 21-163
Design and virtual prototyping of a compact augmented reality system with 2D diffraction grating
This paper presents a computer model of an optically transparent augmented reality system built using a configuration with two diffraction elements. A distinctive feature of this device is that the second diffraction element is a two-dimensional diffraction grating that combines functions of beam expansion and image extraction from the light guide. The technology for manufacturing two-dimensional diffraction gratings is presented. Formulas for calculating the direction of beams diffracted by the diffraction gratings, including the two-dimensional grating, are proposed. Two ray-tracing algorithms for image formation by optical augmented reality systems containing diffraction gratings, including two-dimensional ones, are proposed. Based on these algorithms, virtual prototyping of several augmented reality systems was conducted. Design features, advantages, and disadvantages of these systems were identified. Recommendations for the design and optimization of augmented reality systems using a dual diffraction element configuration are provided.
- 23 November 2024
- Published by SPIE-Intl Soc Optical Eng
- Vol. 13239, 16-118
Two-step method for geometry reconstruction using differentiable rendering
The task of reconstructing real-world objects arises in many areas of human life. Inverse rendering methods show high efficiency in solving this problem. For accurate reconstruction of object geometry from a set of images, we propose a geometry reconstruction method based on differentiable rendering. This method consists of two steps: forming a bounding volume and optimizing the geometry. To achieve high-speed geometry reconstruction, we suggest reducing the time spent on optimizing the silhouette gradients by conducting an initial stage of constructing the bounding volume of the object. The article considers the possibility of using three-dimensional visual hulls as a primary representation for optimizing geometry using differentiable rendering methods. Introducing this stage of geometry reconstruction allows the inverse rendering to start with the condition of matching the reconstructed object's contour and topology. Thus, geometry optimization occurs considering shading gradients, eliminating the need to account for object silhouette gradients. The proposed approach solves the problem related to potential distortion of the object's geometry, such as normal inversion or self-intersecting triangles. The paper presents the results of applying the proposed method to reconstruct test objects from image sets and demonstrates the increase in accuracy of geometry reconstruction.
- 23 November 2024
- Published by SPIE-Intl Soc Optical Eng
- Vol. 13239, 16-118
Two-step method for geometry reconstruction using differentiable rendering
The task of reconstructing real-world objects arises in many areas of human life. Inverse rendering methods show high efficiency in solving this problem. For accurate reconstruction of object geometry from a set of images, we propose a geometry reconstruction method based on differentiable rendering. This method consists of two steps: forming a bounding volume and optimizing the geometry. To achieve high-speed geometry reconstruction, we suggest reducing the time spent on optimizing the silhouette gradients by conducting an initial stage of constructing the bounding volume of the object. The article considers the possibility of using three-dimensional visual hulls as a primary representation for optimizing geometry using differentiable rendering methods. Introducing this stage of geometry reconstruction allows the inverse rendering to start with the condition of matching the reconstructed object's contour and topology. Thus, geometry optimization occurs considering shading gradients, eliminating the need to account for object silhouette gradients. The proposed approach solves the problem related to potential distortion of the object's geometry, such as normal inversion or self-intersecting triangles. The paper presents the results of applying the proposed method to reconstruct test objects from image sets and demonstrates the increase in accuracy of geometry reconstruction.
- 11 June 2024
- Published by Pleiades Publishing Ltd in Programming and Computer Software
- Vol. 50 (3) , 273-282
Dual Representation of Geometry for Ray Tracing Acceleration in Optical Systems with Freeform Surfaces
This paper explores the possibility of using dual representation of geometry to speed up ray tracing and ensure robustness of light propagation simulation in complex optical systems with freeform surfaces defined by high-degree polynomials (up to order 34) or Jacobi polynomials. Traditional methods for representing this geometry both as a triangular mesh and as an analytical expression are analyzed. The analysis demonstrates the disadvantages of the traditional approaches due to the insufficient accuracy of calculating coordinates of the point at which the ray intersects the triangular mesh, as well as the non-robustness of the conventional procedures for finding the point of intersection between the tangent ray and the analytical surface. Thus, it is proposed to use a dual representation of geometry as a rough approximation of the surface by a triangular mesh, which is subsequently used as an initial approximation to find the point at which the ray intersects the surface defined by an analytical expression. This approach significantly speeds up convergence of the analytical methods and improves robustness of their solutions. The use of the Intel® Embree library to quickly find the point of intersection between the ray and the rough triangular mesh, as well as a vector model to refine the coordinates of the point of intersection between the ray and the analytically represented geometry, allows us to develop and implement a ray tracing algorithm for an optical system that has surfaces with dual representation of geometry. Experiments carried out using the developed algorithm show a significant speedup of ray tracing while preserving computational accuracy and high robustness of the results. The results are demonstrated by evaluating the point spread function and glare for two lenses with freeform surfaces defined by Jacobi polynomials. In addition, for these two lenses, an image formed by an RGB-D object that simulates a real scene is calculated.
- 11 June 2024
- Published by Pleiades Publishing Ltd in Programming and Computer Software
- Vol. 50 (3) , 273-282
Dual Representation of Geometry for Ray Tracing Acceleration in Optical Systems with Freeform Surfaces
This paper explores the possibility of using dual representation of geometry to speed up ray tracing and ensure robustness of light propagation simulation in complex optical systems with freeform surfaces defined by high-degree polynomials (up to order 34) or Jacobi polynomials. Traditional methods for representing this geometry both as a triangular mesh and as an analytical expression are analyzed. The analysis demonstrates the disadvantages of the traditional approaches due to the insufficient accuracy of calculating coordinates of the point at which the ray intersects the triangular mesh, as well as the non-robustness of the conventional procedures for finding the point of intersection between the tangent ray and the analytical surface. Thus, it is proposed to use a dual representation of geometry as a rough approximation of the surface by a triangular mesh, which is subsequently used as an initial approximation to find the point at which the ray intersects the surface defined by an analytical expression. This approach significantly speeds up convergence of the analytical methods and improves robustness of their solutions. The use of the Intel® Embree library to quickly find the point of intersection between the ray and the rough triangular mesh, as well as a vector model to refine the coordinates of the point of intersection between the ray and the analytically represented geometry, allows us to develop and implement a ray tracing algorithm for an optical system that has surfaces with dual representation of geometry. Experiments carried out using the developed algorithm show a significant speedup of ray tracing while preserving computational accuracy and high robustness of the results. The results are demonstrated by evaluating the point spread function and glare for two lenses with freeform surfaces defined by Jacobi polynomials. In addition, for these two lenses, an image formed by an RGB-D object that simulates a real scene is calculated.
- 11 June 2024
- Published by Pleiades Publishing Ltd in Programming and Computer Software
- Vol. 50 (3) , 249-256
Conversion of Point Cloud Data to 3D Models Using PointNet++ and Transformer
This paper presents an approach to 3D model reconstruction from point cloud data using modern neural network architectures. The method is based on PointNet++ and Transformer. PointNet++ plays a key role, providing efficient feature extraction and encoding of complex geometries in 3D scenes. This is achieved by recursively applying PointNet++ to nested partitions of the input point set in a metric space. Convex decomposition, which is an important step in the proposed approach, transforms complex 3D objects into sets of simpler convex shapes. This facilitates data processing and makes the reconstruction process more manageable. Then, Transformer trains the model on the extracted features, which enables the generation of high-quality reconstructions. It should be noted that Transformer is used only to determine positions of the walls and detect object boundaries. This combination of technologies allows us to achieve high accuracy of 3D model reconstruction. The main idea of the method is to segment the point cloud into small fragments, which are then reconstructed as polygon meshes. To restore missing points in point cloud data, a method based on the L1-median algorithm and local features of the point cloud is used. This approach can adapt to different geometric structures and correct topological connectivity errors. The proposed method is compared with several modern approaches, demonstrating its potential in various fields, including architecture, engineering, digitalization of cultural heritage, as well as augmented and mixed reality systems. This confirms its wide applicability and great perspectives for further development.
- 11 June 2024
- Published by Pleiades Publishing Ltd in Programming and Computer Software
- Vol. 50 (3) , 249-256
Conversion of Point Cloud Data to 3D Models Using PointNet++ and Transformer
This paper presents an approach to 3D model reconstruction from point cloud data using modern neural network architectures. The method is based on PointNet++ and Transformer. PointNet++ plays a key role, providing efficient feature extraction and encoding of complex geometries in 3D scenes. This is achieved by recursively applying PointNet++ to nested partitions of the input point set in a metric space. Convex decomposition, which is an important step in the proposed approach, transforms complex 3D objects into sets of simpler convex shapes. This facilitates data processing and makes the reconstruction process more manageable. Then, Transformer trains the model on the extracted features, which enables the generation of high-quality reconstructions. It should be noted that Transformer is used only to determine positions of the walls and detect object boundaries. This combination of technologies allows us to achieve high accuracy of 3D model reconstruction. The main idea of the method is to segment the point cloud into small fragments, which are then reconstructed as polygon meshes. To restore missing points in point cloud data, a method based on the L1-median algorithm and local features of the point cloud is used. This approach can adapt to different geometric structures and correct topological connectivity errors. The proposed method is compared with several modern approaches, demonstrating its potential in various fields, including architecture, engineering, digitalization of cultural heritage, as well as augmented and mixed reality systems. This confirms its wide applicability and great perspectives for further development.
- 27 November 2023
- Published by SPIE-Intl Soc Optical Eng
- Vol. 12767, 24-176
Effective caustics simulation for real-time rendering applications
In this paper, we discuss different approaches which may be used to solve the caustic illumination problem including the approximation of the light wavefront method, the backward and forward photon mappings, and propose the method for the effective calculation of the caustic illumination maps to be integrated into a common DirectX rendering pipeline. The proposed method is based on the photon mapping method and adds additional rendering pipeline steps for calculating the visibility maps for forward and backward caustics and then merging them to form the caustic illumination map of the whole scene. The proposed method was implemented, tested, and compared with the ground truth result acquired with realistic rendering software. The test results along with rendered images are presented.
- 27 November 2023
- Published by SPIE-Intl Soc Optical Eng
- Vol. 12767, 24-176
Effective caustics simulation for real-time rendering applications
In this paper, we discuss different approaches which may be used to solve the caustic illumination problem including the approximation of the light wavefront method, the backward and forward photon mappings, and propose the method for the effective calculation of the caustic illumination maps to be integrated into a common DirectX rendering pipeline. The proposed method is based on the photon mapping method and adds additional rendering pipeline steps for calculating the visibility maps for forward and backward caustics and then merging them to form the caustic illumination map of the whole scene. The proposed method was implemented, tested, and compared with the ground truth result acquired with realistic rendering software. The test results along with rendered images are presented.
- 27 November 2023
- Published by SPIE-Intl Soc Optical Eng
- Vol. 12767, 177-195
Federative rendering model
In some cases, there is a problem associated with the synthesis of images of scenes whose data cannot be transferred to the rendering system, while the rendering software cannot be installed on the customer's hardware. This may be related to the fear of possible data leakage and the unlicensed use of the software. To solve this problem, we propose a remote rendering method that uses the remote interaction between the isolated rendering system which controls the rendering process and runs on the server side, and scene data which provides basic rendering functions and runs on the client side.
- 27 November 2023
- Published by SPIE-Intl Soc Optical Eng
- Vol. 12767, 177-195
Federative rendering model
In some cases, there is a problem associated with the synthesis of images of scenes whose data cannot be transferred to the rendering system, while the rendering software cannot be installed on the customer's hardware. This may be related to the fear of possible data leakage and the unlicensed use of the software. To solve this problem, we propose a remote rendering method that uses the remote interaction between the isolated rendering system which controls the rendering process and runs on the server side, and scene data which provides basic rendering functions and runs on the client side.
- 26 May 2023
- Published by Pleiades Publishing Ltd in Programming and Computer Software
- Vol. 49 (3) , 161-171
Using Multilevel Hash Tables to Speed up Rendering
In this paper, we analyze realistic rendering methods in terms of their efficiency in calculating caustic and indirect illumination. As the basic approach to realistic rendering, we choose bidirectional progressive ray tracing with backward photon maps. The main factors that reduce the efficiency of this method are analyzed. It is shown that the main factor that affects its performance is slow access to photon map data. Different techniques for construction of spatial acceleration structures are considered, their advantages and disadvantages are investigated. As the main approaches, we select the regular spatial grid and binary kd-tree. The spatial grid provides high-speed access to photon data at low adaptability of photon map partitioning. The kd-tree is characterized by high spatial adaptability of photon map partitioning but slow access to photon data. We propose a combined solution that takes advantage of the adaptability of the kd-tree and the fast data access of the spatial grid. For this purpose, the regular grid is superimposed on the kd-tree constructed based on the principle of space partitioning of a photon region into geometrically identical halves. To reduce memory consumption, we propose, first, to use multilevel spatial grids superimposed on the selected nodes of the kd-tree and, second, to store spatial grids in the form of hash tables in order to reduce the size of the acceleration structure. Thus, a spatial acceleration structure of a new type—a tree of hash tables—is proposed and implemented. For the spatial structure developed, we implement methods for finding the nearest photons the integration spheres of which cover the illumination point, as well as methods for finding the intersection between a ray segment and photon integration spheres. The proposed software solutions are implemented in the Lumicept software package; for some scenes, the proposed method is compared with the Lumicept method based on the binary tree. The comparison shows that our method can increase the overall speed of the rendering process by more than 40%.
- 26 May 2023
- Published by Pleiades Publishing Ltd in Programming and Computer Software
- Vol. 49 (3) , 161-171
Using Multilevel Hash Tables to Speed up Rendering
In this paper, we analyze realistic rendering methods in terms of their efficiency in calculating caustic and indirect illumination. As the basic approach to realistic rendering, we choose bidirectional progressive ray tracing with backward photon maps. The main factors that reduce the efficiency of this method are analyzed. It is shown that the main factor that affects its performance is slow access to photon map data. Different techniques for construction of spatial acceleration structures are considered, their advantages and disadvantages are investigated. As the main approaches, we select the regular spatial grid and binary kd-tree. The spatial grid provides high-speed access to photon data at low adaptability of photon map partitioning. The kd-tree is characterized by high spatial adaptability of photon map partitioning but slow access to photon data. We propose a combined solution that takes advantage of the adaptability of the kd-tree and the fast data access of the spatial grid. For this purpose, the regular grid is superimposed on the kd-tree constructed based on the principle of space partitioning of a photon region into geometrically identical halves. To reduce memory consumption, we propose, first, to use multilevel spatial grids superimposed on the selected nodes of the kd-tree and, second, to store spatial grids in the form of hash tables in order to reduce the size of the acceleration structure. Thus, a spatial acceleration structure of a new type—a tree of hash tables—is proposed and implemented. For the spatial structure developed, we implement methods for finding the nearest photons the integration spheres of which cover the illumination point, as well as methods for finding the intersection between a ray segment and photon integration spheres. The proposed software solutions are implemented in the Lumicept software package; for some scenes, the proposed method is compared with the Lumicept method based on the binary tree. The comparison shows that our method can increase the overall speed of the rendering process by more than 40%.
- 26 May 2023
- Published by Pleiades Publishing Ltd in Programming and Computer Software
- Vol. 49 (3) , 151-160
3D Scene Reconstruction and Digitization Method for Mixed Reality Systems
Mixed reality systems are a promising direction of research that opens up great opportunities for interaction with virtual objects in the real world. Like any promising direction, mixed reality has a number of unresolved problems. One of these problems is the synthesis of natural lighting conditions for virtual objects, including the correct light interaction of virtual objects with the real world. Since virtual and real objects are located in different spaces, it is difficult to ensure their correct interaction. To create digital copies of real-world objects, machine learning tools and neural network technologies are employed. These methods are successfully used in computer vision for space orientation and environment reconstruction. As a solution, it is proposed to transfer all objects into the same information space: virtual space. This makes it possible to solve most of the problems associated with visual discomfort caused by the unnatural light interaction of real and virtual objects. Thus, the basic idea of the method is to recognize physical objects from point clouds and replace these objects with virtual CAD models. In other words, it implies semantic analysis of a scene and classification of objects with their subsequent transformation into polygonal models. In this study, we use competitive neural network architectures, which can provide state-of-the-art results. The test experiments are carried out on Semantic3D, ScanNet, and S3DIS, which are currently the largest datasets with point clouds that represent indoor scenes. For semantic segmentation and classification of 3D point clouds, we use the PointNeXt architecture based on PointNet, as well as modern methods of data augmentation in the process of learning. For geometry reconstruction, the Soft Rasterizer differentiable rendering method and the Total3Understanding neural network are considered.
- 26 May 2023
- Published by Pleiades Publishing Ltd in Programming and Computer Software
- Vol. 49 (3) , 151-160
3D Scene Reconstruction and Digitization Method for Mixed Reality Systems
Mixed reality systems are a promising direction of research that opens up great opportunities for interaction with virtual objects in the real world. Like any promising direction, mixed reality has a number of unresolved problems. One of these problems is the synthesis of natural lighting conditions for virtual objects, including the correct light interaction of virtual objects with the real world. Since virtual and real objects are located in different spaces, it is difficult to ensure their correct interaction. To create digital copies of real-world objects, machine learning tools and neural network technologies are employed. These methods are successfully used in computer vision for space orientation and environment reconstruction. As a solution, it is proposed to transfer all objects into the same information space: virtual space. This makes it possible to solve most of the problems associated with visual discomfort caused by the unnatural light interaction of real and virtual objects. Thus, the basic idea of the method is to recognize physical objects from point clouds and replace these objects with virtual CAD models. In other words, it implies semantic analysis of a scene and classification of objects with their subsequent transformation into polygonal models. In this study, we use competitive neural network architectures, which can provide state-of-the-art results. The test experiments are carried out on Semantic3D, ScanNet, and S3DIS, which are currently the largest datasets with point clouds that represent indoor scenes. For semantic segmentation and classification of 3D point clouds, we use the PointNeXt architecture based on PointNet, as well as modern methods of data augmentation in the process of learning. For geometry reconstruction, the Soft Rasterizer differentiable rendering method and the Total3Understanding neural network are considered.
- 1 January 2023
- Published by Keldysh Institute of Applied Mathematics
Automatic Creation and Annotation of RGB-D Images for Training Machine Vision Systems
Due to the active development of artificial intelligence technologies, machine vision, and deep learning, as well as the emergence of RGB?D cameras that allow you to get a three-dimensional image of the scene, more and more attention is paid to various tasks of processing three-dimensional data. One of these problems is the problem of point cloud segmentation, which is used in various fields, from robotics to architecture, and is solved by machine vision methods. The training of machine vision systems requires the creation and annotation of datasets, which takes up a significant part of the design and development time. In this paper, it is proposed to automate the process of creating a dataset using a scripting interpreter and realistic rendering computer systems, which can significantly reduce the time required to create a dataset. An example of creating a dataset, training a neural network on this dataset, and using a network trained on this dataset to classify objects in a scene image is given.
- 1 January 2023
- Published by Keldysh Institute of Applied Mathematics
Automatic Creation and Annotation of RGB-D Images for Training Machine Vision Systems
Due to the active development of artificial intelligence technologies, machine vision, and deep learning, as well as the emergence of RGB?D cameras that allow you to get a three-dimensional image of the scene, more and more attention is paid to various tasks of processing three-dimensional data. One of these problems is the problem of point cloud segmentation, which is used in various fields, from robotics to architecture, and is solved by machine vision methods. The training of machine vision systems requires the creation and annotation of datasets, which takes up a significant part of the design and development time. In this paper, it is proposed to automate the process of creating a dataset using a scripting interpreter and realistic rendering computer systems, which can significantly reduce the time required to create a dataset. An example of creating a dataset, training a neural network on this dataset, and using a network trained on this dataset to classify objects in a scene image is given.