Thodoris Betsas

Cultural Heritage (CH) monuments are strongly characterized by detailed architectural elements, inherent complexity, and heterogeneity and therefore present unique challenges regarding 3D Semantic Segmentation (3DSS), which is a useful tool for documentation enhancement and for empowering preservation actions. This study explores the generalization capability of recent deep learning 3DSS architectures applied to cultural heritage (CH) point cloud data. Using the ArCH benchmark, we evaluate five representative models, including PointNet, PointNet++, Point Transformer v1, v2 and Omni-Adaptive CNNs. All models are assessed using a uniform pipeline and limited input features (XYZ and RGB). Both qualitative and quantitative results indicate that Point Transformer v1 achieves strong performance on unseen CH data (61.3 mIoU), suggesting a potential link between architectural design and generalization ability in CH domain. These findings highlighting the need for further research under varying configurations and broader evaluation settings, especially for recent deep learning architecturs e.g., transformers.

Edge detection is essential for numerous applications in various engineering and scientific fields, including photogrammetry and computer vision. Edge detection can be applied to a variety of 2D and 3D data types, enabling tasks like feature extraction, object recognition and scene reconstruction. Currently, 2D edge detection in image data can achieve high accuracy through various automated methods. At the same time, edge detection in 3D space remains a challenge due to the computational demands and parameterization of existing algorithms. However, with the growing volume of data, the need for automated edge extraction that delivers high accuracy and reliable performance across diverse datasets has become more critical than ever. In this context, we propose an algorithm that implements a direct method for automated 3D edge detection in point clouds. The suggested method significantly aids architects by automating the extraction of 3D vectors, a process that is traditionally time-consuming and labor-intensive when performed manually. The proposed algorithm performs edge detection through a five-stage pipeline. More specifically, it utilizes the differences in the direction of normal vectors to identify finite edges. These edges are afterwards refined and grouped into segments which are then fitted to highlight the presence of 3D edges. The proposed approach was tested on both simulated and real-world data with promising results in terms of accuracy. For the synthetic data, the proposed method managed to detect 92% of the straight edges for the higher density meshes.

In this paper, an exhaustive review and comprehensive analysis of recent and former deep learning methods in 3D semantic segmentation (3DSS) is presented. In the related literature, the taxonomy scheme used for the classification of 3DSS deep learning methods is ambiguous. Based on the taxonomy schemes of nine existing review papers, a new taxonomy scheme for 3DSS deep learning methods is proposed, aiming to standardize it and improve the comparability and clarity across related studies. Furthermore, an extensive overview of the available 3DSS indoor and outdoor datasets is provided along with their links. The core part of this review is the detailed presentation of recent and former 3DSS deep learning methods and their classification using the proposed taxonomy scheme along with their GitHub repositories. Additionally, a brief but informative analysis of the evaluation metrics and loss functions used in 3DSS is included. Finally, a fruitful discussion of the examined 3DSS methods and datasets is presented to foster new research directions and applications in the field of 3DSS. In addition to this review, a GitHub repository is provided, including an initial classification of over 400 3DSS methods, using the proposed taxonomy scheme.

In this paper an exhaustive review and comprehensive analysis of recent and former deep learning methods in 3D Semantic Segmentation (3DSS) is presented. In the related literature, the taxonomy scheme used for the classification of the 3DSS deep learning methods is ambiguous. Based on the taxonomy schemes of 9 existing review papers, a new taxonomy scheme of the 3DSS deep learning methods is proposed, aiming to standardize it and improve the comparability and clarity across related studies. Furthermore, an extensive overview of the available 3DSS indoor and outdoor datasets is provided along with their links. The core part of the review is the detailed presentation of recent and former 3DSS deep learning methods and their classification using the proposed taxonomy scheme along with their GitHub repositories. Additionally, a brief but informative analysis of the evaluation metrics and loss functions used in 3DSS is included. Finally, a fruitful discussion of the examined 3DSS methods and datasets, is presented to foster new research directions and applications in the field of 3DSS. Supplementary, to this review a GitHub repository is provided (https://github.com/thobet/Deep-Learning-on-3D-Semantic-Segmentation-a- Detailed-Review) including a quick classification of over 400 3DSS methods, using the proposed taxonomy scheme.

Orthoimages are a common product used as a base in CAD software for vectorization purposes. In fact, vectorization of orthoimages constitutes a tedious and labour-intensive process which should be supervised by experts e.g. architects, chemical engineers etc. On the one hand, deep learning algorithms are used extensively nowadays achieving high quality results. On the other hand, deep learning algorithms require a huge amount of manually annotated data to be trained on, which is a very difficult process especially at pixel level applications like semantic segmentation and instance segmentation. However, the transformation of 2D CAD drawings into a suitable deep learning dataset (CAD2DLD) is underexplored ignoring a large source of data, created by experts. In this effort, the InCAD algorithm is proposed, which aims to automatically create 2D vector drawings using the YOLOv8 instance segmentation algorithm which was trained on CAD2DLD data. Additionally, a methodology for transforming 2D CAD drawings into a suitable deep learning dataset for instance segmentation, is presented. Finally, the proposed workflow is evaluated on the creation of 2D vector drawings of stones of a fortification wall achieving promising results (78.34 mIoU).

Edge detection is supported by extensive research and is part of different photogrammetric and computer vision tasks across numerous application areas. While 2D edge detection may achieve high accuracy results from several automated methods, the automation of edge detection in 3D space remains a challenge. Existing methods are often computationally demanding and heavily parameterized, leading to a lack of adaptability. In real-world applications 3D edges, representing the object boundaries and break lines, are crucial, particularly in fields such as computer vision, robotics and architecture. In this context, we present a method that automates 3D edge detection in 3D point clouds exploiting the normal vectors’ direction differences to detect finite edges, which are further pruned and grouped to edge segments and fitted to indicate the presence of a 3D edge.

The creation of 2D–3D architectural vector drawings constitutes a manual, labor-intensive process. The scientific community has not provided an automated approach for the production of 2D–3D architectural drawings of cultural-heritage objects yet, regardless of the undoubtable need of many scientific fields. This paper presents an automated method which addresses the problem of detecting 3D edges in point clouds by leveraging a set of RGB images and their 2D edge maps. More concretely, once the 2D edge maps have been produced exploiting manual, semi-automated or automated methods, the RGB images are enriched with an extra channel containing the edge semantic information corresponding to each RGB image. The four-channel images are fed into a Structure from Motion–Multi View Stereo (SfM-MVS) software and a semantically enriched dense point cloud is produced. Then, using the semantically enriched dense point cloud, the points belonging to a 3D edge are isolated from all the others based on their label value. The detected 3D edge points are decomposed into set of points belonging to each edge and fed into the 3D vectorization procedure. Finally, the 3D vectors are saved into a “.dxf” file. The previously described steps constitute the 3DPlan software, which is available on GitHub. The efficiency of the proposed software was evaluated on real-world data of cultural-heritage assets.

Point cloud segmentation, is a widespread field of research and it is useful in several research topics and applications such as 3D point cloud analysis, scene understanding, semantic segmentation etc. Architectural vector drawings constitute a valuable platform source for scientists and craftsmen while the production of such drawings is time-consuming because many of the creation steps are done manually. Detecting 3D edges in point clouds could provide useful information for the automation of the creation of 3D architectural vector drawings. Hence, a 3D edge detection method is proposed and evaluated with a proof-of-concept experiment and another one using a professional software. The scope of this effort is twofold, firstly the production of semantically enriched 3D dense point clouds exploiting four-channel images in order to detect 3D edges and secondly the comparison of the detected 3D edges with their corresponding edges in a textured 3D model. Comparing 3D edges in the early step of the 3D dense point cloud production and in the final step of 3D textured mesh, provides useful conclusions of the data used for the automatic creation of 3D drawings. Both of the experiments i.e., the proof-of-concept and using the professional SfM-MVS software were conducted using real world data of cultural heritage objects.