Juuso Autiosalo

Analyzing multi-vendor mechanical system designs requires a significant amount of manual work, resulting in a design paradigm where analysis is conducted only after the design is locked and components are selected. This leads to a suboptimal design with compatibility issues, over-dimensioned components, inferior performance, poor energy efficiency, and a lack of collaboration between OEMs (original equipment manufacturers) and system integrators. To overcome these issues, this paper proposes Co-Des (collaborative design) framework for automated and collaborative multi-vendor system design. The framework relies on standardized digital twin documents (DTD) of system designs, components, and analyses. The discoverability and distribution of these DTDs are enabled with digital twin web (DTW). Co-Des framework allows for finding suitable components for the design task by automatically running selected analyses employing component digital twins. In addition, OEMs can provide customized components for system integrators using the initial system design defined in the system design DTD. The use of the Co-Des framework was demonstrated with a windmill powertrain design use case, and the applicability of the automated assembly analysis for component selection was verified with performance measurements. The adoption of the proposed framework will lead to a paradigm shift from manual and siloed work relying on the exchange of PDFs to a more automated and collaborative design of mechanical systems. The adoption rate is defined by the willingness of system integrators to publish their initial system designs and OEMs their components as public digital twins.

Most deep learning-based depth estimation models that learn scene structure self-supervised from monocular video base their estimation on visual cues such as vanishing points. In the established depth estimation benchmarks depicting, for example, street navigation or indoor offices, these cues can be found consistently, which enables neural networks to predict depth maps from single images. In this work, we are addressing the challenge of depth estimation from a real-world bird’s-eye perspective in an industry environment which contains, conditioned by its special geometry, a minimal amount of visual cues and, hence, requires incorporation of the temporal domain for structure from motion estimation. To enable the system to incorporate structure from motion from pixel translation when facing context-sparse, i.e., visual cue sparse, scenery, we propose a novel architecture built upon the structure from motion learner, which uses temporal pairs of jointly unrotated and stacked images for depth prediction. In order to increase the overall performance and to avoid blurred depth edges that lie in between the edges of the two input images, we integrate a geometric consistency loss into our pipeline. We assess the model’s ability to learn structure from motion by introducing a novel industry dataset whose perspective, orthogonal to the floor, contains only minimal visual cues. Through the evaluation with ground truth depth, we show that our proposed method outperforms the state of the art in difficult context-sparse environments.

Industry 4.0 is moving forward under technology upgrades, utilizing information technology to improve the intelligence of the industry, whereas Industry 5.0 is value-driven, aiming to focus on essential societal needs, values, and responsibility. The manufacturing industry is currently moving towards the integration of productivity enhancements and sustainable human employment. Such a transformation has deeply changed the human–machine interaction (HMI), among which digital twin (DT) and extended reality (XR) are two cutting-edge technologies. A manufacturing DT offers an opportunity to simulate, monitor, and optimize the machine. In the meantime, XR empowers HMI in the industrial field. This paper presents an XR application framework for DT-based services within a manufacturing context. This work aims to develop a technological framework to improve the efficiency of the XR application development and the usability of the XR-based HMI systems. We first introduce four layers of the framework, including the perception layer with the physical machine and its ROS-based simulation model, the machine communication layer, the network layer containing three kinds of communication middleware, and the Unity-based service layer creating XR-based digital applications. Subsequently, we conduct the responsiveness test for the framework and describe several XR industrial applications for a DT-based smart crane. Finally, we highlight the research challenges and potential issues that should be further addressed by analyzing the performance of the whole framework.

Industry 4.0 and Cyber-physical systems require easy access to shop-floor data, which allows the monitoring and optimization of the manufacturing process. To achieve this, several papers have proposed various ways to make OPC UA (Open Platform Communications Unified Architecture), a standard protocol for industrial communication, RESTful (Representational State Transfer). As an alternative to REST, GraphQL has recently gained popularity amongst web developers. This paper compares the characteristics of the REST and GraphQL interfaces for OPC UA and conducts measurements on reading and writing data. The measurements show that GraphQL offers better performance than REST when multiple values are read or written, whereas REST is faster with single values. However, using OPC UA directly outperforms both REST and GraphQL interfaces. As a conclusion, this paper recommends using a GraphQL interface alongside an OPC UA server in smart factories to simultaneously yield easy data access, the best performance, and maximum interoperability.

Digital twin documents are expected to form a global network of digital twins, a “Digital Twin Web”, that allows the discovery and linking of digital twins with an approach similar to the World Wide Web. Digital twin documents can be used to describe various aspects of machines and their twins, such as physical properties, nameplate information, and communication interfaces. Digital twin is also one of the core concepts of the fourth industrial revolution, aiming to make factories more efficient through optimized control methods and seamless information flow, rendering them “smart factories”. In this paper, we investigate how to utilize digital twin documents in smart factory communication. We implemented a proof-of-concept simulation model of a smart factory that allowed simulating three different control methods: centralized client-server, decentralized client-server, and decentralized peer-to-peer. Digital twin documents were used to store the necessary information for these control methods. We used Twinbase, an open-source server software, to host the digital twin documents. Our analysis showed that decentralized peer-to-peer control was most suitable for a smart factory because it allowed implementing the most advanced cooperation between machines while still being scalable. The utilization of Twinbase allowed straightforward removal, addition, and modification of entities in the factory.

Digital twins are expected to form a network, a “Digital Twin Web,” in the future. Digital Twin Web follows a similar structure to the World Wide Web and consists of meta-level digital twins that are described as digital twin description documents and distributed via Digital Twin Web servers. Standards must be established before the Digital Twin Web can be used efficiently, and having an easily accessible server implementation can foster the development of those standards. Twinbase is an open-source, Git-based Digital Twin Web server developed with user-friendliness in mind. Twinbase stores digital twin documents in a Git repository, modifies them with Git workflows, and distributes them to users via a static web server, from which the documents can be accessed via a client library or a regular web browser. A demo server is available at https://dtw.twinbase.org and new server instances can be initialized free-of-charge at GitHub via its browser interface. Twinbase is built with GitHub repository, Pages, and Actions but can be extended to support other providers or self-hosting. We describe the underlying architecture of Twinbase to support the creation of derivative and alternative server implementations. The Digital Twin Web requires permanent, globally accessible, and transferable identifiers to function properly, and to address this issue, we introduce the concept of digital twin identifier registry. Performance measurements showed that the median response times for fetching a digital twin document from Twinbase varied between 0.4 and 1.2 seconds depending on the identifier registry.

Digital twin technology empowers the digital transformation of the industrial world with an increasing amount of data, which meanwhile creates a challenging context for designing a human–machine interface (HMI) for operating machines. This work aims at creating an HMI for digital twin based services. With an industrial crane platform as a case study, we presented a mixed reality (MR) application running on a Microsoft HoloLens 1 device. The application, consisting of visualization, interaction, communication, and registration modules, allowed crane operators to both monitor the crane status and control its movement through interactive holograms and bi-directional data communication, with enhanced mobility thanks to spatial registration and tracking of the MR environment. The prototype was quantitatively evaluated regarding the control accuracy in 20 measurements following a step-by-step protocol that we defined to standardize the measurement procedure. The results suggested that the differences between the target and actual positions were within the 10 cm range in three dimensions, which were considered sufficiently small regarding the typical crane operation use case of logistics purposes and could be improved with the adoption of robust registration and tracking techniques in our future work.

Industrial Internet of Things practitioners are adopting the concept of digital twins at an accelerating pace. The features of digital twins range from simulation and analysis to real-time sensor data and system integration. Implementation examples of modeling-oriented twins are becoming commonplace in academic literature, but information management-focused twins that combine multiple systems are scarce. This study presents, analyzes, and draws recommendations from building a multi-component digital twin as an industry-university collaboration project and related smaller works. The objective of the studied project was to create a prototype implementation of an industrial digital twin for an overhead crane called “Ilmatar”, serving machine designers and maintainers in their daily tasks. Additionally, related cases focus on enhancing operation. This paper describes two tools, three frameworks, and eight proof-of-concept prototypes related to digital twin development. The experiences show that good-quality Application Programming Interfaces (APIs) are significant enablers for the development of digital twins. Hence, we recommend that traditional industrial companies start building their API portfolios. The experiences in digital twin application development led to the discovery of a novel API-based business network framework that helps organize digital twin data supply chains.

A digital twin offers various features such as visualizations, simulations with real data, and performance monitoring. Several of these features or parts of them can be implemented employing existing systems storing product data. However, to create a digital twin, these data being scattered to different systems is needed to be merged. For merging data and straightforward linking of these systems, this paper proposes a Data Link. The Data Link offers a single interface to access the systems via an API (Application Programming Interface) gateway and makes all data of the physical product available and accessible. In addition, it stores metadata about the systems and offers a user interface that allows searching the system that contains the required piece of information or implements the needed feature. The Data Link was implemented for an industrial overhead crane bringing operational data, control, and CAD (computer-aided design) models behind a single interface. The example implementation indicated that the Data Link based architecture for digital twins allows easy implementation of digital twins by using existing systems.

Data collection in an industrial environment enables several benefits: processes and machinery can be monitored; the performance can be optimized; and the machinery can be proactively maintained. To collect data from machines or production lines, numerous sensors are required, which necessitates a management system. The management of constrained IoT devices such as sensor nodes is extensively studied. However, the previous studies focused only on the remote software updating or configuration of sensor nodes. This paper presents a holistic Open Sensor Manager (OSEMA), which addresses also generating software for different sensor models based on the configuration. In addition, it offers a user-friendly web interface, as well as a REST API (Representational State Transfer Application Programming Interface) for the management. The manager is built with the Django web framework, and sensor nodes rely on ESP32-based microcontrollers. OSEMA enables secure remote software updates of sensor nodes via encryption and hash-based message authentication code. The collected data can be transmitted using the Hypertext Transfer Protocol (HTTP) and Message Queuing Telemetry Transport (MQTT). The use of OSEMA is demonstrated in an industrial domain with applications estimating the usage roughness of an overhead crane and tracking its location. OSEMA enables retrofitting different sensors to existing machinery and processes, allowing additional data collection.