Adam Pekarski

This article presented ongoing research efforts at the Institute for Advanced Mining Technologies (AMT) at the RWTH Aachen University, Aachen/Germany, with the goal of enabling the safe integration of autonomous machinery in surface mining operations. It has been highlighted that there is a need for clear standardization of the integration of autonomous machines in mining operations. To this end a novel and structured safety concept for mixed traffic operations is currently being developed. Part of this safety concept is an integrated safety control system. This safety control system heavily relies on perception sensors to maintain a safe state of the Autonomous or Semi-Autonomous Machine (ASAM). Currently there exists no holistic understanding of the impact of the harsh environmental conditions in mining environments on the detection capability of perception systems as well as a standardized testing method to evaluate and validate them under these conditions. For this purpose, a static long-term test stand was built at the AMTs exploratory sandbox, the Nivelstein quartz sand open-pit mine of Nivelsteiner Sandwerke und Sandsteinbrüche GmbH, where perception sensors are examined year-round. This allows for evaluating sensor performance in a wide variety of environmental influences. Overall, these research topics will play an integral role in the safe and economic integration of autonomous machinery in modern mining operations.

The mining industry is in need of automation due to increasing requirements like higher global demands for resources and deposits, which are deeper and more complex. Progressing underground mines lead to longer travel times to the mining face and thus a loss in productive working time, which has to be compensated by automation. Ultimately, stricter health and safety regulations and a decreasing number of skilled operators accelerate the need for automation further. Within the the drill-and-blast cycle in underground mining, the drilling of blast holes is a central step. While semi-automated and supporting systems exist that allow the automated execution of single process steps under supervision, to date, no system is available for the unsupervised blast hole drilling of a mine face. A precondition for unsupervised operation is a perception system, which allows independent decision-making of the machine. To address this gap, this work presents a novel vision system capable of segmenting a mine face into drillable and non-drillable areas, which can serve as the basis for the autonomous adaption of a drilling pattern. An area of the mine face is considered drillable if no leftover blast holes from the previous blast cycle are present and the surface angle is below a certain threshold. The system presented is based on a stereo camera setup mounted on a drill jumbo. The resulting 2D and 3D data are processed by software that employs AI-based computer vision techniques, as well as traditional algorithms. The system was validated, and the performance was verified in the K+S Zielitz mine. Experts assisted in the determination of operational parameters and empirically validated the system’s performance. Additionally, the blast hole detection algorithm underwent a data-based analytical verification.

To increase the safety and efficiency of underground mining processes, it is important to advance automation. An important part of that is to achieve autonomous material loading using load–haul–dump (LHD) machines. To be able to autonomously load material from a muck pile, it is crucial to first detect and characterize it in terms of spatial configuration and geometry. Currently, the technologies available on the market that do not require an operator at the stope are only applicable in specific mine layouts or use 2D camera images of the surroundings that can be observed from a control room for teleoperation. However, due to missing depth information, estimating distances is difficult. This work presents a novel approach to muck pile detection developed as part of the EU-funded Next Generation Carbon Neutral Pilots for Smart Intelligent Mining Systems (NEXGEN SIMS) project. It uses a stereo camera mounted on an LHD to gather three-dimensional data of the surroundings. By applying a topological algorithm, a muck pile can be located and its overall shape determined. This system can detect and segment muck piles while driving towards them at full speed. The detected position and shape of the muck pile can then be used to determine an optimal attack point for the machine. This sensor solution was then integrated into a complete system for autonomous loading with an LHD. In two different underground mines, it was tested and demonstrated that the machines were able to reliably load material without human intervention.