Abstract
Article Jiafa Mao 1,*, and Lu Zhang 2 1 College of Computer Science and Technology, ZheJiang University of Technology, ZheJiang, Hang Zhou 310023, China 2 China Telecom Hangzhou, Hangzhou 311121, China * Correspondence: maojiafa@zjut.edu.cn Received: 23 December 2023 Accepted: 28 June 2024 Published: 25 March 2025 Abstract: The existing object tracking, localization, measurement, and other technologies mostly concentrate on dual cameras or using single camera plus the non-visual sensor technology. These technologies are achieved by increasing the amount of data at the expense of lowing the processing speed to achieve precise localization of machine vision. If machine vision localization can be achieved without increasing the amount of data processing, then only the monocular ranging method can be used. Therefore, monocular ranging is obviously more challenging in actual research. Motivated by this, this paper proposes a novel object learned method based on monocular vision. According to the geometric model of camera imaging and the basic principle of converting analog signals to digital signals, we derive the relationship model between the object distance, object height, camera height, image resolution, image target size, and camera parameters. We theoretically prove the infinite solvability of “self-invariance” and the solvability of “self-change”, which provides a theoretical basis for the object tracking, localization and measurement based on monocular vision. The experimental results show the correctness of our theory. Mobile robots have been widely used in various fields such as surveillance, visual navigation, automatic image interpretation, human-computer interaction, and virtual reality [1−2]. The machine vision technique tends to simulate visual functions of human eyes for extracting information from video sequences and recognizing the morphology and motion of three-dimensional scenes and targets [3−5]. Machine vision plays a vital role for the performance of mobile robots [6−8]. Like human beings relying on vision to obtain most of the information, mobile robot vision systems use cameras to capture and process the information. The machine vision technology is therefore crucial for the robot to properly process its received information. In machine vision, 3D information acquisition remains a challenging problem in the field of autonomous navigation [9], 3D scene reconstruction [10], visual measurements [11, 12] and industrial automation [13]. Existing methods of machine vision can be roughly divided into two categories. The methods in the first category try to acquire the target depth information, including binocular perception methods [14], monocular camera calibration methods [15, 16], target depth information estimation methods [17, 18] and single camera plus plane mirror-based methods [19, 20]. While the methods of the second category are based on the sensor positioning technology. The binocular perception method simulates human stereo vision, which obtains the target depth information from the difference of viewing angles between the direct view of the left and right eye [13]. Yang et al. [14] used a binocular camera stereo matching method for gesture recognition. In this method, parallax is obtained from the relative information between two cameras, which is then used to calculate the depth information of the target. However, the accuracy of this method depends on the performance of the camera, such as illumination and baseline length (i.e., the distance between two cameras). Further, this method has a limited capability for real applications due to high complexity and a relatively large data processing load. Camera calibration methods can be divided into the traditional approach [21, 22] and self-calibration approach [15, 16]. Abdal-Aziz and Karara [23] proposed a camera calibration method based on direct linear transformation. Tsai et al. [24] devised a radial uniform constraint calibration method. Zhang et al. [25] introduced a camera calibration method for active vision. These traditional methods could achieve high accuracy of calibration. However, these methods require specific calibration references. In the calibration process, due to the limitations of equipment, it is still impossible to accurately record the corresponding coordinates of target in the world coordinate and image coordinate system. Further, the accuracy of coordinate conversion fluctuates accordingly. The deep learning-based methods [17, 18] employ neural networks and large-scale data learning to obtain the depth information of targets. Liu et al. [26] designed a Full Convolutional Network (FCN) to perform a single-eye estimation scene depth map. Eigen et al. [27] developed deep neural networks to estimate depth cues in RGB images. He et al. [28] conducted deep learning on a large-scale fixed focal length data set to synthesize a variable focal length data set, thus generating deep cavity in the image. Zhang et al. [29] proposed a deep guidance and regularization (HGR) learning framework for end-to-end monocular depth estimation. Although these methods can be used to estimate target depth, it is difficult to obtain accurate depth of the target, which limits its application to mobile robots. In recent years, monocular depth estimation has been widely applied in the field of simultaneous localization and mapping (SLAM). Loo et al. [30] proposed CNN-SVO where the three-dimensional point information obtained from monocular depth estimation was used to improve SVO. Tateno et al. [31] proposed real-time monocular dense SLAM. With the help of a depth estimator, this method is robust and accurate. David et al. [32] devised a multi-scale architecture to simultaneously predict depth, surface normal and semantic labels. This method can capture many image details without relying on super pixel segmentation. Liu et al. [26] proposed a deep convolutional neural field model for depth estimation. In this method, unary and binary potentials of continuous CRF are learned in a unified deep network. The model is based on a fully convolutional network with a super pixel pooling scheme. Similar methods were also designed in [33, 34], which combine the CNNs and CRF technology. In these methods, two-layer CRF are used to enhance the collaborative operation of global and local prediction values to obtain the final depth prediction results. Cao et al. [35] converted the depth estimation problem to a pixel-level classification problem. The continuous ground truth depth value is first discretized into multiple depth intervals, then a label is assigned to each interval according to the depth value range, and the depth estimation problem is transformed into a sub-classification problem. Finally, the depth is obtained using a fully convolutional depth residual network. Zheng et al. [36] proposed a layout-conscious convolutional neural network (LA-Net) for accurate monocular depth estimation, in which a multi-scale layout map is employed as a structural guide to generate a consistent layout depth map. The monocular camera plus flat mirror schemes [19, 20] are methods designed for observing the fish target in a water tank. These methods could be used to effectively solve 3D occlusion tracking of the fish in a water tank. However, they are not suitable for mobile robots or moving targets when realizing 3D tracking of the target. The non-visual sensor [37, 38] uses information including sound waves, infrared, pressure, and electromagnetic induction to sense the distance of an external target to the robot, thus obtaining the depth information. To work out the 3D coordinates of the target, this approach requires visual imaging and sensors, and this increases the data processing burden of the robot and requires synchronization between the vision and the sensor. As a result, such an approach is generally not efficient in mobile robots. In this paper, we propose a target’s height and distance measurement method. In the proposed method, we adopt an operation of “self-variant” to achieve the measurement of target information in monocular vision, which requires no additional supplement devices. Our proposed method can accurately obtain the target height and distance information of the target. Such a method can overcome the shortcomings of traditional target depth measurement methods, and help robots to obtain target information measurements and track in a moving environment. This paper analyzes the insolvability of “self-invariance” and the solvability of “self-change” through theoretical and experimental analysis. The main contributions of this work are as follows: 1) A target’s height and distance measurement method is proposed of “self-change” and a relationship model of parameters is established including the target distance, target height, camera height, image size, target image size and camera parameters, etc. 2) The experiment proves that under the conditions of unknow target height and “self-invariant”, the target distance cannot be calculated even there are multiple views available. 3) It is theoretically proved that under the condition of “self-variant”, the target distance can be calculated using two different views. The above contributions provide a theoretical basis for researchers engaged in target measurements in the field of computer vision, and make it possible for scholar to use a single image for target depth estimation. At the same time, the target depth can only be calculated when the height or distance of the two images is changed. The rest of the paper is organized as follows. Section 2 introduces the basic concept of “self-change”. Section 3 provides the relationship model between the distance and height of the target when it is in the visual center. The infinite solution analysis of “self-invariance” is given in Section 4. Section 5 presents the solvability analysis of “self-change”, In Section 6, experimental verification is carried out while Section 7 concludes the work. For monocular vision, the target distance should be properly addressed to achieve target tracking and location. Based on our previous research [39], it is easy to achieve target tracking and positioning when the target height is known beforehand. For real applications, however, it is difficult to obtain the height of the target. In this case, certain machine learning technique is required to estimate the target’s height. The performance of monocular vision is mainly affected by following parameters: 1) the focal length of the camera, which is denoted as \( f \) (mm), 2) the size of the photosensitive film of CMOS, which is denoted as \( w\times l\left({\mathrm{m}\mathrm{m}}^{2}\right) \) , 3) the height of the camera, recorded as \( {\mathrm{h}}_{\mathrm{r}} \) , 4) the target distance, denoted as \( \mathrm{m} \) , and 5) the height of target AB, denoted as \( {\mathrm{h}}_{\mathrm{o}} \) . We call the first two parameters as internal parameters while the other three as external parameters. Definition 1: The term of “self-change” refers to the change of camera’s height \( {\mathrm{h}}_{\mathrm{r}} \) or the target distance m. For example, the robot may change the height of the camera \( {\mathrm{h}}_{\mathrm{r}} \) during the process of stretching or bending its legs. The target’s distance \( \mathrm{m} \) . can be changed while it moves forward or backward. Here, we call \( {\mathrm{h}}_{\mathrm{r}} \) and \( \mathrm{m} \) as variable external parameters. The proposition opposite to “self-change” is “self-invariance”. It refers to the cases that the camera’s height \( {\mathrm{h}}_{\mathrm{r}} \) and the target’s distance \( \mathrm{m} \) keep unchanged. It should be noted that the target height \( {\mathrm{h}}_{\mathrm{o}} \) cannot be changed, so we call it an invariable external parameter. To carry out target’s height and distance measurements through "self-change", we first should understand typical relationship models of the target height and distance. To solve the problem of target depth measurement when the target is in the visual center, we first need to define the term of “target is located in the visual center”. Definition 2: From the point of view of the 3D imaging process, the so-called “target is located at the center of vision” is that the line passing through the highest and lowest vertexes of the target can intersect the main optical axis of the camera in space. In the two-dimensional plane of target image, it means the line or extension line between the highest and lowest points of the target image passing through the center point of the visual image. According to the position of the main optical axis of the camera, we divide the relationship model of the target distance and target height into the following categories: 1) the model of imaging geometry when the target vertex falls on the main optical axis of the camera; 2) the model of imaging geometry when the target vertex is higher than the main optical axis of the camera; 3) the model of imaging geometry when the target vertex is lower than the main optical axis of the camera; and 4) the relationship model of the target distance and target height of the angular bisecting image. Before introducing these models, we first explain the concept of the depression angle. Definition 3: The so-called depression angle is the angle between the main optical axis and the horizontal line when the camera is shooting downwards, marked as \( \alpha \) . As shown in Figure 1, if we assume that the horizontal line of the camera is FE and the main optical axis is \( F{B}{{'}} \) , then the depression angle \( \alpha \) is \( \angle EF{B}{{'}} \) . To measure the horizontal distance between target AB and the camera, we first set the coordinates of the center point of the image plane to be \( {\mathrm{C}}{{{'}}}\left(w/2,l/2\right) \) . The size of the digitized image is \( M\times N\left({\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}}^{2}\right) \) , and the digital coordinates of image CD of target AB after image segmentation are \( C\left({X}_{C},{Y}_{C}\right) \) and \( \mathrm{D}\left({X}_{D},{Y}_{D}\right) \) . Then, we can calculate the length \( d \) of the target image CD, the distance \( {d}_{1} \) from point D to the center point of the image plane \( {\mathrm{C}}' \) , and the distance \( {d}_{2} \) from point C to the center point \( {\mathrm{C}}' \) of the image plane as: In the following sections, we should introduce the relationship models. Figure 1. Two-dimensional geometric principle of camera imaging. When the highest point of the target just falls on the main optical axis of camera, it means that the target vertex is at the center of vision. To measure the horizontal distance \( \mathrm{m} \) between target AB and the camera, we use the bottom position of the camera as the center of the coordinate system, the horizontal line of the machine vision direction as X axis, and the line perpendicular to the OX axis on the ground plane as Y axis. Also, we set the camera position as point F, then the coordinate system shown in Figure 2 is established with the side of OF as Z axis. Suppose that the image plane is \( {\text{π}}_{1} \) , the ground plane is \( {{\text{π}}}_{2} \) , and the projected image plane \( {\text{π}}_{1} \) is CD after the target AB is imaged by the camera. Obviously, points A, B, C, D, O and F are all on the plane where XOZ is located. Based on this, we can easily derive the relationship between the length \( d \) , target distance \( m \) , target height \( {h}_{o} \) , and camera height \( {h}_{r} \) : From the basic principle of camera imaging, we know that since the vertex B of target |AB| falls on the main optical axis, the image C of point B must be the center of the image plane \( {\mathrm{\pi }}_{1} \) . Then, we can calculate the target distance \( m \) as: Figure 2. Schematic diagram of imaging with the target vertex falling on the main optical axis When the target vertex is higher than the main optical axis of the camera, then in the image plane \( {\text{π}}_{1} \) , the vertex C of the target image CD is higher than the center point of the image plane as shown in Figure 3. Suppose that the main optical axis of the camera and target AB intersect at point \( {\mathrm{B}}' \) , and set \( \left|\mathrm{A}{\mathrm{B}}'\right|=\mathrm{x} \) , then \( \left|{\mathrm{B}}'\mathrm{B}\right|={\mathrm{h}}_{\mathrm{o}}-\mathrm{x} \) . Consequently, we can easily obtain the following relationship of these parameters: Finally, the relationship between the target height \( {h}_{o} \) and the target distance \( m \) can be written as: Figure 3. Schematic diagram of imaging when the target vertex is higher than the main optical axis of the camera. When the target vertex is lower than the main optical axis of the camera, then in the image plane \( {{\text{π}}}_{1} \) , the vertex C of the target image CD is lower than the center point of the image plane as shown in Figure 4. Suppose that the main optical axis of the camera and the extended line of target AB intersects at point \( {B}' \) , and set \( \left|\mathrm{A}{B}'\right|=\mathrm{x} \) , then \( \left|{B}'\mathrm{B}\right|={\mathrm{x}-\mathrm{h}}_{\mathrm{o}} \) . Since the coordinates of CD in the image plane \( {\text{π}}_{1} \) are \( \mathrm{C}\left({\mathrm{X}}_{\mathrm{C}},{\mathrm{Y}}_{\mathrm{C}}\right) \) and \( \mathrm{D}\left({\mathrm{X}}_{\mathrm{D}},{\mathrm{Y}}_{\mathrm{D}}\right) \) , and \( {C}' \) is the center point of the image plane, the simulation coordinates of \( {C}' \) are \( {C}'\left(w/2,l/2\right) \) . We can express the target distance as: Figure 4. Schematic diagram of imaging when the target vertex is lower than the main optical axis of the camera. Definition 4:In a certain image acquisition process, if the main optical axis of the camera bisects the angle between two endpoints of the target and the camera. This kind of image is called the angle bisection image (ABI). As shown in Figure 1, the main optical axis bisects \( \angle AFB \) , so we can easily derive the relationship between the depression angle \( \alpha \) and the CD length \( d \) : In Eq. (7), \( f \) is the focal length of the camera, \( \mathrm{tan}\left(\cdot \right) \) is the tangent function, \( {\alpha }_{1}={\mathrm{tan}}^{-1}\left({h}_{r}/m\right) \) , \( {\alpha }_{2}={\mathrm{tan}}^{-1} \left(\left({h}_{r}-{h}_{o}\right)/m\right) \) , and \( {\mathrm{tan}}^{-1}\left(\cdot \right) \) is the arctangent function. Fig. 5 is a function diagram of the relationship between the depression angle and the target image length obtained with the target AB height 680 (mm), distance (mm), focal length of 50 (mm), and camera height of (mm). It can be seen from Figure 5 that when the depression angle \( \alpha \) gradually increases from zero, the CD length d gradually decreases until it reaches a certain value. After that, it gradually increases. In other words, \( d \) has a minimum value. Taking the derivative of Eq. (7), we can easily know that when \( \alpha =\left({\alpha }_{1}+{\alpha }_{2}\right)/2 \) , \( d \) reaches the minimum value. This is due to: Therefore, when the main optical axis is the bisector of \( \angle AFB \) (the green part in Figure 1), \( d \) reaches the minimum value. According to the angle bisector theorem in geometry, we can get the following equation: Based on Eq. (9), we can obtain the relationship between the...

This publication has 40 references indexed in Scilit: