Hoeseok Yang

Deep learning has been applied to object detection in remotely sensed images. Typically, remote sensing object detection is performed on the ground rather than on-board due to the limited computational resources available on embedded systems. This offloading introduces delays in acquiring mission-critical information, restricting its applicability to real-time scenarios. To enable on-device object detection, research has focused on designing efficient detectors and applying model compression techniques to reduce inference latency. Nonetheless, there is an absence of relevant research that maintains the accuracy of remote sensing detectors. Currently, the majority of remote sensing detectors operate as two-stage detectors. Therefore, there is a need for efficient research into two-stage detectors. In this paper, we propose a model simplification method for two-stage object detectors. Instead of utilizing a conventional feature pyramid network (FPN), our approach employs a single feature extraction process within the two-stage detector. However, relying solely on a single feature degrades detection accuracy. To compensate for the accuracy drop, we modify the method of selecting positive anchors used in training. We apply a method to determine the threshold for positive anchors based on the intersection over union (IoU), with the k anchors closest to the object. To further enhance accuracy, we apply a high-pass filter to increase the objectness score of hard-to-detect objects on the region proposal network (RPN) score map. The proposed method is compatible with any two-stage detector that integrates an FPN. Experiments conducted on state-of-the-art two-stage detectors, such as Oriented-RCNN and LSKNet, demonstrate that our method reduced computation costs up to 58.1% while achieving a 1.6%p increase in average accuracy on the DOTAv2.0 dataset. With the fewer RoIs, our approach in the Oriented-RCNN achieves a 31.8% improvement in throughput while maintaining the same average accuracy.

This paper presents a novel real-time scheduling technique tailored for sporadic task graphs in multi-processor systems, motivated by the computational demands of intelligent transportation systems (ITS). Unlike traditional deadline-based approaches, the proposed method prioritizes tasks based on their anticipated contribution to system utility, expressed as a value, under runtime uncertainty. Each request is associated with a reputation score, with progressively refined value estimations revealed through sequential task executions. A feasibility-aware value density metric is introduced to dynamically schedule tasks while respecting global deadlines and the non-preemptive nature of execution. Extensive simulations, conducted with 1, 2, and 4 processors, evaluate performance under varying traffic conditions and workload intensities. The results confirm that the proposed method significantly improves the total earned value, particularly under high-load scenarios or limited processing resources.

Deep learning has been successfully applied to object detection from remotely sensed images. Images are typically processed on the ground rather than on-board due to the computation power of the ground system. Such offloaded processing causes delays in acquiring target mission information, which hinders its application to real-time use cases. For on-device object detection, researches have been conducted on designing efficient detectors or model compression to reduce inference latency. However, highly accurate two-stage detectors still need further exploitation for acceleration. In this paper, we propose a model simplification method for two-stage object detectors. Instead of constructing a general feature pyramid, we utilize only one feature extraction in the two-stage detector. To compensate for the accuracy drop, we apply a high pass filter to the RPN's score map. Our approach is applicable to any two-stage detector using a feature pyramid network. In the experiments with state-of-the-art two-stage detectors such as ReDet, Oriented-RCNN, and LSKNet, our method reduced computation costs upto 61.2% with the accuracy loss within 2.1% on the DOTAv1.5 dataset. Source code will be released.

Convolutional Neural Networks (CNNs) are widely used in various systems, including resource-constrained embedded systems or IoT devices. In such systems, it is typical to deploy compressed or pruned CNNs, instead of original ones, at the cost of reduced accuracy. Existing CNN pruning techniques have primarily focused on minimizing resource requirements. However, today’s embedded systems are increasingly dynamic in both resource demands and availability. Thus, the previous techniques that only consider given static cases are no longer efficient. In this paper, we propose a novel multi-phase CNN that enables a multi-objective exploration of a number of pruning candidates out of a single CNN. In the proposed technique, a CNN can operate in various versions depending on which subsets of weights are used and can be transformed to the one best matches to the given constraint adaptively and efficiently. For that, a CNN is first pruned to the sparsest form; then a set of parameters (sub-network) is additionally supplemented as the phase goes by. As a result, a number of network versions for all different phases can be represented by a single network and they form a pareto solution over the accuracy and resource usage trade-off. In this work, we target CPU-based CNN inference engines as most embedded systems do not have the luxury of specialized co-processor support such as GPUs or HW accelerators. The proposed technique has been implemented in a publicly available CPU inference engine, Darknet, and its effectiveness has been validated with a popular CNN in terms of design space exploration capability and runtime switchability.

Deep Neural Networks (DNNs) used for image classification are vulnerable to adversarial examples, which are images that are intentionally generated to predict an incorrect output for a deep learning model. Various defense methods have been proposed to defend against such adversarial attacks, among which, image-reconstruction-based defense methods, such as DIPDefend, are known to be effective in getting rid of the adversarial perturbations injected in the image. However, this image-reconstruction-based defense approach suffers from a long execution time due to its iterative and time-consuming image reconstruction. The trade-off between the execution time and the robustness/accuracy of the defense method should be carefully explored, which is the main focus of this paper. In this work, we aim to improve the execution time of the existing state-of-the-art image-reconstruction-based defense method, DIPDefend, against the Fast Gradient Sign Method (FGSM). In doing so, we propose to take the input-specific properties into consideration when deciding the stopping point of the image reconstruction of DIPDefend. For that, we first applied a low-pass filter to the input image with various kernel sizes to make a prediction of the true label. Then, based on that, the parameters of the image reconstruction procedure were adaptively chosen. Experiments with 500 randomly chosen ImageNet validation set images show that we can obtain an approximately 40% improvement in execution time while keeping the accuracy drop as small as 0.4–3.9%.

In the design of real-time systems, it is becoming increasingly important to take soft-error tolerance into account. While hardening techniques such as error detection and error correction enable us to build systems that can better tolerate soft-errors, they are inevitably accompanied with execution time overhead. In order to mitigate the impact of the increased execution time, Chen et al. proposed to identify the (m, k)-constraints of real-time tasks, which demand that at least m jobs out of k consecutive task invocations must be fault-free, and to design hardening policies based on this constraint. In this paper, we propose a new method to design hardening policies that are adaptive and stochastic. At the heart of our method is a new linear program (LP) formulation that finds an adaptive stochastic policy optimizing the CPU utilization. At the design time, we first identify the task set information of a given system, verify the system schedulability, and solve LPs to find an optimal policy. This policy is represented as a look-up table that specifies the stochastic hardening decisions as a function of the past execution history of the system. At the run time, hardening decisions are made simply by looking up this table. The proposed method finds hardening policies that adaptively reacts to the execution history of the system, allowing improvement in the CPU utilization. The method also deviates from the previous approaches’ viewpoint that reliability must be assessed in an all-or-nothing manner, by devising the notion of stochastic hardening policies. We evaluated the effectiveness of the proposed method using various task sets. In a set of 2,050 benchmarks, the system’s CPU utilization was improved by 2.80%-7.16% on average under different configurations. The improvement was by as high as 18.45% in the best benchmark.