Zoltan Nagy

The present paper proposes an implementation of a hybrid hardware–software system for the visual servoing of prosthetic arms. We focus on the most critical vision analysis part of the system. The prosthetic system comprises a glass-worn eye tracker and a video camera, and the task is to recognize the object to grasp. The lightweight architecture for gaze-driven object recognition has to be implemented as a wearable device with low power consumption (less than 5.6 W). The algorithmic chain comprises gaze fixations estimation and filtering, generation of candidates, and recognition, with two backbone convolutional neural networks (CNN). The time-consuming parts of the system, such as SIFT (Scale Invariant Feature Transform) detector and the backbone CNN feature extractor, are implemented in FPGA, and a new reduction layer is introduced in the object-recognition CNN to reduce the computational burden. The proposed implementation is compatible with the real-time control of the prosthetic arm.

Summary: In this paper we describe an FPGA implementation of the Scale Invariant Feature Transform (SIFT) algorithm. The FPGA is required as its a lightweight device which makes it ideal for vision‐guided hybrid neuro‐prostheses utilised for upper limbs replacement. SIFT point detection is needed for computation of coordinates of the object‐to‐grasp in a wearable multi‐camera system. A modified SIFT algorithm is proposed and an implementation of it into C/C++ language on Xilinx ZCU102 FPGA board. The proposed hybrid hardware/software solution is compared to other hardware or hybrid implementations of the SIFT algorithm and with the baseline software detector OpenSIFT. The algorithm optimised for FPGA gives an average precision of 0.84 and the average recall of 0.94 in SIFT‐point detection compared to the baseline.The proposed solution has lower dissipated power than other solutions like CPU or GPU, and it has better computational speed.This solution allows for processing medium‐sized images in real‐time with low power consumption.

Spiking activity of individual neurons can be separated from the acquired multi-unit activity with spike sorting methods. Processing the recorded high-dimensional neural data can take a large amount of time when performed on general-purpose computers. Methods: In this paper, an FPGA- based real-time spike sorting system is presented which takes into account the spatial correlation between the electrical signals recorded with closely-packed recording sites to cluster multi- channel neural data. The system uses a spatial window-based version of the Online Sorting algorithm, which uses unsupervised template-matching for clustering. Results: The test results show that the proposed system can reach an average of 86% using simulated data (16-32 neurons, 4-10 dB Signal-to-Noise Ratio), while the single-channel clustering version achieves only 74% average accuracy in the same cases on a 128-channel electrode array. The developed system was also tested on in vivo cortical recordings obtained from an anesthetized rat. Conclusion: The proposed FPGA-based spike sorting system can process more than 11000 spikes/second, so it can be used during in vivo experiments providing real-time feedback on the location and electrophysiological properties of well-separable single units. Significance: The proposed spike sorting system could be used to reduce the positioning error of the closely-packed recording site during a neural measurement.

A platform design for the analysis of human myoelectric signals (MES) is presented. Offline recorded multichannel signals of forearm muscles are processed with a Field Programmable SoC in order to classify different movement patterns to control human-assisting electromechanical systems with multiple degrees of freedom (e.g. a prosthetic hand). Benchmark results of an ANSI C implementation are shown to assess the raw performance of the built-in ARM cores of the SoC. Possible computational bottlenecks are located based on the results and custom hardware implementations are shown to fully utilize the flexibility and performance of the used hardware platform.

Numerical simulation of complex computational fluid dynamics problems evolving in time plays an important role in scientific and engineering applications. Accurate behavior of dynamical systems can be understood using large scale simulations which traditionally requires expensive super-computing facilities. In the paper a Field Programmable Gate Array (FPGA) based framework is described to accelerate simulation of complex physical spatio-temporal phenomena. Simulating complicated geometries requires unstructured spatial discretization which results in irregular memory access patterns severely limiting computing performance. Data locality is improved by mesh node renumbering technique which results in a sequential memory access pattern. Additionally storing a small window of cell-centered state values in the on-chip memory of the FPGA can increase data reuse and decrease memory bandwidth requirements. Generation of the floating-point data path and control structure of the arithmetic unit containing dozens of operators is a very challenging task when the goal is high operating frequency. Efficiency and use of the framework is described by a case study solving the Euler equations on an unstructured mesh using finite volume technique. On the currently available largest FPGA the generated architecture contains three processing elements working in parallel providing 75 times speedup compared to a high performance microprocessor.

Many real-life applications of processor-arrays suffer from memory bandwidth limitations. In many cases an unstructured mesh is given (computation on sensor data, simulations of physical systems - PDEs), where the vertices represent computations with dependencies represented by the edges. Utilization of processing elements (PEs) during these computations is mainly depends on the node indexing of the mesh. If the adjacent nodes are stored close to each other in main memory, the reloading of node data can be significantly decreased. In case of FPGA the memory accesses can be fully determined by the designer. The mesh and an ordering of its nodes, define the graph bandwidth, which determines the minimum size of on-chip memory to avoid reloading of the nodes from the off-chip memory. If the required on-chip memory size is higher than the available resources, the mesh must be divided into parts. In this paper a novel geometry-based method is presented, which constructs reordered parts from a given unstructured mesh, where each part meets some predefined constraints on graph bandwidth.

Array computers can be useful in the solution of numerical spatiotemporal problems such as partial differential equations (PDEs). IBM has recently introduced the cell broadband engine (Cell BE) Architecture, which contains 8 identical vector processors in an array structure. In the paper the implementation of the 3-D Princeton Ocean Model on the Cell BE is discussed. The area/speed/power tradeoffs of our solution and different hardware implementations are also compared.