Izaro Goienetxea

Purse seiners play a crucial role in tuna fishing, as approximately 69% of the world’s tropical tuna is caught using this gear. All tuna Regional Fisheries Management Organizations have established minimum standards for the use of electronic monitoring (EM) in fisheries in addition to traditional observers. These EM systems produce a massive amount of video data that human analysts must process. Integrating artificial intelligence (AI) into their workflow can decrease that workload and improve the accuracy of the reports. However, species identification still poses significant challenges for AI, as achieving balanced performance across all species requires appropriate training data. Here, we quantify the difficulty experts face to distinguish bigeye tuna (BET, Thunnus obesus) from yellowfin tuna (YFT, Thunnus albacares) using images captured by EM systems. We found inter-expert agreements of 42.9% ± 35.6% for BET and 57.1% ± 35.6% for YFT. We then present a multi-stage pipeline to estimate the species composition of the catches using a reliable ground-truth dataset based on identifications made by observers on board. Three segmentation approaches are compared: Mask R-CNN, a combination of DINOv2 with SAM2, and an integration of YOLOv9 with SAM2. We found that the latter performs the best, with a validation mean average precision of 0.66 ± 0.03 and a recall of 0.85 ± 0.03. Segmented individuals are tracked using ByteTrack. For classification, we evaluated a standard multiclass classification model and a hierarchical approach. Their performance depended on the fishing trip considered, suggesting that both approaches are promising, but additional testing is needed to determine which strategy generalizes better overall. All our models were cross-validated during training and tested on fishing operations with fully known catch composition. Combining YOLOv9-SAM2 with the hierarchical classification produced the best estimations, with 84.8% of the individuals being segmented and classified with a mean absolute error of 4.5%.

In this paper we report on the design of a pipeline involving Common Spatial Patterns (CSP), a signal processing approach commonly used in the field of electroencephalography (EEG), matrix representation of features and image classification to categorize videos taken by a humanoid robot. The ultimate goal is to endow the robot with action recognition capabilities for a more natural social interaction. Summarizing, we apply the CSP algorithm to a set of signals obtained for each video by extracting skeleton joints of the person performing the action. From the transformed signals a summary image is obtained for each video, and these images are then classified using two different approaches; global visual descriptors and convolutional neural networks. The presented approach has been tested on two data sets that represent two scenarios with common characteristics. The first one is a data set with 46 individuals performing 6 different actions. In order to create the group of signals of each video, OpenPose has been used to extract the skeleton joints of the person performing the actions. The second data set is an Argentinian Sign Language data set (LSA64) from which the signs performed using just the right hand have been used. In this case the joint signals have been obtained using MediaPipe. The results obtained with the presented method have been compared with a Long Short-Term Memory (LSTM) method, achieving promising results.

Video activity recognition, despite being an emerging task, has been the subject of important research due to the importance of its everyday applications. Video camera surveillance could benefit greatly from advances in this field. In the area of robotics, the tasks of autonomous navigation or social interaction could also take advantage of the knowledge extracted from live video recording. In this paper, a new approach for video action recognition is presented. The new technique consists of introducing a method, which is usually used in Brain Computer Interface (BCI) for electroencephalography (EEG) systems, and adapting it to this problem. After describing the technique, achieved results are shown and a comparison with another method is carried out to analyze the performance of our new approach.

Action recognition in robotics is a research field that has gained momentum in recent years. In this work, a video activity recognition method is presented, which has the ultimate goal of endowing a robot with action recognition capabilities for a more natural social interaction. The application of Common Spatial Patterns (CSP), a signal processing approach widely used in electroencephalography (EEG), is presented in a novel manner to be used in activity recognition in videos taken by a humanoid robot. A sequence of skeleton data is considered as a multidimensional signal and filtered according to the CSP algorithm. Then, characteristics extracted from these filtered data are used as features for a classifier. A database with 46 individuals performing six different actions has been created to test the proposed method. The CSP-based method along with a Linear Discriminant Analysis (LDA) classifier has been compared to a Long Short-Term Memory (LSTM) neural network, showing that the former obtains similar or better results than the latter, while being simpler.

This paper presents a music generation method which is an extension of a previously presented method that generates coherent melodies using a melodic coherence structure extracted from a template piece. This extension, which has been applied for generating bertso melodies, adds the generation of the rhythmic content of the melodies, for which a rhythmic coherence structure of the template piece is also created. To do so, a pattern discovery and ranking method is used to discover the rhythmically repeated segments that are interesting, and create a rhythmic coherence structure which can have several levels of nesting. Independent sampling processes have been developed for melodic and rhythmic content, using an adapted optimization method for sampling the rhythmic content of the new pieces. An evaluation process has been carried out to evaluate some of the generated pieces, considering on one hand how the listeners perceive them and on the other hand whether they share the features with bertso melodies. It has been concluded from this evaluation that the method is capable of generating good coherent bertso melodies.

Bach-en obretan oinarritutako bertso doinuen sorkuntza automatikoa produkzioaren informazio-fitxa