Olga Tushkanova

This paper aims to test the hypothesis that the quality of social media bot detection systems based on supervised machine learning may not be as accurate as researchers claim, given that bots have become increasingly sophisticated, making it difficult for human annotators to detect them better than random selection. As a result, obtaining a ground-truth dataset with human annotation is not possible, which leads to supervised machine-learning models inheriting annotation errors. To test this hypothesis, we conducted an experiment where humans were tasked with recognizing malicious bots on the VKontakte social network. We then compared the “human” answers with the “ground-truth” bot labels (‘a bot’/‘not a bot’). Based on the experiment, we evaluated the bot detection efficiency of annotators in three scenarios typical for cybersecurity but differing in their detection difficulty as follows: (1) detection among random accounts, (2) detection among accounts of a social network ‘community’, and (3) detection among verified accounts. The study showed that humans could only detect simple bots in all three scenarios but could not detect more sophisticated ones (p-value = 0.05). The study also evaluates the limits of hypothetical and existing bot detection systems that leverage non-expert-labelled datasets as follows: the balanced accuracy of such systems can drop to 0.5 and lower, depending on bot complexity and detection scenario. The paper also describes the experiment design, collected datasets, statistical evaluation, and machine learning accuracy measures applied to support the results. In the discussion, we raise the question of using human labelling in bot detection systems and its potential cybersecurity issues. We also provide open access to the datasets used, experiment results, and software code for evaluating statistical and machine learning accuracy metrics used in this paper on GitHub.

Cyberattacks on cyber-physical systems (CPS) can lead to severe consequences, and therefore it is extremely important to detect them at early stages. However, there are several challenges to be solved in this area; they include an ability of the security system to detect previously unknown attacks. This problem could be solved with the system behaviour analysis methods and unsupervised or semi-supervised machine learning techniques. The efficiency of the attack detection system strongly depends on the datasets used to train the machine learning models. As real-world data from CPS systems are mostly not available due to the security requirements of cyber-physical objects, there are several attempts to create such datasets; however, their completeness and validity are questionable. This paper reviews existing approaches to attack and anomaly detection in CPS, with a particular focus on datasets and evaluation metrics used to assess the efficiency of the proposed solutions. The analysis revealed that only two of the three selected datasets are suitable for solving intrusion detection tasks as soon as they are generated using real test beds; in addition, only one of the selected datasets contains both network and sensor data, making it preferable for intrusion detection. Moreover, there are different approaches to evaluate the efficiency of the machine learning techniques, that require more analysis and research. Thus, in future research, the authors aim to develop an approach to anomaly detection for CPS using the selected datasets and to conduct experiments to select the performance metrics.

Both parts of this paper discuss the basic aspects of semantic computing, semantic technologies, and semantic applications applied to NL-text big data proc

This paper discusses the basic aspects of the modern understanding of semantic computations, semantic technologies, and semantic applications in the field

The paper presents Semantic Concept Analysis (SCA) framework intended for automatic data-driven design of actionable ontology specifying mobile device user's personal interest's hierarchy together with dual structure reflecting the user's preferences over these interests. The framework integrates known technique for semi-automatic ontology design exploiting DBpedia and Wikipedia categories, on the one hand, and the data-driven Formal Concept Analysis (FCA), on the other one. The framework implements a kind of machine-learning approach integrating algebraic and statistical models of data and knowledge structured as s a pair of dual concept semi-lattices. The proposed technology implementing SCA framework basic ideas is validated experimentally through its software prototyping and subsequent computer experimentation using natural language text data sample.

The paper objectives are twofold: to discuss the essence and challenges of automatic ontology design as applied to the Big data semantic modeling and to present Semantic Concept Analysis (SCA), a framework specifically developed for automatic actionable ontology design in Big data scenario. This framework integrates the data-driven DBpedia-based technology for semi-automatic design of the ontology concept hierarchy and Formal Concept Analysis (FCA), which formal concept specialization structure is built as dual one with regard to the ontology concept hierarchy. The SCA model of big data is built iteratively through interleaving use of data-driven ontology generalization step and subsequent formal concept specialization step. In this procedure, each of the pair of steps controls the other one. Indeed, ontology generalization step determines the dual formal concepts of the next specialization level, whereas the extent cardinality of each generated formal concept is used as attribute of the stopping criterion for the iterative ontology generalization design process. The proposed SCA framework technology is validated experimentally through its software prototyping and subsequent computer experimentation.