Simon Tjoa

With the advent of Artificial Intelligence (AI) technology and immense transitions to industry, it is important to regulate and establish standards for this fascinating technology. As a result, AI-powered applications need to be checked for compliance with AI regulations. Although compliance checks have been extensively discussed by the research community and industry, AI compliance checks and verification still need to be explored, and pitfalls need to be identified. This paper aims at automating the extraction of AI regulations using the knowledge graph and the ontology defined for legal documents. The paper first reports the initial effort to create knowledge graphs for legal documents (e.g., the EU AI Act) using different language models trained in general textual documents where general approaches in extracting Named-Entity Recognition (NER) are employed. Triplets in the form of are extracted followed by enhancement achieved through augmenting NER tags by information captured using dependency parsers. The paper then presents a methodology called Co-Compliance Officer (CO2) where Large Language Models (LLMs) are employed to build knowledge graphs and NER which are solely for parsing and extracting legal documents. The LLM ontology is then used to create effective tags that are specifically designed to extract legal documents and the relationships between legal entities. The created knowledge graphs are then used to perform several verification analysis on legal documents, including inconsistency checks and violations. The results indicate that Large Language Models (LLMs) can be helpful in compliance checking of legal documents, in particular, the documents created for AI compliance checking.

The new availability of powerful Artificial Intelligence (AI) as an everyday copilot has instigated a new wave of attack techniques, especially in the area of Social Engineering (SE). The possibility of generating a multitude of different templates within seconds in order to carry out an SE-attack lowers the entry barrier for potential threat actors. Still, the question remains whether this can be done using openly available tools without specialized expert skill sets on the attacker side, and how these compare to each other. This paper conducts three experiments based on a blueprint from a real-world CFO fraud attack, which utilized two of the most used social engineering attacks, phishing and vishing, and investigates the success rate of these SE attacks based on utilizing different available LLMs. The third experiment centers around the training of an AI-powered chatbot to act as a social engineer and gather sensitive information from interacting users. As this work focuses on the offensive side of SE, all conducted experiments return promising results, proving not only the ability and effectiveness of AI technology to act unethically, but also the little to no implied restrictions. Based on a reflection on the findings and potential countermeasures available, this research provides a deeper understanding of the development and deployment of AI-enhanced SE attacks, further highlighting potential dangers, as well as mitigation methods against this “upgraded” type of threat.

The introduction of Artificial Intelligence (AI) as an everyday tool has instigated a new wave of attack techniques, especially in the Social Engineering (SE) department. The possibility to generate a multitude of different templates within seconds in order to carry out an SE-attack lowers the entry barrier for potential threat actors. This paper conducts three experiments, where it accounts two of the most used social engineering attacks, phishing and vishing, and utilizes them to investigate the success rate of the SE-attack process when assisted by various LLM-agents available to normal, non-expert users. The third one centers around the training of an AI-powered chatbot to act as a social engineer and gather sensitive information from the interacting user. As this work focuses on the offensive side of SE, all conducted experiments return promising results, proving not only the ability and effectiveness of AI-technology to act unethical, but also the little to no implied restrictions or opposed regulations. Gathering an overall opinion on these results, benefits and drawbacks to these findings are reflected upon and potential countermeasures are presented. This research aims to provide a deeper understanding behind the development and deployment of AI-enhanced SE-attacks by non-experts, further highlighting potential dangers.

Our wealth, economy, and society increasingly rely on the effective operation of ICT core services. This is emphasized by EU legislation, which mandates EU Member States to identify and safeguard their critical information infrastructures. In order to achieve this objective, it is essential for critical infrastructure providers to implement sufficient cybersecurity measures, which generally include a range of technical and organizational measures. Establishing a robust cyber defense is essential for safeguarding individual organizations. Management structures and response strategies are typically coordinated for this purpose, requiring participation in different cyber exercises. On a national level, it is very challenging to correctly evaluate the progress of a complete national Cyber Exercise Program (later CXP) including numerous different cyber exercises. This article focuses on various metrics that can be employed to evaluate the progress of national cyber exercises as a program, intending to provide a significant contribution. This article aims to address this challenge by introducing a 5-step CXP Evaluation Framework and a Goal-Question-Metric (GQM) methodology specifically designed for assessing national Cyber Exercise Programs. Alongside the established GQM approach, we present a use case to assess and enhance the efficacy of cyber exercises, ensuring that they fulfill their intended goals and strengthen the overall resilience of critical information infrastructures. Furthermore, we are providing an extensive analysis of the Data Collection and Analysis process, incorporating specific examples derived from the cyber-exercise life-cycle.