Roy Meissner

In this article, we explore the transformative impact of advanced, parameter-rich Large Language Models (LLMs) on the production of instructional materials in higher education, with a focus on the automated generation of both formative and summative assessments for learners in the field of mathematics. We introduce a novel LLM-driven process and application, called ItemForge, tailored specifically for the automatic generation of e-assessment items in mathematics. The approach is thoroughly aligned with the levels and hierarchy of cognitive learning objectives as developed by Anderson and Krathwohl, and takes specific mathematical concepts from the considered courses into consideration. The quality of the generated free-text items, along with their corresponding answers (sample solutions), as well as their appropriateness to the designated cognitive level and subject matter, were evaluated in a small-scale study. In this study, three mathematical experts reviewed a total of 240 generated items, providing a comprehensive analysis of their effectiveness and relevance. Our findings demonstrate that the tool is proficient in producing high-quality items that align with the chosen concepts and targeted cognitive levels, indicating its potential suitability for educational purposes. However, it was observed that the provided answers (sample solutions) occasionally exhibited inaccuracies or were not entirely complete, signalling a necessity for additional refinement of the tool's processes.

Zusammenfassung: Knowledge Graphs (KG) provide us with a structured, flexible, transparent, cross-system, and collaborative way of organizing our knowledge and data across various domains in society and industrial as well as scientific disciplines. KGs surpass any other form of representation in terms of effectiveness. However, Knowledge Graph Engineering (KGE) requires in-depth experiences of graph structures, web technologies, existing models and vocabularies, rule sets, logic, as well as best practices. It also demands a significant amount of work.Considering the advancements in large language models (LLMs) and their interfaces and applications in recent years, we have conducted comprehensive experiments with ChatGPT to explore its potential in supporting KGE. In this paper, we present a selection of these experiments and their results to demonstrate how ChatGPT can assist us in the development and management of KGs.Zusammenfassung. Wissensgraphen (englisch Knowledge Graphs, KGs), bieten uns eine strukturierte, flexible, transparente, systemübergreifende und kollaborative Möglichkeit, unser Wissen und unsere Daten über verschiedene Bereiche der Gesellschaft und der industriellen sowie wissenschaftlichen Disziplinen hinweg zu organisieren. KGs übertreffen jede andere Form der Repräsentation in Bezug auf die Effektivität. Die Entwicklung von Wissensgraphen (englisch Knowledge Graph Engineering, KGE) erfordert jedoch fundierte Erfahrungen mit Graphstrukturen, Webtechnologien, bestehenden Modellen und Vokabularen, Regelwerken, Logik sowie Best Practices. Es erfordert auch einen erheblichen Arbeitsaufwand.In Anbetracht der Fortschritte bei großen Sprachmodellen (englisch Large Language Modells, LLMs) und ihren Schnittstellen und Anwendungen in den letzten Jahren haben wir umfassende Experimente mit ChatGPT durchgeführt, um sein Potenzial zur Unterstützung von KGE zu untersuchen. In diesem Artikel stellen wir eine Auswahl dieser Experimente und ihre Ergebnisse vor, um zu zeigen, wie ChatGPT uns bei der Entwicklung und Verwaltung von KGs unterstützen kann.

Information about the paper titled "FACILITATION OF COMPETENCE ACQUISITION THROUGH FORMATIVE E-ASSESSMENT" at IATED Digital Library

One approach to continuously achieve a certain data quality level is to use an integration pipeline that continuously checks and monitors the quality of a data set according to defined metrics. This approach is inspired by Continuous Integration pipelines, that have been introduced in the area of software development and DevOps to perform continuous source code checks. By investigating in possible tools to use and discussing the specific requirements for RDF data sets, an integration pipeline is derived that joins current approaches of the areas of software-development and semantic-web as well as reuses existing tools. As these tools have not been built explicitly for CI usage, we evaluate their usability and propose possible workarounds and improvements. Furthermore, a real-world usage scenario is discussed, outlining the benefit of the usage of such a pipeline.