Kai-Uwe Kühnberger
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Kai-Uwe Kühnberger has a total of 64 co-authors. Here are the top co-authors by number of shared publications:
- Lucas Bechberger
Osnabrück University
8 shared publications - Tarek Richard Besold
Sony AI, Barcelona, Spain
6 shared publications - Ulf Krumnack
University of Osnabrück
5 shared publications - Alexander Mehler
Goethe University Frankfurt
2 shared publications - Artur D'Avila Garcez
University of London
2 shared publications - Gordon Pipa
Institute of Cognitive Science, Osnabrueck University, 49074 Osnabrueck, Germany
2 shared publications
Latest Publications
- 20 September 2024
- Published by Springer Nature
- p. 269-279
Enhancing Small Language Models via ChatGPT and Dataset Augmentation
- 20 September 2024
- Published by Springer Nature
- p. 269-279
Enhancing Small Language Models via ChatGPT and Dataset Augmentation
- Open Access
- 9 September 2024
- Published by Springer Nature
- p. 61-75
Generating Qualitative Descriptions of Diagrams with a Transformer-Based Language Model
To address the task of diagram understanding we propose to distinguish between the perception of the geometric configuration of a diagram from the assignment of meaning to the geometric entities and their topological relationships. As a consequence, diagram parsing does not need to assume any particular a priori interpretations of diagrams and their constituents. Focussing on Euler diagrams, we tackle the first of these subtasks—that of identifying the geometric entities that constitute a diagram (i.e., circles, rectangles, lines, arrows, etc.) and their topological relations—as an image captioning task, using a Vision Transformer for image recognition combined with language model GPT-2 to generate qualitative spatial descriptions of Euler diagrams with an encoder-decoder model. Due to the lack of sufficient high-quality data to train the pre-trained language model for this task, we describe how we generated a synthetic dataset of Euler diagrams annotated with qualitative spatial representations based on the Region Connection Calculus (RCC8). Results showed over 95% accuracy of the transformer-based language model in the generation of meaning-carrying RCC8 specifications for given Euler diagrams.
- 9 September 2024
- Published by Springer Nature
- p. 61-75
Open AccessGenerating Qualitative Descriptions of Diagrams with a Transformer-Based Language Model
To address the task of diagram understanding we propose to distinguish between the perception of the geometric configuration of a diagram from the assignment of meaning to the geometric entities and their topological relationships. As a consequence, diagram parsing does not need to assume any particular a priori interpretations of diagrams and their constituents. Focussing on Euler diagrams, we tackle the first of these subtasks—that of identifying the geometric entities that constitute a diagram (i.e., circles, rectangles, lines, arrows, etc.) and their topological relations—as an image captioning task, using a Vision Transformer for image recognition combined with language model GPT-2 to generate qualitative spatial descriptions of Euler diagrams with an encoder-decoder model. Due to the lack of sufficient high-quality data to train the pre-trained language model for this task, we describe how we generated a synthetic dataset of Euler diagrams annotated with qualitative spatial representations based on the Region Connection Calculus (RCC8). Results showed over 95% accuracy of the transformer-based language model in the generation of meaning-carrying RCC8 specifications for given Euler diagrams.
- Open Access
Show Me How It's Done: The Role of Explanations in Fine-Tuning Language Models
Our research demonstrates the significant benefits of using fine-tuning with explanations to enhance the performance of language models. Unlike prompting, which maintains the model's parameters, fine-tuning allows the model to learn and update its parameters during a training phase. In this study, we applied fine-tuning to various sized language models using data that contained explanations of the output rather than merely presenting the answers. We found that even smaller language models with as few as 60 million parameters benefited substantially from this approach. Interestingly, our results indicated that the detailed explanations were more beneficial to smaller models than larger ones, with the latter gaining nearly the same advantage from any form of explanation, irrespective of its length. Additionally, we demonstrate that the inclusion of explanations enables the models to solve tasks that they were not able to solve without explanations. Lastly, we argue that despite the challenging nature of adding explanations, samples that contain explanations not only reduce the volume of data required for training but also promote a more effective generalization by the model. In essence, our findings suggest that fine-tuning with explanations significantly bolsters the performance of large language models.
Open AccessShow Me How It's Done: The Role of Explanations in Fine-Tuning Language Models
Our research demonstrates the significant benefits of using fine-tuning with explanations to enhance the performance of language models. Unlike prompting, which maintains the model's parameters, fine-tuning allows the model to learn and update its parameters during a training phase. In this study, we applied fine-tuning to various sized language models using data that contained explanations of the output rather than merely presenting the answers. We found that even smaller language models with as few as 60 million parameters benefited substantially from this approach. Interestingly, our results indicated that the detailed explanations were more beneficial to smaller models than larger ones, with the latter gaining nearly the same advantage from any form of explanation, irrespective of its length. Additionally, we demonstrate that the inclusion of explanations enables the models to solve tasks that they were not able to solve without explanations. Lastly, we argue that despite the challenging nature of adding explanations, samples that contain explanations not only reduce the volume of data required for training but also promote a more effective generalization by the model. In essence, our findings suggest that fine-tuning with explanations significantly bolsters the performance of large language models.
- Open Access
- 9 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (1) , 209-219
Development of Few-Shot Learning Capabilities in Artificial Neural Networks When Learning Through Self-Supervised Interaction
Most artificial neural networks used for object recognition are trained in a fully supervised setup. This is not only resource consuming as it requires large data sets of labeled examples but also quite different from how humans learn. We use a setup in which an artificial agent first learns in a simulated world through self-supervised, curiosity-driven exploration. Following this initial learning phase, the learned representations can be used to quickly associate semantic concepts such as different types of doors using one or more labeled examples. To do this, we use a method we call fast concept mapping which uses correlated firing patterns of neurons to define and detect semantic concepts. This association works instantaneously with very few labeled examples, similar to what we observe in humans in a phenomenon called fast mapping . Strikingly, we can already identify objects with as little as one labeled example which highlights the quality of the encoding learned self-supervised through interaction with the world. It therefore presents a feasible strategy for learning concepts without much supervision and shows that through pure interaction meaningful representations of an environment can be learned that work better for few-short learning than non-interactive methods.
- 9 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 46 (1) , 209-219
Open AccessDevelopment of Few-Shot Learning Capabilities in Artificial Neural Networks When Learning Through Self-Supervised Interaction
Most artificial neural networks used for object recognition are trained in a fully supervised setup. This is not only resource consuming as it requires large data sets of labeled examples but also quite different from how humans learn. We use a setup in which an artificial agent first learns in a simulated world through self-supervised, curiosity-driven exploration. Following this initial learning phase, the learned representations can be used to quickly associate semantic concepts such as different types of doors using one or more labeled examples. To do this, we use a method we call fast concept mapping which uses correlated firing patterns of neurons to define and detect semantic concepts. This association works instantaneously with very few labeled examples, similar to what we observe in humans in a phenomenon called fast mapping . Strikingly, we can already identify objects with as little as one labeled example which highlights the quality of the encoding learned self-supervised through interaction with the world. It therefore presents a feasible strategy for learning concepts without much supervision and shows that through pure interaction meaningful representations of an environment can be learned that work better for few-short learning than non-interactive methods.
- 25 June 2023
- Published by Springer Nature
- p. 665-679
Detecting Loose Wheel Bolts of a Vehicle Using Accelerometers in the Chassis
- 25 June 2023
- Published by Springer Nature
- p. 665-679
Detecting Loose Wheel Bolts of a Vehicle Using Accelerometers in the Chassis
- 25 September 2022
- Published by Springer Nature
- p. 86-106
Grounding Psychological Shape Space in Convolutional Neural Networks
- 25 September 2022
- Published by Springer Nature
- p. 86-106
Grounding Psychological Shape Space in Convolutional Neural Networks
- Open Access
- 3 August 2021
- Published by Springer Nature
- p. 11-36
Generalizing Psychological Similarity Spaces to Unseen Stimuli
The cognitive framework of conceptual spaces proposes to represent concepts as regions in psychological similarity spaces. These similarity spaces are typically obtained through multidimensional scaling (MDS), which converts human dissimilarity ratings for a fixed set of stimuli into a spatial representation. One can distinguish metric MDS (which assumes that the dissimilarity ratings are interval or ratio scaled) from nonmetric MDS (which only assumes an ordinal scale). In our first study, we show that despite its additional assumptions, metric MDS does not necessarily yield better solutions than nonmetric MDS. In this chapter, we furthermore propose to learn a mapping from raw stimuli into the similarity space using artificial neural networks (ANNs) in order to generalize the similarity space to unseen inputs. In our second study, we show that a linear regression from the activation vectors of a convolutional ANN to similarity spaces obtained by MDS can be successful and that the results are sensitive to the number of dimensions of the similarity space.
- 3 August 2021
- Published by Springer Nature
- p. 11-36
Open AccessGeneralizing Psychological Similarity Spaces to Unseen Stimuli
The cognitive framework of conceptual spaces proposes to represent concepts as regions in psychological similarity spaces. These similarity spaces are typically obtained through multidimensional scaling (MDS), which converts human dissimilarity ratings for a fixed set of stimuli into a spatial representation. One can distinguish metric MDS (which assumes that the dissimilarity ratings are interval or ratio scaled) from nonmetric MDS (which only assumes an ordinal scale). In our first study, we show that despite its additional assumptions, metric MDS does not necessarily yield better solutions than nonmetric MDS. In this chapter, we furthermore propose to learn a mapping from raw stimuli into the similarity space using artificial neural networks (ANNs) in order to generalize the similarity space to unseen inputs. In our second study, we show that a linear regression from the activation vectors of a convolutional ANN to similarity spaces obtained by MDS can be successful and that the results are sensitive to the number of dimensions of the similarity space.
- Open Access
- 1 February 2021
- Published by Elsevier in Neural Networks
- Vol. 134, 23-41
Learning sparse and meaningful representations through embodiment
- 1 February 2021
- Published by Elsevier in Neural Networks
- Vol. 134, 23-41
Open AccessLearning sparse and meaningful representations through embodiment
- 9 September 2019
- Published by Springer Nature
- p. 69-75
Combining Deep Learning and (Structural) Feature-Based Classification Methods for Copyright-Protected PDF Documents
- 9 September 2019
- Published by Springer Nature
- p. 69-75
Combining Deep Learning and (Structural) Feature-Based Classification Methods for Copyright-Protected PDF Documents
- Open Access
Formalized Conceptual Spaces with a Geometric Representation of Correlations
The highly influential framework of conceptual spaces provides a geometric way of representing knowledge. Instances are represented by points in a similarity space and concepts are represented by convex regions in this space. After pointing out a problem with the convexity requirement, we propose a formalization of conceptual spaces based on fuzzy star-shaped sets. Our formalization uses a parametric definition of concepts and extends the original framework by adding means to represent correlations between different domains in a geometric way. Moreover, we define various operations for our formalization, both for creating new concepts from old ones and for measuring relations between concepts. We present an illustrative toy-example and sketch a research project on concept formation that is based on both our formalization and its implementation.
Open AccessFormalized Conceptual Spaces with a Geometric Representation of Correlations
The highly influential framework of conceptual spaces provides a geometric way of representing knowledge. Instances are represented by points in a similarity space and concepts are represented by convex regions in this space. After pointing out a problem with the convexity requirement, we propose a formalization of conceptual spaces based on fuzzy star-shaped sets. Our formalization uses a parametric definition of concepts and extends the original framework by adding means to represent correlations between different domains in a geometric way. Moreover, we define various operations for our formalization, both for creating new concepts from old ones and for measuring relations between concepts. We present an illustrative toy-example and sketch a research project on concept formation that is based on both our formalization and its implementation.