Sina Moayed Baharlou
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Sina Moayed Baharlou has a total of 10 co-authors. Here are the top co-authors by number of shared publications:
- Sahand Sharifzadeh
Faculty of Computer Science Ludwig Maximilians University of Munich Munich Germany
2 shared publications - Simon Fetzel
Faculty of Computer Science University of Vienna Vienna Austria
2 shared publications Benjamin Roth
Faculty of Computer Science University of Vienna Vienna Austria
1 shared publication- Benjamin Roth
University of Vienna
1 shared publication Erion Çano
Faculty of Computer Science University of Vienna Vienna Austria
1 shared publication- Erion Çano
Digital Philology Data Mining and Machine Learning University of Vienna Austria
1 shared publication
Latest Publications
- Open Access
MemeGraphs: Linking Memes to Knowledge Graphs
Memes are a popular form of communicating trends and ideas in social media and on the internet in general, combining the modalities of images and text. They can express humor and sarcasm but can also have offensive content. Analyzing and classifying memes automatically is challenging since their interpretation relies on the understanding of visual elements, language, and background knowledge. Thus, it is important to meaningfully represent these sources and the interaction between them in order to classify a meme as a whole. In this work, we propose to use scene graphs, that express images in terms of objects and their visual relations, and knowledge graphs as structured representations for meme classification with a Transformer-based architecture. We compare our approach with ImgBERT, a multimodal model that uses only learned (instead of structured) representations of the meme, and observe consistent improvements. We further provide a dataset with human graph annotations that we compare to automatically generated graphs and entity linking. Analysis shows that automatic methods link more entities than human annotators and that automatically generated graphs are better suited for hatefulness classification in memes.
Open AccessMemeGraphs: Linking Memes to Knowledge Graphs
Memes are a popular form of communicating trends and ideas in social media and on the internet in general, combining the modalities of images and text. They can express humor and sarcasm but can also have offensive content. Analyzing and classifying memes automatically is challenging since their interpretation relies on the understanding of visual elements, language, and background knowledge. Thus, it is important to meaningfully represent these sources and the interaction between them in order to classify a meme as a whole. In this work, we propose to use scene graphs, that express images in terms of objects and their visual relations, and knowledge graphs as structured representations for meme classification with a Transformer-based architecture. We compare our approach with ImgBERT, a multimodal model that uses only learned (instead of structured) representations of the meme, and observe consistent improvements. We further provide a dataset with human graph annotations that we compare to automatically generated graphs and entity linking. Analysis shows that automatic methods link more entities than human annotators and that automatically generated graphs are better suited for hatefulness classification in memes.