Benjamin Roth

Professor of Digital Philology
Faculty of Computer Science and Faculty of Philological and Cultural Studies
Research Group Data Mining and Machine Learning
and Department of European and Comparative Literature and Language Studies
University of Vienna
Office: Kolingasse 14, Room 5.17
Virtual Coffee*: Every Tuesday, 14:00-15:00, via Zoom link
*I'll have a coffee and be online. You can bring any question, idea, topic you want to talk about. Just join with the link, no appointment necessary.
Office Hours: by appointment
Email: [firstname.lastname@univie.ac.at]

Benjamin Roth is a professor in the area of statistical NLP, LLMs, and explainability, leading the WWTF Vienna Research Group for Young Investigators "Knowledge-Infused Deep Learning for Natural Language Processing".
Prior to this, he was an interim professor at LMU Munich. He obtained his PhD from Saarland University and did a postdoc at UMass, Amherst. His research interests are the extraction of knowledge from text with statistical methods and the principled study of LLMs and AI agents.

News:

Current and former PhD students


Publications

PHL de Araujo, P Röttger, D Hovy, B Roth
Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
EMNLP 2025

L Schoenegger, L Thoma, T Blevins, B Roth
Influence-driven Curriculum Learning for Pre-training on Limited Data
EMNLP 2025 BabyLM Workshop

A Tampier, L Thoma, L Schoenegger, B Roth
RecombiText: Compositional Data Augmentation for Enhancing LLM Pre-Training Datasets in Low-Resource Scenarios
EMNLP 2025 BabyLM Workshop

P Hong, B Chen, S Peng, MC de Marneffe, B Roth
Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanations
Preprint, 2025

L Bergmann, R Beckenbach, L Zach, B Roth, US Tran
One and a Half Years of ChatGPT–An Umbrella Review of Large Language Model (LLM) Perception Across Time and Research Fields
Zeitschrift für Psychologie, 2025

Y Xia, A Sedova, PH Luz de Araujo, V Kougia, L Nußbaumer, B Roth
Exploring prompts to elicit memorization in masked language model-based named entity recognition
PLOS ONE, 2025

P Dolog, SD Rico Torres, Y Velaj, Y Sadikaj, A Stephan, B Roth, C Plant
The Impact of Graph Structure, Cluster Centroid and Text Review Embeddings on Recommendation Methods
ACM Transactions on Recommender Systems, 2025

Y Xia, K Zaporojets, B Roth
Learn to pick the winner: Black-box ensembling for textual and visual question answering
KONVENS 2025

T Blevins, S Schmalwieser, B Roth
Do language models accommodate their users? A study of linguistic convergence
Preprint, 2025

Y Xia, PHL de Araujo, K Zaporojets, B Roth
Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles
ACL 2025

A Stephan, D Zhu, M Aßenmacher, X Shen, B Roth
From calculation to adjudication: Examining LLM judges on mathematical reasoning tasks
ACL 2025 Workshop on Generation, Evaluation and Metrics (GEM²)

PH Luz de Araujo, B Roth
Helpful assistant or fruitful facilitator? Investigating how personas affect language model behavior
PLOS ONE, 2025

C Madge, M Camilleri, P Carretero Garcia, V Karan, J Shao, P Jayannavar, J Hough, B Roth, M Poesio
Mdc-r: The minecraft dialogue corpus with reference
Preprint, 2025

B Roth, PHL de Araujo, Y Xia, S Kaltenbrunner, C Korab
Specification Overfitting in Artificial Intelligence
Artificial Intelligence Review, 2025

A Sedova, R Litschko, D Frassinelli, B Roth, B Plank
To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity
EMNLP Findings 2024

PHL De Araujo, B Roth
Functionality learning through specification instructions
EMNLP Findings 2024

M Aßenmacher, A Stephan, L Weissweiler, E Çano, I Ziegler, M Härttrich, B Bischl, B Roth, C Heumann, H Schütze
Collaborative Development of Modular Open Source Educational Resources for Natural Language Processing
ACL 2024 Workshop on Teaching NLP

V Kougia, A Sedova, A Stephan, K Zaporojets, B Roth
Analysing zero-shot temporal relation extraction on clinical notes using temporal consistency
ACL 2024 Workshop on Biomedical NLP

P Dolog, Y Sadikaj, Y Velaj, A Stephan, B Roth, C Plant
The Impact of Cluster Centroid and Text Review Embeddings on Recommendation Methods
ACM Web Conference (WWW) 2024

L Zellinger, A Stephan, B Roth
Counterfactual Reasoning with Knowledge Graph Embeddings
EACL 2024

A Stephan, L Miklautz, K Sidak, JP Wahle, B Gipp, C Plant, B Roth
Text-Guided Image Clustering
EACL 2024

A Baumann, A Stephan, B Roth
Seeing through the mess: evolutionary dynamics of lexical polysemy
EMNLP 2023

A Sedova, B Roth
ULF: Unsupervised Labeling Function Correction using Cross-Validation for Weak Supervision
EMNLP 2023

PHL de Araujo, B Roth
Cross-functional Analysis of Generalisation in Behavioural Learning
TACL 2023

A Sedova, B Roth
ACTC: Active Threshold Calibration for Cold-Start Knowledge Graph Completion
ACL 2023

A Sedova, L Zellinger, B Roth
Learning with Noisy Labels by Adaptive Gradient-Based Outlier Removal
ECML PKDD 2023

V Kougia, S Fetzel, T Kirchmair, E Çano, S Moayed Baharlou, S Sharifzadeh, B Roth
MemeGraphs: Linking Memes to Knowledge Graphs
ICDAR 2023

L März, E Asgari, F Braune, F Zimmermann, B Roth
XPASC: Measuring Generalization in Weak Supervision by Explainability and Association
Preprint, 2022

A Stephan, V Kougia, B Roth
SepLL: Separating Latent Class Labels from Weak Supervision Noise
Findings of EMNLP 2022

PHL de Araujo, B Roth
Checking HateCheck: a cross-functional analysis of behaviour-aware learning for hate speech detection
ACL 2022 Workshop on Efficient Benchmarking in NLP (NLP Power!)

A Stephan, B Roth
WeaNF: Weak Supervision with Normalizing Flows
RepL4NLP 2022

B Roth, E Çano
Focused Contrastive Training for Test-based Constituency Analysis
NeurIPS 2021 Workshop on Self-Supervised Learning

L März, E Asgari, F Braune, F Zimmermann, B Roth
KnowMAN: Weakly Supervised Multinomial Adversarial Networks
EMNLP 2021

A Sedova, A Stephan, M Speranskaya, B Roth
Knodle: Modular Weakly Supervised Learning with PyTorch
RepL4NLP 2021

L März, S Schweter, N Poerner, B Roth, H Schütze
Data Centric Domain Adaptation for Historical Text with OCR Errors
ICDAR 2021

MA Hedderich, B Roth, K Kann, B Plank, A Ratner, D Klakow (editors)
Proceedings of the First Workshop on Weakly Supervised Learning (WeaSuL)
ICLR 2021 Workshop on Weakly Supervised Learning

B Roth, M Wiegand
Python for Linguists (book review)
Computational Linguistics 2021

M Speranskaya, M Schmitt, B Roth
Ranking vs. Classifying: Measuring Knowledge Base Completion Quality
AKBC 2020

J Jungmaier, N Kassner, B Roth
Dirichlet-Smoothed Word Embeddings for Low-Resource Settings
LREC 2020

E Asgari, F Braune, B Roth, C Ringlstetter, M Mofrad
UniSent: Universal Adaptable Sentiment Lexica for 1000+ Languages
LREC 2020

R Rojowiec, M Fink, B Roth
Intent Recognition in Doctor-Patient Interviews
LREC 2020

A Sydorova, N Poerner, B Roth
Interpretable Question Answering on Knowledge Bases and Text
ACL 2019

L März, D Trautmann, B Roth
Domain adaptation for part-of-speech tagging of noisy user-generated text
NAACL 2019

B Roth, C Conforti, N Poerner, S Karn and H Schütze.
Neural Architectures for Open-Type Relation Argument Extraction.
Natural Language Engineering, 2019 (preprint)

M Schmitt, S Steinheber, K Schreiber, B Roth.
Joint Aspect and Polarity Classification for Aspect-based Sentiment Analysis with End-to-End Neural Networks.
EMNLP 2018.

N Poerner, H Schütze, B Roth.
Evaluating neural network explanation methods using hybrid documents and morphological prediction.
ACL 2018.

P Gupta, B Roth, H Schütze.
Joint Bootstrapping Machines for High Confidence Relation Extraction.
NAACL 2018.

M Schulder, M Wiegand, J Ruppenhofer and B Roth.
Towards Bootstrapping a Polarity Shifter Lexicon using Linguistic Features.
IJCNLP 2017.

H Adel, B Roth and H Schütze.
Comparing Convolutional Neural Networks to Traditional Models for Slot Filling.
NAACL 2016.

P Verga, D Belanger, E Strubell, B Roth, and A McCallum.
Multilingual Relation Extraction using Compositional Universal Schema.
NAACL, 2016.

M Schuhmacher, B Roth, S Ponzetto and L Dietz.
Finding Relevant Relations in Relevant Documents.
ECIR 2016.

B Roth, N Monath, D Belanger, E Strubell, P Verga, A McCallum.
Building Knowledge Bases with Universal Schema: Cold Start and Slot-Filling Approaches.
NIST Text Analysis Conference 2015.

A Neelakantan, B Roth and A McCallum.
Compositional Vector Space Models for Knowledge Base Completion.
ACL 2015.

M Wiegand, B Roth and D Klakow.
Combining Pattern-based and Distributional Similarity for Graph-based Noun Categorization.
NLDB 2015.

B Roth, E Strubell, K Silverstein and A McCallum.
Minimally Supervised Event Argument Extraction using Universal Schema.
NIPS 2014 Workshop on Knowledge Extraction (AKBC).

A Neelakantan, B Roth and A McCallum.
Knowledge Base Completion using Compositional Vector Space Models.
NIPS 2014 Workshop on Knowledge Extraction (AKBC).

M Wiegand, B Roth and D Klakow.
Automatic Food Categorization from Large Unlabeled Corpora and Its Impact on Relation Extraction.
EACL 2014.

B Roth, T Barth, G Chrupala, M Gropp, D Klakow.
RelationFactory: A Fast, Modular and Effective System for Knowledge Base Population.
EACL 2014 (software demo).

J Illig, B Roth and D Klakow.
Unsupervised Parsing for Generating Surface-Based Relation Extraction Patterns.
EACL 2014.

B Roth, T Barth, M Wiegand, M Singh, D Klakow.
Effective Slot Filling Based on Shallow Distant Supervision Methods.
NIST Text Analysis Conference 2013.

B Roth, D Klakow.
Combining Generative and Discriminative Model Scores for Distant Supervision.
EMNLP 2013.

B Roth, D Klakow.
Feature-Based Models for Improving the Quality of Noisy Training Data for Relation Extraction.
CIKM 2013.

B Roth, T Barth, M Wiegand, D Klakow.
A Survey of Noise Reduction Methods for Distant Supervision.
CIKM 2013 Workshop on Knowledge Extraction (AKBC).

M Wiegand, B Roth, D Klakow.
Web-based Relation Extraction for the Food Domain.
NLDB 2012.

B Roth, G Chrupala, M Wiegand, M Singh, D Klakow.
Generalizing from Freebase and Patterns using Distant Supervision for Slot Filling.
NIST Text Analysis Conference 2012.

M Wiegand, B Roth, D Klakow.
Knowledge Acquisition with Natural Language Processing in the Food Domain: Potential and Challenges.
ECAI 2012 Workshop: Cooking with Computers.

M Wiegand, B Roth, D Klakow.
Data-driven Knowledge Extraction for the Food Domain.
KONVENS 2012.

M Wiegand, B Roth, E Lasarcyk, S Köser, D Klakow.
A Gold Standard for Relation Extraction in the Food Domain.
LREC 2012.

B Roth, A McCallum, M Dymetman and N Cancedda.
Machine Translation Using Overlapping Alignments and SampleRank.
AMTA 2010.

G Chrupala, G Dinu and B Roth.
Enriched Syntax-based Meaning Representation for Answer Extraction.
SIGIR 2010 Workshop: Query Representation and Understanding.

B Roth and D Klakow.
Cross-Language Retrieval Using Link-Based Language Models.
SIGIR 2010.

L Li, B Roth and C Sporleder.
Topic Models for Word Sense Disambiguation and Token-Based Idiom Detection.
ACL 2010.

M Wiegand, A Balahur, B Roth, D Klakow and A Montoyo.
A Survey on the Role of Negation in Sentiment Analysis.
2010 Workshop on Negation and Speculation in NLP.

B Roth and D Klakow.
Combining Wikipedia-Based Concept Models for Cross-Language Retrieval.
IRF Conference 2010.