Seminar Details
2026-03-03 (14:00) : Consensus is a myth: Human label variation in Natural Language Inference
At Euler building (room A.002)
Organized by Mathematical Engineering
Speaker :
Marie-Catherine de Marneffe (CENTAL, UCLouvain)
Abstract :
Recently, NLP researchers have increasingly begun to acknowledge that humans often diverge in their interpretations of various NLP tasks, and that such variation should be captured if robust language understanding is to be achieved. In this talk, I will focus on analyzing human label variation in the Natural Language Inference (NLI) task, in which, given a premise, one identifies whether a hypothesis sentence is true, false, or undetermined. For instance, if one says: “My friend often travels with a heavy suitcase”, can it be inferred that “My friend often travels with a light suitcase”? I will examine the various sources of NLI label variation and investigate whether or not they can be captured by current LLMs, arguing that, in the presence of variation, labels without explanations are not sufficiently meaningful.