Invited Talks at the 2014 RefNet Summer School (24-30 Aug 2014)

Invited Talks of the Summer School will be presented in the evenings, see the programme of courses and evening lectures.

Susan Brennan, Stony Brook University.

Let the language games begin: Embodied, embedded referring

Referring is where linguistics, psychology, philosophy, ethnography, and computer science meet, in order to understand and model how agents use language and nonverbal cues to communicate about entities of interest. Referring expressions emerge and are used in embodied, embedded contexts that we can think of as language games (extending David Lewis, 1979). Given the diversity of methods and approaches taken in the study of referring, the language game is often unmentioned, implicit, and rather invisible. Using data from some of our old and new projects, I'll argue that it's critical to acknowledge and bring out into the open the language games where referring takes place (in the world, the mind, the medium, the relationship, and the lab).

Susan Brennan is Professor of Psychology, Linguistics, and Computer Science at Stony Brook University (NY, USA). She received a bachelor's degree from Cornell in anthropology, a master's degree from MIT's Media Lab, and a Ph.D. in psychology from Stanford. She has conducted research at Atari, Apple, and HP Labs on topics including computer-generated caricature, mediated communication, natural language interfaces, human-computer interaction, and speech recognition interfaces; since defecting to academia, her research has been continuously supported by the National Science Foundation since 1992. She currently serves as Stony Brook's Graduate Director of Psychology and as Associate Editor for Cognitive Science, and previously served as consulting editor for Discourse Processes, Psychological Science, and Computational Linguistics. In addition to working with graduate students on the psycholinguistics of spoken dialogue and adaptive processing in communication, Brennan teaches cognitive psychology, psycholinguistics, and human factors to undergraduates.

Michael Frank, Stanford University

Predicting pragmatic reasoning

A short, ambiguous message can convey a lot of information to a listener who is willing to make inferences based on assumptions about the speaker and the context of the message. Pragmatic inferences are critical in facilitating efficient human communication, and have been characterized informally using tools like Grice's conversational maxims. They may also be extremely useful for language learning. In this talk, I'll propose a probabilistic framework for referential communication in context. This framework shows good fit to adults' and children's judgments. In addition, it makes interesting novel predictions about both language acquisition and processing, some of which we have already begun to test.

Michael C. Frank is Associate Professor of Psychology at Stanford University. He earned his BS from Stanford University in Symbolic Systems in 2005 and his PhD from MIT in Brain and Cognitive Sciences in 2010. He studies children's language learning and how it interacts with their developing understanding of the social world, using behavioral experiments, computational tools, and novel measurement methods including large-scale web-based studies, eye-tracking, and head-mounted cameras.

Eva Belke, Bochum University

Visual and linguistics representations and processes in generating and comprehending referring expressions

Producing and understanding references are key activities in language processing, yet there are remarkably few psycholinguistic models dedicated to this activity. While there are detailed models of language production, virtually none incorporates a viable interface to visual encoding processes. In particular, it is unclear to what extent visual processing impacts on the conceptual representation and the subsequent linguistic encoding of referents in a visual scene when producing a referring expression. With respect to understanding referring expressions, conceptual short term memory (CSTM) appears to be a viable means of modelling the interface of visual and linguistic processing allowing for the recognition of the intended referent in a visual scene. I will argue that in reference production, too, CSTM is vital in that it imprints on the early conceptual processing of a scene. However, in order to sustain the linguistic encoding processes involved in reference production, longer-lasting and less ad-hoc representations must be involved that allow for the unique identification of the target referent and the ensuing verbal encoding processes. I will discuss these and other differences between the comprehension and production of references and present a working model of reference comprehension and production that might function as a framework for both psycholinguistic models and computational linguistic algorithms of reference processing. Based on this working model, I will outline avenues for future research.

Eva Belke studied clinical linguistics, psychology and maths at Bielefeld University (1994-1998). She obtained a PhD in psycholinguistics in 2001 (Bielefeld University) and went on to do a postdoc at the University of Birmingham. In the academic year 2004/05, she held a lectureship at Aston University, Birmingham. In 2005, she returned to Bielefeld University, where she continued working as a postdoc and completed her habilitation in 2008. Since 2008, she is a professor of experimental psycholinguistics at Bochum University. Her main research interests concern language production processes. Much of her recent work has focused on lexical-semantic encoding in lexical access. In a number of eye tracking studies, she has (co-)investigated the links between visual and linguistic encoding processes in both visual and linguistic tasks. Beyond that, she is interested in bilingual language processing and in the role of executive functions in language processing in healthy and impaired speakers.

Emiel Krahmer, Tilburg University

Widening the gap between psycholinguistics and computational linguistics: the case of referring expressions

The production of referring expressions has been studied extensively, both within psycholinguistics and computational linguistics. In recent years, various researchers have tried to bridge the gap between these two disciplines, and I will argue that this has substantially improved our understanding of reference production. However, sometimes it seems important to look in detail at what is happening at the respective sides of the gap, and inform those at the other side. In this talk, I will first present recent developments in natural language generation (the subfield of computational linguistics aimed at the automatic generation of text), and ask what insights into the production of referring expressions are particularly needed here. Then, I will discuss the kind of computational psycholinguistic models that are being developed on the other shore, and ask how these can be extended to become models of human reference production.

Emiel Krahmer (MA, Computational Linguistics, 1991; PhD., Linguistics, 1995) is a full professor of Language, Cognition and Computation at Tilburg University, and one of the research program leaders in the Tilburg center for Cognition and Communication (TiCC). Before joining Tilburg University in 2001, he worked at Eindhoven University of Technology (1995-2001), as a senior researcher in the Institute of Perception Research (IPO). In his work, he studies how people communicate with each other, both verbally and non-verbally, with the aim of subsequently improving the way computers communicate with human users. To achieve this, he combines computational modelling and experimental studies with human participants. He has co-authored over 250 peer-reviewed publications, successfully co-supervised 12 PhD students, and currently supervises 10 more. He was PI on various externally funded research projects, including an NWO Vici project which ran from 2008 to 2013: “Bridging the gap between psycholinguistics and computational linguistics: The case of Referring Expressions” (

Alexander Koller, University of Potsdam

Reference is interactive

The traditional view of reference in natural language generation is that an (artificial) speaker is supposed to produce an unambiguous referring expression. The hearer then gets one chance to identify the object to which the expression was intended to refer. If the hearer misidentifies the object, typical evaluation measures will count this as evidence against the quality of the generation system.

The reality of human-human dialogue looks very different. Human speakers produce ambiguous referring expressions all the time. Hearers resolve many of these referring expressions effortlessly, and when they do run into trouble, they will ask clarification questions, or the speaker will identify the problem and provide further information. In other words, among humans referring to objects is an interactive process in which both parties can participate actively.

In my talk I will present some of my recent research on the interactive generation of referring expressions. I will discuss the use of virtual 3D environments as a framework for interactive natural language generation; the use of AI planning techniques for interactively changing the communicative context; and various ways for detecting misunderstandings, and how to respond to them, in terms of eyetracking and probabilistic modeling.

Alexander Koller is the Professor of Theoretical Computational Linguistics at the University of Potsdam, Germany. He obtained his PhD from Saarland University in 2004, and was a post-doc at Columbia University and the University of Edinburgh. In 2008 he returned to Saarland University as a junior research group leader, before moving to Potsdam in 2011. He was the main organizer of the GIVE Challenge, an international shared task for interactive natural language generation systems. Alexander's main research interests are currently computational semantics, natural language generation, and parsing.

Ellen Bard and Kees van Deemter, 14 Aug. 2014.