Taught Courses on the 2014 RefNet Summer School (24-30 Aug 2014)

List of Taught Courses. The descriptions below are only approximate. For a complete schedule including the times of all lectures and Practical/Lab sessions, see the programme of courses and evening lectures.

1. Multimodal reference. (Piwek/Gatt/Van der Sluis)

This course focuses on the interplay of language and gesture in reference. It is divided into four parts: (1) we explore issues in the study of gesture in communication from a psycholinguistic and computational perspective; (2) subsequently, we will focus specifically on multimodal corpora, giving an overview of previous work before homing in on our own work on designing and collecting the MREDI (Multimodal Reference in DIalogue) corpus and using it in machine-learning experiments to predict referential actions; (3) we will then focus on annotating multimodal corpora, with particular reference to the design of a reliable annotation scheme based on one's research questions. This third session will be practically oriented. Participants will be provided with some data samples, and be asked (in small groups) to think about interesting questions that could be addressed with such data, and what sort of annotation scheme would help to address them. Finally, session (4) will wrap up: participants will be given time to design a short presentation on the work carried out during the previous session and give the presentation to other members of the group.

(1) Introduction to studying gesture and reference: psycholinguistics of speech and gesture, approaches to computational modelling.
(2) Collecting multimodal corpora: Overview of previous work on multimodal corpora (e.g. Beun and Cremers; work carried out in Bielefeld). The MREDI dialogue corpus, methods and annotation. Results of using machine-learning algorithms on the MREDI corpus. Preparation for the practical session (3).
(3) A practical session on annotation: how to design annotation schemes, as a function of the research questions being addressed.
(4) Wrap-up session with participant presentations based on the practical session. Open discussion.

Material relating to the course can be found here.

2. The development of reference in early childhood (Matthews)

Over the first 5 years of life, children become remarkably effective communicators. By the end of the first year, infants have learned to use gesture and vocalisation in order to direct the attention of those around them. They then quickly begin to use conventional symbols (words, whole utterances) to achieve the same task. Over the preschool years, they carve out a functional space for each of these units in their language such that, for example, they know when an adjective or prepositional phrase might be valuable. Finally, around 5 years of age, children start to demonstrate meta-linguistic insight into the fact that linguistic conventions are normative agreements between speakers. Over the course of three sessions, I will provide an overview of naturalistic and experimental studies that explore the processes that drive these early developments. The first session will focus on preverbal communication (pointing, babbling), the second on the production of referring expressions and the third on methods for researching child language, including how to search the databases of naturalistically occurring child speech hosted on CHILDES.

These sessions will provide an overview of how the ability to refer to things develops over the first 5 years of life. It will include a review of early gestural and vocal reference and later production of pronouns and complex referring expressions. A session on methodology will provide a practical insight into the tools available for conducting research in this domain.

(1) The development of reference in infancy. (Lecture session with interactive bits)
(2) The development of reference in the preschool years (Lecture session with interactive bits)
(3) Methods for studying the development of reference.

Recommended reading:
Stephens, G., & Matthews, D. (2014). The communicative infant from 0-18 months: The social-cognitive foundations of pragmatic development. In D. Matthews (Ed.), Pragmatic Development in First Language Acquisition (pp. 13-35). Amsterdam: John Benjamins
Graf, E., & Davies, C. (2014). The production and comprehension of referring expressions. In D. Matthews (Ed.), Pragmatic Development in First Language Acquisition (pp. 161-181). Amsterdam:John Benjamins.

3. Eye-tracking methods. (Hermens/Hill/Keller)

This course offers a brief introduction to an increasingly important set of techniques in psychological research that has long been used in language comprehension and starting to be used in language production as well.

(1) (Hermens) Eye movements in general and a bit about analyzing them (regions of interest, heatmaps)
(2) (Hermens) Computer lab session, where students work on analyzing data (regions of interest, heatmaps) in pairs or groups of three people (would require sufficient computers with Matlab installed)
(3) (Hermens) Introduction to eye movements in reading (fixations, saccades, typical durations, moving mask, moving window paradigm)
(4) (Keller + Hill) More detailed lecture on eye movements in psycholinguistics and beyond (different reading measures, Experiment Builder, Data Viewer, Cross Recurrence, Joint Eye tracking)
(5) (Keller + Hill) Groups visiting the lab (those who are interested), those left behind could continue to work with Matlab.

Course notes and recommended reading relating to Frouke Hermens' lectures can be found: here. Other relevant papers include the following:

Dale, R., Warlaumont, A. S., & Richardson, D. C. (2011). Nominal cross recurrence as a generalized lag sequential analysis for behavioral streams. International Journal of Bifurcation and Chaos, 21, 1153-1161.
Simon P. Liversedge and John M. Findlay. 2000. Saccadic eye movements and cognition, Trends in Cognitive Science, Volume 4, Issue 1, p6 14.
Keith Rayner and Monica Castelhano (2007). Eye Movements. Scholarpedia, 2(10):3649.

4. Statistics: mixed models. (Corley, Finlayson, Barr)

This course will motivate why psycholinguistic research is increasingly making use of mixed effects models, explain how they work, illustrate them on concrete data, and discuss how they can be applied in cases of unbalanced designs and non-normal data. The course will focus on issues that are most relevant for the study of language production.

Session 1:

First hour [basics], taught by Corley:
* basic regression, understanding coefficients (what is standard error?), etc.
* multiple regression
* (brief intro to graphics and model criticism; plotting residuals)
* non-continuous predictors and effects coding
* interaction terms [optional?]

Second hour [generalized linear models / non-normality], taught by Finlayson:
* production, corpus data, and non-normality
* linking functions (logit for binary, poisson for count)
* drastic non-normality [optional?]

Session 2:

Third hour [dependent data], taught by Barr:
* repeated measures / mixed effects models
* how to match the error structure to your study design

Fourth hour: address remaining issues that arose during hrs 1-3. (all)
If none, then:
* imbalance, collinearity, other likely problems
* convergence problems
* (if time) how to design to avoid some of these more thorny issues

Session 3 (2 hours), taught by Corley, Finlayson, Barr:

Students bring their own data and questions
If no data, work through example dataset

Recommended reading:
Parts 1-4 of Dan Navarro's "Learning Statistics with R", which is freely available here . There is a lot of material there, so they may wish to read as needed.

5. Algorithms for the generation of referring expressions. (van Deemter/Gatt/Kutlak)

This course will offer a brief overview of the field of Referring Expressions Generation (REG). After a summary of classic algorithms, we discuss a recent strand of work that views REG algorithms as tentative models of human language production, akin to psycholinguistic models. We conclude with a discussion of computational models (both symbolic and statistical) of complex human knowledge and their use in REG.

(1) Early REG algorithms
-- The Californian School (Winograd, Appelt, Kronfeld)
-- Dale & Reiter's new assumptions & formulation of the task
-- The classic algorithms (FB, GR, IA)
-- Evaluation: early experiments and the TUNA Challenges

(2) REG as a cognitive model
-- Preference beats Discrimination
-- The effect of domain size
-- Nondeterminism

(3) Modelling the role of knowledge in reference production
-- The role of shared knowledge in existing algorithms
-- Modern Knowledge Representation techniques (especially Description Logic)
-- Ontologies, Rules, and Quantifiers in REG
-- Reference under uncertainty over the hearer's knowledge (Kutlak)

Recommended reading:
Van Deemter, Gatt, van Gompel and Krahmer (2012) Towards a computational psycholinguistics of reference production. Topics in Cognitive Science 4(2), pp.166-183. Published version. Pre-final version.
Krahmer and van Deemter (2012) Computational Generation of Referring Expressions: A Survey. Downloadable from here.

6. Linguistic and philosophical aspects of reference (Cann)

Reference is an important topic in philosophy and linguistics, and this series of three two-hour lectures looks at various aspects of reference and referring expressions from the perspective of these two disciplines.

(1) The first lecture looks at different types of referring expression, pronouns, definite noun phrases, proper names, etc. and discusses some of the issues these raise in the philosophy of language.
(2) The second lecture looks at quantificational and discourse properties of definite and indefinite noun phrases.
(3) The final lecture looks at pronominal anaphora, accessibility of referents and salience in discourse.

For more information see this page .
Recommended reading: four entries from the Stanford Encyclopedia of Philosophy:

reference .

Ellen Bard and Kees van Deemter, 21 Aug. 2014.