Multimodal Corpora:
Combining applied and basic research targets

J. Edlund. D. Heylen, P. Paggio

27 May 2014, Harpa Conference Centre, Reykjavik, Iceland

+++ DEADLINE EXTENDED: 21 Feb 2014 +++

Download the FINAL Call for papers.

Currently, the creation of a multimodal corpus involves the recording, annotation and analysis of several communication modalities such as speech, hand gesture, facial expression, body posture, etc. An increasing number of research areas have transgressed or are in the process of transgressing from focused single modality research to full-fledged multimodality research, and multimodal corpora are becoming a core research asset and an opportunity for interdisciplinary exchange of ideas, concepts and data.

We are pleased to announce that in 2014, the 10th Workshop on Multimodal Corpora will return home and again be collocated with LREC.


As always, we aim for a wide cross-section of the field, with contributions ranging from collection efforts, coding, validation and analysis methods, to tools and applications of multimodal corpora. Given that LREC this year emphasizes the use of corpora to solve language technology problems and develop useful applications and services, we would like this workshop also to focus on the usefulness of multimodal corpora to applied research as well as basic research. Many of the unimodal speech corpora collected over the past decades have served a double purpose: on the one hand, they have enlightened our view on the basic research question of how speech works and how it is used; on the other hand, they have forwarded the applied research goal of developing better speech technology applications. This reflects the dual nature of speech technology, where funding demands often require researchers to follow research agendas that target applied and basic research goals in parallel. Multimodal corpora are potentially more complex than unimodal corpora, and their design poses an even greater challenge. Yet the benefits to be gained from designing with a view to both applied and basic research remain equally desirable. Against this background, the theme for this instalment of Multimodal Corpora is how multimodal corpora can be designed to serve this double purpose. Success stories of corpora that have provided insights into both applied and basic research are welcome, as are design discussions, methods and tools that help towards this dual goal.

This workshop follows similar events held at LREC 00, 02, 04, 06, 08, 10, ICMI 11, LREC 2012, and IVA 2013. All workshops are documented under and complemented by a special issue of the Journal of Language Resources and Evaluation which came out in 2008, a state-of-the-art book published by Springer in 2009 and a special issue of the Journal of Multimodal User Interfaces under publication. There is an increasing interest in multimodal communication and multimodal corpora as visible by European Networks of Excellence and integrated projects such as HUMAINE, SIMILAR, CHIL, AMI, CALLAS and SSPNet. Furthermore, the success of recent conferences and workshops dedicated to multimodal communication (ICMI-MLMI, IVA, Gesture, PIT, Nordic Symposium on Multimodal Communication, Embodied Language Processing) and the creation of the Journal of Multimodal User Interfaces also testifies the growing interest in this area, and the general need for data on multimodal behaviours.

The LREC'2014 workshop on multimodal corpora will feature a special session on the design and use of multimodal corpora for applications and multimodal technology development.

Other topics to be addressed include, but are not limited to:

  • Multimodal corpus collection activities (e.g. direction-giving dialogues, emotional behaviour, human-avatar interaction, human-robot interaction, etc.) and descriptions of existing multimodal resources
  • Relations between modalities in natural (human) interaction and in human-computer interaction
  • Multimodal interaction in specific scenarios, e.g. group interaction in meetings
  • Coding schemes for the annotation of multimodal corpora
  • Evaluation and validation of multimodal annotations
  • Methods, tools, and best practices for the acquisition, creation, management, access, distribution, and use of multimedia and multimodal corpora
  • Interoperability between multimodal annotation tools (exchange formats, conversion tools, standardization)
  • Collaborative coding
  • Metadata descriptions of multimodal corpora
  • Automatic annotation, based e.g. on motion capture or image processing, and the integration with manual annotations
  • Corpus-based design of multimodal and multimedia systems, in particular systems that involve human-like modalities either in input (Virtual Reality, motion capture, etc.) and output (virtual characters)
  • Automated multimodal fusion and/or generation (e.g., coordinated speech, gaze, gesture, facial expressions)
  • Machine learning applied to multimodal data
  • Multimodal dialogue modelling

Important Dates

Deadline for paper submission (minimum title, authors, abstract): 21 February 2014
Deadline for revisions of submitted papers (full paper): 28 February 2014
Notification of acceptance: 10 March
Final version of accepted paper: 17 March
Final program and proceedings: 6 April
Workshop: 27 May


The workshop will consist primarily of paper presentations and discussion/working sessions. Submissions should be 4 pages long, must be in English, and follow the submission guidelines (to be published by LREC).

Demonstrations of multimodal corpora and related tools are encouraged as well (a demonstration outline of 2 pages can be submitted).

Submissions are made through the START V2 conference manager. The submission package will be available shortly.

Time schedule and registration fee

The workshop will consist of a morning session and an afternoon session. There will be time for collective discussions.

The fee will be specified on the LREC 2014 website.

Organizing Committee

Jens Edlund, KTH Royal Institute of Technology, Sweden
Dirk Heylen, University of Twente, The Netherlands
Patrizia Paggio, University of Copenhagen, Denmark/University of Malta, Malta

maintained by kipp