Verbal communication relies on the interaction between many heterogeneous processes involving physical, cognitive and social variables and unfolding over different time scales. The overarching objective of my studies is to understand how this coordination is achieved and how it shapes the linguistic system. In this presentation, I will give an overview of this work and of the methods conceived to support my research program. In the first part of the presentation, I will rapidly summarize the results of recent work showing that 1) invariant coordinative relations link the behaviour of the articulators during the production of speech consonants and that 2) the degree of coordination between the production of syllables and the production of prosodic prominence is a key dimension to understand rhythmic typology. These results have been obtained thanks to the combination of two different approaches to study speech spatiotemporal patterns. These will be illustrated in the second part of the presentation. In the first approach, we use Wavelet Based Functional Mixed Models to estimate the effects of experimental factors on the shape of time series (e.g. F0 curves) and on images (e.g. ultrasound vocal tract images). In the second approach, we adapt state-space methods (originally introduced to characterize dynamical systems) to study the organization of the processes underlying the production of speech patterns (e.g. speech gestures) by analysing their variability. In the last part of the presentation, I will illustrate an ongoing project aimed at studying inter-speaker coordination. The modelling part of the project aims at testing models of inter-speaker coordination by studying the interactions between human speakers and an artificial agent. The artificial agent can coordinate its behaviour with the human speakers during the repetition of simple speech utterances. The exploratory part of the project aims at understanding how and if physiological coordination between speakers involved in a conversation has an impact on the coordination of their speech acts and makes communication more efficient.
Outi Tuomainen (University of Potsdam)
Michael A. Grosvald (Qatar University)
Martin Krämer (The Arctic Unversity of Norway)
Gasper Begus (UC Berkeley)