This line of research seeks to produce computational models that mimic the way we process events in time. In particular, we are interested in the cognitive process by which a person perceives events and extracts patterns that help make predictions. Such predictions alter the way future events are perceived. Human communication in which information is displayed in time is interwoven in the perceive, understand and predict cycle. Both parts of the communication must have such predictions in mind in order to moderate how the message is relayed. Examples of messages with hierarchical structure are paragraphs and chapters in a book, verses and sections in a song or scenes and acts in a film. The details of how events are placed in time make for important features of verbal and non-verbal communication. Having models that predict how we organize events in time can help automatic systems to organize when information is displayed to better entrain with the user’s attention, it can help predict when a speaker will finish speaking by adding the time dimension to prosody analysis, it can improve timing aspects of text-to-speech synthesis or assist automatic analysis of text into summation.
Currently, research is focused on the analysis of structure in music. Music has a very clear hierarchical structure that is present to create and manipulate expectations. Manipulating expectations is one of the mechanisms that music uses to produce emotions. We use music as reasonable examples to develop our first models.
A modeling technique that is fit for the task if that of bayesian inference. This technique is currently being used to model a variety of cognitive processes. Moreover, bayesian inference has been used to extract hierarchical structures from static data. Our aim is to adapt these models to extract hierarchical structure from timed data.
Further reading:
- Joshua B Tenenbaum, Charles Kemp, Thomas L Griffiths, and Noah D Goodman. How to grow a mind: Statistics, structure, and abstraction. science, 331(6022):1279–1285, 2011.
- Ruslan Salakhutdinov, Joshua Tenenbaum, and Antonio Torralba. One-shot learning with a hierarchical nonparametric bayesian model. In Proceedings of ICML Workshop on Unsupervised and Transfer Learning, pages 195–206, 2012.
- Stanislas Dehaene, Florent Meyniel, Catherine Wacongne, Liping Wang, and Christophe Pallier. The neural representation of sequences: from transition probabilities to algebraic patterns and linguistic trees. Neuron, 88(1):2–19, 2015.