Behaviour adaptation

The video below, related to the publication [Milliez et al. (2016)], illustrates how a robot can adapt its behavior to the user preferences and knowledge on each task of the plan.

YARBUS: A simple rule based belief tracker

The source code of the tracker and some tools to explore the DSTC datasets are available at

Recurrent self-organizing neural networks

The videos below show how recurrent self-organizing neural networks learn a representation of the hidden state of a Markov chain where the observations are ambiguous.

Sequence ABCDEFEDCB : a sequence with ambiguous observations

Sequence AAAAAF : a sequence with a long term dependency

Sequence ABCDEFEDCB then Sequence ABCBAFEDEF : a switching sequence

Sequence ABCDEFEDCB with an additional perturbed state from which random observations are received. At the end of the simulation, the perturbed state is dropped out to better appreciate the learned sequence.

We can also train the recurrent self-organzing map to represent the state of a POMDP. We consider a grid world from which only the x position is observed and therefore the observation is ambiguous. The self organizing map then provides an input to an adaptive controller with a linear representation of the Q-values, the features being RBF on the winning position of the self organizing map. The video below shows the performance of the algorithm as learning goes on.

Divergent belief

The video below, related to the publication [Milliez et al. (2014)], illustrates the Sally and Anne test with the PR2 robot: