AURO2017: Using Probabilistic Movement Primitives in Robotics

Movement Primitives are a well-established paradigm for modular movement representation and generation. They provide a data-driven representation of movements and support generalization to novel situations, temporal modulation, sequencing of primitives and controllers for executing the primitive on physical systems. However, while many MP frameworks exhibit some of these properties, there is a need for a uni-fied framework that implements all of them in a principled way. In this paper, we show that this goal can be achieved by using a probabilistic representation. Our approach models trajectory distributions learned from stochastic movements. Probabilistic operations, such as conditioning can be used to achieve generalization to novel situations or to combine and blend movements in a principled way. We derive a stochastic feedback controller that reproduces the encoded variability of the movement and the coupling of the degrees of freedom of the robot. We evaluate and compare our approach on several simulated and real robot scenarios.

  • A. Paraschos, C. Daniel, J. Peters, and G. Neumann, “Using probabilistic movement primitives in robotics,” Autonomous Robots, 2018.
    [BibTeX] [Abstract] [Download PDF]

    Movement Primitives are a well-established paradigm for modular movement representation and generation. They provide a data-driven representation of movements and support generalization to novel situations, temporal modulation, sequencing of primitives and controllers for executing the primitive on physical systems. However, while many MP frameworks exhibit some of these properties, there is a need for a uni- fied framework that implements all of them in a principled way. In this paper, we show that this goal can be achieved by using a probabilistic representation. Our approach models trajectory distributions learned from stochastic movements. Probabilistic operations, such as conditioning can be used to achieve generalization to novel situations or to combine and blend movements in a principled way. We derive a stochastic feedback controller that reproduces the encoded variability of the movement and the coupling of the degrees of freedom of the robot. We evaluate and compare our approach on several simulated and real robot scenarios.

    @article{lirolem27883,
    journal = {Autonomous Robots},
    title = {Using probabilistic movement primitives in robotics},
    publisher = {Springer Verlag},
    author = {Alexandros Paraschos and Christian Daniel and Jan Peters and Gerhard Neumann},
    month = {December},
    year = {2018},
    keywords = {ARRAY(0x5590072111c8)},
    url = {http://eprints.lincoln.ac.uk/27883/},
    abstract = {Movement Primitives are a well-established
    paradigm for modular movement representation and
    generation. They provide a data-driven representation
    of movements and support generalization to novel situations,
    temporal modulation, sequencing of primitives
    and controllers for executing the primitive on physical
    systems. However, while many MP frameworks exhibit
    some of these properties, there is a need for a uni-
    fied framework that implements all of them in a principled
    way. In this paper, we show that this goal can be
    achieved by using a probabilistic representation. Our
    approach models trajectory distributions learned from
    stochastic movements. Probabilistic operations, such as
    conditioning can be used to achieve generalization to
    novel situations or to combine and blend movements in
    a principled way. We derive a stochastic feedback controller
    that reproduces the encoded variability of the
    movement and the coupling of the degrees of freedom
    of the robot. We evaluate and compare our approach
    on several simulated and real robot scenarios.}
    }

This paper is the extended journal version of

  • A. Paraschos, G. Neumann, and J. Peters, “A probabilistic approach to robot trajectory generation,” in 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2013, pp. 477-483.
    [BibTeX] [Abstract] [Download PDF]

    Motor Primitives (MPs) are a promising approach for the data-driven acquisition as well as for the modular and re-usable generation of movements. However, a modular control architecture with MPs is only effective if the MPs support co-activation as well as continuously blending the activation from one MP to the next. In addition, we need efficient mechanisms to adapt a MP to the current situation. Common approaches to movement primitives lack such capabilities or their implementation is based on heuristics. We present a probabilistic movement primitive approach that overcomes the limitations of existing approaches. We encode a primitive as a probability distribution over trajectories. The representation as distribution has several beneficial properties. It allows encoding a time-varying variance profile. Most importantly, it allows performing new operations – a product of distributions for the co-activation of MPs conditioning for generalizing the MP to different desired targets. We derive a feedback controller that reproduces a given trajectory distribution in closed form. We compare our approach to the existing state-of-the art and present real robot results for learning from demonstration.

    @inproceedings{lirolem25693,
    month = {October},
    booktitle = {13th IEEE-RAS International Conference on Humanoid Robots (Humanoids)},
    title = {A probabilistic approach to robot trajectory generation},
    volume = {2015-F},
    publisher = {IEEE},
    author = {A. Paraschos and G. Neumann and J. Peters},
    number = {Februa},
    year = {2013},
    pages = {477--483},
    abstract = {Motor Primitives (MPs) are a promising approach for the data-driven acquisition as well as for the modular and re-usable generation of movements. However, a modular control architecture with MPs is only effective if the MPs support co-activation as well as continuously blending the activation from one MP to the next. In addition, we need efficient mechanisms to adapt a MP to the current situation. Common approaches to movement primitives lack such capabilities or their implementation is based on heuristics. We present a probabilistic movement primitive approach that overcomes the limitations of existing approaches. We encode a primitive as a probability distribution over trajectories. The representation as distribution has several beneficial properties. It allows encoding a time-varying variance profile. Most importantly, it allows performing new operations - a product of distributions for the co-activation of MPs conditioning for generalizing the MP to different desired targets. We derive a feedback controller that reproduces a given trajectory distribution in closed form. We compare our approach to the existing state-of-the art and present real robot results for learning from demonstration.},
    keywords = {ARRAY(0x5590082b13c0)},
    url = {http://eprints.lincoln.ac.uk/25693/}
    }

  • A. Paraschos, C. Daniel, J. Peters, and G. Neumann, “Probabilistic movement primitives,” in Advances in Neural Information Processing Systems, (NIPS), 2013.
    [BibTeX] [Abstract] [Download PDF]

    Movement Primitives (MP) are a well-established approach for representing modular and re-usable robot movement generators. Many state-of-the-art robot learning successes are based MPs, due to their compact representation of the inherently continuous and high dimensional robot movements. A major goal in robot learning is to combine multiple MPs as building blocks in a modular control architecture to solve complex tasks. To this effect, a MP representation has to allow for blending between motions, adapting to altered task variables, and co-activating multiple MPs in parallel. We present a probabilistic formulation of the MP concept that maintains a distribution over trajectories. Our probabilistic approach allows for the derivation of new operations which are essential for implementing all aforementioned properties in one framework. In order to use such a trajectory distribution for robot movement control, we analytically derive a stochastic feedback controller which reproduces the given trajectory distribution. We evaluate and compare our approach to existing methods on several simulated as well as real robot scenarios.

    @inproceedings{lirolem25785,
    journal = {Advances in Neural Information Processing Systems},
    title = {Probabilistic movement primitives},
    booktitle = {Advances in Neural Information Processing Systems, (NIPS)},
    author = {A. Paraschos and C. Daniel and J. Peters and G. Neumann},
    month = {December},
    year = {2013},
    url = {http://eprints.lincoln.ac.uk/25785/},
    keywords = {ARRAY(0x559008296270)},
    abstract = {Movement Primitives (MP) are a well-established approach for representing modular
    and re-usable robot movement generators. Many state-of-the-art robot learning
    successes are based MPs, due to their compact representation of the inherently
    continuous and high dimensional robot movements. A major goal in robot learning
    is to combine multiple MPs as building blocks in a modular control architecture
    to solve complex tasks. To this effect, a MP representation has to allow for
    blending between motions, adapting to altered task variables, and co-activating
    multiple MPs in parallel. We present a probabilistic formulation of the MP concept
    that maintains a distribution over trajectories. Our probabilistic approach
    allows for the derivation of new operations which are essential for implementing
    all aforementioned properties in one framework. In order to use such a trajectory
    distribution for robot movement control, we analytically derive a stochastic feedback
    controller which reproduces the given trajectory distribution. We evaluate
    and compare our approach to existing methods on several simulated as well as
    real robot scenarios.}
    }