Home

My name is Gerhard Neumann and I am Professor of Robotics and Autonomous Systems at the University of Lincoln. I am heading the Computational Learning for Autonomous Systems (CLAS) team which is part of the Lincoln Centre for Autonomous Systems (LCAS). Part of the team is still based at the TU Darmstadt, my former affiliation. We still heavily cooperate with other teams from TUDa such as the IAS team from Prof. Jan Peters or the group from Prof. Johannes Fuernkranz. We work on machine learning methods for autonomous systems, with a focus on reinforcement learning, policy search and imitation learning.  Our group’s research focus is the use of domain knowledge from robotics to develop new data-driven machine learning algorithms that scale favorably to the complexity of robotic tasks.

This site is still under construction. More content will be added soon.

Research Fields

Our research concentrates on the following sub-fields of machine learning:

Applications

We focus on a wide range of applications where machine learning methods could prove a huge benefit in the future. We work on specific applications (agriculture, nuclear robotics) where robots are highly needed in the next years while other application areas serve as proof-of-concept studies to evaluate our algorithms. Our application fields include:

  • Grasping and Manipulation
  • Agri-culture Robotics
  • Sort and Segregate of Nuclear Waste
  • Dynamic Motor Games, Table-Tennis, Beer-Bong…
  • Robot Swarms

News

  • 2 fully funded PhD positions available!

    The positions are part of the National Center for Nuclear Robotics and funding is available for 3.5 years. A very cool opportunity to work on challenging robotics problems. Please apply here!

  • Paper accepted at ICML!
    • O. Arenz, M. Zhong, and G. Neumann, “Efficient Gradient-Free Variational Inference using Policy Search,” in Proceedings of the International Conference on Machine Learning, 2018.
      [BibTeX] [Download PDF]
      @inproceedings{VIPS_and_supplement,
      added-at = {2018-06-28T12:55:36.000+0200},
      author = {Arenz, O. and Zhong, M. and Neumann, G.},
      biburl = {https://www.bibsonomy.org/bibtex/2b82a0dfe17060b859497bcd1099dc194/gerineumann},
      booktitle = {Proceedings of the International Conference on Machine Learning},
      crossref = {p11147},
      interhash = {7e1e7d618e6ef764ac3aa8e6d39efed3},
      intrahash = {b82a0dfe17060b859497bcd1099dc194},
      key = {variational inference, policy search, sampling},
      keywords = {imported},
      timestamp = {2018-06-28T12:55:36.000+0200},
      title = {Efficient Gradient-Free Variational Inference using Policy Search},
      url = {https://www.ias.informatik.tu-darmstadt.de/uploads/Team/OlegArenz/VIPS_and_supplement.pdf},
      year = 2018
      }

  • New paper accepted at JMLR!
    • R. Akrour, A. Abdolmaleki, H. Abdulsamad, J. Peters, and G. Neumann, “Model-Free Trajectory-based Policy Optimization with Monotonic Improvement,” Journal of Machine Learning Research (JMLR), 2018.
      [BibTeX] [Download PDF]
      @article{moto_jmlr18,
      added-at = {2018-06-28T12:56:24.000+0200},
      author = {Akrour, R. and Abdolmaleki, A. and Abdulsamad, H. and Peters, J. and Neumann, G.},
      biburl = {https://www.bibsonomy.org/bibtex/267611294111fcc089a843503bfb11e20/gerineumann},
      crossref = {p11148},
      interhash = {3f677be600711abda1e54fad914e7fa9},
      intrahash = {67611294111fcc089a843503bfb11e20},
      journal = {Journal of Machine Learning Research (JMLR)},
      keywords = {imported},
      timestamp = {2018-06-29T18:50:47.000+0200},
      title = {Model-Free Trajectory-based Policy Optimization with Monotonic Improvement},
      url = {https://www.ias.informatik.tu-darmstadt.de/uploads/Team/RiadAkrour/moto_jmlr18.pdf},
      year = 2018
      }

  • Paper accepted at Ants2018!
    • M. Hüttenrauch, A. Šošić, and G. Neumann, “Local Communication Protocols for Learning Complex Swarm Behaviors with Deep Reinforcement Learning,” in International Conference on Swarm Intelligence (ANTS), 2018.
      [BibTeX] [Download PDF]
      @inproceedings{huttenrauch2018local,
      added-at = {2018-06-29T12:08:40.000+0200},
      author = {H{\"u}ttenrauch, Maximilian and \v{S}o\v{s}i\'{c}, Adrian and Neumann, Gerhard},
      biburl = {https://www.bibsonomy.org/bibtex/2cc4bd057a1b9af23e42a874f07864da3/gerineumann},
      booktitle = {International Conference on Swarm Intelligence (ANTS)},
      interhash = {0efa19c3dab20befafabdb3184100181},
      intrahash = {cc4bd057a1b9af23e42a874f07864da3},
      keywords = {imported},
      publisher = {Springer International Publishing},
      timestamp = {2018-06-30T14:00:03.000+0200},
      title = {Local Communication Protocols for Learning Complex Swarm Behaviors with Deep Reinforcement Learning},
      url = {http://computational-learning.net/wp-content/uploads/ants_camera.pdf},
      year = 2018
      }

  • New Survey Paper: An Algorithmic Perspective on Imitation Learning
    • T. Osa, J. Pajarinen, G. Neumann, A. J. Bagnell, P. Abbeel, and J. Peters, “An Algorithmic Perspective on Imitation Learning,” Foundations and Trends in Robotics, vol. 7, iss. 1-2, pp. 1-179, 2018. doi:10.1561/2300000053
      [BibTeX] [Download PDF]
      @article{ImitationLearningSurvey2018,
      author = {Takayuki Osa and
      Joni Pajarinen and
      Gerhard Neumann and
      J. Andrew Bagnell and
      Pieter Abbeel and
      Jan Peters},
      title = {An Algorithmic Perspective on Imitation Learning},
      journal = {Foundations and Trends in Robotics},
      volume = {7},
      number = {1-2},
      pages = {1--179},
      year = {2018},
      url = {https://doi.org/10.1561/2300000053},
      doi = {10.1561/2300000053},
      timestamp = {Mon, 30 Apr 2018 01:00:00 +0200},
      biburl = {https://dblp.org/rec/bib/journals/ftrob/OsaPNBA018},
      bibsource = {dblp computer science bibliography, https://dblp.org}
      }

  • 3 new ICRA papers!
    • G. H. W. Gebhardt, K. Daun, M. Schnaubelt, and G. Neumann, “Robust Learning of Object Assembly Tasks with an Invariant Representation of Robot Swarms,” in Proceedings of the International Conference on Robotics and Automation (ICRA), 2018.
      [BibTeX]
      @inproceedings{Gebhardt_PICRA_2018,
      author = "Gebhardt, G.H.W. and Daun, K. and Schnaubelt, M. and Neumann, G.",
      year = "2018",
      title = "Robust Learning of Object Assembly Tasks with an Invariant Representation of Robot Swarms",
      booktitle = "Proceedings of the International Conference on Robotics and Automation (ICRA)",
      crossref = "p11126"
      }

    • D. Koert, G. Maeda, G. Neumann, and J. Peters, “Learning Coupled Forward-Inverse Models with Combined Prediction Errors,” in Proceedings of the International Conference on Robotics and Automation (ICRA), 2018.
      [BibTeX]
      @inproceedings{Koert_PICRA_2018,
      author = "Koert, D. and Maeda, G. and Neumann, G. and Peters, J.",
      year = "2018",
      title = "Learning Coupled Forward-Inverse Models with Combined Prediction Errors",
      booktitle = "Proceedings of the International Conference on Robotics and Automation (ICRA)",
      key = "3rd-hand",
      crossref = "p11119"
      }

    • R. Pinsler, R. Akrour, T. Osa, J. Peters, and G. Neumann, “Sample and Feedback Efficient Hierarchical Reinforcement Learning from Human Preferences,” in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2018.
      [BibTeX] [Download PDF]
      @inproceedings{icra18_robert,
      author = "Pinsler, R. and Akrour, R. and Osa, T. and Peters, J. and Neumann, G.",
      year = "2018",
      title = "Sample and Feedback Efficient Hierarchical Reinforcement Learning from Human Preferences",
      booktitle = "Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)",
      key = "ias",
      URL = "http://www.ausy.tu-darmstadt.de/uploads/Team/RiadAkrour/icra18_robert.pdf",
      crossref = "p11125"
      }

  • New Survey Paper: Preference-based Reinforcement Learning

    New survey paper published at JMLR!

    • C. Wirth, R. Akrour, G. Neumann, and J. Furnkranz, “A Survey of Preference-Based Reinforcement Learning Methods,” Journal of Machine Learning Research, vol. 18, iss. 136, pp. 1-46, 2017.
      [BibTeX] [Download PDF]
      @Article{JMLR:v18:16-634,
      Title = {A Survey of Preference-Based Reinforcement Learning Methods},
      Author = {Christian Wirth and Riad Akrour and Gerhard Neumann and Johannes Furnkranz},
      Journal = {Journal of Machine Learning Research},
      Year = {2017},
      Number = {136},
      Pages = {1-46},
      Volume = {18},
      Url = {http://jmlr.org/papers/v18/16-634.html}
      }

  • We are part of the National Center for Nuclear Robotics (NCNR)!

     

    The National Center for Nuclear Robotics (NCNR) is a multi-disciplinary EPSRC RAI (Robotics and Artificial Intelligence) Hub consisting of most leading nuclear robotics experts in the UK including Universities of Birmingham, Queen Mary, Essex, Bristol, Edinburgh, Lancaster and Lincoln. Under this project, more than 40 postdoctoral researchers and PhD researchers form a team to develop cutting edge scientific solutions to all aspects of nuclear robotics such as sensor and manipulator design, computer vision, robotic grasping and manipulation, mobile robotics, intuitive user interfaces and shared autonomy.

  • 2 Innovate UK projects are starting!

    In Robo-Pic we are starting to pick mushrooms while in Automato, we are harvesting vine tomatoes… It almost makes already a nice topping of a Pizza 🙂

  • 2 new IROS papers accepted!
    • J. Pajarinen, V. Kyrki, M. Koval, S. Srinivasa, J. Peters, and G. Neumann, “Hybrid control trajectory optimization under uncertainty,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017.
      [BibTeX] [Abstract] [Download PDF]

      Trajectory optimization is a fundamental problem in robotics. While optimization of continuous control trajectories is well developed, many applications require both discrete and continuous, i.e. hybrid controls. Finding an optimal sequence of hybrid controls is challenging due to the exponential explosion of discrete control combinations. Our method, based on Differential Dynamic Programming (DDP), circumvents this problem by incorporating discrete actions inside DDP: we first optimize continuous mixtures of discrete actions, and, subsequently force the mixtures into fully discrete actions. Moreover, we show how our approach can be extended to partially observable Markov decision processes (POMDPs) for trajectory planning under uncertainty. We validate the approach in a car driving problem where the robot has to switch discrete gears and in a box pushing application where the robot can switch the side of the box to push. The pose and the friction parameters of the pushed box are initially unknown and only indirectly observable.

      @inproceedings{lirolem28257,
      title = {Hybrid control trajectory optimization under uncertainty},
      booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
      month = {September},
      author = {J. Pajarinen and V. Kyrki and M. Koval and S Srinivasa and J. Peters and G. Neumann},
      year = {2017},
      keywords = {ARRAY(0x564e3c6d8f88)},
      abstract = {Trajectory optimization is a fundamental problem in robotics. While optimization of continuous control trajectories is well developed, many applications require both discrete and continuous, i.e. hybrid controls. Finding an optimal sequence of hybrid controls is challenging due to the exponential explosion of discrete control combinations. Our method, based on Differential Dynamic Programming (DDP), circumvents this problem by incorporating discrete actions inside DDP: we first optimize continuous mixtures of discrete actions, and, subsequently force the mixtures into fully discrete actions. Moreover, we show how our approach can be extended to partially observable Markov decision processes (POMDPs) for trajectory planning under uncertainty. We validate the approach in a car driving problem where the robot has to switch discrete gears and in a box pushing application where the robot can switch the side of the box to push. The pose and the friction parameters of the pushed box are initially unknown and only indirectly observable.},
      url = {http://eprints.lincoln.ac.uk/28257/}
      }

    • A. Paraschos, R. Lioutikov, J. Peters, and G. Neumann, “Probabilistic prioritization of movement primitives,” IEEE Robotics and Automation Letters, vol. PP, iss. 99, 2017.
      [BibTeX] [Abstract] [Download PDF]

      Movement prioritization is a common approach to combine controllers of different tasks for redundant robots, where each task is assigned a priority. The priorities of the tasks are often hand-tuned or the result of an optimization, but seldomly learned from data. This paper combines Bayesian task prioritization with probabilistic movement primitives to prioritize full motion sequences that are learned from demonstrations. Probabilistic movement primitives (ProMPs) can encode distributions of movements over full motion sequences and provide control laws to exactly follow these distributions. The probabilistic formulation allows for a natural application of Bayesian task prioritization. We extend the ProMP controllers with an additional feedback component that accounts inaccuracies in following the distribution and allows for a more robust prioritization of primitives. We demonstrate how the task priorities can be obtained from imitation learning and how different primitives can be combined to solve even unseen task-combinations. Due to the prioritization, our approach can efficiently learn a combination of tasks without requiring individual models per task combination. Further, our approach can adapt an existing primitive library by prioritizing additional controllers, for example, for implementing obstacle avoidance. Hence, the need of retraining the whole library is avoided in many cases. We evaluate our approach on reaching movements under constraints with redundant simulated planar robots and two physical robot platforms, the humanoid robot ?iCub? and a KUKA LWR robot arm.

      @article{lirolem27901,
      volume = {PP},
      year = {2017},
      number = {99},
      publisher = {IEEE},
      month = {July},
      journal = {IEEE Robotics and Automation Letters},
      title = {Probabilistic prioritization of movement primitives},
      booktitle = {, Proceedings of the International Conference on Intelligent Robot Systems, and IEEE Robotics and Automation Letters (RA-L)},
      author = {Alexandros Paraschos and Rudolf Lioutikov and Jan Peters and Gerhard Neumann},
      keywords = {ARRAY(0x564e3c35e8e8)},
      abstract = {Movement prioritization is a common approach
      to combine controllers of different tasks for redundant robots,
      where each task is assigned a priority. The priorities of the
      tasks are often hand-tuned or the result of an optimization,
      but seldomly learned from data. This paper combines Bayesian
      task prioritization with probabilistic movement primitives to
      prioritize full motion sequences that are learned from demonstrations.
      Probabilistic movement primitives (ProMPs) can
      encode distributions of movements over full motion sequences
      and provide control laws to exactly follow these distributions.
      The probabilistic formulation allows for a natural application of
      Bayesian task prioritization. We extend the ProMP controllers
      with an additional feedback component that accounts inaccuracies
      in following the distribution and allows for a more
      robust prioritization of primitives. We demonstrate how the
      task priorities can be obtained from imitation learning and
      how different primitives can be combined to solve even unseen
      task-combinations. Due to the prioritization, our approach can
      efficiently learn a combination of tasks without requiring individual
      models per task combination. Further, our approach can
      adapt an existing primitive library by prioritizing additional
      controllers, for example, for implementing obstacle avoidance.
      Hence, the need of retraining the whole library is avoided in
      many cases. We evaluate our approach on reaching movements
      under constraints with redundant simulated planar robots and
      two physical robot platforms, the humanoid robot ?iCub? and
      a KUKA LWR robot arm.},
      url = {http://eprints.lincoln.ac.uk/27901/}
      }