The goal of this project is to develop a hierarchical learning system that decomposes complex motor skills into simpler elemental movements, also called movement primitives, that serve as building blocks of our movement strategy. For example, in a tennis game, such primitives can represent different tennis strokes such as a forehand stroke, a backhand stroke or a smash. As we can see, the autonomous decomposition into building blocks is inherent to many motor tasks. In this project, we want to exploit this basic structure for our learning system. To do so, our autonomous learning system has to extract the movement primitives out of observed trajectories, learn to generalize the primitives to different situations and select between, sequence or combine the movement primitives such that complex behavior can be synthesized out of the primitive building blocks. Our autonomous learning system will be applicable to learning from demonstrations as well as subsequent self improvement by reinforcement learning. Learning will take place on several layers of the hierarchy. While on the upper level, the activation policy of different primitives will be learned, the intermediate level of the hierarchy extracts meta-parameters of the primitives and autonomously learns how to adapt these parameters to the current situation. The lowest level of the hierarchy learns the control policies of the single primitives. Learning on all layers as well as the extraction of the structure of the hierarchical policy is aimed to operate with a minimal amount of dependence from a human expert. We will evaluate our autonomous learning framework on a robot table tennis platform, which will give us many insights in the hierarchical structure of complex motor tasks.