Real world applications often naturally decompose into several sub-tasks. In many settings (e.g., robotics) demonstrations provide a natural way to specify the sub-tasks. However, most methods for learning from demonstrations either do not provide guarantees that the artifacts learned for the subtasks can be safely recombined or limit the types of composition available.

In this work, we use control improvisation to synthesize voluntary lane-change policy that meets human preferences under given traffic environments. We first train Markov models to describe traffic patterns and the motion of vehicles responding to such patterns using traffic data. The trained parameters are calibrated using control improvisation to ensure the traffic scenario assumptions are satisfied. Based on the traffic pattern, vehicle response models, and Bayesian switching rules, the lane-change environment for an automated vehicle is modeled as a Markov decision process.