Human interaction with the physical world is increasingly mediated by autonomy, as planes assist pilots, robots assist surgeons, and cars assist drivers. Automation is introduced in such systems to aid humans and guarantee safety and performance. However, such guarantees are hard to provide, since humans may misunderstand the automation's intentions or behave in an unanticipated manner; tragic examples like the crash of Air France flight 447 illustrate that such confusion between pilots and autopilots can lead to catastrophic outcomes. Although some applications may someday yield to full automation (e.g., cars can already drive themselves in traffic), legal and ethical concerns related to safety, accountability, and non-repudiation will ensure humans and autonomy must be capable of handing off control authority at multiple levels in many such systems for the foreseeable future. The principal investigator (PI) proposes to flexibly deploy degrees of autonomy in the presence of human collaborators to compensate for changes in task or environmental conditions. Though PI focuses on robotic teleoperation for concreteness, the anticipated results will lead to general principles that benefit a variety of CPS with humans in-the-loop. Providing safety and performance guarantees for any system involving humans is a lofty goal. PI proposes to achieve this goal by (i) targeting our effort on applications in robotic teleoperation and (ii) integrating findings from multiple established academic disciplines, including engineering disciplines like human factors and control theory that consider the interaction between people and dynamic physical processes, as well as scientific disciplines like neuromechanical motor control and behavioral game theory that account for how humans interact individually and in groups. The proposed work paves the way for provably-safe teleoperated robots distributed in an urban area to provide services in transportation, manufacturing, telemedicine, and emergency response by developing principles for predictive modeling and automated interventions. Unlike the present day, where incompatibilities in aims or means for humans and autonomy lead to performance degradation ranging from significant to catastrophic, the proposed work envisions a future wherein humans can be safely deployed amidst cyber-physical systems in society with high confidence.