The objective of this Computer and Information Science and Engineering (CISE) Research Initiation Initiative (CRII) proposal is to develop a cognizant learning framework for cyber-physical systems (CPS) that incorporates risk-sensitive and irrational decision making. The necessity for such a framework is exemplified by two observations. First, CPS such as self-driving cars will share an environment with other CPS and human users. Human drivers demonstrate a heightened sensitivity to changes in speed and can easily adapt to changes in the environment and road conditions, which makes it essential for a CPS to have an ability to recognize non-rational behaviors. Second, large amounts of data generated during their operation and limited access to models of their environments can make a CPS reliant on machine learning algorithms for decision making to meet performance requirements such as reachability and safety. Our research will be grounded on improving behaviors of autonomous vehicles in realistic traffic situations. Outcomes from this effort will contribute to the development of a research paradigm unifying control, learning, and behavioral economics. Students at a Primarily Undergraduate Institution will benefit by being directly involved in all aspects of the research process. Research tasks will involve a team of undergraduate students in a vertically integrated manner where more experienced students will mentor newer team members. 

The proposed effort comprises two thrusts. Thrust 1 will construct utilities to encode CPS performance objectives consistent with practical models of risk-sensitive and irrational decision making. Strategies will be learned by formulating and solving a reinforcement learning problem to maximize this utility. Methods to enable learned strategies to adequately consider delays between evaluation and execution of actions arising from the physical components of the CPS will be developed. Thrust 2 will design algorithms to learn decentralized cognizant strategies when multiple CPS operate in the same environment. To improve reliability in uncertain environments, or when feedback is sparse, techniques to identify contributions of each CPS to a shared utility will be identified. Solution methodologies will be evaluated empirically through extensive experiments and theoretically by determining probabilistic performance guarantees. The PI will develop a research agenda and new undergraduate curriculum in CPS and machine learning at Western Washington University (WWU). Research and educational goals of the project will be integrated through the CARLA simulator for autonomous vehicle research and the F1/10 Autonomous Vehicle platform. The multidisciplinary scope of the project will be emphasized in outreach efforts through Student Outreach Services and STEM Clubs at WWU to encourage and broaden participation from traditionally underrepresented student groups.