Learning to Sense Robustly and Act Effectively
Benjamin Kuipers, P.I.
Silvio Savarese, co-P.I.
University of Michigan
Computer Science & Engineering

The physical environment of a cyber-physical system is unboundedly complex, changing continuously in time and space.  An embodied cyber-physical system, embedded in the physical world, will receive a high bandwidth stream of sensory information, and will send continuous control signals.  Traditional embedded systems restrict the environment or the attributes considered relevant, and depend on human supervision.  How can future cyber-physical systems, operating in unrestricted environments, cope with this complexity?

Our hypothesis is that a successful cyber-physical system will need to be a learning agent, learning the properties of its sensors, effectors, and environment from its own experience, and adapting over time.  Inspired by human developmental learning, we believe that foundational concepts such as Space, Object, Action, etc., are essential for such a learning agent to abstract and control the complexity of its world.  To bridge the gap between continuous interaction with the physical environment, and discrete symbolic descriptions that support effective planning, the agent will need multiple representations for these foundational domains, linked by abstraction relations.

We take inspiration from our previous work on the Spatial Semantic Hierarchy (SSH), a multi-representation approach to learning the structure of large- and small-scale space from exploration.  We are developing the Object Semantic Hierarchy (OSH), which shows how a learning agent can create a hierarchy of representations for visual perception of objects it interacts with.  The OSH shows how the "object abstraction" factors the uncertainty in the sensor stream into object models and object trajectories.  To provide a similar multi-level representation for actions that apply to these object models, we are developing the Qualitative Learner about Actions and Plans (QLAP).  QLAP identifies contingencies in low-level motor and sensor features, and builds higher-level models that can be used to define a hierarchy of actions.  We believe that these self-learned object and action abstractions will be essential for cyber-physical systems operating in an unboundedly complex world.

Bio sketch

Benjamin Kuipers joined the University of Michigan in January 2009 as Professor of Computer Science and Engineering.  Prior to that, he held an endowed Professorship in Computer Sciences at the University of Texas at Austin.  He received his B.A. from Swarthmore College, and his Ph.D. from MIT.  He investigates the representation of commonsense and expert knowledge, with particular emphasis on the effective use of incomplete knowledge.  His research accomplishments include developing the TOUR model of spatial knowledge in the cognitive map, the QSIM algorithm for qualitative simulation, the Algernon system for knowledge representation, and the Spatial Semantic Hierarchy models of knowledge for robot exploration and mapping.  He has served as Department Chair at UT Austin, and is a Fellow of AAAI and IEEE.