Hybrid Systems for Modeling, Performing and Teaching the Language of Surgery
Abstract
The aim of this project is to investigate methods for modeling human expert surgeons performing robotic minimally invasive surgery, and to understand how to reflect this expertise back upon students in the form of teaching and training. At a fundamental level, we view modeling the surgical “signal” as one of language modeling. We have pursued two approaches to general-‐purpose low-‐level modeling. In one path, we are extending traditional HMM-‐like models with more complex state dynamics. We have now advanced that work to a structured model in which each high-‐dimensional kinematic data vector is viewed as the linear transformation of a low-‐dimensional gesture-‐dependent motion vector plus a gesture-‐independent nuisance (noise) vector. We then introduced temporal 
dynamics into these two underlying vectors. Finally, we made the parameters of the temporal dynamics state-‐dependent. We call this the structured switching vector autoregressive (SS-‐VAR) model. At the same time, we have looked at global optimization methods for classification of data on multiple manifolds based on structured sparse representation. Our most recent work treats the training data as a self-‐expressive dictionary where the data in each class are organized into several blocks of the dictionary where each block corresponds to a low-‐dimensional subspace of a high-‐dimensional ambient space. We have extended this model to have dynamics, similar to a classical HMM, leading to a modeling technique we refer to as Sparse HMMs (S-‐HMM). S-‐HMMs outperform all other methods that we have investigated to date as shown at the right. In particular, a one-‐state S-‐HMM model has comparable performance to a three-‐state switched dynamical system.
We have used these modeling tools to recognize and create contextual feedback to users. In particular, we have explored methods for automated collaborative (traded control) execution of robot motions. The overall execution of a task can be viewed as a state transition system, where each state may be performed by the human operator or automatically. The states are recognized using the previously describe discrete models, and the motion is a learned representation from training data.
Finally, we have continued to push forward in using our results in surgical assessment. In new work, we have developed a new approach to modeling variation due to skill. This approach makes use of two different representations for motion. The first is to compute a near-‐optimal path for a particular task, and then to measure variation from that path; we find this correlates well with skill. The second is to measure the local consistency of motion along a path; this also correlates well with skill. We find the best performance combines both measures, leading to skill prediction of greater than 90% on average.
Award ID: 0931805
Submitted by Gregory Hager
on
Abstract
The aim of this project is to investigate methods for modeling human expert surgeons performing robotic minimally invasive surgery, and to understand how to reflect this expertise back upon students in the form of teaching and training. At a fundamental level, we view modeling the surgical “signal” as one of language modeling. We have pursued two approaches to general-‐purpose low-‐level modeling. In one path, we are extending traditional HMM-‐like models with more complex state dynamics. We have now advanced that work to a structured model in which each high-‐dimensional kinematic data vector is viewed as the linear transformation of a low-‐dimensional gesture-‐dependent motion vector plus a gesture-‐independent nuisance (noise) vector. We then introduced temporal dynamics into these two underlying vectors. Finally, we made the parameters of the temporal dynamics state-‐dependent. We call this the structured switching vector autoregressive (SS-‐VAR) model. At the same time, we have looked at global optimization methods for classification of data on multiple manifolds based on structured sparse representation. Our most recent work treats the training data as a self-‐expressive dictionary where the data in each class are organized into several blocks of the dictionary where each block corresponds to a low-‐dimensional subspace of a high-‐dimensional ambient space. We have extended this model to have dynamics, similar to a classical HMM, leading to a modeling technique we refer to as Sparse HMMs (S-‐HMM). S-‐HMMs outperform all other methods that we have investigated to date as shown at the right. In particular, a one-‐state S-‐HMM model has comparable performance to a three-‐state switched dynamical system.
We have used these modeling tools to recognize and create contextual feedback to users. In particular, we have explored methods for automated collaborative (traded control) execution of robot motions. The overall execution of a task can be viewed as a state transition system, where each state may be performed by the human operator or automatically. The states are recognized using the previously describe discrete models, and the motion is a learned representation from training data.
Finally, we have continued to push forward in using our results in surgical assessment. In new work, we have developed a new approach to modeling variation due to skill. This approach makes use of two different representations for motion. The first is to compute a near-‐optimal path for a particular task, and then to measure variation from that path; we find this correlates well with skill. The second is to measure the local consistency of motion along a path; this also correlates well with skill. We find the best performance combines both measures, leading to skill prediction of greater than 90% on average.
Award ID: 0931805