Welcome to NRI-CSA

NRI-CSA stands for National Robotics Initiative - Complementary Situational Awareness for Human-Robot Partnerships.

This is a new 5-year collaborative project involving three collaborative teams at Vanderbilt, Carnegie Mellon, and Johns Hopkins University. The Principal investigators on this grant are Dr. Nabil Simaan (Vanderbilt), Dr. Howie Choset (CMU) and Dr. Russell H. Taylor (JHU).

The grant consists of three partner institutions contributing to laying the foundations to a new concept in robotics that we call Complementary Situational Awareness. Robots have been primarily used to augment human skill during manipulation tasks (e.g. for surgical applications, telemanipulation in hazardous environments) and in some cases to augment sensory presence (e.g. by providing force feedback to surgeons in cases where forces are below humanly perceptible thresholds). In our new approach robots will augment the human user not only in manipulation but also in understanding of the task and in action planning and execution. The idea is that the robots in some cases can sense things beyond human perception and this information may be used by the robot controller to create a model of the environment shape and the interaction characteristics. This robot situational awareness is then used to augment user/surgeon skill and situational awareness for carrying out complex tasks.

Key Project Highlights: Force-controlled Exploration for Updating Virtual Fixtures 

This work deals with development of an approach for using exploration data to update and register an a-priori virtual fixture (high-level assistive telemanipulation law) geometry to a corresponding deformed and displaced physical geometry. This is representative of a robotic surgical intervention where a pre-operative surgical path-plan has to be updated due to organ intra-operative organ shift/swelling. Using hydrid force-motion control, exploration data (position and local surface normal) is used to deform and register the a-priori environment model to the exploration data set. The environment registration is achieved using a deformable registration approach based on coherent point drift. The task-description of the virtual fixture is then deformed and registered in the new environment and the new model is updated and used within a model-mediated telemanipulation framework. The approach is experimentally validated using a da-Vinci research kit (DVRK) master interface and a Cartesian stage robot.

Key Project Highlights: Smultaneous compliance and registration (SCAR) using stiffness and exploration data

Leveraging techniques pioneered by the SLAM community, we present a new filtering approach called simultaneous compliance and registration estimation or CARE. CARE is like SLAM in that it simultaneously determines the pose of a surgical robot while creating a map, but in this case, the map is a compliance map associated with a preoperative model of an organ as opposed to just positional information like landmark locations. The problem assumes that the robot is forcefully contacting and deforming the environment. This palpation has a dual purpose: 1) it provides the necessary geometric information to align or register the robot to a priori models, and 2) with palpation at varying forces, the stiffness/compliance of the environment can be computed. By allowing the robot to palpate its environment with varying forces, we create a force balanced spring model within a Kalman filter framework to estimate both tissue and robot position.

Publications 

  1. Z. Li, M. Shahbazi, N. Patel, E. O. Sullivan, H. Zhang, K. Vyas, P. Chalasani, A. Deguet, P. L. Gehlbach, I. Iordachita, G.-Z. Yang, and R. H. Taylor, "Hybrid Robot-assisted Frameworks for Endomicroscopy Scanning in Retinal Surgeries", IEEE Trans Medical Robotics and Bionics, vol. 2- 2, pp. 176-187, May, 2020. 4/17/2020 4/17/2020.     10.1109/TMRB.2020.2988312
  2. L. Wang and N. Simaan, “Geometric Calibration of Continuum Robots: Joint Space and Equilibrium Shape Deviations,” IEEE Trans. Robot., 2019.
  3. Z. Li, M. Shahbazi, N. Patel, E. O. Sullivan, P. Chalasani, H. Zhang, K. Vyas, A. Deuget, P. L. Gehlbach, I. Iordachita, G.-Z. Yang, and R. H. Taylor, "An Image-Based Control Framework for Teleoperated Semi-Autonomous Retina Endomicroscopy Scanning", in Int. Symposium of Medical Robotics (ISMR), Atlanta, Georgia, April 4-5, 2019.
  4. Z. Li, M. Shahbazi, N. Patel, E. O. Sullivan, H. Zhang, K. Vyas, P. Chalasani, P. L. Gehlbach, I. Iordachita, G.-Z. Yang, and R. H. Taylor, "A Novel Semi-Autonomous Control Framework for Retina Confocal Endomicroscopy Scanning", in International Conference on Intelligent Robots and Systems (IROS), Macao, Nov. 4-8, 2019. pp. 7083-7090. 
  5. R. Arun Srivatsan, N. Zevallos, P. Vagdargi, and H. Choset, “Registration with a small number of sparse measurements,” Int. J. Rob. Res., vol. 38, no. 12–13, pp. 1403–1419, Oct. 2019.
  6. R. Arun Srivatsan, M. Xu, N. Zevallos, and H. Choset, “Probabilistic pose estimation using a Bingham distribution-based linear filter,” Int. J. Rob. Res., vol. 37, no. 13–14, pp. 1610–1631, Dec. 2018.
  7. N. Zevallos et al., “A Real-time Augmented Reality Surgical System for Overlaying Stiffness Information,” Robotics: Science and Systems, 2018. 
  8. H. Salman, E. Ayvali, R. Arun Srivatsan, Y. Ma, N. Zevallos, R. Yasin, L. Wang, N. Simaan, and H. Choset, ”Trajectory-Optimized Sensing for Active Search of Tissue Abnormalities in Robotic Surgery,” International Conference on Robotics and Automation (ICRA), pp. 5356-5363, May 2018.
  9. P. Chalasani, A. Deguet, P. Kazanzides, and R. H. Taylor, “A Computational Framework for Complementary Situational Awareness (CSA) in Surgical Assistant Robots,” Second IEEE International Conference on Robotic Computing (IRC), 2018, pp. 9–16.
  10. R. Yasin, L. Wang, C. Abah, and N. Simaan, “Using continuum robots for force-controlled semi autonomous organ exploration and registration,” 2018 International Symposium on Medical Robotics (ISMR), 2018, pp. 1–6.
  11. N. Zevallos et al., “A surgical system for automatic registration, stiffness mapping and dynamic image overlay,” International Symposium on Medical Robotics (ISMR), 2018.
  12. P. Chalasani, L. Wang, R. Yasin, N. Simaan, and R. H. Taylor, “Preliminary Evaluation of an Online Estimation Method for Organ Geometry and Tissue Stiffness,” IEEE Robot. Autom. Lett., vol. 3, no. 3, pp. 1816–1823, Jul. 2018.
  13. P. Chalasani, A. Deguet, P. Kazanzides, and R. H. Taylor, "Software Architecture for Complementary Situational Awareness (CSA)", in IEEE International Conference on Robotic Computing (IRC), Laguna Hills, Jan 31 - Feb 2, 2018. 
  14. N. Zevallos et al., “A surgical system for automatic registration, stiffness mapping and dynamic image overlay,” 2018 International Symposium on Medical Robotics (ISMR), 2018, pp. 1–6.
  15. R. A. Srivatsan, P. Vagdargi, and H. Choset, “Sparse Point Registration,” International Symposium on Robotics Research, 2017.
  16. K. Patath, R. Arun Srivatsan, N. Zevallos and H. Choset, “Dynamic Texture Mapping of 3D models for Stiffness Map Visualization,” IEEE/RSJ International Conference on Intelligent Robots and Systems, Workshop on Medical Imaging, 2017.
  17. L. Li, B. Yu, C. Yang, P. Vagdargi, R. A. Srivatsan, and H. Choset, “Development of an Inexpensive Tri-axial Force Sensor for Minimally Invasive Surgery,” in IEEE International Conference on Intelligent Robots and Systems, 2017, pp. 906–913.
  18. E. Ayvali, H. Salman, and H. Choset, “Ergodic Coverage In Constrained Environments Using Stochastic Trajectory Optimization” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada, September 24–28, 2017.
  19. A. Srivatsan Rangaprasad, M. Xu, N. Zevallos, and H. Choset, “Bingham Distribution-Based Linear Filter for Online Pose Estimation,” Robotics: Science and Systems, 2017.
  20. R. A. Srivatsan, P. Vagdargi, N. Zevallos, and H. Choset, “Multimodal Registration Using Stereo Imaging and Contact Sensing,” Robotics Science and Systems Workshop, July 2017.
  21. E. Ayvali, A. Ansari, L. Wang, N. Simaan and H. Choset, "Utility-Guided Palpation for Locating Tissue Abnormalities," in IEEE Robotics and Automation Letters, vol. 2, no. 2, pp. 864-871, April 2017.
  22. L. Wang, Z. Chen, P. Chalasani, R. M. Yasin, P. Kazanzides, R.H. Taylor, and N. Simaan, “Force-Controlled Exploration for Updating Virtual Fixture Geometry in Model-Mediated Telemanipulation,” Journal of Mechanisms and Robotics, vol. 9, no.2, pp. 021010, April 2017.
  23. P. Chalasani, R. M. Yasin, L. Wang, N. Simaan, P. Kazanzides, R.H. Taylor, “Constrained Semi-autonomous Telemanipulated Palpation with Assistive Virtual Fixtures,” Hamlyn Symposium on Medical Robotics, London, UK, 2017.
  24. R. Roy, L. Wang, and N. Simaan, " Modeling and Estimation of Friction, Extension, and Coupling Effects in Multisegment Continuum Robots," in IEEE/ASME Transactions on Mechatronics, vol. 22, no. 2, pp. 909-920, December 2016.
  25. Srivatsan, R. A. & Choset, H. (2016),”Multiple Start Branch and Prune Filtering Algorithm for Nonconvex Optimization,” accepted to the Workshop on the Algorithmic Foundations of Robotics.
  26. L. Wang, Z. Chen, P. Chalasani, J. Pile, P. Kazanzides, R. H. Taylor, and N. Simaan, "Updating Virtual Fixtures from Exploration Data in Force-Controlled Model-Based Telemanipulation", in ASME 2016 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, Charlotte, Aug. 21-24, 2016. V05AT07A031
  27. Z. Chen, A. Malpani, P. Chalasani, A. Deguet, S. S. Vedula, P. Kazanzides, and R. H. Taylor, "Virtual fixture assistance for needle passing and knot tying", in IEEE International Conference on Intelligent Robots and Systems. Daejeon, 2016, pp. 2343-2350
  28. Ayvali, E., Srivatsan, R. A., Wang, L., Roy, R., Simaan, N. & Choset, H (2016),” Using Bayesian Optimization to Guide Probing of a Flexible Environment for Simultaneous Registration and Stiffness Mapping,” In International Conference on Robotics and Automation (ICRA’2016), pages 931 – 936.
  29. Bajo, A. & Simaan, N. (2016),”Hybrid Motion/Force Control of Multi-Backbone Continuum Robots,” International Journal of Robotics Research, 35(4), 422-434.
  30. Srivatsan, R. A., Rosen, G. T., Ismail, F. N. & Choset, H. (2016),”Estimating SE(3) elements using a dual quaternion based linear Kalman filter,” Robotics: Science and Systems, July, 2016.
  31. Srivatsan, R. A., Wang, L., Ayvali, E., Simaan, N. & Choset, H. (2016),” Simultaneous Registration and Stiffness mapping of a Flexible Environment using Stiffness and Geometric Prior,” The Hamlyn Symposium on Medical Robotics, 2016. 
  32. Roy, R., Wang, L. & Simaan, N (2016b),” Investigation of effects of dynamics on intrinsic wrench sensing in continuum robots,” In International Conference on Robotics and Automation (ICRA’2016). 
  33. P. Chalasani, L. Wang, R. Roy, N. Simaan, R. H. Taylor, and M. Kobilarov, "Concurrent Nonparametric Estimation of Organ Geometry and Tissue Stiffness Using Continuous Adaptive Palpation", in IEEE Conference on Robotics and Automation, Stockholm, May 17-19, 2016.  pp. 4164-4171.
  34. R. Arun Srivatsan, E. Ayvali, L. Wang, R. Roy, N. Simaan and H. Choset, ”Complementary Model Update: A Method for Simultaneous Registration and Stiffness Mapping in Flexible Environments,” IEEE International Conference on Robotics and Automation (ICRA), pages 924-930, Stockholm, Sweden, 2016 
  35. Simaan, N., Taylor, R. H. & Choset, H. (2015),” Intelligent Surgical Robots with Situational Awareness: from Good to Great Surgeons,” ASME Dynamic Systems Magazine, 137, no. 9 (2015): 3-6.
  36. Srivatsan, R. A. & Choset, H. (2014). Using Lie algebra for shape estimation of medical snake robots. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), 2014, 3483-3488.
  37. Wang, L. & Simaan, N (2014),” Investigation of Error Propagation in Multi-Backbone Continuum Robots,” In Advances in Robot Kinematics, pages 385-394. Springer International Publishing

 

Ph.D. Dissertations

  1. Rashid M. Yasin. PERCEPTION AUGMENTATION AND ASSISTANCE FOR IMPROVED SURGICAL AWARENESS. (2019).  Vanderbilt University. 
  2. Long Wang. Modeling, Calibration, and Intelligently-Assisted Telemanipulation of Surgical Continuum Robots. (2018).  Vanderbilt University. 
  3. Preetham Chalasani. Complementary Situational Awareness For Intelligent Telerobotic Surgical Assistant System. (2018).  Johns Hopkins University.
  4. R. A. Srivatsan. Probabilistic Approaches for Pose Estimation. (2018).  Carnegie Mellon University. 
  • You are here:  
  • Home