Deep Reinforcement Learning in Surgical Robotics: Enhancing the Automation Level

Abstract: Surgical robotics is a rapidly evolving field that is transforming the landscape of surgeries. Surgical robots have been shown to enhance precision, minimize invasiveness, and alleviate surgeon fatigue. One promising area of research in surgical robotics is the use of reinforcement learning to enhance the automation level. Reinforcement learning is a type of machine learning that involves training an agent to make decisions based on rewards and punishments. This literature review aims to comprehensively analyze existing research on reinforcement learning in surgical robotics. The review identified various applications of reinforcement learning in surgical robotics, including pre-operative procedure, intra-body procedure, and percutaneous procedure, listed the typical studies, and compared their methodologies and results. The findings show that reinforcement learning has great potential to improve the autonomy of surgical robots. Reinforcement learning can be used to teach robots to perform complex surgical tasks, such as suturing and tissue manipulation. It can also improve the accuracy and precision of surgical robots, making them more effective at performing surgeries.

Comparison of Dynamic Tomato Growth Models for Optimal Control in Greenhouses

Abstract: As global demand for efficiency in agriculture rises, there is a growing interest in high-precision farming practices. Particularly greenhouses play a critical role in ensuring a year-round supply of fresh produce. In order to maximize efficiency and productivity while minimizing resource use, mathematical techniques such as optimal control have been employed. However, selecting appropriate models for optimal control requires domain expertise. This study aims to compare three established tomato models for their suitability in an optimal control framework. Results show that all three models have similar yield predictions and accuracy, but only two models are currently applicable for optimal control due to implementation limitations. The two remaining models each have advantages in terms of economic yield and computation times, but the differences in optimal control strategies suggest that they require more accurate parameter identification and calibration tailored to greenhouses.

Link: https://arxiv.org/abs/2308.06031

A Grasp Controller in Mujoco

This project serves as the final project of the "Introduction to Human Grasping" course, wherein we've developed a grasp controller capable of robustly securing and stabilizing rotating objects of various shapes in Mujoco. The controller is based on position control. Demo video illustrates the application of this controller on the Shadow Hand, successfully grasping a cylindrical, a spherical, and an ellipsoidal object, respectively.