The uneven development of medical services in different regions, coupled with the need to protect doctors from the risk of contracting infectious diseases, has led to the gradual growth of remote robotic medical diagnosis. However, when utilizing robots for remote diagnosis, patients, especially elderly, children or pregnant women, may perform dangerous actions due to their confusion, curiosity, or anxiety. So, in this project with MITI (Minimally invasive Interdisciplinary Therapeutical Interventions) group, we aim to address the issue of potential injury specifically caused by sudden movements.
Here is our pipeline. Firstly, we have a camera on the top that captures both RGB and depth scene images. Then, the image is processed by a human pose detection model to identify key points of the patient. By analyzing the consecutive frames, the system evaluates the motion of each body joint and detects any risky motion patterns that might indicate a potential collision. If there is any detected risky motion, the system immediately retreats the robot arm following a safe trajectory. Otherwise, the examination continues. The demos are displayed below:
Demo video: Notice that, the human pose detection model is based on OpenPose library (developed by CMU)