Robot trained by watching surgeons performs complex procedures with human-level skill
13 November 2024
A new AI-driven surgical robot trained by watching videos of expert surgeons can now perform complex procedures, marking a major advancement toward fully autonomous robotic surgery.
The successful use of imitation learning to train surgical robots eliminates the need to program robots with each individual move required during a medical procedure and brings the field of robotic surgery closer to true autonomy, where robots could perform complex surgeries without human help.
"It's really magical to have this model and all we do is feed it camera input and it can predict the robotic movements needed for surgery," said senior author Axel Krieger.
"We believe this marks a significant step forward toward a new frontier in medical robotics."
The team, which included Stanford University researchers, used imitation learning to train the da Vinci Surgical System robot to perform fundamental surgical procedures: manipulating a needle; lifting body tissue, and suturing.
The model combined imitation learning with the same machine learning architecture that underpins ChatGPT. However, where ChatGPT works with words and text, this model speaks "robot" with kinematics, a language that breaks down the angles of robotic motion into maths.
The researchers fed their model hundreds of videos recorded from wrist cameras placed on the arms of da Vinci robots during surgical procedures. These videos, recorded by surgeons all over the world, are used for post-operative analysis and then archived.
Nearly 7,000 da Vinci robots are used worldwide, and more than 50,000 surgeons are trained on the system, creating a large archive of data for robots to "imitate".
While the da Vinci system is widely used, researchers say it's notoriously imprecise. However, the team found a way to make the flawed input work. The key was training the model to perform relative movements rather than absolute actions, which are inaccurate.
"All we need is image input and then this AI system finds the right action," said lead author Ji Woong "Brian" Kim. "We find that even with a few hundred demos the model is able to learn the procedure and generalise new environments it hasn't encountered."
The team trained the robot to perform three tasks: manipulate a needle, lift body tissue, and suture. In each case, the robot trained on the team's model performed the same surgical procedures as skilfully as human doctors.
"Here the model is so good [at] learning things we haven't taught it," Krieger said.
"Like if it drops the needle, it will automatically pick it up and continue. This isn't something I taught it [to] do."
The model could be used to train a robot quickly to perform any type of surgical procedure, the researchers said. The team is now using imitation learning to train a robot to perform not just small surgical tasks but a full surgery.
Before this advancement, programming a robot to perform even a simple aspect of surgery required hand-coding every step. Someone might spend a decade trying to model suturing, Krieger said. And that's suturing for just one type of surgery.
"It's very limiting," Krieger said. "What is new here is we only have to collect imitation learning of different procedures, and we can train a robot to learn it in a couple [of] days.
“It allows us to accelerate to the goal of autonomy while reducing medical errors and achieving more accurate surgery."