GESTURE CONTROL AND MIXED REALITY-BASED HOLOGRAPHIC MONITOR FOR MINIMALLY INVASIVE SURGICAL ASSISTANCE
EAES Academy. Sánchez-Varo I. 07/05/22; 366541; P284
Dr. Ignacio Sánchez-Varo
Contributions
Contributions
Abstract
OBJECTIVES
To present and evaluate a set of tools for assistance in minimally invasive surgery: (1) a gesture control system for interaction with preoperative studies and (2) a mixed reality-based head-mounted display for laparoscopic surgery.
MATERIALS AND METHODS
The gesture control system consists of a wireless gesture recognition system (Myo armband) that together with the TEDCUBE system (TedCas) allows interaction with preoperative studies remotely and aseptically. Two expert surgeons evaluated its use during the performance of several laparoscopic procedures in a porcine model. During the intervention, they used the system to interact with different types of preoperative information. Additionally, a mixed reality-based application was developed as an auxiliary laparoscopic monitor. The experience with the use of this application was evaluated by sixteen surgeons with different levels of experience during the performance of an eye-hand coordination task on a laparoscopic box trainer.
RESULTS
During the interventions, the gesture control system allowed surgeons to interact with CT and ultrasound studies, as well as with the 3D reconstruction of preoperative models. The surgeons found the system to be a useful, simple and safe tool during surgery. They indicated that they would choose this system as a method for interacting with the patient's preoperative information during surgery, although the accuracy of some gesture control maneuvers would need to be improved beforehand. Regarding the application of the auxiliary monitor for laparoscopic surgery using mixed reality, the participant surgeons completed all the tasks within the maximum time established (300 seconds). Most of them rated positively the effectiveness of using the application in laparoscopic practice. However, they considered the image sharpness and color as aspects that could be improved.
CONCLUSIONS
The gesture control system provides an intuitive interface for interaction with preoperative information during surgery while maintaining the surgeon's aseptic conditions. However, it is necessary to improve its accuracy prior to use in real surgical environments. Participating surgeons reported a positive experience using the mixed reality-based holographic monitor, indicating that they could perform more complex laparoscopic tasks and procedures.
To present and evaluate a set of tools for assistance in minimally invasive surgery: (1) a gesture control system for interaction with preoperative studies and (2) a mixed reality-based head-mounted display for laparoscopic surgery.
MATERIALS AND METHODS
The gesture control system consists of a wireless gesture recognition system (Myo armband) that together with the TEDCUBE system (TedCas) allows interaction with preoperative studies remotely and aseptically. Two expert surgeons evaluated its use during the performance of several laparoscopic procedures in a porcine model. During the intervention, they used the system to interact with different types of preoperative information. Additionally, a mixed reality-based application was developed as an auxiliary laparoscopic monitor. The experience with the use of this application was evaluated by sixteen surgeons with different levels of experience during the performance of an eye-hand coordination task on a laparoscopic box trainer.
RESULTS
During the interventions, the gesture control system allowed surgeons to interact with CT and ultrasound studies, as well as with the 3D reconstruction of preoperative models. The surgeons found the system to be a useful, simple and safe tool during surgery. They indicated that they would choose this system as a method for interacting with the patient's preoperative information during surgery, although the accuracy of some gesture control maneuvers would need to be improved beforehand. Regarding the application of the auxiliary monitor for laparoscopic surgery using mixed reality, the participant surgeons completed all the tasks within the maximum time established (300 seconds). Most of them rated positively the effectiveness of using the application in laparoscopic practice. However, they considered the image sharpness and color as aspects that could be improved.
CONCLUSIONS
The gesture control system provides an intuitive interface for interaction with preoperative information during surgery while maintaining the surgeon's aseptic conditions. However, it is necessary to improve its accuracy prior to use in real surgical environments. Participating surgeons reported a positive experience using the mixed reality-based holographic monitor, indicating that they could perform more complex laparoscopic tasks and procedures.
OBJECTIVES
To present and evaluate a set of tools for assistance in minimally invasive surgery: (1) a gesture control system for interaction with preoperative studies and (2) a mixed reality-based head-mounted display for laparoscopic surgery.
MATERIALS AND METHODS
The gesture control system consists of a wireless gesture recognition system (Myo armband) that together with the TEDCUBE system (TedCas) allows interaction with preoperative studies remotely and aseptically. Two expert surgeons evaluated its use during the performance of several laparoscopic procedures in a porcine model. During the intervention, they used the system to interact with different types of preoperative information. Additionally, a mixed reality-based application was developed as an auxiliary laparoscopic monitor. The experience with the use of this application was evaluated by sixteen surgeons with different levels of experience during the performance of an eye-hand coordination task on a laparoscopic box trainer.
RESULTS
During the interventions, the gesture control system allowed surgeons to interact with CT and ultrasound studies, as well as with the 3D reconstruction of preoperative models. The surgeons found the system to be a useful, simple and safe tool during surgery. They indicated that they would choose this system as a method for interacting with the patient's preoperative information during surgery, although the accuracy of some gesture control maneuvers would need to be improved beforehand. Regarding the application of the auxiliary monitor for laparoscopic surgery using mixed reality, the participant surgeons completed all the tasks within the maximum time established (300 seconds). Most of them rated positively the effectiveness of using the application in laparoscopic practice. However, they considered the image sharpness and color as aspects that could be improved.
CONCLUSIONS
The gesture control system provides an intuitive interface for interaction with preoperative information during surgery while maintaining the surgeon's aseptic conditions. However, it is necessary to improve its accuracy prior to use in real surgical environments. Participating surgeons reported a positive experience using the mixed reality-based holographic monitor, indicating that they could perform more complex laparoscopic tasks and procedures.
To present and evaluate a set of tools for assistance in minimally invasive surgery: (1) a gesture control system for interaction with preoperative studies and (2) a mixed reality-based head-mounted display for laparoscopic surgery.
MATERIALS AND METHODS
The gesture control system consists of a wireless gesture recognition system (Myo armband) that together with the TEDCUBE system (TedCas) allows interaction with preoperative studies remotely and aseptically. Two expert surgeons evaluated its use during the performance of several laparoscopic procedures in a porcine model. During the intervention, they used the system to interact with different types of preoperative information. Additionally, a mixed reality-based application was developed as an auxiliary laparoscopic monitor. The experience with the use of this application was evaluated by sixteen surgeons with different levels of experience during the performance of an eye-hand coordination task on a laparoscopic box trainer.
RESULTS
During the interventions, the gesture control system allowed surgeons to interact with CT and ultrasound studies, as well as with the 3D reconstruction of preoperative models. The surgeons found the system to be a useful, simple and safe tool during surgery. They indicated that they would choose this system as a method for interacting with the patient's preoperative information during surgery, although the accuracy of some gesture control maneuvers would need to be improved beforehand. Regarding the application of the auxiliary monitor for laparoscopic surgery using mixed reality, the participant surgeons completed all the tasks within the maximum time established (300 seconds). Most of them rated positively the effectiveness of using the application in laparoscopic practice. However, they considered the image sharpness and color as aspects that could be improved.
CONCLUSIONS
The gesture control system provides an intuitive interface for interaction with preoperative information during surgery while maintaining the surgeon's aseptic conditions. However, it is necessary to improve its accuracy prior to use in real surgical environments. Participating surgeons reported a positive experience using the mixed reality-based holographic monitor, indicating that they could perform more complex laparoscopic tasks and procedures.
{{ help_message }}
{{filter}}