Use of 3D printed markers for Mixed Reality-guided maxillofacial surgery
EAES Academy. Sánchez-Varo I. 07/05/22; 366554; P297
Dr. Ignacio Sánchez-Varo
Contributions
Contributions
Abstract
Introduction
The use of Mixed Reality (MR) wearable devices such as the HoloLens 2 (Microsoft) together with Vuforia's computer vision libraries allow the development of image-guided surgery applications on bony structures, such as maxillofacial surgeries. In the past, augmented reality libraries required a two-dimensional 83D) marker such as a QR or a pre-trained image, but nowadays these libraries allow the use of three-dimensional markers. This fact opens up a multitude of possibilities in terms of the fabrication of 3D markers due to the rapid spread of the use of 3D printing.
Materials and methods
A series of markers with different angles, shapes and colors have been manufactured and tested for its use in image-guided maxillofacial surgery, finally arriving at the visual marker presented. This marker has been printed with the i3 MK3 3D printer (Prusa) using polylactic acid (PLA) material in two colors easily distinguishable visually: red and black (Figure 1.A). This printed visual marker has been trained using a training set of the Model Target Generator tool (Vuforia) and integrated in an application for Universal Windows Platform developed with the Unity 3D engine (Unity Technologies) using the MixedRealityToolKit libraries. Within this 3D-printed marker, three radiopaque fiducial markers (GoldAnchor) have been embedded. These markers will be used to register the preoperative 3D model of the patient with the visual marker using MR during surgery. To test the feasibility of using these fiducial markers, they have been analyzed by computed tomography (CT) (Figure 1.B) and fluoroscopy (Figure 1.C and Figure 1.D). Finally, the registration by means of MR of the different anatomical elements obtained from a CT scan together with the implants provides the surgical planner for maxillofacial implants (Figure 1.E).
Results
A proof of concept using this MR-based registration system has been successfully conducted using a phantom for maxillofacial surgery (APB, Frasaco). For use, the surgeon has to direct the gaze towards the 3D printed marker with the MR device (Figure 2.A). Automatically, the system will detect this visual reference marker and superimpose on it in holographic form the 3D model of the bone structure together with the implants to be performed (Figure 2.B). Using an interface that can be interacted with by manual gestures or voice commands, the surgeon will be able to make the bone structure transparent and visualize where to perform the implants (Figure 2.C), obtaining a better spatial perception of the surgical procedure.
Discussion and conclusions
Future work should be carried out to determine the level of accuracy obtained in the identification and real-time tracking of the frame for the registration process, as well as to achieve a fully automatic registration method. Other types of structures such as soft tissues pose more complications of registration in terms of accuracy.
The use of Mixed Reality (MR) wearable devices such as the HoloLens 2 (Microsoft) together with Vuforia's computer vision libraries allow the development of image-guided surgery applications on bony structures, such as maxillofacial surgeries. In the past, augmented reality libraries required a two-dimensional 83D) marker such as a QR or a pre-trained image, but nowadays these libraries allow the use of three-dimensional markers. This fact opens up a multitude of possibilities in terms of the fabrication of 3D markers due to the rapid spread of the use of 3D printing.
Materials and methods
A series of markers with different angles, shapes and colors have been manufactured and tested for its use in image-guided maxillofacial surgery, finally arriving at the visual marker presented. This marker has been printed with the i3 MK3 3D printer (Prusa) using polylactic acid (PLA) material in two colors easily distinguishable visually: red and black (Figure 1.A). This printed visual marker has been trained using a training set of the Model Target Generator tool (Vuforia) and integrated in an application for Universal Windows Platform developed with the Unity 3D engine (Unity Technologies) using the MixedRealityToolKit libraries. Within this 3D-printed marker, three radiopaque fiducial markers (GoldAnchor) have been embedded. These markers will be used to register the preoperative 3D model of the patient with the visual marker using MR during surgery. To test the feasibility of using these fiducial markers, they have been analyzed by computed tomography (CT) (Figure 1.B) and fluoroscopy (Figure 1.C and Figure 1.D). Finally, the registration by means of MR of the different anatomical elements obtained from a CT scan together with the implants provides the surgical planner for maxillofacial implants (Figure 1.E).
Results
A proof of concept using this MR-based registration system has been successfully conducted using a phantom for maxillofacial surgery (APB, Frasaco). For use, the surgeon has to direct the gaze towards the 3D printed marker with the MR device (Figure 2.A). Automatically, the system will detect this visual reference marker and superimpose on it in holographic form the 3D model of the bone structure together with the implants to be performed (Figure 2.B). Using an interface that can be interacted with by manual gestures or voice commands, the surgeon will be able to make the bone structure transparent and visualize where to perform the implants (Figure 2.C), obtaining a better spatial perception of the surgical procedure.
Discussion and conclusions
Future work should be carried out to determine the level of accuracy obtained in the identification and real-time tracking of the frame for the registration process, as well as to achieve a fully automatic registration method. Other types of structures such as soft tissues pose more complications of registration in terms of accuracy.
Introduction
The use of Mixed Reality (MR) wearable devices such as the HoloLens 2 (Microsoft) together with Vuforia's computer vision libraries allow the development of image-guided surgery applications on bony structures, such as maxillofacial surgeries. In the past, augmented reality libraries required a two-dimensional 83D) marker such as a QR or a pre-trained image, but nowadays these libraries allow the use of three-dimensional markers. This fact opens up a multitude of possibilities in terms of the fabrication of 3D markers due to the rapid spread of the use of 3D printing.
Materials and methods
A series of markers with different angles, shapes and colors have been manufactured and tested for its use in image-guided maxillofacial surgery, finally arriving at the visual marker presented. This marker has been printed with the i3 MK3 3D printer (Prusa) using polylactic acid (PLA) material in two colors easily distinguishable visually: red and black (Figure 1.A). This printed visual marker has been trained using a training set of the Model Target Generator tool (Vuforia) and integrated in an application for Universal Windows Platform developed with the Unity 3D engine (Unity Technologies) using the MixedRealityToolKit libraries. Within this 3D-printed marker, three radiopaque fiducial markers (GoldAnchor) have been embedded. These markers will be used to register the preoperative 3D model of the patient with the visual marker using MR during surgery. To test the feasibility of using these fiducial markers, they have been analyzed by computed tomography (CT) (Figure 1.B) and fluoroscopy (Figure 1.C and Figure 1.D). Finally, the registration by means of MR of the different anatomical elements obtained from a CT scan together with the implants provides the surgical planner for maxillofacial implants (Figure 1.E).
Results
A proof of concept using this MR-based registration system has been successfully conducted using a phantom for maxillofacial surgery (APB, Frasaco). For use, the surgeon has to direct the gaze towards the 3D printed marker with the MR device (Figure 2.A). Automatically, the system will detect this visual reference marker and superimpose on it in holographic form the 3D model of the bone structure together with the implants to be performed (Figure 2.B). Using an interface that can be interacted with by manual gestures or voice commands, the surgeon will be able to make the bone structure transparent and visualize where to perform the implants (Figure 2.C), obtaining a better spatial perception of the surgical procedure.
Discussion and conclusions
Future work should be carried out to determine the level of accuracy obtained in the identification and real-time tracking of the frame for the registration process, as well as to achieve a fully automatic registration method. Other types of structures such as soft tissues pose more complications of registration in terms of accuracy.
The use of Mixed Reality (MR) wearable devices such as the HoloLens 2 (Microsoft) together with Vuforia's computer vision libraries allow the development of image-guided surgery applications on bony structures, such as maxillofacial surgeries. In the past, augmented reality libraries required a two-dimensional 83D) marker such as a QR or a pre-trained image, but nowadays these libraries allow the use of three-dimensional markers. This fact opens up a multitude of possibilities in terms of the fabrication of 3D markers due to the rapid spread of the use of 3D printing.
Materials and methods
A series of markers with different angles, shapes and colors have been manufactured and tested for its use in image-guided maxillofacial surgery, finally arriving at the visual marker presented. This marker has been printed with the i3 MK3 3D printer (Prusa) using polylactic acid (PLA) material in two colors easily distinguishable visually: red and black (Figure 1.A). This printed visual marker has been trained using a training set of the Model Target Generator tool (Vuforia) and integrated in an application for Universal Windows Platform developed with the Unity 3D engine (Unity Technologies) using the MixedRealityToolKit libraries. Within this 3D-printed marker, three radiopaque fiducial markers (GoldAnchor) have been embedded. These markers will be used to register the preoperative 3D model of the patient with the visual marker using MR during surgery. To test the feasibility of using these fiducial markers, they have been analyzed by computed tomography (CT) (Figure 1.B) and fluoroscopy (Figure 1.C and Figure 1.D). Finally, the registration by means of MR of the different anatomical elements obtained from a CT scan together with the implants provides the surgical planner for maxillofacial implants (Figure 1.E).
Results
A proof of concept using this MR-based registration system has been successfully conducted using a phantom for maxillofacial surgery (APB, Frasaco). For use, the surgeon has to direct the gaze towards the 3D printed marker with the MR device (Figure 2.A). Automatically, the system will detect this visual reference marker and superimpose on it in holographic form the 3D model of the bone structure together with the implants to be performed (Figure 2.B). Using an interface that can be interacted with by manual gestures or voice commands, the surgeon will be able to make the bone structure transparent and visualize where to perform the implants (Figure 2.C), obtaining a better spatial perception of the surgical procedure.
Discussion and conclusions
Future work should be carried out to determine the level of accuracy obtained in the identification and real-time tracking of the frame for the registration process, as well as to achieve a fully automatic registration method. Other types of structures such as soft tissues pose more complications of registration in terms of accuracy.
{{ help_message }}
{{filter}}