Application of Objective Clinical Human Reliability Analysis (OCHRA) Methodology for Near Miss and Error Analysis of Basic Robotic Surgical Skills
EAES Academy. Gorard J. 07/05/22; 363125; P170
Jack Gorard
Contributions
Contributions
Abstract
Aims:
(I) to test the application of intensive formative assessment during basic robotic training and evaluate its impact on learning
(II) Apply Objective Clinical Human Reliability Analysis (OCHRA) to basic, generic robotic skills through video analysis
Method:
112 procedures of basic robotic skills in a simulation lab, assessing basic skills in Virtual Reality (VR) on the da Vinci Skills Simulator (dVSS) and in the dry lab where videos were formatively assessed using Global Evaluative Assessment of Robotic Skills (GEARS) tool as well as the VR scores. Tasks were recorded and analysed with OCHRA methodology. This intensive 5-day training programme had 7 participants ranging from complete novice residents to a specialist, all with no prior robotic experience. Tasks selected were Sea Spikes, Ring Rollercoaster, Needle Driving and Knot Tying. Assessments compared were VR, VR GEARS and Dry model GEARS scores of each task at the start and end of the week. VR and GEARS breakdown scores were analysed and correlated with the OCHRA score methodology developed for basic robotic skills. ANOVA test was used to compare day 1 and day 4 for significant improvement, with Pearson’s correlation coefficient used to compare the different objective assessments.
Results:
Sea Spikes, Ring Rollercoaster, Big Dipper Needle Driver, Knot Tying average scores on day 1 VR were 56%, 14%, 41%, 19% respectively compared to GEARS assessment percentages of the VR video 63%, 55%, 73%, 62% and Dry models 64%, 67%, 69%, and 62%. Day 4 the average assessment scores were 65%, 47%, 67%, 64% for VR, GEARS VR scores 88%, 82%, 92%, 87% and for GEARS scores on the dry model 87%, 90%, 91%, 87%. The participants’ average scores comparing day 1 to day 4 across all four VR and Dry model tasks showed significant improvement (p=0.003). A strong correlation shown between the objective assessment tools, VR scores to VR GEARS r=0.84, VR scores to Dry GEARS r=0.72 and VR GEARS to Dry GEARS r=0.92.
Conclusion:
This feasibility study demonstrates the effective application of formative assessment tools to aid learning of novice surgeons undergoing basic robotic surgery training. OCHRA methodology is applied to assess its feasibility for error analysis of basic robotic skills.
(I) to test the application of intensive formative assessment during basic robotic training and evaluate its impact on learning
(II) Apply Objective Clinical Human Reliability Analysis (OCHRA) to basic, generic robotic skills through video analysis
Method:
112 procedures of basic robotic skills in a simulation lab, assessing basic skills in Virtual Reality (VR) on the da Vinci Skills Simulator (dVSS) and in the dry lab where videos were formatively assessed using Global Evaluative Assessment of Robotic Skills (GEARS) tool as well as the VR scores. Tasks were recorded and analysed with OCHRA methodology. This intensive 5-day training programme had 7 participants ranging from complete novice residents to a specialist, all with no prior robotic experience. Tasks selected were Sea Spikes, Ring Rollercoaster, Needle Driving and Knot Tying. Assessments compared were VR, VR GEARS and Dry model GEARS scores of each task at the start and end of the week. VR and GEARS breakdown scores were analysed and correlated with the OCHRA score methodology developed for basic robotic skills. ANOVA test was used to compare day 1 and day 4 for significant improvement, with Pearson’s correlation coefficient used to compare the different objective assessments.
Results:
Sea Spikes, Ring Rollercoaster, Big Dipper Needle Driver, Knot Tying average scores on day 1 VR were 56%, 14%, 41%, 19% respectively compared to GEARS assessment percentages of the VR video 63%, 55%, 73%, 62% and Dry models 64%, 67%, 69%, and 62%. Day 4 the average assessment scores were 65%, 47%, 67%, 64% for VR, GEARS VR scores 88%, 82%, 92%, 87% and for GEARS scores on the dry model 87%, 90%, 91%, 87%. The participants’ average scores comparing day 1 to day 4 across all four VR and Dry model tasks showed significant improvement (p=0.003). A strong correlation shown between the objective assessment tools, VR scores to VR GEARS r=0.84, VR scores to Dry GEARS r=0.72 and VR GEARS to Dry GEARS r=0.92.
Conclusion:
This feasibility study demonstrates the effective application of formative assessment tools to aid learning of novice surgeons undergoing basic robotic surgery training. OCHRA methodology is applied to assess its feasibility for error analysis of basic robotic skills.
Aims:
(I) to test the application of intensive formative assessment during basic robotic training and evaluate its impact on learning
(II) Apply Objective Clinical Human Reliability Analysis (OCHRA) to basic, generic robotic skills through video analysis
Method:
112 procedures of basic robotic skills in a simulation lab, assessing basic skills in Virtual Reality (VR) on the da Vinci Skills Simulator (dVSS) and in the dry lab where videos were formatively assessed using Global Evaluative Assessment of Robotic Skills (GEARS) tool as well as the VR scores. Tasks were recorded and analysed with OCHRA methodology. This intensive 5-day training programme had 7 participants ranging from complete novice residents to a specialist, all with no prior robotic experience. Tasks selected were Sea Spikes, Ring Rollercoaster, Needle Driving and Knot Tying. Assessments compared were VR, VR GEARS and Dry model GEARS scores of each task at the start and end of the week. VR and GEARS breakdown scores were analysed and correlated with the OCHRA score methodology developed for basic robotic skills. ANOVA test was used to compare day 1 and day 4 for significant improvement, with Pearson’s correlation coefficient used to compare the different objective assessments.
Results:
Sea Spikes, Ring Rollercoaster, Big Dipper Needle Driver, Knot Tying average scores on day 1 VR were 56%, 14%, 41%, 19% respectively compared to GEARS assessment percentages of the VR video 63%, 55%, 73%, 62% and Dry models 64%, 67%, 69%, and 62%. Day 4 the average assessment scores were 65%, 47%, 67%, 64% for VR, GEARS VR scores 88%, 82%, 92%, 87% and for GEARS scores on the dry model 87%, 90%, 91%, 87%. The participants’ average scores comparing day 1 to day 4 across all four VR and Dry model tasks showed significant improvement (p=0.003). A strong correlation shown between the objective assessment tools, VR scores to VR GEARS r=0.84, VR scores to Dry GEARS r=0.72 and VR GEARS to Dry GEARS r=0.92.
Conclusion:
This feasibility study demonstrates the effective application of formative assessment tools to aid learning of novice surgeons undergoing basic robotic surgery training. OCHRA methodology is applied to assess its feasibility for error analysis of basic robotic skills.
(I) to test the application of intensive formative assessment during basic robotic training and evaluate its impact on learning
(II) Apply Objective Clinical Human Reliability Analysis (OCHRA) to basic, generic robotic skills through video analysis
Method:
112 procedures of basic robotic skills in a simulation lab, assessing basic skills in Virtual Reality (VR) on the da Vinci Skills Simulator (dVSS) and in the dry lab where videos were formatively assessed using Global Evaluative Assessment of Robotic Skills (GEARS) tool as well as the VR scores. Tasks were recorded and analysed with OCHRA methodology. This intensive 5-day training programme had 7 participants ranging from complete novice residents to a specialist, all with no prior robotic experience. Tasks selected were Sea Spikes, Ring Rollercoaster, Needle Driving and Knot Tying. Assessments compared were VR, VR GEARS and Dry model GEARS scores of each task at the start and end of the week. VR and GEARS breakdown scores were analysed and correlated with the OCHRA score methodology developed for basic robotic skills. ANOVA test was used to compare day 1 and day 4 for significant improvement, with Pearson’s correlation coefficient used to compare the different objective assessments.
Results:
Sea Spikes, Ring Rollercoaster, Big Dipper Needle Driver, Knot Tying average scores on day 1 VR were 56%, 14%, 41%, 19% respectively compared to GEARS assessment percentages of the VR video 63%, 55%, 73%, 62% and Dry models 64%, 67%, 69%, and 62%. Day 4 the average assessment scores were 65%, 47%, 67%, 64% for VR, GEARS VR scores 88%, 82%, 92%, 87% and for GEARS scores on the dry model 87%, 90%, 91%, 87%. The participants’ average scores comparing day 1 to day 4 across all four VR and Dry model tasks showed significant improvement (p=0.003). A strong correlation shown between the objective assessment tools, VR scores to VR GEARS r=0.84, VR scores to Dry GEARS r=0.72 and VR GEARS to Dry GEARS r=0.92.
Conclusion:
This feasibility study demonstrates the effective application of formative assessment tools to aid learning of novice surgeons undergoing basic robotic surgery training. OCHRA methodology is applied to assess its feasibility for error analysis of basic robotic skills.
{{ help_message }}
{{filter}}