This article focuses on key issues surrounding the needs and application of simulation technologies for technical skills training in otolaryngology. The discussion includes an overview of key topics in training and learning, the application of these issues in simulation environments, and the subsequent applications of these simulation environments to otolaryngology. Examples of past applications are presented, with discussion of how the interplay of cultural changes in surgical training in general along with the rapid advancements in technology have shaped and influenced their adoption and adaptation. The authors conclude with emerging trends and potential influences advanced simulation and training will have on technical skills training in otolaryngology.
The need for surgical simulation in otolaryngology
Despite ever improving less-invasive medical treatment regimens, surgical intervention is still required for many health conditions. As an example, disorders of the temporal bone affect millions of patients in the United States and many require surgical intervention for resolution. To gain surgical proficiency, trainees must possess comprehension of the complex anatomy and associated pathology of the temporal bone. This knowledge must be integrated with refined microsurgical technical skill. This proficiency requires many hours of deliberate practice and considerable clinical experience. Surgical training requires at least 5 years under current methods at a cost of approximately $80,000 per year per resident. Conventional temporal bone laboratories with related equipment cost more than a $1 million to construct and are expensive to maintain (D.B. Welling, personal communication, 2010; recently outfitted complete temporal bone laboratory with 12 stations with instructor station, ∼$2 million). Several factors contribute to inefficiencies in traditional training methodologies, adversely impacting the overall cost of health care. The barriers to progress include less time available for teaching and learning, limitations of instructional resources, and perhaps most importantly the lack of a uniform objective assessment of technical skills.
Less Time Available for Teaching and Learning
There are a few limiting factors adversely influencing the amount of time available for teaching in training centers. First, as health care costs continue to escalate, financial pressures come to bear on teaching physicians. An increased demand for “clinical efficiency” in the operating room and outpatient clinics limits the amount of time faculty can devote to actually teaching. Second, the restrictions of duty hours for trainees imposed to reduce fatigue and related errors also limit the amount of time available for hands-on learning. This presents a disassociation of the traditional model of “apprentice and master” introduced by Halsted and Osler in the late 1800s. Third, this reduction leads to the concern for insufficient development of technical skills acquired in the operating room and requires learning and practice outside of patient care.
Limitations of Instructional Resources
A few limitations of instructional resources are apparent. First, there is a reduced availability of cadaveric material, the previous gold standard for practicing technical maneuvers outside of patient care. Ethical procurement and proper disposal present necessary and continuous challenges. Sources of human material are no longer available from foreign countries and fewer patients and their families are consenting to donation. Second, there is reduced access to faculty expertise, a key instructional resource. Because of financial pressures, faculty have fewer hours available for teaching, especially for critical assessment during formative development that requires considerable time. With financial pressure for more clinical productivity to maintain previous income levels, the numbers of teaching faculty are decreasing. Finally, of continuing concern is exposure to hazardous materials including pathogens present in cadaveric specimens (eg, hepatitis B and C, prion-derived illness, and HIV infection) and increased exposure to formalin that present increased dangers to health. Use of simulation technologies presents the capacity to greatly mitigate these limitations of instructional resources.
Lack of a Uniform Objective Assessment of Technical Skills
Perhaps most importantly, there is a lack of uniform and objective standardized metrics for use in the assessment of technical skills. Without standardized metrics, uniform formative feedback during training and measurement of professional technical proficiency is not being achieved. This has resulted because no methodology previously existed to objectively apply metrics that were valid, reliable, and practical.
Recently, it has been asserted that today’s training methodologies are antiquated and that a new balance between patient safety and physician training is necessary. A more standardized and structured approach to curriculum development, continuous assessment of skills, constructive feedback, and provision for deliberate practice outside of direct patient care are necessary. Elements required in technical skills development include not only facility to practice psychomotor skills but also elements that reinforce adequate knowledge of a specific procedure, such as relevant anatomy, instrumentation, indications, complications, and postoperative management. This reinforcement requires demonstration of a procedure, delineation of the key steps, assurance of comprehension of the key steps, and single component mastery followed by entire procedure mastery. Formative continuous and summative assessment of skill is necessary. Simulation technologies provide the mechanism by which this reinforcement can be achieved with efficient use of the expert.
Simulation in support of technical skills training
Simulation environments are uniquely suited to allow deliberate practice in a nonthreatening environment ( Fig. 1 ). For these systems to be effective they require the integration of automated expert feedback based on rigorous standards and more than just an environment for the replication of a real-world task.
Johnson points out that simulation is particularly suited to provide (1) errors without putting patients at risk (ie, error-based learning); (2) objective performance measurements and a standardized learning process; and (3) that a realistic simulation provides “situational context,” with more effective reification of the elements to be learned.
This article presents the issues of applying advanced computing and simulation environments for supporting technical skills training and meeting the proscribed criteria established by Johnson. It should be noted that this discussion excludes other avenues used for simulation training, including mannequins and box-trainers. The main focus of this article is simulation environments that support the development of technical skills needed to successfully execute surgical procedures and that integrate automated standards.
Simulation in support of technical skills training
Simulation environments are uniquely suited to allow deliberate practice in a nonthreatening environment ( Fig. 1 ). For these systems to be effective they require the integration of automated expert feedback based on rigorous standards and more than just an environment for the replication of a real-world task.
Johnson points out that simulation is particularly suited to provide (1) errors without putting patients at risk (ie, error-based learning); (2) objective performance measurements and a standardized learning process; and (3) that a realistic simulation provides “situational context,” with more effective reification of the elements to be learned.
This article presents the issues of applying advanced computing and simulation environments for supporting technical skills training and meeting the proscribed criteria established by Johnson. It should be noted that this discussion excludes other avenues used for simulation training, including mannequins and box-trainers. The main focus of this article is simulation environments that support the development of technical skills needed to successfully execute surgical procedures and that integrate automated standards.
Technical skills assessment and simulation
The need for accurate assessment of skill is paramount for any effective training and maintenance of skill. Two types of assessment are defined and necessary: formative assessment for improvement and development and summative assessment for evaluation of competency. Formative assessments need to be specific and concrete to suggest actions for improvement (feedback). Summative assessment is data driven and requires statistical rigor and validation to make a valid judgment of an individual’s skill level. The goal with respect to simulation is to demonstrate that competency within the simulator translates into competency in the clinical realm. Competency can be assessed in the simulator by objective measures, such as time to task, error rate, and economy of movement. For this level of assessment to occur, establishment of standards of competency must be developed. Minimally acceptable performance needs to be determined by a panel of experts. These benchmarks must then be established within the simulation and validated in that context by the experts. The inclusion of expert-defined benchmarks helps provide a theoretical link to patient outcomes in the early adaption and adoption of a technical skills training simulator.
Well-designed simulators are posed to provide a paradigm shift in objective assessment of technical skill. Current methods to assess technical competency are seriously flawed and continue to receive little attention from certification bodies. The core competencies defined by the Accreditation Council for Graduate Medical Education do not provide an adequate, objective assessment of technical skill. The reasons for this are many. However, the largest roadblock to improving technical skills assessment is the lack of an objective methodology that is reliable, valid, and practical. Reliability refers to consistency, repeatability, and dependability of the measures. Validity refers to the concept of a metric actually measuring what it is supposed to measure and able to be defined by criterion (correlates with other measures), construct (correlates with level of training), content (reflects content of domain), and face (extent of measure reflecting real life situation). Use of simulators in domains other than otolaryngology has been shown to provide valid, reliable, and practical assessment of technical skill. It is imperative that the field of otolaryngology follows suit in this area of objective assessment.
Technical skills assessment requires a gold standard with which other assessment methodologies can be compared. No truly objective, validated assessments are widely available in otolaryngology. Validated assessment tools are beginning to emerge through the use of expert opinions and surveys. The otologic experience lags behind that which has been established in sinus surgery (see the discussion of the Endoscopic Sinus Surgery Simulator [ES3]). In the context of otologic surgery, these assessment tools are still cumbersome, ill defined, and impractical to administer; they often require long hours of expert review of individual performances and validation. As a result study sizes are small. With respect to that developed for the sinus surgery simulator studies, most are small and further limited because of the cost of the simulator hardware. Only through large-scale, multiinstitutional studies can these tools be more standardized, rigorously studied, validated, and accepted.
The establishment of technical skills assessment tools with expert defined metrics integrated into a simulation-based training system will be the underpinning for proficiency-based training programs. Brydges and coworkers describe a methodology for developing such a simulation-based training program where trainees progress from less to more technically demanding skills and tasks, only after achieving defined criteria. This methodology is, in a sense, implicit in the “apprentice and master” training system but not well defined. The current execution has been marked by nonstandardized and subjective influences with the ultimate criteria being the completion of a prescribed time in training or “adequate number of procedures.” This current concept of surgical skills training and assessment does not provide objective and valid measures of performance. Simulation-based training and assessment provides the means by which valid and objective measures can be instituted and provides a safe environment for surgical trainees to assess their performance rigorously without risk to patients. The experience with the ES3, although on a small scale, has demonstrated that this can be accomplished within the otolaryngology community.
Simulation systems in otolaryngology
Since the introduction of simulation in medicine, otolaryngology has increasingly been involved in promoting its development and validation, most notably in endoscopic sinus surgery and temporal bone surgery. A complete review of these two areas was recently published. The following provides a brief summary of past developments and more current progress.
Endoscopic sinus surgery simulator
The ES3 was the first sinus surgery simulator developed and remains a leading system with several validation studies completed. Inspired by aviation training simulators for military aircraft, the ES3 was developed between 1995 and 1998 by Lockheed Martin, in association with the University of Washington, The Ohio State University (OSU), and the Ohio Supercomputer Center, with sponsorship from the US Army Medical Research and Materiel Command. The ES3 was first used in the Army in 1997 and gained popularity among medical students and residents within the armed forces. The simulation system involves CT-derived three-dimensional paranasal sinus anatomy models and interactions with a virtual endoscopic instrument with haptic feedback. The ES3 uses an “expert surgical assistant” that interprets multimodal input to provide automated feedback to the user and warnings as critical structures are approached. It also allows the user to query the system regarding relevant anatomy and procedural maneuvers. Yale University has developed a state-of-the-art curriculum to standardize training on the ES3 regardless of level, available on a compact disk. The ES3 consortium is currently led by Albert Einstein College of Medicine and includes the Agency for Healthcare Research and Quality (funding); Yale University (curriculum development); New York University Medical Center; New York Eye and Ear Infirmary; Mount Sinai Medical Center (data collection); and the University of Washington–Human Interface Technology Laboratory (web database) ( Fig. 2 ).
“The ES3 is one of the few virtual reality simulators with a comprehensive validation record,” a notion that allows it to be claimed as one of the leading ESS simulation systems. An initial study demonstrating construct validity of the ES3 showed significant correlation between performance on the ES3 and performance on other validated tests of innate ability in psychomotor, visuospatial, and perceptual capacities (to parallel the ESS-required skill of two-handed coordination of surgical instruments in a three-dimensional space). A second study compared the performance of medical students, otolaryngology residents, and attendings on the ES3. The ES3 was clearly able to distinguish between the three levels in initial trials, with the expert performing at the highest level, followed by residents, then medical students. This study also showed that all groups achieved a remarkably similar plateau score by the tenth trial on the simulator, demonstrating the ability of the ES3 to consistently achieve a standard performance goal in users. The most recent validation study (Virtual Reality to Operating Room), which provides the strongest clinical correlation, evaluates whether training on the ES3 translates to improved performance in actual surgery. This study showed that otolaryngology junior residents who received both conventional sinus surgery training and ES3 training (experimental group) performed significantly better than residents who received only conventional training (control group). Improved performance in the experimental group includes significantly shorter operating time, demonstration of higher confidence, better skills in instrument manipulation, and fewer technical errors. Of interest, the ES3 has also been shown to be an effective tool in training ophthalmology residents in endoscopic endonasal dacryocystorhinostomy at the Albert Einstein College of Medicine, effectively extending its use beyond the field of otolaryngology. The limitations of the ES3 are that it is no longer in production and there are only a handful of systems in existence. Additionally, there has been no update to the underlying technology supporting the system since its inception over a decade ago. Dr Marvin Fried of Albert Einstein College of Medicine has continued to champion the use of this simulator and has been a pioneer in its application ( Fig. 3 ).
Other ESS simulators
Although the ES3 remains the primary ESS simulation system in the United States, several other ESS simulators are worthy of mention. The University of Hamburg-Eppendorf group in Germany (developers of the VOXEL-MAN TempoSurg simulator) recently developed a new paranasal sinus surgery simulator that is compatible with standard personal computer hardware. This system uses three-dimensional models of human skulls obtained from high-resolution CT images with mucosa and vital relevant organs added manually. It uses a lower-cost haptic feedback device to promote affordability. Learning effects of this simulation have yet to be quantified. The Innovation Center for Computer Assisted Surgery group (ICCAS) at the Medical Faculty of the University of Leipzig, Germany, is also developing a virtual reality functional ES3 and a transphenoidal pituitary surgery simulator. The ICCAS projects generally place a stronger emphasis on clinical applications (ie, preoperative planning and intraoperative guidance) rather than training. Another group at Colombia is using “telesimulation” to create a virtual reality tool aiding trainees in resource-limited countries to gain skills to perform functional endoscopic sinus surgery. Their project, the Web Environment for Surgery Skills Training on Otolaryngology, uses an Internet-based educational cycle that simulates the stages of a real procedure. This system still requires work in its development, but represents a valuable concept of telesimulation to promote distance learning in disadvantaged countries. Researchers at Stanford have recently implemented a sinus surgery simulation environment that introduces automatically derived data sets from preoperative, patient-specific imaging.
Simulation of temporal bone surgery
Traditional Media in the Otologic Curriculum
To date, temporal bone surgery has been learned through contemporary media: textbooks and atlases, illustrations, CD-ROMS, models, and cadaver dissections. Although CD-ROMs provided a cost-effective solution through the integration of photographs, illustrations, movies, computer graphics, and tomographic images, the solutions from interaction remain predetermined and provide limited, if any, task fidelity (ie, correlation between the training and performance environment). Selections are from a limited number of choices, are schematic in representation, and the results are not unique to the individual.
It is difficult to achieve a consummate comprehension of the subtle spatial relationships required for temporal bone surgery without diligent studies through dissection over a period of 4 to 5 years. To facilitate understanding of the intricacies of the regional anatomy of the temporal bone, some authors have presented techniques for tissue preservation and display. The resulting displays provide the student with only limited and passive means to study the intricate relationships of structures found in the temporal bone. In addition, the physical limitation of the material, associated risk of infection (HIV, hepatitis B and C), exposure to formalin, and decreasing availability make this method increasingly problematic. To increase the availability of material and reduce the risks of infection, Pettigrew introduced a plastic model of the otic capsule and middle ear for drilling practice. These models are structurally realistic and serve as substitute materials for drilling practice. However, plastic models are subject to similar physical limitations as cadaver specimens (ie, to start over, a new plastic specimen must be used), and provide a limited force correlate (ie, biofidelity) to actual bone because of the homogeneity of the plastics.
Use of Computer Simulations in Histopathologic and Morphologic Studies
Three-dimensional reconstructions from CT have been extensively integrated with computer-aided design techniques to provide a non–real-time system for use in the diagnosis and surgical planning of craniofacial disorders. Methods for characterizing the morphology and histopathology of the temporal bone soon followed. Subsequently, these methods were combined with computer-aided reconstruction techniques for visualization and morphometric analysis. Although specimen preparation and integration of the photomicrographs through manual methods was time intensive, the advantage of three-dimensional reconstructions to demonstrate subtle morphologic relationships and changes clearly became evident. More recently, several groups have used photomicrographs to create elegant data sets of the human temporal bone. Although providing exquisite detail and near natural coloration, these preparations take considerable time and effort, and consequently provide limited variance.
Harada and coworkers first introduced the concept of exploiting three-dimensional volumetric reconstructions from CT for emulating drilling and exposing the intricate regional anatomy of the temporal bone. However, because of the computational overhead of volumetric representation at the time, real-time interactions were unavailable. Subsequently, surface-based (isosurfaces) approaches were predominantly used for modeling structure to exploit the hardware-accelerated surface rendering techniques that have been developed for other applications, such as video gaming. Similar techniques using reconstructions from histologic sections to derive isosurfaces have focused on clarifying the spatial relationships of the regional anatomy. Stereo presentations of surface-based models acquired from the Visible Human Project have been presented for transpetrosal, retrosigmoid, and middle fossa approaches to the cerebropontine angle. Although surface-based representations of soft tissues and bone structures have been developed, the systems do not provide haptic feedback to the user, and only provide a schematic emulation of dissection and surgical technique. The development of stereo volumetric, physically based simulations has been presented. These systems do not support real-time viewing and aural simulation. Although volumetric data sets have been integrated, no multiscale data (ie, data acquired at multiple scales) has been reported. Agus has extensively explored efforts to present secondary characteristics, such as bleeding, debris formation, and fluid flow simulation. Ray-casting techniques have been developed to provide stereo simulations of cutting and drilling based on reconstructions from CT. Although all investigators present initial enthusiasm from local surgeons and residents, local or extensive multiinstitutional studies have not been conducted to validate the efficacy of these systems compared with traditional methods of training and assessment.
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
