A Randomized Trial of the “Preoperative Warm-up” Effect, Including Improvement in Operating Room Performance
Kanav Kahol, PhD, Richard M. Satava, MD, Aaron Ashby, MS, Marshall L. Smith, MD, PhD, John Ferrara, MD
INTRODUCTION
The Oxford English Dictionary defines warm-up as “The act or process of ‘warming up’ for a contest, etc., by light exercise or practice.” More generically, warm-up is “The act or process of raising the temperature of an engine, electrical appliance, etc., to a level high enough for efficient working,” which captures the concept of activating and preparing to a high level of efficiency before beginning an activity. In sports and the sports training literature, several published papers and books highlight the importance of the warm-up in improving performance and avoiding errors. The stretching hand-book published by the Stretching Institute1 captures the fundamental principles behind the benefit of warming up:
Warming up prior to any physical activity does a number of beneficial things, but primarily its main purpose is to prepare the body and mind for more strenuous activity…it’s important to start with the easiest and most gentle activity first, building upon each part with more energetic activities, until the body is at a physical and mental peak. This is the state in which the body is most prepared for the physical activity to come.
While the positive effect of a warm-up on strenuous physical activity is well known, research has also shown positive effects of a warm-up on cognitive skills2 in physical sports. Research has also shown that subjects tend to perform better at cognitive exercises in follow-up tasks. The overall effect is seen as a cognitive arousal phenomenon that enables subjects to more fully concentrate their resources on the task at hand. As such, cognitive arousal has been shown to cause increased somatic and cortical activity3 and may even aid in resisting sleep. In addition, cognitive arousal also activates the “attention center” of the brain,3 helping focus attention on the task at hand, as well as initiating the process that controls automating simple psychomotor skills and tasks. The effect of short- term practice is not only limited to sports or physical activity. High-stakes, high- performance professions with significant psychomotor skills proficiency, such as dancing, playing a musical instrument, being a sculptor or painter, all have for centuries used short-term practice or warm-up as a method of getting ready for the task at hand.
The common thread running through all the activities that promote use of the warm-up is:
- Strenuous physical activity;
- Focused mental activity with requirements of cognitive arousal;
- The ability to perform both (a) and (b) within required coordination and task performance constraints.
While the warm-up has a documented positive effect on activities that involve the above- mentioned requirements, it is also known to be necessary to avoid deleterious consequences, such as injury to self and others in the environment, and to trigger cognitive arousal that can significantly enhance performance ability.
Modern day surgery certainly qualifies as a high-stakes, expertise-driven activity.4 The advent of minimally invasive surgery, while being revolutionary for patients and the operating room, poses specific challenges for surgeons and residents. It requires surgeons to conduct procedures with hard to manipulate tools that impose undesirable constraints on the movements of a surgeon. While the advent of robotic surgery and the development of more intuitive manipulators address some of these issues, modern day surgery, and laparoscopic surgery in particular, still requires considerable additional cognitive and psychomotor proficiency. There is significant research showing that surgery requires both strenuous mental and cognitive activity.5 In a seminal article by Hall et al5 published in British Journal of Surgery, the link between surgical proficiency and cognitive processes was investigated. The article outlines a key factor that has been the basis of surgical curriculums and teaching practices. Surgery is not only viewed as a mechanical task- based skill. The article states that “surgical competence combines the intellectual exercise of decision making with the ability to perform mechanical tasks.”5 Today, the fact that it is impossible to be a competent surgeon without using higher order cognitive skills is widely accepted. Spencer6 hypothesized that 75% of the important events in an operation are related to making decisions and only 25% to manual skill. Sir Frederick Treves wrote in 1891, “The actual manipulative part of surgery requires no very great skill and many an artisan shows infinitely more adeptness in his daily work. . .It is in the mental processes involved in an operation that not a few fail. There is some lack in. . .the capacity for forming a ready judgment, which must follow each movement of the surgeon’s scalpel.”7
These factors raise many concerns for the medical community including surgical education and training requirements, objective proficiency assessment measurement, effective feedback mechanisms, and eventually issues of certification. A considerable amount of research has been performed to address these issues. Surgical simulation has emerged at the forefront of technologies and educational methodologies that can augment and/or replace existing methods of training.8 Objective proficiency metrics have been developed that measure hand movement and tool movement during procedures to analyze surgical proficiency.9 Feedback mechanisms involving adult education principles, didactic training, and novel user interfaces with information visualization have been proposed.10,11
Training methodologies and tools emerging from these efforts have significantly affected the future of surgical practice and are centered on treating surgery and surgical practice as a skilled exercise with high performance requirements. This raises an important question that if surgery involves strenuous physical and mental activity, then would a “preoperative warm-up” (or simply “warm-up” for the purposes of this manuscript) activity that involves surgeons performing surgical exercises before the main task improve surgical performance in the main task at hand? This research embarks on defining a systematic methodology to study the effect of short-term pretrial practice on surgical proficiency. We use simulation tasks targeted to hone both psychomotor and cognitive skills, to provide warm-up exercises that will enhance the performance of the main surgical procedures.
The concept of a preoperative warm-up in surgery is largely facilitated by the advent of surgical simulation. Today, most surgical education programs include simulators and simulation laboratory experience as a part of the medical curriculum. The Residency Review Committee of the Accreditation Council on Graduate Medical Education has mandated the use of simulation-based training as a required component in surgical residency programs. Simulators are increasingly becoming affordable, and it is not far in the future that we could place a simple practicing simulator in every preoperative area for warm-up before the surgeon performs the procedure on a patient. It is hence logical to use surgical simulators for providing warm-up exercises. Furthermore, several studies12-14 have shown the face, content, concurrent, and construct validity of using simulators for analyzing surgical proficiency and simulation environments that closely mimic realistic work requirements. Hence, evaluating user’s proficiency in a simulation environment will serve as an effective design as a first phase. In the first phase, if the simulation-based evaluation showed significant promise then in phase II of the experiment we would test the benefit of simulation in the actual OR.
Phase I: Evaluation of the Effect of Warm-up in Simulation Environments
1.1 Hypotheses.
Several key questions need to be addressed to study the effect of warm-up on surgical proficiency and present a case for including preoperative warm-up as an advantageous activity for surgeons. Some of these key questions and hypotheses are as follows:
1) What effect does warm-up have on overall proficiency of a surgeon? The key to including warm-up exercises for surgeons resides in understanding its effects on the subsequent tasks. A methodology and metrics need to be defined to measure proficiency during both warm-up and follow-up tasks to analyze the effect on proficiency. Proficiency measures should preferably be cross compatible and not depend specifically on the task but focus on generic cognitive and psychomotor skills.
2) Hypothesis 1: Warm-up is not dependent on the level of experience of a surgeon. Do warm-up requirements vary with level of experience of a surgeon? All the possible outcomes from studies designed to test this hypothesis would provide important cues in the relevance of warm-up in laparoscopic surgery, and address the question as towhether there are important physical (psychomotor) and cognitive (arousal, alertness, working memory, and other) functions that can always be “activated” by a short warm-up, regardless of the experience or fund of knowledge of a surgeon.
3) Hypothesis 2: Warm-up and its intensity do not vary with short-term practice. It could be hypothesized that warm-up is required only during training stages, and once a necessary level of learning has occurred for a particular task, there is no further requirement for warm-up. In essence, warm-up is “learned’ and does not provide benefit after an initial time frame. To test this hypothesis, experiments that track the performance of subjects over several sessions are required. It is a priori that no matter how experienced a surgeon might be, there is value added to performing a warm-up before every surgical procedure.
4) Hypothesis 3: Warm-up is significantly effective in reducing errors due to fatigue. Research has shown that fatigue due to night call and sleep deprivation can lead to decreased surgical proficiency.15 However, whether warm-up would help in alleviating some of the effects of fatigue is an intriguing question with many pragmatic and positive implications.
5) Hypothesis 4: Warm-up affects both cognitive and psychomotor skills. An important research question lies in understanding the effect of warm-up on both psychomotor skills, such as accuracy, movement smoothness, as well as on cognitive skills, such as attention, memory, intermodal coordination abilities, and others. In related fields, such as sports and dance, it has been established that warm-up leads to both cognitive arousal as well as movement proficiency.2,16 Whether such effects are seen in surgical tasks may be critical in increasing patient care and safety, and the possible mechanism for performance improvement (memory, attention, psychomotor coordination, and others) needs to be elucidated.
6) Hypothesis 5: The effectiveness of warm-up is independent of the follow-up task to be performed? From a scientific point of view, this is an important investigation that will reveal whether effects produced by generic practice and exercises can carry over to more complex tasks; from a pragmatic standpoint, this study will reveal whether a single “standard” warm-up is sufficient, or will multiple different types of warm-up sessions need to be developed.
Some previous work has been reported to address a few of these issues. Do et al17 used a laparoscopic “box trainer” simulator, to study the effect of warm-up exercises on follow- up tasks. The study involved 3 OB/GYN residents from 4 postgraduate year groups and 12 medical students to study the effect of warm-up on different experience levels. The participants performed a task involving grasping pill-like plastic elements from a Petri dish and placing them into a bud vase. A total of 30 pills were required to be transferred. The time elapsed to complete the task, and the number of pills successfully transferred to the bud vase was used as a measure of performance. The groups performed the warm-up task and then performed follow-up tasks after a 5-minute respite. It may be noted that the same task was used for warm-up and follow-up exercises. The independent sample t test was used to study the difference in performance between prewarm-up and follow-up conditions. Overall laparoscopic performance was significantly improved for both residents (all years combined), and the medical student control group (P<0.0001). The overall performance of the residents (all PGY) improved by 25%. Likewise, the medical students’ performance was improved by 29.3%. The study showed encouraging results of warm-up being used prior to performance of the main exercise. However, the study has certain limitations. Although it does provide statistically significant data to indicate the overall effect of warm-up on surgical proficiency and studies the difference of the effect of warm-up on groups with different experience levels, to a certain extent it fails to distinguish between learning effect and warm-up effect. The increase in proficiency may have occurred due to subjects learning and adapting to the task. To isolate the effect of warm-up on surgical proficiency from the improvement in psychomotor activation and cognitive arousal, it is important to assimilate data over multiple trials with sufficient iterations to allow learning to occur in order to account for the “learning curve.” Also, their study does not include sophisticated measures of surgical proficiency like hand movement analysis and tool movement analysis that can quantify psychomotor proficiency in detail. Likewise, the tasks used in their experiment did not include a cognitive dimension thereby not allowing for studying the effect of warm-up on cognitive arousal. Finally, because the study used the same exercise as the warm-up and follow-up exercise, it does not sufficiently indicate that the increase in proficiency will carry over to other tasks, especially those with more clinical relevance.
1.2 Definitions
Attention: Although several definitions are available for attention, this chapter focuses on the definition provided by Mesulam, who defined attention as “preferential allocation of limited processing resources and response channels to events that have become behaviorally relevant.”18 This comprehensive definition encompasses the behavioral aspects as well as the neural aspects of attention. This definition has been shown to suitably explain various attentional effects.
Automaticity: Posner and Snyder 19 proposed 3 criteria for a process to be purely automatic. The process occurs (a) involuntarily, (b) without giving rise to conscious awareness, and (c) without producing interference with ongoing mental activity (not using limited capacity resources of the organism). The authors noted that it is possible to show that many complex but habitual processes can operate automatically. Automaticity is measured in experimental paradigms through a pop-up effect that allows procedures to be accomplished in a limited amount of response time. Posner and Snyder’s seminal work focused on automaticity as an emergent property of long-term learning. However Shiffrin and Schneider20 showed that with training, subjects can obtain automaticity in processing inputs and accomplishing tasks with small response times. A key element in training that led to emergence of automaticity resided in a constant condition wherein the learned task had a consistent logical framework that could be learned with practice. In the absence of such a condition, learning did not lead to automaticity. The phenomenon of chunking wherein discrete sub tasks are combined to generate an automatic continuous response has been associated with automaticity and learning.21
It is also important to note that recent theories have shown that automaticity does not equate to unconscious processing. Research has shown that automaticity is obtained gradually through learning and is subject to attentional control. Further, once a procedure obtains automaticity, it is difficult to suppress by simple environmental cues. Such environmental cues, when presented to the subjects in a test phase, trigger the learned automatic processing, which is then difficult to suppress. Warm-up activities may well provide the necessary environmental cues in addition to allowing selective attention to focus on such cues, triggering automatic fast processing, while preparing the subject for achieving high performance during follow-up tasks.
Cognitive Arousal: Cognitive arousal can be defined as chronological change in condition, raising the organism from a lower to higher level of sensory, perceptual, and cognitive excitation. It may be noted that cognitive arousal can have both positive and negative effects. Excessive arousal can often lead to deleterious effects like insomnia22 or reduced performance. However, appropriate arousal can significantly improve performance.2 Arousal can be measured at a behavioral level through specially designed tests and arousability scales.2 It can also be measured at a neural level through various mechanisms, such as EEG, MEG, or fMRI. The onset of cognitive arousal is generally seen as a function of selective attention. Being a goal-driven activity, selective attention is inherently related to warm-up like activities that allow for subjects to focus on the task at hand, concentrate on environmental and proprioceptive cues, and achieve required levels of performance in follow-up tasks.
Intermodal Coordination, Intermodal Transfer: In everyday life as well as in specialized environments, exchanges between individuals and environment are multimodal. A key perceptual process lies in the human ability to perceive varied perceptual information from different modalities and coordinate it into unified wholes to produce adequate responses to each situation. The act or the ability of humans to coordinate information from various modalities to produce unified perceptual experiences is known as intermodal coordination.23 In intermodal transfer, information provided by one modality is used or processed by another modality. An everyday life example of intermodal transfer lies in predicting haptic characteristics of an object by only viewing the object. Humans are adept at transferring information learned in one modality (like vision) to another modality (like haptics).23 In surgery, the ability to transpose knowledge and information from visual stimuli to haptic sensations is a key skill. Surgeons have to often rely on experience to interpret visual stimuli from a camera and decipher tactile property of organs. There is no reported literature on warm-up and its ability to enhance basic functions like intermodal transfer and coordination. However, including tasks that specifically target intermodal coordination and transfer skill may have a beneficial effect on surgical proficiency and help elucidate the mechanisms of warm-up for cognitive and psychomotor arousal.
Working Memory: Working memory may be defined as the system for the temporary maintenance and manipulation of information, necessary for the performance of such complex cognitive tasks as comprehension, learning, and reasoning.24 Many researchers have explored working memory capacity. G. A. Miller25 in a seminal paper summarized evidence that people can remember about 7 chunks in short-term memory (STM) tasks. Since that paper, however, considerable debate in the psychological and neurological literature has led to the consensus that that capacity limit is smaller (Cowan26 in a summary article put the number at 4 with all other performance constraints accounted). The most basic working memory test is known as the digit span test (DST) that involves subjects perceiving a sequence of characters and recalling the sequence after a delay.27
Using this test and its variations, researchers have shown that working memory capacity can vary according to level of expertise21 and level of fatigue.28
Sensorimotor coordination: Activities that involve motor action following multimodal sensing fall under the realm of sensorimotor actions. Sensorimotor coordination is the ability of organisms to respond appropriately to sensory perception through motor actuation. For skilled tasks, such as surgery, dance, sports, and others, obtaining efficient sensorimotor coordination is central to good performance.2,29 Sensorimotor coordination is measured through a variety of tests that may include activities like dietetic gestures, grasping, tracking, and others. The relation between improved sensorimotor coordination and warm-up is well established in the sports literature.2,16,17 The report by Do et al17 on warm-up for laparoscopic skill involved tasks that are primarily sensorimotor in nature. Their results showed significant improvement in common laparoscopic skills with warm- up that indicates that sensorimotor coordination might improve in surgical tasks. These results are consistent with improvements noted in other task domains.
2. METHODS
Before any experiments were conducted, prior approval was obtained from the Institutional Review Board at Banner Good Samaritan Medical Center.
The present study focuses on a series of experiments designed to systematically explore the effect of warm-up. The experimental design extends the methodology proposed by Do et al,17 by using exercises in warm-up condition and follow-up condition. Subjects across varying specialties and experience levels (including senior trauma surgeons) were involved in multiple sessions performing exercises. These sessions were held in both precall and postcall conditions, allowing for study of warm-up in alleviating the effect of night call on both cognitive and psychomotor fatigue. In an additional experiment, different exercises were used for warm-up and follow-up tasks. This allowed quantification of carry-over effect (generalization) of warm-up exercises. The study design, controls, and others are discussed in sections below.
We used a specially designed simulator for our studies. Conventional simulators are designed to hone and measure procedural and psychomotor skills. The present study warranted measurement of general laparoscopic and psychomotor skills, as well as cognitive skills that included attention, working memory, intermodal transfer, tracking, and orientation. These terms that have been previously defined above have different connotations in different domains and in the next subsection will be clarified as to the goals of the tasks and their design. Further, we elaborate on interaction mechanisms between these basic processes that may enable a neuro-cognitive view of skills involved in modern day surgery.
2.1 Task Definition
A simulation was designed for a variety of virtual ring transfer tasks that served as warm- up and follow-up exercises. In the virtual ring transfer task [shown in Figure 1 (a)], residents were tasked with grasping a series of a virtual rings and placing each on randomly highlighted pegs on a board. The simulation was implemented by using the Sensable haptic joystick. The Sensable haptic joystick allows for generation of 3 degrees of force feedback in response to events in the virtual environment. OpenHL programming API was used to design the simulation. The simulation allows for measurement of tool tip in the virtual environment. Additionally, the sessions could be played back in the software with traces (path) of the tool tip movement being shown at various speeds, allowing for visual analysis of movement.
2.2 Psychomotor Skills Tasks
2.2.1 Sensorimotor coordination exercise: This basic task involved 10 rings (Figure 1). After the participant places a ring on a highlighted peg, another peg is randomly highlighted for the participant to put the ring on it. This is repeated till all the 10 rings are correctly placed. The time taken for completing the task is displayed on the bottom middle part of the screen for the participant to follow. This basic ring transfer task is a psychomotor task used in many simulators to hone tool manipulation skills. A cognitive error is marked for every time a ring is placed on the wrong (non-highlighted) peg. It may be noted that the simulation does not allow placement of the ring on a wrong peg, and the user must continue until the correct peg is chosen. In the basic simulation, the number of cognitive errors in practicality is very low, because the simulation is not intended to challenge attentional or memory resources. It is primarily a sensorimotor coordination task that involves visual perception and motor actions, and errors only occur during the habituation phase, wherein subjects are familiarizing with the apparatus and task. Sensorimotor coordination in this task is primarily measured through movement smoothness of the surgeons’ hands as well as the tool movement smoothness. This test is inspired from the Purdue Pegboard Test, which measures hand movement dexterity.33
2.3 Cognitive Skills Tasks
For the purposes of this study, it was necessary to design novel tasks that require both cognitive and psychomotor skill. Although both psychomotor skills and cognitive skills are involved, the tasks were designed so the main emphasis is on the cognitive component. Note also that there is an assumption that the psychomotor skill (sensorimotor) above has been partially if not completely automated. Thus, the basic ring transfer task was modified based on cognitive psychology methodologies for suitable tasks.23
2.3.1 Slow 2-Dimensional Tracking Task: In addition to the basic ring transfer task, 3 variations of the task were designed to target orientation. In the sensorimotor coordination exercise, the ring is stationary and can be grasped by picking it from a fixed location. To include more manipulations and greater range of motion, a variation of the basic simulation was designed where the ring moved slowly (approximately 25 pixels per second on a 1124×1048 pixel display) in the environment. This variation required the participants to track the ring movement and pick it while it is in motion. In this variation, the ring moved in the plane of the pegboard. This task is inspired by visio-motor tracking tasks that involve temporal sensorimotor coordination recruiting attentional resources, working memory resources, and motor planning.27
2.3.2 Fast 3-Dimensional Tracking Task: In this variation, the ring was allowed to move anywhere in a 3D environment at a faster rate (approximately 50 pixels per second on a 1124×1048 pixel display). This task requires more sophisticated tracking in 3 dimensions with the ring in a variable orientation.
2.3.3 Orientation Task: Complex motor behavior is composed of serial movements. To plan serial movements, serial sensory information is often transiently stored in working memory and converted to a movement program with the help of multisensory stimuli.30 The ability to plan serial movements to execute complex movements is an important basic skill for modern day surgery. We designed a task inspired by the Grooved Pegboard task,31 which is a variation of the Purdue Pegboard Task. The grooved pegboard task is like the original pegboard task that requires placing rings on pegs, but in the Grooved Pegboard task, rings are grooved to fit the pegs only in a particular orientation.
In this variation of the pegboard task, the entire pegboard moves slowly, creating different orientations in which the surgeons need to accomplish the task. These tasks trained the user to accomplish the tasks in different orientations; this is similar to a common requirement in actual surgical procedures where the tissues or organs may actually be slightly moving. In addition, it also allowed for capture of rotation movements of the wrist. These movements and assignments form the orientation tasks. The orientation tasks increase the cognitive load required to complete the task including working memory usage for motor responses.27 Users have to divide their resources between tracking rings and pegboard and placing them on the correct peg in a continuously changing orientation.
2.3.4 Preparatory Attention Task: As shown in Figure 1(a), a peg is highlighted to indicate to the user where to place the ring. In the primary sensorimotor orientation, the peg is lit up throughout the simulation, allowing the user to focus solely on the psychomotor skills. In order to make this an attentional task, a randomly chosen peg is highlighted only for 500ms after which the highlight is removed. This task requires the user to pay attention to the peg that lights up and then execute the ring transfer to the This task requires users to pick visual cues to identify the correct peg and then execute the motion, engaging preparatory attentional circuits in addition to recruiting processes for accomplishing the basic sensorimotor coordination task.
2.3.5 Working Memory Task: In the memory task, a sequence of randomly chosen pegs light up instead of a single peg. Each peg lights up for 500ms, requiring the user to memorize the sequence. Two variations are programmed. The first variation presents a sequence of 3 randomly chosen pegs, while the second variation presents a sequence of 4 randomly chosen pegs. This test is inspired by the digit span test,27 which requires users to memorize a sequence of numbers. This test combines working memory recruitment with motor processes to offer a challenging exercise for surgeons. Such variations of the digit span test have been designed for measuring visiospatial attention and working memory in neuropsychological evaluation (the Korsi Block Tapping Test, in Lezak et al Neuropsychological Assessment32). This test reveals that the digit span in such tasks runs one element lower than the conventional digit span task does. We chose to highlight a maximum of 4 pegs based on previous work that suggests 4 items to be the limit of working memory.26 It may be noted that the designed task included sensorimotor functions in addition to working memory. Informal tests with senior surgeons who did not participate in the final study revealed that a span of more than 4 pegs led to a considerable increase in errors in judgment of correct pegs.
2.3.6 Visio-Haptic Transfer: In this task, much like the attentional task, a peg is randomly chosen and highlighted for 500ms with the task being to put a ring on the highlighted peg. However, after the peg is highlighted, the entire pegboard appears and disappears from the visual display every second. During appearance of the pegboard, the user can both see and palpate the entire board and ring. However, during the disappearance of the board from visual stimuli, haptic sensations of the board and ring are still present in the simulation. The user has to now rely solely on the sense of haptics to place the ring on the correct peg. This task requires the user to memorize the spatial location of the pegboard and individual pegs learned from the visual display. To effectively accomplish this task, the information learned from the visual display needs to be transferred to the haptic modality, which allows the user to place the ring on the pegboard even in the absence of visual stimuli. This is a novel task designed to specifically measure and promote intermodal transfer.
Table 1 summarizes the tasks used and the primary psychomotor and cognitive processes they target.
2.4 Objective Proficiency Evaluation Measures
This section on methodology defines the hand movement data capture system and variables, and the tool movement capture system and variables. For measuring laparoscopic proficiency, we used a combination of hand movement and tool movement. Tool movement measured as movement of the tool tip in virtual environment is a validated measure for surgical proficiency.12 Kahol et al33 introduced hand movement measured through Cyberglove and the Polhemus Liberty Tracker as an effective measure of surgical proficiency. Both tool movement and hand movement are representative of economy of motion; overall smoothness in execution and their construct validity has been established. The smoothness of tool movement was calculated by using the following formula:
Tool Movement Smoothness = 1 – normalized (tool acceleration) (1)
Tool acceleration was calculated for the entire duration of a task and normalized in a range of 0 through 1. Smoothness of tool movement as predicted through this measure is 1 when overall acceleration is close to 0. This is generally the case in well-executed motion with controlled accelerations. On the other hand, jerky motions show higher normalized acceleration and hence lower smoothness. Similarly, the hand movement smoothness is calculated using the following formula:
Hand Movement Smoothness = 1 – normalized (wrist acceleration) (2)
The data capture setup shown in Figure 1(b) depicts the wireless Cyberglove glove and the Polhemus Liberty tracker. The wrist acceleration is calculated through tracking of the sensor placed on the wrist. In regards to tool movement smoothness, jerky hand motions lead to less smoothness while controlled movements lead to increased smoothness.
For every simulation exercise the time required to complete a task is also recorded. Further cognitive errors are recorded for every type of exercise quantified by the number of selections of placing the ring on the wrong peg. These 4 objective measures are also supplemented with gesture level proficiency measure. Task decomposition has emerged as a validated method to measure surgical proficiency.33,34In this approach, hand movement or tool movement is decomposed into smaller gestures (such as in, out, grasping, rotation, etc.). Each individual gesture is analyzed and, based on its similarity to an optimal occurrence of a gesture as determined by setting a benchmark criteria from an experts group performance of that gesture, is given a benchmark proficiency rating. For this paper, hand movement was used for task decomposition. The algorithm for this purpose was described by Kahol et al35 and was shown to correlate highly with subjective proficiency ratings obtained by surgeon teachers. The scoring method used is a standard Likert scale between 0 and 10 for an entire exercise. Zero implies least proficiency in accomplishing the task while 10 implies highest proficiency. This measure is estimated through a combination of time elapsed and kinematic analysis of hand motion. These 5 measures (gesture level proficiency, hand movement smoothness, tool movement smoothness, elapsed time, and cognitive errors) provide a broad framework for the global assessment of proficiency.
2.5 Experimental Design
Two experiments were performed. The first experiment consisted of 8 sessions, while the second experiment consisted of a single session of data capture. Experiment 1 was designed to evaluate the overall effect of warm-up on surgical proficiency and its relation to experience level, short-term practice, fatigue level, and cognitive and psychomotor skills. In Experiment 1, all subjects acted as their own control, wherein their performance was measured during warm-up and then during follow-up trials. Experiment 2 was designed to test whether warm-up conducted in basic skills was able to effect performance in a complex surgical task. A control group of subjects who performed a complex task without warming up was compared with an experimental group of subjects who warmed up with simple laparoscopic exercises before performing a complex task.
2.5.1 Experiment 1: Subjects
Forty-six surgeons participated in the study. Fourteen PGY1 OB/GYN and general surgery residents, 10 PGY2 OB/GYN and general surgery residents, 11 PGY3 OB/GYN and general surgery residents, and 10 attending trauma surgeons were involved in the experiments. Nineteen females and 27 males comprised the subject pool, with 21 subjects in the OB/GYN specialty and 25 subjects in the general surgery specialty.
2.5.2 Experiment 1: Design
Each participant was involved in 8 sessions spread over 4 weeks. Four of these sessions were held precall before the residents performed their night call, and the remaining 4 sessions were held postcall. In each session, 3 exercises were performed after participants filled in the fatigue questionnaire. These 3 exercises were randomly chosen from the 8 variations of the ring transfer exercises described in 2.1. Each exercise was repeated 2 times. The first iteration was marked as warm-up trial, while the second iteration was marked as the follow-up exercises. Hence, this experiment permitted an in-subject design wherein every subject was his or her own control. During each session, subjects wore a Cyberglove and the Polhemus Liberty tracker on their dominant hand. In the first session, the glove was calibrated to a participant’s hand, and the calibration was stored. For every session performed by a participant, the calibrated glove was used to accurately record hand movements and wrist movements. Each session lasted approximately 15 minutes to 20 minutes. The exercises in the precall condition and the postcall condition were not matched. This was done to control for learning effect that could have been produced for a particular exercise. During the course of 8 sessions, each subject performed 6 iterations of 8 exercises described in 2.1. Three of these iterations were for warm-up, while the remaining 3 iterations were marked as follow-up condition.
Data captured in the 8 sessions on hand movement, tool movement, time elapsed, and errors were processed as per methodology defined in section 2.2 to calculate the 5 objective proficiency measures for each exercise iteration. Statistical analysis was performed to compare performance of surgeons during warm-up and follow-up exercises. ANOVA was used to study the difference between warm-up and follow-up conditions.
In the first level of analysis, all the trials performed during warm-up were grouped together and compared with all the trials during follow-up grouped. This analysis provided statistical data to study the global differences between the follow-up and warm- up condition and designed toward investigating the question of whether warm-up improves the performance of a subsequent task.
In the second level of analysis, data were grouped according to experience level of the surgeons (PGY1, PGY2, PGY3, and senior surgeons) and the groups’ warm-up, and follow-up exercise iterations were compared using ANOVA. This analysis allowed study of hypothesis #1 about differences in the effect of warm-up on surgical proficiency of the surgeons with different experience levels.
In the third type of analysis, exercise iterations were grouped according to sessions in which they were performed. Overall change in proficiency measures in the follow-up condition and warm-up condition was plotted as a curve across the iterations of exercises and sessions. This graph helped provide insight into hypothesis #2 about whether the need for warm-up alters with short-term learning of surgical skills and exercises spread over 8 sessions.
The fourth type of analysis grouped precall warm-up and follow-up sessions and compared them with postcall warm-up and follow-up sessions. This comparison allowed for studying hypothesis #3 whether warm-up has a positive effect on surgical proficiency during the postcall fatigue state, thus improving the performance of fatigued surgeons.
In the fifth type of analysis, warm-up and follow-up iterations of sensorimotor coordination exercise was compared with the warm-up and follow-up iterations of all the other types of exercises grouped together. Table 1 shows that all exercises except the sensorimotor coordination exercise had a significant cognitive component. This analysis was performed for hypothesis #4 to reveal whether warm-up had a considerable effect on psychomotor skills and cognitive skills and to compare the performance differential.
2.5.3 Experiment 2: Subjects
Eight PGY2 and 4 PGY3 residents (6 OB/GYN residents and 6 general surgery residents) participated in the second experiment designed to study the effect of warm-up on surgical proficiency in complex surgical procedures. The participants were taken from the same pool as in experiment 1, but this experiment was performed after the completion of experiment 1. Both the groups had equal exposure and learning of the ring transfer exercises before they began experiment 2.
2.5.4 Experiment 2: Design
The residents were divided into 2 groups of 6 residents each. The control group performed only the surgical electrical diathermy task on the ProMIS simulator (with no preliminary warm-up exercises). The surgical diathermy task mimics an actual electrosurgery procedure performed on a gallbladder. The experimental group performed the simulated surgical diathermy task after performing warm-up exercises. Three warm- up exercises were randomly chosen from the 8 exercises defined in 2.1 for warm-up.
Each exercise was repeated twice by the experimental group. The surgeons wore Cyberglove and Ascension Tracker while performing the diathermy simulation. The tool movement was available from ProMIS simulator. However, no direct cognitive errors could be recorded in the ProMIS simulators, because it does not allow for any such recording. This analysis was performed for hypothesis #5 to determine whether warm-up improves performance independent of the type of follow-up task (or procedure). The performance of the control group and experimental group was compared on the diathermy simulation task by using ANOVA.
3. Results
3.1 What effect does warm-up have on overall proficiency?
Figure 2 shows the plot for comparison of the objective proficiency measures evaluated in the experiment in warm-up and follow-up condition. Each of the measures showed a statistically significant difference between the warm-up and the follow-up condition (P<0.005). The time required for completion of tasks as well as number of cognitive errors decreased while the gesture level proficiency, hand movement smoothness, and tool movement smoothness increased significantly. Overall reduction in errors was 33%.
3.2 Hypothesis 1: Warm-up is not dependent on the level of experience of a surgeon.
Figure 3 shows plots of the 5 objective proficiency measures, grouped according to level of experience of the participants.
The plots show that before warm-up, a statistically significant correlation did not appear between the groups. However in the follow-up condition, the groups show a statistically significant difference (P<0.001). Consistent with the results of overall analysis, the time required for completion of tasks and the number of cognitive errors decreased while the gesture level proficiency, hand movement smoothness, and tool movement smoothness increased significantly. The percentage reduction in errors was calculated for each of the groups. PGY1, PGY2, PGY3, and senior surgeons showed a 31%, 30%, 44%, and 29% reduction in errors. These reductions show that each subject group showed consistent and significant improvement supporting the hypothesis that warm-up is needed for subjects independent of their level of expertise. An interesting observation is that following warm- up, the gesture proficiency and hand movement smoothness of all the residents actually exceeded that of the surgeons initial score before warm-up; however after warm-up, the surgeons greatly exceeded all the residents.
3.3. Hypothesis 2: Warm-up and its intensity do not vary with short-term practice.
Figure 4shows the improvement in objective proficiency in 3 sessions of the exercises. Data were grouped in terms of each session of the exercise performance, and the improvement in proficiency was calculated for each session. Figure 4 shows that the improvement in proficiency introduced due to warm-up is shown in all 3 sessions consistently and does not increase or decrease with short-term learning. Only gesture proficiency showed a statistically significant difference between the 3 sets of exercise performance; however, there was not a continuous improvement (eg, learning curve), but rather simply differences in amount of improvement from one set to the next.
3.4 Hypothesis 3: Warm-up is significantly effective in reducing errors due to fatigue.
Figure 5 shows plots of objective proficiency measures in warm-up and follow-up conditions with data being grouped into sessions captured precall and sessions captured postcall. The data show that warm-up does positively affect proficiency during postcall condition characterized by fatigue and sleep deprivation36 to a point where the warm-up returns performance in a fatigued subject to the same level as that of a precall subject who has not warmed-up. The proficiency obtained in the postcall fatigue follow-up condition is less than the proficiency measures obtained in the pre-call follow-up group. Hence, these results indicate that warm-up does help in countering some of the effect of fatigue and sleep deprivation by returning performance in the fatigued subject to the equivalent of a non-fatigue baseline (with no warm-up). However, it appears that fatigue prevents enhancing beyond baseline performance of a fresh (precall) subject. This interesting phenomenon deserves further, more deliberate investigations than these preliminary trials. It is important to emphasize that even in a fatigued subject, every single parameter improved (hand motion smoothness improved 198%, tool motion improved 23.8%, gesture proficiency improved 103%, time decreased by 32%, and errors decreased 34%), demonstrating that significant performance improvement can be achieved even in a fatigued individual, and that the improvement can return performance to approximately that of the baseline performance of a rested individual who has not warmed up.
3.5 Hypothesis 4: Warm-up affects both cognitive and psychomotor skills.
Figures 6 and 7 show respectively the plots of objective proficiency measures (means) in warm-up and follow-up conditions for the sensorimotor coordination (psychomotor skills) exercise as well as the cognitive exercises (grouped together), while Figure 8 shows t-statistic measure for gesture proficiency and cognitive error for each exercise separately. The t-statistic value represents the change in objective proficiency in the warm-up and follow-up condition and shows how warm-up affects skills associated with each of the exercises separately. The highest value was noted for orientation exercise while the lowest values were noted for the memory 2 exercise. These results support the hypothesis that warm-up does lead to improvement in both types of skills.
3.6 Hypothesis 5: The effectiveness of warm-up is independent of the follow-up task to be performed
Figure 9 shows the objective proficiency measures (except cognitive errors) for the 2 groups involved in experiment 2. The control group that did not warm-up before performing the diathermy task on the ProMIS simulator achieved inferior proficiency scores compared with the experimental group. The experimental group that warmed up with the exercises designed for this experiment showed statistically significant better performance (P<0.01) except for time elapsed where P<0.08 was noted. This demonstrates at least preliminarily that simple warm-up tasks can be generalized for more than one type of procedure; however, this trial is quite limited and needs much further exploration.
4. Discussion
The results of this study provide substantive evidence that short-term practice (warm-up) with exercises designed to target both psychomotor and cognitive skills that are involved in surgical procedures can greatly enhance skill proficiencies during a follow-up procedure. Gesture level proficiency, movement smoothness, and tool movement smoothness are favorably enhanced with simple practice. Errors show a significant reduction, which is arguably the most important positive effect of warm-up exercises.
Time required for completion of a task, which is one important indicator of efficiency, is also reduced significantly.
Participants of varying experience levels all can benefit from warm-up. It is important to note that before warm-up there was no statistically significant difference among the different levels of experience, (although there was a tendency for the more senior levels to have a slightly higher, though not statistically significant, baseline performance capability in the hand and tool movement tasks). After the warm-up exercises, there are significant differences within each of the expertise levels between the warm-up and the follow-up tasks, with approximately equal improvement for all levels of expertise. Since the improvement is so strong among all levels (25% to 40% reduction in error), it is fair to conclude that warm-up provides a quantifiable performance improvement for subsequent tasks, regardless of experience level or expertise. What cannot be concluded is whether improved task performance will transfer to improvement in the operation room (VR to OR), although this has preliminary evidence in the final task that demonstrates the generalizability of preoperative warm-up to a different (electrocautery) task, and more important whether improvement in the operating room will result in improved overall clinical outcomes.
Another possible explanation for the favorable effect of warm-up exercises on follow-up exercises could be short-term familiarization effect. The key element in clarifying such an explanation requires that warm-up exercises and follow-up exercises are exactly the same. When the same task is performed before and after warm-up, the results unequivocally demonstrate improvement. The results also show that the group that warmed up with simple exercises prior to a complete procedure performed significantly better than the control group that directly performed the task. Because improvement was demonstrated in both the same follow-up task and a different follow-up task, the improvement is NOT due to familiarization or the learning curve, rather the improvement supports the generalizability of the chosen tasks. In a similar fashion, the results demonstrate that the immediate onset (first trial) of improvement from the warm-up exercises implies that this rapid improvement in skill cannot be attributed to the familiarization effect or the learning effect.
The analysis of data collected during precall and postcall conditions indicated that warm- up can improve proficiency in the fatigued individual and can bring their performance to a level equivalent to their baseline nonfatigued condition. However, warm-up cannot take a fatigued person and improve their performance to their potential best performance (i.e., follow-up performance after warm-up in nonfatigued state).
In trying to understand the reasons for improved performance, analysis of the portfolio of tests demonstrates that warm-up exercises enhance performance not only through basic psychomotor skills (sensorimotor coordination), but also by intensifying cognitive faculties. Previous research suggests that 4 concurrent tasks may be close to the limit of working memory26 beyond which skill deteriorates; this was also suggested by the above studies that demonstrated a decrease in performance when the subject was required to remember 4 (rather than 3) sequential pegs. Hence, the improvement caused in proficiency may be only due to improvements in psychomotor skill (sensorimotor coordination) and some cognitive skills, but not due to expanding or enhancing the available working memory.
Further analysis indicates that warm-up exercises may allow for surgeons to achieve cognitive arousal coupled with exercises invoking motor circuits in the brain and warming up of hand and shoulder muscles to achieve sensorimotor coordination. This is based on the improvements shown in all types of exercises including the ones with a significant cognitive component. This hypothesis is consistent with findings in the sports domain16 as related to warm-up. Such effect lasts until a follow-up task is completed.
Warm-up decrement has been shown to occur in the sports domain when a period of rest is introduced between exercises and the task. Future studies will investigate whether such effects are also a factor in laparoscopic surgery.
From a very pragmatic standpoint, it is encouraging to note that improvement in performance with warm-up is independent of the type of task that follows (experiments 1 and 2). Should further research confirm these preliminary results, it will be possible to develop a few, simple preoperative warm-up exercises that will be sufficient for whatever procedure will follow.
While the focus of the study has been the effect (both cognitive and psychomotor) of a preoperative warm-up caused by physical stimulation (i.e., the performance of a specific task), there is a possible confound in the study– the psychological anticipation to perform a surgical procedure. This confounding factor may be that surgeons do not require such preoperative warm-up when conducting actual surgery, because the environment and expectation leads to required cognitive arousal, tuning of sensorimotor coordination and preparatory attention. Further studies need to be designed to address the issue of psychological anticipation and formed the basis of phase II of this study.
Phase II Validation of the effect of warm-up in the OR
Experiments in phase I of the trials show that warm-up significantly improves performance in the simulated environments. Upon these conclusions, the next logical step was to enter the operating room for actual study of surgeons performing preoperative exercises for determination of the transferability of the laboratory findings into the setting that most matters. A table was constructed to highlight the major environmental differences between this and the previous study (Table 2). These differences enforced a number of restrictions also described in this table and accounted for in the experimental design section. The decision was made to confine the experimentation to a single laparoscopic surgical procedure, removal of the gallbladder (GB), due to its suitability within the study’s confines of type, time, sample size, frequency, patient safety, accessibility, and participation pool.
Methods
A participant pool of 40 surgeons from Banner Good Samaritan Hospital in Phoenix, Arizona, was recruited for this investigation. These had a wide distribution of expertise levels ranging from the first year of medical residency after medical school and seasoned “attending” surgeons. This pool was randomly broken up into 2 groups, an experimental and a control, while matching for expertise in each group. Members of the former performed an approximately 10-minute series of exercises on a mobile, virtual-reality laparoscopic trainer before surgery. The control group would keep to its typical presurgical routine. Over the course of 3 months, video was collected from 17 surgeries, 2 of which were later excluded due to data issues. Nine surgeons participated. The surgeries were all of the same type, namely laparoscopic cholecystectomies. The participant and date of surgery depended on the diagnosis of need for that procedure and the normal scheduling strategies used at that location.
Warm-up was performed by the experimental group using the apparatus shown in Figure 1. To allow for warm-up within the OR, we put the setup on a rolling cart as shown in Figure 10. After the warm-up was performed on the simulator for 2 repetitions of 3 randomly chosen variations of the ring transfer game, the surgeons in the experimental group performed the surgery. We mandated that warm-up should be for a minimum of 8 minutes and maximum of 10 minutes based on findings of phase I. Those surgeries were video recorded. Each surgery was recorded from 2 different cameras–the endoscopic camera within the patient’s abdominal cavity and an external camera focused on the surgical field to capture hand movement. The 2 videos of each were then merged into a single split-screen video for rating by an expert physician. See Figure 11 for a sample of the video screen used for rating. The videos were broken up into three 5- to 7-minute clips of the most relevant portions of the surgery. Those areas are in chronological order– dissection of the Triangle of Calot, clipping the cystic artery and ducts, and electrocautery separation of the gallbladder from the liver. Three surgeons provided the ratings of the surgery in terms of smoothness, efficiency and accuracy in a blinded fashion. We used pair-wise correlation coefficient between raters as a measure of interrater reliability of at least P>80. Further, to compare the performance of the groups, we used 2-way unbalanced ANOVA.
Results
The raters scored 70 individual surgical clips on a scale of 1 to 5 described in the rating schema section. Noninteger ratings were allowed. When comparing the 3 raters to each other (first vs. second, first vs. third, and second vs. third) using the pairwise correlation, a high correlation was reported among all 3 (0.86, 0.88, 0.88, respectively). ANOVA comparison between the control group and experimental group for the 3 measures of ratings namely smoothness, efficiency, and accuracy, showed a statistically significant difference between the groups, with the experimental group outperforming the control group (P<0.03, P<0.05, and P<0.03, respectively) (Figure 12).
Discussions
The investigation into preoperative warm-up prior to actual laparoscopic surgeries resulted in significant evidence that such activity provides advantageous benefits to performance in both psychomotor and cognitive skills. This small pilot experiment also demonstrates that there are significant benefits to warm-up in the OR and that the effects are generalizable to tasks other than the warm-up tasks. It is noteworthy that Catalayud D et al37 replicated the results of benefit of warm-up in the OR. Although these preliminary findings have rather small numbers of subjects and therefore need verification with larger multi-institutional studies, this study strongly suggests that warm-up is beneficial and has serious implications not only for training and patient safety, but also for policy and practice.
5. Conclusions and Implications on policy and practice
Laboratory studies and VR to OR validation demonstrate preliminarily that preoperative warm-up improves performance and decreases errors. It is encouraging that another independent investigator has confirmed these results. The result is that conclusions can be drawn regarding a few fundamental precepts. First and foremost, the results confirm the a-priori impression that performing a preoperative warm-up exercise before a surgical procedure can improve operative performance in terms of more efficient motion, reduced operative time (procedure efficiency), and reduced errors (patient safety).
Second, the improvement occurs from both psychomotor and cognitive skill enhancement, although working memory does not seem to be affected. This improvement is completely independent of the expertise of the surgeon, and the improvement will occur before every operative procedure. There is no learning curve after which warm-up is not helpful.
Third, preoperative warm-up can improve performance in a fatigued surgeon, although the performance improvement will only be to the surgeon’s baseline level and not to his or her optimal potential level. This finding will require further validation and investigation into other areas, such as a longer warm-up period, quantifying the amount of fatigue, and other such things.
There are significant policy implications from these results. The most important implication is whether there should be a required preoperative warm-up period before every surgical procedure (in the name of efficiency and patient safety). Should there be a specific, quantifiable level of proficiency (benchmark) the surgeon should obtain before beginning a surgical procedure? Should the surgeon be required to document that the preoperative warm-up was complete before starting the procedure (thereby documenting that the surgeon was at his or her optimal performance)? Will it be necessary to require that operating suites have readily available access to preoperative warm-up stations? If a surgeon does not warm-up and an adverse event occurs, will the surgeon be liable? These are but a few of the possible implications, all of which need to be thoroughly investigated and deliberated before any hasty regulations are put into place.
Once again, healthcare is beginning to confront a “common sense” practice that is routine in many other high-stakes professions. The results of this study are intended to begin to form the scientific foundation for a new approach to surgical practice: the preoperative warm-up. Just as the “see one, do one, teach one” tradition of surgical education is yielding to quantitative training and assessment of technical skills and criterion-based benchmarks to demonstrate proficiency, so too may preoperative warm-up become a new standard for improving operative performance and patient safety.
References
- Stretching I. The Stretching Handbook: http://www.thestretchinghandbook.com/archives/warm-up.php. Accessed May 15th, 2006.
- Anshel MH, Wrisberg CA. Reducing warm-up decrement in the performance of a tennis serve. Journal of Sport and Exercise Psychology 1993; 15: 290-303.
- Valck E, Cluydts R, Pirrera S. Effect of cognitive arousal on sleep latency, somatic and cortical arousal following partial sleep depriva. Journal of Sleep Research 2004; 13(4):295-304.
- Aggarwal R, Moorthy, K., and Darzi, A. Laparoscopic skills training and British Journal of Surgery 2004; 91:1549-1558.
- Hall JC, Ellis.C, Hamdorf.J. Surgeons and cognitive processes. British Journal of Surgery 2002; 90(1):10-16.
- Spencer F. Teaching and measuring surgical techniques – the technical evaluation of competence. Bull Am Coll Surg 1978; 63:9-12.
- Treves F. A Manual of Operative Surgery. London: Cassell andCompany, 1891. 26-27.
- Satava R, Gallagher AG, Pellegrini CA. Surgical Competence and Surgical Proficiency: Definitions, Taxonomy, and Metrics. American College of Surgeons 2003; 196(6):933-936.
- Kahol K, Balasubramanium VN, Krishnan CN, et al. Measuring movement expertise in surgical tasks. 14th Annual ACM International conference on Multimedia Santa Barbara, CA: IEEE, 2006. pp. 719 – 722.
- Kahol K, French J, McDaniel T, et al. Augmented virtual reality for laparoscopic surgical tool training. HCII International Beijing, China, 2007.
- Kahol K, McDaniel T, Smith M, et al. The effect of real-time visualization of skill on surgical training. Medicine Meets Virtual Reality Long Beach, CA, 2007.
- Gallagher AG, Richie K, McClure N, McGuigan J. Objective psychomotor skills assessment of experienced, junior and novice laparoscopists with virtual reality. World Journal of Surgery 2001; 25:1478-1483.
- McNatt SS, Smith CD. A computer-based laparoscopic skills assessment device differentiates experienced from novice laparoscopic surgeons. Surgical Endoscopy 2001; 15:1085-1089.
- Torkington J, Smith SGT, Rees B, Darzi A. The role of basic surgical skills course in the acquisition and retention of laparoscopic skill. Surgical Endoscopy 2001; 15:1076- 1079.
- Leyba M, Kahol K, Smith M, et al. The effect of fatigue on psychomotor and cognitive skills of surgical residents. Annual Meeting of American College of Surgeons, 2007.
- Anshel MH, Wrisberg CA. ED283791:The Effect of Arousal and Focused Attention on Warm-Up Decrement. 1987.
- Do AT, Cabbad M, Kerr A, et al. A Warm-up Laparoscopic Exercise Improves the Subsequent Laparoscopic Performance of Ob-Gyn Residents: a Low-Cost Laparoscopic Trainer. Journal of the Society of Laparoendoscopic Surgeons. 2006; 10:297-301.
- Mesulam M. Spatial Attention and Neglect: Parietal, frontal and cingulate contributions to the mental representations and attentional targeting of salient extrapersonal events. Phil. Trans. R Soc Lond B. 1998;354:1325-1346.
- Posner MI, Snyder CRR. Attention and Cognitive Control. In Solso RL, ed. Information processing and cognition, The Loyola symposium: Lawrence Erlbaum Associates, 1975. pp. 55-85.
- Shiffrin RM, Schneider W. Controlled and automatic information processing: II Perceptual learning, automatic attending and a general theory. Psychological Review 1977; 84:127-189.
- Barbara K, Nespoulous J. Working memory performance in expert and novice interpreters. Interpreting 2006; 8(1):1-23.
- Marchetti LM, Biello SM, Broomfield NM, et al. Who is pre-occupied with sleep? A comparison of attention bias in people with psychophysiological insomnia, delayed sleep phase syndrome and good sleepers using the induced change blindness paradigm Journal of Sleep Research 2006; 15(2):212-221.
- Hatwell Y, Streri A, Gentaz E. Touching for knowing : cognitive psychology of haptic manual perception. Amsterdam , Philadelphia: John Benjamins Pub., 2003.
- Baddeley A. Working Memory: The interface between memory and cognition. In Schacter DL, Tulving E, eds. Memory Systems. Cambridge: MIT Press, 1994.
- Miller GA. The magical number seven, plus or minus two: Some limits on our capacity for processing information. . Psychological Review 1956; 63:81-97.
- Cowan N. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Brain and Behavioral Science 2000; 24:87-185.
- Lezak MD. Neuropsychological Assessment. New York: Oxford, 1995.
- Hodge M. Factors Affecting Cognitive Function in Acute Care Nurses Working 12 Hour Shifts. Abstr Acad Health Serv Res Health Policy Meet 2002; 19:6.
- Satava R, Gallagher AG, Pellegrini CA. Surgical Competence and Surgical Proficiency:Definitions, Taxonomy, and Metrics. Journal of the American College of Surgeons 2003; 196(6).
- Miller GA, Galanter E, Pribram KH. Plans and the Structure of Behavior New York: Holt, 1960.
- Klove H. Clinical Neuropsychology. The medical clinics of North America. New York: Saunders, 1963.
- Lezak MD, Howieson DB, Loring D. Neuropsychological Assessment. New York: Oxford, 2004.
- Kahol K, Smith M, Tripathi P. Gesture Based Hand Movement Analysis and Haptic Feedback for Surgical Training (Selected as Best Poster). Medicine Meets Virtual Reality 14. Long Beach, CA., 2006.
- Rosen J, Brown JD, Chang L, et al. Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model. Biomedical Engineering, IEEE Transactions on 2006; 53(3):399-413.
- Kahol K, Krishnan NC, Balasubramanian VN, et al. Measuring movement expertise in surgical tasks. Proceedings of the 14th annual ACM international conference on Multimedia. Santa Barbara, CA, USA: ACM, 2006. pp. 719-722.
- Deaconson TF, O’Hair DP, Levy MF, et al. Sleep deprivation and resident performance. The journal of the American Medical Association 1988; 260:1721-1727.
- Catalayud D, Arora S, Aggarwal R, et al. Warm-up in a Virtual Reality Environment Improves Performance in the Operating Room. Annals of Surgery 2010; 251:1181-1185.