Purpose
To develop a structured tool to assess strabismus surgical skills and to determine the face and content validity of this tool.
Design
Development of a surgical assessment tool and its validation by an expert panel.
Methods
A structured subjective evaluation form was developed to evaluate a trainee’s technique and skill in strabismus surgery. The tool consists of strabismus surgery-specific components, such as conjunctival incision, muscle exposure, and suture placement, and global indices, such as hemostasis, tissue control, and knowledge of instruments. Each of these components was evaluated on a 5-point Likert scale. A survey of experienced strabismus surgeons was performed to establish the face and content validity of the tool. The results of the survey were used to amend the tool.
Results
The final tool comprises 10 strabismus surgery-specific competencies and 7 global indices. The final version of the tool was deemed to address the vital aspects of a strabismus surgical procedure (face validity), and the components of the assessment tool were deemed to assess appropriate competencies and skills associated with the procedure (content validity).
Conclusions
The strabismus surgery skill assessment tool is a structured quantitative instrument designed to aid surgical evaluation and training of ophthalmic surgical trainees. The Strabismus Surgical Skill Assessment Tool is a paper-based tool that is easy to use, provides the trainee with detailed feedback and a measure of progression of their surgical skills, and stimulates discussion between trainee and trainer to direct further training.
There has been increasing emphasis over recent years of the importance of assessing surgical skills of ophthalmologists in training. Several tools to assess surgical skills have been developed and published. The Royal College of Ophthalmologists (RCO) in the United Kingdom require trainees to show progression in developing surgical skills in strabismus surgery using a generic Objectively Structured Assessment of Technical Skills form. We sought to develop a tool specific to strabismus surgery.
Methods
The strabismus surgical skill assessment tool was devised comprising 2 areas of assessment: first, a task-specific breakdown of the essential steps of strabismus surgery, and second, global skill parameters reflecting general surgical techniques practiced by ophthalmic surgeons. This format has been validated elsewhere. This format allows trainees to gain some insight into their basic surgical skills and to provide generic end points that they would encounter in strabismus surgery while allowing for differences in technique among trainers.
A 5-point Likert scale was used, as validated by several groups. In accordance with RCO assessment criteria, the trainee is assessed against what would be expected from a trainee at their stage of training, rather than against a gold standard operation. For this reason, the descriptors used are general and the trainer is allowed to interpret how the trainee measures against their expectations. The column on the extreme right allows for an entry to be made if particular steps were not undertaken, such as in a modular teaching technique, while still allowing the trainee to receive feedback on their progress.
Survey
The tool was presented to 20 highly experienced strabismus surgeons to assess face validity (the extent to which the items addressed the vital aspects of the surgical technique being assessed) and content validity (the extent to which the items addressed the trainee’s competence and skill). Structured feedback and comments were collected using a face and content validity form. The form listed the components of the assessment and asked the specialists whether they agreed with the content and weighting of each component and asked them to record any comments to alter or improve the form. The assessment tool was modified with the help of comments from the survey. This method has been used by several authors.
Results
The Table summarizes the response of experts to questions in the face and content validity form. The Strabismus Surgical Skill Assessment Tool was modified accordingly after all feedback had been received. The final tool comprises 10 generic competencies and 7 global indices. No components were removed, components were added, and 2 components were modified after the survey.
| Questions | No. of Experts Who Said Yes | No. of Experts Who Said No | Comments Received | Comments Implemented |
|---|---|---|---|---|
| Does the tool include the most essential elements of the procedure? | 19 | 1 | 1 | 1 |
| Does the sequence represent a strabismus surgical procedure in real life? | 19 | 1 | 1 | 0 |
| Are the skills listed representative of the skills that trainees should be mastering during their training? | 20 | 0 | 0 | 0 |
| Will this list capture important deficiencies, if present, in trainee’s surgical technique? | 20 | 0 | 0 | 0 |
| Is there anything else that should be added/changed? | 13 | 7 | 9 | 3 |
| Are there any essential elements of the surgical skill that are not assessed in this tool? (the tool intends to assess surgical technique and competency rather than be an exhaustive assessment of knowledge or ability) | 4 | 16 | 4 | 0 |
| Overall form design | N/A | N/A | 15 | 4 |
The question “Is there anything else that should be added/changed?” in the survey elicited the most comments from experts (see Table). The comments implemented in the tool were complexity of the case, meets expectations/does not meet expectations, measures position of muscle, combine conjunctival incision and Tenon dissection, and combine speed and efficiency of movement and overall flow of procedure. Keeping in line with the RCO Objectively Structured Assessment of Technical Skills Tool, the outcome section of the assessment was amended to maintain the format of a familiar structured feedback to the trainee. Thus, good points, suggestions for development, and agreed action were incorporated into the tool.
It was not feasible to implement some of the comments into the tool. Comments such as dose of surgery (how much to recess/resect), surgical plan, and postoperative complications were not incorporated into the tool because this tool intends to assess surgical technique and competency rather than to be an exhaustive assessment of knowledge or ability. Not all strabismus surgeons practice the use of adjustable sutures, and hence comments calling for a section on adjustable sutures were not implemented in the tool. The comment calling for an explanation of planned surgery to nurse/assistant was not implemented because there was already a component “Communication with surgical team” in the section Global Indices. Two experts queried the lack of descriptors for points 2 and 4 on the 5-point Likert scale. This 5-point Likert scale has been used in several assessment tools, and we believe 3 descriptors are sufficient, and the familiarity with this scale coupled with the uniformity makes the form easier to complete. The final version of the Strabismus Surgical Skill Assessment Tool is presented in the Figure .
