Purpose
To assess the psychometric properties of a new version of the Glaucoma Treatment Compliance Assessment Tool, a survey evaluating health behavior and glaucoma adherence using constructs from the Health Belief Model.
Design
Psychometric analysis.
Methods
We administered the 47-statement Glaucoma Treatment Compliance Assessment Tool to 201 participants who were using a single bottle of an ocular hypotensive agent, and objectively measured adherence with medication event monitoring system devices over 60 days. Adherence was the percentage of days with correctly timed bottle openings. We used principal components analysis to determine construct validity, Cronbach’s alpha for internal consistency reliability, frequency analysis for floor and ceiling effects, and Spearman rho for test-retest reliability. We determined predictive validity using univariate and multiple regression.
Results
The mean (±SD, range) adherence percentage was 79.9% (±18.5%, 20.3%–100.0%). Principal component analysis loaded 24 questions into 6 components that were consistent with the Health Belief Model. All 6 components had Cronbach’s alpha reliability between 0.601 and 0.797. No statements had floor or ceiling effects, and all statements had acceptable test-retest reliability. Multiple regression analysis showed 4 Health Belief Model statements, white race, older age, and married marital status to be associated with higher adherence (adjusted R 2 = 0.27, P < .001).
Conclusions
The newest version of the Glaucoma Treatment Compliance Assessment Tool showed acceptable psychometric properties. With further refinement, clinicians and researchers could use it to examine factors related to adherence and measure improvement in adherence with a change in health behavior attitudes.
Glaucoma is the second-leading cause of blindness worldwide and will affect 76 million people by 2020. Treatment with ocular hypotensive medications can reduce the development or worsening of glaucoma by at least 60%. However, research shows overall poor adherence to glaucoma medications. Clinicians and researchers would benefit from a tool to measure and predict risk of glaucoma adherence.
The Health Belief Model, a cognitive-based model, theorizes that patients will take action to prevent disease if they believe they are susceptible; that the disease is severe; and that the benefits of action outweigh the barriers. Other constructs of the Health Belief Model include cues to action and self-efficacy. Researchers have used the Health Belief Model to understand and predict health behaviors across a spectrum of medical conditions.
We used the framework of the Health Belief Model to create the Glaucoma Treatment Compliance Assessment Tool, a questionnaire designed to evaluate multiple behavioral factors associated with glaucoma medication adherence. We have published the results of a previous assessment of the psychometric properties of the Glaucoma Treatment Compliance Assessment Tool in a single-center study. We revised the current version of the Glaucoma Treatment Compliance Assessment Tool to include a 5-point Likert scale, standard anchoring definitions, and additional statements addressing patient-physician relationship and patient mood. We hypothesized that the updated Glaucoma Treatment Compliance Assessment Tool has acceptable psychometric properties to measure behaviors related to adherence in a multicenter study with a large sample size and diverse enrollment. Future researchers and clinicians may use the Glaucoma Treatment Compliance Assessment Tool to determine risk of poor adherence; measure change in health behavior attitudes with an intervention; and provide a focused discussion of patient factors related to adherence.
Methods
Participants
We included 201 open-angle glaucoma or ocular hypertension patients from 3 tertiary glaucoma clinics. A previous manuscript describes the study methods in detail. In brief, the inclusion criteria were patients who self-administered a single eye drop bottle (ie, single-agent or combination ocular hypotensive medication); were living independently; and had a recent visual field test to measure the severity of their glaucoma. The Institutional Review Boards at Legacy Health System, University of Colorado, and Vanderbilt University approved the study prior to the beginning of data collection, and all participants signed an informed consent. The study adhered to the tenets of the Declaration of Helsinki. We registered the trial on the clinicaltrials.gov website ( ClinicalTrials.gov ID# NCT01409421).
Glaucoma Treatment Compliance Assessment Tool Development
We created the Glaucoma Treatment Compliance Assessment Tool using constructs of the Health Belief Model, expert opinion, and previous studies regarding adherence in glaucoma patients. We also included other information associated with adherence including age, race, medical history, sex, income, and education levels. The Glaucoma Treatment Compliance Assessment Tool includes 47 statements with a 5-interval Likert-type scale response with anchoring definitions (eg, 1 = disagree a lot, 5 = agree a lot) and contains language at a 6th grade reading level as shown by a Flesch-Kincaid reading level of 5.7 and reading ease of 72.1 (11- to 12-year-old level). Patients self-administered the Glaucoma Treatment Compliance Assessment Tool during initial enrollment (baseline testing) and again 2 months later (repeat testing). The Glaucoma Treatment Compliance Assessment Tool is available online ( http://www.deverseye.org/public/Self-Administered-GTCAT.pdf ) or by contacting the corresponding author.
Objective Measurement of Adherence
We measured adherence with a medication event monitoring system bottle cap. Medication event monitoring system caps contain microchips that record the date and time when patients open and close the bottle. Several previous studies have used medication event monitoring system caps to measure adherence to eye drop medications. We trained all participants to place their ocular hypotensive medication in the medication event monitoring system cap system as soon as they returned home from their initial visit, to open it only when taking drops, and to close the bottle again afterwards. To avoid trial openings by the patients or investigators, we excluded the first day that the patients had the medication event monitoring system cap from data analysis, and we also excluded the day of their final visit.
We used AARDEX PowerView (version 3.5.1; Mead Westvaco Ltd, Sion, Valais, Switzerland) to view the time of each bottle opening and total number of bottle openings. For those on a once-a-day medication, we counted a “valid opening” if it occurred within plus or minus 2 hours of the 24-hour interval of a previous opening. Therefore, we did not count openings less than 22 hours before, or more than 26 hours after, the previous opening. The interval clock reset the next time they took a dose when someone missed a full dose for 1 or more days. For patients taking once-per-day medication, we defined adherence percentage as the number of correctly timed bottle openings divided by the total number of days in the monitoring period. We did not apply dose timing rules for those on twice-a-day (n = 25/201, 12.4%) or 3-times-a-day (n = 4/201, 2.0%) medications because clinicians’ instructions did not specify the exact time of day to use these medications. Our analysis considered an “adherent” day if and only if the medication event monitoring system cap measured the correct number of bottle openings expected during a 24-hour period.
Data Analysis: Organizational Structure
We used IBM SPSS statistics (version 21; International Business Machines Corporation, Armonk, New York, USA) to conduct all statistical analyses. We used the baseline Glaucoma Treatment Compliance Assessment Tool for all analyses and the repeat Glaucoma Treatment Compliance Assessment Tool for only the test-retest reliability evaluation. Similar to a previous publication, we used principal components analysis with an orthogonal rotation to determine if the Glaucoma Treatment Compliance Assessment Tool contained an organizational structure consistent with the Health Belief Model. We used eigenvalues (≥1.0) and examination of the scree plot to determine the appropriate number of components to include. Our analysis excluded statements from the model if: (1) they did not load strongly (≥0.50) on any component; (2) they cross-loaded (≥.30) onto multiple components; or (3) only a single statement loaded on any component. We then used Cronbach’s alpha (α) to examine the internal consistency reliability of each component, with cutoffs of acceptable (α ≥ 0.70), borderline (α ≥ 0.60), and poor (α < 0.60), and examined Cronbach’s alpha after removal of each statement.
Data Analysis: Floor/Ceiling Effects and Test-Retest Reliability
We assessed floor and ceiling effects by examining the frequency distribution of each statement. We considered a statement to have a ceiling effect if ≥90% of the responses were a “5,” and a statement to have a floor effect if ≥90% if the answers were a “1.” We report the Spearman rho correlation as an assessment of test-retest reliability between the baseline Glaucoma Treatment Compliance Assessment Tool and the repeat testing. All analyses occurred prior to any intervention.
Data Analysis: Predictive Validity
To assess predictive validity, we used a 3-stage linear regression model with standard model-building techniques after confirming appropriate residual plots for linear regression. The first stage determined demographic factors that were associated with adherence percentage. Candidate variables included age in years at enrollment, sex, education level, race/ethnicity (white vs not white), marital status (married vs not married), intraocular pressure, visual field mean deviation, and medication type (prostaglandin analogues vs other ocular hypotensive medication). We then included variables with P ≤ .10 in univariate regression as candidates in a multiple linear regression equation. This regression used an automated backward stepwise selection procedure with t-score where P ≤ .05 for inclusion and P > .10 for removal. The second stage examined the Glaucoma Treatment Compliance Assessment Tool statements associated with adherence percentage. Similar to above, we used adherence percentage as the dependent variable and used univariate and multiple linear regression analysis to determine the statements associated with adherence. The third stage incorporated the variables from stages 1 and 2 to create a final model with demographic factors and Glaucoma Treatment Compliance Assessment Tool statements that predicted adherence percentage.
Results
Demographics
We included 201 participants. Table 1 describes the demographic information. Most participants (78.1%, 157/201) used a prostaglandin analogue and most (85.6%, 172/201) were on a 1-drop-a-day regimen. A small percentage of patients were on multiple drops per day, with 12.4% (25/201) on 2 drops per day and 2.0% (4/201) on 3 drops per day. The average visual field mean deviation was −3.2 decibels (range −16.9 to 5.3).
| Variable | All Participants | Univariate | Multiple Regression | ||
|---|---|---|---|---|---|
| B (95% CI) | P Value b | B (95% CI) | P Value b | ||
| White primary ethnicity, % (n/total n) | 69.2 (139/201) | 12.05 (5.75–18.34) | <.01 | 10.50 (4.41–16.59) | <.01 |
| Age in years, mean (SD) | 66.9 (11.2) | 0.23 (0.00–0.49) | .07 | 0.23 (0.00–0.48) | .06 |
| Female, b % (n/total n) | 63.2 (127/201) | 1.02 (0.65–1.61) | .92 | — | — |
| Married, b % (n/total n) | 61.4 (121/197) | 8.97 (3.20–14.75) | <.01 | 7.59 (1.94–13.24) | <.01 |
| Education level, % (n/total n) | 2.9 ([−0.4]−6.2) | .09 | — | — | |
| No high school diploma | 7.3 (14/192) | ||||
| High school diploma/General Educational Development degree | 16.1 (31/192) | ||||
| Some college | 28.1 (54/192) | ||||
| College degree or higher | 48.4 (93/192) | ||||
| Intraocular pressure in mm Hg, mean (SD) | 15.1 (3.9) | 0.77 (0.00–1.53) | .05 | — | — |
| Visual field mean deviation in decibels, mean (SD) | −3.2 (3.6) | .14 ([−0.67]–0.96) | .73 | — | — |
| Prostaglandin eye drop use, b % (n/total n) | 78.2 (154/197) | −1.21 ([−8.50]–6.08) | .74 | — | — |
a Glaucoma treatment compliance assessment was the number of correctly timed bottle openings divided by the total number of days in the monitoring period using the medication event monitoring system cap.
b P value comparing adherence between white primary ethnicity to other, female to male sex, married marital status to other, higher education level, and prostaglandin eye drops to other classes of ocular hypotensives. The symbol “—” indicates that the statement was eligible for inclusion in the model but the backward stepwise multiple regression removed it because of a P value >.10.
Construct Validity
Principal components analysis
Table 2 shows 6 extracted components containing 24 statements. Each of these statements loaded ≥.50 on 1 component and <.30 on all other components except for Statement 25 (“Sometimes the drops aren’t with me when it is time to take them”), which loaded 0.62 on Component 2 (forgetting owing to a lack of cues-to-action component) and 0.31 on Component 4 (self-efficacy component). We kept Statement 25 in Component 2 because it most strongly loaded on this component. Of the 23 statements that did not load onto a component, all failed to load ≥.50 on any component, and 5 of them also cross-loaded ≥.30 on multiple components. Examination of components suggested Component 1 indicates a knowledge component; Component 2 indicates a forgetting owing to lack of cues-to-action component; Component 3 indicates a susceptibility component; Component 4 indicates a self-efficacy component; Component 5 indicates a severity component; and Component 6 indicates a barrier as well with medication side effects. This model explained 38.6% of the total variance in responses.
| GTCAT Statements a | Component b | |||||
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | |
| S42. There are things I can do to prevent my glaucoma from getting worse | 0.71 c | −0.04 | −0.06 | −0.03 | 0.04 | −0.10 |
| S41. There are things I can do to control my glaucoma | 0.70 c | −0.04 | −0.13 | −0.05 | −0.03 | −0.15 |
| S9. Major vision loss from glaucoma can be prevented with treatment | 0.67 c | −0.09 | −0.00 | 0.05 | 0.05 | −0.07 |
| S6. A person can have glaucoma and not know it | 0.65 c | 0.01 | −0.10 | −0.05 | 0.02 | 0.03 |
| S1. My personal knowledge of the risk factors for glaucoma is excellent | 0.60 c | −0.17 | 0.21 | −0.15 | −0.05 | −0.03 |
| S2. My personal knowledge of the symptoms of glaucoma is excellent | 0.58 c | −0.27 | 0.16 | −0.01 | −0.24 | 0.10 |
| S11. Vision lost from glaucoma is permanent | 0.53 c | 0.11 | 0.14 | −0.13 | −0.13 | 0.07 |
| S15. I completely agree with my doctor’s diagnosis of glaucoma in my eye(s) | 0.51 c | −0.14 | 0.04 | −0.16 | −0.04 | 0.03 |
| S23. Sometimes I forget to use my drops | −0.14 | 0.74 c | −0.02 | −0.03 | −0.19 | 0.12 |
| S24. Sometimes I fall asleep before dosing time | −0.04 | 0.67 c | 0.01 | −0.11 | −0.16 | 0.10 |
| S22. Over the last month I have not missed taking my eye drops | 0.06 | −0.63 c | 0.22 | −0.04 | 0.22 | 0.14 |
| S25. Sometimes the drops aren’t with me when it is time to take them | −0.04 | 0.62 c | −0.13 | 0.31 | 0.05 | −0.11 |
| S37. I think I will go blind in 10 years if I DO NOT use my eye drops | 0.15 | −0.09 | 0.77 c | 0.08 | −0.16 | −0.06 |
| S36. I think I will go blind in 5 years if I DO NOT use my eye drops | 0.12 | −0.07 | 0.76 c | 0.13 | −0.10 | 0.05 |
| S39. A friend or family member’s experience with eye drops has encouraged me to use my eye drops | 0.14 | −0.18 | 0.51 c | 0.01 | −0.10 | 0.19 |
| S29. I need assistance putting drops in my eyes | −0.01 | 0.16 | 0.06 | 0.78 c | −0.06 | 0.05 |
| S35. My eye drops are difficult to use | −0.11 | 0.04 | 0.07 | 0.77 c | 0.11 | 0.14 |
| S40. I can place the eye drops into my eye correctly without any assistance | 0.13 | 0.10 | −0.12 | −0.76 c | 0.05 | −0.03 |
| S17. If I lost the same amount of vision over the next five years as I have over the past five, it would have no effect on my quality of life | −0.03 | −0.20 | −0.05 | −0.06 | 0.63 c | −0.06 |
| S16. I have lost none of my vision due to glaucoma | 0.00 | −0.11 | 0.08 | 0.12 | 0.58 c | −0.25 |
| S26. Sometimes the drops are painful or uncomfortable to take | −0.00 | 0.12 | 0.10 | 0.02 | −0.01 | 0.75 c |
| S33. My eye drops cause me no pain or discomfort | 0.03 | 0.01 | 0.00 | 0.01 | 0.11 | −0.68 c |
| S31. I will suffer from side effects when using my drops | 0.05 | −0.10 | 0.00 | 0.20 | 0.04 | 0.64 c |
b A 6-component solution was found. Component 1 indicates a knowledge component; Component 2 indicates a forgetting owing to a lack of cues-to-action component; Component 3 indicates susceptibility component; Component 4 indicates self-efficacy component; Component 5 indicates severity component; Component 6 indicates a side effect barriers component.
Reliability analyses
Component 1 (α = 0.797) and Component 2 (α = 0.775) had acceptable internal consistency reliability, while Component 3 (α = 0.660), Component 4 (α = 0.682), Component 5 (α = 0.618), and Component 6 (α = 0.601) showed borderline reliability. Item-total statistics showed the removal of statements would not increase the alpha level in any component.
Floor/Ceiling Effects and Test-Retest Reliability
Statement 27 (I don’t need to take drops) nearly had a floor effect, with 89.9% answering “disagree a lot.” The other responses to this statement were: “disagree a little” (7.7%), “no opinion/don’t know” (2.2%), “agree a little” (15.3%), and “agree a lot” (1.6%). No other statements had a floor effect or ceiling effect. All statements had acceptable test-retest reliability (average r = 0.48, range = 0.21–0.65) when comparing the results from the Glaucoma Treatment Compliance Assessment Tool at baseline to repeat testing.
Predictive Validity
Table 1 shows the results of the univariate and multiple regression analysis of demographic variables. Age, race/ethnicity, and marital status were included in the final model.
Table 3 indicates 10 statements significantly associated ( P < .10) with adherence in univariate analysis. These statements represent the Health Belief Model constructs of knowledge (Statement 8), severity (Statements 16 and 17), forgetting owing to lack of cues-to-action (Statements 23, 24, and 25), barriers (Statements 28 and 33), and self-efficacy (Statement 41). Statement 22 represents self-reported adherence. Six of these statements remained in the multiple regression model: Statement 8 (“Eye pain is a common symptom of glaucoma”), Statement 17 (“If I lost the same amount of vision over the next five years as I have over the past five, it would have no effect on my quality of life”), Statement 22 (“Over the last month I have not missed taking my eye drops”), Statement 23 (“Sometimes I forget to use my drops”), Statement 28 (“Sometimes I am out of drops”), and Statement 41 (“There are things I can do to control my glaucoma”).
