(In)Consistency of Preference Ratings: Results Using Three Methods of Preference Elicitation
- Presenter:
Mon June 5, 2006 9:30-10:45 Room Alumni Lounge
Background. The direct elicitation of preferences for different health states is one method suggested for measuring quality of life. Direct elicitation methods include the standard gamble (SG), time trade-off (TTO), and visual analog scale (VAS). Though theoretically appealing, in practice these methods produce some troubling results. First, the distribution of elicited preferences can be both left and right censored, with many persons rating health scenarios with a zero or one value. Second, health states that should have a logical ordering may receive reverse rankings. Finally, little is known about how the direct elicitation methods correlate with quality of life ratings derived from scales such as the Quality of Well-Being (QWB) scale.
Setting. Eighty-eight persons rated health states using the SG, TTO, and VAS methods and completed the Quality of Well-Being (QWB) scale. Thirty-four of the persons were randomly recruited; the other 54 were persons with disabilities. All persons rated 9 health states (the subject’s own health, single-eye blindness, double-eye blindness, and 6 scenarios that included various disabilities) using the 3 methods. Each of the 6 disability scenarios had two variants that differed only in health status; respondents were randomly assigned one of the two versions. Preferences were elicited using computerized software.
Methods. The mean preferences from the SG, TTO, and VAS methods were compared and the distributions were graphed. Multinomial logistic regression to predict zero, intermediate, and one responses used patient demographics and preference elicitation method as explanatory variables. The number of preference reversals within and across methods was counted. Correlation with preference imputation was examined by comparing the SG, TTO, and VAS utility values for own health with the QWB score.
Results. The SG and TTO methods gave higher mean preferences than the VAS method. However, the higher means were due primarily to a large number of ones. There were also many zeros, so that the distributions were both left and right censored. The VAS method yielded smoother single-peaked distributions of preferences across all health states. Inconsistent preferences occurred relatively infrequently. Blindness health states were rated inconsistently by 9% of respondents using the SG, 3% using the TTO, and 11% using the VAS. Across disabilityscenarios, persons with disabilities gave preference ratings that were 0.02 (SG) to 0.09 (VAS) higher than randomly selected participants, although among 36 options of health status and rating technique, there were several comparisons that were reversed. Randomly selected subjects were more consistent in giving the higher health state scenario a higher rating. None of the three methods were significantly correlated with the QWB scores for own health, the VAS method had the highest correlation (r=.179; p=.108).
Discussion. The distributions of preferences suggest that use of the SG and TTO methods is problematic. The large numbers of zeros and ones could be valid responses, or could indicate difficulty understanding the methods or protest valuations. The number of preference reversals was a small minority of all possible comparisons, but is troubling nonetheless. None of the three methods was correlated with QWB scores when rating own health.
