Evaluating Estimators of Treatment Effects Using Detailed Measures of Illness Severity From Medicare Claims Data
- Presenter:
Chair: Gabriel Picone; Discussant: Joe Terza Tue June 6, 2006 10:45-12:15 Room 332
An old and constant problem in health economics has been to estimate the effects of choices made by doctors or patients on health outcomes in the presence of unobserved heterogeneity. Economists have a broad range of estimators available, which fall into two categories. The first assumes that treatment is exogenous conditional on a set of covariates (selection on observables). Estimators based on this approach include ordinary least squares with a large number of controls, flexible regression methods, propensity score methods, and matching estimators. The second approach uses instrumental variables to identify the treatment effects of interest. However, each of these estimators makes different sets of assumptions and may even estimate different effects. In this study we compare these different estimators through their ability to estimate the effects of different treatments on survival outcomes for individuals with Acute Myocardial Infarction (AMI). The treatment effects studied are the effects of: (i) catheterization, (ii) admission to for-profit hospitals, and (iii) admission to low-volume hospitals. Our dependent variables are 30-day and one-year mortality following the AMI. We use data from the Cooperative Cardiovascular Project (CCP) merged with the American Hospital Association’s (AHA) annual survey of hospitals and the National Inpatient Survey (NIS). One significant limitation of the Medicare claims data used previously to study these treatment effects is that it does not contain complete measures of severity of illness. The strategy of this study is to use a richer data set to analyze how sensitive different estimators are to the addition of detailed severity of illness measures. Severity of illness is a primary factor driving selection into treatments and if omitted from the estimation leads to bias in the estimation of the treatment-outcome relationship. Lacking a combination of experimental and non-experimental data, as used in previous studies comparing treatment effect estimators, we use the availability of detailed severity of illness to compare the performance of different estimators. Our results show that for estimating average treatment effects, methods that rely on the selection on observables assumption produce almost identical results using similar sets of control variables and there are no advantages over using a standard least squares regression. Across methods, the addition of controls for severity of illness reduces the estimated average treatment effects. In fact, conditional on detailed patient data, the addition of hospital characteristics has little impact on the average treatment effects. Methods that rely on instruments tend to be very unstable across specifications with different sets of controls. We conclude that in estimating treatment effects there may be no alternative but to rely on good data.
