This entry to the One Post Challenge comes from Matt (who is withholding his last name). Matt is an employee in the development office of a nonprofit devoted to providing fracture care in developing countries. These are his views, not those of his employer.
The Burden of “Burden of Disease”
Working in development, I get frustrated with some foundations’ tendency to have rigid, mathematically determined goals that we simply can’t furnish the data to meet.
Individual people respond well to our campaigns – they intuitively grasp that fractures are a real problem. We can speak to them on an emotional basis, and they don’t feel patronized; we can use anecdotes, and they don’t cry out for footnoted tables. Foundations, on the other hand, seem to have an obsessive need for facts and figures. This is understandable, given their duty to weed out programs that aren’t necessary or cost-effective. What irks me is the extreme that this bean-counting goes to. I’m speaking of the apples-to-oranges comparison known as Disability-Adjusted Life Years – a measurement of mortality and morbidity, or the “burden of disease”. What I want to say is, “Here – we have numbers on how much money we save patients, how much time in the hospital they can avoid, how many patients we’ve served, how many dollars we’ve spent, how many instruments and implants we’ve produced – the only figure we don’t have is DALYs. Pick something else, anything else, in any units – furlongs per nanosecond, I don’t care. No more DALYs.” We simply don’t have that data. We can estimate, extrapolate – we could even prevaricate (I would get fired, but we could). But we don’t have the numbers.
That puts us at a severe disadvantage writing to a large foundation, one that’s never considered injuries to be comparable to AIDS, TB, malaria, and the like. Providing our own metric doesn’t wash. Nobody wants us controlling the terms; that would be akin to allowing OJ to handle the glove. Instead, I’m forced to say things like, “Assuming one surgery can avert at least three DALYs, our methods are more cost-effective than preventing the mother-to-child transmission of HIV.” It’s a flattering comparison, but “assuming” casts it into doubt. I have little choice in the phrasing, though, because it is an assumption.
Measuring the impact of our treatments with any certainty and objectivity would require 1) a precise estimation of a disability weight (difficult to do) and 2) much more data from our patients. Asking them to return to the hospital to submit that data would be, frankly, inexcusable, considering the hardship some of them go through to even get to the hospital in the first place. The only remotely comparable demand in our lives is a summons to jury duty. Imagine being summoned to jury duty hundreds of miles away, over rocky terrain, while you’re working twelve hours a day to feed your family of five, then being greeted by a battery of tests and dismissed without any compensation. We could conceivably do this. By giving them a surgical implant, we create a feeling of gratitude and indebtedness, and we could exploit that. What I want to know is, who in their right mind would think it justifiable to exploit patients’ gratitude, waste their time, waste their doctors’ time, and waste our own funding (paying for the doctors’ time), simply on account of one column of data? When foundations demand DALYs from organizations like mine, that’s what they’re asking us to do.
This may seem like an exaggeration.
1) Disability weights can’t be that hard to guess, can they? Take a look at this table. There’s no differentiation between treated and untreated broken bones. There’s no definition of “short term” or “long term.” There’s no “long term” data for many fractures. And this is only the start of the problem.
2) What more data do you need? We know that we prevent treatment by traction or amputation. What we don’t know is the ratio of those treatments or the increase in DALYs averted due to the change to our treatment. There’s no DALY weight for traction; it’s not an illness. If we had data on the percentage of traction cases that end in failure – malunion, nonunion, gangrene – perhaps we could calculate DALYs for those conditions, right? Nope, there’s no values for them. Well, what about amputation? The difference between “treated femur fracture, long-term” and “leg amputation, long-term”: .028. Essentially none. This is no doubt due to “treated” being a misnomer in the data table. So, what can we use instead of that weight? Data from the patient. If they report normal leg function, for example, that’s tantamount to zero disability. If they don’t – well, we’re back to guesswork, but at least we have data to work with.
3) Why must patients return in person? Ask someone how they’re doing, and they’ll reply “fine.” Ask a patient how their fracture has healed, and they’ll reply “fine” – even if your questions are so precise as to demand a numerical response, no questionnaire will be accurate. They need to be examined in person for the data to be objective.
4) Say you do this only in urban areas, where patients can return without too much trouble – wouldn’t that work? In a certain sense. We would be able to estimate DALYs averted on some basis – a skewed basis that we couldn’t justify using for rural patients, who have different injury patterns and post-recovery activity. Further problems that would arise: it would still be an estimate. It would still be unreliable. It would still cost time and money to collect. And it would still be on the foundations’ terms. Even in the very best case, we can’t change the basis of the argument to something rational, because foundations don’t accept that DALYs aren’t a nice little simulacrum of reality. We’re stuck within that framework until foundations realize that DALYs are the wrong measurement for complex conditions.