In the post Demonstrating Impact, which has become my most widely read post of the conference, I wrote:
Knickman gave as an example a project that The James Irvine Foundation did on end of life care (if readers know more about this project, email me the relevant links). He called it “the most successful failure I’ve ever seen”, because it really changed the way people approached the issue since the project really seemed like it should have worked.
Thank you to everyone who wrote to say that Knickman was referring to a project of the Robert Wood Johnson Foundation, not the James Irvine Foundation. An explanation of why the project failed and an analysis of what was learned can be found here. I found this part of the introduction to be of particular note and useful in thinking about why philanthropy should work on being more transparent:
The findings from the demonstration project at the core of the study were negative: the interventions did not achieve the goals expected. However, the large investment by the Foundation–which has totaled approximately $29 million to date–may have other payoffs. The findings clarified that changes in care at the end of life are not going to happen with marginal adjustments in the way we organize services. It takes a much more sustained effort on many fronts to refocus priorities for the care of the critically ill. Changes in social norms, professional values, and social priorities all need to be part of the solution.
SUPPORT suggests another lesson for a philanthropy that uses some of its resources to support research and analysis. The project was expensive in part because of the detailed, high-quality data-collection effort designed to measure outcomes associated with different interventions. This dataset is providing a range of collateral payoffs as the research team explores the data. For example, an important study published by some members of the team raises serious questions about the efficacy of the Swan-Ganz catheter, a common intervention to monitor cardiovascular function in critically ill patients in hospitals. Thus, investments in quality datasets can lead to important research beyond the questions that motivated the data collection.