One of the most prevalent themes on this blog and in much of the high profile conversations in philanthropy, is the idea that grants and donations should be made based on evidence that they will actually make a difference. On the one hand, this seems like an entirely uncontroversial statement. On the other hand, it is widely agreed that most people and even most large foundations do not base their giving on any significant level of evidence.
I have good news. This debate is completely normal and I can tell you how it will end. Evidence-based grantmaking will rise to become the status quo, but it will take another decade or two to get there. We only have to look at the medical community for our road map.
According to the journal article Evidence-Based Medicine: A Unified Approach (hat tip to friends at the Robert Wood Johnson Foundation), the concept of evidence-based medicine, is actually quite new.
According to the article:
“Up until about forty years ago, medical decisions were doing very well on their own, or so people thought. The complacency was based on a fundamental assumption that through the rigors of medical education, followed by continuing education, journals, individual experiences, and exposure to colleagues, each physician always thought the right thoughts and did the right things. The idea was that when a physician faced a patient, by some fundamentally human process called the "art of medicine" or "clinical judgment," the physician would synthesize all of the important information about the patient, relevant research, and experiences with previous patients to determine the best course of action.”
This pretty much describes what our field expects of foundation program officers. But we can see that this approach has evolved dramatically in the medical field.
“Beginning in the early 1970s, however, two major flaws began to appear in this fundamental assumption. One was a growing body of research showing that key aspects of the assumption were simply wrong. In 1973 John Wennberg and his colleagues began to document wide variations in practice patterns. The implications for the fundamental assumption were undeniable: When different physicians are recommending different things for essentially the same patients, it is impossible to claim that they are all doing the right thing…
…One problem was the lack of good evidence for many important practices. An estimate that only 15 percent of medical practices were based on solid clinical trials became famous. Another problem was that many practices taken for granted by physicians were actually found to be ineffective when subjected to clinical trials. Archie Cochrane, among others, argued persuasively for much more attention to randomized controlled trials (RCTs).”
Wow. Sound familiar? This is exactly what we are seeing in philanthropy today.
The article continues:
“It was into this supersaturated solution that the term "evidence based" was dropped. The first published use of the term was… in 1990. During the next twenty years, fueled by the forces described above, more and more organizations began to apply evidence-based methods to their work.”
I’d say that philanthropy is somewhere in this stage when it comes to evidence based grantmaking. The downside of this conclusion is that it means our field is decades behind the medical field when it comes to using evidence in our decision making. On the other hand, it means that even in a field where today the idea of basing decisions on evidence is widely accepted and expected, it has been only 20 years that the term has been in use and only 40 years that the idea has even been a mainstream topic.
Importantly, the medical community has realized that while evidence-based decision making is critical, it does not suggest that removing the human doctor from the equation is the end game. Evidence-based decision making does not mean turning to spreadsheets and statistics, but instead allows doctors to draw on a deep evidence base when making case by case decisions.
Evidence-based decision making has been a revolution in health care. Yet it is a revolution that has been going on for 40 years and has not yet reached anywhere close to its full potential. A New York Times Magazine article from last year, Making Health Care Better, concludes that increasing evidence-based medicine is the key to fixing health care and profiles a number of clinics who use evidence-based practices that are far ahead of the field as a whole.
We are in the midst of a revolution in philanthropy. While our field might not be practicing the equivalent of doctors bleeding their patients using leaches, philanthropy clearly acts very regularly without any real evidence that our actions will make a difference. However, the rise of evidence-based grantmaking is real. It might take a few decades to play out. But that’s just how real change happens.
Great post Sean and one that helps frame the philanthropic evidence-based efforts in a different context. I would hope, though, that the future evidence-based framework also allows for experimental designs and innovative approaches that will further test what would be considered best science (or best philanthropy). The other area I would hope philanthropy does not emulate is the long gap in translating research into practice – about 19 years. Last, and an ongoing challenge we will face, is that many evidence-based approaches involve control groups or some form of clinical trials in order to isolate variables. That might not be the best approach within philanthropy. Imagine a foundation purposely deciding to withhold a health service for a community in order to test out the effectiveness of its approach in another community. In my opinion, we might end up reconciling ourselves to approaches that establish evidence-based without the bona fide clinical trial investigation.
We were recently asked by the Annie E. Casey Foundation to explore evidence-based practice in fields like medicine and education to see what we could learn that might apply to leadership and leadership development. The RCT is not usually an appropriate method for producing evidence about what works to develop and connect leadership, however, we are finding that bringing an evidence-based lens to our evaluation and research work focuses attention on how to produce and be accountable for results. One area we are pursuing is to develop better hypotheses about leadership development that can be tested by those in the field. Right now, too often, leadership programs have lofty goals but they are not systematically learning about what is and is not working to produce results. We all need to do better. We are still in the early stages but we invite others who might be interested in this topic to join in creating resources at a wiki we set up http://www.leadershipforanewera.org/page/Evidence-Based+Practice+Resources
Sounds like a great project. I’d love to hear more about what you learn as you go.