This is a guest post by George Overholser of Third Sector Capital Partners, which works with service providers, government and funders on the development of social impact bonds, the Pay For Success program and other social sector capital market projects. George was formerly with Nonprofit Finance Fund where he pioneered the SEGUE accounting system for tracking philanthropic equity investments.
By George Overholser
As is so often the case with innovation, there are many perils to be wary of. I agree completely that Pay For Success can’t work without better methods for measuring impact. There is a real risk of building incentives around metrics that might sound good in a sound bite, but, under the harsh light of RCT social science, wouldn’t hold up. Also, as Melinda correctly points out, we could easily be surprised when, a la “freakonomics” behavioral economics, social metrics look good, but fiscal savings fail to materialize. I also worry about problems of teaching to the test, or, as my friend Nigel Morris used to call it, “monovariablitus”. And then there’s the problem of putting wealthy foundations into a position to call the shots, in a sense, for how taxpayer money is allocated. These are just a few of the many challenges.
All that said, I like to think of Pay For Success as being one of many innovations that can be tested and improved without causing great harm. Let’s not jump to judgment before getting some experience under our belts.
Perhaps we will end up thinking of this as trading one set of difficulties for another. Remember Churchill’s quote: “It has been said that democracy is the worst form of government, except for all the others that have been tried.”
For example, I certainly do not support the bludgeoning of smaller nonprofits with onerous evaluation requirements. But when the checks get big, (as they do with government), and when taxpayer money (not voluntary philanthropic money) is involved, it seems to me that we might have an obligation to raise the burden of proof, whenever it might be possible to do so.
Moreover, I have been persuaded by Jon Baron and others of two truths: (1) There exist highly proven and scalable interventions that are deeply undercapitalized relative to the good they can do, and (2) Under the harsh light of social science, it is clear that hundreds of billions of government dollars are spent each year towards programs that may have once worked, but, today, bring about no discernible change in the life course of the individuals who are served, as compared to randomly selected peers who are not served.
To me, there is a moral imperative to reallocate the “fund what once worked” dollars towards whatever “here’s what we can prove that works” programs may exist.
So while it is true that most programs, indeed most areas of social need, lack sound evidence-based stories of impact… some actually do have RCT proven programs that work. For those, Pay For Success seems like a good way to experiment.
Finally, I am optimistic that the cost of evaluation, both in terms of money and in terms of entangling operations, is dropping precipitously, and that we will therefore soon detect a growing number of high impact/proven programs. The reason: huge databases are now coming online that are a byproduct of running schools, jails, homeless shelters, emergency rooms, etc. Our ability to track individual outcomes, (while also protecting their privacy) is being revolutionized.