President Obama’s budget proposal for 2012 includes a $100-million idea that could revolutionize the way social programs are financed—or could put a ticking time bomb next to the heart of some of our society’s most needed social programs.
…The main concern will be how to define success; in the real world, it is difficult, if not impossible, to agree on rigorous measures that make sense.
…If all government-financed programs were to rely on the social-impact-bond approach, the debate over measurements of success would more often than not set up social programs to fail, with a nonprofit’s program, not the unrealistic expectations attached to it, becoming the scapegoat.
And that could unwittingly undermine society’s support for government financing of programs to help the needy just at a time when relatively little support already exists.
Melinda Tuan, a co-founder of REDF and expert in social return on investment frameworks, writes:
I think very careful attention must be paid to the projected positive outcomes of a program and whether those are intended to generate cost savings for the government.
In my experience with social return on investment (SROI) at REDF, where we tried to calculate the social savings to society generated by nonprofit social enterprises that employed formerly homeless individuals, I clearly recall at least one important finding from that work.
We funded a fabulous nonprofit called Youth Industry that tried to get homeless youth off the streets and into wage-earning jobs in social enterprises such as a thrift store, bicycle repair shop, neighborhood restaurant, etc. The positive outcomes achieved in the lives of these homeless youth were remarkable. Based on follow-up interviews conducted two years after youth were hired at a Youth Industry enterprise, 72% of youth were still employed, and 45% were living in a stable home (as opposed to 81% homeless at time of hire).
However, the youth were also using more social services at two-years after hire than they were at the time of hire — which meant they were costing more to society than they had previous to being employed in Youth Industry social enterprise. Their “SROI” numbers were not good.
But the real story is that these were youth who were largely disenfranchised from society at the time of hire: 76% had used illegal substances, 68% had mental health issues, 41% had a previous criminal conviction; and as a staff members said, “they were not working the system, they wanted to get out of the system.”
The youth outcomes showing an increase in usage of social services provided by the government were a positive outcome for the youth. For example, they were re-engaging in society, accessing much-needed mental health services, and obtaining food stamps to feed themselves (as opposed to scrounging in the streets). In the case of these formerly homeless youth, costing more money to society was a step in the right direction towards their ultimate greater self-sufficiency and well-being.
Sean, I was glad to see that you cautioned any Pay for Success program be piloted using effective programs with evidence of success AND where positive program results would produce cost savings to the government. It would be a huge mistake to simply assume that cost savings are tantamount to positive outcomes in individuals’ lives.
Reader Jacob M comments:
While all this sounds great, to me the question whenever anyone sets up an incentive-based system that uses outcomes is a question of monitoring. No Child Left Behind’s effect on state education standards is probably the best recent example of this – when the Feds asked the states to hold their schools accountable for passing State tests, most states just dumbed down their tests so students would pass.
There’s a similar issue at play here. If the intermediaries get compensated for the outcomes which they fund, there better be some independent entity that is collecting the data and ensuring quality. Otherwise, there will be strong pressure, especially if investors get involved, to deliver on those outcomes.
While some of this pressure might lead to program efficiencies, it also will encourage people to game the system. Outright lying about outcomes might be possible to catch, but other more subtle methods such as “creaming” for the least needy may prove more difficult. And even if on paper outcomes may look good, the neediest people may become more likely to be overlooked.
I’ll respond to some of these comments tomorrow as all of them raise important points.