I thought I’d follow up on all the comments regarding Pay For Success in bullet point format.
- Regarding the difficulty of measuring impact: Pay For Success sets up a contract that specifies the population to be served, the outcomes to be achieved and the measurement methodology to be used. While the contract may be flawed, the process is agreed upon and so you avoid the second guessing of evaluation that happens in an research setting. If the contract is flawed, then both sides will learn and changes will be made to future contracts. But each contract once executed does not need to get mired in debates about the validity of the evaluation process.
- Regarding cost savings to tax payers not being the same thing as philanthropic impact: I think this is correct. Government and philanthropy have different goals and how they measure success should be different. In the case of Pay for Success, the government is the payer and so their needs must be met. The government engages in public activity for the public good and has a natural interest in providing that good at the lowest possible cost. But low cost isn’t the only goal. For instance, a Pay For Success project that aimed to improve educational outcomes might be in the interest of the government even if the cost was higher. However, the current fiscal and political realities suggest that an initial focus on projects that will lower tax payer expenses makes the most sense. Longer term Pay For Success contracts could be crafted with philanthropic funders as the payer, in which case pure social impact, non-cost related goals might be sought.
- Regarding setting the correct outcomes: It is true that since the contract must set specific outcomes, if those outcomes are too narrow they might create a “teaching to the test” reaction from the intermediary. For instance, if a homelessness reduction project simply specified that a certain percentage of beneficiaries have a place to spend the night, the intermediary may be incentivized to place people into extremely low cost, unsafe housing that is actually detrimental to their longer term life outcomes. However, I believe that the risk of paying for outcomes is far less than the risk of paying for inputs (paying a nonprofit to serve the homeless without any discussion or tracking of outcomes).
- Regarding the time frames used in the contract: Short term incentives can be hugely destructive. One major contributing factor to the financial crisis was the short term incentives that mortgage lenders pursued. Luckily the Pay For Success program breaks away from the traditional one to two year appropriation cycle and allows the contracts to support long term outcome measurement. That being said, there will always be some tension around time frame and working to reward short, medium and long term success will be important.
Pay For Success is an experiment. However, it represents a rare opportunity to shift government spending from input based to outcome based. That shift, from funding activity to funding results, is one of the key unifying elements of the effective philanthropy movement. I believe that those of us who care about results based philanthropy need to lend support to the Pay For Success concept, while not downplaying the challenges, of which philanthropy is well aware, of results based funding.
One Comment
Hi Sean, I’ve really appreciated the discussion this week on “pay for success” and concur that this is a promising but very new idea so we shouldn’t kill the seedling by continually pulling it up out of the ground to examine it. The question is, how can we help it take root as an experiment that we can learn from? In this regard I am struck by how the discussion of this approach, as with other outcome- or performance-based contracting schemes typically focus on the challenges in the social sector of delivering (identifying and tracking the right outcomes to measure, avoiding creaming, etc). But there are as many if not more barriers on the government side of these contracts that also need to be overcome. One of the biggest obstacles here for example is the fact that while government agencies might say they want to contract and pay for outcomes, they don’t want to give up the specification and tracking of inputs and outputs along the way toward those outcomes. And in many areas (e.g., child welfare) they are obliged by law and policy and public scrutiny to insist on inputs and outputs. It is also much easier for government contracting officers to specify and count beds filled and people served than it is for them to assess whether the people in those beds end up stably housed or sober or employed two or three years down the road, hence the attraction to government of traditional input/output vs. outcome/success based contracts. And then there is the broader problem of how the government is organized, in very siloed agencies, and that the savings arising from success in the application of one prevention program may well accrue down the road in the budget of another agency, thereby making it all the more challenging to align the incentives and planning for government (whether it be at the federal, state, or local level) to contract so as to pay for success, as resonable as it might seem to do so. One potentially highly leveraged investment a philanthropist could make in this regard would be to support a government body in taking stock of and developing and documenting ways to overcome these barriers. It might seem counterintuitive for philanthropy to invest in government capacity-building in this regard, but given the fiscal reckoning that will be unfolding for the foreseeable future at all levels of government this type of investment is not likely to be made internally with public funds. Outside support in developing and disseminating this research would also provide continuity over time as particular administrations and political support for this idea might wax and wane.