Holden Karnofsky on Investing in Nonprofits

GiveWell was one of the organizations I listed as taking the approach of Investing in Nonprofits. GiveWell founder Holden Karnofsky has added his thoughts:

On one hand, I don’t believe it’s wise for a funder to withdraw completely from the “What sorts of programs work?” discussion. Many programs simply don’t work, and the state of knowledge about effective programs is very poor – just because a program has raised a lot of money, has an excellent reputation, or even has excellent people working on it doesn’t mean it’s an effective program. “Investing in nonprofits” has to include separating better from worse nonprofits, and I believe part of that evaluation should include what they do and whether there’s a case for it as an effective program.

That said, we see our primary role as identifying excellent charities with strong track records and funding them – definitely not as “subcontracting out” our own programs based on our own theories of change. More thoughts on this idea here and here.

I’ve taken the liberty of posting below the content of the first link Holden refers to in his last sentence. This post was in response to a question I had emailed him in April 2008:

Sean asks (via email):

What’s your view on whether funders should do research on techniques and then fund organizations that use those techniques or do research on organizations and let them decide on techniques? I was intrigued with your education research post, but was wondering if it might make more sense to find smart dynamic nonprofits who will figure out the best techniques to use and change strategy as more information becomes available.

My literal response is that it depends on the funder’s priorities and techniques – I don’t think there is much to be gained by debating the approach “funders” should take in the abstract. But I want to share how we deal with this question, as naive funders (i.e., not experts in the issues) aiming to serve more naive funders (i.e., individual donors), because we do have a specific philosophy on it and we’d appreciate feedback.

My ideal is to fund at the highest level I can have confidence in, i.e., delegate as many decisions as possible to to someone who I feel confident will make those decisions well.

So, my ideal would be to donate not to a charity, but to another funder. If a major foundation, such as the Gates Foundation, could convince me that they consistently make decisions using (a) a strong process, (b) good reasoning, and (c) subjective/philosophical values that are close to mine, I would give to them and let them do the rest (and get rid of our own, now redundant overhead). This was one of the first things we tried when GiveWell was still a part-time volunteer club. What stopped us was that we couldn’t find a single foundation that publicizes substantive information about how it makes its decisions, why it chooses to do A instead of B, and what evidence there is regarding its past and likely future impact. We couldn’t be confident in the institutions without such information; we couldn’t think of a way to get them to share information, since such institutions generally don’t have incentives that we can affect. So we moved on to trying to find great charities.

Again, the goal was ultimately to find a great organization – one that’s better at what it does than we could ever be, and can make its own compelling, evidence-based case for its effectiveness – and give with no strings attached. In some cases, we found exactly this: for example, the Nurse-Family Partnership’s outcomes evaluation is available via peer-reviewed publications, its basic model is clearly described on its website, and it provided documents to fill in gaps in our understanding. PSI was a similar case: after some independent checks on its estimates, we felt we could trust its process as a whole, even for activities we haven’t researched.

In other causes, the strongest applicants could provide some pieces of the puzzle, but not the full top-down case for why their approach was the best available. That’s where we had to start looking on our own for information about what approaches are likely to work, and pick organizations that fit with what we had found. There’s a spectrum here. KIPP gave us about 60% of what we needed to have confidence in it, and after some independent analysis, we ended up feeling that it was our best bet. By contrast, our Cause 2 (global poverty) applicants gave us so little to go on that we ended up betting on an approach, more than an organization.

Between blind faith and micromanagement is conditional confidence: trusting an organization to make decisions because of an evidence-based case that they can make them well. That’s our ideal; when it isn’t available, some degree of micromanagement (i.e., picking an organization based on its approach) seems preferable to blind faith.