Demonstrating Impact: Philanthropy’s Urgent Call to Action

Before I came to Seattle, I told my readers that they could assign me to cover the sessions they found most interesting. Holden Karnofsky, whose project GiveWell he refers to as The World’s First Transparent Grankmaker, dropped me the following note:

To me [Demonstrating Impact: How Foundations Can Show Their Value] is the one that you most have to go to. You have argued consistently (and strongly) for more foundation transparency. You should ask the foundations the questions you pose in your posts: why not share everything they have on a website? Why not publish things that are currently “internal,” like evaluations of specific charities (as opposed to “overall programs”)? Why not blog?

Mark Sedway, director of the brand new Philanthropy Awareness Initiative, had shot me an email last week encouraging me to attend. Reading the description of the session it looked like a class on how foundations can engage in better PR.

I was wrong.

Demonstrating Impact was an incredible session. The room was packed, every corner of the room full of people pressed against the walls and people outside trying to poke their heads in through the doorways. I have over five pages of single-space, type written notes. What I want to share with you isn’t just a blog post. I’m going to have to figure out how to put this all down on paper in a more comprehensive way, but I’ll share some highlights now.

The panelists were:

  • James Canales, CEO, The James Irvine Foundation
  • James Knickman, CEO, New York State Health Foundation
  • Joel Fleishman, author of The Foundation: A Great American Secret

The discussion was nominally about how foundations can do a better job letting influential Americans and the general public know about the good work they do. But at the root of this is a discussion about identifying and measuring the impact of foundation grantmaking. And at the root of that is a discussion about transparency and the sharing of information about what works AND what doesn’t work.

Yes, foundations do great things and most people don’t understand the magnitude of these good works (the very existence of emergency 911 service is attributable to the Robert Wood Johnson Foundation, Fleishman pointed out). But what I found so incredible about the session was the advocating for communication to be designed right into the program strategy from day one. Fleishman talked about The Wallace Foundation and how they do grantmaking in teams with Program, Research and Communications all taking part in strategy design from the beginning.

To me the most important part of Fleishman’s message, the most important message of the session and maybe the conference, is that transparency is NOT about public accountability, it is about improving the sector of philanthropy. It is about improving the way that all of us do our jobs. It is about transforming ourselves from a series of silos to an integrated, robust intellectual capital platform upon which all future grantmakers, big and small, can draw.

Sharing the good works of philanthropy is not enough. A good bit of debate centered on how and why foundations should share their failures as well. To me the issue of why failures should be shared, and why doing so is not a risk for the organization if handled correctly, was nailed by James Knickman:

“We need to frame our release of “failures” as an attempt to learn. No one tells scientists they are a failure when one of their experiments don’t work!”

That’s it right there. What philanthropy is engaged in is an experiment. An experiment in how we can all make the world a better place. We don’t know what the right answer is. In fact, the “answer” is probably evolving as quickly as we can design experiments. But by being transparent, by sharing successful ideas and failed ideas. By judging ourselves not on the outcomes of each grant, but on the body of knowledge that we contribute to the field, we will truly transform philanthropy.

Knickman gave as an example a project that The James Irvine Foundation did on end of life care (if readers know more about this project, email me the relevant links) (see correction). He called it “the most successful failure I’ve ever seen”, because it really changed the way people approached the issue since the project really seemed like it should have worked.

Humans don’t like to talk about their own “failures”. But halfway through the session, someone from the audience who identified herself as a professor of marketing stood up to say that people who admit their mistakes publicly are viewed with more trust afterwards. We need to reframe transparency away from some sort of thing that philanthropy is being forced to consider by outside forces and instead celebrate transparency as the mark of an organization that is truly committed to improving the field.

I’m writing furiously in a Starbucks with a half eaten sandwich next to me. I have another session I have to attend soon. The great thing about blogging is that there is no page limit and there is no final draft (the same could be said of philanthropy). I wanted to get my first impression of the Demonstrating Impact session down while it was still fresh. I’ve completely ignored important themes of the debate that I can’t cover right now. But I’ll be returning to this topic frequently.

Thanks for reading.


  1. Holden says:

    Conceptually, sounds great … so why isn’t it happening? Or is it, and if so, where? Where do I go to look at foundations’ transparent and honest assessments of what does and doesn’t work?

  2. Don says:

    I love the idea of foundations sharing their failures and successes. That sort of transparency almost can’t help but improve the philanthropic process for all, assuming we heed the lessons learned.

    The main problem, as Holden points out, is that there’s no single place to go to get this information. It almost seems that philanthropy needs something akin to the education field’s ERIC (Educational Resources Information Clearinghouse). A centralized repository of lessons learned would be fantastic.

    Another problem that Sean alludes to is getting people to admit their failures. While Dr. Knickman suggests that scientists aren’t called failures when their experiments don’t work, they also tend not to be published. (The dynamics of academic publication are another discussion entirely….) His essential point, however, is right on target — great success often comes as the culmination of repeated failure. It seems like we could all enjoy greater success if we learned from each others’ failures. As the saying goes, “Experience is learning from your mistakes. Wisdom is learning from others’ mistakes.”

  3. Holden says:

    I think centralization is an issue, as Don points out, but further than that, I can’t even find one single foundation that is embracing the kind of transparency Sean writes about.

    Every foundation website I’ve seen has a list of grantees and a list of publications, but no way to see substantive information about what particular grantees do and what the evidence is that it works (and never, ever anything about grantees that have been turned down). As to the list of publications, it’s generally (a) incredibly long (b) alphabetized [thanks, guys, that’s useful] (c) all the publications I look are incredibly general pieces with either “lessons learned” or “program accomplishments” filled with sweeping statements and no details or evidence. I have called a bunch of them asking for more of the information I have in mind (what did your grantees DO? What did you measure and what did you see?) and just gotten nowhere. There are hundreds of foundations so it’s possible I’ve just missed the good ones, although I know I haven’t missed the Wallace Foundation.

    If I had found any good information, I’d harp more on the fact that I’ve never seen a meaningful attempt to organize it and make it usable to a casual party. But I haven’t found any good information.

    I am exactly the guy these people should be targeting with their communications: someone who just wants information concrete and useful enough to help me with my decision of where to donate, and is willing to put in a good deal of legwork to get it. I haven’t gotten a single useful thing from a single foundation. Where’s the beef?

  4. Holden, the presenters were advocating for more transparency, not saying that it already exists. I have not yet done follow up work but you should spend some time looking at Robert Wood Johnson Foundation and James Irvine Foundation.

  5. Hewlett recently published a report on a failure.

  6. Holden says:

    Sean – right, my first question was more directly in response to your post. You answered the “where can I go” question (and I’ll check it out) – the next thing I want to know, which I hope you’ll address if you do a longer writeup, is why the sort of transparency described here (for which I can think of about 15 “pro” arguments and 3-4 embarrassingly bad “con” arguments) isn’t happening. I’m asking about what they said at the session.

    My second comment was more in response to Don – just saying it seems that publicization has to come before centralization (although I agree both should happen).

  7. Don says:

    Fair enough, Holden. Not much point in having a clearinghouse if there’s not anything to centralize. 🙂

  8. As the head of The Communications Network, an organization whose mission it is to promote increased transparency, I believe, as do our members, that there is a link between foundation effectiveness and transparency. However, that hasn’t been a long held belief. Instead for years, and for reasons too many to list, foundations felt how they did their business — aside from what the government required them to report — was nobody else’s. The combination of increased regulation — real and threatened — as well as the realization that they were hurting themselves by being secretive, has led to a push in recent years for foundations to open themselves up. Enlightened foundations know they have to routinely explain who they are, how and why they do their work, and what they are trying to accomplish in order to build awareness, support, and the partnerships and alliances critical to their success. Some foundations are even going the extra step to find appropriate ways to be held accountable for their actions. The belief is that in lieu of market forces, this increases the pressure to perform. Or so it should.