Maximizing vs Measuring Impact

Gabriel Kasper, one of the authors of the What’s Next for Philanthropy report that I mentioned last week commented on my post:

“Thanks for the provocative commentary on the report Sean. We couldn’t agree with you more about the importance and power of leverage. In the coming years, we believe that the most effective funders will increasingly be able to leverage outsized impact and trigger resources that are far larger than their own, whether it’s by influencing other funders (institutional or individual), catalyzing government support, mobilizing public will and opinion, or stimulating markets to sustainably provide services…

But I wonder, though, whether the main distinction you are really highlighting is less about “internal” versus “external” leverage (many of the examples of Acting Bigger from our report—especially in the section on “Leveraging Others’ Resources”—are primarily about influencing the flow of external resources, after all), as much as it’s about “direct” and “indirect” impact.

And in that respect, your point remains a powerful one. In the end, the greatest impact of the Women Moving Millions campaign, for example, may not be in exactly how many dollars the initial gift ends up leveraging directly, but instead in how the effort raises awareness about women’s and girl’s issues and how it shifts the behaviors and attitudes of generations of women who are empowered and encouraged to give in the years to come.”

Gabriel makes a good point in questioning the internal/external vocabulary I used. My point in the post (and in the presentation I gave on Friday to the Social Venture Partners conference) was to suggest that the impact of a funder’s actions may be felt much more largely away from the targeted area intended by the funder.

In the post I wrote:

“I believe that in many cases, external leverage is FAR more powerful than internal leverage. But internal leverage is far more measurable. We can see this dynamic at work in Warren Buffett’s gift to the the Gates Foundation. The internal leverage (Buffett’s utilization of the Gates Foundation’s resources to distribute his giving) is significant. But as I’ve argued many times, the big impact of Buffett’s gift is the external leverage generated through the richest people in America modeling a changing attitude to philanthropy.”

Gabriel suggests that direct/indirect might be better than internal/external. I hesitate to use this vocabulary, because I think the “external”/”indirect” impact of Buffett’s gift, for example, is actually very direct. His actions are directly influencing the way Americans think about philanthropy. It may be difficult to measure and it may be difficult to control, but it is very real.

But Gabriel is right that internal/external isn’t very good either. Clearly a funder can leverage internal sources of leverage (non-monetary resources) or external sources (other fund flows). These actions can be directed towards precise, targeted activities or towards influencing larger spheres.

Regardless of what we call it, an important take away for me is that a real tension can exist between measuring impact and maximizing impact. The relationship between the two is neither completely correlated or negatively correlated. Often measuring impact can be a key tool for maximizing it. Yet I think it is important for philanthropy to grapple with the idea that those things which can be measured are not always the most important things. As we strive towards measuring impact, it is critical that we don’t end up thinking too small. Achieving measureable objectives while ignoring opportunities to create much larger impact simply because the larger impact isn’t measureable would be a tragedy of epic proportions.

8 Comments

  1. Sean, I hope you can have some impact with articles like this, that might be measured by other writers sharing this in their own blogs, or by more donors adopting this thinking.

    Last week I wrote a blog article asking which is more possible, putting great teachers in every classroom in every high poverty area of the country, and keeping them for 10 or 15 years, or creating great non-school programs that surrounded kids in poverty with extra adults and extra learning so they were motivated to go to school demanding that educators help them learn.
    http://tutormentor.blogspot.com/2010/10/education-reform-big-question.html

    In either scenario, it would take many years and a load of continuously flowing dollars going to thousands of locations to put great teachers in every classroom, or great non-school mentoring, enrichment and learning centers in every poverty neighborhood. No single donor, even the government, can fund all that would need to be funded.

    However, each donor who shares the vision, and funds part of this, could multiply his/her impact by encouraging others to add their own donations to the effort.

    Over time, that might add up to much more than we have right now.

  2. I am not sure I understand this sentence:

    Achieving measureable objectives while ignoring opportunities to create much larger impact simply because the larger impact isn’t measureable would be a tragedy of epic proportions.

    How do you know that the other impact is “much larger” if it is not measurable?

    • Good question Joel. Regardless of whether something is measurable, it can still be real and significant. My point wasn’t to argue that unmeasurable things are always larger impact. I’m suggesting that a focus on measurable outcomes creates a creative tension because it steers us away from unmeasurable projects even though some of them might be much higher impact.

      • Hi Sean. My point was that it is meaningless to use words like “larger” or “higher” or phrases like “more significant”, which all imply measurement, if no measurement has taken place. Again: how can you say something is “larger” or “more significant” if you cannot measure its size or significance?

        • Joel, what’s more important in judging the success of your life, the number of friends you have or the depth of your relationships? Some people might feel that depths of relationships are more important than number, even though depth is much harder or even impossible to measure.

          Traditional advertising is much harder to measure than google ads. But we keep seeing super bowl ads, bill boards in Time Square and blimps with brand names on them because they work even though they aren’t measurable.

          • Hi Sean. Depth of relationship is far from being difficult or impossible to measure; it is a staple of psychological and sociological research. And while we still have superbowl commercials, advertisers spend far more money on Google ads now than they do on blimps or the Superbowl – precisely because their effect, and therefore their value, can be more exactly measured.

            You seem to insist that certain things cannot be measured but are at the same time “larger”, ” greater”, or “more significant” in your eyes than things which can be measured with effort – a judgment which couldn’t be reasonably made without measurement. How do you know things are “more significant” without measurement? Gut feeling?

            Effectiveness of philanthropy may be harder to measure, and involve greater effort, than simply providing our gut feelings, but that doesn’t mean that those two evaluative methods are equivalent, and it doesn’t mean that that effort of rigorous evaluation isn’t worth the effort.

            We may have to simply agree to disagree here. 🙂

  3. Joel, We’ll have to agree to disagree. I’m all for measuring. But I think that our ability to measure things is limited. I’m just saying that our limited ability to measure things should not be seen as an excuse to only do things which we can measure. Most importantly, from a practical standpoint, I think the issue is less about doing things that are measurable vs unmeasurable, but to make sure we don’t only focus on those things which are most measurable. The easiest to measure results aren’t always the best (ie. see how many people focus weight rather than “health”/”fitness”).