When I first wrote about the Demonstrating Impact session at the Council on Foundations conference, I wrote:
The discussion was nominally about how foundations can do a better job letting influential Americans and the general public know about the good work they do. But at the root of this is a discussion about identifying and measuring the impact of foundation grantmaking. And at the root of that is a discussion about transparency and the sharing of information about what works AND what doesn’t work.
In my first post about the session, I focused on the implications for transparency. Today I want to return to the conversation and focus on the panelists’ discussion of identifying and measuring impact.
The panelists were:
- James Canales, CEO, The James Irvine Foundation
- James Knickman, CEO, New York State Health Foundation
- Joel Fleishman, author of The Foundation: A Great American Secret
Measuring impact is a big topic right now. Back in March, I held a Giving Carnival debate on the topic. What I found important about the discussion during the Demonstrating Impact session was the panelist insistence that while communicating impact was important, we shouldn’t get bogged down in trying to precisely measure and quantify the metrics. One of the reasons cited was the difficulty in proving causality. For instance, even if test scores of low-income students rise 12% after a foundation makes a large grant to a local tutoring program, this doesn’t mean that the grant caused the test scores to rise. Using the dollar amount of the grant, the percentage rise in test scores and the average increase in lifetime earning potential due to the higher test scores, it might be possible to calculate a specific “return on investment” for the grant. But did the grant have any impact or were the increased test scores due to something else entirely?
James Knickman made a good analogy:
A good soup is made up of lots of parts, it is hard to identify if the garlic or the carrots make it good. But you can identify if the soup is good. If you’re making a bad soup, do something different.
As readers of this blog know, I think that for-profit business knowledge can be used to improve the effectiveness of nonprofits and philanthropists. But don’t be fooled into thinking that business is always precisely measured and numbers driven. Many investors, including (or even especially) professionals tend to find a kind of comfort in precisely measuring and calculating long strings of numbers before making investment decisions. But I agree more with investor Whitney Tilson, a big fan of Warren Buffett, who wrote the following:
If the future were predictable with any degree of precision, then valuation would be easy. But the future is inherently unpredictable, so valuation is hard — and it’s ambiguous. Good thinking about valuation is less about plugging numbers into a spreadsheet than weighing many competing factors and determining probabilities. It’s neither art nor science — it’s roughly equal amounts of both.
The lack of precision around valuation makes a lot of people uncomfortable. To deal with this discomfort, some people wrap themselves in the security blanket of complex discounted cash flow analyses. My view of these things is best summarized by this brief exchange at the 1996 Berkshire Hathaway annual meeting:
Charlie Munger (Berkshire Hathaway’s vice chairman) said, “Warren [Buffett] talks about these discounted cash flows. I’ve never seen him do one.”
“It’s true,” replied Buffett. “If (the value of a company) doesn’t just scream out at you, it’s too close.”