Albert Ruesga, who blogs at White Courtesy Telephone, was a panel member at the recent Metrics Mania debate at the Bradley Center for Philanthropy & Civic Renewal. Albert posted the text of his comments today and they are outstanding.
At a time when we are seeing a growing backlashing against “philanthrocapitalism”, it is interesting to look at what is being grouped under that term. For many people, “metrics” and the push for more “evaluation” of philanthropy is an unwelcome element of a “business-like” approach to giving. I believe that evaluating nonprofits and philanthropy in general is necessary for a the Third Sector to become a high-performing, high-impact driver of social good. But as I wrote last week, I think that much “evaluation” takes a scientific approach to measurement that is borrowed from the hard sciences, while the lessons of the liberal arts (under which investing and financial markets should be categorized) are more appropriate.
Albert, I think, would agree. He writes:
Measurement and evaluation, when done properly, are not just a bit of value-added for philanthropic or nonprofit work, they’re absolutely essential. Only a fool would disagree with that proposition.
But here I mean not just the kinds of formal evaluations described by Gary Walker in his essay, but informal evaluation as well: the kinds of course corrections we naturally make when we embark on a project, take a false step, and adjust what we do accordingly. Evaluation is not and should not be the sole province of the highly compensated consultant. We evaluate all the time; our own eyes and ears notice things the most astute consultant will never notice; and we’ll often be our own worst critics.
Now here’s where the metrics schmetrics comes in, perhaps: More nonsense has been spoken and written about evaluation than about any other subject in philanthropy. The number of people practicing evaluation without a license and without a proper scientific and philosophical grounding in the subject is, in my view, a scandal. Worries about evaluation, engendered in part by logic models the length of whale intestines, have become the math anxiety of the philanthropic world…
…I want to make clear that I’m not in the least anti-evaluation. As I’ve written elsewhere, I’m concerned that we tend to seek a kind of scientific or moral certainty from a formal evaluation where none exists. The questions that funders most often bring to an evaluator—Was this program worth our $25,000 investment? Should we continue funding it?—are questions only they, the funders, can answer. Say we measure a 25 percent drop in the truancy rate for a hundred kids in some program, and a 25 percent increase in their test scores. Is that worth $25,000 to you? Each donor needs to answer that question for him- or herself. As donors we will never be absolved of our responsibility to use our good judgment.
“Evaluation… the math anxiety of the philanthropic world.” What a great line. Evaluating nonprofits and social good in general is not about math. When I advocate for a financial markets-type approach to evaluation, I am not calling for number crunching. I wrote about this concept in January and will repost my comments from then rather than repeating myself:
Economics is often called the “dismal science”. I know that many people think that finance is boring. But the vision of financial markets as nothing but numbers and spreadsheets does not capture the reality. Do investors buy stock in Apple because they spent hours and hours processing spreadsheet calculations? No. While at the end of the day, buyers of Apple stock believe that the return on capital being generated by the company will make for a profitable investment, the information they use to determine that are not just numbers. The way in which Apple has captured the imagination of the consumer, (an intangible piece of data that cannot be added to a spreadsheet) is by far the most valuable asset that Apple has and it is a major reason why investors have flocked to the stock.
Have you ever watched CNBC, the news channel of the financial markets? It is far from some kind of spreadsheet crunching lecture. Every day, investors or all types come on the show and make passionate arguments for why certain companies are good investments. While numbers and calculations underlie much of their thinking, it is the story, the human story of the companies they discuss that take center stage.
Warren Buffet is widely considered the best for-profit investor of his generation. Does he sit in a corner office reading a spreadsheet the way that Phil suggests? The quote below is from noted investor Whitney Tilson (Tilson is a huge fan of Buffet and a fellow columnist of mine at the Financial Times):
If the future were predictable with any degree of precision, then valuation would be easy. But the future is inherently unpredictable, so valuation is hard — and it’s ambiguous. Good thinking about valuation is less about plugging numbers into a spreadsheet than weighing many competing factors and determining probabilities. It’s neither art nor science — it’s roughly equal amounts of both.
The lack of precision around valuation makes a lot of people uncomfortable. To deal with this discomfort, some people wrap themselves in the security blanket of complex discounted cash flow analyses. My view of these things is best summarized by this brief exchange at the 1996 Berkshire Hathaway annual meeting:
Charlie Munger (Berkshire Hathaway’s vice chairman) said, “Warren talks about these discounted cash flows. I’ve never seen him do one.”
“It’s true,” replied Buffett. “If (the value of a company) doesn’t just scream out at you, it’s too close.”
Taking liberties with Tilson’s quote, I would argue that donors should not “wrap themselves in the security blanket of metrics” because “the lack of precision around measuring the impact that nonprofits achieve makes them uncomfortable.”
World-class investors do not sit in their office crunching spreadsheets all day. Neither should world-class donors. But the underlying logic of both should be that of achieving the highest return on investment.