I just read the fascinating article “What Makes People Give” from the March 9, 2008 edition of the New York Times Magazine. The article chronicles the attempts by John List and Dean Karlan (economists at Yale and the University of Chicago), to understand why people give.
List and Karlan considered the usual answers (to make the world a better place, to see your name printed in the back of an annual report and the like) too pat, too simple — and sometimes just wrong. Over the years, whenever one of them asked fund-raisers why they did what they did, the responses were vague and unimpressive. There didn’t seem to be much empirical evidence to support the strategies employed by most fund-raisers. So the two economists wondered whether charities were wasting a lot of effort…
…When charities are designing their donor appeals, they often go by nothing more than a few rules of thumb, some of which may be profoundly insightful and others a good deal less so. “I think some fund-raisers have developed terrific intuitions, passed on through the fraternity of fund-raisers,” says Paul Brest, president of the William and Flora Hewlett Foundation in Menlo Park, Calif., which often works with charities. “But a lot of the intuitions don’t work. Look at how much junk mail you get.” Matching gifts were another good example. People figured that they worked, because — well, how could they not? They seem so sensible…
The story reminds me of two of my favorite books, Moneyball and Freakonomics. In Moneyball, Michael Lewis studied the Oakland A’s use of statistical analysis to drive the way they built their baseball team and played the game. In Freakonomics, Steven Levitt and Stephen Dubner used economic analysis techniques to understand falling crime rates, the organizational structure of street gangs and the inner workings of professional sumo wrestling.
What both books (and the NY Times article) use as their premise, is that quantitative analysis is incredibly useful in understanding our world. Yet all three also understood that statistics do not themselves give you answers, they just help you understand your environment better so that you can more easily find the answers you are looking for. This is the promise of metrics and other statistically, quantitative measurements in philanthropy. They are not themselves the answers we seek, but they help describe the world we live in.
When used as tools to advance our understanding, metrics in philanthropy are wonderful. But when viewed as some sort magical answer that shows us the Truth, we are better off with Mark Twain as a source of insight than Moneyball or Freakonomics:
“There are three types of lies – lies, damn lies, and statistics.” – Mark Twain