The “Evaluation Revolution” & Problems with Measuring Nonprofits

In preparation for tomorrow’s Metrics Mania debate at the Hudson Institute, Gary Walker (former president of Public/Private Ventures) has written an essay titled “Reflections on the ‘Evaluation Revolution'”. As I write about the need for more evaluation of nonprofits and more funding for high performing nonprofits (and less for low performing nonprofits) I realize that many readers assume I am calling for more of the outcome and impact studies that have been used for some time. In fact, I believe that the kind of analysis needed is quite different.

The evaluations that Walker so eloquently discusses use a scientific framework. A framework that says we can quantitatively measure certain types of outcomes and that these results should be repeatable. This framework is borrowed from the hard sciences and I wonder how applicable they are in the social sciences. I believe that a better evaluation framework is that used in financial markets. Finance is not a hard science, it is a social science. One of my favorite books on investing is Investing: The Last Liberal Art by Robert Hagstrom in which the author lays out an approach to investing that draws on the lessons of literature, psychology, sociology and philosophy.

In financial markets much times is spent performing massive historical studies of what has or has not worked in the past. But every new investment decision must be made with the understanding that we’re not talking about physics here. Experiments are not repeatable. What worked last time, might not work this time. However, in the social sciences and in finance, we can study what has worked in the past and we can perform rational analysis about what might work in the future.

I’ve taken criticism in the past from people like Phil Cubeta who have suggested that I see philanthropy as a desk job where spreadsheets are run all day long. That is simply wrong and does not capture the kind of proactive philanthropic decision making within a social capital market framework that I think will lead to a nonprofit sector that performs at a much higher level.

I urge you to read Walker’s essay for a deconstruction of why traditional evaluation has not lived up to the promise.