Most donors, both individuals and institutions, want to understand the impact of their donation. When you buy a consumer product, you get something tangible in return and you can evaluate your purchase directly. When you buy stock in a for-profit company, you get feedback via the performance of the stock and/or the financial performance of the company. But when you donate to a nonprofit, most people don’t have any sense of what their money actually does.
Some nonprofits try to remedy this by creating “gift opportunities” (ie. “Your $100 will provide immunizations for baby Sarah and her two siblings”). Frequently, this “offer” are not actually literal (the $100 will provide immunizations for three kids, but your money will not specifically buy the shots that go to Sarah and her siblings).
For large grants, nonprofits are often under pressure to report on the impact of the specific grant on the nonprofit’s operations. But what is missed when this request is made, is that money is fungible. Fungible means that it is interchangeable. If you give $10,000 to a nonprofit for their new childcare center, did your money really go to the childcare center? What if before your grant, the nonprofit had budgeted $100,000 of their existing funds for the center and after your grant they reduce their budget to $90,000. “Your” $10,000 goes to the childcare center, but the center still has the same budget before and after your grant.
I think that trying to measure the impact of specific grants doesn’t generally make a lot of sense. In the for-profit sector, companies measure things like “Return on Invested Capital” so that investors can get a sense of what the company achieved with the money they spent on the business. Since we’re never going to be able to numerically calculate the “good” that a nonprofits does, the nonprofit sector will never have a true metric that works as well as ones used by for-profits. But the concept is still valuable. For-profit businesses are not as numbers drive as many people think. Advertising is a perfect example; you cannot truly measure the return on advertisement spending because you can’t really connect ads to sales. But even without strong metrics, companies spend money on advertising because they know it works. They know it works through using both qualitative and quantitative evaluation. Even without a single, easily measured ratio, companies still know that advertising works.
So weaving these various concepts together, my point is that as nice as it would be to have tangible, numerical feedback, it is not going to happen and it is not necessary. Trying to force the issue, requiring tangible, numerical feedback when it is not possible, is a destructive process. How do you know someone loves you? Imagine how misleading it would be to try to assign numerical scores to the relationships in your life. But that does not mean that you don’t need feedback from your friends and family.
Nonprofits need to demonstrate that they do great things with the donations they receive; they need to prove that they are high-impact enterprises. Donors need to accept that there is never going to be a numerical score that determines if a nonprofit is doing a good job. The work of nonprofits, the work of social benefit, is far too messy and far too wonderful to be fully captured by statistics.
Update: A comment from Bruce Trachtenberg spurs me to clarify that I don’t believe that there is any single metric that will work across multiple cause. And I think that focusing too heavily on metrics can lead to misleading conclusions. But I definitely believe that evaluation is critical and that good evaluation will bring together both quantitative metrics and qualitative observations.
9 Comments
Yes, we need good metrics, indicators, and data.
But, what amazes me is how so many donorati and philanthropoids cling to the notions that if we just map out the system and do the ROI, everything will fall into place.
Here’s why I think we get stuck in a groundhog day conversation about measuring impact:
We never took that class in research because it required algebra.
We have not read the research in our own fields because we are too busy making tiny $10,000 grants and tying them to unrealistic expectations.
We have not looked at how healthcare, government, business does a sketchy job of ROI and impact measurement on a wide range of issues.
We think “fungible” is something used to epoxy roof shingles.
We have never funged, funds or really seen how the money really flows in a nonprofit.
—-Your point about not forcing the issue is key.
Lets not force the issue.
Maybe we can finesse it for a change.
Sean,
I wouldn’t be so quick to give up on the hope that you can come up with meaningful metrics to help demonstrate social return. In fact one nonprofit that you’ve featured before, Nurse Family Partnership, has demonstrated that for every dollar invested in the program, the public receives a return of $2.28 in reduced outlays for such things incarceration, remediation, government assistance, along with increased tax payments from newly employed parents. Maybe you can’t tie performance to every grant or donation. But any organization committed to “doing good” should be able to show what outcomes they are achieving and the cost of achieving those returns. That’s all measurable, meaningful, and gives donors an ability to discern which organizations are doing better compared to others addressing the same issues or needs.
For generations, American philanthropists and charitable foundations have given money to improve the lives of others. For generations, these charitable gifts and grants have flowed as “tax-privileged” dollars, neither incurring nor paying taxes.
Tax-privileged foundation assets reached a record $550.6 billion in 2005.
And overall, foundation grant dollars reached an estimated $40.7 billion in 2006, up from $36.4 billion in 2005 (Foundation Center).
Charity is no small business, and $41 billion is no small investment. But for foundation grants, the IRS requires reports of transactions, not analysis of results. Project evaluations, when done, typically come after all the money is spent. In the world of foundations, good intentions have usually been good enough. But with the advent of the Internet and online databases, the world of data collection, aggregation and analysis has changed. Reporting impact is difficult, but not impossible.
With $41 billion of tax-free cash flowing each year, are good intentions and transaction reports really good enough? What will provide the motivation to move from announcing good intentions to documenting genuine impact?
Mary, what do you think will “provide the motivation to move from announcing good intentions to documenting genuine impact”?
“In fact one nonprofit that you’ve featured before, Nurse Family Partnership, has demonstrated that for every dollar invested in the program, the public receives a return of $2.28 in reduced outlays for such things incarceration, remediation, government assistance, along with increased tax payments from newly employed parents.”
BS BS BS BS BS BS BS BS!
See Tidy Sum’s point: “We never took that class in research because it required algebra … We have not read the research in our own fields … We have not looked at how healthcare, government, business does a sketchy job of ROI and impact measurement”
Being guilty of all these mistakes is the ONLY way you could ever take a number like that “$2.28” at face value.
Predicting how a program will change people’s lives, relative to how they would have changed otherwise, and how that will affect the incredibly complex interactions of the public and private sector to reduce “public spending” simply CANNOT be done to that level of precision.
Read all the details of how they calculated that number and see how good you feel about it. I bet the analysis itself will even explicitly say, “This is the roughest of rough rough estimates” – analysis like that always does – but of course, that gets buried once people have the number to wave around out of context.
Which is exactly why Sean’s point is dead on. You can’t and won’t take the human judgment out of giving. What you can do is make it more informed and more transparent.
Nice post sean. As the woman who manages Paul Wolfowitz’ money once said to me, if you make giving look like investing, you lose the payoff you get from giving. Making something that has the narrative value characteristics of a luxury good conform to commodity centric cost benefit analysis turns measurement into a net value detractor, imho, as business like as it feels from an aspirational standpoint.
some value retains its value only when it is encapsulated in a narrative. some value loses value when you break it down into it’s pure numerical form. the iphone is a luxury good with elastic pricing. the trio is a hand held communications device bought on a cost benefit basis. where would you rather be?
Kevin, I think you make a great point. How would you respond to this post on Holden’s blog, which suggests that framing “giving” as a “luxury good” is “crass and misguided and yuck”?
if you equate luxury good with, say, needlessly frivolous perfumes, etc. i completely agree. but the point is that prices are elastic in that category; people don’t think about price and cost benefit, they care about it, want it, feel they need it in some way. giving is not a luxury good, but the decisions around value are more in that camp than in commodity goods. making giving look investing reduces it to the commodity level where people slice and dice it. that framing takes away its real value.
You mean like leaving 12 million for a dog?