My post on Friday set off a heated debate over whether measuring organizational performance was actually more important than measuring impact. Remember, I didn’t argue that high performance nonprofits are better than high impact nonprofits, I said that high impact is the holy grail and that high performing organizations are best positioned to complete the journey towards high impact.
Three comments from highly respected, senior people in philanthropy caught my attention in particular.
Mario Marino, founder of Venture Philanthropy Partners wrote:
I’d ask you to consider a further delineation that takes this one premise into account: “good operating performance” does not necessarily translate to ‘good social benefit and results,”…
As this discussion goes forward, I only hope we never lose sight that the discussion of “high performance” should be always be in the context of the benefit/value for those served. It can be alluring to focus on operational effectiveness, while losing sight of what it is all about. And, this is why it gives me great concern that some are seeking to use “operational metrics” as a way to assess nonprofits.
David Hunter, a social and public sector consultant and former Director of Evaluation and Knowledge Development at the Edna McConnell Clark Foundation wrote:
I think the distinction that Sean makes between “high performance” and “high impact” nonprofits is conceptually precise but in a practical sense not of much help…a spurious precision if you will. In the nonprofit sector – which, it is useful to recall, was specifically created to address public needs – what pragmatic sense does it make to applaud “high performance” that does not include a notion of effectiveness in meeting public needs and improving the conditions or consequences of those needs? To do so trivializes the concept of “high performance”…which is at least an aspect of what I took Mario to be saying.
George Overholser, founder of NFF Capital Partners, a division of the Nonprofit Finance Fund offered a short but critically important comment:
I would simply add that one hallmark of high performance is the presence of a clear road map to high impact.
So the question remains, does identifying and funding high performing organizations offer the best path to philanthropy creating high impact? Or does this strategy run too great a risk of funding “good operating performance” that doesn’t turn into “good social benefit and results”? In other words, is the correlation between high performance and high impact simply spurious?
My answer in my next post.
7 Comments
All good stuff, indeed, but it might be helpful to move beyond “theoretical” positioning and start to share some concrete examples that differentiate between “high performing” and “high impact.” Maybe your readers can provide some links to case studies, reports, even overviews of what they consider to be organizations having high impact. If nothing else, those organizations probably deserve the attention.
I agree with you Bruce, I think it’s time for hard examples rather than theory.
Sean, I agree with your clarification: “Remember, I didn’t argue that high performance nonprofits are better than high impact nonprofits, I said that high impact is the holy grail and that high performing organizations are best positioned to complete the journey towards high impact.” I think the best non-profits are both high impact and high performing, and it should be the goal of every non-profit to be such.
As I commented on Holden’s blog (http://blog.givewell.net/?p=403), it would be helpful to have a baseline of expectations when defining “high impact” and “high performance”. Could you please provide us with your definitions to ensure everyone is on the same page?
Kristen
Kristen & Bruce,
I’m working on it. I think high impact is pretty easy. It is an organization that has produced proven impact. Organizations like this are few and far between. Very few are able to prove their impact.
I’ve started defining high performance here.
Adding a further wrinkle to this highly evocative conversation about impact v. performance, I would only like to suggest that some allowance be made for context. An organization’s local operating environment is frequently very difficult to overcome. Should they not be funded, despite being the only game in town, if they don’t measure up to some absolute standard of impact (or performance, for that matter)? We wrestle with this all the time and end up paying a lot of attention to anecdotal evidence that speaks more to our gut than to our “evaluation matrices.”
Alex,
Yes context matters a ton. I’m a big fan of the concept, “don’t let the perfect be the enemy of the good.”
Impact Evaluation is defined differently for different groups. But one area of interest is the issue of quantitative impact evaluation.
This type of evaluation is focused on the issue of attribution. More
specifically, quantitative impact evaluation isolates the welfare
effect of a specific project by comparing the actual observed
outcomes of project participants with counterfactual outcomes,
i.e., the hypothetical outcomes that would have prevailed in the
absence of the project. Since people are either in or not in the
project and cannot be both, these hypothetical counterfactual
outcomes cannot be observed.
The central objective of
quantitative impact evaluation is to estimate these unobserved
counterfactual outcomes.
Because of this counterfactual analysis, quantitative impact
evaluation makes possible clear specification of the project
impact being estimated. It is therefore generally regarded as
more authoritative and is usually referred to as rigorous impact
evaluation.
Our organization sponsored a conference on this topic in March.
See:
http://www.impactevaluation2009.org
Thanks Ron. I’ll check it out.