My post earlier today and this one is an attempt to air the differing opinions that have been circling my recent posts (of course you can read the 30+ comments on this post to catch a whiff as well).
The many comments, emails and tweets that have pushed back and applauded my series of posts have been incredible useful to me. I’ve realized that there is one critically important element of a high performing organization that I did not highlight and yet is integral to the success of the model I’m advancing. The fact that this element even needs to be mentioned is one of the keys to understanding how radically different for-profit and nonprofit analysis is. I’ll examine this issue tomorrow.
Today I present a guest post from John MacIntosh of SeaChange Capital Partners, a nonprofit firm that seeks to arrange transformational fundings of outstanding nonprofit organizations.
By John MacIntosh
There’s no tension between “performance” and “impact” but they are very different and so it’s hard for one person to evaluate both well even though effective philanthropy sometimes requires it.
Let me start with Sean’s definitions:
· A high performance nonprofit is a very well run organization. It has outstanding leadership, clear goals, an ethic of monitoring performance and making adjustments as needed, and it is financially healthy.
· A high impact nonprofit is one whose efforts have been proven to cause sustainable, positive change.
First, Sean uses the ambiguous “efforts” when what he really means is “program”, “treatment” or “intervention” (collectively, “program”). “Efforts” hides the fact that that high impact is a feature of programs not organizations. In this respect, impact and performance apply to different kinds of things.
Second, the definition of “high performance” makes no mention of program! Would anyone claim that a nonprofit was high performance despite knowing that its "efforts" were believed (or proven!) to cause (or even be associated with) negative change?
Third, the tools for evaluating for “impact” and “performance” come from different disciplines. “Impact” is a concept from social science where ideally we define the treatment, develop measures of impact (wages, employment rates, test scores, etc.), identify a comparison or control group, worry about selection and omitted variable biases, organize for large Ns, and hope for p-values above 2.0!
“Performance” is about leadership, culture, management information systems, accounting, unrestricted net assets, opportunities for donor engagement, and the like. Few people I have met are equally comfortable in these two domains; very, very few are masters of both.
Here’s a more realistic definition:
A high performance nonprofit has a high impact program that it is likely to be able to deliver over time under a variety of changing conditions.
Under this definition, the question of performance can only be considered together with that of impact and Sean’s list of organizational virtues are means to the end of sustained impact.
Under this definition, there is no false dichotomy between performance and impact yet it is only over time and/or under changing conditions that high performance matters much over and above impact.
For a typical one-year grant made to a local nonprofit not going through any major change, impact is all that matters. If the program “works” the money will be well spent. If the organization falters because of a leadership change, shaky finances, or some other changing condition, then there won’t be a renewal. At the other end of the spectrum – multi-year growth capital grant to a growing organization already operating under various conditions in multiple sites – the question of “performance” is paramount since conditions will change over the course of the grant.
Performance may also be the best proxy for “impact” where impact is very difficult to measure directly. This might be because the time between treatment and effect is long, fidelity is a challenge, background knowledge is weak, or the program is new. Education programs often have all these characteristics. When I think about giving a drug to my children I want to know what it does but I don’t care who makes it or how. When I consider where to send my kids to elementary school I may not care much about test scores but I really want to meet the principal.
Let me finish with a concrete example:
Uncommon Schools is a charter management organization currently operating 11 schools across New York and New Jersey. SeaChange Capital Partners is arranging a $30 million financing to support their growth to 33 schools over the next five years of which we have committed $2.0 million of our own funds.
Uncommon’s program – its 11 schools that are already open — is high impact as measured by a combination of (i) the academic results and college enrollment rates of its students relative to the relevant comparison groups (including through the natural experiment of over-subscribed lotteries for admission) and (ii) the opinion of well-informed experts in the field of education. This "high impact" represents a very cost effective philanthropic opportunity because Uncommon, as an operator of public schools, receives approximately $7-$10 dollar in public money for each $1 of philanthropy.
However, none of the $30 million will be used to support the historical performance of the current schools! A small portion will be used to support these schools over the next five years but most will be used to open 22 schools. In considering this grant the relevant question is: will the new schools be high impact and will the current schools stay high impact? Unfortunately, there is no magic formula for schooling; every school is different in its location, staffing, leadership, etc. so the ONLY way to judge (today) whether the future schools will be high impact is to assess whether the organization that will open and manage them is high performance.
We believe that Uncommon is high performance because of its leadership (a team of six with deep experience and an organization that has already gone through the transition from the initial founder), its track record of success (four years as Uncommon; 12+ years as individuals), its culture (self-reflective and relentless on behalf of children), its systems (for instructing new teachers, inspecting schools, using data to tailor instruction, budgeting and forecasting), its use of real-time measures delivered through dashboards, its financial health/model, and its well-developed plan for growth. None of these are virtues on their own – they are simply what gives us confidence that we’ll be more likely to get what we really want – more high impact program — in the future.
* * *
One theory of change says that “if grants flow to organizations with high impact programs then they will grow and the others will shrink”. Well maybe. But in the interim executive directors and board members need to think about what they can do now in the service of higher impact. What that should do is strive to make their organizations higher performing. (There may be cases where the program is so deeply flawed – a program based on “praying for grades” – that they should just shut it down but it is rare that a program is so self-evidently bad and that even the organization would acknowledge it.) “High impact” is no excuse to rest for conditions will change. “Low impact” should not be a death sentence but a call to action.