There are two types of "metrics” that philanthropy needs to figure out.
- Metrics used to evaluate nonprofit organizations or programs to determine if they should be supported.
- Metrics used to evaluate the impact or performance of philanthropic investments.
I believe that for the most part, the first set of metrics should not and will not ever be standardized. Experience with for-profit evaluation shows that even with the evaluation advantage of money being both an input and an output, investors focus on different metrics for different companies. Even when analyzing the same company, different investors focus on different metrics.
There will never be a set of universal metrics that allow for good evaluation across nonprofit organizations.
However, performance metrics are different. With the simplifying situation of cash being both input and output, for-profit investing has completely standardized performance reporting. How good a for-profit investor will perform in the future can be debated. But historical performance is objective and factual.
I think philanthropic performance will likely converge on a standardized evaluation framework. I’m wondering if the new government backed Social Innovation Fund will be the trigger that sets this in motion.
According to the White House Blog, “The Fund will identify the most promising, results-oriented non-profit programs and expand their reach throughout the country.” However, according to the actual Edward M. Kennedy Serve America Act that created the fund, the Social Innovation Fund will:
Award competitive matching grants to social entrepreneur venture funds in order to provide community organizations with the resources to replicate or expand proven solutions to community challenges…
Now realize how different those two statements are. The first suggests the fund managers will identify high impact nonprofits. The second states the fund managers will identify high performing venture philanthropy funders. I believe that this second strategy is best and will be the one that wins out. See my remarks in the comments section of this post for more.
The Social Innovation Fund is currently only $50 million. But it will participate in creating a pipeline of organizations that may very well get much more government funding once they’ve grown. Note that the 2010 federal budget calls for $8.5 billion to support already scaled Nurse-Family Partnership, an organization President Obama cited when talking about the Social Innovation Fund.
In order for the government fund to identify great venture philanthropy funds, they will need to evaluate the historical performance of these funds. If it becomes clear that big, big money will become available to organizations that make it through the scaling process, venture philanthropy funds will begin to actively compete for the attention of the Social Innovation Fund.
If this happens, the Fund will be in a position to demand a standardized set of performance metrics from the venture philanthropy funds to whom they are providing matching grants.
These metrics might not be perfect, but they will be standardized. I would suggest that it is very much in the interest of the philanthropic field to define these metrics in advance and encourage the Social Innovation Fund to adopt them. If we don’t, we may very well be stuck with bad performance metrics being broadly adopted. They will prove difficult to change once they are in place.
So the task is before us. How should the Social Innovation Fund evaluate the historical performance of philanthropic funders?
10 Comments
Interesting post Sean. (As always!)
For the most part, the SIF will use intermediaries to make and manage investments out of the fund. Each of the intermediaries will be required to leverage the SIF money by deploying matching amounts of private philanthropic capital into a public-private pool.
Although no single metric is a panacea, I agree strongly that there are promsing ways to consider using when measuring the performance of investors of growth capital (or, as we call it at NFF, “Philanthropic Equity”).
Here’s a description of what I would call “ROPE”, or Return on Philanthropic EQuity.
Nonprofits are in the business of turning money into program execution.
Investors of Philanthropic Equity are in the business of giving rise to nonprofits… (which, in turn, are in the business of turning OTHER PEOPLE’S MONEY into program execution).
The cost-benefit to society of that program execution is in the eyes of the beholder, and can be devilishly hard to quantify in a way that builds consensus.
But its easier to answer this question: “If Philanthopic Equity was what we used to build the nonprofit, then how much program execution ultimately came about, by virtue of its having been built?”
So, for example, about $80 million of Philanthropic Equity was required to get Nurse Family Partnership to the point where policy-makers will use the program to turn billions of dollars into nurse visits that change people’s lives and save tax dollars.
It took thirty years, and the $80 million paid for three scientific evaluations, a whole lot of tinkering with the program, and a long period where NFP hadn’t yet learned how to land government contracts. But finally now, they are nearing the point where government contracts actually pay for everything, and philanthropic support is no longer needed.
Thus, the price tag of building a sustainable NFP will be about $80 million of Philanthropic Equity.
Without the Philanthropic Equity, NFP would never have gotten the program right enough, proven enough, or well-sold enough to attract major streams of government expenditure.
And NFP isn’t quite there yet.
Soon, though, I think we may be able to claim that $80 million was the PRICE TAG of building NFP — a sustainable enterprise that, each year, turns, lets say, a billion dollars of government money into changed lives.
IN other words, an $80 million investment of Philanthropic Equity GAVE RISE to an annuity of, say $1 billion per year towards a program that, in turn, generates more than $1 billion per year of social benefit.
That’s a heck of a ROPE!
And here’s what echoes your point: We can quantify a heck of a return without having to pinpoint what the exact cost-benefit of the program is.
For example, there’s a good case to be made that every dollar of government money spent on NFP program execution eventually relieves tax payers of $6. If this is true, then, in our example, the $80 million investment of Philanthropic Equity is throwing off an annual social return of $6 billion per year, or a whopping 7,500 percent per year!
But what if people don’t agree that the cost-benefit is right? Well, even if the net cost-benefit were only $.50 for every government dollar spent, NFP would generate half a billion of net social benefit to tax payers PER YEAR.
Still very compelling. And much higher than an equity investment that fails to entice others into turning their money into program execution.
NFP is the goose that lays golden eggs, and all for the low price tag of $80 million.
At root, growth capital is all about building an enterprise that prompts OTHER PEOPLE into paying for program execution.
Build it and they might come.
If they come, then the ROPE is higher. If they don’t then the ROPE is a bust.
“I’m an investor of Philanthropic Equity. I love this program and think it has a great cost-benefit to society. I am going to use my equity to pay the bills while an enterprise gets built and learns how to become so compelling that OTHER PEOPLE will pay for the program I like so much. Those other people pay for program execution. I pay for building the firm they use to turn their money into program execution.”
So…. a key ROPE metric for SIF intermediaries might be: The ratio of program dollars compared to Philanthropic Equity, throughout the lifetime of a nonprofit firm.
Of course, it can’t just be the equity put in by the SIF intermediaries. It has to be the total amount of paid-in equity to the nonprofit, only some of which will come from the intermdiaries. And that’s where Philanthropic Equity accounting becomes very important…
But that’s another topic for another time!
ROPE. Hmm. I like that. Doesn’t have the financial markets branding “baggage” of SROI.
So, when tracking the performance of venture philanthropy funds, it sounds like you’re saying that we can simply track the program dollars/philanthropic equity. Much like Return on Equity (what Warren Buffett says is the most important metric in for-profit investing), but the “return” is measured by program dollars spent.
Am I correct that this methodology values $1 of program execution as $1 from a cost-benefit standpoint? In other words, if $1 in program execution saved $3 in tax payer dollars, the return would be even higher.
For-profit companies, even after they no longer need to raise outside equity, still retain earnings and therefore their equity base increases. Help me understand how the equity level changes after a nonprofit is done raising outside equity (growth capital). The money they take in each year as revenue is 1) spend on program, 2) spend on “overhead”, 3) retained in a reserve fund or endowment. Would program + reserved money/revenue be the “operating margin” and would the amount reserved be booked to “equity”?
One thing to remember is that the “Buyers” or providers of “Sustainable Business Model Revenues” may have rationales for their funding that differs radically from the social return thinking in the minds of other funders…. and that can be okay!
So, for example, I might like the program because it reduces net tax dollars. Someone else might like it because they have empathy for families in need. Someone else might like it because making a donation makes them look good in front of their peers.
The ROPE mentality stands above all this and says: “I as an investor of Philanthropic Equity, will help build an enterprise that entices other people to pay for the program I like, for reasons of my own.”
My sense is that the equity investors, who tend to write big checks, and who can afford to conduct deep due diligence, are most likely to leave emotion and self-aggrandizement out of the equation and to justify their investments using broad measures of social well-being.
My short answer to your retained earnings question is: You’re on the right track, but I believe it is more complicated than that. 🙂 Will try to post on this later….
Sean,
You know that I respect the rigorous focus of your writings. And, in that same spirit, I’d like to push back on two counts.
I would respectfully suggest that it is not philanthropy that needs to figure out the metrics to evaluate nonprofits, but rather that philanthropy has to provide the encourage, funding and support that will allow nonprofits to figure out the metrics they need to fulfill their mission and do their work. In attempts in other venues to “define metrics, there is a tension and risk that instead of focusing on the metrics most needed, people give in and move to those that are more readily available.
First, I believe that we should be doing more to encourage and enable those nonprofit leaders who really want to find ways to assess how well they are doing with they claim to do, using such successes as Youth Villages, Family Nurse Partnership, and a few select others, maybe like Latin American Youth Center as a starting point. And, I would suggest that the focus should be to:
assess how well they are achieving the social outcomes they claim for those they serve;
assess the community impact that the achievement these social outcomes should create; and
provide the indicators they need relative to running the non-programmatic side of their operation on issues of effectiveness, efficiency, financial health, human resource management and development and more.
When nonprofits have the impact, outcome, and output information they need to manage their own organizations and constituencies they serve, their donors and supporters will, in turn, have the information — quantitatively and qualitatively — they need to understand if their grants or investments are making the difference they envisioned.
One more point about the risk of moving to performance metrics is that it could be just as damaging as using an overhead number. Let me share a story from my corporate life. As you know, we helped organizations develop the metrics and performance reporting systems to manage their “IT plants,” and we were thoroughly immersed in this at both the theoretical and practical application levels. One of the lead industry magazines came out with a ranking system to assess IT performance. The irony was that a good number of those who enjoyed “good operational performance” metrics for this purpose, were being challenged within their own corporations for not being effective in helping the firm increase effectiveness, reduce costs, and/or gain differentiated competitive advantage. Certainly it is worth while to consider performance metrics as you suggest, so long as we remember they are, at best, a means to the true end.
In wrapping up, as long as we continue to place this responsibility on philanthropy versus nonprofit leadership, we will serve the wrong master and, ultimately, not achieve the systemic change the sector seeks.
Mario Morino
let me offer an example, from a strategy paper put to US government 3 years ago. It describes am investment mix of several components each with their own SROI and varying degrees of financial ROI. The design is such that less that full cost recovery components such as provision of family type homes for institutionalised children is balanced by more than full cost recovery from broadband rollout.
http://www.p-ced.com/projects/ukraine/national/
The call is for investment in partnership with another government and for the means of deployment via a social investment fund.
Impact to date has been governmnent agreement for creation of 400+ rehab centres for disabled children, doubling the adoption allowance and at Davos 2008, when Bill Gates described Creative Capitalism, the launch of the East Europe Foundation to promote CSR and support sustainable community enterprise.
This is based on a for-profit approach, in which those businesses wishing to do so, may contribute to increase the social investment fund. The delivery model, one of profit for social purpose, a business with both financial and social ROI, was conceived in the US 1996 and in 2004 was put into practice by a UK based social enterprise, whose social product is the research and development of the strategy paper linked above.
http://www.p-ced.com/about/history/
Here, here, Mario!
The key is putting talented leadership into a position where they can make the right choices and, with their board, choose measures along the way that best support their missions and strategies.
To that end, our standard deal terms call for reporting package that is a) shared across all investors of philanthropic equity and b) can be changed by management/board over time as needed to support the mission.
Here’s the language our clients typically use:
“Investor Report. Beginning in the first quarter of 2010 and ending fourth quarter 2016, Acme Nonprofit will produce a quarterly report that provides a comprehensive view of financial and programmatic results as compared to annual and quarterly operating objectives. From time to time, at the discretion of Acme Nonprofit’s management and board, the metrics included in the report may be changed to reflect Acme Nonprofit’s evolving nonprofit market dynamics. Each quarter, a copy of this report will be sent to each Philanthropic Equity Investor.”
So, the team and the board (not the external investors) set the annual operating plan goals. I believe that this is the way it needs to be.
In the meantime, there are a small number of long-term impact and sustainability goals that are spelled out in the prospectus, and can be referred to throughout the multi-year investment period. I believe the key is to avoid attempting to drive the selection near term goals from the removed and usually divergent perspectives of a group of funders.
In some cases, a lead funder may choose to participate more strongly in day to day stewardship. If so, they tend to do so in the context of the boardroom, which is the proper forum for stewardship.
Sean,
Thank you for raising such a critical issue. I would like to respectfully suggest adding a third “type” of metric: “Metrics used to evaluate nonprofit organizations or programs to enable nonprofit leaders to make decisions and improve performance”. These would be the similar to the metrics that you describe in Type 1, but the emphasis is on giving nonprofits the tools they need to choose and focus programs, improve services, and manage their organizations effectively. Ultimately, performance measurement should ensure the best programs are delivered to those in need and achieving social change.
To your second type of metrics, I’d like to share how Venture Philanthropy Partners measures its performance. But before I go into detail, let me emphasize that above all other metrics, the ultimate measure of success for VPP and its investors is social return – the benefit for children, youth, families and our communities.
We base our assessments of our investments on a performance to milestone system. We work closely with our investment partners (the nonprofits in which we invest) to establish mutually agreed upon annual milestones in our seven distinct areas of focus:
• clarity of mission and goals
• building strong senior management teams
• creating highly effective boards
• defining long-term financing strategies
• improving fiscal accountability
• creating systems of performance management
• improving program and service models.
As organizations develop their performance management systems, their leaders also report to us on achievement of outcomes for children and youth.
We have an extensive annual review process involving the management and board leadership to determine progress towards achieving milestones.
So our first measure of performance is the accomplishments and growth of our investment partners.
Another important metric of success for us is the growth and engagement of our investor community and our ability to leverage our funds on behalf of our investment partners. We also have annual milestones for own organizational capacity that we report bi-annually to our board.
Additionally, we have contracted with an outside resource, the Boston-based Chatham Group, to conduct two, comprehensive perception surveys about our work, one in 2004-2005 and another that has recently been completed. These surveys have been invaluable to us in improving our execution and effectiveness.
You can read more detail about our tools for assessment on our site:
http://www.vppartners.org/impact/assess.html
I believe any standard set of metrics to evaluate venture philanthropy/social investment funds, created by the Social Innovation Fund or others, should have three key components:
• The organizational strength and capacity of the organization including leadership (board and executive); fiscal stability (including ability to leverage funds and partner with others – none of us can do this alone); system of performance management; ability to identify successful nonprofit investments; thoughtfulness and rigor of model, and success of execution.
• Demonstrated performance — A track record of successful investments;
• And ultimately, the social benefit created for people and communities served.
As we say at VPP, our success is dependent on the success of our nonprofit partners. The ultimate goal is to improve the lives of others and the future for us all.
Carol Thompson Cole
President and CEO, Venture Philanthropy Partners
Sean….
I’d like to second a portion of what Mario said regarding metrics and nonprofits. In his comments he stated that philanthropy has to provide the “encouragement, funding and support that will allow nonprofits to figure out the metrics they need to fulfill their mission and do their work.” He went on to envision a day when “nonprofits have the impact, outcome, and output information they need to manage their own organizations and constituencies they serve.”
This is a point I have made several times in these pages and elsewhere.
Figuring out the appropriate metrics, or indicators, of meaningful and sustainable outcomes is a key to this situation. Understanding how this results-centered approach differs from more traditional accents on a problem to be solved or upon activities and outputs is even more seminal. Yet the “encouragement, funding and support” that would allow nonprofits to reach the necessary level of competence in these matters is often lacking.
For many years many well-meaning funders, both private and governmental, have primarily focused their dollars on what charities do, as opposed to how well they do it. Venture Philanthropy Partners and organizations like Social Venture Partners are beginning to change that paradigm by shifting a portion of their attentions to the ability of nonprofits to carry out their missions. Part of this is being accomplished through a vigorous examination of, and guidance on managing, the outcomes and metrics their partner charities adopt.
This is an inspiring start, but if it does not find echoes through larger portions of our sector, the success stories of organizations like VPP and SVP will remain the exception and not the rule.
Until the investor community does more to provide the wherewithal to allow the charities they support to figure out, understand, and use the metrics they need to fulfill their mission and do their work, much knowledge that is otherwise available and there for the asking will not benefit the vast majority of nonprofits and charities.
By the way, the above post speaks to the accountability relationship between EQUITY funders and the organizations they’ve invested in. These are what I sometimes call the Builders.
The other type of funder relationship would be the Buyer. Unlike Builders, Buyers aren’t about changing what the organization does. Rather, they pay the organization to do what it does so well.
This calls for a very different accountability setup. Very specific metrics can be laid out, for example, as to how much program execution is being paid for. And very specific mapping can be done to determine which part of the overall budget is being covered by the specific funder.
The key to remember is that the choice of program design and strategic direction would be largely predetermined. In a sense, the organization has decided what it wants to sell, and now the philanthropic Buyers (with strong accountability measures in place)are choosing to exchange their grants for the execution.
This is hard for an organization to pull off, unless it (with help from Builder funders) has put together a highly compelling plan and track record, so compelling that prospective Buyers will be brought into alignment. that’s why we sometimes say: “Philanthropic Equity is what pays the bills while an organization learns to be compelling enough to promote true Buyer behavior.”
Thanks to all of you for commenting. Unfortunately I did a terrible job of trying to explain my point in my original post. So I’ve taken another stab at making my point in a new post.
Let’s move all future comments to the new post.
Separately, I want to state that I agree completely with the point many of you made that much of the measurement conversation should be driven by the nonprofits, not funders. This is in fact the way it is done in financial markets where companies tend to set the frame for appropriate measurements of organizational performance, not investors.
But I was trying to speak to a different issue, which I’ve now tried to lay out in my new post.