Yesterday I wrote about the importance of nonprofits tracking their outputs, outcomes and impact to the best of their ability. Reader Nick Temple of the School for Social Entrepreneurs commented:
“[Nonprofits should see outputs, outcomes and impact]… not just as ‘reporting’ to donors, philanthropists and investors of all types, but also an important part of improving an organisation’s work: we need to prove that what we are doing is making a difference, but we also need to improve those services wherever possible, learning from what we’ve done so far.”
This point of view is one that I agree with completely and one that I think needs to be more broadly accepted for nonprofits to fully embrace tracking their results.
In our current system, funders typically decide what sorts of reports they want to see from nonprofits. Nonprofits then have to create these reports for each funder. These might be elaborate, custom reports for major foundations or it might require spending time responding to the idiosyncratic requests of an individual major donor.
I believe that nonprofits are best positioned to know what information is most important for them to track. The most relevant information is that which reveals how well a nonprofit is performing and the degree to which its programs are making a difference. This information should be mission critical for the nonprofit, even if no funders are asking for reports. It is also the information that funders should be most interested in receiving.
This means that high performing nonprofits should be tracking their outputs, outcomes and impact for their own purposes and funders should be seeking to support these sorts of high performing nonprofits. The reporting burden then becomes much lighter on the nonprofit and specialized reporting might only be available for very large donors.
For too long, the conversation around measuring and tracking results has been framed as a sort of finger wagging campaign by the funding side of the table who often waste their grantees time by requesting information that is of little relevance and which ends up sitting on the shelf at the foundation.
But nonprofits perpetuate this dysfunction by not bothering to track mission critical outputs, outcomes and impact.
This is why my definition of a high performing nonprofit focuses on how they interact with the data they track. While nonprofits may often view the process of collecting this data as a waste of time, that is only because the dysfunctional grantee/funder relationship drives nonprofits to report on information with little value to their work. But if a nonprofits leads the creation of their data tracking process, the result is a better run organization, that has more impact and which can far more easily fulfill the understandable reporting needs of funders.
High performing nonprofits…
- …base their programs on research about what works.
- …actively collect information about the results of their programs.
- …systematically analyze this information.
- …adjust their activities in response to new information.
- …operate with an absolute focus on producing results.
Thinking from the non-profit side, I think the reason we focus on “outputs” more than “outcomes” isn’t just that outputs are easier to measure. I think it’s also that outputs are much easier to influence. In the comments to the first post, I was really struck by Melinda’s question — “what will you do with the data once you have it?” and I’m glad you drew attention to it in this second post. It’s such a critical issue and it’s challenging when you’re dealing with outcomes or even impact data. Outputs you can control: you know if you gave away more food or less food and if you spent more hours exercising. Knowing what to do with outcomes data is a real challenge.
I think non-profits are often nervous about collecting and sharing outcomes data because it inevitably includes lots of variables they cannot control. Everything from the economy to quirks of the particular population they serve might be influencing the outcomes. Staff members fear being compared to each other or with other organizations when there is so much they can’t control influencing the metrics. The result is concern (both founded and unfounded) among non-profit staff members that outcomes or impact data will be used irresponsibly by their managers, their board members, and most especially by their funders. Staff members have to reach a place where they feel they know how to handle data and can trust that others will handle it responsibly too. You need quite strong leadership in a non-profit to have the staff trust that the data will be used responsibly internally. You also need strong partnerships with funders to trust that the data will be viewed responsibly externally.
The same impact focus has to guide the fundraising side of a nonprofit’s work. The same commitment to data-gathering and data study has to be made. The same willingness to use the data to recalibrate and re-act has to be embraced. But it isn’t. All too often, disappointing fundraising results are swept under the rug for fear of retribution and public/media/donor outrage. We are systematically stifling the data pool on fundraising activity. In many cases, you’d literally need forensic accountants to uncover the outcomes of specific fundraising activities, either because they get buried in the organizations overall accounting in order to soften blemishes, or joint cost allocations are applied without obvious or detailed disclosure for the same purpose.
That we have shame about, instead of interest in, disappointing fundraising results, and that we inflate results to make costs seem lower than what they actually are, is perhaps the greatest tragedy occurring in the sector today. because if we don’t understand fundraising we can never achieve the kind of program impact we seek.
I agree with you that organizations burying their failed fundraising attempts don’t help anyone, and that if we want the sector to evolve to be more efficient, we must share this information. On a large scale, this (external) sharing is extremely important. We cannot move forward without it.
However, for individual organizations, there’s a lot to lose in sharing this information (and I don’t think organizations are *too* at fault for keeping lessons internal while they are still kicking). After all, donors may be swayed not to give to organizations that publicly “fail.” And bad press is a realistic concern with significant consequences for these kinds of organizations. It’s easy to say something like “share your organization’s failures to contribute to sector evolution”… but when you consider that these organizations’ own health could suffer even more if they go public with failure, it just makes sense that most bury the evidence. From a business perspective, that’s what they should do to survive.
The trick, I think, is to change public perception of nonprofits to allow them to take risks and share stories without being branded as an organization “wasting” money that’s supposed to go to a good cause. I don’t know how to get to this point, but right now organizations with failed fundraising attempts have two decisions: 1) go public with it and risk bad press, which could lead to loss of donors, funding, and grant leverage, but helps the sector in an unmeasurable way in the future. or 2) keep the lessons internal and forgo aiding the sector in this sense but maintain your grasp on serving your mission.
One decision really isn’t better than the other. Perhaps the key is to change the decision. What do we need to do in society to allow nonprofits the room that they need to evolve?
Thanks for including me in the post, Sean. Much appreciated, and glad it resonated.
I really like some of the comments above. To Emily’s point about measuring what’s easy, I also like the phrase “It’s better to be vaguely right than precisely wrong” in this context; which means it’s better to have a proper attempt to measure the difficult, softer, intangible stuff (and be transparent about how you’ve gone about it), than ignore it altogether and be “correct”.
And to Dan’s point about sweeping bad stuff under the carpet, we’ve always found that communicating the bad along with the good (or the learning points along with the successes) only serves to build credibility in the organisation. [Admittedly in relation to outcomes / impact, rather than fundraising per se.] But that openness and honesty, though it can feel counter-intuitive, is what I try and get across to our early-stage social entrepreneurs. Or as a funder over here put it recently, “transparency leads to credibility which leads to trust which leads to support”.
Great posts and debate.
Appreciate your comments.
You put the qualification about the fact that you’re admittedly not referring to fundraising in brackets, as if it’s a minor matter. You’re not alone. That’s the way most of the world is conditioned to think about the fundraising side of things – you know – no harm if it doesn’t apply to fundraising so much, as the program side of things is where the rubber really meets the road. But the fact that the social change sector doesn’t find fundraising as academically interesting as the program side is significant, because fundraising makes the program side possible.
And the principle of openness and honesty works differently with fundraising, sadly, as Colleen rightly points out above.
For example, my company was a model of transparency. Check out the granular detail on fundraising results at http://www.pallottateamworks.com/financial_detailed.php . But this practice of disclosure absolutely worked against us. The media used our own data to denigrate us. The folks that followed after us learned that lesson, and so don’t publish any data like this. They are silent on the matter. Like the military’s “Don’t Ask, Don’t Tell” policy, we are institutionalizing camouflaging – and a kind of dishonesty – when it comes to fundraising.
I really do appreciate your post, but I find constructing the disclaimer about fundraising as a parenthetical to be fascinating and endemic.
Dan, maybe there is way to work through a transitional strategy: have fundraisers report to an independent body (coalition) who reports in the aggregate about lessons learned with certain types of fundraising efforts. This type of reporting could ‘protect’ the independent agency and yet the lessons would not be lost to the sector.
Also, I think the issue is really about risk. NFPs are not comfortable with risk for some of the reasons stated above (and I absolutely believe work is needed to support funders to get to this point), yet in the FP sector risks are encouraged with the belief that in order to find the next big thing you have to fail – learn from that failure and move on. Of course this is an oversimplification of the issue but the fact remains, we need to create a culture that embraces risks if we are going to enable the innovation needed to help us solve some of our intractable social issues.
Hi Dan – sorry, that wasn’t what I meant. My point was agreeing with you about transparency, and that this builds trust in, as you say, enlightened funders + donors. The only reason I put it in parentheses is because we are not an organisation that raises funds in this type of way, so have no experience of transparency in this context/way (i.e. we don’t run fundraising events, get direct donations, legacies, as per ‘traditional’ charity…), or indeed on this scale.
I would agree that predominantly learning of this sort, about what’s worked and what hasn’t for fundraising, gets used internally in larger organisations (I was listening to some Institute of Fundraising podcasts the other day that were advocating exactly that process for making judgements internally…but nothing about sharing externally)
I would also agree that data alone isn’t enough (a point often made when charities are criticised for their %administration costs and so on). How we generate culture change and awareness without penalising the pioneers is a valid question to raise. Perhaps governments can take a line in leading also? Our current coalition government is making more and more data (about spending + costs especially) open to all which is certainly opening up the dialogue and conversation more across the piece: http://www.conservatives.com/Policy/Where_we_stand/Technology.aspx
Btw, Allyson, I totally agree about your point about risk. Risk awareness, learning by doing; not risk aversion and safety through inaction.
Per the transparency issue, the new DISCLOSE act being debated in Congress would require more donor disclosure by non-profits. This is definitely leading to some fear and panic in the NGO sector, but it would mandate more sunshine.