This is my latest column for the Chronicle of Philanthropy. You will find the full archive of my past columns here.
Probing Questions All Donors Should Ask Before Making a Significant Gift
By Sean Stannard-Stockton | Chronicle of Philanthropy
One of the holy grails of nonprofit evaluation is to be able to compare nonprofits with different missions. Concepts like “social return on investment” strive to measure how much “good” an organization is creating, regardless of whether it is a soup kitchen or a job-training program. Given the difficulties of comparing the results of different types of organizations, it makes more sense for potential donors to ask a specific set of questions of all organizations.
Successful programs often look quite different from one another. However, high-performing organizations, those that have the ability to carry out successful programs, have similar characteristics. These organizations base their programs on research about what works, actively collect information about the results of their programs, systematically analyze this information, adjust their activities in response to new information, and focus all their energy on producing results.
To figure out whether nonprofits meet those standards, donors can ask specific questions that will help them decide whether a group is worth supporting. Each answer must be interpreted in the context of the nonprofit’s operations.
For instance, it would be unreasonable to expect a small grass-roots organization to present extensive evidence supporting its answers. However, no matter how big or small the group, donors can still assess whether an organization has the proclivity to become a high-performing group and whether it has put in place all that it can to move toward that goal. Even most large organizations do not possess all of the attributes of a high-performing organization. It is important that donors not let the perfect be the enemy of the good.
With that caveat in mind, here are the questions that donors should ask when they are considering a significant gift.
On what research or evidence did the organization design its programs? Whether you are evaluating a local after-school tutoring program or a global disaster-relief program, a high-performing nonprofit should be able to speak about the evidence and research that shaped its programs. While many organizations have not conducted extensive evaluations of their programs’ results, all programs should at least be based on knowledge about what works. When a nonprofit is exploring an unproven approach, it is critical that the program is treated as a research project to test an idea—and that donors are told that.
What information does the nonprofit collect about the results of its programs?For-profit organizations can track their revenue and expenses to determine exactly how much profit they are producing. Nonprofits need to track not only their financial transactions but also the social results that their programs achieve. The relevant information will vary at different organizations. However, all high-performing nonprofits should be making a consistent effort to collect the information that they believe is most relevant to measuring progress toward their goals.
How does the organization systematically analyze the information it collects? It is not enough simply to collect information; the whole point of gathering data is to better understand a situation. Whether information analysis takes the form of sophisticated statistical analysis or simply regularly scheduled reviews and discussion among the board and staff members, high-performing nonprofits should diligently attempt to understand the meaning of the information they collect.
How has the nonprofit adjusted its activities in response to new information?Unfortunately, knowing what should be done is not enough to spur every organization to action, just as knowing that eating less and exercising more are the keys to losing weight does not guarantee a successful diet. Nonprofits must be ready and able to adjust their activities as needed. High-performing nonprofits should be able to discuss specific instances in which they responded to new information by stopping or significantly altering their activities.
Does the organization have an absolute focus on producing results? In the business world, the intended result—a profit—is also the fuel that sustains the organization. In the nonprofit world, program results don’t usually pay the rent. That means that nonprofits are at risk of giving higher priority to fund raising and other revenue-generating activities than producing program results. Producing the revenue needed to run an organization is critical, but it is a means to an end. Sustaining an organization is useful only to the extent that it enables the delivery of program results. High-performing nonprofits should be able to speak convincingly to their absolute focus on results.
Plenty of nonprofit groups can show financial success. But donors need to ask the probing questions that will make sure that dollars flow only to organizations that turn financial resources into program results.
Sean Stannard-Stockton is chief executive of Tactical Philanthropy Advisors, in Burlingame, Calif., and author of the Tactical Philanthropy blog. He is a regular columnist for The Chronicle of Philanthropy.
16 Comments
With the surge in conversation about metrics and effective measurement, these questions come at a great time. I’m curious about your recommendations for how a nonprofit should communicate the answers to these questions. I’ve been involved in dialogue within my organization and others about posting this info on the web site, in annual reports, etc. But our analytics tell a different story about how this type of information is used. Your recommendations? What’s most effective for nonprofits to provide these answer to prospective donors?
Great question.
I think these questions should generally be answered in plain English. Don’t think of them as technical questions, think about them being posed to you by a friend who doesn’t really understand your work all that well.
Imagine the questions being posed informally over dinner or a drink:
***
Heather, how did you figure out the best way to help kids do better in school (for example)?
What a great idea! What do you look for to know you are making progress?
That’s so interest! What do you do you with all that information you collect?
That makes sense. Have you run into any problems or figured out better ways to do things since you started?
It sounds like your organizations is great, but it must take a lot of money to run. How do you maintain your focus on the kids when you have to fundraise and run the organization?
***
I think that we too often frame analysis as a spreadsheet crunching process that is so technical it can’t be understood by mere mortals. I wrote this column to layout a commonsense approach to thinking about whether a nonprofit is good at what they do.
The best way to answer these questions, in person or on your website, is to explain your answers in a way that is compelling and easy to understand.
“If you can’t explain something simply, you don’t understand it well enough”
Some orgs, like media ministries broadcasting into closed or nearly closed countries in th Middle East cannot, or almost can’t, measure results as some Amer foundations drone results.
As some foundations define results.
Gray, you bring up an important point. That’s part of the reason I wrote this column. See my answer to Heather above. This exercise isn’t about quantifying your impact, it is about explain how you do what you do.
Mr. Stannard-Stockton,
I enjoyed the post and find your underlying premise intriguing. I am not the biggest fan of financial ratios and metrics. They serve a great purpose but as you point out it can be a misleading tool of comparison. Additionally, when so much emphasis is placed on those figures, I begin to question the motives of the organizations’ leadership. Is their mission to help others or to generate favorable expense ratios? I always try to use those metrics as a deal breaker; nonprofits must first show that they can efficiently manage their donations, but the qualitative and intangibles tend to paint a better picture for me.
If I could offer one suggestion, it would be to elaborate on the fifth question posed. I, too, believe that the goal of a nonprofit should be a heavy focus on the actual programming, but I fear that some donors (and even nonprofits for that matter) do not place enough value on internal investment. Many of these donors may be looking for large, established nonprofits. If that is where they feel their dollar will be most effective so be it, who am I to argue with them? However, I tend to think that nonprofits would do well by acknowledging the success of their for-profit counterparts who place a major emphasis on investing within themselves. I believe that in the long run, donors would find they might do a great deal of good by allowing these companies to invest in quality infrastructure and employees. Better IT systems and personnel development programs increase efficiency just as higher salaries attract more qualified candidates. I am not convinced that raises in nonprofit employee pay will ever be seen as an acceptable use of contributions, but hopefully donors will realize the importance of other forms of internal investment.
Tom, I definitely agree. See this old column I wrote on this topic.
When I write that a nonprofit should have an absolute focus on producing results, I don’t mean they should prioritize funding their programs over funding their internal infrastructure. Instead, I mean they should make every decision, both strategic and financial, based on a goal to produce results (as opposed to simply perpetuate the organization).
These questions are important but I believe they are only a beginning and solid data can and must support the answers provided. This is neither as difficult as it sounds nor as focused on resource development as implied. For example, a museum could provide information on how many constituents are served, the demographic composition of that audience and how it is changing year over year. This could be done by administering a brief survey to all those entering the institution. The results would help to demonstrate how well the organization serves various communities, its success in reaching underserved audiences and any changes on a longitudinal basis. It would also help the institution in its fundraising efforts by providing the first outreach, set up an opportunity to follow up and build its constituent file, making for a more independent and community-sustainable institution. This is just one simple method used by all too few organizations to gather discrete information so valuable to the understanding of institutional success. It is also one more simple argument for holding all organizations and donors to some simple concrete standards for measuring impact.
Jay, I hope you don’t read anything in my post that contradicts your point. Solid data is the underpinning for answering all of these questions. As you point out, good data (or I would simply say good information, since good information isn’t always quantitative data) is critical. When I talk about using good evidence, gathering information, analyzing it and adjusting, all in the service of producing results, I hope I’m speaking exactly to what you are looking for.
Sean, Thank you for your blog post and your reply to my comment. While I do not read anything into your post that contradicts my point, I do think that the questions you offered too easily lend themselves to anecdotal answers precisely because nonprofits usually lack strong quantitative data on their constituencies and program impacts. The reason I used museums as an example in my comment is because they rarely gather specific information on all those passing through their doors. This data would enable them to analyze what types of people they are reaching and how they are reaching them as well as how that is changing over time. This data both informs analysis and sets the stage for better resource development, making the institutions engaging in this type of data collection more financially independent as well as more responsive to the needs and interests of their constituencies. The same argument for better quantitative data gathering and analysis can be made in higher education, the target of most large scale charitable investment, where peer screening, data modeling, constituent surveys and electronic screening are all used to help gain a sense of the resource development opportunity and may also be helpful in measuring the impact of the educational experience on the institution’s alumni. While I suspect you do not disagree with these statements, you do not appear to give them similar emphasis, instead focusing on the necessity to better analyze the data the organizations do in fact collect. In my experience, however, nonprofits have very far to go before data collection catches up to the type of analysis you rightly recommend. The place where we are likely to disagree is on the relative importance of resource development. You write, “In the business world, the intended result—a profit—is also the fuel that sustains the organization. In the nonprofit world, program results don’t usually pay the rent. That means that nonprofits are at risk of giving higher priority to fund raising and other revenue-generating activities than producing program results.” I disagree. First, nonprofits are businesses. They simply try to achieve a positive fund balance rather than a profit and any surplus is (or should be) invested in endowment, additional hires, program expansion or other activities benefitting the stakeholders rather than as distributions to shareholders. Second, while “program results” may not pay the rent any more than good nutrition brings profits to a cereal company, in both for-profit and not-for-profit enterprises programs and products do bring revenue both directly (fees for service/purchases of products) and indirectly (investment in the enterprise through stock purchases or charitable investments). Third and most importantly, nonprofits must focus squarely on fundraising because it is what keeps them independent and responsive to the community at large. When organizations are supported by a small number of donors they can easily develop a myopia about both the problems they seek to address and their effectiveness in that pursuit, much as for-profit enterprises inflexibly following the vision of their entrepreneurial founders can lose touch with contemporary markets. It is the very act of having to reach out to new donors and prove their worth each and every day which forces those in leadership to continually assess how they can be most relevant and effective. Fundraising is after all a sales activity. Rather than this being at odds with pursuit of “public good,” I believe it enhances it. In fact, one of the best measures of whether a nonprofit is “high performing” is that it is “able to speak convincingly to their absolute focus on results” as demonstrated by their ability to attract revenue. In the end, both not-for-profits and for-profits have to produce a good and attractive product (or service) in order to meet payroll. As they say in the movies, “No bucks, No Buck Rogers.”
One of the reasons I developed these questions the way I did is because they allow the potential funder to decide for themselves what sort of answers are convincing. It sounds to me like you would want very data heavy, quantitative answers to these questions. Another donor might not be convinced by the data you seek and instead want to hear the management team provide a detailed qualitative description of their activities.
But both you and the funder seeking qualitative info could still use these questions as the basis for your analysis.
Re: your point about fundraising, I think we do disagree. When you write that nonprofits “simply try to achieve a positive fund balance”, I have to ask “to what end”? Profit is the end and the means for for-profits. Positive fund balances are only a means for nonprofits. They are not a goal, they are only a resource to help achieve a mission focused goal.
Just to clarify, I like your questions very much. I simply don’t believe most nonprofits have the data to answer these questions in anything other than anecdotal fashion. That is why I emphasize the need to acquire this type of information.
As for the matter of resource development, I agree that fundraising is not the goal. Mission is the goal. On the other hand, mission is not possible or sustainable without resources. The two should not be separated.
In the final analysis, our real disagreement may be with who or what should determine the control or future direction of a nonprofit. In my view, donors should play a far more important role. Having a diverse and broad fundraising base helps to ensure that money is available to achieve mission, that an organization is not beholden to any one particular interest and that nonprofits are truly responsive to the interests of the broader public.
The great thing about these questions, Sean, is they are exactly what everyone interested in the organization should be asking. That’s donors, board members, staff, the community at large, the constituents. You name it. Yes, there should be solid data behind all the answers, but what people really want is an interpretation of the data. Nonprofits have complex programs providing complex issues. What everyone really wants to know, and your questions break this down nicely, is “Is it working? Is it making a difference? Is it doing what it says it does?” And realistically, most people don’t want a huge binder bulging with information. They need to understand the basic outline of how this nonprofit is measuring and living up to its promises. If most boards could start with that, a lot more of the numbers would make sense to everyone.
Thanks Alexandra,
I believe that any information that a nonprofit gathers to explain what they do to funders should be useful to them as well. Otherwise, the funders are probably asking for the wrong data. Thanks so much for your comment, you got right to the heart of the issue!
Great questions, Sean. While everybody seems to get lost in the conversation around metrics and numbers, I think one line in your post says it all: “[organizations] that have the ability to carry out successful programs”. That’s what we’re all looking for, and the questions you raised help uncover that feeling during a conversation. It’s not necessarily about the metrics themselves, it’s about believing that the organizational structure can support pursuing success.
Looking forward to seeing you at the SVPI conference next week.
Thanks David! See you in Long Beach.