This is my newest column for the Chronicle of Philanthropy. You’ll find an archive of my past columns here.
Foundations and nonprofits are constantly looking for the right tools to measure success.
One of the most effective sources of information might come from the people who rely on an organization, suggests a new the book, The Ultimate Question 2.0, in which the veteran management consultant Fred Reichheld demonstrates that asking one simple question of a business’s customers can often reveal more about their performance than more traditional financial or product analyses.
The question is: "How likely is it that you would recommend Company X to a friend or colleague?"
Since recipients of nonprofit services don’t typically pay the costs of the services they receive, this approach to measuring results must be modified when applied to the social sector.
But a number of organizations are working on ways to reach out to beneficiaries of a nonprofit or foundation as well as to the general public. In so doing they may find the same connection Mr. Reichheld did.
The Center for Effective Philanthropy is one of the pioneers of this work.
For more than a decade, it has conducted studies of grant recipients and others to help foundations figure out how to reinforce strengths and fix weaknesses. Its flagship Grantee Perception Report has now been commissioned by more than 190 foundations.
The center has also been working on a Beneficiary Perception Report. Its pilot program is called YouthTruth and gathers the feedback of high school students who attend schools supported by The Bill & Melinda Gates Foundation.
In a report published this month, the Center for Effective Philanthropy notes that few foundations collect information from beneficiaries. The fact that the intended recipients of foundation programs are rarely asked for feedback highlights how much room there is in the social sector for Ultimate Question type measurement approaches to be deployed.
Keystone, a London charity, is also doing important work to gather and analyze the views of people a charity tries to serve. They are working with students in university service learning programs to gather what they call constituent voice; feedback from the beneficiaries of nonprofit programs
Since foundations serve the public good and not just one group of beneficiaries, they must reach out to a lot of different kinds of people to assess their work.
The James Irvine Foundation, for instance, has been publishing their Grantee Perception Report from the Center for Effective Philanthropy and actively seeking public feedback on their annual performance. Importantly, the foundation’s president, Jim Canales specifically says they are looking for feedback from critics of their work.
"The power dynamic inherent in philanthropy makes it critical that we resist the temptation to talk more than listen," Mr. Canales writes, "precisely because people will always listen politely to anything we have to say, regardless of its utility."
He recognizes that nonprofits tend not to tell foundations when they’re doing a bad job and it is the rare foundation that has ever lacked enough groups eager to take its money.
A similar dynamic exists between many beneficiaries and nonprofits, because a person in need of social services rarely is in a position to turn down subpar assistance. Few nonprofits ask such clients what they could do to better serve them. But organizations could make more vigorous efforts to encourage such feedback.
It is with this dynamic in mind that I’m reminded of a plea by Frederick Hess of the American Enterprise Institute for foundations to not react defensively when they are criticized but instead to actively seek out and encourage criticism.
"Foundations need to make it conscious policy to welcome-and even encourage-criticism." Mr. Hess wrote in Philanthropy, the magazine published by the Philanthropy Roundtable.
"Given that even tart-tongued observers will be unusually reluctant to share their thoughts, foundations need to make it extravagantly clear that they will not blacklist critics-or look kindly upon those who do. Only this kind of scrutiny, will flag blind spots, wishful thinking, or ineffective spending.
Whether the foundation personnel agree with such assessments, engaging with them is essential to forestalling the plagues of hubris and groupthink that are so much a part of human nature."
It is asking a lot of any organization to actively seek out criticism. But it is only by asking for constructive feedback that nonprofits and foundations can expect to improve the quality of their contributions to society.
6 Comments
Nice write up, the one caution I’d throw out here is that a positive outcome and a positive experience don’t always line up. I’ve worked with organizations in the past that ran interventions with no discernible social impact, yet their client bases adored agency staff and had warm feelings about programs that were not improving their lives.
While I’d encourage any organization to get feedback from constituents about whether they would recommend an agency’s services to others, I’d have concerns about using this feedback as a proxy for social impact.
I got a little lost in this one Sean–somewhere between beneficiaries, nonprofits, foundations and groups like CEP. That one question about recommending a company or not is the oldest question in the book in philanthropy. Donors of all stripes are always asking their friends, colleagues, advisors, and others whether or not they would recommend a given nonprofit for funding. The points you raise about foundations and feedback are all valid ones, though I think appeals based on this being the right thing to do are not likely to have lasting impact in our sector. The vast majority of foundations are endowed institutions that do not run for office, sell products on the market, or raise funds. That autonomy and independence is what gives them the ability to experiment and take risks. We need to understand (and think about how to re-design) incentives within foundations to seek and act on such feedback. CEP, as you mention, has made inroads in this regard by getting enough foundations to do perception reports that they are able to benchmark themseves against their peers. This appeals to the kind of reputational competition that is part of the philanthropic sector as well as a desire to improve performance in the absence of market or other pressures to do so. I imagine even CEP would agree that there is much still to understand about the interaction between external perception/feedback, internal performance and social impact when it comes to foundations.
Nice piece, and an important perspective. The nonprofit and foundation sectors are finally waking up to a basic tenet that the private sector has long been aware of: listening to your customers is not only a good idea, it’s essential to success.
I appreciate your perspective on the recipient of any kind of service or monetary award. Perhaps a charitable organization’s ultimate challenge is not to solicit donations but to provide effective service to the beneficiary of their work. You make a great point as to why a recipient would ever give bad grades to an organization that has just given them something without having to do anything for it.
I would be interested to know how the assessments are secured. Secondly, if adjustments have been made by those who have been assessed what have been the results of a secondary assessment after any changes have been made to their services.
A great turn of events! No one knows better than the recipients whether their needs are being served. It is a special kind of arrogance to tell people what they need rather than to ask them.
Sean, nice blog entry. Several years ago, I was part of a panel here in NYC sponsored by PhilanthropyNY that was for young foundation program officers who were thinking about making a change. Many really enjoyed their work, and they were very comfortable in well-paid positions that allowed them to survey leadership in their field of interest. Being a program officer also does tend to make your popular at the party.
One of the points that I stressed (and my fellow panelists took up the refrain as well) is that as program officers we need to put ourselves in the position of those seeking support from us. The best way to do this, honestly, is to join the board of an organization who’s leadership your trust and support, and then go out and raise money on their behalf (like any good board member should). Use your contacts in the philanthropic sector and make introductions, attend pitch meetings with your ED, help them craft their proposals – in short, use your direct expertise and network to advance a social mission.
It’s humbling – no two ways about it. As much as we like to think ourselves fair and equitable, giving away money is simply very hard to do in a fair and equitable way. How many good organizations have you turned away because they couldn’t meet your guidelines? How many phone calls or emails asking for an opportunity to discuss your foundation priorities have you neglected to answer, or failed to answer in a timely way? How may proposals have lingered in your in-box because of other priorities? When you’re the person asking for money, you experience firsthand how all these things add up to give the impression of disorganization at best, and capriciousness at worst.
It takes a lot of work to make sure that giving away money doesn’t become an exercise in making people and institutions with a lot of money feel better about being wealthy. Listening, developing, responding, coaching, exploring — the onus is on us as grantmakers to be leaders in our field of expertise through our accessibility. Any perspective on evaluation needs to start there.