I wrote yesterday about the Chronicle of Philanthropy article about Great Nonprofits. In the article, some people voice the criticism that the site has just positive reviews. So I emailed founder Perla Ni and put the criticism to her.
(Full Disclosure: Perla is a friend of mine. I know her professionally, but I also went to a party at her house earlier this month. Also, the photo of me that appears in the Chronicle of Philanthropy article is from when I volunteered at a Great Nonprofits event a few months ago. Clearly, I’m not the person to argue the merits of Great Nonprofits. So I’ll let Perla’s response to my question speak for itself. If readers have further questions, I’m sure Perla will respond in the comments section.)
Sean: “The criticism I know that you will hear and I will for writing about your project, will be that all the reviews are positive. The lowest rating on the site for a nonprofit with two or more reviews is 4+. How would you respond to someone who complains about the lack of negative reviews?
Perla’s response:
Great question. The answer is long – but I think your question deserves a full answer.
First, your readers shouldn’t look at the star ratings so much. It’s a short-hand really and cannot fully reflect the nuances of someone’s opinion about a nonprofit. Unlike product reviews, for instance, where the product is supposed to do a specific thing, nonprofits do a lot of different things sometimes and so it’s much more valuable to read the full review, especially the part on the “Great” things about the organization as well as “How to make it even better” which offers very concrete ideas for how the nonprofit can make its programs/services better.
Secondly, we have a selection bias right now in 2 ways because we’re just starting out.
1. As you’d expect, nonprofits that feel confident about their programs/services are more likely to participate on a website like ours. Though “consumer reviews” are standard now in just about every other sector – restaurant reviews, book reviews, car reviews, hotel reviews, movie reviews – it’s a new concept in the nonprofit world.
And so only the nonprofits that feel pretty confident about the quality of their services/programs are going to participate. My theory is that these probably are nonprofits that do indeed provide better than average programs/services.
2. Because since we’re starting out and the larger world of nonprofit stakeholders hasn’t heard of us, we rely on the nonprofits to get the word out to their clients, volunteers, donors and other stakeholders.
And the natural inclination is to ask one’s most trusted and closest stakeholders to write a review. But in time – as with anything on the internet – because the information is so readily available and transparent, other stakeholders become aware of the site, they will chime in and react to the reviews on the site. There’s a counterbalancing effect on the web because information is so transparent. People tend to feel more free, in fact, to disagree online than in person.
So in time, those two selection biases should correct itself in time.
Now, there’s a third bias that will continue even when the site gets more counterbalancing reviews. It’s the psychological tendency that people remember good experiences more strongly than negative experiences. This is well documented in psychology studies – and it’s the reason why the average review about consumer products is not 3 out of 5, but 3.4 out of 5. But this is still probably more accurate than the “Expert” reviews (ie: staff writers for consumer products magazines) where the average rating is 4 out of 5.
I hope that helps.
2 Comments
I want your project to succeed, so take these comments in that vein.
1. It is essential to acknowledge what’s not happening if you want it to happen. It seems to me that you are not doing this with your “First” point. I just scanned the reviews for your top-ranked nonprofit, and all of the “criticism” came down to: “need more money,” “need more awareness,” or jargon.
People are not used to thinking critically about charities. I’m not surprised at all to see that there is nothing of substance here. In fact, the website itself seems to be perpetuating this problem with its phrasing, “Ways to make it better!”
In my experience, people in this sector are nice, and nice people FEAR criticism. That’s why they use that kind of wishy-washy language. Having the same kind of language in the fabric of the site seems like a problem.
It is really important that you acknowledge this rather than saying “it isn’t just the stars.” It is just the stars, in the sense that all of the “feedback” seems to be no less positive than the stars. (If I haven’t looked carefully enough, please point me to substantive criticisms.)
2. Your plan for how things are going to improve sounds unrealistic. I believe that just about every successful community-driven website started with a strong base of content, not just a structure. Why? Because people don’t contribute unless they’re engaged, and people don’t engage with things that aren’t useful.
Wikipedia started as a small project in a particular community. By the time it was public, there was already a lot of useful information there; people wanted to engage with it and add more. It drew people in and THEN they became a part of it. If a site isn’t useful, people bounce right off.
Sitting back and expecting GreatNonprofits to develop as “word spreads” will, I predict, ensure that it does not happen. Actively forcing useful content onto the site seems like a much better path to making it useful, which will then allow it to become more useful.
Interesting site… thanks for pointing it out. I just added my organization, Central Dallas Ministries.