After reading a United Way blogger’s reaction to my prediction that UW would develop an industry standard, narrative outcome measurement form and my discussion of Google Finance, I’ve started a discussion thread on the Google Finance page for United Way.
The United Way’s focus on Outcome Measurement is wonderful. I wish there were more resources like the ones you list at http://www.unitedway.org/Outcomes/ elsewhere on the web. I was wondering if someone at United Way could explain to me a little bit about how your organization thinks about qualitative vs. quantitative evaluation of nonprofits. It seems to me that quantitative metrics are probably easier to measure, but less valuable than more difficult to measure qualitative outcomes.
Any help you can provide in thinking through this issue would be wonderful. Thanks!
FYI: While the Red Cross has stopped posting to the thread I started on their page, I understand from sources that they are taking the questions about their effectiveness quite seriously.
I wonder how the United Way will respond?
3 Comments
Sean,
My role at United Way (of Massachusetts Bay and Merrimack Valley*) is in communications, so I’m a bit of an engaged onlooker when it comes to evaluation, but the challenge of measuring and communicating an organization’s effectiveness is something that we talk about throughout the organization. I’m hoping that some people a little closer to the data in my own organization and in other local United Ways (and perhaps United Way of America), will chime in after me with some more in-depth thoughts on your question.
But, hey, in the meantime, here are a couple of my own thoughts. For me, it comes down to context. Quantitative metrics are clearly an essential component of nonprofit evaluation and will remain so. The strength of qualitative evaluation however is in its ability to give context to the numbers and, when placed into a narrative, be helpful as tool for learning and sharing effective strategies throughout the sector. If are talking about advancing the field, and not just counting mosquito nets (a la Denise Caruso’s recent NYT article), I think that qualitative measurements has to be part of the conversation. I’m glad to see that there’s some energy around this discussion.
* quick point of clarification –I work at a local United Way not the national office, where the outcomes measurement resource network is based. The post you referred to is from Speak United, a blog of United Way of Massachusetts Bay and Merrimack Valley employees.
I’d love to have others from UW weigh in. Glad you just went ahead and added your own thoughts.
All measurements need context. That’s something I think people miss when they seek a universal metric.
This is actually an incredibly timely conversation, as we at the United Way of Massachusetts Bay and Merrimack Valley are just about to execute our first data collection for our new community investment model. To your point about quantitative versus qualitative evaluation of nonprofits, my own personal opinion and the position of our local United Way (I honestly can’t speak for United Way of America) is that they both add value in helping to understand what results an organization is able to achieve in the community. In general, we are asking for quantitative information (number of people served in a program, number achieving a particular outcome etc.) and in each category of investment, we are asking for a narrative reflection on the information provided. This narrative will hopefully give us a clearer view of qualifications on data (where estimates need to be made etc.) and more importantly, challenges encountered and successes achieved while working towards specific targets. In the area of early childhood for example, we are going to be tracking a few data points around expulsion in early education and care programs, such as the number of children who are screened, the number who are recommended for services and most importantly I believe, the number who ultimately are able to remain in the program as a result of services received and of course, the number of children who are expelled from the program. In addition to the numbers, we are also asking organizations who are working in this area to provide a narrative describing the steps that they take to help prevent expulsion. The numbers themselves are important in that they answer the question of scale, (how many children benefit), while the narrative is important because it provides detail on the activities that can significantly impact the numbers reported. When the two are put together, a powerful story can be seen if a relationship exists between, in this case,expulsion prevention activities and the number of children retained in programs. The story behind the quantitative data can also powerfully drive toward program improvement. A funder like United Way, with relationships with a large number of service providers, is in the unique position of bringing the successful experiences of some organizations to many.