Last month I posted a question on the Google Finance profile page of the United Way of America. I got an email today from Meg Plantz, director, Impact Design and Learning for United Way. You can catch up on the Google Finance for nonprofits discussion here.
United Way of America’s approach to program outcome measurement, reflected in its manual, Measuring Program Outcomes: A Practical Approach (United Way of America, 1996), encourages health and human service agencies to develop ways to measure outcomes quantitatively. This often is a challenge because many human-service outcomes seem at first glance to be un-quantifiable, and many agencies are used to describing their successes with narrative vignettes rather than with numbers. When they have used numbers, the numbers often have reflected outputs (e.g., number of classes offered, number of clients served) rather than outcomes (e.g., improvement in parenting skills). However, for purposes of creating and tracking program improvement over time and demonstrating results to increasingly data-oriented funders, donors, and publics, numerical indications of program performance are important.
Our approach consciously responds to the challenges that quantitative measurement presents. For example, in our approach, intended outcomes do not have to meet the “measurable” test. In fact, when agencies are identifying what their outcomes are, we encourage them not to worry about how they will measure them, but instead to focus on what results are meaningful to the program and its clients.
Once the agency is comfortable that it has described the appropriate outcomes for its activities, then we advise them to identify measurable indicators of those outcomes. Questions such as “What will tell you if clients achieve the intended outcome (e.g., improvement in parenting skills)? What will you be able to see, count, or measure?” help agencies identify critical aspects of their outcomes (e.g., using age-appropriate discipline methods) and think about ways to quantify them (e.g., record observations of parents in role plays and tally entries that parents make in a journal). Agencies are encouraged to pick indicators that will provide useful data – data that will help them with program improvement, in communications, and in other management tasks.
Qualitative information complements and can help with quantitative measurement. For example, agencies can use their narrative success stories to identify intended outcomes and then use quantitative measurement as a way to learn whether the stories are unique or are representative of other clients. Agencies can use their narrative stories to illustrate their outcome data rather than offering the stories as evidence of outcomes. Qualitative information such as focus groups discussions of quantitative data can help agencies understand the meaning of the data and make appropriate program improvements.
Meg Plantz
Director, Impact Design and Learning for United Way