Join us in tracking the evolving landscape of social innovation.
POST
New metrics and a new mindset for measuring movements By Noah Rimland Flower / January 2012 / Leave a comment

When you’re trying to build a movement, how do you know if your effort is reaping results? Across all the types of enterprises in social change, movement organizers are often seen as the polar opposite of straightforward service providers such as soup kitchens. It might be hard to quantify the emotional benefit of getting a meal when you’re hungry, but it’s often enough for the people running the kitchen (or providing the funds) to know that those in need are being helped. The situation is far more complicated for the people organizing civic action, whether one as wide-ranging as Occupy Wall Street or as focused as the campaign against fracking. You can measure the number of people who show up at an event, but how can you measure the gradual shift in mindsets, greater levels of political awareness, and the strength of grassroots leadership?

It’s clear from watching the history of social movements that they can play a powerful role in achieving cultural shifts and policy change, and any organizer can tell you about changes that they’ve witnessed in the people they work with. But after decades of organizing and supporting movements, nonprofits and foundations still struggle with how to open the black box of a movement and accurately describe its progress. This is a considerable problem in the midst of the movement throughout the social sector towards evidence-based decisionmaking.

The essence of the challenge is that there is often a mismatch between the outputs that can be counted but might not be meaningful and the outcomes that might not be countable but are the reason for doing the work. I had to wrestle with that challenge in the work we did to design volunteer programs for Service as a Strategy, an initiative that gives mayors a set of instructional “blueprints” for calling on volunteers to help advance important policy priorities. It was essential for every blueprint to provide clear metrics for showing the value of the work, but my team often found that the outcomes operated on a far longer time horizon than the outputs that we could recommend counting.

The good news from three USC academics is that there are now better metrics for measuring movements and more support for proving results among organizers. Manuel Pastor, Jennifer Ito, and Rachel Rosner released Transactions, Transformations, Translations:  Metrics That Matter for Building, Scaling and Funding Social Movements last October, based on input from over sixty experienced organizers and others actively involved in movement-building. (Thanks to Beth Kanter for bringing it to my attention, and to Mario Morino for bringing it to hers.) In their conversations, the authors observed a marked shift in both the mindset around measurement and the tools for accomplishing it.

From my perspective watching the field, the mindset shift is the most interesting news. They report that whereas “community organizers have not always helped matters” by being “resistant to numeric measures on the grounds that these fail to capture the… one-on-one epiphanies of their members,” today “the field is changing: Recognizing the gravity of the times, hoping to gauge their effectiveness, and wanting to add up to more than the sum of their parts, the movement builders we interviewed are eager to come up with a common language and common framework.” They also say that a change is afoot among funders: “There are a wide range of program officers and philanthropic leaders who are eager for the evidence to make the argument that movements matter. They are actively looking for the tools and the stories that can help their institutions see the bigger picture.”

But if you’re doing this work, whether as a funder or a nonprofit, it will be the meat of the report that you’ll be most interested in: a set of suggested metrics for every aspect of movement-building. (Heads up to those who participated in the Network of Network Funders and anyone else interested in networks other than movements: a great deal of what they describe applies equally well to measuring the progress of growing networks of all kinds.) They divide them among community organizing, civic engagement, leadership development, alliance building, campaigns, research and policy analysis, communications and framing, media, organizational development, and movement building (on the whole). In each area they suggest telling a story of progress using two types of metrics: transactional and transformational. They define them as follows:

Transactions involve the quantifiable markers both internal (e.g., how much funding, how
many members, etc.) and external to the organization (e.g., voter turnout, policies passed, etc). While the data is not always easy to collect (especially with transient or mobile groups), such measures tend to be easier to track because they are more tangible. But transactions only tell part of the story and tend to skip over the richness of experience and momentum that can be precursors to big change.

Transformations, on the other hand, are the vital but sometimes “invisible” work. They show how people, organizations, and movements have been altered through the collective efforts. Taking the transformation further, they can show how societal and political views have shifted or been impacted by movement building.

They also have a special section for funders that recommends the following as goals:

  • Build the Movement Metrics Toolbox
  • Develop Movement Capacities to Use Metrics
  • Nurture Leadership and Leadership Pathways
  • Link Policy Outcomes with Broader Social Change
  • Communicate Transformative Shifts
  • Document Innovation and Experiments
  • Adopt a Movement Frame to Evaluation
  • Co-Create the New Metrics of Movement Building

Their suggestions are far from a final solution to the problem, but they do represent a real step forward. If you do this work, I heartily recommend you take the time to read their analysis. Please let me know:

What challenges do you face in quantifying your work to build movements, and how do their suggestions compare to your current practices?

You can find the full report on the USC website, or flip through it below:

Leave a comment, question, or resource