“How do we measure for epiphany?” Rob Stein’s question from MCN2011 haunts me.

Measurement is on my mind. As a theorist, I don’t tend naturally towards the quantifiable. Neither do museums in some ways. Much of the value of museums is other-than-economic, and not easily measurable. But we live in a society that demands success in quantifiable packaging. We really need a Wondermeter, but since we don’t have one finding the right questions to ask, the right things to measure becomes critical.

It occurs to me, then, that changing the conversation on online collections or museum websites (or anything that we know is important) will demand that we have the right metrics; metrics that funders or board members can make sense of in order to benchmark new ways of thinking about digital against other priorities; metrics that alter the way we think about what we do, and why.

When I think about how I conceive of the online museum collection and its potential role in the broader information ecosystem, it doesn’t make sense to me if success is measured simply by numbers of object records online or visits to the website. That doesn’t tell us anything about whether new knowledge is being created using the collection, or how the collection records are meaningful beyond the museum – and I think those are much more interesting questions. As Jasper Visser put it:

I recently realised that we, cultural institutions, are using the wrong metrics to measure our online success, because we’re measuring just that: generic success. We’re using statistics and software that is perfectly fine when you’re selling Cokes, but might not be ideal for culture, heritage and the arts.

There are some important general questions that should be asked when considering digital success and metrics (Clairey Ross’ post following Seb’s MW2011 web metrics course is a great starting place here). But I’m going to imagine some new metrics, ones that measure the things I think are *really* important, beyond the basics. You might not agree with me that these are the right things to measure. That’s great. What is?

1. What new innovations, knowledge, and digital inventions have come from people using your collection/information?
Patrick Hussey recently considered how crowdfunding is changing culture, and asked “Is it possible that crowdfunding is telling us something rather profound – that the most important and popular form of creativity at this point in history is not ‘useless’ art, but digital invention?” It’s a question that recalls to me one of the conclusions that Koven drew from his unconference session on online collections at MW2012, that “The after-market for collections data may be the most important one.” What if making possible digital invention and new collection-driven discovery was the point of museum collections? How would we measure that? Would the right metric be measuring how many new inventions/innovations have come from people utilising collection data, or what new knowledge had come of it? And if that was a metric, would that change the way your museum acted? Would you run more hack days to encourage innovation around the collection? Would you make sure your data was available as an API? Would you change your image licensing allow image downloads for non-commercial and academic use as the UK National Portrait Gallery have just done?

2. How does your collection link to the broader information ecology?
If we were to start thinking of the online museum collection as a living historical document rather than a mere catalogue, one in which we can discover things about our collections that we’ve never known before, what would indicate success? Would we measure how many Wikipedia entries lead back to the collection? How many external links lead out from the collection to authoritative sources? As Nate Solas has reminded us previously, authority online is conferred by linking to the right sources and places, by making the information that people need and want findable and available. You only have to look at the Walker’s website to see how they are making use of this kind of thinking. What could we measure to encourage more of this?

3. How is the world different because your collection, your museum is online?
Ok, this one is getting more towards the esoteric end of things, but go with me. Maxwell Anderson, in You Get What You Measure writes of answering the question “how is the world different because your museum exists?” It would be interesting to try to find measurements that answer the question “how is the world different because your collection is online?” If we cannot find ways to answer that, maybe we aren’t really having an impact at all? In which case, why bother? Of course, this isn’t something that is easily quantifiable (I told you, I’m a theorist not an evaluator), but I’d love to find new ways to measure the real importance of a museum’s online presence, of its impact as an educational institution, and the impact of the online collection. As Jasper puts it, our significance, not just our success.

I want museums and collections to be meaningful, online and offline. I think they are; or should be; or can be. But maybe they aren’t yet as meaningful as they will be. It’s not quite measuring for epiphany, but maybe it’s not that far from it.

What do you think? Am I completely wrong with my imagined new metrics? What would be better? What crazy things would you want to measure, and how would you do it? Feel free to talk about your own area of fascination, right across the institution. Don’t just limit yourself to mine.

BTW – there are some good general posts about museum metrics that have recently surfaced. Worth reading too.