{learning} The Art of Evaluation_#ICM820

We at #ICM820 course have discussed successful strategies of community-building as well as cases gone very wrong. But evaluating – quantitatively measuring and qualitatively assessing – successes is a tricky issue.

From a macro-level vantage point of societies and its institutions, we could ponder how to assess media systems (or, as many tend to say about the digital era of multiplicity, media ecosystems) work effectively, democratically, openly, and so on. I have collected some links to projects, ideas, and cases that aim at measuring media systems and media development from a global perspective.

The meso-level of organizations outlook would be to look at effectiveness of particular political, economic, social, etc. communities, organizations, or campaigns. Is it about eye-balls, likes/shares/follows, comments, retweets/repins etc.? Is it about the ratio between lurkers vs. active participants? Professionally, do we value media differently than we did before, in terms of it as an advertising distribution tool, a news source, a forum for debate, an entertainment source? Here are just some examples of the infinite amount of views on how to measure success and impact in the digital age:

Also:

Finally, at the micro – or individual – level: How do you (does one) measure a digital community? Usability, access, relevance, engagement/familiarity, security…?

As experts of digital communities, how do we balance structural/technological concerns, big data metrics, and individual experiences?

{research} Big Data, Big Challenges

Will Big Data save the world, as some claim?

I’m currently enrolled in two excellent online courses: Data Journalism by the European Journalism Centre (#ddjmooc) and the TechChange course on Mapping for International Development (#TC141). I’m not a journalist or a crisis mapper but both of these themes, I suspect, will be relevant to global/media development and the UN post-2015 goals (something that I am passionately interested in).

The more I learn of the amazing tools, the fascinating projects, and the impressive visualizations, the more cautious I become about the possible lack of multidimensional perspectives.

I am wondering whether we are sometimes falsely seduced by massive numbers. Given the debates about diminishing resources in journalism, can the big data be an easy — too easy — solution to research? What kind of stories are not being told? Or, given that new data is often created by the means of people’s digital activities, whom does it leave in shadows? And, crucial in all cases but very, very important in development work: How do we detect the (Western?) biases of big data?

I’m, of course, not the only one wondering. As Michael Brodie of MIT comments in a recent issue of The Economist (May 24th, pg., 16), ‘much harm is being done by people asserting causality as a consequence of data analysis. […] Big Data’s pursuit of “what” should be symbiotically linked to empiricism’s pursuit of “why”…’.

The Wired says we should focus on Long Data. I suspect the rise of qualitative approaches, as this wise blog post by an applied anthropologist also claims. Am I right or wrong? Ahead or behind the curve? What are your experiences of using / being informed by Big Data?