Swarm Intelligence Journalism

kentbye's picture
| | | | | | |

Dan Gillmor points to Lance Knobel's lecture titled "Nullius in verba: navigating through the new media democracy." The Latin phrase comes from the Royal Society which translates to "Don't trust in anyone's word."

Since we shouldn’t trust any one person's word, then is there a way that we can trust the Swarm Intelligence contained within the Wisdom of Crowds? And is there a way that we can somehow extract this swarm intelligence through a technological tagging mechanism such as del.icio.us?

Bloggers don't rely upon single sources of information, but they instead triangulate many different perspectives -- bloggers do an internal calculus to determine their perception of the truth.

Instead of starting with facts provided from the mainstream news, many news consumers are first reading the subjective interpretations from the bloggers they trust. Jay Rosen pointed out at Bloggercon III that the old news consumption paradigm was that people first consume Facts, then Analysis, and then Opinion. But the true paradigm may be in fact that most people start with the opinion and debate, and then hunt down the facts and other interpretations.

So news consumers aren't solely relying upon newspaper editors any more -- they're creating a "Daily Me" mix of specialized blogs who serve as the new filtering mechanism -- what Knobel describes as "trusted intermediaries" who now "navigate through this thicket" of information for us.

I've created my own thicket of information by interviewing 45 different perspectives on how the media covered the build-up to the war in Iraq. -- All of the perspectives have grains of truth. There are universal truths across all of the perspectives. There are also definite points of departure of what "the truth" is.

The challenge for me and other journalists to find these universal truths and points of departure among the multitude of perspectives. Sifting through all of these perspectives is too large a task for one person -- or even a small group of experts. It really takes a very large group of people who are contributing small chunks of insight.

The Sandia National Lab's Mark Boslough is helping US intelligence agencies figure out how to scale the collaborative process of analyzing intelligence across many different organizations. Boslough writes in his paper titled "A Distributed Dynamic Intelligence Aggregation Method":

Much progress is underway to improve data collection, distribution, and mining methods. The ultimate problem with this approach is that it is not scalable. Even if all the relevant information were available to an expert panel of analysts, it would be too much for them to digest. An effective means of “swarm intelligence” is required to overcome non-scalability, as well as the distributed and dynamic properties of the data.

Boslough cites James Surowiecki's Wisdom of Crowds for the four conditions that need to be met for a large group of people to be smarter than a few experts:

There are four key qualities that make a crowd smart. It needs to be diverse, so that people are bringing different pieces of information to the table. It needs to be decentralized, so that no one at the top is dictating the crowd's answer. It needs a way of summarizing people's opinions into one collective verdict. And the people in the crowd need to be independent, so that they pay attention mostly to their own information, and not worrying about what everyone around them thinks. (Emphasis Added)

The blogosphere is diverse and decentralized. Folksonomic tagging systems can be used to aggregate opinion's into one collective verdict (e.g. del.icio.us/popular aggregates "votes" on what people find interesting).

The biggest question is how independent each blogger is versus how they're influenced by their set of trusted information sources. As Knobel says:

"How can we circumvent, or at least ameliorate, the effect of the echo chamber of the Daily Me?"

Knobel continues:

One of the necessary requirements that does not yet exist, I believe, is an effective way to encourage – and perhaps enforce – serendipitous discoveries outside the comforts of the Daily Me.

The Echo Chamber Project wants to help find out how to overcome this "Echo Chamber of the Daily Me." How can we get people to break out of their dominant paradigms driven by ideology and worldview?

I'm open sourcing the text of the documentary interviews -- which is nice from a transparency perspective. But how do I make it more interactive where people will be forced to evaluate perspectives they might not necessarily agree with?

Well, my first step is to create a software infrastructure to facilitate a del.icio.us-like tagging that summarizes "people's opinions into one collective verdict."

In other words, I plan assigning a URL to each interview soundbite -- In Drupal terms, each soundbite will become it's own node with a unique URL. Then by creating an internal system where users can tag that soundbite node with a number of different tags, then I'll be able to see what the dominant tags are for each soundbite. These tags are votes of how people are subjectively percieve the soundbite.

This will be a way for me to tap into the Wisdom of the Crowd in order to quantify the context and meaning of each soundbite. Gathering this type of metadata will help me edit a film that will be much richer in meaning because I'll be using the insights of many different participants who have contributed their perspectives -- Call it Swarm Intelligence Journalism.

But subjective perceptions need to be balanced with objective facts. By Phase 7 of post-production, I hope to address how to fully implement a New Media Ecosystem that uses many insights from the field of intelligence analysis.