Friday, May 14, 2010

Octopus Kills Shark

Via my dad via HuffPo via Boing Boing.

Reviews in the Age of Aggregation

One of the most dramatic elements of the new internet age has been the rise of aggregation. Before the internet, data was limited. Now we have unlimited data, and the limiting factor on people's consumption of data is time. In the previous generation, important people may have been 'personalities' who provided interesting outlooks on the world. Now important people are collectors who organize data in useful ways.

I think that aggregation has fundamentally changed the role of criticism in our culture in a way which most critics have not yet realized.

First let's take a look at the role of criticism before the rise of aggregation. And here I am talking about friendly accessible newspaper criticism which people use to decide whether or not to consume a particular work of art. I do not care about ponderous useless academic criticism which people use to gain jobs as tenured humanities professors.

So here's how people's relationship to criticism used to look: You want to see a movie this weekend. So you open up the arts section of your newspaper of choice and you read the movie reviews. Maybe you pay attention to the byline of the reviewer, and certain reviewers--A. O. Scott, say, or Ebert--will influence your opinion more than other reviewers.

In this situation, the reviewer is trying to get at a kind of objective criticism of the work in question. Since the reader is only going to read one--or at most two--reviews of a given work, the reviewer will do his or her best to set aside their individual opinion and try instead for a more universal opinion. The reviewer does not answer the question "Did I like it?" but answers the question "Is it good?"

This stance was a result merely of a technological limitation. The collection of large sets of data about people's opinions was difficult and time consuming. So the reviewer had to take the position of the sole judge.

Yet now, with the rise of aggregation, that has all changed. Data is cheap and plentiful.

These days, if I want to judge a given work of art, I will look at a site which aggregates opinion. Sites like Metacritic and Rotten Tomatoes digest professional reviews into numerical scores which provide a good guess at how good a particular work of art will be. Other sites like and Imdb allow users to contribute their own reviews. Sites like Netflix and the iTunes music store use complicated algorithms to look at individuals' tastes and try to figure out what art and music will appeal to them. And then just look at the plethora of different top ten, top 100 and top 1000 lists out there. We love aggregation. Sometimes we might look at the reviews of a particular favorite reviewer, but that reviewer is no longer the only source we use to judge whether we give our time to a work of art.

So there is now no more need for the reviewer who positions himself as the sole arbiter of a piece of art. The reviewer can merely give his or her own personal opinion and then let the aggregation do the work of providing a more 'objective' criticism. There is no harm now in saying "I don't like this film because the actor reminded me of my brother-in-law." The peculiarities of individual opinion are leveled by aggregation.

But criticism has not yet changed. To show what I think modern criticism might look like, I will periodically post reviews of things on this site, giving the most biased and personal reviews possible. To balance out these biased reviews, I will also include as much aggregated criticism of the given work of art as possible--its Metacritic, Amazon, and if possible, Imdb rating. In this way I hope to show both sides of aggregation--the small-scale, personal and biased individual reviewer, and the numerical, sphinx-like and sage aggregated number.

Things Which Don't Suck, Part 1

Rhapsody in Blue

My first sip of coffee in the morning.

Jess' jokes.

Gordon S. Wood.

Dostoyevsky and Salinger.

Dorris Lessing and Laurence Durrell.

Tulips and Irises.

Squids and octopuses.

Miyazaki and Pixar.

Keroro Gunso

What else doesn't suck?

Tuesday, May 11, 2010

The Narrative Fallacy

We tend to look at the events of our lives as stories. While our stories will fit the data of the world most of the time, there will be rare but significant moments when our stories about the world will fail. These incidents are as distressing as they are inevitable.

Anyway. Here’s something I thought of on the bus today.

Since we understand the world in terms of stories, we judge the validity of particular storylines not only by how well they fit the world but also by how successful they are as stories.

This isn’t a huge problem when we have a lot of data (in our own life, for instance). Since we have a lot of data we’re probably constrained to make storylines which are a closer fit to the world.

Where this effect is really significant, I think, is in parts of our knowledge where we rely mostly on hearsay—where we don’t have a lot of easily available data to whittle down our storyline options.

Now tons of stuff we believe in relies on hearsay. Our beliefs about the nature of the universe, our beliefs about politics, and history are mostly formed by stories other people tell us rather than from our own observation.

In these situations since we don’t have a lot of data with which to falsify or verify these stories, we will tend to choose stories which provide more narrative satisfaction.

For instance, I am interested in the US health care debate, but I do not have time enough to adequately educate myself about it. But I still need some kind of opinion about it. So what opinion do I choose? I will choose a story which makes sense to me. “President Obama is selling us out because he is a liar.” Or, “President Obama is standing up for me and he will choose the best possible policy option.”
Now this problem is especially pernicious when it comes to journalism. The success of a journalist is dependent on his or her ability to construct meaningful stories out of the data of the world—not on their ability to construct more accurate stories about the world.

You can see this in reporting on the stock market. The condition of the stock market effects the well-being of many people, so it is a story which we have to tell. So when there is an unusual movement of the stock market journalists have to explain it. “The stock market crashed and then recovered very quickly today because of a computer error.” “The stock market rose today because of investor hopes of a change in the interest rate.” But the stock market is a notoriously complex system which defies easy analysis. The same conditions which are used today to explain a rise in the stock market will be used tomorrow to explain a crash in the stock market. The real reasons for these rises and falls are too complicated for non-experts to understand (and probably too complicated even for the experts to understand.) So we tell ourselves stories about these events which make sense but don’t actually convey the nature of reality.

This narrative fallacy can explain religious and political enthusiasm, too: having a particular storyline which can explain everything in the world is very comforting. “Everything would be better if people just followed Marx.” “The world is bad because nobody is a real Christian.” “America is corrupted by special interests.” All of these ideas provide very compelling stories which explain the world in narratively satisfactory ways. The problem is that they don’t accurately reflect the world. Since we don’t actually have a lot of commerce with data about politics, we are able to keep these inaccurate beliefs safe from recalcitrant experience. In our personal lives we are held to greater accuracy about our stories because we are more likely to encounter recalcitrant experience.

Here is Tocqueville talking about something similar. He's discussing how pre-Revolutionary France was led by its writers.

When we study the history of our Revolution, we realize that it was prompted by precisely the same outlook which inspired so many books on the theory of government. They reflected the same attractions for universal theories, comprehensive systems of legislation and an exact summary in the laws; the same contempt for existing facts; the same faith in theory; the same taste of the original; the ingenious and the novel in reshaping institutions; the same desire to reconstruct the entire constitution at one and the same time following the rule of logic and according to a single plan instead of seeking to reform it in its separate parts. A frightening spectacle! For what is a good quality in a writer is a failing in a politician and the very themes which have often produced fine books may lead to great upheavals.

If I was Malcom Gladwell I’d call this something like ‘The Storyteller’s Illusion’. But I’m not. So it’s ‘the Narrative Fallacy.’