Tuesday, February 23, 2010

The Long Tail of the Internet and the election of 2010

The Long Tail is one of the most important books about how electronic distribution of information via the Internet is shattering old business models, and replacing them with new ones. In it, author Chris Anderson uses the music industry as an illustration. Radio air time or Big Box retailer shelf space have hard limits to the quantity of music they can offer: after all, there are only 24 hours in a day, so you probably can't ever play more than about 400 songs a day.

As a result, the radio industry looks for hits, and plays pretty much only those. Hit albums are pretty much the only ones you see on the shelf down at Wally World. But what about all the rest of the music? Anderson looked at customer track listening data from Rhapsody, and found something very interesting:
But by then I had some hard data, thanks to Rhapsody, which is one of the online music companies. They had given me a month's worth of customer usage data, and when I graphed it out, I realized that the curve was unlike anything I'd seen before.

It started like any other demand curve, ranked by popularity. A few hits were downloaded a huge number of times at the head of the curve, and then it fell off steeply with less popular tracks. But the interesting thing was that it never fell to zero. I'd go to the 100,000th track, zoom in, and the downloads per month were still in the thousands. And the curve just kept going: 200,000, 300,000, 400,000 tracks - no store could ever carry this much music. Yet as far as I looked, there was still demand. Way out at the end of the curve, tracks were being downloaded just four or five times a month, but still the curve wasn't at zero.
Anderson graphed the curve. He calls it "the long tail" of the Internet:

The hits are all at the front of the curve, the "head" (the red stuff). Everything else is in the tail (yellow).

OK, this is all very interesting and everything, but what (I hear you ask) does this have to do with business models shattering? Well, the recording industry is paid to act as gatekeepers: they have traditionally scanned the tail for interesting new artists, and have promoted them to the head. Think Col. Tom Parker "discovering" Elvis. Nobody had ever heard of Elvis before, and Parker made him a star.

But in the Internet Age, who needs a Parker? Listen to Anderson again:
Way out at the end of the curve, tracks were being downloaded just four or five times a month, but still the curve wasn't at zero.
Business models are collapsing because the Gatekeeper function is being eliminated, or at least massively changed. A lot of recording labels will never figure out how to respond, and will go out of business. Some will figure it out, and will have massive success. It will take another decade for us to know who's who.

It's not just music. Any information product exhibits Long Tail market properties today. We're seeing this in Climate Science, where the folks in the long tail have blown the peer reviewed science out of the water. The ClimateGate emails show how a small team of scientists worked to rig the peer review process. They wanted control of the gatekeeping system, to let their friends in and keep their enemies out. Their "business model" was gaining control of the traditional distribution networks, e.g. getting the editor of Geophysical Research Letters fired.

But in the Internet Age, information wants to - and will - be free. So long, Dr. Phil Jones of the CRU. Michael Mann of Penn State, you're next. The scientific review "business model" is already shattering. Dr. Les Halton is a Climatologist who was interested in the IPCC's statement that Global Warming is making Hurricanes more destructive. He downloaded hurricane data from NOAA and did a statistical analysis, and discovered that contrary to the IPCC claims, there's no correlation. He's placed his code and data on the Internet for Long Tail review:
Before sending me hate-mail if you are a warmist or love-mail if you are a coolist or denialist or whatever the parlance is this week, I am neither. I am a scientist and trained to be sceptical. To make it easier than the CRU have made it, the data is readily accessible at the link below. Go check it yourself. I'm damned if I know how the IPCC came to the conclusions it did.
For something this important, all the software and models and all the data should be publicly available in easily accessed form to allow anybody to contribute. Anything less is insane.
Information wants to be free, despite the best effort of the Gatekeepers to keep it locked up. Which brings us to the election of 2010. We see a bunch of Long Tail phenomena, including the Tea Parties, Scott Brown, and Sarah Palin. The are all outsiders. They all use the Internet to organize, to raise money, to get the message out. They all drive the gatekeepers crazy.

The Gatekeepers are the two main political parties (it's a mistake to think that Tea Partiers don't hate the Republican party machine, and vice versa). It's the media: The New York Times and CBS News, which is finding to their dismay that they can no longer control the message (ask John Edwards or Dan Rather). But most importantly, it's the Ivy League.

I wondered for years why anyone would spend a quarter million dollars to send their child to Harvard. Like one of the grand old movie stars, it's living off its past reputation. I didn't get small, the movies did. So what gives with the price tag?

It's the gatekeeper into the political class. Both George W. Bush and John Kerry were Skull and Bones men. It's where the political elite sends their children to meet the future movers and shakers. To become a future mover and shaker. To find their rightful place in the head of the political power curve, not down on the tail.

Most of the members of the Political Elite aren't very bright (I'm looking at you, Joe Biden). Rather, they're the beneficiaries of the gatekeeping system. Some have, through talent, personal exertion, and great personal expense passed through the gates dividing the Head from the Long Tail - say, an outsider graduating from Columbia Journalism School. Now in the Ruling Class, and saddled with mountains of student loans, they find that Palin (or Scott Brown, or the Tea Parties) seem to be "jumping the queue". Playing by different rules. Shattering the business model for entering into the Halls Of Power.

How could they not be incandescent with rage? They'd have to be a saint not to be. Welcome to the New Revolution.

When you hear talk of "teabaggers", or how "dumb" Palin is, or how Brown won "because he stood in the cold shaking hands outside Fenway Park", you know that you're talking to one of the people that the Internet is turning into a loser, as their political business model collapses around them. They'll all be gone in ten years.

These memes don't - can't - stand up to scrutiny. They're actually not intended to, any more than Jones and Mann thought that their scientific papers could stand up to skeptical challenges: the suppression of alternate ideas is all the proof you need to understand how strong they thought their own arguments were. It was all posturing - signaling - to the other Gatekeepers, as is chatter about "teabaggers".

The election of 2010 won't be the last in this Revolution. We're looking at a decade of upheaval, as the current "business model" of political power gets reshaped. It will probably take ten years before it settles down into a new Conventional Wisdom of how to succeed in the political market. Most of the current incumbents will be gone, and a bunch of people we've never heard of will be major players. All of these new faces will have something in common.

Like Sarah Palin and Scott Brown, they will have come out of the Long Tail, and will have bypassed the old Gatekeepers on their way to success.

The dinosaurs smell a change in the air, and roar their defiance.


TOTWTYTR said...

Most of us (bloggers) will remain down at the pointy end of the long tail. Some such as you, JayG, and AD, will end up closer to the head. Which is good because many have to fail (or have limited success) in order for a few to succeed.

What's important is that people are no longer passive consumers of the news product of the Lame Stream Media.

The gates are being crashed and the gate keepers are falling back in disarray.

It will be ugly, but it will be worth it if the media and power elite realize that they work for us, not the other way around.

If they don't, they won't be in power much longer.

doubletrouble said...

Wow- good one BP.

You should file this under "Remember when I said..." for future reference.

You packed that one full 'o smaht.

Borepatch said...

TOTWTYTR, I'm not in the same league as JayG and AD. I also think that it's not that blogs down at the tail are failing; on the contrary, I think that they're a massive amplifier.

But the rest of what you say is spot on. The top-down dinosaurs of the media and politics are sinking into a swamp. The bottom-up blogs are eating all their eggs. I don't know how it will play out, but it sure won't look like it looks today.

Doubletrouble, thanks. As you know, the problem is not getting me to talk about stuff for future reference; it's getting me to shut up ...

ASM826 said...

Perhaps your best post so far. I go and link now.

Anonymous said...

Well said! I applaud your analysis of the situation and think that your hypothesis will bear up under the test of time.

Dan in NC

bluesun said...

Is it the end of the world, or the beginning? I'm sometimes kinda scared at where we are going, but sometimes I hear something like this and it makes me a bit excited. One thing's for sure--it's gonna be interesting. Thanks.

Keith said...

Great post,

And now to ask the questions that the gate keepers have avoided hearing.

The British home office has never asked the question of whether gun control actually does any good to ordinary folks.

We know that all forms of violent crime were lower before each and every British gun control act than after.

A Tory multi millionaire is said to have bought the publisher of the British Gun mag "Guns review" before the '97 election to silence dear old Colin Greenwood.

Not anymore!

TOTWTYTR said...

Bluesun, it is both and end and a beginning. The end of media gatekeepers "shaping the message" to achieve their own ends and the beginning of "open source news" where you not only report, you decide.

Which is probably why Bill O'Reilly hates blogs and bloggers. Despite his populist veneer, he's a gatekeeper too, and will soon be watching the Internet eat his lunch.

Finally the promise of the First Amendment has come to life. The Founding Fathers would approve.

Anonymous said...

Perhaps it is that the internet is doing what movable type did before; the gatekeepers have been again denied to tool they used to keep the people in the dark, and we saw what happened the last time.


Lissa said...


stbaguley said...

May this (quite excellent) post never get filed under "counted chickens too early". Remember that the dinosaurs have the controls to the internet within their reach. Look to China, North Korea, the Fairness Doctrine, efforts by the commerce commision to protect us all from lies and smut because we don't know how ourselves. This fight has barely begun. "Interesting Times" was an ancient chinese curse as I recall.

Borepatch said...

stbaguley, there's an old saying: "The Internet interprets censorship as damage, and routes around it." Nobody has come up with a way to censor/control the Internet. The best they can do is to cut themselves off from it, and nobody (other than North Korea) is willing to do that.

Anonymous said...

Interesting post. 'Gatekeepers' is only one name for this: nomenklatura is another. THe insiders who get the benefits that all the other 'equal' people don't get.
And you are not alone: James deLong.


on another way to view the death of the special interest state.