It's Rishi

The goal of any news delivery medium is to provide maximum signal-to-noise ratio to its target audience. “Signal” is the set of news items that is of interest to a person. “Noise” is everything else. The reality is that an infinitesimally small percentage of news is interesting to any given person. And that percentage is shrinking every day because more news is being created on a daily basis: more frequently are more people documenting more people who are doing more newsworthy stuff every day.

In order to keep SNR high, news mediums need to focus on the news interests of their audiences more intensely than ever before. However, trying to create a single focus for a group of individuals, each of whose interests differ somewhat, is not a long-term solution. Sites like PerezHilton.com, a leading Hollywood gossip blog, and TechMeme, a leading (especially here in the SV) tech news aggregator, provide a certain segment of the news to an audience specifically interested in that segment. However, over time, the amount of news created in the news segment grows and the the segment bulges. The news publisher either must choose to further narrow their segment, which will alienate some of their existing audience, or publish a higher volume of news, which ultimately lowers the SNR to any given audience member. Either of these options is not a good choice.

Long-term, the only news deliver medium which is viable is the roll-your-own news concept. Geeks here this and start throwing out terms like RSS and OPML but the bottom line is that you don’t have to know technology in order to determine whether a piece of news is interesting to you. Over the past months, I’ve found myself going to news sites, including TechMeme, less and instead refreshing Google Reader more. I’ve added many feeds and the news that arrives is astonishingly interesting to me. Most importantly, my Reader is astonishingly uninteresting to most other people. This kind of relevance is ultimately impossible to achieve by any news publisher that tries to appeal to more than a handful of people.

I don’t want you to conclude from this that I think the penultimate solution is the RSS Reader. The concept of explicitly adding feeds to a reader is just not going to fly with mainstream folks. So what is the perfect news medium that allows you to roll your own news but doesn’t require any tech savvy? Attempts have been made (NewsVine, etc..) but I think we have yet to see the killer news app.

Oh how mainstream media has changed over the past decades. Back in the 1960’s, during JFK’s presidency, news outlets wouldn’t publish any stories about the president’s infidelities. News editors had a sense of responsibility towards upholding the values and code of our society. There was no need to blemish the president’s name for little good would have come from it. Back then, news was controlled by a handful of agencies. Not only did these agencies have control over what news was received by citizens across the nation, but also when they received news. There were no 24-hour cable news channels and of course there was no Internet.

The landscape of news exchange/delivery today could not be more different. Major news outlets source and publish news around the clock and around the world. Americans are able to receive news wherever and whenever. News is no longer thought of as a single collection of headlines that you consume at once. Instead, news is a continuous flow of stories and headlines that is streaming whether you’re there to catch it or not. The consumption of news went from being a 30-minute event each morning or evening to being a virtually constant activity. How did this happen? Where is all this news coming from?? From two places:

The world shrank – Digital information networks enables news to efficiently travel across the globe in an instant. Now only can data travel at the speed of light, but there is a connected path from the news source to the news consumer. Often, very little human intervention is involved.

Citizen journalism – Digital cameras/videocams, camera phones, laptops, and wireless connectivity allow every one of us to capture the events of the world. I would venture a guess that the majority of Americans under the age of 30 now have atleast one device capable of digital capture with them at all times. We then take this digital information and disseminate it to the world via social-networking sites, blogs, online photo albums/streams, YouTube, message boards, etc.. An average citizen doesn’t have the reach of NBC or CNN, but as is seen every day on the Web, viral citizen media can spread like wildfire and ultimately achieve the same or greater reach as a mainstream media broadcast.

With so much news being created and so many new ways by which news can be spread, there is tremendous competition for people’s attention. I’m not suggesting that the big media companies are going to be extinct any time soon, but I am suggesting that their role in society is. Let’s face it, NBC was thrilled when Cho’s package arrived in their mailroom. NBC said they spent hours deciding whether to air footage from Cho’s videos on the air. There’s little doubt in my mind that they were going to broadcast it. How could they not? The fact that Cho chose to send the package to NBC affirms NBC’s stature as a dominant media outlet. The only issue that they may have been wrestling with was whether to air it and get a backlash from the public, politicians, or special interest groups who might denounce NBC for sensationalizing the Va Tech shooter. However, if NBC didn’t air the footage, they would have no doubt posted it on their news website, MSNBC.com. I’m sure the NBC execs realized that if they didn’t release it, eventually the material would at some point get leaked and in this case, NBC wouldn’t get the limelight for having the scoop.

If Cho would have simply posted all his videos to a MySpace page or YouTube, he would have demonstrated that the big media companies are simply becoming irrelevant. But, whether he knew it or not, what he did was smart. He knew that NBC would whore out the video footage as much as it possibly could since they would have the exclusive and others would inevitably do the job for him of ensuring that the video got on MySpace, YouTube, etc… The reach of his videos was maximized as a result.

Unlike 40 years ago during JFK’s presidency, the media companies a) can’t afford to ignore stories which will garner them attention and b) simply have little to no control over what stories make it to the public. If they don’t cover a story, someone else will. AOL Time Warner realized this a couple years ago and launched TMZ.com. TMZ.com is a hollywood news/gossip site that basically runs stories that AOL Time Warner couldn’t on their mainstream sites. TMZ.com stories often lack the journalistic integrity that a mainstream news organization would want to uphold. AOL Time Warner knew that this segment of news was too much in demand and too lucrative to ignore. And they were right: TMZ.com has been enormously successful and one of the fastest growing blogs on the Web. Moreover, TMZ.com relies heavily on citizen- captured stories, photos and videos and not a dedicated news team. TMZ.com is an example of an old media giant embracing the fact they are losing control of the news rather than trying to combat this fact. There can be little doubt that other media giants will follow suit with sites of their own which embrace citizen media.

A big part of being a trusted news source is providing comprehensive information. Increasingly, this means relying on sources beyond a dedicated news team. Dedicated news teams simply will not be able to scale to meet the volume of news consumption in the future. News sites like TMZ.com, which rely on citizen journalism, can scale and will be a crucial strategy for the big media companies to maintain their significance.

Hmm I know I’ve got some more thoughts on this but enough for now… =)

Memeorandum and other news aggregators are focused on what’s going on NOW which isn’t really always super interesting. However, geek bloggers know that news aggregators are a great way to build traffic to their own blogs. A simple trackback post to the news item will likely result in publicity for the blog in the form of a link in the discussion for that news item on the aggregator’s site. Keen to this, many bloggers are quick to post such follow-ups. Sometimes the blogger adds some great insight to the news item in which case their presence in the discussion is certainly merited, but often it’s the case that the follow-up post is light on value to the reader and thus adds little to the discussion.

Gabe, if you’re somehow reading this, my suggestion would be to track out-going clicks such that if there is a link to a post in a discussion where many users are clicking and then within a few seconds clicking browser-back and returning to Memeorandum, then treat that as a negative importance vote for that link (post). If the post has a high-frequency of users that do click thru and quickly return to Memeorandum and also if the post has 0 comments, just go ahead and boot it from the discussion links for the news item. Hopefully that would encourage bloggers to only post follow-ups if they truly have something meaningful to add. Furthermore, it would help keep a high signal/noise ratio for Memeorandum readers.

In my own experience, I have noticed that the most insightful and mind-tingling posts are those where the author/blogger has clearly spent some time composing their thoughts and not just trying to garner some quick attention. For some good reads, I suggest you check out some of the blogs listed over on the right column under “Some Feeds I Read”. Happy reading!

Blogs to Riches: The Haves and Have-Nots of the Blogging Boom. Oh, and just for reference while reading the article, I’m not an A, B, or C-list blogger. I’m probably a Z-lister. =(

The author, Clive Thompson, referencing Clay Shirky, an instructor at New York University specializing in the social dynamics of the Internet, has this scientific explanation of the grossly disproportionate traffic flow in the blogging world:

Economists and network scientists have a name for Shirky’s curve: a “power-law distribution.” Power laws are not limited to the Web; in fact, they’re common to many social systems. If you chart the world’s wealth, it forms a power-law curve: A tiny number of rich people possess most of the world’s capital, while almost everyone else has little or none. The employment of movie actors follows the curve, too, because a small group appears in dozens of films while the rest are chronically underemployed. The pattern even emerges in studies of sexual activity in urban areas: A small minority bed-hop, while the rest of us are mostly monogamous.

The power law is dominant because of a quirk of human behavior: When we are asked to decide among a dizzying array of options, we do not act like dispassionate decision-makers, weighing each option on its own merits. Movie producers pick stars who have already been employed by other producers. Investors give money to entrepreneurs who are already loaded with cash. Popularity breeds popularity.

Anyways, the article is a bit longish but well worth the read. It offers some great insight into the blogging community.

UPDATE: Clay Shirky’s original article Power Laws, Weblogs, and Inequality published back in 2003.

I just read on Scobleizer that coComment is launching a service that tracks the comments that you make in the blogosphere. Using coComment, you can:

1) See a centralized view of all the conversations (conversation = blog post + comment thread) that you have commented in. It’s not clear if they handle the recursive nature of this. For example, Scoble posts, gets 10 comments, I trackback and add my 2 cents, I get 5 comments, …. This is all one conversation but spanning many blogs.

2) Put a little widget on your blog that shows your readers what comments you’ve been making on other blogs. This is really what gets me most excited. Every comment you make on another blog is a little piece of your identity that now belongs to someone else. By making these comments visible on your blog as well, your comments become tied to your identity. That’s why, often times, if i have something insightful to say on a topic, I would rather post about it on my own blog rather than comment on someone else’s post.

3) Get alerts when conversations you’re involved in get updated.

My take: Great idea. Questionable implementation. However, to be totally fair, I will reserve judgement until I get a chance to really sink my teeth into it.

I have discussed a similar idea with several friends over the past couple months and just about everyone has agreed that there is a need for this. The concept of centralizing the decentralized nature of the blogosphere has already manifested itself and this is yet another example. Sort of making a personalized Usenet reader out of the blogosphere.

However, the implementation that I have sketched out is simpler and would not involve third-party bookmarklets. I’m not going to go into details right now but if I do get time to hack it together you’ll see what I have visioned. If anyone is curious, shoot me an e-mail.

At 4:02PM (Eastern), Google posts their Q4 earning results on the Business Wire. The big, big news (definitely the biggest news out of the valley for today) is that their numbers fell short of consensus estimates. At 5:11PM Reuters posts their summary of this news item and at 5:29PM, AP does the same. At around 5:30PM, this news cluster lands on Google News under the Business section. At 5:45PM, a CNN (via CNNMoney) writer has published an article covering this news.

It’s 6PM (Eastern), a full 2 hours since this news landed, and no sign of it on Memeorandum. This exhibits a limitation of pure algorithm-based aggregators is that in their attempt to maintain a high signal-to-noise ratio, they have a hard time grabbing big stories that are just breaking. However, I know several people that consider Memeorandum to be the best source for real-time Tech-business news. Clearly in this case, it is not.

What’s somewhat amusing about this is that the first news organization to post a follow-up to Google’s own announcement was not even US-based. It was The Financial Times, a London-based publication.

UPDATE: At around 6:15PM (Eastern), the news hits Memeorandum. The head story is the AP article and it has a couple posts from the blogosphere connected to it. I’m guessing what happens is that since there’s tons of news items posted by AP every day, there’s no way to isolate immediately which few are actually big news. Big news publications, in this case like CNN or TheStreet.com, publish fresh copy on the news and do not generally back-link. So, unless you are clustering news by relevance, you’re not going to be able to figure out what’s big until bloggers, for which back-linking is common practice, start posting about it.
Also, one could certainly argue that for 99.9% of people, a 2 hour delay is totally justifiable especially if it means keeping a high signal-to-noise ratio. I know for myself, this would usually be my preference as well.

Over the weekend I had dinner in the city with several friends. During our delicious meal, I was chatting with one of the friends about my recent blog post on Structured Blogging when something occurred to me. At the table was ten young 20-something successful professionals yet I was the only one there that had a blog. On the drive home, I realized that because I spend my days obsessing over the latest tech news/developments, my world is probably very skewed. In fact, a couple weeks ago at a poker game with some friends, I had a side conversation with a friend about Y!’s acquisiton of del.icio.us and nobody else in the room had heard of the news much less had even heard of a website called del.icio.us. When I get emails from friends sharing their recent photos, those emails are coming from old names like ImageStation and SnapFish not Flickr. Today, I had to spell out M-e-e-b-o several times to a friend who was looking for an IM solution to get around his company’s firewall. I could go on and on with examples…but you get my point.

Many of the techie bloggers in the blogosphere have a grossly skewed view of the world. We get so used to this community of early-adopters that focus is lost on the other 99% of the population. However, for a consumer application/service to achieve real success, it is crucial, of course, to capture the mainstream user which represents the overwhelming majority of the market. And, because “Web 2.0″ (I put that in quotes for a reason..heh) apps tend to be community-focused, attracting a wide audience of users is seemingly more important than ever.

One example of this skewed view is the recent talk about decentralized content and the power of self-publishing. The concept of Structured Blogging is built on this principle. As I mentioned in my post on the topic, if I’m going to write a movie review, I want to post it on my blog so I own it and it remains a part of my online identity. Similarly, posts like this predict the end of centralized sites (the author calls them “Walled Gardens”) like Craigslist and eBay because users will inevitably prefer to self-publish their classifieds ads on their own blogs.

The problem with these discussions is the reality that the number of Internet users who blog regularly is tiny. It’s hard to say exactly how many blogs there really are since I don’t trust most of the statistics on # of blogs because lots of people have blogs (sometimes several) but few actually post to it. According to this survey, only 7% of American adults read blogs regularly. If this is true, then Americans who actively publish via blogs has got to be no more than a couple %. Yet all this talk of self-publishing requires one fundamental thing: a place for the self to freely publish on the Web which for most people means having a blog. (Note: I say “freely” publish to exclude sites like MySpace which do limit the format of content that can be published by the user). And just a very small fraction of Americans blog.

From what I’m always reading about, the number of bloggers is rapidly rising so maybe, down the road, models involving decentralized content may become more and more of a reality. But, it does seem that we are not nearly as close as many tech bloggers make it seem.

It’s approaching 3AM right now and I’m not asleep. In fact, over the past year, my sleeping time has gotten later and later and later. Why you ask? Partly it’s because I’ve been busy working on my startup Dontbuyjunk and I’m often working late into the night until I’m satisfied with the progress that I’ve made for the day. But, I’m increasingly finding that what really is preventing me from getting to bed is information overload courtesy of the Internet. Let me explain.

I’ve been spending hours per day on the Internet for several years now. The big difference though is that recently the time I spend is shifting away from entertainment (mindless chatting on message boards, gaming, etc.) to information exchange activities such as reading/writing in the blogosphere. Every night, after I’m done working, I do one last catch up with my RSS reader and almost without fail, I end up spending a couple hours bouncing from one blog to the next and then to aggregators like del.icio.us and memeorandum.

Today, publishing (via the Web) is essentially free. And when I say “free” I mean that it both has no cost and is without rules or barriers. Furthermore, the second you publish your content, it is instantly accessible to a billion people. Because of all this, the rate at which information id created and disseminated is astonishing. So this is a good thing right?

Well…sure. enabling people to express and share both knowledge and opinions is great for society in countless ways. The problem that develops is that with so much publishing going on, how can I keep track of that tiny subset of information that is relevant, unique (remember that the majority of content published everyday is either syndication or basically duplicate) and valuable in my world? It’s getting harder by the day. Further exacerbating my problem is the wanting to not just read the facts behind a topic/news bit, but also read the opinions and participate in the many insightful discussions that branch from it.

So what’s the solution to my problem? Lunesta? Maybe. The next-generation of aggregators? Bingo.
One big trend that we are starting to see develop and I believe will be a major area of focus in the years to come is in information filtering and aggregation. Search engines like Google and centralized information sources like ESPN and Wikipedia allow me to pull in specific pieces of information when I am actively seeking it. However, their limitation stems from the fact that most of the information I absorb on a daily basis is new and could not have been searched for. In other words, if I didn’t know the information existed, how could I have searched for it? Instead, I must rely on my set of trusted sources to push this new information to me. Information aggregations, either human-derived (digg, reddit, del.icio.us) or algorithmic (memeorandum, blogniscient, Google News), are a step in the right direction. But aggregators have a long way to go before they truly are accurate and encompassing tools for information.

Anyways, it’s now 4:30AM and I’m basically just blabbing. Aggregators is an area that I’m becoming increasingly interested in myself and I have some of my own ideas brewing in my head about what the perfect aggregator would be and how it would work. I’ll be thinking and blogging about it in the coming weeks.

For some more discussions on aggregators, check out a blog post on memeorandum I was reading earlier that I found insightful:

http://mashable.com/2005/11/08/hacking-memeorandum-more-proof-that-algorithms-dont-work/

Be sure to read the comments thread.