Marc Gayle

Google just announced that they have rolled out a new indexing system for their search engine, called Caffeine. Here are some interesting quotes:

Some background for those of you who don't build search engines for a living like us: when you search Google, you're not searching the live web. Instead you're searching Google's index of the web which, like the list in the back of a book, helps you pinpoint exactly the information you need. (Here's a good explanation of how it all works.)

What's even more intriguing is the amount of data they process:

Caffeine lets us index web pages on an enormous scale. In fact, every second Caffeine processes hundreds of thousands of pages in parallel. If this were a pile of paper it would grow three miles taller every second. Caffeine takes up nearly 100 million gigabytes of storage in one database and adds new information at a rate of hundreds of thousands of gigabytes per day. You would need 625,000 of the largest iPods to store that much information; if these were stacked end-to-end they would go for more than 40 miles.

I can't even fathom that amount of data. To read the official Google announcement, check it out here.

This is a real-life example of the User Interface from the movie 'The Minority Report'.

Alexis your recent blog post to Kevin Rose re: Digg struck a wrong chord with me. For those not familiar with what he is talking about, a new video giving details about upcoming changes for Digg version 4 was released today.

I can see how on the surface these changes look like a clone of other more popular features, but I suspect you might be mis-interpreting what's happening here.

Yes, many of the features look and feel like competitors. But so does the iPod. It plays mp3s? Yuck! So does Rhapsody's player, why would I want one from Apple. So does a Mercedes/BMW, it has wheels + an engine.

The issue is the way it does what it does. I have not used it just yet, but from what I have seen, this could fundamentally be as much of a game changer as the first version of digg was to giving individual publishers a bigger voice.

Imagine...now, as a publisher, I have 200 'followers' whose sole interest is in seeing the links/stories that I publish. Jason Fried talks about building an audience. So far, there is no other tool that 'aggregates' all your content and pushes it directly to an audience like digg v4 is suggested to do.

People use Twitter to do this, but it wasn't built to do that. It was built for messages of 140 characters. Valuable in it's own right, but not as a 'media publishing' tool.

People use status updates on Facebook to do that, but it wasn't made for that. It was made to update friends on your latest activities (i.e. uploaded new videos/photos, add new quotes, friended new people, got engaged/married....other things that your social graph might find of value).

But for publishers trying to build an audience, there isn't one tool that allows an audience to 'subscribe' to everything of interest to them.Yes, you can have a blog with an RSS feed. Your twitter feed has an RSS feed, etc. But RSS feed readers suck!

The mere fact that I can have one place, that streamlines all of the links published by people I am interested in hearing from + links they found interesting, is absolutely game-changing (in my opinion).Ofcourse, it all comes down execution, but based on what I am seeing, I think I see Kevin back in his original form. Quite contrary to your assertion about pandering to his VCs, I don't see any evidence of driving revenues. I see mainly product development. I see small UI elements that seem to enhance the link discovery process. I see giving value to small-time publishers and further 'democratization' of news.

So while I respect that everyone is entitled to their own opinion, I would caution us all to wait and see how it unfolds, and actually use it - before casting judgement. Many high profile people have done that recently and have had to do mea cuplas.

Maybe I am biased, because I am one of those that submit stories that stall or get a handful of diggs. This looks like it could be a significant game changer for me. Maybe it won't guarantee all my stories will make it to the front page, but I would not be surprised if my content got easily 200% - 800% more traffic, as a direct result of this change.

THAT...would be game changing.

This is pretty ingenius. If I were hiring a creative guy, I would definitely hire this guy. 

Idea: Scratch and win.

Concept: Give consumers free stuff (on a very consistent basis) and give advertisers more brand engagement. Think woot.com but with free, quality, stuff (or actually even deeply discounted – per the coupon in the win).

Existing Competitors: http://itunes.apple.com/us/app/scratch-off-now/id358970551?mt=8 & www.scratchoffnow.com

That’s the idea in a nutshell. Here are some more details:

This app would be free in the App Store. Naturally, this idea would work best with folks that have existing relationships either with an ad agency (or agencies) or with brands and other advertisers.

The graphics would be holding a simple coin, and you use your real finger to scratch the ticket with the coin.

Game mechanics could be included, so points could be accumulated that users can redeem for some discounts (think Visa, American Express, Airlines, etc. kind of points).

The trick is to start with a bang, say giving away 500 – 1,000 iPads in the first 48 hours.

Then, once you have the momentum going you continue to provide small wins for the users even when they don’t ‘win’ a major prize (supplied by a vendor or brand).

The app can even be branded (charge for impressions above the results of the prize, or the prize itself is sponsored by some other brand).

Build functionality to allow users to ‘share points’ with others. Add a social aspect to it, so I can send 1,000 points to my sister so she can do whatever she wants with it.

Integrate it with Facebook Game Developers (like Zynga) where points and ‘tickets’ are hidden in the games – work out some sort of kickback commission with them. This can also be extended to other game platforms – like World of Warcraft, HeyZap, etc.

Implement scavenger hunts for ‘golden tickets’, where each ticket is a guaranteed prize – this comes back to the game mechanics idea earlier.

That’s all I can think of right now, and many people might ask…why don’t I do it. Well, to be honest, I am quite busy right now and I am not a big fan of the advertising model (i.e. working with brands and advertisers…it’s just not my forte. I am not bashing it…just saying it’s not my thing).

Hope this sparks some ideas and someone wants to run with it, that would be awesome. I would definitely download the app (provided it can work on my iPod Touch 2Gen).

You are free to take the idea and run with it. If you do run with it, some credit would be nice 

I have some additional ideas on other possibilities, so if you would like me to be involved in any way, feel free to reach out to me. You don’t have to…just saying.

P.S. Image courtesy of meddygarnet on Flickr.

 

Ok…hate is such a strong word. I don’t exactly hate them…I don’t hate anyone actually (not that I can think of ). However, the image below illustrates why they have fallen from grace, and shows what is fundamental wrong with them.

To explain, I just finished doing a fresh install of XP for a friend and am installing the pre-requisite stuff. Acrobat (PDF reader) was one of them, just because I needed something to read PDFs and that’s the first thing that comes to mind. So I google ‘Acrobat Download’ and am given a link that takes me to their download page. I click download and the screenshot captures what happens next (click to view fullscreen):

As you can see….they force me to download ‘Adobe Download Manager’ plug-in by ‘getPlus’ (who the hell is getPlus? – That’s rhetorical, I don’t know and don’t care they are also in my bad books) for Firefox. Then, I have to restart Firefox, which then loads Adobe Download Manager, which then hits me with a ton of banner ads that slide from side to side (that I can’t close or block) and, while downloading Acrobat – which I then have to install. On top of that, once acrobat is installed, they also installed ‘Adobe Air’. It could have been that they gave me the option to opt-out, but I was so pissed by this install process that I glossed over it, but it just made the entire process even worse.

Is Adobe joking? This reminds me of some shady ‘third-party’ app site. I will not be going through that process again, that’s for sure.

The worst part is, Acrobat isn’t even the best free PDF reader. Next time I will go with my usual default.

Adobe…stop worrying about Apple’s 3.3.1 change in their iPhone TOS and start focusing on your own customers and improving their experiences and lives. Maybe if you did that, and made Flash better people wouldn’t allow Apple to get away with the shafting they are now giving you and many other developers.

I have been stumbling across many of these infographics lately, and I couldn’t not share them here.

Intac.net recently put out an awesome infographic that illustrates the companies that dominate the internet (in terms of number of servers). Guess who is the leader, by a long shot. It’s quite staggering actually.Make sure to read the disclaimer at the bottom though….there’s always a disclaimer. Still informative and quite telling.

To check out the full post, go here.

P.S. Image 1 courtesy of Patrick H Lauke on Flickr.

Gizmodo recently put out a very interesting infographic that illustrates the many product spaces that the big three are directly competing with. Oh how I love free markets. Competition is one the of the best mechanisms ever!

To check out the full post at Gizmodo,go here

P.S. Image 1 courtesy of Dunechaser on Flickr.

Idea: Content Delivery Network powered by 'opted-in' desktops - i.e. a legal botnet CDN. This post was inspired by this comment on Hacker News. Explanation: CDNs now, are traditionally powered by data centers scattered all over the globe - that in many cases have edge networks where blade servers cache the most frequently requested content (i.e. content requested in the last 15-minutes). The good thing about this model, as opposed to downloading from a centralized server, is when you request content you can be downloading from the closest server to you - which results in faster download times. The problem is that CDNs nowadays are still only able to have a handful of these datacenters all over the world. Some people estimate that Amazon (the great Amazon) only has data centers in approximately 14 cities. I know that many CDNs have arrangements with major ISPs to house local edge servers, so users that belong to those ISPs can get access to those data much quicker. That is a step in the right direction, in terms of reducing total amount of network hops to the source file, but can't match the distribution network of an illegal P2P service. Suppose when you wanted to download a large file, though, you could download it from everybody in your neighborhood - at the same time. This drastically changes the equation. That's what P2P networks and clients kind of allow you to do. Even some games allow you to search for the closest server (hosted by an individual) to your IP. So, back to the billion $$ idea. If you were able to create a CDN that distributed content (you would have to determine whether or not the content would be encrypted and the trade-offs therein) to a major network of peers all over the world. Think of a legal botnet. Not a spyware botnet, where users are tricked into installing 'Anti-virus 2010', but a network that people register to join and are paid per GB of disk storage and bandwidth used. Let's look at some math. From the company's perspective - using Amazon's price sheet as benchmark (as of Jan 8, 2010):

• Charge $0.15/GB for storage
• Charge $0.17/GB for bandwidth

For calculations sake, we are only going to use these figures. The calculations can also be applied to a graduated scale of bandwidth used and storage consumed - for the user.

• Pay a peer $0.01/GB for storage
• Pay a peer $0.02/GB for bandwidth

The main issues we can see with this model are that for the lower allocations of disk space - provided by the user - and slower internet pipes, the user takes home a pittance (approximately $4.83 for a 1mbit connection with 50GB storage space).  However, for more powerful connections with higher quantities of space, a user can be more than doubling or tripling the cost of their internet connection every month (with prices falling so rapidly, especially in the US). With a 50mbit connection, and 50GB storage, a user can be taking home approximately $220 per month. Assuming 50mbit will be $50 throughout the US, following the lead of a town in Minneapolis, they have quadrupled the cost of their broadband connection - assuming the CDN uses their connection for the amount of time they allocate it for (and also assuming their computer is left running 24x7). Note: The Discount Factor in the spreadsheet model above is used to reflect the amount of time the user will allow the CDN to use their computer and bandwidth. So a 0.70 discount factor means that 70% of the time the CDN will have access to the users computer and bandwidth. Just to do a quick comparison to the large CDNs (in terms of their cost structure), here is something to think about. As far as I can tell, the largest line item expenses that the 'non-fibre-owning CDNs' have is the cost of bandwidth. Looking at Limelight Networks (LLNW), we see that their Cost of Revenue was approximately 64%. An unscientific comparison with the above model highlighted, shows our Cost of Revenue being 6.7% for storage, and 11.8% for bandwidth = approximately 18.5% of Revenues. Even if our model inputs were doubled ($0.02 for storage, $0.04 for bandwidth) cost of revenues would be approximately 37% (i.e. 50% lower than the traditional CDN). The CDN business is a multi-billion dollar business. With a model like this, scaling becomes more of a software problem - than a hardware problem - which is a beautiful place to be. You scale on the backs of ISPs and users are responsible for maintaining their machines. It's Google's Adsense business model implemented as a CDN - in short. Please feel free to tinker with the numbers, and leave comments for how I can improve the model. Disclaimer: This post assumes a number of variables, all of which are not fully fleshed out - due to the difficulty in fully conveying the complete picture in a simple blog post.

[caption id="" align="aligncenter" width="427" caption="David Plouffe - Obama for America"] [/caption]

Read the rest of this post »