Take the 2-minute tour ×
Web Applications Stack Exchange is a question and answer site for power users of web applications. It's 100% free, no registration required.

I heard some friends talking about Google's indexing percentage. They were saying that Google estimates the size of the whole internet as 5 million terabytes, and Google only indexes 0.04% of it.

Could that be true? Where can I find info about Google's indexing statistics?

ADDENDUM: I found a page containing this info: 18 Fun Interesting Facts You Never Knew About The Internet (Dec 31st, 2010). It's in number 9.

Google estimates that the Internet today contains about 5 million terabytes of data (1TB = 1,000GB), and claims it has only indexed a paltry 0.04% of it all! You could fit the whole Internet on just 200 million Blu-Ray disks.

share|improve this question

closed as too localized by ChrisF Jan 30 '11 at 10:51

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.If this question can be reworded to fit the rules in the help center, please edit the question.

3  
They index my blog pages and I'm sure I'm not in the "interesting" 0.04% of the internet. –  ChrisF Jan 27 '11 at 12:21
    
Someone just voted to close this question as subjective or argumentative. How come? This is an objective question which asks for a confirmation about an info related to Google. –  Mehper C. Palavuzlar Jan 28 '11 at 15:12
    
They should have voted to close as too localized since this will always be out of date –  Eight Days of Malaise Jan 30 '11 at 2:39
add comment

2 Answers

up vote 3 down vote accepted

I think this depends on what your definition of "internet" is. Strictly speaking Google only indexes the Web, which is a subset of the internet. If you think about all the things that Google doesn't index then that would include:

  1. Email
  2. P2P Traffic
  3. Video and audio streams
  4. Stuff sitting on private FTP servers
  5. Instant message and VOIP chat
  6. All the other thousands of protocols that exist

Even then, there is lots of the stuff on the Web that Google can't index, either because it can't find it (content not linked anywhere), it's on a private intranet or extranet or a robots directive excludes it.

So whilst 0.04% of the Web doesn't sound correct, it could well be true that only 0.04% of internet "content" is indexed (this being the sub-set of the Web that Google can "see").

share|improve this answer
add comment

This paper estimated that in 2005 Google had indexed 76.16% of the web.

http://www.cs.uiowa.edu/~asignori/papers/the-indexable-web-is-more-than-11.5-billion-pages/

I don't think that is necessarily true though. There is a huge wealth of information on the web that can only be found via queries and not links, which Google can't index.

share|improve this answer
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.