Tell me more ×
Web Applications Stack Exchange is a question and answer site for power users of web applications. It's 100% free, no registration required.
up vote 4 down vote favorite
1

I heard some friends talking about Google's indexing percentage. They were saying that Google estimates the size of the whole internet as 5 million terabytes, and Google only indexes 0.04% of it.

Could that be true? Where can I find info about Google's indexing statistics?

ADDENDUM: I found a page containing this info: 18 Fun Interesting Facts You Never Knew About The Internet (Dec 31st, 2010). It's in number 9.

Google estimates that the Internet today contains about 5 million terabytes of data (1TB = 1,000GB), and claims it has only indexed a paltry 0.04% of it all! You could fit the whole Internet on just 200 million Blu-Ray disks.

share|improve this question
3  
They index my blog pages and I'm sure I'm not in the "interesting" 0.04% of the internet. – ChrisF♦ Jan 27 '11 at 12:21
Someone just voted to close this question as subjective or argumentative. How come? This is an objective question which asks for a confirmation about an info related to Google. – Mehper C. Palavuzlar Jan 28 '11 at 15:12
They should have voted to close as too localized since this will always be out of date – Eight Days of Malaise Jan 30 '11 at 2:39

closed as too localized by ChrisF♦ Jan 30 '11 at 10:51

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, see the FAQ.

2 Answers

up vote 3 down vote accepted

I think this depends on what your definition of "internet" is. Strictly speaking Google only indexes the Web, which is a subset of the internet. If you think about all the things that Google doesn't index then that would include:

  1. Email
  2. P2P Traffic
  3. Video and audio streams
  4. Stuff sitting on private FTP servers
  5. Instant message and VOIP chat
  6. All the other thousands of protocols that exist

Even then, there is lots of the stuff on the Web that Google can't index, either because it can't find it (content not linked anywhere), it's on a private intranet or extranet or a robots directive excludes it.

So whilst 0.04% of the Web doesn't sound correct, it could well be true that only 0.04% of internet "content" is indexed (this being the sub-set of the Web that Google can "see").

share|improve this answer
up vote 1 down vote

This paper estimated that in 2005 Google had indexed 76.16% of the web.

http://www.cs.uiowa.edu/~asignori/papers/the-indexable-web-is-more-than-11.5-billion-pages/

I don't think that is necessarily true though. There is a huge wealth of information on the web that can only be found via queries and not links, which Google can't index.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.