How Much of the Internet Is Hidden?

Related on TestTube:
Who Created the Internet?
How Do Conspiracy Theories get started?

Each week on TestTubePlus, we cover one topic from multiple angles and perspectives, and this week Trace tackles something most of us take for granted but dominates our lives: the internet. Yesterday, he went over what exactly the internet is and how it works. Today, he's going to try and figure out: exactly how big is the internet, and how much of it is hidden from search engines?

As discussed yesterday, when the internet first came online back in 1969, it consisted of just four computers. Needless to say, it's grown quite a bit since...but just how big has it gotten? It's impossible to know for sure. Eric Schmidt, Google's former CEO, estimated that it consists of roughly five million terabytes of data. That's over five trillion megabytes. Google's web search has indexed about 200 terabytes of this data. That's a lot of data, but that comprises only 0.004 percent of the whole thing.

So what is happening on the other 99.996 percent of the internet? This is known as The Deep Web, and it's basically any content that's not indexed by standard search engines like Google or Bing. The Dark Web is a part of this, and you'll need specific software, configurations or authorization to access it. The most popular browser to access the deep web, TOR, uses a "virtual tunneling" system that acts like a multi-layered Virtual Private Network. It is for users to conduct anonymous and untraceable interactions online, and it's generally thought to be a haven for illegal activity. But it also provides a place for whistleblowers and political dissidents--so maybe it's not all bad? Join Trace as he tries to untangle this complicated web.

TestTube Plus is built for enthusiastic science fans seeking out comprehensive conversations on the geeky topics they love. Each week, host Trace Dominguez probes deep to unearth the details, latest developments, and opinions on big topics like black holes, the history of religion, dreams, space travel, gender, and more. TestTube Plus is also available as a podcast--click here to subscribe!

Learn More:

How Big Is The Internet? (WiseGeek)
"Assessing the size of the Internet is a somewhat difficult proposition, since it is a distributed body, and no complete index of it exists. What we mean by asking how large the Internet is also plays into how we answer the question."

How The Deep Web Works (How Stuff Works)
"About 40 percent of the world's population uses the Web for news, entertainment, communication and myriad other purposes [source: Internet World Stats]. Yet even as more and more people log on, they are actually finding less of the data that's stored online. That's because only a sliver of what we know as the World Wide Web is easily accessible."

What Is the IP Address of Google? (About Tech)
"Web site administrators like to know the IP addresses used by Google when it is spidering. You can also sometimes use Google's IP address to reach the site www.google.com if the domain cannot be contacted by its name."

Political misquotes: The 10 most famous things never actually said (CS Monitor)
"Everybody knows that Al Gore claimed to have invented the Internet. But like many things that everyone knows, it's not actually true. So where did it come from? In a March 9, 1999 interview on CNN, Wolf Blitzer asked the candidate to describe what distinguished him from his Democratic challenger, New Jersey Sen. Bill Bradley."