The Internet is not as open as we think. There are a lot of things that the internet is hiding from us. We all know that Google is one of the best search engines, and it has answers to almost every single question. How would you react if I say that the internet we can access through Google is not more than 8-10% of the whole internet?
Yeah, it’s true. Search engines like Google, Yahoo, and others can only crawl and index 8-10% of pages. The rest of 90% of pages never get indexed by these search engines, which means we can only access a small portion of the internet.
The remaining part of the internet is known as the Deep Web and Dark Web. In this article today, I am going to define the concept of all these three layers of the internet. Along with that, I will compare and help you in understanding the difference between the Surface Web vs. Deep Web vs. Dark Web.
Surface Web vs. Deep Web. Bs Dark Web
If we consider the internet as the ocean, then the surface part of the sea that we can see that looks to spread all around you is the surface web. We can see it, and it seems easily accessible. But what about the more profound part? An ocean is not only the surface part. Well, the same thing is with the internet too. The deeper part beneath the surface considered as Deep Web, while the bottom part recognized as Dark Web. Still, confusing? Let’s discuss it in brief-
Before moving to the discussion part, we need to understand a term called indexing. So, indexing is a method search engines use to show the data on a web page. Search engines have crawlers or web spiders that visit a page then crawl all the pages available on that website and send it for indexing. If a site doesn’t allow crawlers to send the data for indexing, the site doesn’t get indexed and won’t show up on the search engines.
We are starting with the surface web. After all, we are familiar with this layer only.
Surface Web
The surface web is the outer layer of the internet. It is also very famous with names like Index Web, Light net, or Visible Web. We are eyeing the surface web only. Almost all our daily online activity that we perform takes place on the surface web. Many of us consider that it is the place where most of the internet exists. But in reality, it’s not true.
Almost all the web pages or internet that can be indexed by the search engines come under the surface web. It is easily accessible, and you can access it with the help of any normal browser such as Chrome or Mozilla. Even the web page where you’re reading all this information is part of the Surface web.
Deep Web
Deep Web is the layer of the internet beneath the surface web. Which means you can’t access it easily. The deep web is popular with the names like under web and hidden web. It usually consists of the data that you can’t find with a regular google search. There is no such report that claims how big the deep web is, but we can assume that it is at least 100-1000x bigger than the surface web, and it is growing bigger every day.
We can divide the deep web into two different categories. The first one is obscurity, which simply refers to anonymity. Websites in the categories block the search engine’s crawler to index the site. It can be easily achieved by adding a robot. txt file. Because of that, search engines can’t show the pages in search results.
The other part of the deep web is authenticated. Websites in this category are much secure than sites in obscurity as anyone who needs to access their resource need to enter valid credentials. Along with that, there are so many other ways to enhance security. Whitelisting IPs, giving access to authenticate persons, and some encryption are popular ways to improve security and anonymity.
However, all the websites stored in the deep web are not hidden necessarily. Even in a web blog or website, there might be some pages that the webmasters don’t want you or search engine to access, and they block those pages from the crawlers to index them. Those pages are also part of the deep web.
Mostly the data in the deep web are stored in databases instead of pages. That’s why accessing it becomes a bit difficult for a normal user. It usually contains the data of organizations, hospitals, inaccessible web pages like admin pages, Military armed forces, journalists, police, and crime units, etc. However, these kinds of data are highly secured and encrypted for regular internet users and accessing it is almost impossible as you need to go through so many security checks.
Dark Web
Commonly known as the darkest part of the internet. It is a tiny part of the deep web but contains highly secured and inaccessible data from the search engines as well as standard browsers. If you want to access the dark web, you need to download the TOR browser, which gives you anonymity at a certain level. Also, you need to have the URL of the dark web page, which ends with .onion. or I must say .onion links However, they don’t guarantee that you’ll become entirely anonymous. The TOR browser creates multiple layers of IP addresses to make you anonymous.
According to sources, the dark web was initially introduced and built by US Military researchers in the 1990s. The main goal was to create a space where spies and intelligence agencies can send and receive messages anonymously.
However, the dark web is also famous because of illegal activities such as selling documents, weapons, computer viruses, child pornography, hitman hiring, and many other unlawful activities. Most of the data on the dark web is illegal, hence those who use it keep it secretive.
I hope now you know the difference between the surface web, the deep web, and the dark web. If you have any queries, feel free to ask in a comment.
