To tell you the truth. In our previous articles, we talked about Search engines. But in this article, we are going to talk in detail about “how search engines work?”
Have you ever wondered how search engines work so quickly and show you the most relevant answers to your questions? Just imagine when you type and search something, the search engine will search through the whole internet and choose the results that are relevant and show them to you.
(Food for Thought: It is estimated that there are more than 1.7 billion websites on the internet)
So how to google, yahoo, bing or other search engines are sorting out these 1.7 billion websites.
Well before reading remember when you type a keyword and search it, you are not actually searching the live internet. You are searching the index files which are being crawled by search engine bots.
When “Archie”, the world’s first search engine was launched, it was designed as a downloadable directory listings site. During these 31 years, search engines have changed the whole world in a way that no one predicted.
Look how google looked like in January 1998
Source: Internet Archive
There are many search engines. But only a few of them are popular. Google holds first place with more than 86% of the market share. After that bing, Baidu, Yahoo!, Yandex and Ask are a few popular search engines on the internet.
These search engines use their own software and tools to deliver you information. But if you look closely they all act in the same manner.
Search engines will send bots (spiders) to the webpage and after getting the permission these bots will read the entire web page.
These spiders will start from a page and they will read the entire content. Then they will find the links on that particular page and will go to the other pages. Then they will read the content on those pages. Spiders/Bots will repeat this entire process until they read the entire content.
This is time to remember the google product called “Search Console”
In a future article, we will talk about how to check the crawling status of your website domain or certain page. How to opt-out of crawling? or how do make a request for crawling?
Crawlers (bots or spiders) will follow the link to the link and read all content. Then they will bring this content back to search engine servers. It acts as the head office of a multinational organization. Actually, search engines can be called the main centers of internet libraries.
With billions of web pages and trillions of content, search engines will store this information in an easy-to-find manner. This process is called indexing.
This is the part you want to keep special attention to. Because as digital marketers when a user searches for something related to our products or services we need to know how to get into the higher rank on the search result page.
To explain this simply let’s take an example. Imagine you are searching for “youtube” on google. Then google will search its index and select the relevant information. When I searched it, Google gave me fifteen billion two hundred ten million results. 15,210,000,000 youtube results?
You don’t trust me?
Look at the following image
How google gave me these search results in a fraction of a second. Actually in 0.47 seconds.
You won’t get the same results when you search in different search engines. Different search engines use different algorithms to display your results. This process is called “Ranking”. They will rank their indexed content according to the search queries.
The objective of ranking is to provide the best result to searchers with the most relevant information.
Search algorithms will decide your rank depending on various factors. Like the relevance to the search query, Word count, Location and freshness of the information, Quality and reliability, etc.
In this article We talked about the fundamentals of how search engines read website pages using their automated bots programs, in the indexing part how they store the pages they find through crawling, and how they rank those results in order to show the most relevant data.
See you in the next article. Until then stay safe.