Tweet
Google, the greatest search engine, maintains an index of all the pages that it crawls in the database. This index is updated or reformed after a rough time interval of about one month however the exact dates fluctuate and are not disclosed. The update process of this mega search engine is popularity known as the “Google Dance”. Google also happens to be one of the very few search engines offering free submission, and that is why it is having over 3.4 billion pages in its database. SEO industry seems to revolve around this giant engine as nearly 77% search results on internet are powered by Google. So it becomes extremely important for a webmaster to understand the process of Google dance in order to plan optimization for a website.
Every dance starts with a deep crawl that entrails spidering the whole web from the start which ends up to many days. Google uses around 15000 servers spread all over the world at its data centers. Obviously an index update cannot be proceeded on all those servers at the same time. One server after the other has to be updated with the new index. Servers used by Google are placed at data centers mainly located in US. It is possible for Google to record all queries centrally and then distribute them to the data centers but this would obviously be inefficient. During the Google Dance, the data centers do not receive the new index at the same time. One after the other the data centers receive the updated index. So if a user queries Google during the Google Dance, it is possible that he may get the results from a data center which still has the old index at one point of time and after a few minutes from a data center which has the new index because DNS is resolved to that latest data center.
Google Dance can be identified by querying the IP addresses of its data centers which are unique for every center. Generally all queries on the IP addresses of data centers are redirected to www.google.com but there are special domains by Google that resolve to these data centers individually. Those domain and IP addresses are
Domain IP Address
www-ex.google.com 216.239.33.100
www-fi.google.com 216.239.41.100
www-ab.google.com 216.239.51.100
www-in.google.com 216.239.53.100
www-zu.google.com 216.239.55.100
www-va.google.com 216.239.37.100
www-dc.google.com 216.239.39.100
www-cw.google.com 216.239.57.100
The process of updation of index can be seen by visiting these domains. The results at each center vary from one another which is a proof of the process going on. Also each domain mentioned above i.e www-xx.google.com has an additional domain www-xx2.google.com and the IP address of such a domain ends on .101 instead of .100. These pairs of domains belong to the same data center and hence search at these two pairs shows same results. Index updates at an individual data center seem be done at one point of time. Servers seem to update by one half first and during this first-half update time the remaining half of server becomes active and serves search results for queries. Once the half under progress completes it becomes active and serves the results and the other half undergoes the same process. Users are very unaware about the changes occurring behind Google and feel normal search experience.
Google has employed test domains www2.google.com and www3.google.com that resolve to a fixed IP address. This fixed IP address is of the data center that updates the index at the earliest. As soon as the dance starts least of these domains is assigned the IP of the data center that receives the very first updated index. So you can check for future Google listings (listings after update) on these test domains. Webmasters are eager to view the changes in position of their listings at these test domains. If everything goes fine then its time to change the DNS records at Google, if no problems are found then Google’s DNS is resolved to the data center that is updated at the earliest. However if certain problems arise then the update at other data centers is cancelled, now the domain google.com will not resolve the IP of the malfunctioning data center but it will resolve the previous index and hence general users are unable to get what is happening behind the scene.
However nobody can bet about the exact internal workings of Google but a technical overview like this one is all the way very necessary and helpful to plan an SEO campaign. Other than that it can also help webmasters and SEO experts in planning new developments & updates to be launched online by a certain date and time so as to be properly included in the search engine database. Now you know when and how to determine Google updates and do a lot to improve your rankings.
Best of Luck!
Every dance starts with a deep crawl that entrails spidering the whole web from the start which ends up to many days. Google uses around 15000 servers spread all over the world at its data centers. Obviously an index update cannot be proceeded on all those servers at the same time. One server after the other has to be updated with the new index. Servers used by Google are placed at data centers mainly located in US. It is possible for Google to record all queries centrally and then distribute them to the data centers but this would obviously be inefficient. During the Google Dance, the data centers do not receive the new index at the same time. One after the other the data centers receive the updated index. So if a user queries Google during the Google Dance, it is possible that he may get the results from a data center which still has the old index at one point of time and after a few minutes from a data center which has the new index because DNS is resolved to that latest data center.
Google Dance can be identified by querying the IP addresses of its data centers which are unique for every center. Generally all queries on the IP addresses of data centers are redirected to www.google.com but there are special domains by Google that resolve to these data centers individually. Those domain and IP addresses are
Domain IP Address
www-ex.google.com 216.239.33.100
www-fi.google.com 216.239.41.100
www-ab.google.com 216.239.51.100
www-in.google.com 216.239.53.100
www-zu.google.com 216.239.55.100
www-va.google.com 216.239.37.100
www-dc.google.com 216.239.39.100
www-cw.google.com 216.239.57.100
The process of updation of index can be seen by visiting these domains. The results at each center vary from one another which is a proof of the process going on. Also each domain mentioned above i.e www-xx.google.com has an additional domain www-xx2.google.com and the IP address of such a domain ends on .101 instead of .100. These pairs of domains belong to the same data center and hence search at these two pairs shows same results. Index updates at an individual data center seem be done at one point of time. Servers seem to update by one half first and during this first-half update time the remaining half of server becomes active and serves search results for queries. Once the half under progress completes it becomes active and serves the results and the other half undergoes the same process. Users are very unaware about the changes occurring behind Google and feel normal search experience.
Google has employed test domains www2.google.com and www3.google.com that resolve to a fixed IP address. This fixed IP address is of the data center that updates the index at the earliest. As soon as the dance starts least of these domains is assigned the IP of the data center that receives the very first updated index. So you can check for future Google listings (listings after update) on these test domains. Webmasters are eager to view the changes in position of their listings at these test domains. If everything goes fine then its time to change the DNS records at Google, if no problems are found then Google’s DNS is resolved to the data center that is updated at the earliest. However if certain problems arise then the update at other data centers is cancelled, now the domain google.com will not resolve the IP of the malfunctioning data center but it will resolve the previous index and hence general users are unable to get what is happening behind the scene.
However nobody can bet about the exact internal workings of Google but a technical overview like this one is all the way very necessary and helpful to plan an SEO campaign. Other than that it can also help webmasters and SEO experts in planning new developments & updates to be launched online by a certain date and time so as to be properly included in the search engine database. Now you know when and how to determine Google updates and do a lot to improve your rankings.
Best of Luck!
Comment