Page 1 of 6 1 2 3 ... LastLast
Results 1 to 10 of 57

  1. #1

    Default What is a web crawler?

    Many of the website owners, SEO experts and forum members have been discussing about a web crawler. They consider it to be the most important element for the website owner. Is it a computer program? How does it help the sites? Does it help in browsing the World Wide Web in a methodical, automated manner? I heard that the web crawlers are also termed as ants, automatic indexers, bots, and worms. Is that true? Will this assist the website to be visible on the search engine? Can I expect the search engine spider to crawl the web pages of my site?

  2. #2


    Web crawler is a computer program that browses the web in a methodical, automated manner or in an orderly fashion. Sometimes called Spiders. Spiders also it identifies all the hyperlinks in the page and adds them to the list of URLs to visit.

  3. #3


    Website owners, SEO experts and many of the members of the forum discussion about crawler.Web a web crawler is a computer program that a systematic, automated manner or in an orderly fashion web browses.

  4. #4


    It is a machine that visit your site everyday.

  5. #5


    it can track your record and direct you when you require them another time. it can load the web pages as very quickly.

  6. #6


    It is a computer program that surfs the World Wide Web in a methodical way, automatically. Other terms for web crawlers are ants, automatic indexers, bots and worms or spider, web robot, or specifically in the FOAF community SCUTT web.

  7. #7


    I just by a spider crawl sitemap to provide a platform for

  8. #8


    A web crawler is a relatively simple automated program, or script, that methodically scans or "crawls" through Internet pages to create an index of the data it's looking for. Alternative names for a web crawler include web spider, web robot, bot, crawler, and automatic indexer.


  9. #9


    The web crawler is really a program which visits Internet sites and scans their pages along with other information to be able to create records for the search engines index.

  10. #10


    web crawler are search engine bots which collects the data from web and use that data to display results in Searching. The Google bot is the name of google web crawler.

Page 1 of 6 1 2 3 ... LastLast

Similar Threads

  1. spider/crawler
    By jaydee in forum Google Forum
    Replies: 3
    Last Post: 04-02-2010, 04:59 AM


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
| UKC | VPS Germany | Webmaster Marketplace | Web Hosting Forum | Domain Name Forum | VPS Forum | Webmaster Forum | Website Design Forum | Sites For Sale |