What is robots.txt nad What is Robot's Meta Tag?

  • Filter
  • Time
  • Show
Clear All
new posts

  • What is robots.txt nad What is Robot's Meta Tag?

    What is robots.txt nad What is Robot's Meta Tag?

  • #2
    Robot.txt is instructions file for website crawler and bots, we just allow or disallow any files or folder of website. For example if we are using admin panel and url is domain/admin then we can disallow admin folder from robot.txt.

    Same like robot.txt robot meta tags instruct bots that this page should index or no index because we can let him know by robot meta tag. In robot meta tag we use index means search engine will index that page and no index means search engine should not index that page.
    SEO Manual Submission - Quality and manual services guarantee or money back.
    Rank Web SEO - Grow your business by rank website in search engines.


    • #3
      Robots.txt is the text file for crawler or bots, but Robots meta tags is the tag for crawler or bots.
      agencia de viajes a la india, viaje a la india desde mexico, triangulo de oro viaje india, tour india en espanol, viaje a la india desde colombia, agencia viajes india, viaje privado a la india


      • #4
        Robots.txt is a text file that tells search engine crawlers which pages on your website to index and which to ignore. You can use robots.txt to exclude pages containing sensitive information that you don't want to be indexed for any reason.

        Robot's meta tag is an HTML tag that tells web crawlers whether or not they should index a page. If the value of the robot's meta tag is "noindex," the page will not be indexed. Therefore, if you want a page to be indexed, you need to ensure that it is accessible to crawlers and does not have any noindex tags.

        So, robot.txt is suitable when you want to disallow whole sections (multiple pages), whereas the robot meta tag is preferable for individual pages.
        Buy Best WordPress Hosting


        • #5
          Robot. txt file is just a text file placed within the website, It allows Google bot to Crawl the listed URL's , Otherwise the bot not crawled.


          Unconfigured Ad Widget



          1 of 2 < >

          FreeHostForum Rules and Guidelines

          Webmaster forum - Web Hosting Forum,Domain Name Forum, Web Design Forum, Travel Forum,World Forum, VPS Forum, Reseller Hosting Forum, Free Hosting Forum


          Board-wide Policies:

          Do not post links (ads) in posts or threads in non advertising forums.

          Forum Rules
          Posts are to be made in the relevant forum. Users are asked to read the forum descriptions before posting.

          Members should post in a way that is respectful of other users. Flaming or abusing users in any way will not be tolerated and will lead to a warning or will be banned.

          Members are asked to respect the copyright of other users, sites, media, etc.

          Spam is not tolerated here in most circumstances. Users posting spam will be banned. The words and links will be censored.

          The moderating, support and other teams reserve the right to edit or remove any post at any time. The determination of what is construed as indecent, vulgar, spam, etc. as noted in these points is up to Team Members and not users.

          Any text links or images contain popups will be removed or changed.

          Signatures may contain up to four lines

          Text in signatures is subject to the same conditions as posts with respect decency, warez, emoticons, etc.

          Font sizes above 3 are not allowed

          Links are permitted in signatures. Such links may be made to non-Freehostforum material, commercial ventures, etc. Links are included within the text and image limits above. Links to offensive sites may be subject to removal.

          You are allowed ONLY ONE picture(banner) upto 120 pixels in width and 60 pixels in height with a maximum 30kB filesize.

          In combination with a banner/picture you can have ONLY ONE LINE text link.

          Webmaster related advertising is allowed in Webmaster Marketplace section only. Free of charge.

          Shopping related (tangible goods) advertising is allowed in Buy Sell Trade section only. Free of charge.

          No advertising allowed except paid stickies in other sections.

          Please make sure that your post is relevant.

          More to come soon....
          2 of 2 < >

          Advertise at FreeHostForum

          We offer competitive rates and a many kinds of advertising opportunities for both small and large scale campaigns.More and more webmasters find advertising at is a useful way to promote their sites and services. That is why we now have many long-term advertisers.

          At here, we also want to thank you all for your support.

          For more details:

          More ad spots:

          See more
          See less