AD MANAGEMENT

Collapse

BEHOSTED

Collapse

GOOGLE

Collapse

How to create robot.txt File ?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • How to create robot.txt File ?

    How to create robot.txt File ?

  • #2
    This is relatively easily, you just need to open Notepad and save a text file as robots.txt, for information on what to put in these files, search Google for "robots.txt".
    Free Web Space - PHP, CGI, MYSQL and more from a friendly hosting provider.

    Comment


    • #3
      Hello jcworlditsme,

      I like your amazing information.
      Thanks

      Comment


      • #4
        Robot.txt files tell your instructions to a search engine robot. it depends on what you what it to do. if you dont have the robot.txt file then when a robot visits your site and if it doesnt find the file, it will start to visit all the pages and contents of your site. these files are like walls which helps in preventing some files like java scripts, some images or any other files from indexing. it is also important to make a note that it is not a method that prevents search engines from crawling your site. so it is important to create a robot.txt file. if you need this file to be created you can get it from Tucktail.com

        A robots.txt is a simple text file that can be created with Notepad. each robot.txt file should contain two parts
        1. User agent: * means all search bots should use the instruction to crawl the website. unless your site is a complex one you need not seperate instructions for seperate spiders.
        2.disallow : contains the files which should be avoided from crawling.
        Eg.
        User-agent: *
        Disallow: /wp-

        Comment


        • #5
          Robots.txt file is created to stop crawling access on some of the pages. You can easily find code for this in Google. Just copy and paste the code and give the page/folder urls you want to stop access and save the file as robots.txt format and upload at root of your web server

          Comment

          Unconfigured Ad Widget

          Collapse

          Announcement

          Collapse
          1 of 2 < >

          FreeHostForum Rules and Guidelines

          Webmaster forum - Web Hosting Forum,Domain Name Forum, Web Design Forum, Travel Forum,World Forum, VPS Forum, Reseller Hosting Forum, Free Hosting Forum

          Signature

          Board-wide Policies:

          Do not post links (ads) in posts or threads in non advertising forums.

          Forum Rules
          Posts are to be made in the relevant forum. Users are asked to read the forum descriptions before posting.

          Members should post in a way that is respectful of other users. Flaming or abusing users in any way will not be tolerated and will lead to a warning or will be banned.

          Members are asked to respect the copyright of other users, sites, media, etc.

          Spam is not tolerated here in most circumstances. Users posting spam will be banned. The words and links will be censored.

          The moderating, support and other teams reserve the right to edit or remove any post at any time. The determination of what is construed as indecent, vulgar, spam, etc. as noted in these points is up to Team Members and not users.

          Any text links or images contain popups will be removed or changed.

          Signatures
          Signatures may contain up to four lines

          Text in signatures is subject to the same conditions as posts with respect decency, warez, emoticons, etc.

          Font sizes above 3 are not allowed

          Links are permitted in signatures. Such links may be made to non-Freehostforum material, commercial ventures, etc. Links are included within the text and image limits above. Links to offensive sites may be subject to removal.

          You are allowed ONLY ONE picture(banner) upto 120 pixels in width and 60 pixels in height with a maximum 30kB filesize.

          In combination with a banner/picture you can have ONLY ONE LINE text link.


          Advertising
          Webmaster related advertising is allowed in Webmaster Marketplace section only. Free of charge.

          Shopping related (tangible goods) advertising is allowed in Buy Sell Trade section only. Free of charge.

          No advertising allowed except paid stickies in other sections.

          Please make sure that your post is relevant.


          More to come soon....
          2 of 2 < >

          Advertise at FreeHostForum

          We offer competitive rates and a many kinds of advertising opportunities for both small and large scale campaigns.More and more webmasters find advertising at FreeHostForum.com is a useful way to promote their sites and services. That is why we now have many long-term advertisers.

          At here, we also want to thank you all for your support.

          For more details:
          http://www.freehostforum.com/threads...eHostForum-com

          More ad spots:
          http://www.freehostforum.com/forums/...-FreeHostForum
          See more
          See less
          Working...
          X