![]() |
|
|
#1
|
||
|
Well i want to do some testing in my forum so i made a clone of it in another domain i want to stop search engines spidering it and index. How to do it with a robot txt file ?
__________________
|
|
|
|
||
|
#2
|
||
|
User-agent: *
Disallow: / |
|
|
|
||
|
#3
|
||
|
Thanks raider, I've been looking for that everywhere. I am so glad that someone was finally capable of answering this question
|
|
|
|
||
|
#4
|
||
|
Thanks for the answer. Didn't know it was that easy. How you would do it you want to stop a particular search engine ? I'm sure is something pretty easy though I have no idea about it.
__________________
|
|
|
|
||
|
#5
|
||
|
Can I use this robot.txt file in custom script too?
I see this file in joomla dir too but I don't know how I can use this in general script. |
|
|
|
||
|
#6
|
||
|
What do you mean by a custom script file ? If you are going to build custom script and you want to stop search engines spidering pages you can use a robot.txt file. its same as we use in normal websites. Heres general commands of robot.txt file
http://www.hasitha.info/how-to-use-...gines-spidering
__________________
|
|
|
|
||
|
#7
|
||
|
usually i place a clone of a site in a subfolder and use robots.txt to avoid spidering these and with any custom scripts and general CMS like joomla and pligg u can have robots.txt of own.
|
|
|
|
||
![]() |
| Thread Tools | Search this Thread |
| Display Modes | |
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| robot.txt | puffin | Site & Server Administration | 2 | 01-05-2008 06:50 |