POST thought I find out about the robots. txt from not too long ago. However, this specific key article retains me scratching my head, now i always re-read it:
what’s meant by a new squiggle () within this article
In particular, this example:
() = replace this squiggle by using any letters because of this example Or…
() = actually utilize this squiggle in any robots. txt file to be a wildcard Or…
() = several URLs actually contain a squiggle…
can a particular directive exclude several " image" versions on multiple levels
With all the standard method:
Disallow: /images/
Disallow: forum/images/
Disallow: clients/joes-diner/images/
Disallow: clients/hotel-albion/images/
Can doing this and more be replaced because of the following single series… or something same…
Is it advisable to place your blank index. htm report in every " disallowed" folder (such because the images folders) to discourage snooping But will it discourage snooping
POST. e., if a folder includes no index file–anyone whom types that folder throughout his address pub is fed a convenient report on everything in in which folder. I have read that this sort of snoop particularly likes to visit the robots. txt report for directions wherever to snoop. Which an index file which is blank–or has your notice " about to catch authorized to type in this directory… " –may considerably deter snooping…
Also, by the way…
Used to do a search right here at WDF with regard to threads with " robots" in the title. I only found this 2007 thread:
http: //www. webdesignforums. net/showthread. phpt=13061
It seems to me that statement in this thread is mistaken ():
Nonetheless I suspect this specific statement is accurate ():
() = several URLs actually contain a squiggle
usually denotes a house directory, so about some web contains, especially crappy versions, your account is at /username.