Robots.txtRequesting confirmation

Spiders.txt – Requesting confirmation

Hi all,

I get some sites which can be being built off from a structure I’ve personalized.The problem is the fact means many of the " documents" will probably be very equivalent, only changing inside the places the location where the name can be referenced (dynamic).

Please ok , i’ll know whether you think you will have any problems using the following that will block robot admission to those specific pages, which are however directory, while not restricting entry to all other pages.

Policy: User-Agent:*
Allow:/ Regards for what.


This entry was posted in Website Development and tagged , , , , , , , . Bookmark the permalink.

Related Posts

4 Responses to Robots.txtRequesting confirmation

Leave a Reply

Your email address will not be published. Required fields are marked *