Robots.txtRequesting confirmation

Spiders.txt – Requesting confirmation

Hi all,

I get some sites which can be being built off from a structure I’ve personalized.The problem is the fact means many of the " documents" will probably be very equivalent, only changing inside the places the location where the name can be referenced (dynamic).

Please ok , i’ll know whether you think you will have any problems using the following that will block robot admission to those specific pages, which are however directory, while not restricting entry to all other pages.

Policy: User-Agent:*
Allow:/ Regards for what.


This entry was posted in Website Development and tagged , , , , , , , . Bookmark the permalink.

Related Posts

4 Responses to Robots.txtRequesting confirmation

  1. Levitra says:

    Aloha dude! I quite agree with your thoughts. I’ve just shared it on the social networks.

  2. Jeni Manely says:

    Very interesting details you have noted, thank you for putting up.

  3. Cialis says:

    You points about the topic discussed can’t be thought doubtful! Keep on writing!

Leave a Reply

Your email address will not be published. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>