Quote:
Originally Posted by Cole001 View Post
It is great when search engines frequently visit your site and index your content but often there are cases when indexing parts of your online content is not what you want.
Yes, for example my site has an admin section which is for my use only and which is disallowed by robots.txt