My article, "Robots.txt Files".

Because I understand that members of HackThis.co.uk have found it particularly useful for one of the levels. Although it is basic, it is a useful thing to know, for hackers and webmasters.

It is located here if you are interested:
http://www.go4expert.com/showthread....3844#post43844