Yep the robots.txt file useful for the website to protect the personal information from the public.