1. 

   Izaan New Member

   Joined:

   Oct 16, 2007

   Messages:

   215

   Likes Received:

   2

   Trophy Points:

   0

   I would like to know if Yahoo follows the robots.txt rules.
   
   Some folders are disallowed In the robots.txt for bots to crawl and index. But Yahoo still indexed files under these folders, so I want to know if Yahoo bot DO recognize and comply with the robots.txt?

    

   Izaan, Apr 24, 2009

   SHARE #1
2. 

   shabbir Administrator Staff Member

   Joined:

   Jul 12, 2004

   Messages:

   15,375

   Likes Received:

   388

   Trophy Points:

   83

   They do but if you had the robots.txt updated recently it would take longer for the indexed pages to be removed from Yahoo

    

   shabbir, Apr 24, 2009

   SHARE #2

• Log in with Facebook

Your name or email address:

Do you already have an account?

• No, create an account now.
• Yes, my password is:
• Forgot your password?

Stay logged in