1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Does Yahoo Follow robots.txt rules?

Discussion in 'Search Engine Optimization (SEO)' started by Izaan, Apr 24, 2009.

  1. Izaan

    Izaan New Member

    Joined:
    Oct 16, 2007
    Messages:
    215
    Likes Received:
    2
    Trophy Points:
    0
    I would like to know if Yahoo follows the robots.txt rules.

    Some folders are disallowed In the robots.txt for bots to crawl and index. But Yahoo still indexed files under these folders, so I want to know if Yahoo bot DO recognize and comply with the robots.txt?
     
  2. shabbir

    shabbir Administrator Staff Member

    Joined:
    Jul 12, 2004
    Messages:
    15,287
    Likes Received:
    364
    Trophy Points:
    83
    They do but if you had the robots.txt updated recently it would take longer for the indexed pages to be removed from Yahoo
     

Share This Page