Go4Expert - Does Yahoo Follow robots.txt rules?

Go4Expert (http://www.go4expert.com/)

- Search Engine Optimization (SEO) (http://www.go4expert.com/forumdisplay.php?f=33)

- - Does Yahoo Follow robots.txt rules? (http://www.go4expert.com/showthread.php?t=17100)

Izaan

24Apr2009 17:43

Does Yahoo Follow robots.txt rules?

I would like to know if Yahoo follows the robots.txt rules.

Some folders are disallowed In the robots.txt for bots to crawl and index. But Yahoo still indexed files under these folders, so I want to know if Yahoo bot DO recognize and comply with the robots.txt?

shabbir

24Apr2009 18:48

Re: Does Yahoo Follow robots.txt rules?

They do but if you had the robots.txt updated recently it would take longer for the indexed pages to be removed from Yahoo

All times are GMT +5.5. The time now is 03:16.