Go4Expert (http://www.go4expert.com/)
-   Internet Marketing (http://www.go4expert.com/forums/internet-marketing-forum/)
-   -   It will work for stopping google bots? (http://www.go4expert.com/forums/stopping-google-bots-t29369/)

davidgaffer 24Dec2012 23:55

It will work for stopping google bots?

I think I have messed up big time all my websites.
6 months ago I decided to change the robots.txt files in 9 of my websites, a few weeks later I noticed a big drop in traffic(I thought it was the google updates), now 6 months later, none of them are indexed in Google, only bing.

I am now suspecting that my robots.txt files were wrong.

Could you please tell me is this code rejects Google bots please?

User-agent: *

I have now replaced everything with:
User-agent: *
Disallow: /cgi-bin/

Thanks for your help everyone,

ozsubasi 28Dec2012 17:55

Re: It will work for stopping google bots?
Go to Google Webmaster Tools, click on Health from the menu on the left and then click on Blocked URL's from the dropdown menu.
This will show you how your robots.txt file is currently affecting your site, and you can test different variations of it to see what they do.

All times are GMT +5.5. The time now is 00:12.