1. 

   infoway USA New Member

   Joined:

   Jun 14, 2010

   Messages:

   34

   Likes Received:

   0

   Trophy Points:

   0

   Occupation:

   Webmaster

   Location:

   West Hills, CA, USA

   Home Page:

   http://www.infoway.us/

   what is the code through which I can ask any search engine not to crawl my own site or part of the site?

    

   infoway USA, Jul 20, 2010

   SHARE #1
2. 

   shabbir Administrator Staff Member

   Joined:

   Jul 12, 2004

   Messages:

   15,375

   Likes Received:

   388

   Trophy Points:

   83

   Multiple ways.
   
   1. In robots.txt add the following line
   User-agent: *
   Disallow: /
   
   2. In Head Meta you can specify no crawl or index like
   
   <meta name="robots" content="noindex,nofollow" />
   
   3. Last but by no means least is make it a password protect directories using Apache

    

   shabbir, Jul 20, 2010

   SHARE #2
3. 

   nabeelarkisdata New Member

   Joined:

   Jun 19, 2010

   Messages:

   11

   Likes Received:

   0

   Trophy Points:

   0

   Occupation:

   Webmaster

   Location:

   Mauritius

   Home Page:

   http://rajeevkistoo.com/

   Disallow: / this is the code. this prevents search engines from crawling your site

    

   nabeelarkisdata, Jul 28, 2010

   SHARE #3

• Log in with Facebook

Your name or email address:

Do you already have an account?

• No, create an account now.
• Yes, my password is:
• Forgot your password?

Stay logged in