What Code stop SE crawl a site?

infoway USA's Avatar, Join Date: Jun 2010
Go4Expert Member
what is the code through which I can ask any search engine not to crawl my own site or part of the site?
0
shabbir's Avatar, Join Date: Jul 2004
Go4Expert Founder
Multiple ways.

1. In robots.txt add the following line
User-agent: *
Disallow: /

2. In Head Meta you can specify no crawl or index like

<meta name="robots" content="noindex,nofollow" />

3. Last but by no means least is make it a password protect directories using Apache
0
nabeelarkisdata's Avatar, Join Date: Jun 2010
Go4Expert Member
Disallow: / this is the code. this prevents search engines from crawling your site