How to prevent crawling and indexing some dynamic pages in same directory using robots.txt For eg: www.abc.com/page.php?mid=5040 www.abc.com/page.php?mid=416 -----------and so on I want to prevent all pages how I do the same. Please suggest if possible