Advanced Crawler Optimization Methods
 

I am looking to construct web pages in a way that search engines will crawl it best.

I understand the basic tagging and on page SEO keyword related material.

I am looking for something more in the way the code is constructed or possible commands that can be given from the server or from more advanced programming so that pages can be crawled the best.

Any links, references, guidelines, articles, and subjects on what I am looking for would be very helpful and very much appreciated.

Re: Advanced Crawler Optimization Methods
 

Do you believe their is any server side or HTML bound commands I could give so that the most important content is crawled first or is this not possible?

Re: Advanced Crawler Optimization Methods
 

Content should be better, if you can gather a good page rank through link exchange then there is a chance of getting crawled easily and quite often.

Re: Advanced Crawler Optimization Methods
 

I don't believe that there is. I know that you can restrict what is being crawled, but am not not aware of any way to tell the crawlers in what order you want them to crawl.

Re: Advanced Crawler Optimization Methods
 

Hi,
In my knowledge, there is no such code to drive/force crawler to crawl your site. You can do it only through submission in active high PR sites. Active sites are those sites, which are updating and crawled frequently.
For example. Directory sites, Social bookmarking sites and forum sites. These sites updates very frequently and therefore, crawled fast.

Re: Advanced Crawler Optimization Methods
 

It is apparently true that sites with higher PR get crawled more often, but that isn't what the OP is looking to achieve.

Re: Advanced Crawler Optimization Methods
 

Again, the OP isn't looking to get the site crawled more often but to have it crawled in a certain order.

Re: Advanced Crawler Optimization Methods
 

+1 to this, as you can only make your web page search engine friendly and you can not do anything to get it crawled faster than search engine does it in its normal way.

Re: Advanced Crawler Optimization Methods
 

[You have used a quote in your comment but you have answered something different. The question was whether important content could be crawled first, not faster.

Re: Advanced Crawler Optimization Methods
 

The OP asked if there was an HTML command to make crawlers crawl their most important content first.
However, no-one has been able to offer such a solution though we have had answers concerning crawling faster, which is not the subject of the thread.

Therefore, if anyone can suggest a way to do what the OP asks, please post it.
Anyone who has any thoughts on getting crawled faster, please keep them to yourself as far as this thread is concerned.