The whole SEO process can be divided among three major parts i.e.
Crawlabilty, Content and Links. These three things are pillars of
Search Engine Optimization and so to your site’s success path.
First keep this thing in your mind that Search Engine Spiders are not
all that smart. If a spider can’t find content (suppose there is a
broken link, for example), it’s programmed to end what it’s doing and
wouldn’t go look around for that great article you wrote. It’s going
to move on to the subsequent link and keep crawling, crawling, and
crawling. That’s what it does.
So, it’s obvious that if a spider can’t access your content then your
content will not get indexed and so will never be found on search
engines. That’s why crawlability is the basic and most important
component of SEO and we will highlight these crawlabilty issues.
1) Screwing up Robots.txt file.
Make sure your text is like the one written as User-agent:* Disallow:
It should not be like the one written as: User-agent:* Disallow: /
This mistake can drop your organic rankings drastically.
2) Too many variables or parameters in your URLs
Although Search Engines are getting better and better in means of
crawling long and ugly looking links but still they don’t like them.
Short URL’s also do get clicked more often in the result pages and
they are good for crawlabilty and clickabilty.
3) Session ID’s in your URL’s
Search Engines straight away do not like session ID’s in URL’s. If you
are using Session ID’s then just store them in cookies rather than
including them in your URL’s. Session ID’s can cause a single page
content to be visible at multiple URL’s and thus obstructing the
SERP’s.
4) Your site suffers from code bloat
Even though Spiders are normally good at sorting out code from
content, but that doesn’t mean you should make it more complex by
having so much code that the content is tough to find.
5) Your navigation and internal linking
Designers and developers can be pretty creative while designing a web
site but this creativity may come in the way Search Engine Crawlers.
Your site’s navigation build in complicated DHTML, java script code,
flash or Ajax based can stop a crawler in its track.
Though crawlers have become advanced that they can crawl these entire
things but why take a chance.Crawlability is often ignored in the name
of creativity and coding, but it’s as vital to your SEO efforts as
content development, link building, and any other element of the SEO
Strategy.
more details:
http://www.creeper-seo.com/archives/831/5-mistake-kill-seo-process/