I know Google Matt has said that he prefers a tree-like structure in a
website as his personal preference.  I do to.

Trouble is, I'm got literally hundreds of product categories, and I am
using a CSS expanding menu to make jumping to ANY category a one-click
operation from any page (make pages for users, not for search engines,
right?).

Trouble is, GoogleBot gets confused by all the links.

Is their a method of causing GoogleBot to ignore certain links, and if
so, what is Google's prefered method of tagging links as such.

The site in question is www.lingerieplace.com.  You can see why I like
being able to have the expanding menu, it pleases the humans.  I want
to please GoogleBot and humans!

Any suggestions?

Hi phizzler,

huh, this looks weird for sure ... well using ie6, but anyway ... I
can't use the sub-menus 'cause  they are kinda transparent, so I see
through them having text and links optically mingled ^^

-luzie-

That's strange.  It seems to work fine for me in IE6 IE7 Firefox2
Firefox3 Opera Safari and Chrome.

I just checked the server error log, it found no problems.

The menu does load a gradient image to form a backdrop for the texts.
Perhaps it takes a while for IE6 to load.

Come to think of it, I think there was some version of IE6 which had a
bug relating to that very thing.  Though, you are the first one who
seems to have run into this problem on this site.

Anyone else?

On Oct 23, 4:10 am, luzie wrote:

Yes, you are right, but it took quite long here ^^

Ok, back to the questions:

Yes. It looks far better than I thought it would reading all this ;-)

If you lok at it it IS a tree-like-structure (optically) ;-)

Now, that's the interesting part of it all: confused in which way?

-luzie-

Google guidelines recommend a max of 100 links on a page, and although
it's not a fixed limit, I believe that GoogleBot stops readng links
after around 200.  This would mean that any lines within the actual
page would be ignored, and links towards the top of the menu would
automatically be favored.

I have a sitemap file for the robots, but I'm still concerned that it
may be better for the robot to ignore the links within the menu (or at
least the links more than 1 level deep).  I'm not sure whether I'm
right to be concerned, nor about how to go about marking the inner
links so that GoogleBot ignores them.

When I say Googlebot is getting confused, I mean that it has no
context to recognize the greater importance of higher-level menu links
vs lower-level links.

All pages can be accessed without the expanding menu (with multiple
clicks).

On Oct 23, 4:24 am, luzie wrote:

Right (may differ from site to site depending on 'strength' (~"PR").

I don't think so! (look at the cache-version, it's the complete page
they cached)

There's the point. They may not want to crawl pages beyond link number
100, or 150 or 180 (or something), so these pages would never show up
in the index.

You can NOT rely on the sitemap, they'll index only 'by links', not
'by lines in the sitemap', they don't do that, at least not before
they "trust" your site quite much.

You could mark second-level links as unimportant using "nofollow"? Or
use javascript instead of hypertext-anchors?

Which of course would be very necessary then, else the bot wouldn't
find the pages marked "nofollow" (or linked by javascript) in the main
menu.

It's not bad ... but still really very big, loading slowly, needing
several hundred pics (symbols) ... i don't really know ...

-luzie-

I like the nofollow idea.  That would probably be most in the spirit
of following Google's guidelines, which I wish to do.

If we assume that:
1. Google does, in fact, only consider the first, lets say, 200 links,
2. You have, say, 600 links.
3. The first, say, 400 links are tagged nofollow
does that mean:
a.  Google will only see only the 200 links which aren't tagged with
nofollow    or;
b.  Google will see no links, since the first 200 links were nofollow
and it "stopped trying"

Does that question make sense?

On Oct 23, 4:47 am, luzie wrote:

It does to me ... ;-)

Now, that's my opinion only: i'd say they'll consider following up to
100 (150, whatever) links per page, regardless of where they are in
relation to anything not crawlable (nofollowed links, javascript-
links), be that before, after or between it.

So the answer would be: a.!

;-)

-luzie-

THis came up before (a few times) about the css menu's etc.

I think the general responses was akin to...
... how is Google meant to identify the 'important' pages in that big
list of links ? ...

Which kind of makes sense.
Technically, when we structure navigation in such a way, we are
defining groupings... parents, children, grand-children etc...
users/visitors/bots have to "drill-down".

BUT
believe it or not - the important stuff is meant to be top-level
(parent)!
This is so that visitors can easily locate such info - straight away.
This is part of the problem with getting "inner-pages" indexed.

.

So - thats the theory side covered - and the clues are there.
We already have the suggestion of nofollow.
Brilliant ... now - can we make that dynamic?

For an example... lets say we have 2 Parents... Parent1 and Parent2.
In each of those, we have Children (7 each) and Grandchildren (2 per
child)

So we have a total of 44 pages (is that right?)
That 44 is split into 2 branches.
IF I go to Parent1 -> Child3(P1) -> Grandchild2(P1C3) ...
I don't really want the bots wandering off and looking at ANYTHING in
Parent2.
In fact ... the only things i want them to look at is the sibling (the
first Grandchild of P1C3) and the immediate Ancestor (child3) ... that
way it can move sidewards and back a step - thats it.

Make sense?

Now - I'm not saying that it will work - but the logic is there ...
you should be able to control what path the bots can take.  This means
you can (hopefully) stop them wandering off and looking at other
things rather than following the prescribed/logically path/branch ...
plus it means at any one time, you are automatically reducing the
number the of links that the bot should follow.

To go a step further, you could not only apply this logic to the path
being followed.... but to the depth as well.
Make the menu so that it only allows 1 step from current.
So if you are at Root and looking at the 2 Parents... the only links
you could follow are to the Parent pages...
If you are on a Parent page, the only places you could go is to the
other Parent pages, the child of the current Parent, or back to Root.

Again - this means we should be able to focus the bots attention on
what we want.

.

Does that make sense?

On Oct 23, 10:10 am, luzie wrote:

Just for the record, we can process more than 100 links per page :-).
We do however recommend the limit of 100 because it generally makes
sense for users (and search engines).

At any rate, I like how you did the CSS menu and I think the use of
rel=nofollow there is a great idea (good stuff, Luzie!)! Since we can
find the lower level pages from the higher level ones, you should be
all set.

As Autocrat mentioned, it sometimes helps us (and users) when the
higher level categories are easy to find, which is something your menu
does naturally for users -- and with the rel=nofollow search engines
can spot it that as well.

Keep up the good work, everyone!!

John