Filters for Xcart and vBulletin

[forno]

I like GSiteCrawler so much, I installed Parallel and XP on my Mac OS X
just to use it. :-)

My site uses xcart (e-store), vbulletin and linkmachine (link
management). Are there any filters I should be using to not crawl
registration pages, etc.

The store items go into the hundreds and there are 6,000+ forum
postings, but the site registers 20K+ pages including all the php
stuff. There are about 1,000 static html pages.

Thanks for in the input.

Also, this is only related -- would you recommend using an SEO package
for modifying the .php pages from xCart and vBulletin. There are
add-ons for both, I am considering using them. We sell stuff, and are
motivated to keep moving up in Google ranking.

FB

[forno]

Here's an update. The crawl has been going for over 2 hours, and I
still have 25K URLs to go, and the number is still going up.

I sounds like I really need to filter pages.

Any recommendations?

FB

[forno]

After more reading, I excluded store/, forum/ and linkmachine/ and
gsitecrawler shot through my static html pages, created the map, ftp'd
the .xml to my site and pinged google. Not bad.

Now, I need to decide how to make sure the store and forum are indexed
properly. For example, there are sitemap programs for vBulletin and
X-Cart, but how do I manage three sitemaps and present it to Google
correctly?

VB

[Tim L. Walker]

There are several good sitemap programs for vB that create them
automatically via a cron job. Check out the vBSEO sitemap program:
http://www.vbseo.com/f44/vbseo-google-yahoo-sitemap-generator-vbulletin-3-5-x-vbulletin-3-0-x-2648/

Cheers,
Tim L. Walker

--
Tim L. Walker

www.PhotographyCorner.com
www.WorldPhotoSeries.com

[forno]

Hi Tim,

I was hoping to hear from you. I have been reading vBSEO for a while
and have a question or two.

1. How to I consolidate my forum/ sitemap from vBSEO with the
public_html/ sitemap.xml that I just built using gsitecrawler?

2. How do I build a good sitemap for X-cart (there are SEO mods) and
consolidate it all into something that Googe likes?

We have a relatively large static html site that is probably more
important than forum/ or store/, so those pages really matter.

Help!
FB

[Tim L. Walker]

Just submit several sitemaps. For instance, I have 24 sitemaps for
PhotographyCorner.com - several that are created for the static pages,
several for the galleries, and a bunch for vB... You can add as many
sitemaps as you need to the Google Webmaster Console - just make sure
you give them different names. ;)

For the X-cart, while I'm not familiar with it myself, just browse the
site for a while, and look for redundant pages with different URLs,
and look for which parameters to drop, and keep adding them to
GSiteCrawler until you're satisfied you're not submitting dups. It'll
take a couple of run-throughs at first, but once you have it right,
you'll never have to worry about it until you change your site. ;)

Good luck,
Tim L. Walker

On 12/12/06, forno <ja...@fornobravo.com> wrote:
>

[forno]

Ah ha. Thanks. As the webmaster, how do I go about making sure they
don't overlap and create duplicate information? Do you just call them
sitemap1.xml, sitemap2.xml, etc.?

For example, I could build a map for the html, the forum/ and the
store/. Right?

I also want to double check. You definitely think it is better
installing vBSEO and creating a new forum sitemap.xml, rather than
putting a series of filters on gsitecrawler.com. Right? I am dragging
my feet at learning a new tool, mods, etc.
FB

[Tim L. Walker]

Yeah... I've found the auto sitemaps built for vB, at the very least,
take a lot less time time crawl (done automatically through the
database)... so very important if you have a large forum.

You can pretty much name them whatever you want, but sitemap1,
sitemap2, vbsitemap1, etc is what I use.

Cheers,
tlw

On 12/12/06, forno <ja...@fornobravo.com> wrote:
>

[forno]

Our forum is about 1,000 users and growing. We have about 6K postings
and 20-30 new ones a day. I am jumping back to vBSEO right now.

How does Google monitor and prioritize different sitemaps? Just
curious.
FB

[Tim L. Walker]

Yeah, even a forum that size would benefit from using the vBSEO
sitemap program (or there's another one that's good called vbgsitemaps
or something).

I don't think Google prioritizes them one way or another...

On 12/12/06, forno <ja...@fornobravo.com> wrote:
>

[softplus]

Just some short comments on this, here's what I would do:

- If you can split your site into logical sections and can use
"internal" or database-based sitemap generators for those sections,
then I would definitely aim to do it that way. It is an absolute waste
of resources to crawl a site which has the URL structure in a database
already :-) (plus it takes forever for a large site)

- vbSEO is the way to go -- if I were to set up a new forum, it's
probably only add-on which I would want from the start. Forums are
terrible with regards to SEO, absolute crawler-traps. They need a good
structure and help to push the crawlers into the right areas.

- I would set it up as follows:

/sitemap-index-forum.xml
- with links to the sitemap files for the forum, perhaps in a
sub-directory (which is now possible), eg:
- /forum/sitemap-1.xml
- /forum/sitemap-2.xml

/sitemap-index-shop.xml
- ditto like above

/sitemap-general-index.xml
- with all the URLs which are not in the forum or shop, crawled with
the GSiteCrawler, with filters set to keep it from entering the shop or
forum sections.

Then you would only need to submit those 3 sitemap-index files to
Google. If you add more sitemap files, you can just adjust the
sitemap-index files and ping Google about it. I hope that makes sense
:-)

John

[forno]

Thanks John,

I don't think the forum is large enough to justify a sub-directory.
Just a root sitemap.xml.

So the plan is that I will:

1. leave the gsitecrawler maps at root, as is (or rename it).
2. install vBSEO and their sitemap, and ftp that to root.
3. find, buy and install an X-Cart SEO/map tool, and do the same

I will then monitor and maintain the three sitemaps.xml.

Make sense?
FB