I love gSiteCrawler!

[Chris Lund]

I have a few comments as well as a few "I wishes"...

I own many websites, some of which are very large. LundList Business
Directory being one of them, with 101,000,000 unique urls. I
originally started out with gSiteCrawler a few years ago, and have
tried many others since, but I always go back to GSC.
Needless to say, with that many url's, my SQL 2008 gets a nice workout
(its dedicated just to GSC). We have even been running the tuning
advisor in the SQL box weekly as well as index rebuilts weekly as
well.

I can say this: gSiteCrawler is a VERY well built program. A little
slow on the larger websites, but still an awesome program.

I do wish that the original developers would update it though... it's
been 3 years!

For instance, some the querirs that GSC executes use 'SELECT Top 20
*..." thus returning possibly unused columns un-nec.
This has a huge effect on performance...

I also think GSC would be better served utilizing some of the newer
features of SQL Server 2003 & 2008, like ROWS, better stored
procedures, and triggers to offload some off the workload onto the SQL
instance...

Those things being said, I still LOVE this program!
Let me know if you ever need a beta tester in the future!

-Sincerely,
Chris Lund
Nevada Racks Hosting