Tracking GSiteCrawler by User-Agent

3 views
Skip to first unread message

Fer

unread,
Feb 13, 2008, 10:47:34 AM2/13/08
to SOFTplus GSiteCrawler
Hi, we are trying to exclude the traffic generated by GSiteCrawler
using the user-agent string. We were expecting to see a big spike in
the reports, but it seems the crawler is not being tracked at all. We
cannot find it using the U-A neither the IP. Anybody knows what could
the reason be?
Thanks a lot!

webado

unread,
Feb 13, 2008, 11:31:00 AM2/13/08
to SOFTplus GSiteCrawler
The IP is your own, since the program runs on your own computer.

The default user agent string is:

GSiteCrawler/[VERSION] (http://gsitecrawler.com/)

but you can set it to any value in GSC > File > Global Optins >
Crawlers

webado

unread,
Feb 13, 2008, 11:34:09 AM2/13/08
to SOFTplus GSiteCrawler
You will only find GSC traffic in your server logs, not in any third
party tracking logs like Goolge Analytics or Statcounter.
The third aprty trackers cannot detect it because GSC is not javascrit
enabled (so GA doesn't detect it), nor is it image enabled (so
Statcounter's html-only or noscript tracking code won't detect it
either).



On Feb 13, 10:47 am, Fer <fer....@gmail.com> wrote:

Fer

unread,
Feb 13, 2008, 11:42:16 AM2/13/08
to SOFTplus GSiteCrawler
webado, very much appreciated!
Reply all
Reply to author
Forward
0 new messages