Glassfish / Tomcat Custom log

ssoler

unread,

Feb 3, 2010, 2:36:52 PM2/3/10

to Urchin Software from Google - help forum - Urchin 6

I'm running a couple of Glassfish servers and they have a log format
like this:

"75.90.99.253" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/
2009:10:03:32 -0500" "GET /edis/UW251 HTTP/1.1" 503 1037 "http://
www.google.com/search?hl=en&source=hp&q=black+snake+with+white+belly&aq=0&oq=black+snake+with+white+&aqi=g9"
"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET
CLR 1.1.4322; .NET CLR 1.0.3705; .NET CLR 2.0.50727; .NET CLR
3.0.4506.2152; .NET CLR 3.5.30729)" "NULL-COOKIE-VALUE"
"72.30.81.175" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/
2009:10:03:32 -0500" "GET /edis/document_fr258 HTTP/1.0" 503 1037
"NULL-REFERER" "Mozilla/5.0 (compatible; Yahoo! Slurp;
http://help.yahoo.com/help/us/ysearch/slurp)" "NULL-COOKIE-VALUE"

I'm trying to setup a custom log

PrimaryPositions: "157,158,0,3,91,10,11,92,13,14"
SecondaryPositions: -
PrimaryKey: -
SecondaryKey: -
PrimaryContent: HIT
SecondaryContent: -
CommentKey: #
FieldSeparator1: \s
FieldSeparator2: \t
QuotesEscapeSep: YES
BracketsEscapeSep: NO
MergeSuccessiveSep: NO
CleanWhiteSpace: NO
StatusRequired: YES
CustomDateFormat: -
CustomTimeFormat: -
TimeZoneOffset: 0

I'm not getting anything out of the logs it reads the file but returns
0

Any help would be greatly appreciated

Jeremy Aube

unread,

Feb 4, 2010, 7:38:48 AM2/4/10

to Urchin Software from Google - help forum - Urchin 6

You logfiles don't seem to contain a status code. This is usually
essential to get good data in Urchin. You do have the option, however,
to set StatusRequired to NO in the custom log format so that Urchin
doesn't expect a status code. There may be other issues as well, but I
would give that a try first and see if that fixes it.

Jeremy Aube
http://www.roirevolution.com/urchin

On Feb 3, 2:36 pm, ssoler <ssole...@gmail.com> wrote:
> I'm running a couple of Glassfish servers and they have a log format
> like this:
>
> "75.90.99.253" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/

> 2009:10:03:32 -0500" "GET /edis/UW251 HTTP/1.1" 503 1037 "http://www.google.com/search?hl=en&source=hp&q=black+snake+with+white+belly&..."

Geoff McQueen - Internetrix

unread,

Feb 4, 2010, 7:43:58 AM2/4/10

to urchin...@googlegroups.com

Also, it looks like you're getting a 503 error code on those log hits... you'll really want a 200 or a 304 to have the page views/hits count...

Jeremy Aube
http://www.roirevolution.com/urchin

--
You received this message because you are subscribed to the Google Groups "Urchin 6" group.
To post to this group, send email to urchin...@googlegroups.com.
To unsubscribe from this group, send email to urchin-help-...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/urchin-help-6?hl=en.

ssol...@gmail.com

unread,

Feb 6, 2010, 12:06:19 AM2/6/10

to urchin...@googlegroups.com

That is just a sample :) it happens that i pasted ones with 503 errors

Rakesh

unread,

Feb 11, 2010, 4:38:36 PM2/11/10

to Urchin Software from Google - help forum - Urchin 6

I can use your suggestions for Urchin with Glassfish.
How well they both go?
We want to have Urchin with Glassfish for our enterprise web
application.

Thanks,
Rakesh

On Feb 5, 11:06 pm, <ssole...@gmail.com> wrote:
> That is just a sample :) it happens that i pasted ones with 503 errors
>
> -----Original Message-----
> From: Geoff McQueen - Internetrix <geoff.mcqu...@internetrix.com.au>
> Sent: February 04, 2010 7:43 AM
> To: urchin...@googlegroups.com <urchin...@googlegroups.com>
> Subject: RE: [urchin-user] Re: Glassfish / Tomcat Custom log
>
> Also, it looks like you're getting a 503 error code on those log hits... you'll really want a 200 or a 304 to have the page views/hits count...
>
> -----Original Message-----
> From: urchin...@googlegroups.com [mailto:urchin...@googlegroups.com] On Behalf Of Jeremy Aube
> Sent: Thursday, 4 February 2010 11:39 PM
> To: Urchin Software from Google - help forum - Urchin 6
> Subject: [urchin-user] Re: Glassfish / Tomcat Custom log
>
> You logfiles don't seem to contain a status code. This is usually
> essential to get good data in Urchin. You do have the option, however,
> to set StatusRequired to NO in the custom log format so that Urchin
> doesn't expect a status code. There may be other issues as well, but I
> would give that a try first and see if that fixes it.
>

> Jeremy Aubehttp://www.roirevolution.com/urchin

>
> On Feb 3, 2:36 pm, ssoler <ssole...@gmail.com> wrote:
> > I'm running a couple of Glassfish servers and they have a log format
> > like this:
>
> > "75.90.99.253" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/

> > 2009:10:03:32 -0500" "GET /edis/UW251 HTTP/1.1" 503 1037 "http://www.google.com/search?hl=en&source=hp&q=black+snake+with+white......"

> For more options, visit this group athttp://groups.google.com/group/urchin-help-6?hl=en.

santos soler

unread,

Feb 16, 2010, 5:09:02 PM2/16/10

to urchin...@googlegroups.com

I have not been able to get anything from the access_log. It reads the contents, but nothing is parsed. A bigger / better sample. Any help would be appreciated. :)

"192.5.2.196" "if-srvv-edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/robots.txt HTTP/1.0" 200 1375 "NULL-REFERER" "ufl-gsa-crawler (Enterprise; S5-H6XPT6HTC8JJB; webadmin...@lists.ufl.edu)" "NULL-COOKIE-VALUE"
"192.5.2.196" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/robots.txt HTTP/1.0" 200 1375 "NULL-REFERER" "ufl-gsa-crawler (Enterprise; S5-H6XPT6HTC8JJB; webadmin...@lists.ufl.edu)" "NULL-COOKIE-VALUE"
"192.5.2.196" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/robots.txt HTTP/1.0" 200 1375 "NULL-REFERER" "ufl-gsa-crawler (Enterprise; S5-H6XPT6HTC8JJB; webadmin...@lists.ufl.edu)" "NULL-COOKIE-VALUE"
"66.249.71.45" "www.edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/robots.txt HTTP/1.1" 200 1375 "NULL-REFERER" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "NULL-COOKIE-VALUE"
"192.5.2.196" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/robots.txt HTTP/1.0" 200 1375 "NULL-REFERER" "ufl-gsa-crawler (Enterprise; S5-H6XPT6HTC8JJB; webadmin...@lists.ufl.edu)" "NULL-COOKIE-VALUE"
"192.5.2.196" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/robots.txt HTTP/1.0" 200 1375 "NULL-REFERER" "ufl-gsa-crawler (Enterprise; S5-H6XPT6HTC8JJB; webadmin...@lists.ufl.edu)" "NULL-COOKIE-VALUE"
"192.5.2.196" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/robots.txt HTTP/1.0" 200 1375 "NULL-REFERER" "ufl-gsa-crawler (Enterprise; S5-H6XPT6HTC8JJB; webadmin...@lists.ufl.edu)" "NULL-COOKIE-VALUE"
"192.5.2.196" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/robots.txt HTTP/1.0" 200 1375 "NULL-REFERER" "ufl-gsa-crawler (Enterprise; S5-H6XPT6HTC8JJB; webadmin...@lists.ufl.edu)" "NULL-COOKIE-VALUE"
"192.5.2.196" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/robots.txt HTTP/1.0" 200 1375 "NULL-REFERER" "ufl-gsa-crawler (Enterprise; S5-H6XPT6HTC8JJB; webadmin...@lists.ufl.edu)" "NULL-COOKIE-VALUE"
"72.30.81.175" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/style/edis-pub.css HTTP/1.0" 200 9964 "http://edis.ifas.ufl.edu/IG081" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)" "NULL-COOKIE-VALUE"
"128.227.160.143" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/pdffiles/FS/FS14600.pdf HTTP/1.1" 200 106519 "NULL-REFERER" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) [UsableNet Lift Assistive]" "NULL-COOKIE-VALUE"
"98.208.160.209" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/IN638 HTTP/1.1" 503 1037 "http://www.google.com/url?sa=t&source=web&ct=res&cd=1&url=http%3A%2F%2Fedis.ifas.ufl.edu%2FIN638&ei=9DSZSs3BNaid8QbTg9SmBQ&rct=j&q=chili+thrips+control&usg=AFQjCNHjWSQ8wVB1WdasqC3rwnudwJYdqA" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 GTB5" "139740647.3739565926248663600.1240394344.1245236802.1251111658.4"
"65.55.108.123" "edis.ifas.ufl.edu" "NULL-AUTH-USER" "29/Aug/2009:10:02:45 -0500" "GET /edis/LyraEDISServlet?command=getThumbnailImage&oid=15044673 HTTP/1.1" 503 1037 "NULL-REFERER" "WinHttp" "NULL-COOKIE-VALUE"

santos soler

unread,

Feb 17, 2010, 5:31:04 PM2/17/10

to urchin...@googlegroups.com

Found the problem and a possible solution :)

The problem is the (") in the IP address and the date example ("192.5.2.196" ... "29/Aug/2009:10:02:45 -0500") these have to be like this (192.5.2.196 ... [29/Aug/2009:10:02:45 -0500])

a sed command like this will take care of the clean up:

sed -e 's/"\([0-9\.]*\)"/\1/g' -e 's/"\([^"]*-0500\)"/[\1]/g' /path_to_glassfish_logs/access_log.%YYYY-0-%MM-0-%DD-12.txt

But this can be AVOIDED :)

1) Open your domain.xml and search for "access-log"
2) Change format="%client.name% %header.host% %auth-user-name% %datetime% %request% %status% %response.length% %header.referer% %header.user-agent% %cookie.value%" to format="combined"
3) Save and restart the domain
4) Look at your logs it should look like a standard Apache log
5) Configure Urchin profile log type as auto
6) Happy logs from Glassfish :)

Now to clean the old logs. :(

Rakesh

unread,

Feb 23, 2010, 12:17:17 PM2/23/10

to Urchin Software from Google - help forum - Urchin 6

Santos,

I did the same way, but now I have found that there is no cookie
information.

My logging as as follows after changing the format to combined:

127.0.0.1 - - [23/Feb/2010:09:03:49 -0600] "GET /wings/images/
__utm.gif?
utmwv=1&utmn=2144264443&utmcs=ISO-8859-1&utmsr=1280x1024&utmsc=32-
bit&utmul=en-us&utmje=1&utmfl=10.0%20r42&utmdt=WINGS%20Reporting
%20System&utmhn=wingsanalyticstest.com&utmr=-&utmp=/wings/spring/wp/
job-order/job-matching?execution=e6s1 HTTP/1.1" 200 35 "http://
wingsanalyticstest.com:8081/wings/spring/wp/job-order/job-matching?
execution=e6s1" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US)
AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.89 Safari/532.5"

127.0.0.1 - - [23/Feb/2010:09:03:55 -0600] "POST /wings/spring/wp/job-
order/job-matching?execution=e6s1 HTTP/1.1" 302 238 "http://
wingsanalyticstest.com:8081/wings/spring/wp/job-order/job-matching?
execution=e6s1" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US)
AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.89 Safari/
532.5" (****do you think in this line of logging also there should be
cookie information****)

Can you verify the above two lines of logging and tell me if it is
correct format or not?

On Feb 17, 4:31 pm, santos soler <ssole...@gmail.com> wrote:
> Found the problem and a possible solution :)
>
> The problem is the (") in the IP address and the date example ("192.5.2.196"
> ... "29/Aug/2009:10:02:45 -0500") these have to be like this (192.5.2.196
> ... [29/Aug/2009:10:02:45 -0500])
>
> a sed command like this will take care of the clean up:
>
> sed -e 's/"\([0-9\.]*\)"/\1/g' -e 's/"\([^"]*-0500\)"/[\1]/g'
> /path_to_glassfish_logs/access_log.%YYYY-0-%MM-0-%DD-12.txt
>

> But this can be *AVOIDED* :)

>
> 1) Open your domain.xml and search for "access-log"
> 2) Change format="%client.name% %header.host% %auth-user-name% %datetime%
> %request% %status% %response.length% %header.referer% %header.user-agent%
> %cookie.value%" to format="combined"
> 3) Save and restart the domain
> 4) Look at your logs it should look like a standard Apache log
> 5) Configure Urchin profile log type as auto
> 6) Happy logs from Glassfish :)
>
> Now to clean the old logs. :(
>

> On Tue, Feb 16, 2010 at 5:09 PM, santos soler <ssole...@gmail.com> wrote:
> > I have not been able to get anything from the access_log. It reads the
> > contents, but nothing is parsed. A bigger / better sample. Any help would be
> > appreciated. :)
>
> > "192.5.2.196" "if-srvv-edis.ifas.ufl.edu" "NULL-AUTH-USER"
> > "29/Aug/2009:10:02:45 -0500" "GET /edis/robots.txt HTTP/1.0" 200 1375
> > "NULL-REFERER" "ufl-gsa-crawler (Enterprise; S5-H6XPT6HTC8JJB;

> > webadmin-searc...@lists.ufl.edu)" "NULL-COOKIE-VALUE"