[Dspace-tech] Stats on 1.5

[Gary Browne]

Hi all

 

Can anyone point me in the right direction to get the statistics (stat-initial etc) working in 1.5.

 

When I run stat-initial, I get:

 

Exception in thread "main" org.postgresql.util.PSQLException: ERROR: syntax error at or near "FROM"
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:1512)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1297)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:188)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:430)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:346)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:250)
        at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:92)
        at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:92)
        at org.dspace.storage.rdbms.DatabaseManager.query(DatabaseManager.java:209)
        at org.dspace.storage.rdbms.DatabaseManager.querySingle(DatabaseManager.java:279)
        at org.dspace.app.statistics.LogAnalyser.getNumItems(LogAnalyser.java:1273)
        at org.dspace.app.statistics.LogAnalyser.getNumItems(LogAnalyser.java:1302)
        at org.dspace.app.statistics.LogAnalyser.processLogs(LogAnalyser.java:543)
        at org.dspace.app.statistics.CreateStatReport.statInitial(CreateStatReport.java:291)
        at org.dspace.app.statistics.CreateStatReport.main(CreateStatReport.java:162)

 

Don't really know where to start with this one.

 

In fact, I don't really know where to start with the stats reports in general - I find the wording of the documentation ambiguous and confusing.

 

Are stat-initial and stat-report-initial scripts that you would only run once in the beginning and then do nightly runs of the other four scripts?

 

The system seems to have changed since 1.4 eg:

 

1.4 dstat.cfg - contains dspace.log parameter

 

1.5 dstat.cft - contains host.name, host.url, start.year, start.month parameters

 

There are other anomalies, for example the stat-initial script in 1.5 states:

 

"This script starts from the year and month specified below.."

 

But I don't see any year or month specified??

 

Etc etc, I'll stop whinging now. I'm doing a server migration and a 1.4->1.5 upgrade at the same time and it has been quite a painful process. It would be great if someone could provide a straightforward explanation of how to initialise the stats and then keep them running daily? Or any ideas regarding the above exception.

 

Thanks a million

Gary

 

 

Gary Browne
Development Programmer
Library IT Services
University of Sydney

ph: 9351-5946

 

[Stuart Lewis]

Hi Gary,

> In fact, I don't really know where to start with the stats reports in general
> - I find the wording of the documentation ambiguous and confusing.
>
> Are stat-initial and stat-report-initial scripts that you would only run once
> in the beginning and then do nightly runs of the other four scripts?

Correct.

> The system seems to have changed since 1.4 eg:

Yes. The old stat scripts used to be written in Perl with embedded (and
repeated) variables such as start year and month. The scripts have been
re-written in Java to make them more portable, and the location of variables
has been changed (some use values from dspace.cfg, some have been moved into
dstat.cfg).

Make sure you follow step 15 of the 1.4->1.5 instructions carefully as they
tell you how to upgrade your stats scripts.

> There are other anomalies, for example the stat-initial script in 1.5 states:
>
> "This script starts from the year and month specified below.."
>
> But I don't see any year or month specified??

Whoops - so they do :(

Ignore those comments - the start year and month are now set in dstat.cfg.
I'll fix those scripts in svn now.

Report back if you're still having the exception, and we'll look into it
further. As it is an sql query issue, further diagnosis can be undertaken by
turning up your level of logging to DEBUG so that you can see the sql
queries that are throwing the exceptions.

Thanks,


Stuart
_________________________________________________________________

Gwasanaethau Gwybodaeth Information Services
Prifysgol Aberystwyth Aberystwyth University

E-bost / E-mail: Stuart...@aber.ac.uk
Ffon / Tel: (01970) 622860
_________________________________________________________________

[Gary Browne]

Hi Yin

Thanks very much for that, I wasn't aware of that bug. I'm forwarding
this reply to the list so that anyone else experiencing this problem can
update this class (LogAnalyser.java) to resolve the issue.

Cheers

Gary



Gary Browne
Development Programmer
Library IT Services
University of Sydney
ph: 9351-5946

-----Original Message-----
From: Yin Latt [mailto:y.l...@auckland.ac.nz]
Sent: Friday, 6 June 2008 9:27 AM
To: Gary Browne
Subject: RE: Stats on 1.5

Hi Gary,

It is reported as bug in LogAnalyser.java
http://sourceforge.net/tracker/index.php?func=detail&aid=1931799&group_i
d=19984&atid=119984

Fixed in check out code from the 1.5x branch:
http://dspace.svn.sourceforge.net/viewvc/dspace/branches/dspace-1_5_x/ds
pace-api/src/main/java/org/dspace/app/statistics/LogAnalyser.java?view=l
og

Based on my understanding,

stat-initial and stat-report-initial need to run once only (with very
initial installation of your DSpace)

"stat-initial" script will scan your dspace.log* and create "the *.dat
files"

From - dspace-log-monthly-(start.year)-(start-month).dat To
dspace-log-monthly-(current.year)-(current-month).dat
So, your system need to keep all dspace.log* to get correct
monthly statistics

"stat-monthly" script will scan your dspace.log* create/override "a
*.dat file"
dspace-log-monthly-(current.year)-(current-month).dat

"stat-general" script will scan your dspace.log* and create "a ***.dat
file"
dspace-log-general-(current.date).dat

stat-report-initial, stat-report-monthly, stat-report-general are to
convert **.dat to html under [dspace]/reports/

For migration or upgrade, if you re-run "stat-initial and
stat-report-initial" it will re-generate all "dspace-log-monthly.dat"
And if you don't keep all dspace.log you will lost the previous data of
monthly statistics.

For nightly cron job, you just need to run monthly and general only

<From documentation>
In order that statistical reports are generated regularly and thus kept
up to date you should set up the following cron jobs:
# Run stat analyses
0 1 * * * [dspace]/bin/stat-general
0 1 * * * [dspace]/bin/stat-monthly
0 2 * * * [dspace]/bin/stat-report-general 0 2 * * *
[dspace]/bin/stat-report-monthly

Cheers,

Yin Yin
------------------------------------------
Research Systems Analyst
Digital Services
The University of Auckland Library
Email y.l...@auckland.ac.nz
Phone: 64-9-3737599 Ext
------------------------------------------
1883-2008: Celebrating 125 Years


------------------------------

Message: 2
Date: Thu, 5 Jun 2008 16:22:08 +1000
From: "Gary Browne" <G.Br...@library.usyd.edu.au>
Subject: [Dspace-tech] Stats on 1.5
To: <dspac...@lists.sourceforge.net>
Message-ID:
<A479DFBFDD8F5948ADE9...@MAIL.mcs.usyd.edu.au>
Content-Type: text/plain; charset="us-ascii"

Don't really know where to start with this one.

In fact, I don't really know where to start with the stats reports in
general - I find the wording of the documentation ambiguous and
confusing.

Are stat-initial and stat-report-initial scripts that you would only run
once in the beginning and then do nightly runs of the other four
scripts?

The system seems to have changed since 1.4 eg:

1.4 dstat.cfg - contains dspace.log parameter

1.5 dstat.cft - contains host.name, host.url, start.year, start.month
parameters

There are other anomalies, for example the stat-initial script in 1.5
states:

"This script starts from the year and month specified below.."

But I don't see any year or month specified??

Etc etc, I'll stop whinging now. I'm doing a server migration and a
1.4->1.5 upgrade at the same time and it has been quite a painful
process. It would be great if someone could provide a straightforward
explanation of how to initialise the stats and then keep them running
daily? Or any ideas regarding the above exception.

Thanks a million
Gary


Gary Browne
Development Programmer
Library IT Services
University of Sydney
ph: 9351-5946

-------------- next part --------------
An HTML attachment was scrubbed...

------------------------------