I have got ReportMagic running perfectly via Analog and the ProFTPD
logs. The problem is this;
I want to be able to view what users are logging in, what they are
downloading and for how long. I believe this all relates to the log
file format. Does anyone have any experience with ReportMagic and
ProFTPD ?
You can get the users who logged in, out of your basic report by including the User report (USERS ON in analog.cfg).
You can tell what files everyone is downloading using the Request report (REQUEST ON in analog.cfg), but that should be on by default.
Unfortunately, Analog does not do multivariate (or cross-reference) reports directly <http://www.analog.cx/docs/faq.html#faq128>, so you can't get this out-of-the box without some extra work.
Once you have that you can find out what each user has downloaded by running a new report for that user (USERINCLUDE xxxx in analog.cfg). See the include command <http://www.analog.cx/docs/include.html> for details. It probably makes sense to only run this for your top N users.
You can't precisely tell how long someone was on the site under HTTP<http://www.analog.cx/docs/webworks.html>(which is what Analog was designed for) so it doesn't do time-on-site reports. However, you could use the Processing Time report (PROCTIME ON in analog.cfg) to tell how long the files take to download, in a historgram. Of course, you could also use the USERINCLUDE method to get this for individual users if you wish.
On Wed, Sep 22, 2010 at 2:33 AM, cultavix <eaton.ja...@gmail.com> wrote: > Hi all,
> I have got ReportMagic running perfectly via Analog and the ProFTPD > logs. The problem is this;
> I want to be able to view what users are logging in, what they are > downloading and for how long. I believe this all relates to the log > file format. Does anyone have any experience with ReportMagic and > ProFTPD ?
Thank you for your quick reply, I am very thankful. I will try these
things out and see how it goes.
As of now, I can get a near perfect report of xferlog from ProFTPD but
the problem is, that log only collects information of the transfers,
not the users....
I think that the only log to contain the users and everything else I
want is the "Extendedlog" that Proftpd can generate.
Which log should I be analyzing?
Cheers!
James
On Sep 22, 5:21 pm, Jeremy Wadsack <jeremy.wads...@gmail.com> wrote:
> You can get the users who logged in, out of your basic report by including
> the User report (USERS ON in analog.cfg).
> You can tell what files everyone is downloading using the Request report
> (REQUEST ON in analog.cfg), but that should be on by default.
> Unfortunately, Analog does not do multivariate (or cross-reference) reports
> directly <http://www.analog.cx/docs/faq.html#faq128>, so you can't get this
> out-of-the box without some extra work.
> Once you have that you can find out what each user has downloaded by running
> a new report for that user (USERINCLUDE xxxx in analog.cfg). See the include
> command <http://www.analog.cx/docs/include.html> for details. It probably
> makes sense to only run this for your top N users.
> You can't precisely tell how long someone was on the site under
> HTTP<http://www.analog.cx/docs/webworks.html>(which is what Analog was
> designed for) so it doesn't do time-on-site
> reports. However, you could use the Processing Time report (PROCTIME ON in
> analog.cfg) to tell how long the files take to download, in a historgram. Of
> course, you could also use the USERINCLUDE method to get this for individual
> users if you wish.
> --
> Jeremy Wadsack
> On Wed, Sep 22, 2010 at 2:33 AM, cultavix <eaton.ja...@gmail.com> wrote:
> > Hi all,
> > I have got ReportMagic running perfectly via Analog and the ProFTPD
> > logs. The problem is this;
> > I want to be able to view what users are logging in, what they are
> > downloading and for how long. I believe this all relates to the log
> > file format. Does anyone have any experience with ReportMagic and
> > ProFTPD ?
I really don't know much about ProFTPD logs. A quick look at the docs<http://www.proftpd.org/docs/howto/Logging.html>, though, suggests that the ExtendedLog is what you want to use. It looks like you could configure it to have all the fields you want and then can analyze that.
I don't think you'll need to create both logs, just one or the other, and Extended sounds right. If you do have both logs and they don't all have the information you want, Analog can sometimes read both sets of logs and still build reports. You just add a LOGFORMAT command for each file *before* the LOGFILE command that loads it.
On Thu, Sep 23, 2010 at 2:08 AM, cultavix <eaton.ja...@gmail.com> wrote: > Thank you for your quick reply, I am very thankful. I will try these > things out and see how it goes.
> As of now, I can get a near perfect report of xferlog from ProFTPD but > the problem is, that log only collects information of the transfers, > not the users....
> I think that the only log to contain the users and everything else I > want is the "Extendedlog" that Proftpd can generate.
> Which log should I be analyzing?
> Cheers!
> James
> On Sep 22, 5:21 pm, Jeremy Wadsack <jeremy.wads...@gmail.com> wrote: > > You can get the users who logged in, out of your basic report by > including > > the User report (USERS ON in analog.cfg).
> > You can tell what files everyone is downloading using the Request report > > (REQUEST ON in analog.cfg), but that should be on by default.
> > Unfortunately, Analog does not do multivariate (or cross-reference) > reports > > directly <http://www.analog.cx/docs/faq.html#faq128>, so you can't get > this > > out-of-the box without some extra work.
> > Once you have that you can find out what each user has downloaded by > running > > a new report for that user (USERINCLUDE xxxx in analog.cfg). See the > include > > command <http://www.analog.cx/docs/include.html> for details. It > probably > > makes sense to only run this for your top N users.
> > You can't precisely tell how long someone was on the site under > > HTTP<http://www.analog.cx/docs/webworks.html>(which is what Analog was > > designed for) so it doesn't do time-on-site > > reports. However, you could use the Processing Time report (PROCTIME ON > in > > analog.cfg) to tell how long the files take to download, in a historgram. > Of > > course, you could also use the USERINCLUDE method to get this for > individual > > users if you wish.
> > -- > > Jeremy Wadsack
> > On Wed, Sep 22, 2010 at 2:33 AM, cultavix <eaton.ja...@gmail.com> wrote: > > > Hi all,
> > > I have got ReportMagic running perfectly via Analog and the ProFTPD > > > logs. The problem is this;
> > > I want to be able to view what users are logging in, what they are > > > downloading and for how long. I believe this all relates to the log > > > file format. Does anyone have any experience with ReportMagic and > > > ProFTPD ?
Thank you once again for your quick reply. There is not much help out
there and if there is, I cant find it!! :P
Anyhow, me and my work-mate were working on this today and we cannot
seem to get the correct output from analog that we need for
reportmagic to work. We simply want the name of the usernames(local
userid), his IP address/hostname, the date/time, the files that were
dl/upl and a few other things. We can only get it working by simply
using the defaults, which do not include the local userid. So if we
could somehow add on to the default configuration the userid at least,
that would be good enough for now.
I can supply you with my conf file if you would like, here is a sneak
peek;
This is from our analog.cfg;
#LOGFORMAT (%j %M %d %h:%n:%j %Y %t %S %b %r %j %j %j %j %u %j %j %j)
LOGFILE /var/log/proftpd/extendedlog.log
This is from our proftpd.conf;
LogFormat test-string "%h %l %u %t \"%r\" %s %b"
ExtendedLog /var/log/proftpd/extendedlog.log ALL test-string
Any help is apreciated...
Thank you very much!!!!
- James (and jacob)
On Sep 23, 5:36 pm, Jeremy Wadsack <jeremy.wads...@gmail.com> wrote:
> I really don't know much about ProFTPD logs. A quick look at the
> docs<http://www.proftpd.org/docs/howto/Logging.html>,
> though, suggests that the ExtendedLog is what you want to use. It looks like
> you could configure it to have all the fields you want and then can analyze
> that.
> I don't think you'll need to create both logs, just one or the other, and
> Extended sounds right. If you do have both logs and they don't all have the
> information you want, Analog can sometimes read both sets of logs and still
> build reports. You just add a LOGFORMAT command for each file *before* the
> LOGFILE command that loads it.
> --
> Jeremy Wadsack
> On Thu, Sep 23, 2010 at 2:08 AM, cultavix <eaton.ja...@gmail.com> wrote:
> > Thank you for your quick reply, I am very thankful. I will try these
> > things out and see how it goes.
> > As of now, I can get a near perfect report of xferlog from ProFTPD but
> > the problem is, that log only collects information of the transfers,
> > not the users....
> > I think that the only log to contain the users and everything else I
> > want is the "Extendedlog" that Proftpd can generate.
> > Which log should I be analyzing?
> > Cheers!
> > James
> > On Sep 22, 5:21 pm, Jeremy Wadsack <jeremy.wads...@gmail.com> wrote:
> > > You can get the users who logged in, out of your basic report by
> > including
> > > the User report (USERS ON in analog.cfg).
> > > You can tell what files everyone is downloading using the Request report
> > > (REQUEST ON in analog.cfg), but that should be on by default.
> > > Unfortunately, Analog does not do multivariate (or cross-reference)
> > reports
> > > directly <http://www.analog.cx/docs/faq.html#faq128>, so you can't get
> > this
> > > out-of-the box without some extra work.
> > > Once you have that you can find out what each user has downloaded by
> > running
> > > a new report for that user (USERINCLUDE xxxx in analog.cfg). See the
> > include
> > > command <http://www.analog.cx/docs/include.html> for details. It
> > probably
> > > makes sense to only run this for your top N users.
> > > You can't precisely tell how long someone was on the site under
> > > HTTP<http://www.analog.cx/docs/webworks.html>(which is what Analog was
> > > designed for) so it doesn't do time-on-site
> > > reports. However, you could use the Processing Time report (PROCTIME ON
> > in
> > > analog.cfg) to tell how long the files take to download, in a historgram.
> > Of
> > > course, you could also use the USERINCLUDE method to get this for
> > individual
> > > users if you wish.
> > > --
> > > Jeremy Wadsack
> > > On Wed, Sep 22, 2010 at 2:33 AM, cultavix <eaton.ja...@gmail.com> wrote:
> > > > Hi all,
> > > > I have got ReportMagic running perfectly via Analog and the ProFTPD
> > > > logs. The problem is this;
> > > > I want to be able to view what users are logging in, what they are
> > > > downloading and for how long. I believe this all relates to the log
> > > > file format. Does anyone have any experience with ReportMagic and
> > > > ProFTPD ?
Looking at the ProFTPd docs<http://www.proftpd.org/docs/directives/linked/config_ref_LogFormat.html>and your logformat below, I don't think you need %l in there — nobody supports ident anymore. If you want to know what they downloaded you may find %F (or %f) more useful than %r. Adding %T might also be useful if you want to know how long downloads are taking.
The first step is getting the information in your log files that you want to be there.
I'm guessing that this doesn't produce W3C Extended Log files<http://www.w3.org/TR/WD-logfile.html>(that are self-documenting), so you'll have to provide a LOGFORMAT command to Analog. I don't know how ProFTPd writes out time or request details, but if you're having trouble getting Analog to parse the file, feel free to post a line and I can give it a shot.
Also, note that the analog-help list<http://www.analog.cx/docs/mailing.html>may have more people with experience with ProFTPd, if you want to try the request over there until you get something that works in Analog.
Once Analog is working, it sounds like you have Report Magic mostly figured out.
On Mon, Sep 27, 2010 at 7:24 AM, cultavix <eaton.ja...@gmail.com> wrote: > Hi Jeremy,
> Thank you once again for your quick reply. There is not much help out > there and if there is, I cant find it!! :P
> Anyhow, me and my work-mate were working on this today and we cannot > seem to get the correct output from analog that we need for > reportmagic to work. We simply want the name of the usernames(local > userid), his IP address/hostname, the date/time, the files that were > dl/upl and a few other things. We can only get it working by simply > using the defaults, which do not include the local userid. So if we > could somehow add on to the default configuration the userid at least, > that would be good enough for now.
> I can supply you with my conf file if you would like, here is a sneak > peek; > This is from our analog.cfg; > #LOGFORMAT (%j %M %d %h:%n:%j %Y %t %S %b %r %j %j %j %j %u %j %j %j) > LOGFILE /var/log/proftpd/extendedlog.log
> This is from our proftpd.conf; > LogFormat test-string "%h %l %u %t \"%r\" %s %b" > ExtendedLog /var/log/proftpd/extendedlog.log ALL test-string
> Any help is apreciated...
> Thank you very much!!!!
> - James (and jacob)
> On Sep 23, 5:36 pm, Jeremy Wadsack <jeremy.wads...@gmail.com> wrote: > > James -
> > I really don't know much about ProFTPD logs. A quick look at the > > docs<http://www.proftpd.org/docs/howto/Logging.html>, > > though, suggests that the ExtendedLog is what you want to use. It looks > like > > you could configure it to have all the fields you want and then can > analyze > > that.
> > I don't think you'll need to create both logs, just one or the other, and > > Extended sounds right. If you do have both logs and they don't all have > the > > information you want, Analog can sometimes read both sets of logs and > still > > build reports. You just add a LOGFORMAT command for each file *before* > the > > LOGFILE command that loads it.
> > -- > > Jeremy Wadsack
> > On Thu, Sep 23, 2010 at 2:08 AM, cultavix <eaton.ja...@gmail.com> wrote: > > > Thank you for your quick reply, I am very thankful. I will try these > > > things out and see how it goes.
> > > As of now, I can get a near perfect report of xferlog from ProFTPD but > > > the problem is, that log only collects information of the transfers, > > > not the users....
> > > I think that the only log to contain the users and everything else I > > > want is the "Extendedlog" that Proftpd can generate.
> > > Which log should I be analyzing?
> > > Cheers!
> > > James
> > > On Sep 22, 5:21 pm, Jeremy Wadsack <jeremy.wads...@gmail.com> wrote: > > > > You can get the users who logged in, out of your basic report by > > > including > > > > the User report (USERS ON in analog.cfg).
> > > > You can tell what files everyone is downloading using the Request > report > > > > (REQUEST ON in analog.cfg), but that should be on by default.
> > > > Unfortunately, Analog does not do multivariate (or cross-reference) > > > reports > > > > directly <http://www.analog.cx/docs/faq.html#faq128>, so you can't > get > > > this > > > > out-of-the box without some extra work.
> > > > Once you have that you can find out what each user has downloaded by > > > running > > > > a new report for that user (USERINCLUDE xxxx in analog.cfg). See the > > > include > > > > command <http://www.analog.cx/docs/include.html> for details. It > > > probably > > > > makes sense to only run this for your top N users.
> > > > You can't precisely tell how long someone was on the site under > > > > HTTP<http://www.analog.cx/docs/webworks.html>(which is what Analog > was > > > > designed for) so it doesn't do time-on-site > > > > reports. However, you could use the Processing Time report (PROCTIME > ON > > > in > > > > analog.cfg) to tell how long the files take to download, in a > historgram. > > > Of > > > > course, you could also use the USERINCLUDE method to get this for > > > individual > > > > users if you wish.
> > > > -- > > > > Jeremy Wadsack
> > > > On Wed, Sep 22, 2010 at 2:33 AM, cultavix <eaton.ja...@gmail.com> > wrote: > > > > > Hi all,
> > > > > I have got ReportMagic running perfectly via Analog and the ProFTPD > > > > > logs. The problem is this;
> > > > > I want to be able to view what users are logging in, what they are > > > > > downloading and for how long. I believe this all relates to the log > > > > > file format. Does anyone have any experience with ReportMagic and > > > > > ProFTPD ?
