pflog

[James J. Lippard]

Has anybody built any tools for monitoring pf logs in realtime?
I used to use something like swatch with ipflogs, but the tcpdump
binary format is not easily conducive to such monitoring.

--
Jim Lippard lip...@discord.org http://www.discord.org/
GPG Key ID: 0xF8D42CFE

[Alexandre Dulaunoy]

Hello,

I have the same issue (for using with IPFC) and I have tried multiple
solution :

- a modified pflogd with a dump in ascii format instead of the libpcap.
(should we ask to the author ?)
- a modified tcpdump -e that reads all and export it to "parser friendly"
format (there is some issue you have to take care like the rotation of
pcap capture files...) (The standard tcpdump output is good for human not
so good for parser 8-))

I think the better way is to get a pflogd supporting a standard ascii log
format. (is there somebody working on that ? Daniel ?)

Alx

On Fri, 21 Dec 2001, James J. Lippard wrote:

> Has anybody built any tools for monitoring pf logs in realtime?
> I used to use something like swatch with ipflogs, but the tcpdump
> binary format is not easily conducive to such monitoring.
>
>

--
Alexandre Dulaunoy adu...@conostix.com
http://www.conostix.com/

[James J. Lippard]

Thanks for the FAQ reference and the suggestion to use

tcpdump -ttt -e -i pflog0

but those don't answer the question I asked.

I suppose I could start up a

tcpdump -ttt -e -i pflog0 >ascii-log

and run swatch on that, but that seems pretty inefficient, as well
as risks the potential security problems with tcpdump. What would
be ideal would be an ascii logfile option to pflogd.

On Sat, Dec 22, 2001 at 11:27:05AM -0600, Jon Trembley wrote:
> Have you read the FAQ? http://www.openbsd.org/faq/faq6.html#6.2
> It talks about monitoring the pflog files.
>
> --
> Jon

> --
> Unix is very simple, but it takes a genius to understand the simplicity.
> (Dennis Ritchie)

[Theo de Raadt]

> - a modified pflogd with a dump in ascii format instead of the libpcap.
> (should we ask to the author ?)

It started out that way. I rejected it. The less code is in the
critical path, the better. There have been passive remote holes in
tcpdump. Quite a few, in fact.

> - a modified tcpdump -e that reads all and export it to "parser friendly"
> format (there is some issue you have to take care like the rotation of
> pcap capture files...) (The standard tcpdump output is good for human not
> so good for parser 8-))

Well, I am not sure if we yet another new file format.

> I think the better way is to get a pflogd supporting a standard ascii log
> format. (is there somebody working on that ? Daniel ?)

It does support a standard ascii log format. tcpdump output. Run it
offline, where it is less likely to suffer from a passive attack. I have
some ideas for how to do this in another way, but it is a lot of work.

[Alexandre Dulaunoy]

> > I think the better way is to get a pflogd supporting a standard ascii log
> > format. (is there somebody working on that ? Daniel ?)
>
> It does support a standard ascii log format. tcpdump output. Run it
> offline, where it is less likely to suffer from a passive attack. I have
> some ideas for how to do this in another way, but it is a lot of work.
>

Yes, you are right but for continuous monitoring it's a long stream
and "administrator error compliant(tm)".

If I resume, tell me if I'm wrong :

kernel part :
pf.c -> log into device (/dev/pflog*) in binary (if logging active)

from pflogd :
reading /dev/pflog0
pcap_dispatch (the /dev/pflog0 to a pcap files)
the pcap files is reopened on regular basis. (by sending a HUP to pflogd)

another daemon running for making text logging possible:
reading the pcap files using a method like "tcpdump -e -tt"
and output into text.

Why not to implement a possibility to log into ascii format in pflogd ?
Yes, there is some issue about speed (because we need to do some parsing
like tcpdump does). But the advantage is big, the tcpdump like daemon is
not running anymore. (I'm doing that for the moment)

a logging format like this :

action="rule 0/1(match)",direction="pass out on wi0",srcip=192.168.1.1,srcport=3222,dstip=192.168.2.1,dstport=22
....

Is it possible or there is some issue coming from that ?

Thanks a lot.

alx

[Theo de Raadt]

> Yes, you are right but for continuous monitoring it's a long stream
> and "administrator error compliant(tm)".

Damn it, I wish you had read what I said before replying. You realize how
annoying it is when you don't even read what I say?

> If I resume, tell me if I'm wrong :
>
> kernel part :
> pf.c -> log into device (/dev/pflog*) in binary (if logging active)
>
> from pflogd :
> reading /dev/pflog0
> pcap_dispatch (the /dev/pflog0 to a pcap files)
> the pcap files is reopened on regular basis. (by sending a HUP to pflogd)
>
> another daemon running for making text logging possible:
> reading the pcap files using a method like "tcpdump -e -tt"
> and output into text.
>
> Why not to implement a possibility to log into ascii format in pflogd ?

Because it makes code running as root more complicated.

> Yes, there is some issue about speed (because we need to do some parsing
> like tcpdump does).

No.

You obviously did not bother to read what I said.

> But the advantage is big, the tcpdump like daemon is
> not running anymore. (I'm doing that for the moment)

Then you are crazy. We do not recommend that you do so. You are placing
yourself at risk.

> a logging format like this :
>
> action="rule 0/1(match)",direction="pass out on wi0",srcip=192.168.1.1,srcport=3222,dstip=192.168.2.1,dstport=22
> ....
>
> Is it possible or there is some issue coming from that ?

I really don't see the point of yet another format.

[Peter Galbavy]

Er, what's wrong with 'tcpdump -nvi pflog0' ?

Peter

[Alexandre Dulaunoy]

On Mon, 24 Dec 2001, Theo de Raadt wrote:

> > But the advantage is big, the tcpdump like daemon is
> > not running anymore. (I'm doing that for the moment)
>
> Then you are crazy. We do not recommend that you do so. You are placing
> yourself at risk.

Yes, you are right. That's why I'm looking for a solution for dumping
directly in ascii format without using tcpdump 8-)

Thanks.

Alx

[Darren Reed]

Don't worry, I'll get down and do some work on ipfilter & openbsd 3.0 RSN
and then you won't have to use pf if you're using 3.0 and you won't need
to worry about this problem any more. Plus you'll be able to use any of
the various tools that already work with the ipfilter log format. Just be
patitent and have a little bit of faith.

Merry Christmas,
Darren

[Alfred Breull]

You have your merits. However, that attitude you've shown during
last weeks ... Please, come down to the ground again.

Merry Christmas to everyone !

[Patrick Greer]

My $.02 -

I appreciate your point about keeping the critical path as clean as
possible, and while I don't fully understand the vunlerability issues with
tcpdump, I'll take your word for it. My question is that as long as you are
considering an offline method of producing ascii logs, would you consider
making the output conformant with SANS' dshield format
(http://www.dshield.org/specs.html#dshield_format) ? Since it appears that
a great many of us are using OpenBSD in a firewalling role, the ablility to
quickly and easily exchange hostile activity reports with the rest of the
community seems it would be a Good Thing.

Thanks!

Patrick