New Features: Rewriters and Occurrence Filter

7 views
Skip to first unread message

Tor Johnson

unread,
Oct 10, 2013, 6:13:39 AM10/10/13
to megatron...@googlegroups.com
Hello List!

I have added two new features to the repo:

* Rewriters: Attribute values (parsed items) can be modified by using a rewriter.
Values can be rewritten at parsing or at export, using the property "parser.rewriters"
or "export.rewriters". Examples of usage:

- Rewrite malicious URLs from "http" to "hxxp".

- Mask IP addresses (10.20.30.40 to 10.20.x.x).

- Can simplify reg-exp parsing for tricky log file formats.

- Some Shadowserver reports contains "0xdffffffff" as hostname. This can be
rewritten to an empty string, and then HostnameDecorator will add a correct
hostname. I have not tested this, but it should work.

* Occurrence filter: Filter log entries by occurrence. Examples of usage:

- For each unique IP address keep the first 20 log entries and discard the
rest. This is very handy for log files that includes a lot of rows for a
single IP address.

- Include log entries with more than 10 occurrences of the same URL, but
include only occurrences between 10-20. Occurrence filter supports a list
of intervals.

I hope you find the new features useful. List of not yet released features:

https://github.com/cert-se/megatron-java/blob/master/doc/release-notes.txt

/Tor

Reply all
Reply to author
Forward
0 new messages