NxFilter and AD Settings

[Duncan del Toro]

Well first off, I just recently installed NxFilter as a PoC.  So far I quite like it, it seems to do most everything I need so I am quite happy about that.  I was curious though, is there any way to make it so all groups are excluded except for a subset of groups, that way instead of having over 100 groups imported into nxfilter I would just have a handful of groups that I would be mapped directly to a policy?

Also for the syslog logging, is it possible to specify a port or add a static portion to the line to more easily identify the log entries as coming from an NxFilter appliance?  

Thanks!

[Jinhee]

There's 'Exclude keyword' option on AD import setup. Try 'edit' button on the list. What kind of info do you want to include in syslog exportation?

[Duncan del Toro]

I'm somewhat looking for the opposite.  Sort of an "exclude all groups except those which match".  Reasoning is, I have 100+ groups in AD and I only plan on using ~10 of those for policy control in nxfilter.  If I could name them all similar (i.e. filter-admins, filter-financial, filter-sales, etc) and just include only groups that match filter- it would be a bit cleaner than excluding ~20-30 different patterns which I need to be careful are not vague enough to actually exclude users.  Actualy even a exclude list for users and one for groups split up would make things a bit simpler there.

As for the syslog, right now I'm matching on the fact there are at least 3 | in the message.  Being able to just add "|NXFILTER|" to all syslog lines would let me match for specifically that before parsing it.  For instance (taking the line from the documentation:)

2013-01-28 10:53:23|Y|www.bbc.co.uk|pwuser|192.168.0.101|admin|news|Blocked by admin|33

to

2013-01-28 10:53:23|NXFILTER|Y|www.bbc.co.uk|pwuser|192.168.0.101|admin|news|Blocked by admin|33

I hope that makes sense.   Thanks for the response!

[Jinhee]

If we have 'keyword for inclusion' that would make it too much complicated. Many people don't understand how to use AD import even. And I think you can solve it on AD side. It's basically LDAP so you can create OU and just import the groups belong to that OU.

I think being able to differentiate the syslog message from NxFilter is required when there are multiple syslog into one logging server. But is it the only way to add 'NxFilter' on every message? I think you can use source IP address. From my old memory when I was developing ESM which is taking syslog and SNMP log from all kinds of security devices I could do that. But maybe it's because I was a developer. Do you use some standard software receiving these syslog data?

[Duncan del Toro]

Understandable about the inclusion thing.   I'll look into implementing it via having multiple directory lists then.

No, adding 'nxfilter' wouldn't be the only way.  Right now I have it done via  regular expression which searches for series of pipes.  This works just fine in my current setup as nothing else transmits syslogs with pipes.  I could do it via sourceip or hostname, but I wanted to avoid going that route as if an IP address or name changes I need to update filters. I aggregate all of my syslogs from ~500 devices at the moment into an Elasticsearch/Logstash/Kibana stack and then try to parse out the lines with grok to normalize the field names.  

[Duncan del Toro]

Hmm, not sure if its a bug or my understanding.  I deleted my AD settings, added a new AD server with a base DN farther up the tree, which only has users in it and no groups.  After importing it successfully imported the correct number of users, but imported every group listed in AD, all of which were not housed within the base DN.

As an example it added the group called #Distro which is located in companyname.int/Distribution Lists/ while the base dn was ou=users,ou=cityname,dc=companyname,dc=int. (modified of course)

User import was correct, only users in that base DN were added.  Does the Base DN only limit users?

[Jinhee]

I tested with my OU again which is 'ou=students,dc=rainbowx,dc=local'? Only the users and groups in the OU imported.

[Duncan del Toro]

Hmm yeah that is what I would expect but not what I was geting.  In the end I just made several exceptions, good enough for now and seems to be working just fine.  In my case our Server Admins have AD as below:

OU=Users,OU=<location>,DC=company,DC=int which only contains users

OU=Security Groups,OU=<location>,DC=company,DC=int contains Security groups

OU=Distribution Groups,OU=<location>,DC=company,DC=int contains Distribution groups

I set the base DN to OU=Users,OU=<location>,DC=company,DC=int and it pulled security/distribution groups from at least from the other two.  I think I'll just make the base dn DC=company,DC=int and filter out all the groups I don't want, a little more work but it shouldn't affect anything.

[Duncan del Toro]

Just noticed what it is.  It adds groups based on users membership to those groups that are in that OU even if the group is not in that base DN.  Makes sense to do that.

[Jinhee]

Yeah, we use 'memberOf' attribute of a user for importing groups. About the syslog I will think about it. Maybe we need more flexible format. Something like 'time=201502211143 product=NxFilter version=2.6.2 host=192.168.0.5 user=john'. But the current simpler one looks also good to me.

[Eric Harrison]

For syslog, I named all of the hosts nxfilter-<something> and match on that. The syntax using rsyslog to send these to port 1234 on my logstash server:

     :HOSTNAME, contains, "nxfilter" @logstash:1234


All of the messages start with a timestamp and |Y| or |N|, so if you prefer regular expressions something like should work well (untested)

      :msg, regex, ^[0-9\ -:]*|[YN]|  @logstash:1234


Jinhee, a good option would be to add a syslog tag as recommended in section 5.3 of RFC 3164 (https://tools.ietf.org/html/rfc3164#section-5.3)

That would give us something easy to match on without changing the log message content itself. For example if a syslog tag of "NXFILTER" was set, the resulting log entry would look like this:

     Feb 23 06:30:05 myserver.domain.com NXFILTER: 2015-02-23 06:30:05|N|www.google.com|Default|1.2.3.4|Default|searchengines||28

which can be matched in rsyslog.conf with:

     :syslogtag, contains, "NXFILTER" @logstash:1234



-Eric

[Duncan del Toro]

Hmm I had not though of checking for specifically a Y or N, i'll go ahead and modify based on that.  I wanted to avoid matching based on originating host name, not for any super important reason though.

I do like the real ease of use of the current syslog format using pipes, and the key value pair mentioned a bit earlier by Jinhee works well too.   The RFC 3164 syslog tag is a much better way of implementing what I was originally after though, good note on that one.

[Jinhee]

OK. I will look into syslog tag as well.

[Jinhee]

I think the simplest one would be this one,

  NXFILTER|2013-01-28 10:53:23|Y|www.bbc.co.uk|pwuser|192.168.0.101|admin|news|Blocked by admin|33

Just moved 'NXFILTER' to the front so that you can remove it and just parsing it as it was. I will change it
on v2.6.3.