ntopng 3.0 released but not compatible with Security Onion yet

[Kevin Branch]

Ntop has very recently released ntopng 3.0 which depends on a newer PF_RING version (6.6.0)  than what SO is tied into at this time (6.4.1).    This means our SO-specific ntopng installer is not going to work for now:

https://github.com/branchnetconsulting/so1404-ntopng-installer

I've reached out to Luca about possibly packaging ntopng 3.0 for SO which would really help keep ntopng SO-compatible in the future.  If that works out I'll make sure to test it and make an announcement here.

Eventually SO will upgrade to a newer PF_RING, but I don't see anything specific on the roadmap yet, just this:

Issue 819: soup: check to see if PF_RING updates are available

which doesn't quite sound like an upgrade to PF_RING 6.6.0.

If something better isn't worked out before then, I'll plan to update the SO custom ntopng installer to install a static version of ntopng 2.4 instead of what it sitting of the latest stable packages found at packages.ntop.org.

Kevin

[Doug Burks]

Hi Kevin,

I've created Issue 1101 for PF_RING 6.6:
https://github.com/Security-Onion-Solutions/security-onion/issues/1101

> --
> Follow Security Onion on Twitter!
> https://twitter.com/securityonion
> ---
> You received this message because you are subscribed to the Google Groups
> "security-onion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to security-onio...@googlegroups.com.
> To post to this group, send email to securit...@googlegroups.com.
> Visit this group at https://groups.google.com/group/security-onion.
> For more options, visit https://groups.google.com/d/optout.



--
Doug Burks

[Marco]

I didn't realize there was a SO-specific implementation of ntopng, that's actually quite amazing!
I was just looking up ntopng a couple days ago because I was searching for an open source solution to monitor my switches via ipfix and the best thing would obviously be to use the box already running SO, because it's really potent hardware anyways.
Thanks for bringing this to my attention!

[Kevin Branch]

Hi Luca,

Do you mean libntapi.so and libntos.so are no longer part of the PF_RING 6.6.0 deb package?  That would simplify things in the SO custom installer of ntopng.  Thanks for pointing that out.

The primary issue behind needing the custom installer is that the ntopng deb has a dependency on a very specific pfring package:

dpkg: dependency problems prevent configuration of ntopng:

 ntopng depends on pfring (= 6.6.0-1253); however:

  Package pfring is not installed.

but I can't safely install the packages.ntop.org pfring deb on SO because it conflicts with Doug's build of PF_RING 6.4.1 which is spread across a few securityonion-pfring-* packages.  Installing it breaks SO.

I know ntopng opportunistically makes use of PF_RING if present, but the ntopng deb package appears to mandate the install of the pfring deb from packages.ntop.org.  Building from tarball would be an alternative, except that as I recall, trying to build ntopng from source on a SO platform did not work out last time I explored that option.  It might be nice to publish ntopng-nopfring versions of the deb files so that environments where PF_RING is not wanted or possibly already present under a different package name, could still take advantage of the deb packages.  What do you think?

Since Doug is planning on upgrading SO to PF_RING 6.6 this month anyway, I'm happy to wait for that and then tweak my custom installer as needed.  

Regards,

Kevin

On Wed, Jun 7, 2017 at 9:28 AM, Luca Deri <de...@ntop.org> wrote:

Hi Kevin

the new PF_RING 6.6 removed all the dependencies on external libraries you have in the GIT link you sent me below

ntopng does NOT require PF_RING to operate (we support many non-Linux OSs so it’s not compulsory), but it will use it if available. If you do not plan to upgrade PF_RING to 6.6 (I advise you to do that as the new release has many improvements) the other option I see is to build ntopng without PF_RING.

What do you think?

Regards Luca

[Kevin Branch]

Hi Luca,

I agree that Security Onion catching up to the latest stable PF_RING is ideal, but it would also be nice to avoid these recurring time windows in which the latest stable ntopng deb file requires a PF_RING package newer that what is currently rolled into Security Onion.  PF_RING is the only dependency I am having any trouble with related to ntopng on SO.

With an ntopng-nopfring_3.0 deb file, would ntopng still detect and make use of the existing PF_RING 6.4.1 kernel module already in place on SO, or would it not use PF_RING at all?

Kevin

On Wed, Jun 7, 2017 at 1:41 PM, Luca Deri <de...@ntop.org> wrote:

Hi Kevin

they are not, correct. In 6.6 if you have a proprietary NIC (such as the Napatech NIC that uses the libntapi.o for instance), when you start PF_RING, in case you have such device present, we load the shills via dlopen(). This has removed all dependencies on this party binary files

The dependency is because sometimes people forced installation of custom packages and this lead to crashes sometimes. So in this way we have avoided that. 

The solution IMHO is Doug to move to 6.6, but if it can help we can consider building a different ntopng package with limited dependencies (we do for nprobe or for platform with limited resources such as ARM). As you kind of requested that (reading below), beside PF_RING what other dependency would you like me to avoid?

Finally: will you have @ https://sharkfest.wireshark.org so we can discuss all this in person?

Thanks Luca

[Kevin Branch]

I just finished adapting the SO installer for ntopng to be compatible with version ntopng-3.0.

Based on my own testing, it seems to install ntopng-3.0 just fine.  It appears ntopng-3.0 is willing to use the SO-packaged PF_RING 6.4.1 even though the original deb file for ntopng-3.0 depends on PF_RING 6.6.0 from packages.ntop.org.  I confirmed it is allocating a pcap ring and that counters for that ring really are going up.

Consider ntopng-3.0 to be only lightly tested on Security Onion.

Here it is:

https://github.com/branchnetconsulting/so1404-ntopng-installer

Please give a shout it you have trouble with it.

Kevin