Bug#1042535: nfdump doesn't work with profiles using nfsen 1.3.9
Marcelo Gondim
Version: 1.7.1-2+deb12u1
Severity: normal
X-Debbugs-Cc: gon...@gmail.com
Dear,
nfdump author Peter Haag released nfsen version 1.3.9
(https://github.com/phaag/nfsen) which has support for nfdump 1.7.x. Many
sysadmins working with Networks use nfsen to generate graphs with the data
collected by nfdump. The nfdump package, which was released with Debian
Bookworm, has a bug that does not work when creating new profiles in the nfsen
application. The author has fixed this issue in his latest nfdump update at
https://github.com/phaag/nfdump on the master branch.
-- System Information:
Debian Release: 12.1
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 6.1.0-10-amd64 (SMP w/12 CPU threads; PREEMPT)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=pt_BR.UTF-8, LC_CTYPE=pt_BR.UTF-8 (charmap=UTF-8), LANGUAGE=pt_BR:pt:en
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages nfdump depends on:
ii init-system-helpers 1.65.2
ii libbz2-1.0 1.0.8-5+b1
ii libc6 2.36-9+deb12u1
ii libpcap0.8 1.10.3-1
ii librrd8 1.7.2-4+b8
ii sysvinit-utils [lsb-base] 3.06-4
nfdump recommends no packages.
nfdump suggests no packages.
-- no debconf information
Marcelo Gondim
bug does not occur.
Bernhard Schmidt
> Just to complement, using the recent version of nfdump from github, the
> bug does not occur.
fixes this issue? Is the fix in 1.7.2 or post-1.7.2 (only in git master)?
I can update unstable to 1.7.2 or even the latest git snapshot, but I
need a specific commit to backport the fix to bullseye.
Bernhard
Marcelo Gondim
Hi Bernhard,
First thanks for helping to fix this issue. Even though I'm not a
developer, I hope I'm helping to improve our favorite system even
more.
Sorry. As I'm not a developer, I couldn't tell you exactly which commit that fixes. When I spoke to Peter about the problem here https://github.com/phaag/nfsen/issues/19, the solution he passed was to use the latest version of nfdump's git master and really when I used that version, the bug didn't occurred. I'll ask him what exactly was the commit that solved the problem and report it here or is it not necessary?
Gondim
Marcelo Gondim
> Hi Marcelo,
>
>
> > Just to complement, using the recent version of nfdump from github, the
> > bug does not occur.
>
> Thanks for the report. Do you by any chance know which exact commit
> fixes this issue? Is the fix in 1.7.2 or post-1.7.2 (only in git master)?
>
> I can update unstable to 1.7.2 or even the latest git snapshot, but I
> need a specific commit to backport the fix to bullseye.
>
> Bernhard
>
>
Hi Bernhard,
I asked Peter which commit solved the problem and I'm waiting for a response from him. While waiting for his response, I looked at the 1.7.2 release commits at https://github.com/phaag/nfdump/compare/v1.7.2...master and saw this line:
Update nfprofile:
phaag committed on May 5
https://github.com/phaag/nfdump/commit/18a34c16b6d070323d3d44cb22af48a85ac9b0c5
But I can't tell you exactly if it was this commit that solved
the problem.
Bernhard Schmidt
Hi Marcelo,
I asked Peter which commit solved the problem and I'm waiting for a response from him. While waiting for his response, I looked at the 1.7.2 release commits at https://github.com/phaag/nfdump/compare/v1.7.2...master and saw this line:
Update nfprofile:
phaag committed on May 5
https://github.com/phaag/nfdump/commit/18a34c16b6d070323d3d44cb22af48a85ac9b0c5
But I can't tell you exactly if it was this commit that solved the problem.
Have you tested with the plain 1.7.2 release and it was still broken? So the commit that fixes _your_ issue was introduced after 1.7.2 has been released?
The commit you mention is quite intrusive (a lot of source cleanup mixed with the bugfix) and does not apply to 1.7.1
patching file
src/lib/nffile.c
Hunk #2 FAILED at 39.
Hunk #3 succeeded at 50 (offset -1 lines).
Hunk #4 succeeded at 180 (offset -6 lines).
Hunk #5 succeeded at 210 (offset -6 lines).
Hunk #6 succeeded at 233 (offset -6 lines).
Hunk #7 succeeded at 256 (offset -6 lines).
Hunk #8 succeeded at 285 (offset -6 lines).
Hunk #9 succeeded at 318 (offset -6 lines).
Hunk #10 FAILED at 890.
Hunk #11 succeeded at 911 (offset -4 lines).
2 out of 11 hunks FAILED
patching file src/nfdump/nfdump.c
patching file src/nfsen/nfprofile.c
Hunk #1 succeeded at 100 (offset 1 line).
Hunk #2 succeeded at 122 (offset 1 line).
Hunk #3 succeeded at 164 (offset 1 line).
Hunk #4 succeeded at 176 (offset 1 line).
Hunk #5 succeeded at 191 (offset 1 line).
Hunk #6 succeeded at 205 (offset 1 line).
Hunk #7 succeeded at 218 (offset 1 line).
Hunk #8 succeeded at 244 (offset 1 line).
Hunk #9 FAILED at 292.
Hunk #10 succeeded at 317 (offset 1 line).
1 out of 10 hunks FAILED
patching file src/nfsen/profile.c
There are so many code changes between 1.7.1 and this commit
that I would feel _very_ uncomfortable beating this specific
commit into applying.
Bernhard
Marcelo Gondim
Hi Bernhard,
I downloaded and used version 1.7.2 of nfdump (github) and the bug still remained. What I noticed, was that this bug was fixed after the release of version 1.7.2 in some commit. That's why Peter told me to use the master version. Only when I compiled and used the nfdump/master branch version did the bug disappear.
As I see it, fixing the problem involves using nfdump 1.7.2 + recent patches.
Gondim
Bernhard Schmidt
Control: forward -1 https://github.com/phaag/nfsen/issues/19
Control: found -1 1.7.1-1
On 31/07/23 08:16 AM, Marcelo Gondim wrote:
Hi,
> > The commit you mention is quite intrusive (a lot of source cleanup mixed
> > with the bugfix) and does not apply to 1.7.1
With my installation (still on Bullseye) the official backport of 1.7.1
somewhat works. A few random channels in a profile sometimes show 0 and
throw errors like
Uncompress_Block_LZO() error decompression failed in nffile.c line 342: LZO error: -6
Unable to read appendix block of file: /opt/nfsen/profiles-data/ECIX/ECIX-BER-in/2023/08/18/nfcapd.202308181520
Uncompress_Block_LZO() error decompression failed in nffile.c line 342: LZO error: -6
Uncompress_Block_LZO() error decompression failed in nffile.c line 342: LZO error: -6
Unable to read appendix block of file: /opt/nfsen/profiles-data/ECIX/ECIX-MUC-in/2023/08/18/nfcapd.202308181520
Uncompress_Block_LZO() error decompression failed in nffile.c line 342: LZO error: -6
Uncompress_Block_LZO() error decompression failed in nffile.c line 342: LZO error: -6
Uncompress_Block_LZO() error decompression failed in nffile.c line 342: LZO error: -6
ptr error - elementHeader > eor
when running a query against it, but it mostly works.
The patch you mentioned can be applied with just a single manually fixed
reject, it builds cleanly and the testsuite also works. But with this
patch it actually is worse, no data is ever created by nfprofile. No
error in the logs.
Plain 1.7.2 does not work either, same issue.
On 1.7.2 the patch you mentioned does not apply either, it has other
rejects. Looking at the changes src/lib/nffile.c had since 1.7.2 has
been released I would not be comfortable to do this.
The current git head appears to work fine, but that's not an option for
stable.
Looking at the commits I think it's virtually impossible to get a clean
"this minimally intrusive commit fixes the bug on top of the 1.7.1 in
stable", so I believe the only viable option would be
- upload current snapshot to unstable fixing this bug
- as soon as there is a 1.7.3 release, upload that and provide a
bookworm-backport for people using nfsen
Bernhard
Marcelo Gondim
version 1.7.3 and make it available in the Bookworm backport.
>
> Bernhard
Marcelo Gondim
Version 1.7.3 of nfdump has been released. Can we release this version
in backports for Bookworm?
https://github.com/phaag/nfdump/releases/tag/v1.7.3
Best regards
Em 18/08/2023 19:19, Bernhard Schmidt escreveu: