Maven plugin does not detect version numbers #Heisenbug

[Vít Šesták]

Hello,
I am trying to integrate the OWASP Dependency Check to a build process. I wanted to use Maven plugin rather than the CLI tool, as the Maven plugin has access to more information, so I hoped it to be more accurate. Well, there were some cases when the Maven plugin actually found less information than the CLI tool, so I wanted to debug it.

One more reason why I prefer the Maven plugin is that it is more debugger friendly. Debugging the CLI tool slows it down very much for some unknown reason.

However, when I started debugging the Maven plugin, the ODC tool started behaving better. I have checked out v1.2.11 from the Git.

• When I use the CLI tool, everything is OK.
• When running Maven plugin without debugger, there are missing version numbers in Evidence in the HTML report.
• When I use a debugger with no breakpoints, noting changes.
• When I add a breakpoint to JarAnalyzer:589 or to CentralAnalyzer:190 (or to both of them), the issue with missing version numbers disappears. This seems to apply for both conditional and unconditional breakpoints. I had a breakpoint dependency.fileName.indexOf("some-name…") != -1 in order to be able to focus on just one dependency. If there is the condition, the bug disappears just for the dependencies affected by the condition. If there is no condition, the bug disappears for all the dependencies.

Well, the first thing to blame there might be the parallelism. But there is virtually no parallelism. Updates are disabled in this case. (They are not needed when debugging and they make the process longer…) And there seems to be little-to-no parallelism outside the update mechanism. When I check what is happening at the breakpoint (note that the checkpoints are configured to pause the whole JVM, not just one thread), I see some JVM internal threads, the main thread and one or two H2 threads (in case of using H2 DB).

I had a suspicion about one H2 DB being opened twice. But switching to MySQL did not change anything (except the performance). Well, I had one DB thread (i.e. “Abandoned connection cleanup thread”) instead of two H2 threads.

While I know that no other interacting threads being present at the time of breakpoint do not imply there is no parallelism-related issue, I think that a parallelism-related issue is improbable there, as grepping the code for “concurrent” and some similar terms did not find anything interesting.

Do you have any idea of this issue?

Regards,

Vít Šesták 'v6ak'

[Jeremy Long]

Vit,

Unfortunately, I have no idea what is going on. Can you tell me what exactly what evidence is missing? Possible generate a log file for both executions (one normal and one in the debugger)?

These types of bugs were my favorite - heisenberg was here...

Jeremy

[Vít Šesták]

Hi Jeremy,
I tried to create a very simple project (a minimal reproduction case). It seems to be reproducible on any project with at least one dependency.

My experiment was:
1. Configure breakpoints as described above.
2. mvn      org.owasp:dependency-check-maven:check -Dautoupdate=false -X > out.log 2> err.log (and copy the output files somewhere)
3. mvnDebug org.owasp:dependency-check-maven:check -Dautoupdate=false -X > out.log 2> err.log (and copy the output files somewhere)
4. compare results

Results:

• err.log is completely uninteresting. Moreover, they differ only in timestamps
• out.log is more complex, but after replacing all uninteresting info (like timestamps, memory usage and so on) by „…“, it shows no difference. Moreover, there is some potentially sensitive info related to my global config.
• those reports differ in evidence provided. Following items are present only with debugger:
• file: name = 18.0
• Manifest: Bundle-Version = 18.0.0ur.
• pom: version = 18.0
  

I am attaching the example project with the test results. I am not including the logs.

By the way, with an improper H2 DB configuration, I've experienced some random behavior when debugger was not running. (More context: I passed a custom H2 JDBC URL without lock config, just jdbc:file:some-path. I haven't tested it with debugged.) It sometimes succeeded and sometimes failed with a message saying that the DB is already locked. This looks like some hidden parallelism. Not sure if it is related to this issue, but it might.

Regards,
Vít Šesták 'v6ak'

[Jeremy Long]

Vit,

Thanks, I'll take a closer look over the next few days. Can you tell me which OS and version of Maven (i.e. mvn -info)?

Next, if it is just missing the version evidence the bug is likely in the CPE matching code - which uses an in memory lucene index. This is because, in 1.2.11, only the evidence that was used in library identification is displayed; this is going to change with the next release.

--Jeremy

--
You received this message because you are subscribed to the Google Groups "Dependency Check" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dependency-che...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jeremy Long]

Vit,

While I am still setting up a linux environment I did recently find a bug in the code that *might* be the root of the dread heisenbug. Can you confirm whether or not you are still seeing this behavior in the latest code base?

Thanks!

Jeremy

[Vít Šesták]

Hello Jeremy,
I downloaded the latest code (master), compiled and tried it on the example project without debugger. It seems you have actually fixed the Heisenbug. Thank yu!

BTW, my base GUI (window manager and so on) runs on Fedora 20, whose codename is “Heisenbug”. This can't be coincidence ;-)

Regards,
Vít Šesták 'v6ak'