Two Maven projects sharing same parent pom get different results connecting to CVE DB

[corey....@cerner.com]

We have a local copy of nist data and I have one maven project, lets call it A, that gets these errors when trying to update (because it's been more than 4 hours) 

[INFO] Central analyzer disabled

[INFO] Checking for updates

[INFO] starting getUpdatesNeeded() ...

[ERROR] IO Exception: HEAD request returned a non-200 status code

[ERROR] IO Exception: HEAD request returned a non-200 status code

[ERROR] IO Exception: HEAD request returned a non-200 status code

[ERROR] IO Exception: HEAD request returned a non-200 status code

[ERROR] IO Exception: HEAD request returned a non-200 status code

[ERROR] IO Exception: HEAD request returned a non-200 status code

[ERROR] IO Exception: HEAD request returned a non-200 status code

[ERROR] IO Exception: HEAD request returned a non-200 status code

[ERROR] IO Exception: HEAD request returned a non-200 status code

[ERROR] IO Exception: HEAD request returned a non-200 status code

[ERROR] IO Exception: HEAD request returned a non-200 status code

[ERROR] IO Exception: HEAD request returned a non-200 status code

[WARNING] Unable to download the NVD CVE data; the results may not include the m

ost recent CPE/CVEs from the NVD.

[INFO] If you are behind a proxy you may need to configure dependency-check to u

se the proxy.

[WARNING] Unable to update Cached Web DataSource, using local data instead. Resu

lts may not include recent vulnerabilities.

[ERROR] No documents exist

Unable to continue dependency-check analysis.

If I run maven project B, which shares the same configurations for dependency check maven plugin it gets this:

[INFO] Central analyzer disabled

[INFO] Checking for updates

[INFO] starting getUpdatesNeeded() ...

[INFO] NVD CVE requires several updates; this could take a couple of minutes.

[INFO] Download Started for NVD CVE - 2003

[INFO] Download Started for NVD CVE - 2004

[INFO] Download Started for NVD CVE - 2002

[INFO] Download Started for NVD CVE - 2005

[INFO] Download Started for NVD CVE - 2006

[INFO] Download Started for NVD CVE - 2007

[INFO] Download Started for NVD CVE - 2008

[INFO] Download Started for NVD CVE - 2009

[INFO] Download Started for NVD CVE - 2010

[INFO] Download Started for NVD CVE - 2011

[INFO] Download Started for NVD CVE - 2012

[INFO] Download Started for NVD CVE - 2013

[INFO] Download Complete for NVD CVE - 2003  (1790 ms)

[INFO] Download Started for NVD CVE - 2014

[INFO] Processing Started for NVD CVE - 2003

[INFO] Download Complete for NVD CVE - 2004  (2645 ms)

[INFO] Download Started for NVD CVE - 2015

[INFO] Processing Started for NVD CVE - 2004

[INFO] Processing Complete for NVD CVE - 2003  (1022 ms)

[INFO] Processing Complete for NVD CVE - 2004  (968 ms)

[INFO] Download Complete for NVD CVE - 2009  (3751 ms)

[INFO] Download Started for NVD CVE - 2016

[INFO] Processing Started for NVD CVE - 2009

[INFO] Download Complete for NVD CVE - 2005  (3884 ms)

[INFO] Download Started for NVD CVE - 2017

[INFO] Processing Started for NVD CVE - 2005

[INFO] Download Complete for NVD CVE - 2010  (4539 ms)

[INFO] Download Started for NVD CVE - 2018

[INFO] Processing Started for NVD CVE - 2010

[INFO] Download Complete for NVD CVE - 2006  (4567 ms)

[INFO] Download Started for NVD CVE - Modified

[INFO] Processing Started for NVD CVE - 2006

[INFO] Download Complete for NVD CVE - 2018  (1229 ms)

[INFO] Processing Started for NVD CVE - 2018

[INFO] Download Complete for NVD CVE - 2013  (6117 ms)

[INFO] Processing Started for NVD CVE - 2013

[INFO] Download Complete for NVD CVE - 2008  (6327 ms)

[INFO] Processing Started for NVD CVE - 2008

[INFO] Download Complete for NVD CVE - Modified  (1948 ms)

[INFO] Processing Started for NVD CVE - Modified

[INFO] Download Complete for NVD CVE - 2002  (6548 ms)

[INFO] Download Complete for NVD CVE - 2007  (6554 ms)

[INFO] Download Complete for NVD CVE - 2014  (5011 ms)

[INFO] Download Complete for NVD CVE - 2012  (7644 ms)

[INFO] Processing Complete for NVD CVE - 2018  (2011 ms)

[INFO] Processing Started for NVD CVE - 2002

[INFO] Download Complete for NVD CVE - 2016  (4122 ms)

[INFO] Download Complete for NVD CVE - 2011  (8601 ms)

[INFO] Processing Complete for NVD CVE - Modified  (2372 ms)

[INFO] Processing Started for NVD CVE - 2007

[INFO] Download Complete for NVD CVE - 2015  (8592 ms)

[INFO] Download Complete for NVD CVE - 2017  (8969 ms)

[INFO] Processing Complete for NVD CVE - 2005  (15134 ms)

[INFO] Processing Started for NVD CVE - 2014

[INFO] Processing Complete for NVD CVE - 2009  (17693 ms)

[INFO] Processing Started for NVD CVE - 2012

[INFO] Processing Complete for NVD CVE - 2010  (23907 ms)

[INFO] Processing Started for NVD CVE - 2016

[INFO] Processing Complete for NVD CVE - 2013  (29263 ms)

[INFO] Processing Started for NVD CVE - 2011

[INFO] Processing Complete for NVD CVE - 2002  (29567 ms)

[INFO] Processing Started for NVD CVE - 2015

[INFO] Processing Complete for NVD CVE - 2006  (37738 ms)

[INFO] Processing Started for NVD CVE - 2017

[INFO] Processing Complete for NVD CVE - 2007  (43090 ms)

[INFO] Processing Complete for NVD CVE - 2008  (48709 ms)

[INFO] Processing Complete for NVD CVE - 2012  (44720 ms)

[INFO] Processing Complete for NVD CVE - 2011  (51687 ms)

[INFO] Processing Complete for NVD CVE - 2014  (69070 ms)

[INFO] Processing Complete for NVD CVE - 2016  (64276 ms)

[INFO] Processing Complete for NVD CVE - 2015  (56419 ms)

[INFO] Processing Complete for NVD CVE - 2017  (63842 ms)

[INFO] Begin database maintenance.

[INFO] End database maintenance.

[INFO] Check for updates complete (108050 ms)

[INFO] Analysis Started

[INFO] Finished File Name Analyzer (0 seconds)

[INFO] Finished Dependency Merging Analyzer (0 seconds)

[INFO] Finished Version Filter Analyzer (0 seconds)

[INFO] Finished Hint Analyzer (0 seconds)

[INFO] Created CPE Index (1 seconds)

[INFO] Skipping CPE Analysis for npm

[INFO] Finished CPE Analyzer (1 seconds)

[INFO] Finished False Positive Analyzer (0 seconds)

[INFO] Finished Cpe Suppression Analyzer (0 seconds)

[INFO] Finished NVD CVE Analyzer (0 seconds)

[INFO] Finished Vulnerability Suppression Analyzer (0 seconds)

[INFO] Finished Dependency Bundling Analyzer (0 seconds)

[INFO] Analysis Complete (2 seconds)

Then of course if I run maven project A again it success because project B updated the nist mirror data.  

How can two projects that share the same settings run on the same machine behave differently?

Thanks,

Corey

[Hans Aikema]

Corey,

forgot to include the DependencyCheck maillist, and missed an additional note:

if by 'local copy of the NIST data' you mean a local mirror, than where my mail says 'NIST webserver' you should read 'the webserver of your local mirror'

regards, Hans

On 2 Mar 2018, at 01:46, Hans Aikema wrote:

Corey,

The issue has nothing to do with the project being scanned, but all to do with errors in the internet connectivity towards the NIST website at the time dependency-check ran its up-to-date check. signalled by the logs as "HEAD request returned a non-200 status code"

Which translates to 'some web error occured while checking the up-to-dateness of the most recently downloaded NIST NVD data' when project A ran it's up-to-date check. By the time project B ran the web errors resolved themselves so DC was able to check the up-to-dateness and downloaded the updated NVD data (which from the logs appears to be all NVD datastreams, so I would guess that there was even a full DC database reset in between the two project runs (or an update at the side of NIST which updated the timestamp of all NVD datastreams from 2002 up to 2018).

The cause of the issue can range from issues at a proxy server all the way to some temporary errors due to overloads or maintenance at the NIST website.

200 is the status (return code) that a website returns with its response when a web request is processed OK and the resulting data is sent back to the requestor (to the dependency-check software in this case). 

Other frequently observed status codes include 404 (when information for the URL is not found), 401 (to sgnal that you need to login), 403 (to signal that you are not allowed to access the URL), 500 (when something went wrong internally in the webserver while it tried to create the responsemessage for the URL). 

A full list can be found at http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml Of all these codes only the case of status code '200' means that the NIST website was able to send the appropriate results back to the DC software, hence the error message of Dependency-Check only mentions 'non-200 status code'.

kind regards,

Hans Aikema

--
You received this message because you are subscribed to the Google Groups "Dependency Check" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dependency-che...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[corey....@cerner.com]

Thank you for your response Hans.  That makes sense that it wouldn't be a project specific error.  The strange thing is that it is consistent.  Project A always gets the error if it's been 4 hours since the last update.  And Project B never gets the error no matter when it's run.  I'll keep digging.  

Thanks,

Corey