How to reference to CVE modified and base offline on local directory for CLI?

[melv...@csit.gov.sg]

Tried changing within the core jar and amending to C:\ but still gives the error of "SEVERE: No documents exist" when I run the CLI.

This email is intended only for the named addressee(s) and may contain confidential and/or privileged information. If you are not the named addressee (or have received this e-mail in error), please notify the sender immediately. The unauthorised use, disclosure, distribution or copying of the contents in this e-mail is prohibited.

Thank you

[Jeremy Long]

If I am understanding your question correctly - you want to know how to reference a local copy of the NVD/CVE data files for use with dependency-check. I have two suggestions depending on your situation.

1) If you just want to use an offline copy of the database because you the machine that needs to perform the scan can't be on the Internet do the following: Put dependency-check on a machine that can access the Internet (you may need to configure a proxy via the command line options) and run a scan. Then copy the data directory to the internal system and when you run a scan using the internal system use the "no update" option.

2) If you have a process that is mirroring the NVD CVE data locally you should be able to create a properties file with the following entries (of course changing the HTTPS to the appropriate local FILE:/// protocol):

# the path to the modified nvd cve xml file.
cve.url-1.2.modified=https://nvd.nist.gov/download/nvdcve-Modified.xml.gz
cve.url-2.0.modified=https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-Modified.xml.gz
cve.url-1.2.base=https://nvd.nist.gov/download/nvdcve-%d.xml.gz
cve.url-2.0.base=https://nvd.nist.gov/feeds/xml/cve/nvdcve-2.0-%d.xml.gz

Note, the %d in the above properties will be replaced by 2002-2015. You can then load the custom properties file by specifying the `--propertyfile <file>` command line argument (and I just noticed that this is missing from the online documentation - I'll fix that shortly).

Does that help? or did I misunderstand the question?

--Jeremy

--
You received this message because you are subscribed to the Google Groups "Dependency Check" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dependency-che...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jeremy Long]

This thread continued off list. The resolution was that dependency-check local database needed to be deleted and re-created using the local NVD files. After this, using the properties file to process the local copy of the NVD CVE data worked.

--Jeremy

[Jeremy Long]

I might be somewhat confused by your question. Are you trying to use dependency-check? Or are you trying to figure out how to download and parse the XML files from the NVD to put the NVD data in your own database?

If you are just trying to use the data you could utilize the command line tool in update only mode (see the configuration page). If you wanted to put the data into a different database (other then the embedded H2) take a look at the centralized database information.

Another option - if you are just trying to get at the NVD data would be to take a look at vFeed.

--Jeremy

On Thu, May 26, 2016 at 6:57 AM, Avi Chitranshu <avi.chi...@gmail.com> wrote:

Hello Sir,

I am a php developer but this time i'm very frustrate with NVD data to download and use.
Please give me some ideas or steps -:
how to download?
how to use and store in our database?

[MM]

Hi Jeremy,

I am having issues with running this offline and setting the settings as well.

import org.owasp.dependencycheck.utils.Settings;

Settings.initialize();

Settings.setString(Settings.KEYS.CONNECTION_TIMEOUT, "10000");

Settings.setBoolean(Settings.KEYS.ANALYZER_CENTRAL_ENABLED, false);

Settings.setString(Settings.KEYS.CVE_MODIFIED_12_URL, "file:///C:/Users/Zach/Desktop/Temp/nvdcve-Modified.xml.gz");

Settings.setString(Settings.KEYS.CVE_MODIFIED_20_URL, "file:///C:/Users/Zach/Desktop/Temp/nvdcve-2.0-Modified.xml.gz");

Settings.setString(Settings.KEYS.CVE_SCHEMA_1_2, "file:///C:/Users/Zach/Desktop/Temp/nvdcve-%d.xml.gz");

Settings.setString(Settings.KEYS.CVE_SCHEMA_2_0, "file:///C:/Users/Zach/Desktop/Temp/nvdcve-2.0-%d.xml.gz");

Settings.setString(Settings.KEYS.AUTO_UPDATE, "false");

Settings.setBoolean(Settings.KEYS.AUTO_UPDATE, false);

Engine engine = new Engine();
.......

Even though i set auto update to false (using both string and boolean methods), the value is still true:

[main] DEBUG o.o.dependencycheck.utils.Settings - Properties loaded:
autoupdate='true'

I have set the cve.url-1.2.modified, cve.url-2.0.modified, cve.url-1.2.base, cve.url-2.0.base using file:/// protocol.

However, I face many errors after that:

12:17:44.731 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer Archive Analyzer

12:17:44.731 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer File Name Analyzer

12:17:44.731 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer Jar Analyzer

12:17:44.731 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer Hint Analyzer

12:17:44.731 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer CPE Analyzer

12:17:44.731 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer False Positive Analyzer

12:17:44.731 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer Cpe Suppression Analyzer

12:17:44.747 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer Dependency Bundling Analyzer

12:17:44.747 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer NVD CVE Analyzer

12:17:44.747 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer Vulnerability Suppression Analyzer

12:17:44.747 [main] INFO  o.o.d.analyzer.CentralAnalyzer - Central analyzer disabled

12:17:44.747 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer Central Analyzer

12:17:44.747 [main] DEBUG o.o.d.analyzer.NexusAnalyzer - Nexus analyzer disabled, using Central instead

12:17:44.747 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer Nexus Analyzer

12:17:44.747 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer Nuspec Analyzer

12:17:44.747 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer Assembly Analyzer

12:17:44.747 [main] DEBUG o.o.d.analyzer.AnalyzerService - Loaded Analyzer OpenSSL Source Analyzer

12:17:45.263 [main] DEBUG o.o.d.data.nvdcve.CveDB - Database dialect: H2

12:17:45.303 [main] ERROR org.owasp.dependencycheck.Engine - No documents exist

Unable to continue dependency-check analysis.

12:17:45.303 [main] DEBUG org.owasp.dependencycheck.Engine - 

Multiple Exceptions Occurred

org.owasp.dependencycheck.exception.NoDataException: No documents exist

org.owasp.dependencycheck.exception.ExceptionCollection: Unable to continue dependency-check analysis.

at org.owasp.dependencycheck.Engine.ensureDataExists(Engine.java:751) ~[AndroidScanner-0.1-all.jar:na]

No documents exist

at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:500) ~[AndroidScanner-0.1-all.jar:na]

at org.owasp.dependencycheck.Engine.throwFatalExceptionCollection(Engine.java:773)

at com.temp.AndroidScanner.checkers.DependencyChecker.check(DependencyChecker.java:54) [AndroidScanner-0.1-all.jar:na]

at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:502)

at com.temp.AndroidScanner.AndroidScanner.check(AndroidScanner.java:78) [AndroidScanner-0.1-all.jar:na]

at com.temp.AndroidScanner.checkers.DependencyChecker.check(DependencyChecker.java:54)

at com.temp.AndroidScanner.AndroidScanner.main(AndroidScanner.java:170) [AndroidScanner-0.1-all.jar:na]

at com.temp.AndroidScanner.AndroidScanner.check(AndroidScanner.java:78)

12:17:45.303 [main] INFO  c.m.A.c.MisconfigurationChecker - MisconfigurationChecker started

at com.temp.AndroidScanner.AndroidScanner.main(AndroidScanner.java:170)

Next Exception:

org.owasp.dependencycheck.exception.NoDataException: No documents exist

at org.owasp.dependencycheck.Engine.ensureDataExists(Engine.java:751)

at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:500)

at com.temp.AndroidScanner.checkers.DependencyChecker.check(DependencyChecker.java:54)

at com.temp.AndroidScanner.AndroidScanner.check(AndroidScanner.java:78)

at com.temp.AndroidScanner.AndroidScanner.main(AndroidScanner.java:170)

How do I resolve this?

Hope to hear from you soon.

[MM]

Just an update, I cleaned the project and rebuilt it. Ran the program with internet (and commented out all my local cve urls), first run was OK, database was built at "data\dc.h2.db"

Then I ran program again without internet and uncommenting back my local cve urls, everything was OK too.

Now I set a cron job to mirror the NVD db from NIST weekly, and set the cve.check.validforhours='168'.

Should I set autoupdate='false' so it doesn't require internet when running? However I want the "data\dc.h2.db" to be automatically updated based on the latest files mirrored from NIST (location speficied in cve.url for both 1.2 and 2.0).

I want to run this offline mode. Please advise, thanks!! :)

[Jeremy Long]

When you configure ODC to use different locations of the NVD XML files you need to have autoupdate enabled; otherwise it will ignore the file locations and won't process the results and as such you will get a 'no documents exists' error.

As to the logging of the configuration values - there is a bug in the code and it is logging the base values not the updates based off any individual plugin/cli configuration provided. I'll fix this bug in the next release.

--Jeremy

To unsubscribe from this group and stop receiving emails from it, send an email to dependency-check+unsubscribe@googlegroups.com.