About hash hunt

[Roberto]

Hello All,

I'm new here and new with Velociraptor, so sorry if I post obvious questions. I'm trying to get most out of the hash storage and search.

If I understand well the artifact Generic.Forensic.LocalHashes.Init  creates an SQLite database on the endpoint, but without any data on it.

Then, the artifact Generic.Forensic.LocalHashes.Glob  maintains a local (client side) database of file hashes based on the query specified on the HashGlob.

There is also the artifact  Generic.Forensic.LocalHashes.Query which queries and maintains the database. Here, I don't understand the difference between the parameter "Hash" and the parameter " CommaDelimitedHashes" .

What if I would like maintain a database of all hashes of all files in the computer? Or all most interesting paths? Would that be high cost for the computer? 

How can I indicate more than one path on the Generic.Forensic.LocalHashes.Glob search parameter?

I've searched for detailed info about all of this on Velocidex.com but I did not find it.

I would like to ask also why some of the artifacts are not explained on velocidex.com, is it because it's difficult to maintain?

Thank you in advance for your help.

Kind Regards,

Robert

[Mike Cohen]

Hi Roberto,

    The artifacts you describe show how one can maintain a per host hash database on the end point itself. They are an experimental set of artifacts - primarily designed to answer the question "does this machine have this hash" very quickly.

As you know, calculating the hash is an expensive operation. These artifacts are written to work together with Windows.Forensics.LocalHashes.Usn which follows the USN journal to update file hashes as the files are changing on the endpoint. So the idea is that one can simply query the sqlite database and get instant hits. The Generic.Forensic.LocalHashes.Glob artifact can be used the first time to populate files existing on the endpoint (but it takes a lot of cpu cycles to calculate a lot of hashe).

The Windows.Forensics.LocalHashes.Usn artifact is still experimental - we still have to come up with reasonable defaults.

Probably you will need to whitelist some file types to hash (the default hashes all files but it turns out that some files are changing all the time so we end up hashing them for no reason). Probably we need to restrict to only .exe or .dll files in practice.  Also maybe increase the frequency of USN scans (10 seconds is a bit short). Try to enable that artifact and see how new files are being hashed. 

Also you can just upload the sqlite file in its entirety to the server to get an offline record of all file hashes.

So to summarise - the Generic.Forensic.LocalHashes.Glob is used to initialize the sqlite database with existing hashes of existing files. Windows.Forensics.LocalHashes.Usn is used to watch the USN journal for new files into the future and Generic.Forensic.LocalHashes.Query is used to query the local sqlite database (Hashes parameter is a csv string with hashes or additionally you can specify CommaDelimitedHashes as a comma delimited set of hashes to check.

Thanks

Mike

 

Mike Cohen  Digital Paleontologist,  Velocidex Enterprises M  ‭+61 470 238 491‬  E mi...@velocidex.com

--
You received this message because you are subscribed to the Google Groups "velociraptor-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to velociraptor-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/velociraptor-discuss/e3c05e7b-bff8-41ea-be4d-ec773622a5b5n%40googlegroups.com.

[Roberto]

That's a great reply! I've managed to make it query all files but with lower frequency.

I have 2 more questions about Windows.Forensics.LocalHashes.Usn; please, let me know if I need to open another thread:

1. In order to look for some determined files like, how would we indicate the PathRegex (what format): *.exe, *.dll?

2. Can we use this artifact to query for recently modified files? Would it be possible to add the timestamp of modified file as a different column to the database?

Thanks again in advance.

Roberto

[Mike Cohen]

On Thu, Apr 15, 2021 at 1:43 AM Roberto <roberto...@gmail.com> wrote:

That's a great reply! I've managed to make it query all files but with lower frequency.

I have 2 more questions about Windows.Forensics.LocalHashes.Usn; please, let me know if I need to open another thread:

1. In order to look for some determined files like, how would we indicate the PathRegex (what format): *.exe, *.dll?

It is a regex that applies to the entire path - so something like C:\\Windows\\.+exe$

 

2. Can we use this artifact to query for recently modified files? Would it be possible to add the timestamp of modified file as a different column to the database?

You can query for recently modified files directly in the USN journal - Windows.Forensics.Usn.

The sqlite database that is maintained by the Generic.Forensic.LocalHashes.Glob contains a timestamp column which represents the time in which the hash was inserted into the database. We could, I guess add an mtime column. In the case of Generic.Forensic.LocalHashes.Glob it makes sense because it is the mtime from the file but in the case of Windows.Forensics.LocalHashes.Usn it will always be similar to the timestamp since we insert the hash pretty soon after the file is modified and the journal is updated (maybe a few seconds to minutes).

Thanks

Mike

[Roberto]

Thank you Mike for your reply, very instructive.