handling CSV parameter in VQL

[Fusao Tanida]

Hi. We have a VQL file that has a CSV parameter

some artifacts accesses data of specified column in <parameter_name>.<column_name> as array as follows:

--

parameters:
  - name: search_files_glob
    type: csv
    default: |
      Pathes
      C:\temp\1\**
      C:\temp\2\**

sources:

  SELECT * FROM glob(globs=search_files_glob.Pathes)

--

Can we always assume that <parameter_name>.<column_name> includes all values if the parameter type is csv ?

Thank you 

Fusao.

[Mike Cohen]

When you define an artifact parameter as type csv two things happen:

1. The GUI builds a table to help users enter the table with the specified columns

2. The VQL compiler generates statements to parse the CSV into a list of dicts, keyed by the CSV headers. That list will then be available in the VQL as the parameter name. (You can see the compiler statements by looking in the Requests table of a collection :-).

The next point is accessing search_files_glob.Paths - what is happening here is "search_files_glob" is an array of dicts (as guaranteed by the parameter decleration). Then we apply the "." operator on an array which creates an array of elements by applying the "." operator on each member (that is just how "." behaves with arrays). So it will return an array of paths which is what glob expects.

So it looks like this:

[{"Path": "C:/Temp/1/**"}, {"Path": "C:/Temp/2/**"}] -> search_globs.Path ->  ["C:/Temp/1/**", "C:/Temp/2/**"]

The reason we recommend doing exactly what you are doing above is that glob() is smart enough to optimize the filesystem passes when given multiple globs, so it is better to give it all the globs at once rather than do something like

SELECT * FROM foreach(row=search_files_glob,

query={

   SELECT * FROM glob(globs=Paths)

})

Which will perform multiple passes on the filesystem. It probably does not matter in the example you posted but something like this:

C:/Temp/**/*.exe

C:/Temp/**/*.dll

will benefit from being combined.

Thanks

Mike

Mike Cohen  Digital Paleontologist,  Velocidex Enterprises M  ‭+61 470 238 491‬  E mi...@velocidex.com

--
You received this message because you are subscribed to the Google Groups "velociraptor-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to velociraptor-dis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/velociraptor-discuss/c994cf6c-055b-4cb6-ab26-a9c600a60752n%40googlegroups.com.

[Fusao Tanida]

Hi Mike. Thank you for detailed explanation with internal behaviors.

My question was completely cleared.

Best Regards

Fusao