Log2timeline parameters to run on a HFS 500GB drive

[pm]

What would be the most optimal parameters for log2timeline to process a 500GB HFS drive (unlocked) on the analysis machine with about 16vCPUs and 122GB of Memory.

Currently, experimenting with the below:

log2timeline.py --parsers macos  --workers 100 --status_view window --process_memory_limit 0 --hasher_file_size_limit 30000000     --queue_size 5000000  --buffer_size 300000 --logfile  log2time_data.log log2timeline_data.plaso mac_hfs.dd

I have not tried using the artifacts switches.  As of now it seems to take about 20+ hours to process the 500GB drive, I would like to reduce the processing time. 

Pratik 

 

[Joachim Metz]

It highly depends on the data in the image, e.g. a lot of small files,
a lot of large text log files, etc.

Any indication which parser / type of file is taking the most processing time?

> --
> You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.
> To post to this group, send email to log2timeli...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Pratik Mehta]

Its a mix of small files and mostly related to Spotlight and a portion of gzip. I have experimented with --skip_compressed_streams but it leaves out log files(syslog etc), with the unified logs I guess we can exclude that. 

I can use a machine with more memory but I am trying to get the optimal buffer, workers and any memory limits imposed. 

--

Pratik Mehta

[Daniel White]

log2timeline.py is usually CPU bound, so increasing the worker count and disabling the memory limit is unlikely to make things faster. Unless there's something unusual happening, the default worker count will take all CPU cores to 100%.

You could try disabling hashing with --hashers=none if you don't need hashing support, but I suspect the biggest speedup would be from creating a filter file: https://github.com/log2timeline/plaso/wiki/Collection-Filters for the syntax, 

https://github.com/log2timeline/plaso/blob/master/data/filter_windows.txt for a Windows example.

We don't have a default filter file for MacOS yet, so if you do create one, please share it.

-Daniek