Re: [DFIR] Log2timeline - bodyfile creation [SEC=UNOFFICIAL]

[Daniel White]

+Joachim Metz author of libvshadow and +log2timeline-discuss where we typically discuss Plaso issues.

Hi Sven,

I chatted about this with Joachim today, and it sounds like what you're seeing is the general processing time required to unpack volume shadow copies. How many VSS on the image, and how far apart are they in time?

-Daniel

On Mon, 21 Aug 2017 at 10:00 Sven Püschel <spues...@gmail.com> wrote:

Hi Kristinn,

thanks for the hint regarding the new plaso version.

I did some testing with the plaso release 2017-08-05.

The parsing with zero-mq looks better and doesn't get stuck any more.

But I have the feeling that as soon as I activate the VSS parsing plaso becomes realy slow.

For Testing purposes I created an E01-Disk-Image of a Windows-Testing VM (60GB).
 Using the plaso filestat parser on the mounted image without vss it takes about 10 to 15 minutes.

To get a better feeling I also did another run with more parsers like "filestat, evt(x), syslog, iis, registry" on the mounted image and it was finished after 15 to 20 minutes.

After starting the processing on the image with vss plaso became very slow. I canceled the parsing (only filestat) after 8 to 10 hours.

It seems that plaso stucks in directories with many little files like "winsxs".

Can you send me any suggestions for increasing the vss parsing ?

Thanks.

Sven

2017-08-12 20:36 GMT+02:00 Kristinn Gudjonsson <kris...@log2timeline.net>:

+Daniel White 

Hi

Sorry for the late answer, I'm adding Daniel to this thread so that he can answer some of the concerns here.

If you read the blog post that was pointed out earlier in the thread. http://diftdisk.blogspot.com/2016/11/plaso-15-inconsistent-results.html there was a bug that has since then been fixed.

So I would recommend that you upgrade to the latest development release to see if that has not resolved your issue. This should be fixed in the next release.

On Thu, Aug 10, 2017 at 10:55 PM Sven Püschel <spues...@gmail.com> wrote:

Hi John,

We're using Plaso very often. We figured out that due to changes in plaso 1.5.1, log2timeline will get stuck on whole disk images. In such cases we're using the older version 1.4.0.

It seems that this has something to do with the queuing (zero mq) that they are using in plaso 1.5.1. (
http://diftdisk.blogspot.com/2016/11/plaso-15-inconsistent-results.html).

So one thing that i suggest is using it without zero mq. You can find the argument in the help.

You could also use the image_export.py to extract the files from the image and afterwards you can start plaso and let it parse the extracted files. Sometimes this is faster.

As Nick already suggested it's better to set the number of workers manually.

If you wanna try plaso 1.4.0 the easiest way is to use ubuntu 16.04 server because version 1.4.0 is included in the default repository an can be installed via apt.

Hope that helps

Cheers,

Sven

Nick Klein <ni...@kleinco.com.au> schrieb am Do. 10. Aug. 2017 um 03:10:

Hey John, I’ve found that when processing VSS, manually selecting the number of workers provides much better performance. 

I believe Plaso automatically spawns a number of workers equal to one less than your total number of cores. I’m guessing that by selecting one less frees another worker for VSS mounting. For example, if you’ve got 8 cores, use the option --workers 6

This is just my experience and I’ve not tested the internals to confirm, but it’s worked well for me.

Hope that helps,

Nick Klein Director ni...@kleinco.com.au 0407 614 143 1300 444 500 Suite 502, Level 5 28 O'Connell Street Sydney NSW 2000 kleinco.com.au

On 10 Aug 2017, at 8:48 am, Walker, John PO 2 <john.w...@defence.gov.au> wrote:

UNOFFICIAL

I have been working on creating the bodyfile for an image of a laptop HDD (256GB), however it has been taking an extremely long time. There are 3 VSS stores which have been included in the creation.

 

First time I ran the l2t script, I let it run over the weekend, after a total of approx 5 days it still hadn't completed. I have restarted the process and it has currently been running for over 46 hours. I am running htop to observe the processing with most of the CPU's running at 80-100% with some drops for short periods.

 

Is this a common occurance? Should I just drop the VSS stores and process those at a later date?

 

Any suggestions appreciated.

 

Thanks,

 

John

 

 

_______________________________________________
DFIR mailing list
DF...@lists.sans.org
https://lists.sans.org/mailman/listinfo/dfir

Sent to ni...@kleinco.com.au

_______________________________________________
DFIR mailing list
DF...@lists.sans.org
https://lists.sans.org/mailman/listinfo/dfir

Sent to spues...@gmail.com

_______________________________________________
DFIR mailing list
DF...@lists.sans.org
https://lists.sans.org/mailman/listinfo/dfir

Sent to kris...@log2timeline.net

--

--
with regards

Kristinn

p.s sorry for all spleling mistakes, this email may have been written by large thumbs on a tiny mobile screen. If not and it was written using a real keyboard I've got no excuses what so ever.