Pytsk3 missing when trying to build dfvfs...?

[Rodger Moore]

Hi!

Because of this bug I am working on a fresh dev install of Plaso on CentOS7 64bit in VMware Workstation 9 and I'm having trouble building dfvfs. Steps taken so far (all under root user):

1. Installed CentOS7 x64 minimal install.
2. CentOS7 comes with Python 2.7
3. yum update
4. installed VMware tools
5. Installed Java jdk1.8.0_45 and added to environment
6. yum install wget git kernel-devel make gcc 
7. yum groupinstall "Development Tools"
   
8. yum install gcc-c++ python-devel python-setuptools rpm-build git mercurial
   
9. yum install flex byacc zlib-devel bzip2-devel openssl-devel fuse-devel
   
10. cd /tmp
    
11. wget https://www.samba.org/ftp/talloc/talloc-2.1.2.tar.gz
    
12. tar xzf talloc-2.1.2.tar.gz
    
13. cd talloc-2.1.2
    
14. ./configure
    
15. make install
    
16. cd /tmp
    
17. wget http://sourceforge.net/projects/sleuthkit/files/sleuthkit/4.1.2/sleuthkit-4.1.2.tar.gz
    
18. tar xzf sleuthkit-4.1.2.tar.gz
    
19. cd sleuthkit-4.1.2
    
20. ./configure
    
21. make 
22. make install
23. cd /tmp
24. wget https://pypi.python.org/packages/source/s/setuptools/setuptools-15.2.tar.gz#md5=a9028a9794fc7ae02320d32e2d7e12ee
    
25. tar xzf setuptools-15.2.tar.gz
    
26. cd /setuptools-15.2
27. python setup.py build
    
28. python setup.py install
    
29. cd /tmp
30. wget https://pypi.python.org/packages/source/p/pytsk3/pytsk3-3.2.3-20150406.tar.gz
31. tar xzf pytsk3-3.2.3-20150406.tar.gz
32. cd pytsk3-3.2.3-20150406
    
33. make
    
34. make install
    
35. cd /tmp
36. git clone https://github.com/log2timeline/dfvfs.git
    
37. cd dfvfs
38. python run_tests.py --> result (never mind about the rest of the failures):

[FAILURE]       missing: construct.

[FAILURE]       missing: google.protobuf.

[FAILURE]       missing: six.

[OK]            sqlite3 version: 3.7.17

[FAILURE]       missing: pytsk3 ???????????????????????????

[FAILURE]       missing: pybde.

[FAILURE]       missing: pyewf.

[FAILURE]       missing: pyqcow.

[FAILURE]       missing: pysigscan.

[FAILURE]       missing: pysmdev.

[FAILURE]       missing: pysmraw.

[FAILURE]       missing: pyvhdi.

[FAILURE]       missing: pyvmdk.

[FAILURE]       missing: pyvshadow.

What am I doing wrong?

Thanks!

[Joachim Metz]

    cd pytsk3-3.2.3-20150406
    make
    make install

pytsk uses distulits/setuptools

so this should be:
./setup.py build
sudo ./setup.py install

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.
To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rodger Moore]

Hi Joachim,

Thanks for your quick response. I restored a snapshot and I followed your suggestion so under root user I did:

1. cd /tmp
2. wget https://pypi.python.org/packages/source/p/pytsk3/pytsk3-3.2.3-20150406.tar.gz
3. tar xzf pytsk3-3.2.3-20150406.tar.gz
4. cd pytsk3-3.2.3-20150406
   

1. ./setup.py build
2. sudo ./setup.py install

1. cd /tmp
2. git clone https://github.com/log2timeline/dfvfs.git
   
3. cd dfvfs
4. python run_tests.py --> result (never mind about the rest of the failures):

[FAILURE]       missing: construct.

[FAILURE]       missing: google.protobuf.

[FAILURE]       missing: six.

[OK]                sqlite3 version: 3.7.17

[FAILURE]       missing: pytsk3 ???????????????????????????

[FAILURE]       missing: pybde.

[FAILURE]       missing: pyewf.

[FAILURE]       missing: pyqcow.

[FAILURE]       missing: pysigscan.

[FAILURE]       missing: pysmdev.

[FAILURE]       missing: pysmraw.

[FAILURE]       missing: pyvhdi.

[FAILURE]       missing: pyvmdk.

[FAILURE]       missing: pyvshadow.

So unfortunately I'm still stuck on the same error. Any other suggestions?

Many thanks.

Op vrijdag 8 mei 2015 13:59:44 UTC+2 schreef Joachim Metz:

[Joachim Metz]

What is the output of the pytsk build commands?

Also see: https://github.com/py4n6/pytsk/wiki/Troubleshooting

[Joachim Metz]

And: https://github.com/py4n6/pytsk/wiki/Building

[Rodger Moore]

Thanks again Joachim! 

:--|| 

I feel a bit stupid because there are no errors building and installing dfvfs! So is there something wrong with the test script(s)? I ask this because after cloning the Plaso git and running ./utils/check_dependencies.py I get the same error on pytsk3:

Checking availability and versions of plaso dependencies.

[FAILURE]       missing: artifacts.

[FAILURE]       missing: bencode.

[FAILURE]       missing: binplist.

[FAILURE]       missing: construct.

[FAILURE]       missing: dateutil.

[OK]            dfvfs version: 20150503

[FAILURE]       missing: dpkt.

[FAILURE]       missing: google.protobuf.

[FAILURE]       missing: hachoir_core.

[FAILURE]       missing: hachoir_parser.

[FAILURE]       missing: hachoir_metadata.

[FAILURE]       missing: IPython.

[FAILURE]       missing: pefile.

[FAILURE]       missing: psutil.

[FAILURE]       missing: pyparsing.

[FAILURE]       missing: pytz.

[FAILURE]       missing: six.

[OK]            sqlite3 version: 3.7.17

[FAILURE]       missing: yaml.

[FAILURE]       missing: pytsk3.

[FAILURE]       missing: pybde.

[FAILURE]       missing: pyesedb.

[FAILURE]       missing: pyevt.

[FAILURE]       missing: pyevtx.

[FAILURE]       missing: pyewf.

[FAILURE]       missing: pyfwsi.

[FAILURE]       missing: pylnk.

[FAILURE]       missing: pymsiecf.

[FAILURE]       missing: pyolecf.

[FAILURE]       missing: pyqcow.

[FAILURE]       missing: pyregf.

[FAILURE]       missing: pysigscan.

[FAILURE]       missing: pysmdev.

[FAILURE]       missing: pysmraw.

[FAILURE]       missing: pyvhdi.

[FAILURE]       missing: pyvmdk.

[FAILURE]       missing: pyvshadow.

Thanks again.

Op vrijdag 8 mei 2015 14:28:55 UTC+2 schreef Joachim Metz:

And: https://github.com/py4n6/pytsk/wiki/Building

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Kristinn Gudjonsson]

Have you done

ipython

import pytsk3

Does that work? If not then could you go over the steps you used to install pytsk3?

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Joachim Metz]

> I feel a bit stupid because there are no errors building and installing dfvfs!

What do you mean? dfVFS uses pytsk as well

> So is there something wrong with the test script(s)?

It currently works on travis and appveyor so unlikely

> I ask this because after cloning the Plaso git and running ./utils/check_dependencies.py I get the same error on pytsk3:

This makes sense since they use the same code.

I repeat, what is the output of building pytsk?

[Rodger Moore]

Hi Kristinn,

iPython

import pytsk3 gives the following error:

ImportError: libtsk.so.10: cannot open shared object file: No such file or directory

libtsk.so.10 is located in /usr/local/lib so is this the wrong location?

I installed pytsk3 following the steps in my initial post as root user:

1. ./setup.py build
2. ./setup.py install

Cheers,

Rodger

Op vrijdag 8 mei 2015 14:40:16 UTC+2 schreef Kristinn Gudjonsson:

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Joachim Metz]

> libtsk.so.10 is located in /usr/local/lib so is this the wrong location?

Depends on your settings, but if this location is not part of your system library path, this could be cause.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Kristinn Gudjonsson]

What is the permission of the so file?

ls -l /usr/local/lib/libtsk.so.10

? That could be the issue too ... make sure it is readable by everyone and try again.

[Rodger Moore]

I checked the permissions and added the directory to $PYTHONPATH and $PATH but (after reboot) still the same error. 

I Will do a build of dfvfs and put the output here.

Op vrijdag 8 mei 2015 16:22:45 UTC+2 schreef Kristinn Gudjonsson:

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Joachim Metz]

> I Will do a build of dfvfs and put the output here.

I opt to just focus on building sleuthkit and pytsk first

Remove the current installs and make sure to install them closely following:
https://github.com/py4n6/pytsk/wiki/Building-SleuthKit
https://github.com/py4n6/pytsk/wiki/Building

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Rodger Moore]

Ok so both sleuthkit and pytsk were build and installed successfully. Here is the bash history output for building pytsk3:

[root@hera tmp]# cd pytsk/

[root@hera pytsk]# python setup.py build

Sleuthkit headers found in: /usr/local/include

Sleuthkit version found: 4.1.3

Pytsk version found: 20150406

running build

running build_ext

[root@hera pytsk]# python setup.py install

Sleuthkit headers found in: /usr/local/include

Sleuthkit version found: 4.1.3

Pytsk version found: 20150406

running install

running build

running build_ext

running install_lib

copying build/lib.linux-x86_64-2.7/pytsk3.so -> /usr/lib64/python2.7/site-packages

running install_data

creating /usr/share/doc/pytsk

copying LICENSE -> /usr/share/doc/pytsk

copying README -> /usr/share/doc/pytsk

running install_egg_info

Writing /usr/lib64/python2.7/site-packages/pytsk3-4.1.3_20150406-py2.7.egg-info

[root@hera pytsk]# cd /tmp

[root@hera tmp]# git clone https://github.com/log2timeline/dfvfs.git

Cloning into 'dfvfs'...

remote: Counting objects: 3322, done.

remote: Total 3322 (delta 0), reused 0 (delta 0), pack-reused 3322

Receiving objects: 100% (3322/3322), 60.13 MiB | 2.52 MiB/s, done.

Resolving deltas: 100% (2625/2625), done.

[root@hera tmp]# cd dfvfs/

[root@hera dfvfs]# ./run_tests.py

Checking availability and versions of dfvfs dependencies.

[FAILURE]       missing: construct.

[FAILURE]       missing: google.protobuf.

[FAILURE]       missing: six.

[OK]            sqlite3 version: 3.7.17

[FAILURE]       missing: pytsk3.

[FAILURE]       missing: pybde.

[FAILURE]       missing: pyewf.

[FAILURE]       missing: pyqcow.

[FAILURE]       missing: pysigscan.

[FAILURE]       missing: pysmdev.

[FAILURE]       missing: pysmraw.

[FAILURE]       missing: pyvhdi.

[FAILURE]       missing: pyvmdk.

[FAILURE]       missing: pyvshadow.

Maybe important to mention is that I did not added or changed anything yet to the Python config or something else. So its just has been downloading, building and installing. Nothing else, besides what is advised in the manuals has been executed. I will look into this further on tuesday as I have no time this weekend. Thanks again for the support so far.

Cheers,

Op vrijdag 8 mei 2015 16:50:38 UTC+2 schreef Joachim Metz:

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Joachim Metz]

> Sleuthkit headers found in: /usr/local/include

the Sleuthkit installation is still in /usr/local/include

note that you have to tell configure that Sleuthkit should be installed in /usr if you want it there instead of /usr/local

From: https://github.com/py4n6/pytsk/wiki/Building-SleuthKit#using-gnu-compiler-collection-gcc

This will put the library and tools in /usr/local, if you want to change this to e.g. /usr add --prefix=/usr when running ./configure.

./configure --prefix=/usr

if /usr/local is the correct place then make sure it is in your LD configuration

http://linux.die.net/man/8/ldconfig

Also make sure you don't have multiple installation of the Sleuthkit on your system before you build pytsk, since it will use the first one it detects

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Joachim Metz]

to be verbose note that http://linux.die.net/man/8/ldconfig might not fully represent your systems manual so check "man ldconfig" as well if needed

[Rodger Moore]

Hi Joachim,

I was too curious to let this wait till tuesday and installed a new VM at home. As I put all the commands in the first post it was easy to rebuild. Your last comment solved it! It was my lack of knowledge regarding ldconfig causing the problem. 

I used ./configure --prefix=/usr and now the output is this:

[root@hera dfvfs]# ./run_tests.py

Checking availability and versions of dfvfs dependencies.

[FAILURE]       missing: construct.

[FAILURE]       missing: google.protobuf.

[FAILURE]       missing: six.

[OK]            sqlite3 version: 3.7.17

[OK]            SleuthKit version: 4.1.2

[OK]            pytsk3 version: 20150406

[FAILURE]       missing: pybde.

[FAILURE]       missing: pyewf.

[FAILURE]       missing: pyqcow.

[FAILURE]       missing: pysigscan.

[FAILURE]       missing: pysmdev.

[FAILURE]       missing: pysmraw.

[FAILURE]       missing: pyvhdi.

[FAILURE]       missing: pyvmdk.

[FAILURE]       missing: pyvshadow.

So we are good to go. You can mark this post as solved.

Thanks so much for your help! 

Cheers,

Rodger

Op zaterdag 9 mei 2015 08:43:27 UTC+2 schreef Joachim Metz:

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Rodger Moore]

Hi Joachim,

After installing all dependencies and Plaso building it from Github (git clone https://github.com/log2timeline/plaso) I tried to run log2timeline but got an error. I created an E01 system image with ewfacquire and after finishing I was able to successfully mount it with ewfmount.

[root@hera tmp]#ewfmount /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01 /mnt/ewf/

ewfmount 20140427

[root@hera tmp]# mmls /mnt/ewf/ewf1

DOS Partition Table

Offset Sector: 0

Units are in 512-byte sectors

      Slot      Start             End               Length           Description

00:  Meta    0000000000   0000000000   0000000001   Primary Table (#0)

01:  -----      0000000000   0000002047   0000002048   Unallocated

02:  00:00   0000002048   0000206847   0000204800   NTFS (0x07)

03:  00:01   0000206848   0439554047   0439347200   NTFS (0x07)

04:  -----      0439554048   0488397167   0048843120   Unallocated

This is setup:

Checking availability and versions of plaso dependencies.

[OK]            artifacts version: 20150409

[OK]            bencode

[OK]            binplist version: 0.1.5

[OK]            construct version: 2.5.2

[OK]            dateutil version: 1.5

[OK]            dfvfs version: 20150503

[OK]            dpkt version: 1.8

[OK]            google.protobuf

[OK]            hachoir_core version: 1.3.3

[OK]            hachoir_parser version: 1.3.4

[OK]            hachoir_metadata version: 1.3.3

[OK]            IPython version: 3.0.0

[OK]            pefile version: 1.2.10-139

[OK]            psutil version: 1.2.1

[OK]            pyparsing version: 2.0.3

[OK]            pytz

[OK]            six version: 1.9.0

[OK]            sqlite3 version: 3.7.17

[OK]            yaml version: 3.10

[OK]            SleuthKit version: 4.1.2

[OK]            pytsk3 version: 20150406

[OK]            libbde (pybde) version: 20150204

[OK]            libesedb (pyesedb) version: 20150409

[OK]            libevt (pyevt) version: 20150105

[OK]            libevtx (pyevtx) version: 20150105

[INFO]          libewf (pyewf) version: 20140427 installed, version: 20150126 available.

[OK]            libfwsi (pyfwsi) version: 20150124

[OK]            liblnk (pylnk) version: 20150105

[OK]            libmsiecf (pymsiecf) version: 20150314

[OK]            libolecf (pyolecf) version: 20150413

[OK]            libqcow (pyqcow) version: 20150105

[OK]            libregf (pyregf) version: 20150315

[OK]            libsigscan (pysigscan) version: 20150125

[OK]            libsmdev (pysmdev) version: 20150105

[OK]            libsmraw (pysmraw) version: 20150105

[OK]            libvhdi (pyvhdi) version: 20150110

[OK]            libvmdk (pyvmdk) version: 20150325

[OK]            libvshadow (pyvshadow) version: 20150106

Before I could even run log2time I had to change dfvfs/resolver/context.py

self, maximum_number_of_file_objects=128

to

self, maximum_number_of_file_objects=262144

After this running log2timeline.py gave the following error.

[root@hera tmp]# log2timeline.py -o 206848 systemx.dump /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

[WARNING] Unable to scan for a supported filesystem with error: Unable to scan source, with error: Unable to process source path specification with error: 'pyewf_handle_read_buffer: unable to read data. libewf_chunk_data_initialize: invalid chunk data. libewf_read_io_handle_read_chunk_data: unable to create chunk data. libewf_handle_read_buffer: unable to read chunk data: 6966691.'

FS_Info_Con: (tsk3.c:207) Unable to open the image as a filesystem: Cannot determine file system type

Most likely the image format is not supported by the tool.

Should I open an issue on Github for this or is this a known problem (with a solution)?

Cheers,

Rodger

Op zaterdag 9 mei 2015 11:16:00 UTC+2 schreef Rodger Moore:

...

[Joachim Metz]

> Before I could even run log2time I had to change dfvfs/resolver/context.py

Which version of plaso are you running? older versions of plaso are incompatible with newer version of dfvfs due more strict caching policy in recent dfvfs

> [INFO]          libewf (pyewf) version: 20140427 installed, version: 20150126 available.

Try the latest stable version libewf-20140608.tar.gz not the one in the git repo you can find it via the downloads link on:
https://github.com/libyal/libewf/wiki

--

You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Rodger Moore]

Hi Joachim,

My Plaso version is: 

plaso - log2timeline version 1.2.1_20150509

I changed libewf so my config is now:

[INFO] libewf (pyewf) version: 20140608 installed, version: 20150126 available.

[OK] libfwsi (pyfwsi) version: 20150124
[OK] liblnk (pylnk) version: 20150105
[OK] libmsiecf (pymsiecf) version: 20150314
[OK] libolecf (pyolecf) version: 20150413
[OK] libqcow (pyqcow) version: 20150105
[OK] libregf (pyregf) version: 20150315
[OK] libsigscan (pysigscan) version: 20150125
[OK] libsmdev (pysmdev) version: 20150105
[OK] libsmraw (pysmraw) version: 20150105
[OK] libvhdi (pyvhdi) version: 20150110
[OK] libvmdk (pyvmdk) version: 20150325
[OK] libvshadow (pyvshadow) version: 20150106

But the error remains. Maybe I should start with a smaller and simpler image, I'll try the Sift demo image.

Rodger

Op maandag 11 mei 2015 15:21:40 UTC+2 schreef Joachim Metz:

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Joachim Metz]

maybe the issue is masked (e.g. BitLocker encrypted volume?) can you fls both partitions?

Hi Joachim,

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Rodger Moore]

fls is working fine on the mounted image, not the image itself:

[root@hera plaso]# ewfmount 11052015_SYSTEMX.E01 /mnt/ewf/

[root@hera plaso]# fls -o 2048 /mnt/ewf/ewf1

r/r 4-128-4: $AttrDef
r/r 8-128-2: $BadClus
r/r 8-128-1: $BadClus:$Bad
r/r 6-128-4: $Bitmap
r/r 7-128-1: $Boot
d/d 11-144-4: $Extend
r/r 2-128-1: $LogFile
r/r 0-128-1: $MFT
r/r 1-128-1: $MFTMirr
r/r 9-128-8: $Secure:$SDS
r/r 9-144-11: $Secure:$SDH
r/r 9-144-14: $Secure:$SII
r/r 10-128-1: $UpCase
r/r 3-128-3: $Volume
d/d 35-144-5: Boot
r/r 85-128-1: bootmgr
r/r 96-128-3: BOOTSECT.BAK
d/d 97-144-1: System Volume Information
d/d 256: $OrphanFiles

AND

[root@hera plaso]# fls -o 206848 /mnt/ewf/ewf1

d/d 276-144-8: Program Files (x86)
d/d 23089-144-1: Recovery
d/d 489-144-5: Users
r/r 4-128-4: $AttrDef
r/r 8-128-2: $BadClus
r/r 8-128-1: $BadClus:$Bad
r/r 6-128-4: $Bitmap
r/r 7-128-1: $Boot
d/d 11-144-4: $Extend
r/r 2-128-1: $LogFile
r/r 0-128-1: $MFT
r/r 1-128-1: $MFTMirr
d/d 57-144-1: $Recycle.Bin
r/r 9-144-18: $Secure:$SDH
r/r 9-144-16: $Secure:$SII
r/r 9-128-19: $Secure:$SDS
r/r 10-128-1: $UpCase
r/r 3-128-3: $Volume
d/d 280345-144-5: Config.Msi
d/d 13910-144-1: Documents and Settings
r/r 91668-128-1: pagefile.sys
d/d 58-144-1: PerfLogs
d/d 60-144-6: Program Files
d/- * 0: Program Files
r/r * 15685-128-4(realloc): WIM2321.tmp
r/r * 15686-128-4(realloc): WIM2322.tmp
r/r * 15867-128-1(realloc): WIM37DB.tmp
r/r * 15868-128-3(realloc): WIM37DC.tmp
r/d * 15870-144-1(realloc): WIM37DD.tmp
r/r * 15871-128-4(realloc): WIM37EE.tmp
r/d * 15872-144-1(realloc): WIM37EF.tmp
d/d 651-144-7: Windows
r/d * 54-144-1(realloc): WinPEpge.sys
d/d 395-144-6: ProgramData
d/- * 0: Python27
r/- * 0: Restoration_Tmp1.tmp
r/- * 0: Restoration_Tmp10.tmp
r/- * 0: Restoration_Tmp11.tmp
d/- * 0: _945466_
r/- * 0: WinPEpge.sys
d/d 16191-144-6: System Volume Information
-/d * 357784-144-1: adobeTemp
-/d * 362607-144-6: Config.Msi
d/d 362752: $OrphanFiles

Its an image taken from a SSD, can this be an issue? I didn't mention it before but in this situation log2timeline.py is sucking up 1 CPU thread to 100% and it needs to be forced quit (ctrl-c).

Op maandag 11 mei 2015 20:16:39 UTC+2 schreef Joachim Metz:

Hi Joachim,

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit

...

[Kristinn Gudjonsson]

and did you try to run log2timeline both on the E01 file and the /mnt/ewf/ewf1 fusemount?

Hi Joachim,

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit

...

--

You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Rodger Moore]

Yes I did, in both situations the issue occurred.

Op maandag 11 mei 2015 21:01:30 UTC+2 schreef Kristinn Gudjonsson:

...

[Joachim Metz]

> fls is working fine on the mounted image, not the image itself:

How did you compile tsk with EWF support or without?

img_stat should tell you more if your TSK build supports EWF directly

> Its an image taken from a SSD, can this be an issue? I didn't mention it before but in this situation log2timeline.py is sucking up 1 CPU thread to 100% and it needs to be forced quit (ctrl-c).

can you tell which process? is it really hot spinning or also processing? the output should be able to tell you if it is generating event objects or not.

--

[Kristinn Gudjonsson]

is there only a single log2timeline process running?

On Mon, May 11, 2015 at 7:51 PM Rodger Moore <rodge...@gmail.com> wrote:

Just to make sure I recompiled and re-installed tsk with the following config:

[root@hera sleuthkit-4.1.2]#make uninstall

[root@hera sleuthkit-4.1.2]#./configure --prefix=/usr --with-libewf=/usr
[root@hera sleuthkit-4.1.2]#make
[root@hera sleuthkit-4.1.2]#make install

So assuming tsk is build with ewf support I ran:

[root@hera ~]# img_stat -i ewf -v /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

tsk_img_open: Type: 64   NumImg: 1  Img1: /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01
ewf_open: found 159 segment files via libewf_glob
IMAGE FILE INFORMATION
--------------------------------------------
Image Type: ewf
Size of data in bytes: 250059350016
MD5 hash of data: 8b50171c4d2f649250a39cf7880834ef

So I believe TSK is working correct?

CPU sticks to 100% all the time and no stdout in terminal (blinking cursor) see screens:

Op maandag 11 mei 2015 21:08:55 UTC+2 schreef Joachim Metz:

...

[Rodger Moore]

htop was setup showing "no userlands threads" but enabling all threads its still only 1 thread yes:

Op vrijdag 8 mei 2015 13:51:54 UTC+2 schreef Rodger Moore:

Hi!

Because of this bug I am working on a fresh dev install of Plaso on CentOS7 64bit in VMware Workstation 9 and I'm having trouble building dfvfs. Steps taken so far (all under root user):

1. Installed CentOS7 x64 minimal install.
2. CentOS7 comes with Python 2.7
3. yum update
4. installed VMware tools
5. Installed Java jdk1.8.0_45 and added to environment
6. yum install wget git kernel-devel make gcc 
7. yum groupinstall "Development Tools"
   
8. yum install gcc-c++ python-devel python-setuptools rpm-build git mercurial
   

1. git clone https://github.com/log2timeline/dfvfs.git
   
2. cd dfvfs
3. python run_tests.py --> result (never mind about the rest of the failures):

[FAILURE]       missing: construct.

[FAILURE]       missing: google.protobuf.

[FAILURE]       missing: six.

[OK]            sqlite3 version: 3.7.17

[FAILURE]       missing: pytsk3 ???????????????????????????

[FAILURE]       missing: pybde.

[FAILURE]       missing: pyewf.

[FAILURE]       missing: pyqcow.

[FAILURE]       missing: pysigscan.

[FAILURE]       missing: pysmdev.

[FAILURE]       missing: pysmraw.

[FAILURE]       missing: pyvhdi.

[FAILURE]       missing: pyvmdk.

[FAILURE]       missing: pyvshadow.

[Kristinn Gudjonsson]

tell me again how you ran the tool?

And do you not see any information on the console you ran the tool that suggests the tool is working? That is the foreman should show information like this:

$ log2timeline.py storage.plaso test.dd 

Source path                             : /test_images/test.dd

Is storage media image or device        : True

Partition offset                        : 32256 (0x00007e00)

[INFO] Processing started.

[INFO] [PreProcess] Set attribute: sysregistry to /WINDOWS/system32/config

[INFO] [PreProcess] Set attribute: systemroot to /WINDOWS

[INFO] [PreProcess] Set attribute: windir to /WINDOWS

[INFO] [PreProcess] Set attribute: users to [{u'path': u'%systemroot%\\system32\\config\\systemprofile', u'name': u'systemprofile', u'sid': u'S-1-5-18'}, {u'path': u'%SystemDrive%\\Documents and Settings\\LocalService', u'name': u'LocalService', u'sid': u'S-1-5-19'}, {u'path': u'%SystemDrive%\\Documents and Settings\\NetworkService', u'name': u'NetworkService', u'sid': u'S-1-5-20'}, {u'path': u'%SystemDrive%\\Documents and Settings\\Mr. Evil', u'name': u'Mr. Evil', u'sid': u'S-1-5-21-2000478354-688789844-1708537768-1003'}]

[INFO] [PreProcess] Set attribute: programfiles to Program Files

[INFO] [PreProcess] Set attribute: programfilesx86 to None

[INFO] [PreProcess] Set attribute: osversion to Microsoft Windows XP

[INFO] [PreProcess] Set attribute: code_page to cp1252

[INFO] [PreProcess] Set attribute: hostname to N-1A9ODN6ZXK4LQ

[INFO] [PreProcess] Set attribute: time_zone_str to CST6CDT

[INFO] Parser filter expression changed to: winxp

[INFO] Setting timezone to: CST6CDT

[INFO] Starting extraction in multi process mode.

[INFO] Starting processes.

[INFO] Storage writer (PID: 18145) started.

[INFO] Worker 0 (PID: 18146) started monitoring process queue.

[INFO] Worker 2 (PID: 18148) started monitoring process queue.

[INFO] Worker 1 (PID: 18147) started monitoring process queue.

[INFO] Worker 3 (PID: 18150) started monitoring process queue.

[INFO] Worker 4 (PID: 18152) started monitoring process queue.

[INFO] Worker 5 (PID: 18153) started monitoring process queue.

[INFO] Worker 6 (PID: 18156) started monitoring process queue.

[INFO] Worker 7 (PID: 18158) started monitoring process queue.

[INFO] Worker 8 (PID: 18160) started monitoring process queue.

[INFO] Worker 9 (PID: 18162) started monitoring process queue.

[INFO] Collector (PID: 18164) started

[INFO] Worker_0 (PID: 18146) - events extracted: 152 - file: TSK:/Documents and Settings/All Users/Start Menu/Programs/Accessories/Entertainment/Windows Media Player.lnk - running: True <disk-sleep>

[INFO] Worker_1 (PID: 18147) - events extracted: 20 - file: TSK:/$LogFile - running: True <disk-sleep>

[INFO] Worker_2 (PID: 18148) - events extracted: 155 - file: TSK:/Documents and Settings/All Users/Start Menu/Programs/Accessories/Entertainment/Volume Control.lnk - running: True <disk-sleep>

[INFO] Worker_3 (PID: 18150) - events extracted: 4 - file: TSK:/hiberfil.sys - running: True <disk-sleep>

[INFO] Worker_4 (PID: 18152) - events extracted: 118 - file: TSK:/Documents and Settings/All Users/Start Menu/Programs/Accessories/System Tools/desktop.ini - running: True <disk-sleep>

That is you should indication on how many workers are getting started, etc. And running ps aux should then show something similar to this then:

$ ps aux | grep log2timeline

foobar    19620 68.5  0.1 272852 60036 pts/60   S+   13:15   0:08 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19625 60.5  0.1 357308 61520 pts/60   Sl+  13:15   0:02 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19626 88.2  0.1 494308 52788 pts/60   Rl+  13:15   0:03 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19627 89.0  0.1 498612 57132 pts/60   Rl+  13:15   0:03 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19628 88.5  0.1 493696 51560 pts/60   Rl+  13:15   0:03 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19629 90.2  0.1 503932 62200 pts/60   Rl+  13:15   0:03 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19631 89.2  0.1 495740 54500 pts/60   Rl+  13:15   0:03 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19633 88.7  0.1 497856 56276 pts/60   Rl+  13:15   0:03 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19635 89.7  0.1 419120 51236 pts/60   Rl+  13:15   0:03 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19636 87.7  0.1 495348 53780 pts/60   Rl+  13:15   0:03 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19638 87.7  0.1 498348 56800 pts/60   Rl+  13:15   0:03 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19643 87.5  0.1 577724 62764 pts/60   Rl+  13:15   0:03 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

foobar    19645 94.5  0.1 423136 54108 pts/60   Rl+  13:15   0:03 /usr/bin/python /usr/local/bin/log2timeline.py storage.plaso test.dd

--

[Rodger Moore]

log2timeline.py -o 2048 systemx.dump /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

System is CentOS7, Python 2.7 comes with the system. I believe foreman workers aren't even initialized because the image can't be loaded. There is no output in the console, just the cursor blinking. This is the trackback log after I hit ctrl-c:

Traceback (most recent call last):
File "/usr/bin/log2timeline.py", line 4, in <module>
 __import__('pkg_resources').run_script('plaso==1.2.1.post20150509', 'log2timeline.py')
  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 729, in run_script
  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 1642, in run_script
  File "/usr/lib/python2.7/site-packages/plaso-1.2.1.post20150509-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 576, in <module>
    if not Main():
  File "/usr/lib/python2.7/site-packages/plaso-1.2.1.post20150509-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 559, in Main
    tool.ProcessSource()
  File "/usr/lib/python2.7/site-packages/plaso-1.2.1.post20150509-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 504, in ProcessSource
    vss_stores=self._vss_stores)
  File "/usr/lib/python2.7/site-packages/plaso-1.2.1.post20150509-py2.7.egg/plaso/frontend/storage_media_frontend.py", line 450, in ScanSource
    self._scan_context, scan_path_spec=scan_path_spec)
  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 406, in Scan
  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 265, in _ScanNode
  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 513, in ScanForVolumeSystem
  File "build/bdist.linux-x86_64/egg/dfvfs/analyzer/analyzer.py", line 342, in GetVolumeSystemTypeIndicators
  File "build/bdist.linux-x86_64/egg/dfvfs/analyzer/analyzer.py", line 184, in _GetTypeIndicators
IOError: pysigscan_scanner_scan_file_object: unable to scan file. pysigscan_file_object_read_buffer: unable to read from file object with error: 'pyewf_handle_read_buffer: unable to read data. libewf_chunk_data_initialize: invalid chunk data. libewf_read_io_handle_read_chunk_data: unable to create chunk data. libewf_handle_read_buffer: unable to read chunk data: 0.'. pysigscan_file_object_io_handle_read: unable to read from file object. libbfio_handle_read_buffer: unable to read from handle. libsigscan_scanner_scan_file_io_handle: unable to read buffer.

The image is loaded from a VMWare shared folder and mounted to /mnt/hgfs/ 

Op maandag 11 mei 2015 22:16:36 UTC+2 schreef Kristinn Gudjonsson:

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Joachim Metz]

> fls is working fine on the mounted image, not the image itself:

let's focus on this first, since if TSK does not work, plaso is not going to work either

To be verbose the following does not work?

fls -o 2048 /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

fls -o 206848 /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

>  img_stat -i ewf -v /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

> So I believe TSK is working correct?

Not sure how you concluded that I can only deduce from it that it recognizes EWF as an image format,

I hence can conclude that your build of TSK has EWF support it built in.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Rodger Moore]

Jumping to conclusions too quick is one of my bad habitats I'm trying to get rid off...sorry for that. 

>  img_stat -i ewf -v /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

> So I believe TSK is working correct?

Not sure how you concluded that I can only deduce from it that it recognizes EWF as an image format,

I hence can conclude that your build of TSK has EWF support it built in.

Thats what I meant, yes. I concluded my build of TSK has EWF support build in.

To answer your questions:

To be verbose the following does not work?

fls -o 2048 /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

fls -o 206848 /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01 

This IS actually working:

[root@hera log]# fls -o 2048 /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

r/r 4-128-4:    $AttrDef

r/r 8-128-2:    $BadClus

r/r 8-128-1:    $BadClus:$Bad

r/r 6-128-4:    $Bitmap

r/r 7-128-1:    $Boot

d/d 11-144-4:   $Extend

r/r 2-128-1:    $LogFile

r/r 0-128-1:    $MFT

r/r 1-128-1:    $MFTMirr

r/r 9-128-8:    $Secure:$SDS

r/r 9-144-11:   $Secure:$SDH

r/r 9-144-14:   $Secure:$SII

r/r 10-128-1:   $UpCase

r/r 3-128-3:    $Volume

d/d 35-144-5:   Boot

r/r 85-128-1:   bootmgr

r/r 96-128-3:   BOOTSECT.BAK

d/d 97-144-1:   System Volume Information

d/d 256:        $OrphanFiles

[root@hera log]# fls -o 206848 /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

I can try to load this image in Autopsy using Kali Linux and see if its working on that instance of TSK? Maybe this image is just causing the troubles...

Op dinsdag 12 mei 2015 08:53:19 UTC+2 schreef Joachim Metz:

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Joachim Metz]

> This IS actually working:

That information changes the trouble shooting

Then the issue looks very much like an issue seen earlier with dfvfs-plaso caching incompatibilites

> Is this a version of dfvfs where you changed? self, maximum_number_of_file_objects=262144

This should not be needed and might indicate another issue. If you change this back do you get an cache full exception?

do the dfvfs and plaso tests run correctly?

./run_tests.py

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Rodger Moore]

Rebuilding dfvfs with default cache settings gives the cache error:

[root@hera plaso]# log2timeline.py -o 2048 /tmp/systemx.dump /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

Traceback (most recent call last):

  File "/usr/bin/log2timeline.py", line 4, in <module>

    __import__('pkg_resources').run_script('plaso==1.2.1.post20150509', 'log2timeline.py')

  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 729, in run_script

  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 1642, in run_script

  File "/usr/lib/python2.7/site-packages/plaso-1.2.1.post20150509-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 576, in <module>

    if not Main():

  File "/usr/lib/python2.7/site-packages/plaso-1.2.1.post20150509-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 559, in Main

    tool.ProcessSource()

  File "/usr/lib/python2.7/site-packages/plaso-1.2.1.post20150509-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 504, in ProcessSource

    vss_stores=self._vss_stores)

  File "/usr/lib/python2.7/site-packages/plaso-1.2.1.post20150509-py2.7.egg/plaso/frontend/storage_media_frontend.py", line 450, in ScanSource

    self._scan_context, scan_path_spec=scan_path_spec)

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 406, in Scan

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 265, in _ScanNode

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 513, in ScanForVolumeSystem

  File "build/bdist.linux-x86_64/egg/dfvfs/analyzer/analyzer.py", line 342, in GetVolumeSystemTypeIndicators

  File "build/bdist.linux-x86_64/egg/dfvfs/analyzer/analyzer.py", line 182, in _GetTypeIndicators

  File "build/bdist.linux-x86_64/egg/dfvfs/resolver/resolver.py", line 113, in OpenFileObject

  File "build/bdist.linux-x86_64/egg/dfvfs/file_io/file_io.py", line 75, in open

  File "build/bdist.linux-x86_64/egg/dfvfs/file_io/file_object_io.py", line 63, in _Open

  File "build/bdist.linux-x86_64/egg/dfvfs/file_io/ewf_file_io.py", line 82, in _OpenFileObject

  File "build/bdist.linux-x86_64/egg/dfvfs/resolver/resolver.py", line 113, in OpenFileObject

  File "build/bdist.linux-x86_64/egg/dfvfs/file_io/file_io.py", line 79, in open

  File "build/bdist.linux-x86_64/egg/dfvfs/resolver/context.py", line 52, in CacheFileObject

  File "build/bdist.linux-x86_64/egg/dfvfs/resolver/cache.py", line 83, in CacheObject

 dfvfs.lib.errors.CacheFullError: Maximum number of cached values reached.

running run_tests.py in dfvfs gives no errors:

Ran 319 tests in 29.075s

OK

running run_tests.py in Plaso gives:

Test the magic class functions. ... Key: \Software\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{FA99DFC7-6AC2-453A-A5E2-5E2AFF4507BD} not found

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{FA99DFC7-6AC2-453A-A5E2-5E2AFF4507BD} not found

Key: \Software\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{F4E57C4B-2036-45F0-A9AB-443BCFE33D9F} not found

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{F4E57C4B-2036-45F0-A9AB-443BCFE33D9F} not found

Key: \Software\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{F2A1CB5A-E3CC-4A2E-AF9D-505A7009D442} not found

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{F2A1CB5A-E3CC-4A2E-AF9D-505A7009D442} not found

Key: \Software\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{CEBFF5CD-ACE2-4F4F-9178-9926F41749EA} not found

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{CEBFF5CD-ACE2-4F4F-9178-9926F41749EA} not found

Key: \Software\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{CAA59E3C-4792-41A5-9909-6A6A8D32490E} not found

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{CAA59E3C-4792-41A5-9909-6A6A8D32490E} not found

Key: \Software\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{B267E3AD-A825-4A09-82B9-EEC22AA3B847} not found

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{B267E3AD-A825-4A09-82B9-EEC22AA3B847} not found

Key: \Software\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{A3D53349-6E61-4557-8FC7-0028EDCEEBF6} not found

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{A3D53349-6E61-4557-8FC7-0028EDCEEBF6} not found

Key: \Software\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{9E04CAB2-CC14-11DF-BB8C-A2F1DED72085} not found

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{9E04CAB2-CC14-11DF-BB8C-A2F1DED72085} not found

ERROR:root:Error in  while parsing file OS:/tmp/plaso/test_data/NTUSER.DAT: Unsupported value data size: 8

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{75048700-EF1F-11D0-9888-006097DEACF9} not found

ERROR:root:Error in  while parsing file OS:/tmp/plaso/test_data/NTUSER.DAT: Unsupported value data size: 8

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{5E6AB780-7743-11CF-A12B-00AA004AE837} not found

Key: \Software\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{0D6D4F41-2994-4BA0-8FEF-620E43CD2812} not found

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{0D6D4F41-2994-4BA0-8FEF-620E43CD2812} not found

Key: \Software\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{BCB48336-4DDD-48FF-BB0B-D3190DACB3E2} not found

Key: \Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\UserAssist\{BCB48336-4DDD-48FF-BB0B-D3190DACB3E2} not found

ok

testTopLevelMethods (tools.preg_test.PregToolTest)

Test few of the top level methods in the preg module. ... ok

======================================================================

FAIL: testProcess (plaso.parsers.sqlite_plugins.gdrive_test.GoogleDrivePluginTest)

Tests the Process function on a Google Drive database file.

----------------------------------------------------------------------

Traceback (most recent call last):

  File "./plaso/parsers/sqlite_plugins/gdrive_test.py", line 65, in testProcess

    self.assertEqual(event_object.document_type, u'DOCUMENT')

AssertionError: u'PRESENTATION' != u'DOCUMENT'

- PRESENTATION

+ DOCUMENT

----------------------------------------------------------------------

Ran 583 tests in 72.102s

FAILED (failures=1)

Op dinsdag 12 mei 2015 09:53:14 UTC+2 schreef Joachim Metz:

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Joachim Metz]

just to be sure, do you have multiple versions of log2timeline installed?

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Kristinn Gudjonsson]

And or multiple versions of dfvfs?

[Rodger Moore]

I didn't have the intention to install 2 versions but maybe I did this by accident. Silly question... what is the easiest way to check if I have multiple installations of Plaso?

Op dinsdag 12 mei 2015 10:46:42 UTC+2 schreef Joachim Metz:

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Joachim Metz]

regarding the cache full issue this might be related to the number of segments (159) I'll do some tests to see if that is an issue

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Kristinn Gudjonsson]

Can ddo

sudo updatedb
locate presets.py

And see if there are multipe instances of plaso installed.

Same with dfvfs, something like

locate tsk_file_system

[Rodger Moore]

Results:

[root@hera /]# updatedb

[root@hera /]# locate presets.py

/usr/lib/python2.7/site-packages/plaso-1.2.1.post20150509-py2.7.egg/plaso/frontend/presets.py

/usr/lib/python2.7/site-packages/plaso-1.2.1.post20150509-py2.7.egg/plaso/frontend/presets.pyc

locate tsk_file_system gives no results. locate tsk_ gives more

[root@hera /]# locate tsk_

/usr/bin/tsk_comparedir

/usr/bin/tsk_gettimes

/usr/bin/tsk_loaddb

/usr/bin/tsk_recover

/usr/include/tsk/tsk_incs.h

/usr/include/tsk/auto/tsk_auto.h

/usr/include/tsk/base/tsk_base.h

/usr/include/tsk/base/tsk_os.h

/usr/include/tsk/fs/tsk_ext2fs.h

/usr/include/tsk/fs/tsk_fatfs.h

/usr/include/tsk/fs/tsk_ffs.h

/usr/include/tsk/fs/tsk_fs.h

/usr/include/tsk/fs/tsk_hfs.h

/usr/include/tsk/fs/tsk_iso9660.h

/usr/include/tsk/fs/tsk_ntfs.h

/usr/include/tsk/fs/tsk_yaffs.h

/usr/include/tsk/hashdb/tsk_hashdb.h

/usr/include/tsk/img/tsk_img.h

/usr/include/tsk/vs/tsk_bsd.h

/usr/include/tsk/vs/tsk_dos.h

/usr/include/tsk/vs/tsk_gpt.h

/usr/include/tsk/vs/tsk_mac.h

/usr/include/tsk/vs/tsk_sun.h

/usr/include/tsk/vs/tsk_vs.h

/usr/lib/libtsk_jni.a

/usr/lib/libtsk_jni.la

/usr/lib/libtsk_jni.so

/usr/lib/libtsk_jni.so.0

/usr/lib/libtsk_jni.so.0.0.0

/usr/share/man/man1/tsk_comparedir.1

/usr/share/man/man1/tsk_gettimes.1

/usr/share/man/man1/tsk_loaddb.1

/usr/share/man/man1/tsk_recover.1

Op dinsdag 12 mei 2015 13:48:49 UTC+2 schreef Kristinn Gudjonsson:

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Kristinn Gudjonsson]

Sorry do

locate file_system_searcher

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Rodger Moore]

This also gives no results.

Op dinsdag 12 mei 2015 15:43:54 UTC+2 schreef Kristinn Gudjonsson:

Sorry do

locate file_system_searcher

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Joachim Metz]

So the CacheFullError is triggered by your image having + 128 segments

Sorry do

locate file_system_searcher

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Rodger Moore]

Ok, and what can I do to fix this?

Op dinsdag 12 mei 2015 22:15:40 UTC+2 schreef Joachim Metz:

Sorry do

locate file_system_searcher

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

[Joachim Metz]

You can increase the number for now to 128 + 160, I'll add a more structural fix to dfvfs for this

normally approx 128 open files should be more than enough

Sorry do

locate file_system_searcher

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Rodger Moore]

Ok so it seems Log2Timeline is still running after the adjustment of 128 to 280 (harddrive is spinnin' loud)  but I still don't get any stdout in my console? I'm running the process from SSH console. There should be output about foreman and stuff right? And htop is still showing only 1 CPU thread on 100%. Is this expected behaviour? Its only running for a couple of minutes, should I be more patient and wait for any output to come?

[root@hera plaso]# log2timeline.py -o 2048 /tmp/systemx.dump /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

Op dinsdag 12 mei 2015 22:20:09 UTC+2 schreef Joachim Metz:

Sorry do

locate file_system_searcher

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.
To unsubscribe from this group and stop receiving ema

...

[Joachim Metz]

Normally you should see:

Source path : test.E01

Is storage media image or device : True

Partition offset : 32256 (0x00007e00)

...

the open of the EWF files can take a bit

Sorry do

locate file_system_searcher

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.
To unsubscribe from this group and stop receiving ema

...

--

You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Joachim Metz]

a bit, depends on how large the image is, and how fast the IO is

also how large is the uncompressed image (the RAW in the EWF files) ?

[Rodger Moore]

ok lets wait and see.. thanks again.

Op dinsdag 12 mei 2015 22:40:53 UTC+2 schreef Joachim Metz:

Sorry do

locate file_system_searcher

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout

...

[Joachim Metz]

Also can you try the following:

can you run python dfvfs/examples/recursive_hasher2.py on the image

python dfvfs/examples/recursive_hasher2.py test.E01

Sorry do

locate file_system_searcher

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout

...

--

You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Rodger Moore]

So I let the log2timeline.py process run overnight. I had to kill it this morning as the console had no output at all and 1 CPU thread was still 100%. I think the process is in endless loop? This was the output after killing it with ctrl-c:

[root@hera plaso]# log2timeline.py -o 2048 /tmp/systemx.dump /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

^C

[WARNING] Unable to scan for a supported filesystem with error: Unable to scan source, with error: Unable to process source path specification with error: 'pyewf_handle_read_buffer: unable to read data. libewf_chunk_data_initialize: invalid chunk data. libewf_read_io_handle_read_chunk_data: unable to create chunk data. libewf_handle_read_buffer: unable to read chunk data: 6952542.'

FS_Info_Con: (tsk3.c:207) Unable to open the image as a filesystem: Cannot determine file system type

Most likely the image format is not supported by the tool.

I tried your second suggestion and this gave the same symptoms. Killing it with ctrl-c gave this output:

[root@hera ~]# python /tmp/dfvfs/examples/recursive_hasher2.py /mnt/hgfs/Kali_Im

^CTraceback (most recent call last):

  File "/tmp/dfvfs/examples/recursive_hasher2.py", line 393, in <module>

    if not Main():

  File "/tmp/dfvfs/examples/recursive_hasher2.py", line 374, in Main

    base_path_spec = recursive_hasher.GetBasePathSpec(options.source)

  File "/tmp/dfvfs/examples/recursive_hasher2.py", line 258, in GetBasePathSpec

    scan_context, scan_path_spec=scan_path_spec)

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 406, in Scan

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 299, in _ScanNode

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 339, in _ScanNode

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 429, in ScanForFileSystem

dfvfs.lib.errors.BackEndError: Unable to process source path specification with error: 'pyewf_handle_read_buffer: unable to read data. libewf_chunk_data_initialize: invalid chunk data. libewf_read_io_handle_read_chunk_data: unable to create chunk data. libewf_handle_read_buffer: unable to read chunk data: 6891302.'

FS_Info_Con: (tsk3.c:207) Unable to open the image as a filesystem: Cannot determine file system type

Op dinsdag 12 mei 2015 22:44:33 UTC+2 schreef Joachim Metz:

Sorry do

locate file_system_searcher

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-discuss+unsub...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://group

...

[Joachim Metz]

So this looks like an issue triggered in the source scanner, now to figure out why and what the issue could be.

Sorry do

locate file_system_searcher

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://group

...

--

You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.

[Joachim Metz]

I'll try to think of some test to pin point the issue.

[Rodger Moore]

Do you need some info about the image I created? Its an image of my personal PC so I can't share this one. But if you need some meta info let me know.

Op woensdag 13 mei 2015 08:58:11 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

I've prepared some changes to dfvfs: https://codereview.appspot.com/238170044/

as a first step in analyzing source scanner issues.

if you know how to patch the changes in manually I opt you give them a try, otherwise wait until they are submitted and checkout the latest dfvfs source

And then provide the output of the following command is:

PYTHONPATH=. python examples/source_analyzer.py --no-auto-recurse

--

[Rodger Moore]

So I downloaded a diff file from https://codereview.appspot.com/238170044/ and placed it in the root of dfvfs. Tried the following command but getting a reject:

[root@hera dfvfs]# patch -u -p1 -i issue238170044_40001_50001.diff

Any suggestions to get the files patched manually (this is new for me but like to learn it.

Tnx

Op donderdag 14 mei 2015 13:05:07 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

try:

patch -p1 < issue238170044_40001_50001.diff

--

[Rodger Moore]

Are the patches applied to Github repo already? Getting notifications about already applied patches. Tried to apply patch after re-cloning from Github.

Op donderdag 14 mei 2015 19:52:29 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

yes patches are applied to the git repo as indicated in: https://codereview.appspot.com/238170044/

--

[Rodger Moore]

Sorry... noob

Op donderdag 14 mei 2015 20:22:30 UTC+2 schreef Joachim Metz:

...

[Rodger Moore]

It starts ok then same symptom, 1 CPU thread at 100% and need to break it. This is the output:

[root@hera dfvfs]# PYTHONPATH=. python examples/source_analyzer.py --no-auto-recurse /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

Scan level: 0

Source type : storage media image

OS: location: /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

  EWF: 

Scan level: 1

Source type : storage media image

OS: location: /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

  EWF: 

    TSK_PARTITION: location: /

      TSK_PARTITION: 0, start offset: 0 (0x00000000)

      TSK_PARTITION: 1, start offset: 0 (0x00000000)

      TSK_PARTITION: 2, start offset: 1048576 (0x00100000), location: /p1

      TSK_PARTITION: 3, start offset: 105906176 (0x06500000), location: /p2

      TSK_PARTITION: 4, start offset: 225051672576 (0x3466200000)

Scan level: 2

Source type : storage media image

OS: location: /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

  EWF: 

    TSK_PARTITION: location: /

      TSK_PARTITION: 0, start offset: 0 (0x00000000)

      TSK_PARTITION: 1, start offset: 0 (0x00000000)

      TSK_PARTITION: 2, start offset: 1048576 (0x00100000), location: /p1

        TSK: location: /

      TSK_PARTITION: 3, start offset: 105906176 (0x06500000), location: /p2

      TSK_PARTITION: 4, start offset: 225051672576 (0x3466200000)

Scan level: 3

Source type : storage media image

OS: location: /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

  EWF: 

    TSK_PARTITION: location: /

      TSK_PARTITION: 0, start offset: 0 (0x00000000)

      TSK_PARTITION: 1, start offset: 0 (0x00000000)

      TSK_PARTITION: 2, start offset: 1048576 (0x00100000), location: /p1

        TSK: location: /

      TSK_PARTITION: 3, start offset: 105906176 (0x06500000), location: /p2

        VSHADOW: location: /

          TSK: location: /

      TSK_PARTITION: 4, start offset: 225051672576 (0x3466200000)

^CTraceback (most recent call last):

  File "examples/source_analyzer.py", line 250, in <module>

    if not Main():

  File "examples/source_analyzer.py", line 235, in Main

    source_analyzer.Analyze(options.source, output_writer)

  File "examples/source_analyzer.py", line 62, in Analyze

    scan_path_spec=scan_path_spec)

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 486, in Scan

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 349, in _ScanNode

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 404, in _ScanNode

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 509, in ScanForFileSystem

dfvfs.lib.errors.BackEndError: Unable to process source path specification with error: 'pyewf_handle_read_buffer: unable to read data. libewf_chunk_data_initialize: invalid chunk data. libewf_read_io_handle_read_chunk_data: unable to create chunk data. libewf_handle_read_buffer: unable to read chunk data: 6914818.'

FS_Info_Con: (tsk3.c:207) Unable to open the image as a filesystem: Cannot determine file system type

 

Op donderdag 14 mei 2015 13:05:07 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

Thx for the output.

> It starts ok then same symptom, 1 CPU thread at 100% and need to break it. This is the output:

That is ok for now that means the issue is reproducible. 

Can you try running vshadowinfo on the second volume:

ewfmount /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01 /mnt/ewf/

vshadowinfo -o $(( 0x06500000 )) /mnt/ewf/ewf1

To be verbose vshadowinfo is part of libvshadow

--

[Rodger Moore]

[root@hera plaso]# vshadowinfo -o $(( 0x06500000 )) /mnt/ewf/ewf1

vshadowinfo 20150106

Volume Shadow Snapshot information:

Number of stores: 0

Op donderdag 14 mei 2015 20:41:58 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

thx, we'll have to dig a bit deeper, as far as I can currently tell the issue occurs somewhere in ScanForFileSystem()

I'm having a look what could be a possible culprit

--

[Rodger Moore]

Hi Joachim,

I see a new release is on the horizon. Is this issue fixed or do we need to add it to Github?

Cheers

Op donderdag 14 mei 2015 21:31:50 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

> Is this issue fixed or do we need to add it to Github?

yeah that is typically easier to track than email I've created.

https://github.com/log2timeline/plaso/issues/229

> I see a new release is on the horizon.

yes, we'll release a release candidate first and then I opt that we try again after release candidate (RC1) 

There are some worker/foreman issues that need to be fixed first seeing they affect testing

--

[Rodger Moore]

Ok tnx

Op vrijdag 5 juni 2015 13:12:06 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

Let's put some life back into trying to solve this issue.

I've made some changes to the source scanner, can you give those a try.

git clone -b source_scanner https://github.com/joachimmetz/dfvfs.git

PYTHONPATH=dfvfs python dfvfs/examples/source_analyzer.py --no-auto-recurse

If this doesn't work I'll have to add some debugging to a separate branch to see why it is looping.

--

[Rodger Moore]

After this command: 

[root@hera test]# PYTHONPATH=dfvfs python dfvfs/examples/source_analyzer.py --no-auto-recurse /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

I get this back:

Scan step: 0

Source type             : storage media image

OS: location: /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

EWF:

Traceback (most recent call last):

  File "dfvfs/examples/source_analyzer.py", line 286, in <module>

    if not Main():

  File "dfvfs/examples/source_analyzer.py", line 271, in Main

    source_analyzer.Analyze(options.source, output_writer)

  File "dfvfs/examples/source_analyzer.py", line 75, in Analyze

    for locked_scan_node in scan_context.locked_scan_nodes:

AttributeError: 'SourceScannerContext' object has no attribute 'locked_scan_nodes'

Op dinsdag 23 juni 2015 07:12:48 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

this error indicates python is not finding dfvfs in the PYTHONPATH and uses an older version of dfvfs

--

[Rodger Moore]

Ok well, don't know what I'm doing wrong with PYTHONPATH (need to learn that later on I guess) so I compiled and installed your dfvfs branch. Its running but it seems to get stuck at iteration (step) 6. After breaking the operation I get this error:

Scan step: 6

Source type             : storage media image

OS: location: /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

  EWF:

    TSK_PARTITION: location: /

      TSK_PARTITION: 0, start offset: 0 (0x00000000)

      TSK_PARTITION: 1, start offset: 0 (0x00000000)

      TSK_PARTITION: 2, start offset: 1048576 (0x00100000), location: /p1

        TSK: location: /

      TSK_PARTITION: 3, start offset: 105906176 (0x06500000), location: /p2

        VSHADOW: location: /

          TSK: location: /

      TSK_PARTITION: 4, start offset: 225051672576 (0x3466200000)

^CTraceback (most recent call last):

  File "dfvfs/examples/source_analyzer.py", line 286, in <module>

    if not Main():

  File "dfvfs/examples/source_analyzer.py", line 271, in Main

    source_analyzer.Analyze(options.source, output_writer)

  File "dfvfs/examples/source_analyzer.py", line 59, in Analyze

    scan_path_spec=scan_path_spec)

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 552, in Scan

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 474, in _ScanNode

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 575, in ScanForFileSystem

dfvfs.lib.errors.BackEndError: Unable to process source path specification with error: 'pyewf_handle_read_buffer: unable to read data. libewf_chunk_data_initialize: invalid chunk data. libewf_read_io_handle_read_chunk_data: unable to create chunk data. libewf_handle_read_buffer: unable to read chunk data: 6940055.'

FS_Info_Con: (tsk3.c:207) Unable to open the image as a filesystem: Cannot determine file system type

Op dinsdag 23 juni 2015 07:12:48 UTC+2 schreef Joachim Metz:

Let's put some life back into trying to solve this issue.

...

[Joachim Metz]

OK thanks, I'll add some more debug output here to see where and why it might be looping.

--

[Joachim Metz]

Created a test branch for now: https://github.com/joachimmetz/dfvfs/tree/test

Can you try with that version and send me the output (the loop is still there but trying to pinpoint it)

git clone -b test https://github.com/joachimmetz/dfvfs.git

PYTHONPATH=. python ./examples/source_analyzer.py --no-auto-recurse

Regarding PYTHONPATH not working earlier try using absolute paths instead of relative

[Rodger Moore]

Here you go:

[WARNING] _GetTypeIndicators enter

[WARNING] _GetTypeIndicators pysigscan enter

[WARNING] _GetTypeIndicators pysigscan exit

[WARNING] _GetTypeIndicators results enter

[WARNING] _GetTypeIndicators results exit

[WARNING] _GetTypeIndicators other enter

[WARNING] _GetTypeIndicators other exit

[WARNING] _GetTypeIndicators exit

Scan step: 0

Source type             : storage media image

OS: location: /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

  EWF:

[WARNING] _GetTypeIndicators enter

Traceback (most recent call last):

  File "./examples/source_analyzer.py", line 286, in <module>

    if not Main():

  File "./examples/source_analyzer.py", line 271, in Main

    source_analyzer.Analyze(options.source, output_writer)

  File "./examples/source_analyzer.py", line 59, in Analyze

    scan_path_spec=scan_path_spec)

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 560, in Scan

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 387, in _ScanNode

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 685, in ScanForVolumeSystem

dfvfs.lib.errors.BackEndError: Unable to process source path specification with error: pyewf_handle_open_file_objects: unable to open file. pyewf_file_object_read_buffer: unable to read from file object with error: (5, 'Input/output error'). pyewf_file_object_io_handle_read: unable to read from file object. libbfio_handle_read_buffer: unable to read from handle. libbfio_pool_read_buffer: unable to read from entry: 117. libewf_segment_file_read_file_header: unable to read file header. libewf_handle_open_file_io_pool: unable to read segment file header.

Op dinsdag 23 juni 2015 21:04:51 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

Thanks, but this looks like a different issue

Last time you came to "Scan step: 6" now it seems it stops at "Scan step: 0" to what seems it cannot read from the EWF files.

--

[Rodger Moore]

Rebooted and retried, here is the new output:

Scan step: 6

Source type             : storage media image

OS: location: /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

  EWF:

    TSK_PARTITION: location: /

      TSK_PARTITION: 0, start offset: 0 (0x00000000)

      TSK_PARTITION: 1, start offset: 0 (0x00000000)

      TSK_PARTITION: 2, start offset: 1048576 (0x00100000), location: /p1

        TSK: location: /

      TSK_PARTITION: 3, start offset: 105906176 (0x06500000), location: /p2

        VSHADOW: location: /

          TSK: location: /

      TSK_PARTITION: 4, start offset: 225051672576 (0x3466200000)

[WARNING] _GetTypeIndicators enter

[WARNING] _GetTypeIndicators pysigscan enter

[WARNING] _GetTypeIndicators pysigscan exit

[WARNING] _GetTypeIndicators results enter

[WARNING] _GetTypeIndicators results exit

[WARNING] _GetTypeIndicators other enter

[WARNING] _GetTypeIndicators other exit

[WARNING] _GetTypeIndicators exit

[WARNING] CP3: call to: ScanForFileSystem type: OS, location: /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

type: EWF

type: TSK_PARTITION, part index: 4, start offset: 0x3466200000

[WARNING] ScanForFileSystem entry: type: OS, location: /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

type: EWF

type: TSK_PARTITION, part index: 4, start offset: 0x3466200000

[WARNING] GetFileSystemTypeIndicators enter

[WARNING] _GetTypeIndicators enter

[WARNING] _GetTypeIndicators pysigscan enter

[WARNING] _GetTypeIndicators pysigscan exit

[WARNING] _GetTypeIndicators results enter

[WARNING] _GetTypeIndicators results exit

[WARNING] _GetTypeIndicators other enter

^CTraceback (most recent call last):

  File "./examples/source_analyzer.py", line 286, in <module>

    if not Main():

  File "./examples/source_analyzer.py", line 271, in Main

    source_analyzer.Analyze(options.source, output_writer)

  File "./examples/source_analyzer.py", line 59, in Analyze

    scan_path_spec=scan_path_spec)

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 560, in Scan

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 482, in _ScanNode

  File "build/bdist.linux-x86_64/egg/dfvfs/helpers/source_scanner.py", line 585, in ScanForFileSystem

dfvfs.lib.errors.BackEndError: Unable to process source path specification with error: 'pyewf_handle_read_buffer: unable to read data. libewf_chunk_data_initialize: invalid chunk data. libewf_read_io_handle_read_chunk_data: unable to create chunk data. libewf_handle_read_buffer: unable to read chunk data: 6890410.'

FS_Info_Con: (tsk3.c:207) Unable to open the image as a filesystem: Cannot determine file system type

Op woensdag 24 juni 2015 06:26:34 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

The output indicates that the file system scan loops in the "other" file systems analyzers which is the TSKAnalyzerHelper that tries to run pytsk3.FS_Info() on the volume.

Can you send me output/behavior of the following commands:

mmls /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

fls -o $(( 225051672576 / 512 )) /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

If you don't have TSK with EWF support using a mounted EWF is fine for now.

--

[Rodger Moore]

Here you go:

# mmls /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

DOS Partition Table

Offset Sector: 0

Units are in 512-byte sectors

     Slot    Start        End          Length       Description

00:  Meta    0000000000   0000000000   0000000001   Primary Table (#0)

01:  -----   0000000000   0000002047   0000002048   Unallocated

02:  00:00   0000002048   0000206847   0000204800   NTFS (0x07)

03:  00:01   0000206848   0439554047   0439347200   NTFS (0x07)

04:  -----   0439554048   0488397167   0048843120   Unallocated

# fls -o $(( 225051672576 / 512 )) /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01

Cannot determine file system type

Error so I try again after mounting

# ewfmount /mnt/hgfs/Kali_Image/11052015_SYSTEMX.E01 /mnt/ewf/

ewfmount 20140608

#fls -o $(( 225051672576 / 512 )) /mnt/ewf/ewf1

Cannot determine file system type

Same error

Op woensdag 24 juni 2015 06:49:02 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

Thanks, that does not loop, so the issue is likely somewhere in the interaction between TSK and the other code.

--

[Rodger Moore]

ok... what can we do now?

Op woensdag 24 juni 2015 11:10:15 UTC+2 schreef Joachim Metz:

...

[Joachim Metz]

This will be hard to pin-point, I'm largely thinking of removing TSK from this code path and getting an alternative solution in place:
https://github.com/log2timeline/dfvfs/issues/34

--

[Rodger Moore]

Now my case is just a single use-case. This seems pretty drastic if no other users are reporting issues with this? Should I try imaging the disk again using other tools and check again after that?

Op woensdag 24 juni 2015 11:42:07 UTC+2 schreef Joachim Metz:

...

[greg.f...@gmail.com]

Joachim,

Does it make sense at this time to have plaso simply ignore the unallocated space at the end of the disk?

You could work on the n problem for the next major release.

Greg

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Joachim Metz]

> This seems pretty drastic if no other users are reporting issues with this?

Should not be too drastic it's just replacing the existing approach with a signature check.

> Does it make sense at this time to have plaso simply ignore the unallocated space at the end of the disk?

How do you know it is unallocated? Just based on the information in the partition table?

That probably requires more work than changing to a signature based approach.

Also I'm not sure now roughly knowing what the issue is, that it is a show stopper for the next release.

[greg.f...@gmail.com]

On June 24, 2015 6:02:58 AM MDT, Joachim Metz <joachi...@gmail.com> wrote:>
>> Does it make sense at this time to have plaso simply ignore the
>unallocated space at the end of the disk?
>
>How do you know it is unallocated? Just based on the information in the
>partition table?

Yes, purely taking the partition table at face value. Seems like a reasonable thing to do for the 1.3 release.

Greg

[Joachim Metz]

Signature based approach:

https://github.com/joachimmetz/dfvfs/commit/440a6c17cbe99483756655205976700e4922341d

[Rodger Moore]

Succes!

Test was successful as well as log2timeline. Log2timeline is now processing. Don't forget to turn off debug logging, my console is a war zone ;) 

Thanks Joachim

Op woensdag 24 juni 2015 19:59:34 UTC+2 schreef Joachim Metz:

Signature based approach:

https://github.com/joachimmetz/dfvfs/commit/440a6c17cbe99483756655205976700e4922341d

On Wed, Jun 24, 2015 at 2:08 PM, <greg.f...@gmail.com> wrote:

On June 24, 2015 6:02:58 AM MDT, Joachim Metz <joachi...@gmail.com> wrote:>
>> Does it make sense at this time to have plaso simply ignore the
>unallocated space at the end of the disk?
>
>How do you know it is unallocated? Just based on the information in the
>partition table?

Yes, purely takin

...

[Joachim Metz]

Good to hear, I'll sync with Kristinn on this if this is something we move into 1.3 (RC2) or keep it as is.

> Don't forget to turn off debug logging, my console is a war zone ;) 

Yeah we've been splitting off status information and debug information (give the --status-view=window a try)

we're not there yet where I would like to see it, we'll get there eventually ;)

--

[Rodger Moore]

Attached is the log of source_analyser.py.

I'll give it a try with --status-view=window

Tnx

Op donderdag 25 juni 2015 08:54:55 UTC+2 schreef Joachim Metz:

Good to hear, I'll sync with Kristinn on this if this is something we move into 1.3 (RC2) or keep it as is.

> Don't forget to turn off debug logging, my console is a war zone ;) 

Yeah we've been splitting off status information and debug information (give the --status-view=window a try)

we're not there yet where I would like to see it, we'll get there eventually ;)

On Thu, Jun 25, 2015 at 8:34 AM, Rodger Moore <rodge...@gmail.com> wrote:

Succes!

Test was successful as well as log2timeline. Log2timeline is now processing. Don't forget to turn off debug logging, my console is a war zone ;) 

Thanks Joachim

Op woensdag 24 juni 2015 19:59:34 UTC+2 schreef Joachim Metz:

Signature based approach:

https://github.com/joachimmetz/dfvfs/commit/440a6c17cbe99483756655205976700e4922341d

On Wed, Jun 24, 2015 at 2:08 PM, <greg.f...@gmail.com> wrote:

...

[Joachim Metz]

To be verbose the status view option is for log2timeline.py

--

[Rodger Moore]

I understand, it's running already ;) Looks a lot better.

Op donderdag 25 juni 2015 09:10:31 UTC+2 schreef Joachim Metz:

To be verbose the status view option is for log2timeline.py

On 25 Jun 2015 09:07, "Rodger Moore" <rodge...@gmail.com> wrote:

Attached is the log of source_analyser.py.

I'll give it a try with --status-view=window

Tnx

Op donderdag 25 juni 2015 08:54:55 UTC+2 schreef Joachim Metz:

Good to hear, I'll sync with Kristinn on this if this is something we move into 1.3 (RC2) or keep it as is.

> Don't forget to turn off debug logging, my console is a war zone ;) 

Yeah we've been splitting off status information and debug information (give the --status-view=window a try)

we're not there yet where I would like to see it, we'll get there eventually ;)

On Thu, Jun 25, 2015 at 8:34 AM, Rodger Moore <rodge...@gmail.com> wrote:

Succes!

Test was successful as well as log2timeline. Log2timeline is now processing. Don't forget to turn off debug logging, my console is a war zone ;) 

Thanks Joachim

Op woensdag 24 juni 2015 19:59:34 UTC+2 schreef Joachim Metz:

...

[Rodger Moore]

Hmmm running into new issues. Process aborted because of workers being idle for too long:

[WARNING] (MainProcess) PID:2776 <multi_process> Processing aborted with error: Workers idle for too long.

Error in atexit._run_exitfuncs:

Traceback (most recent call last):

  File "/usr/lib64/python2.7/atexit.py", line 24, in _run_exitfuncs

    func(*targs, **kargs)

  File "/usr/lib64/python2.7/multiprocessing/util.py", line 319, in _exit_function

    p.join()

  File "/usr/lib64/python2.7/multiprocessing/process.py", line 145, in join

    res = self._popen.wait(timeout)

  File "/usr/lib64/python2.7/multiprocessing/forking.py", line 154, in wait

    return self.poll(0)

  File "/usr/lib64/python2.7/multiprocessing/forking.py", line 135, in poll

    pid, sts = os.waitpid(self.pid, flag)

KeyboardInterrupt

Error in sys.exitfunc:

Traceback (most recent call last):

  File "/usr/lib64/python2.7/atexit.py", line 24, in _run_exitfuncs

    func(*targs, **kargs)

  File "/usr/lib64/python2.7/multiprocessing/util.py", line 319, in _exit_function

    p.join()

  File "/usr/lib64/python2.7/multiprocessing/process.py", line 145, in join

    res = self._popen.wait(timeout)

  File "/usr/lib64/python2.7/multiprocessing/forking.py", line 154, in wait

    return self.poll(0)

  File "/usr/lib64/python2.7/multiprocessing/forking.py", line 135, in poll

    pid, sts = os.waitpid(self.pid, flag)

This happens dic files in TSK:/Program Files (x86)/Steam/tenfoot/resource/wordlists/

Should we open a new thread for this?

Op donderdag 25 juni 2015 09:14:45 UTC+2 schreef Rodger Moore:

I understand, it's running already ;) Looks a lot better.

Op donderdag 25 juni 2015 09:10:31 UTC+2 schreef Joachim Metz:

To be verbose the status view option is for log2timeline.py

On 25 Jun 2015 09:07, "Rodger Moore" <rodge...@gmail.com> wrote:

Attached is the log of source_analyser.py.

I'll give it a try with --status-view=window

Tnx

Op donderdag 25 juni 2015 08:54:55 UTC+2 schreef Joachim Metz:

Good to hear, I'll sync with Kristinn on this if this is something we move into 1.3 (RC2) or keep it as is.

> Don't forget to turn off debug logging, my console is a war zone ;) 

...

[Joachim Metz]

> Should we open a new thread for this?

Please create an issue on https://github.com/log2timeline/plaso/issues

This is likely due to all workers taking too long to process these files.

--

[Rodger Moore]

I hadn't updated plaso-dev git for a couple of weeks because we were working on dfvfs. Updated to latest code and seems to be solved too.

Op donderdag 25 juni 2015 09:53:54 UTC+2 schreef Joachim Metz:

...

[Yaniv Schiff]

Does the --status-view window option work? when i try to use it in SIFT3 i get "unrecognized arguments". i've also tried "--status_view window" and "--status-view=window" and "--status_view=window".

Running plaso backend 1.2.0

Thanks.

On Friday, May 8, 2015 at 6:51:54 AM UTC-5, Rodger Moore wrote:

Hi!

Because of this bug I am working on a fresh dev install of Plaso on CentOS7 64bit in VMware Workstation 9 and I'm having trouble building dfvfs. Steps taken so far (all under root user):

1. Installed CentOS7 x64 minimal install.
2. CentOS7 comes with Python 2.7
3. yum update
4. installed VMware tools
5. Installed Java jdk1.8.0_45 and added to environment
6. yum install wget git kernel-devel make gcc 
7. yum groupinstall "Development Tools"
   
8. yum install gcc-c++ python-devel python-setuptools rpm-build git mercurial
   
9. yum install flex byacc zlib-devel bzip2-devel openssl-devel fuse-devel
   
10. cd /tmp
    
11. wget https://www.samba.org/ftp/talloc/talloc-2.1.2.tar.gz
    
12. tar xzf talloc-2.1.2.tar.gz
    
13. cd talloc-2.1.2
    
14. ./configure
    
15. make install
    
16. cd /tmp
    
17. wget http://sourceforge.net/projects/sleuthkit/files/sleuthkit/4.1.2/sleuthkit-4.1.2.tar.gz
    
18. tar xzf sleuthkit-4.1.2.tar.gz
    
19. cd sleuthkit-4.1.2
    
20. ./configure
    
21. make 
22. make install
23. cd /tmp
24. wget https://pypi.python.org/packages/source/s/setuptools/setuptools-15.2.tar.gz#md5=a9028a9794fc7ae02320d32e2d7e12ee
    
25. tar xzf setuptools-15.2.tar.gz
    
26. cd /setuptools-15.2
27. python setup.py build
    
28. python setup.py install
    
29. cd /tmp
30. wget https://pypi.python.org/packages/source/p/pytsk3/pytsk3-3.2.3-20150406.tar.gz
31. tar xzf pytsk3-3.2.3-20150406.tar.gz
32. cd pytsk3-3.2.3-20150406
    
33. make
    
34. make install
    
35. cd /tmp
36. git clone https://github.com/log2timeline/dfvfs.git
    
37. cd dfvfs
38. python run_tests.py --> result (never mind about the rest of the failures):

[FAILURE]       missing: construct.

[FAILURE]       missing: google.protobuf.

[FAILURE]       missing: six.

[OK]            sqlite3 version: 3.7.17

[FAILURE]       missing: pytsk3 ???????????????????????????

[FAILURE]       missing: pybde.

[FAILURE]       missing: pyewf.

[FAILURE]       missing: pyqcow.

[FAILURE]       missing: pysigscan.

[FAILURE]       missing: pysmdev.

[FAILURE]       missing: pysmraw.

[FAILURE]       missing: pyvhdi.

[FAILURE]       missing: pyvmdk.

[FAILURE]       missing: pyvshadow.

What am I doing wrong?

Thanks!

[Kristinn Gudjonsson]

yes, running "--status_view window" should do the trick.

However you are running an older version that doesn't support that. You should be able to do "sudo apt-get update && sudo apt-get upgrade" and update plaso to 1.3.0 which has the status view.

If that doesn't work you'll need to add the GIFT repo to your SIFT station:

sudo add-apt-repository ppa:gift/stable
sudo apt-get update

sudo apt-get upgrade

That should get you to version 1.3.0 of plaso, which supports the status view.

--

You received this message because you are subscribed to the Google Groups "log2timeline-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to log2timeline-dis...@googlegroups.com.
To post to this group, send email to log2timeli...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

--
with regards

Kristinn

p.s sorry for all spleling mistakes, this email have been written by large thumbs on a tiny mobile screen. If not and it was written on the web I've got no excuses what so ever.

[Yaniv Schiff]

Thanks Kristinn, using the updated version the --status_view works.

--

Yaniv Schiff
Director Of Digital Forensics
Forensicon, Inc.
226 S. Wabash Suite 300
Chicago, IL 60604
Email: yaniv.schiff@forensicon.com
Main: 312-427-5667 x128
Direct: 312-893-5568
Fax: 312-427-5668
Add my business card to your address book by clicking: http://www.forensicon.com/vcard/yschiff.vcf

Download our brochure at: http://www.forensicon.com/wp-content/uploads/2012/04/forensicon_brochure.pdf

===================================================
http://www.forensicon.com/
Computer Forensics | eDiscovery | Litigation Risk Management | Cyber Security Breach Response | Online Identity Investigations
 
****Important Notice to Recipients****
This communication and attachments may contain legally privileged, confidential, or otherwise private information intended for a specific individual and purpose.  If you are not the intended recipient, you should delete this message and are hereby notified that any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited.
DISCLAIMER:  Any information contained in any communication from Forensicon employees is not intended to be legal advice.  Any person seeking legal advice should contact a licensed attorney.