Error when trying to tag events within a plaso file

[Colin]

Hi all,

I have generated a .plaso file from a windows system and now I'm trying to tag the events and my google fu is failing me.

Version info:

log2timeline - v1.5.1

psort.py - v1.5.1

The command I'm running:

psort.py -o null --analysis tagging --tagging-file /full/path/to/tagging/file/windows.tags /full/path/to/plaso/file/l2t.plaso

The result I'm getting:

Traceback (most recent call last):

  File "/usr/local/bin/psort.py", line 5, in <module>

    pkg_resources.run_script('plaso==1.5.1', 'psort.py')

  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 748, in run_script

    self.require(requires)[0].run_script(script_name, ns)

  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1517, in run_script

    exec(code, namespace, namespace)

  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.1-py2.7.egg/EGG-INFO/scripts/psort.py", line 880, in <module>

    if not Main():

  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.1-py2.7.egg/EGG-INFO/scripts/psort.py", line 866, in Main

    tool.ProcessStorage()

  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.1-py2.7.egg/EGG-INFO/scripts/psort.py", line 773, in ProcessStorage

    self._options, analysis_plugin)

  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.1-py2.7.egg/plaso/cli/helpers/manager.py", line 76, in ParseOptions

    helper.ParseOptions(options, config_object)

  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.1-py2.7.egg/plaso/cli/helpers/tagging_analysis.py", line 64, in ParseOptions

    analysis_plugin.SetAndLoadTagFile(tagging_file)

  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.1-py2.7.egg/plaso/analysis/tagging.py", line 228, in SetAndLoadTagFile

    self._tag_rules = self._ParseTaggingFile(self._tagging_file_name)

  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.1-py2.7.egg/plaso/analysis/tagging.py", line 145, in _ParseTaggingFile

    for label_name, rules in self._ParseDefinitions(tag_file_path):

  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.1-py2.7.egg/plaso/analysis/tagging.py", line 96, in _ParseDefinitions

    query = self._ParseRule(rule)

  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.1-py2.7.egg/plaso/analysis/tagging.py", line 126, in _ParseRule

    return efilter_query.Query(rule, syntax=syntax)

  File "/usr/local/lib/python2.7/dist-packages/efilter/query.py", line 93, in __init__

    (self.syntax, self.source))

ValueError: Cannot find parser for syntax u'objectfilter'. Source was "data_type is 'windows:prefetch'".

I have downloaded the tag file from here: https://github.com/log2timeline/plaso/blob/master/data/tag_windows.txt

The questions I have:

1) How can I see what the data_type list looks like because it seems that 'windows:prefetch' or any of the other data types listed in the tag file are missing.

2) Is there something I need to install to get this 'preset' list, if there is indeed a preset list.

3) Do I need to generate this list myself, if so, how?

Thanks in advance,

Colin

[Joachim Metz]

> ValueError: Cannot find parser for syntax u'objectfilter'. Source was "data_type is 'windows:prefetch'".

Colin, this looks like an issue with the efilter dependency.

Also 1.5.1 is somewhat old, for plaso sake
I opt to upgrade to the latest version, for more information see:
https://github.com/log2timeline/plaso/wiki/Users-Guide

> 1) How can I see what the data_type list looks like because it seems that 'windows:prefetch' or any of the other data types listed in the tag file are missing.

not sure what you mean, how did you conclude "data types listed in the
tag file are missing" ?
data types are defined in the event data objects e.g.
https://github.com/log2timeline/plaso/blob/master/plaso/parsers/winprefetch.py#L35
at the moment there is no tool to generate the full list of these types

> 2) Is there something I need to install to get this 'preset' list, if there is indeed a preset list.

what preset list are you looking for? that of the data types?

> --
> You received this message because you are subscribed to the Google Groups
> "log2timeline-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to log2timeline-dis...@googlegroups.com.
> To post to this group, send email to log2timeli...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Colin van Niekerk]

Hi Joachim,

I will update plaso and give it another try.

With regards to your questions about my questions. I now understand a bit more about how these data_types are built/registered (by the parsers). Thanks for your input. I believe I misread that error message which led me to think the data_types were an issue.

I'll feedback after an update. Thanks again!

Regards,

Colin 

[Colin van Niekerk]

Hi again,

I have updated to plaso 20180127.

Still getting the same (or very similar) stack trace:

Traceback (most recent call last):

  File "/usr/local/bin/psort.py", line 4, in <module>

    __import__('pkg_resources').run_script('plaso==20180127', 'psort.py')

  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 748, in run_script

    self.require(requires)[0].run_script(script_name, ns)

  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1517, in run_script

    exec(code, namespace, namespace)

  File "/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/EGG-INFO/scripts/psort.py", line 68, in <module>

    if not Main():

  File "/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/EGG-INFO/scripts/psort.py", line 30, in Main

    if not tool.ParseArguments():

  File "/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/plaso/cli/psort_tool.py", line 391, in ParseArguments

    self.ParseOptions(options)

  File "/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/plaso/cli/psort_tool.py", line 472, in ParseOptions

    self._analysis_plugins = self._CreateAnalysisPlugins(options)

  File "/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/plaso/cli/tool_options.py", line 51, in _CreateAnalysisPlugins

    options, analysis_plugin)

  File "/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/plaso/cli/helpers/manager.py", line 77, in ParseOptions

    helper_class.ParseOptions(options, config_object)

  File "/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/plaso/cli/helpers/tagging_analysis.py", line 70, in ParseOptions

    analysis_plugin.SetAndLoadTagFile(tagging_file_path)

  File "/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/plaso/analysis/tagging.py", line 229, in SetAndLoadTagFile

    self._tag_rules = self._ParseTaggingFile(tagging_file_path)

  File "/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/plaso/analysis/tagging.py", line 156, in _ParseTaggingFile

    for label_name, rules in self._ParseDefinitions(tag_file_path):

  File "/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/plaso/analysis/tagging.py", line 104, in _ParseDefinitions

    event_tagging_expression.group(1))

  File "/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/plaso/analysis/tagging.py", line 137, in _ParseEventTaggingRule

    return efilter_query.Query(event_tagging_expression, syntax=syntax)

  File "/usr/local/lib/python2.7/dist-packages/efilter/query.py", line 93, in __init__

    (self.syntax, self.source))

ValueError: Cannot find parser for syntax u'objectfilter'. Source was u"data_type is 'windows:prefetch'".

I have looked for a parser that handles the windows:prefetch' data type and found this:

/usr/local/lib/python2.7/dist-packages/plaso-20180127-py2.7.egg/plaso/parsers# cat winprefetch.py

# -*- coding: utf-8 -*-

"""Parser for Windows Prefetch files."""

from __future__ import unicode_literals

import pyscca

<snip>

  DATA_TYPE = 'windows:prefetch:execution'

<snip>

So the prefetch parser seems to be tabling its data type as 'windows:prefetch:execution' but the tagging file was looking for 'windows:prefetch'

Although even after updating the tagging file to use 'windows:prefetch:execution' I still got a very similar error, although then it obviously referenced the newly updated 'windows:prefetch:execution' parser that it couldn't find.

Not sure what else to try. Any pointers would be greatly appreciated.

Thanks,

Colin

[Joachim Metz]

Could you also provide us with some context:

* which version of efilter to do have installed?
* how was it installed?

[Colin van Niekerk]

Sure. It'll be the same version that ships with version 20180127.

How do I check that version specifically?

I removed all trace of plaso from the python dist-packages directory. After trying to remove it using pip and apt-get to be sure.

Once it was removed I installed the 20180127 version of plaso by running 'python setup.py install'. No errors reported.

Not sure if that is clear enough?

[Joachim Metz]

> How do I check that version specifically?

There is a check dependencies script that can make life easier:
https://github.com/log2timeline/plaso/wiki/Troubleshooting-installation-issues#ubuntu

> After trying to remove it using pip and apt-get to be sure.

It is not recommended to mix the 2 package managers, it can leave your
system in an unstable state
if you have to use pip do it in combination with virtualenv
also see: https://github.com/log2timeline/plaso/wiki/Running-plaso-in-virtualenv

[Colin van Niekerk]

Thanks for the info.

I have setup a virtual environment and deployed inside it and all seems fine so far. I just need to regenerate the plaso file and then I'll test the tagging again.

thanks for your help.

Regards,

Colin