pyvmiaddressspace.py crash

[T0MA]

I'm a bit stuck at using LibVMI with Volatility, any help would be
much appreciated. Some of the example codes provided with LibVMI work
fine including the python example but with Volatility I just see a
crash.

I copied pyvmiaddressspace.py to volatility/plugins/addrspaces/ and
ran Volatility with python vol.py -l honey-xp --profile=WinXPSP3x86
modules.

This is what I get:

Volatile Systems Volatility Framework 2.1_alpha
Traceback (most recent call last):
File "vol.py", line 135, in <module>
main()
File "vol.py", line 126, in main
command.execute()
File "/share/src/volatility-svn/volatility/commands.py", line 101,
in execute
func(outfd, data)
File "/share/src/volatility-svn/volatility/plugins/modules.py", line
38, in render_text
for module in data:
File "/share/src/volatility-svn/volatility/win32/modules.py", line
33, in lsmod
PsLoadedModuleList = tasks.get_kdbg(addr_space).PsLoadedModuleList
File "/share/src/volatility-svn/volatility/win32/tasks.py", line 48,
in get_kdbg
kdbgo = obj.VolMagic(addr_space).KDBG.v()
File "/share/src/volatility-svn/volatility/obj.py", line 808, in v
return self.get_best_suggestion()
File "/share/src/volatility-svn/volatility/obj.py", line 834, in
get_best_suggestion
for val in self.get_suggestions():
File "/share/src/volatility-svn/volatility/obj.py", line 826, in
get_suggestions
for x in self.generate_suggestions():
File "/share/src/volatility-svn/volatility/plugins/overlays/windows/
windows.py", line 661, in generate_suggestions
for val in scanner.scan(self.obj_vm):
File "/share/src/volatility-svn/volatility/plugins/kdbgscan.py",
line 67, in scan
for offset in scan.DiscontigScanner.scan(self, address_space,
offset, maxlen):
File "/share/src/volatility-svn/volatility/scan.py", line 145, in
scan
for match in BaseScanner.scan(self, address_space, o, l):
File "/share/src/volatility-svn/volatility/scan.py", line 103, in
scan
data = address_space.read(self.base_offset, l)
File "/share/src/volatility-svn/volatility/plugins/addrspaces/
intel.py", line 295, in read
return self.__read_bytes(vaddr, length, pad = False)
File "/share/src/volatility-svn/volatility/plugins/addrspaces/
intel.py", line 276, in __read_bytes
buf = self.__read_chunk(vaddr, chunk_len)
File "/share/src/volatility-svn/volatility/plugins/addrspaces/
intel.py", line 260, in __read_chunk
return self.base.read(paddr, length)
File "/share/src/volatility-svn/volatility/plugins/addrspaces/
pyvmiaddressspace.py", line 52, in read
return self.vmi.read_pa(addr, length)
ValueError: Unable to read memory at specified address

I assume volatility is trying to read an invalid memory location but
I'm not sure why that would happen.

[Bryan D. Payne]

I'm happy to see that you're using the Volatility / pyvmi / LibVMI
tool chain :-)

Could you provide a little more information about your setup? Is this
KVM or Xen? If KVM, are you connecting via GDB or via the qemu patch?
Also, it could be helpful to see the LibVMI debug trace output as
well (recompile with DEBUG uncommented in libvmi.h).

Cheers,
bryan

> --
> You received this message because you are subscribed to the Google Groups "vmitools" group.
> To post to this group, send email to vmit...@googlegroups.com.
> To unsubscribe from this group, send email to vmitools+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/vmitools?hl=en.
>

[T0MA]

Hi Bryan, thanks for the fast response. I'm using Xen 4.1.2 and the
vm
is running in HVM.
Feature      | Option             | Reason
-------------|--------------------|-----------------------
Xen Support  | --enable-xen=yes   | yes
KVM Support  | --enable-kvm=no    | missing libvirt
File Support | --enable-file=yes  | yes
#> cat /etc/libvmi.conf
honey-xp {
    ostype      = "Windows";
    win_tasks   = 0x88;
    win_pdbase  = 0x18;
    win_pid     = 0x84;
}




The output with the debug flag is:
#> python vol.py -l honey-xp --profile=WinXPSP3x86 pslist
LibVMI Version 0.6
--found Xen
LibVMI Mode 2
--got id from name (honey-xp --> 4)
**set image_type = honey-xp
**set hvm to true (HVM).
--completed driver init.
**set page_offset = 0x00000000
--MEMORY cache set 0x002e1000
--MEMORY cache set 0x002e2000
--MEMORY cache hit 0x002e2000
--MEMORY cache hit 0x002e2000
--MEMORY cache hit 0x002e1000
--MEMORY cache hit 0x002e2000
--MEMORY cache set 0x02c80000
--MEMORY cache hit 0x002e2000
--MEMORY cache hit 0x02c80000
--MEMORY cache hit 0x002e2000
--MEMORY cache hit 0x02c80000
--MEMORY cache set 0x002e5000
--MEMORY cache hit 0x002e5000
Offset(V) Name PID PPID Thds Hnds Time
---------- -------------------- ------ ------ ------ ------
-------------------
....
--MEMORY cache hit 0x002e4000
--MEMORY cache hit 0x002ee000
--MEMORY cache set 0x0009f000
--MEMORY cache hit 0x002e4000
--MEMORY cache hit 0x002ee000
--MEMORY cache set 0x000a0000

Traceback (most recent call last):
  File "vol.py", line 135, in <module>
    main()
  File "vol.py", line 126, in main
    command.execute()
  File "/share/src/volatility-svn/volatility/commands.py", line 101,
in execute
    func(outfd, data)

  File "/share/src/volatility-svn/volatility/plugins/taskmods.py",
line 122, in render_text
    for task in data:
  File "/share/src/volatility-svn/volatility/win32/tasks.py", line
70,
in pslist
    PsActiveProcessHead = get_kdbg(addr_space).PsActiveProcessHead

[Bryan D. Payne]

I'd like to get to the bottom of this. Unfortunately, I've been
unable to reproduce the problem on my system at this end. The debug
output you provided seems truncated. Is this actually all that was
printed to the console?

Also, I see that you posted an bug report on the Volatility Google
code page as well. There's a few comments in there about ways to
improve the libvmi addressspace. Did any of those help or provide
better context for the problem?

-bryan

On Mon, Feb 20, 2012 at 12:34 PM, T0MA <tamas.k...@gmail.com> wrote:
> Hi Bryan, thanks for the fast response. I'm using Xen 4.1.2 and the vm
> is running in HVM.
>
> Feature      | Option             | Reason
> -------------|--------------------|-----------------------
> Xen Support  | --enable-xen=yes   | yes
> KVM Support  | --enable-kvm=no    | missing libvirt
> File Support | --enable-file=yes  | yes
>
> #> cat /etc/libvmi.conf
> honey-xp {
>    ostype      = "Windows";
>    win_tasks   = 0x88;
>    win_pdbase  = 0x18;
>    win_pid     = 0x84;
> }
>
> The output with the debug flag is:

> #> python vol.py -l honey-xp --profile=WinXPSP3x86 pslist

> ....
> --MEMORY cache hit 0x002e4000
> --MEMORY cache hit 0x002ee000
> --MEMORY cache set 0x0009f000
> --MEMORY cache hit 0x002e4000
> --MEMORY cache hit 0x002ee000
> --MEMORY cache set 0x000a0000

> Traceback (most recent call last):
>  File "vol.py", line 135, in <module>
>    main()
>  File "vol.py", line 126, in main
>    command.execute()
>  File "/share/src/volatility-svn/volatility/commands.py", line 101,
> in execute
>    func(outfd, data)

>  File "/share/src/volatility-svn/volatility/plugins/taskmods.py",
> line 122, in render_text
>    for task in data:
>  File "/share/src/volatility-svn/volatility/win32/tasks.py", line 70,
> in pslist
>    PsActiveProcessHead = get_kdbg(addr_space).PsActiveProcessHead

> On Feb 20, 10:01 am, "Bryan D. Payne" <br...@thepaynes.cc> wrote:

[Tamas K Lengyel]

Hi Bryan,
the suggestions made on the volatility forum were only cosmetic, noone has used the libvmi address space before.

I did truncate the output a bit as it had a really long list of cache hit/set stream. If you need that, I can post the entire log but other then that, that's all that was printed.

Tamas

[Bryan D. Payne]

> noone has
> used the libvmi address space before.

This isn't quite true. I use it here without problems :-)

> I did truncate the output a bit as it had a really long list of cache
> hit/set stream. If you need that, I can post the entire log but other then
> that, that's all that was printed.

I'm surprised that there wasn't more info displayed at the init stage
for LibVMI. I don't need all of the cache information. Either way,
no worries.

I did notice that you're using Volatility 2.1_alpha. I built this
using the 2.0 release. I haven't been tracking the latest changes in
Volatility. Is it possible that something changed in there that would
be causing this bug?

-bryan

[Tamas K Lengyel]

Hi Bryan,
I ment noone has used it from the folks who replied there ;)

My first thought too was reverting so I tried it with 2.0 as well, but crashed the same way unfortunatelly (i can include the output if needed, I saw no difference). Let me know if there is anything additional I could send you that might aid this bug-hunt.

Tamas

[Bryan D. Payne]

Ok, I'm getting closer. The Volatility integration seems to work with
KVM, but not with Xen right now. See the attached error log. Does
this look like what you are seeing? Specifically the
xen_get_memory_mfn failure right before the python Traceback?

-bryan

[Tamas Lengyel]

Hi Bryan,

I haven't see the xen_get_memory_mfn failure in my output. I applied the modification to libvmi/driver/memory_cache.c mentioned in https://groups.google.com/group/vmitools/browse_thread/thread/75e1b2a5581349d8, not sure if that's related or if that would change my output from what you are seeing.

Tamas

-bryan

[Bryan D. Payne]

I have fixed the problem on my machine. Let me know if this updated
address space fixes it for you.
-bryan

[Tamas K Lengyel]

Thanks Bryan, it works now! Great job, keep it up =)

> --
> You received this message because you are subscribed to the Google
> Groups "vmitools" group. To post to this group, send email to
> vmit...@googlegroups.com. To unsubscribe from this group, send email to
> vmitools+u...@googlegroups.com. For more options, visit this
> group at http://groups.google.com/group/vmitools?hl=en.
>

<Attachment>  pyvmiaddressspace.py

[T0MA]

Hi Bryan,
in case you are not following the volatility thread, they made some
fixes in the unstable svn related to LibVMI, but the "scan" plugins
still don't seem to work with this address space:

#> python vol.py -l vmi://honey-xp-sp2 imageinfo

Volatile Systems Volatility Framework 2.1_alpha

Determining profile based on KDBG search...

Suggested Profile(s) : WinXPSP3x86, WinXPSP2x86
(Instantiated with WinXPSP2x86)
AS Layer1 : JKIA32PagedMemoryPae (Kernel AS)
AS Layer2 : PyVmiAddressSpace (Unnamed AS)
PAE type : PAE
DTB : 0x2f3000
KDBG : 0x80544ce0
KPCR : 0xffdff000
KUSER_SHARED_DATA : 0xffdf0000
Image date and time : 2012-03-12 07:52:00 UTC+0000
Image local date and time : 2012-03-12 02:52:00 -0500
Number of Processors : 1
Image Type : Service Pack 2

#> python vol.py -l vmi://honey-xp-sp2 psscan

Volatile Systems Volatility Framework 2.1_alpha

Offset(P) Name PID PPID PDB Time
created Time exited

---------- ---------------- ------ ------ ----------

------------------------ ------------------------
LibVMI Version 0.6
--found KVM
LibVMI Mode 4
--got id from name (honey-xp-sp2 --> 5)
**set image_type = honey-xp-sp2
--libvirt version 9010
--qmp: virsh qemu-monitor-command honey-xp-sp2 '{"execute":
"pmemaccess", "arguments": {"path": "/tmp/vmiwVMZqg"}}'
--kvm: using custom patch for fast memory access

--completed driver init.
**set page_offset = 0x00000000

--qmp: virsh qemu-monitor-command honey-xp-sp2 '{"execute": "human-
monitor-command", "arguments": {"command-line": "info registers"}}'
LibVMI Version 0.6
--found KVM
LibVMI Mode 4
--got id from name (honey-xp-sp2 --> 5)
**set image_type = honey-xp-sp2
--libvirt version 9010
--qmp: virsh qemu-monitor-command honey-xp-sp2 '{"execute":
"pmemaccess", "arguments": {"path": "/tmp/vmiCEtBOi"}}'
--kvm: using custom patch for fast memory access

--completed driver init.
**set page_offset = 0x00000000

--qmp: virsh qemu-monitor-command honey-xp-sp2 '{"execute": "human-
monitor-command", "arguments": {"command-line": "info registers"}}'
--MEMORY cache set 0x002f3000
--MEMORY cache set 0x002f4000
--MEMORY cache set 0x002f4000
--MEMORY cache set 0x002f4000
--MEMORY cache hit 0x002f3000
--MEMORY cache hit 0x002f4000
--MEMORY cache set 0x017c0000
--MEMORY cache hit 0x002f4000
--MEMORY cache set 0x017c0000
--MEMORY cache hit 0x002f4000
--MEMORY cache set 0x017c0000
--MEMORY cache set 0x002f7000
--MEMORY cache hit 0x002f7000
libvir: Domain error : invalid domain pointer in virDomainFree
libvir: error : invalid connection pointer in virConnectClose

[Bryan D. Payne]

Thanks. I am following that thread. I'll explore it a bit this morning.
-bryan