Re: How to use libVMI

[S. Biedermann]

Hi Hoang,

here you can find a nice tutorial which explains how to find the correct values for windows VMs:

http://www.peterklemperer.com/blog/2012/09/24/vmitools-setup-instructions/

[Tamas Lengyel]

I think that tutorial is a bit overcomplicated with WinDbg for finding the Windows offsets. There are tools included in LibVMI for creating the configs.

Windows: https://github.com/bdpayne/libvmi/blob/master/tools/windows-offset-finder/README

As a side-note for the Windows offset finder, you can use VMIFS to mount the guest's memory as a file without having to run dump-memory.

Linux: https://github.com/bdpayne/libvmi/blob/master/tools/linux-offset-finder/README

--
You received this message because you are subscribed to the Google Groups "vmitools" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vmitools+u...@googlegroups.com.
To post to this group, send email to vmit...@googlegroups.com.
Visit this group at http://groups.google.com/group/vmitools?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[hoang dinh]

Thanks for your help,

I am trying to run ./dump-memory win7 win7.dd but it returns an error message:

VMI_ERROR: Address translation failure.

Failed to init LibVMI library.

What can I do about it?

Thanks

[hoang dinh]

Oh I forgot to mention that I am using KVM with LibVMI

[Bryan D. Payne]

It would be helpful to see the full debug output.  Also, are you running the command as root?

-bryan

[hoang dinh]

Yes, I run command as root.

How can I gain the debug report? 

Today, I just start the machine and run ./dump-memory win7-32bit and it said "segmentation fault" :-/

[Bryan D. Payne]

https://code.google.com/p/vmitools/wiki/LibVMIInstallation

See you the bottom for debug instructions.

-bryan

[hoang dinh]

Thanks Bryan, 

I have enable debug feature, 

# ./dump-memory

LibVMI Version 0.10

--found KVM

LibVMI Mode 4

VMI_ERROR: Must specifiy either id or name.

Failed to init LibVMI library.

(process:7655): GLib-CRITICAL **: g_hash_table_size: assertion `hash_table != NULL' failed

--MEMORY cache cleanup round complete (cache size = 0)

But when I use

./dump-memory win7-32bit-2

it just outputs 

"Segmentation fault"

Any idea?? 

[Bryan D. Payne]

This example code takes two arguments:

https://github.com/bdpayne/libvmi/blob/master/examples/dump-memory.c#L51-55

./dump-memory <vm name> <output file name>

Also, note that since you're using KVM that this will be a *very* slow process.  You can watch the output file size grow in another window to get a since for how long it will take.  In the end, that file should be the same size as the amount of RAM allocated to your VM.

-bryan

[hoang dinh]

Thanks Bryan. 

I made a big mistake there.

Currently ./dump-memory win7-32bit-2 win7.dd is running. It outputs a lot of information, are there any ways I can put all of the output into one single debug file? The terminal does not show everything of the output. 

I have to use KVM since Xen kernel does not work with my machine. The machine hangs while booting with xen kernel. /:)

[Bryan D. Payne]

I'm not sure what you mean.  The dump memory command *will* put all of the VM's memory into a single output file.

-bryan

[hoang dinh]

I meant how to put debug info into a file. sorry for confusing you.

[Bryan D. Payne]

http://stackoverflow.com/questions/637827/redirect-stderr-and-stdout-in-a-bash-script

[hoang dinh]

Thanks for the link, Bryan

Today I have successfully dumped the memory, it took a long time. 

Now I am trying this 

./getGUID winxp.dd |./downloadPDB.py | ./dumpPDB.py -o debugSymbols.txt

sh: msexpand: command not found mv: cannot stat `ntkrnlmp.pd': No such file or directory Traceback (most recent call last): File "./dumpPDB.py", line 194, in <module> main() File "./dumpPDB.py", line 186, in main dump_types(infile, opts.outfile) File "./dumpPDB.py", line 159, in dump_types pdb = pdbparse.parse(pdbFile) #call the parse function in __init__ of the pdbparse library File "/usr/lib/python2.6/site-packages/pdbparse/__init__.py", line 435, in parse f = open(filename, 'rb') IOError: [Errno 2] No such file or directory: 'Windows_7.pdb'

and getting stuck at finding a package named msexpand.

I am using CentOS 6.4 x86_64

[Tamas Lengyel]

Not sure about CentOS but on Debian it's available as

#> apt-cache search msexpand

mscompress - Microsoft "compress.exe/expand.exe" compatible (de)compressor

This is the website where you can get it manually if it's not in your repos: http://gnuwin32.sourceforge.net/packages/mscompress.htm

[hoang dinh]

Thanks Tamas,

I'll try to get it manually

[hoang dinh]

I got mscompress already, it require 32bit version of glibc, I installed it, now it can run without the error, however it raises another one

# ./getGUID win7.dd |./downloadPDB.py | ./dumpPDB.py -o debugSymbols.txt

ntkrnlmp.pd_: This is not a MS-compressed file

Traceback (most recent call last):

  File "./dumpPDB.py", line 194, in <module>

    main()

  File "./dumpPDB.py", line 186, in main

    dump_types(infile, opts.outfile)

  File "./dumpPDB.py", line 159, in dump_types

    pdb = pdbparse.parse(pdbFile) #call the parse function in __init__ of the pdbparse library

  File "/usr/lib/python2.6/site-packages/pdbparse/__init__.py", line 445, in parse

    raise ValueError("Unsupported file type")

ValueError: Unsupported file type

So what now? Does this imply that the mscompress I got does not work and it cannot use the ms-compressed file?

Thanks

[Tamas Lengyel]

Can you verify that the file ntkrnlmp.pd_ was downloaded successfully
(ie. it's not an empty file or such)?

If you can't get mscompress to work and the Windows VM you have is
just a regular Windows (not debug build or something funky like that),
you can just read out the offsets manually from the vtypes generated
with pdbparse, they are included for example in Volatility
(https://code.google.com/p/volatility/source/browse/#svn%2Ftrunk%2Fvolatility%2Fplugins%2Foverlays%2Fwindows).

[hoang dinh]

My VM runs a regular windows version. So it requires a debugging version? I have to install debugging tools for the VM, right?

[Bryan D. Payne]

No, you do not need a debug build of windows.  Tamas was simply saying that as long as you *didn't* have a debug build, then you could most likely find the offsets you need by referring to the Volatility project.

-bryan

[hoang dinh]

Thanks Bryan, 

For ease, I followed Bierdermann's site above to take the win_tasks, win_pdbase and win_pid. I ran ./process-list and it asked for win_pname. I did a search and I found that it was patched. What can I do about this? 

Currently I am using libvmi 0.10.1

Thanks

[Bryan D. Payne]

I'm not sure what you mean by "I did a search and I found that it was patched".  This value is usually found dynamically by libvmi. Perhaps that's not working for some reason?  Could you provide a debug trace?

-bryan

[hoang dinh]

LibVMI Version 0.10.1

--found Xen

LibVMI Mode 2

--got id from name (win7-32bit --> 4)

**set image_type = win7-32bit

**set hvm to true (HVM).

--completed driver init.

--looking for config file at /home/pokemon/etc/libvmi.conf

--looking for config file at /root/etc/libvmi.conf

--looking for config file at /etc/libvmi.conf

**Using config file at /etc/libvmi.conf

     1 |win8-32bit {

     2 | ostype="Windows";

     3 | win_tasks=0x0b8;

     4 | win_pdbase=0x018;

     5 | win_pid=0x0b4;

     6 |}

     7 |

     8 |win7-32bit {

     9 | ostype="Windows";

    10 | win_tasks=0x0b8;

    11 | win_pdbase=0x018;

    12 | win_pid=0x0b4;

    13 |}

--got sysmap from config ().

--reading in windows offsets from config file.

--got ostype from config (Windows).

**set os_type to Windows.

**set page_offset = 0x80000000

**set size = 1077907456 [0x403f9000]

**set pae = 1

**set pse = 1

**set lme = 0

**PAE paging

**set cr3 = 0x0000000000000000

--windows symbol lookup (KernBase)

--MEMORY cache set 0x1000

--MEMORY cache set 0x2000

--MEMORY cache set 0x3000

--MEMORY cache set 0x4000

--MEMORY cache set 0x5000

--MEMORY cache set 0x6000

...

...

--MEMORY cache set 0x3ffff000

--MEMORY cache set 0x40000000

--xen_get_memory_pfn failed on pfn=0x40000

--MEMORY cache set 0x40001000

--xen_get_memory_pfn failed on pfn=0x40001

--MEMORY cache set 0x40101000

--xen_get_memory_pfn failed on pfn=0x40101

--MEMORY cache set 0x40201000

--xen_get_memory_pfn failed on pfn=0x40201

--MEMORY cache set 0x40301000

--xen_get_memory_pfn failed on pfn=0x40301

--failed to find pname_offset

Failed to find win_pname

--MEMORY cache cleanup round complete (cache size = 0)

On Thu, Jun 6, 2013 at 11:42 PM, hoang dinh <inv...@gmail.com> wrote:

I attached the output here.

Btw, I tried Volatility, #python vol.py -l vmi://win7-32bit pslist 
it run but showed no process list.

Thanks

[Bryan D. Payne]

What version of win7 are you using?  Is it an international language edition?

-bryan

[hoang dinh]

It's Windows 7 home Premium. I am not sure about the international language edition.

[Bryan D. Payne]

If it is possible to provide a full memory dump, I could help debug this issue.

-bryan

[hoang dinh]

Bryan, a full memory dump is the full debug output from my previous post or a *.dd file made by ./memory-dump? Sorry for asking this

[Bryan D. Payne]

The dd file made by memory-dump.  If you can post it on a web server somewhere, I can download it and see what is happening.  Best to compress it first (dd.gz).

-bryan

[hoang dinh]

Ok here is my dd file, I uploaded it to my Googld Drive

https://docs.google.com/file/d/0B_TlCtOBZy7udlozZmV6Q1BJT1E/edit?usp=sharing

Thanks a lot, Bryan

[Bryan D. Payne]

This doesn't appear to be a raw memory dump.  How did you create this image?

-bryan

[hoang dinh]

I compressed it, # tar -zcvf win7.dd.gz win7.dd

I created win7.dd using memory-dump. 

[Bryan D. Payne]

Can you try increasing the ram size of this vm?

-bryan

[hoang dinh]

I increased the size to 1.5G, the same problem as previous. Here is the link for the new dd file

https://docs.google.com/file/d/0B_TlCtOBZy7uYVRhUGwyaktfR3c/edit?usp=sharing

Thanks Bryan

[Bryan D. Payne]

Just a quick update.  I've found one issue related to the magic number libvmi uses when scanning for processes in memory.  However, fixing this did not completely resolve the problem.  I will continue to explore.

I did note that Volatility also has problems with this image.

-bryan

[hoang dinh]

Thank you for taking care of it!

For Volatility and LibVMI, it works.
I tried:python vol.py -l vmi://win7-32bit –profile=Win7SP1x86 and it worked.

Visit this group at http://groups.google.com/group/vmitools.

[Bryan D. Payne]

Interesting.  Volatility isn't working for me.  Perhaps there's some corruption in the images you sent me?  Note that I'm using Volatility 2.2.  What version did you use?

python vol.py pslist -f win7-15k.dd –profile=Win7SP1x86

Volatile Systems Volatility Framework 2.2

No suitable address space mapping found

Tried to open image as:

 LimeAddressSpace: lime: need base

 WindowsHiberFileSpace32: No base Address Space

 WindowsCrashDumpSpace64: No base Address Space

 WindowsCrashDumpSpace32: No base Address Space

 AMD64PagedMemory: No base Address Space

 JKIA32PagedMemory: No base Address Space

 IA32PagedMemoryPae: Module disabled

 JKIA32PagedMemoryPae: No base Address Space

 IA32PagedMemory: Module disabled

 LimeAddressSpace: Invalid Lime header signature

 WindowsHiberFileSpace32: No xpress signature found

 WindowsCrashDumpSpace64: Header signature invalid

 WindowsCrashDumpSpace32: Header signature invalid

 AMD64PagedMemory: Incompatible profile WinXPSP2x86 selected

 JKIA32PagedMemory: No valid DTB found

 IA32PagedMemoryPae: Module disabled

 JKIA32PagedMemoryPae: No valid DTB found

 IA32PagedMemory: Module disabled

 FileAddressSpace: Must be first Address Space

-bryan

[Tamas Lengyel]

Bryan, the FileAddressSpace is complaining that it "Must be first
Address Space". Just remove some of the other address spaces from
Volatility that you don't use, that usually fixes it.

[Bryan D. Payne]

python vol.py pslist -f ~/EVERYTHING/MemoryImages/win7-15k.dd –profile=Win7SP1x86

Volatile Systems Volatility Framework 2.2

No suitable address space mapping found

Tried to open image as:

 FileAddressSpace: Must be first Address Space

[Tamas Lengyel]

I tried with a semi-recent SVN version of Volatility, it seems to be
working fine:

vol.py -f ./win7-512.dd --profile=Win7SP1x86 imageinfo
Volatile Systems Volatility Framework 2.3_beta
Determining profile based on KDBG search...

Suggested Profile(s) : Win7SP0x86, Win7SP1x86
AS Layer1 : IA32PagedMemoryPae (Kernel AS)
AS Layer2 : FileAddressSpace
(/home/ssjtoma/Downloads/win7-512.dd)
PAE type : PAE
DTB : 0x185000L
KDBG : 0x8272bbe8
Number of Processors : 1
Image Type (Service Pack) : 0
KPCR for CPU 0 : 0x8272cc00
KUSER_SHARED_DATA : 0xffdf0000
Image date and time : 2013-06-11 03:08:28 UTC+0000
Image local date and time : 2013-06-10 20:08:28 -0700

vol.py -f ./win7-512.dd --profile=Win7SP1x86 pslist
Volatile Systems Volatility Framework 2.3_beta
Offset(V) Name PID PPID Thds Hnds Sess
Wow64 Start Exit
---------- -------------------- ------ ------ ------ -------- ------
------ ------------------------------ ------------------------------
0x839b0ad0 System 4 0 77 409 ------
0 2013-06-11 03:08:10 UTC+0000
0x84505640 smss.exe 248 4 4 29 ------
0 2013-06-11 03:08:10 UTC+0000
0x84a76530 csrss.exe 316 308 8 299 0
0 2013-06-11 03:08:12 UTC+0000
0x8450dd40 wininit.exe 352 308 8 86 0
0 2013-06-11 03:08:12 UTC+0000
0x84904d40 csrss.exe 364 344 7 82 1
0 2013-06-11 03:08:12 UTC+0000
0x84b13d40 winlogon.exe 396 344 6 99 1
0 2013-06-11 03:08:12 UTC+0000
0x84b41030 services.exe 468 352 20 200 0
0 2013-06-11 03:08:13 UTC+0000
0x84b46ac8 lsass.exe 476 352 9 419 0
0 2013-06-11 03:08:13 UTC+0000
0x84b454a8 lsm.exe 484 352 11 143 0
0 2013-06-11 03:08:13 UTC+0000
0x84b8f030 svchost.exe 592 468 16 340 0
0 2013-06-11 03:08:14 UTC+0000
0x84baba48 svchost.exe 668 468 9 181 0
0 2013-06-11 03:08:14 UTC+0000
0x84bced40 LogonUI.exe 744 396 10 193 1
0 2013-06-11 03:08:14 UTC+0000
0x84bd9d40 svchost.exe 756 468 18 341 0
0 2013-06-11 03:08:14 UTC+0000
0x84c12718 svchost.exe 836 468 12 181 0
0 2013-06-11 03:08:15 UTC+0000
0x84a44530 svchost.exe 864 468 48 730 0
0 2013-06-11 03:08:15 UTC+0000
0x84c435d0 svchost.exe 968 468 15 256 0
0 2013-06-11 03:08:16 UTC+0000
0x84c7dbd0 svchost.exe 1060 468 21 359 0
0 2013-06-11 03:08:17 UTC+0000
0x84a41030 spoolsv.exe 1156 468 6 77 0
0 2013-06-11 03:08:18 UTC+0000
0x844f1030 svchost.exe 1192 468 22 320 0
0 2013-06-11 03:08:18 UTC+0000
0x84a3fbd0 svchost.exe 1276 468 15 172 0
0 2013-06-11 03:08:18 UTC+0000

[Tamas Lengyel]

Volatility 2.2 doesn't work though, I get the same error as you did Bryan.

On Mon, Jun 17, 2013 at 8:45 PM, Tamas Lengyel

[Bryan D. Payne]

Interesting.  Perhaps the answer we are looking for lies somewhere in the diff between those two versions.

-bryan

[Tamas Lengyel]

Hard to tell, would be nice to check with Volatility 2.2 through
PyVmiAddressSpace to see if that works or not. It might be just
Volatility 2.2's address space initialization that's broken (which has
been funky). Hoang, can you try that?

[hoang dinh]

Hum, I dont know why but Volatility 2.2 works for both of my images, win7-512.dd and win7-15k.dd, no problems at all

python vol.py -f win7-15k.dd --profile=Win7SP1x86 pslist

Volatile Systems Volatility Framework 2.2

Offset(V)  Name                    PID   PPID   Thds     Hnds   Sess  Wow64 Start                Exit                

---------- -------------------- ------ ------ ------ -------- ------ ------ -------------------- --------------------

0x8410a020 System                    4      0     79      405 ------      0 2013-06-13 00:09:10                      

0x85024020 smss.exe                248      4      4       29 ------      0 2013-06-13 00:09:10                      

0x85178d40 csrss.exe               320    312      8      295      0      0 2013-06-13 00:09:13                      

0x84bf2030 wininit.exe             356    312      8       89      0      0 2013-06-13 00:09:14                      

0x8517cc38 csrss.exe               368    348      7       81      1      0 2013-06-13 00:09:14                      

0x85239338 winlogon.exe            420    348      6       99      1      0 2013-06-13 00:09:14                      

0x8524f030 services.exe            472    356     22      202      0      0 2013-06-13 00:09:14                      

0x85269498 lsass.exe               480    356      9      420      0      0 2013-06-13 00:09:15                      

0x852571c0 lsm.exe                 488    356     11      142      0      0 2013-06-13 00:09:15                      

0x852ab030 svchost.exe             588    472     16      340      0      0 2013-06-13 00:09:16                      

0x852c8b18 svchost.exe             668    472     10      183      0      0 2013-06-13 00:09:16                      

0x852e8908 LogonUI.exe             740    420      9      186      1      0 2013-06-13 00:09:16                      

0x852f3d40 svchost.exe             752    472     18      341      0      0 2013-06-13 00:09:16                      

0x85328240 svchost.exe             852    472     12      181      0      0 2013-06-13 00:09:17                      

0x85362030 svchost.exe             884    472     45      727      0      0 2013-06-13 00:09:17                      

0x8535b030 svchost.exe             964    472     14      249      0      0 2013-06-13 00:09:17                      

0x853b1728 svchost.exe            1052    472     19      355      0      0 2013-06-13 00:09:19                      

0x8537daf0 spoolsv.exe            1152    472      6       77      0      0 2013-06-13 00:09:20                      

0x8544d3a8 svchost.exe            1188    472     22      314      0      0 2013-06-13 00:09:20                      

0x854aebd0 svchost.exe            1288    472     13      170      0      0 2013-06-13 00:09:20            

python vol.py -f win7-512.dd --profile=Win7SP1x86 pslist

Volatile Systems Volatility Framework 2.2

Offset(V)  Name                    PID   PPID   Thds     Hnds   Sess  Wow64 Start                Exit                

---------- -------------------- ------ ------ ------ -------- ------ ------ -------------------- --------------------

0x839b0ad0 System                    4      0     77      409 ------      0 2013-06-11 03:08:10                      

0x84505640 smss.exe                248      4      4       29 ------      0 2013-06-11 03:08:10                      

0x84a76530 csrss.exe               316    308      8      299      0      0 2013-06-11 03:08:12                      

0x8450dd40 wininit.exe             352    308      8       86      0      0 2013-06-11 03:08:12                      

0x84904d40 csrss.exe               364    344      7       82      1      0 2013-06-11 03:08:12                      

0x84b13d40 winlogon.exe            396    344      6       99      1      0 2013-06-11 03:08:12                      

0x84b41030 services.exe            468    352     20      200      0      0 2013-06-11 03:08:13                      

0x84b46ac8 lsass.exe               476    352      9      419      0      0 2013-06-11 03:08:13                      

0x84b454a8 lsm.exe                 484    352     11      143      0      0 2013-06-11 03:08:13                      

0x84b8f030 svchost.exe             592    468     16      340      0      0 2013-06-11 03:08:14                      

0x84baba48 svchost.exe             668    468      9      181      0      0 2013-06-11 03:08:14                      

0x84bced40 LogonUI.exe             744    396     10      193      1      0 2013-06-11 03:08:14                      

0x84bd9d40 svchost.exe             756    468     18      341      0      0 2013-06-11 03:08:14                      

0x84c12718 svchost.exe             836    468     12      181      0      0 2013-06-11 03:08:15                      

0x84a44530 svchost.exe             864    468     48      730      0      0 2013-06-11 03:08:15                      

0x84c435d0 svchost.exe             968    468     15      256      0      0 2013-06-11 03:08:16                      

0x84c7dbd0 svchost.exe            1060    468     21      359      0      0 2013-06-11 03:08:17                      

0x84a41030 spoolsv.exe            1156    468      6       77      0      0 2013-06-11 03:08:18                      

0x844f1030 svchost.exe            1192    468     22      320      0      0 2013-06-11 03:08:18                      

0x84a3fbd0 svchost.exe            1276    468     15      172      0      0 2013-06-11 03:08:18           

[hoang dinh]

Another question here, does libvmi pause the virtual machine when the process-list, for example, starts? and how about when I run libvmi with volatility? 

Thanks

[Tamas Lengyel]

The examples in libvmi do pause the VM as you can see at
https://github.com/bdpayne/libvmi/blob/master/examples/process-list.c#L91.
When you run with Volatility, it's up to you to make sure the VM is
paused.

[hoang dinh]

@Tamas: sorry but I dont get you at this point "When you run with Volatility, it's up to you to make sure the VM is paused." ?

When the VM is paused, we cannot use the VM (e.g open files, folders,.. in the VM), right? 

Another thing, I still cannot run process-list successfully, win_pname error.

[Tamas Lengyel]

That's correct. If you want to have a consistent view of the memory
you need to pause, otherwise you may encounter problems when things
change while you are reading it.

[hoang dinh]

Another thing here; I have read a bit about kernel symbol, we need it, for both linux and windows, to map the initial point of 'task_structs' then we can trace the rest of the 'task list', then we can get every information of the VM, right?
ps: Is it ok if I ask other questions (about libvmi and volatility) within this topic like I am doing? I'm afraid I would mess it up!