Xen domU to domU inspection by libVMI

[Winai Wongthai]

Hi all,

In Xen, I would like to use domU, instead of dom0, to inspect another domU using libVMI. 

It will be great if someone did this and please let me know.

Best,

Winai

[Bryan D. Payne]

> In Xen, I would like to use domU, instead of dom0, to inspect another domU
> using libVMI.
> It will be great if someone did this and please let me know.

I've heard people talking about wanting to do this in the past, but I
haven't ever seen it in action. I do believe it should be possible.
It's just a matter of setting the domU's permissions properly. If you
figure out the details, perhaps you could do a quick writeup to go on
the wiki (http://code.google.com/p/vmitools/w/list)?

Cheers,
-bryan

[Winai Wongthai]

Thank you very much for you response, yes I also think that is about setting  domU's permissions. I will do it and let you know as soon as possible.

Best,

Winai

--
You received this message because you are subscribed to the Google Groups "vmitools" group.

To post to this group, send email to vmit...@googlegroups.com.
To unsubscribe from this group, send email to vmitools+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/vmitools?hl=en.

[Tawfiq Shah]

Hello Bryan and Winai

I was wonder were you able to use libvmi in domU to access another domU memory? I am currently working on a similar issue where i want Domu to have the ability to access the memory of another domu and perform analysis without the use of Dom0.

I have been looking at grant tables (grant tables enabling the sharing or transfer of memory frames between unprivileged domains in xen)                       http://xenbits.xen.org/docs/4.2-testing/misc/grant-tables.txt

if you have any tips on how i could do this using libvmi or any resource suggestions i could look into.

Thank you 

Tawfiq  

[Bryan D. Payne]

I was wonder were you able to use libvmi in domU to access another domU memory? I am currently working on a similar issue where i want Domu to have the ability to access the memory of another domu and perform analysis without the use of Dom0.

I know that this is possible, but I believe it requires some newish versions of Xen and Linux to make it happen.  I believe Tamas and/or Steve were working on this previously, so perhaps they can chime in with the details.

-bryan

[Tamas Lengyel]

Correct, it is possible to do domU to domU inspection, or even domU to dom0. You need at least Xen 4.3 with XSM enabled and a policy where such inspection is allowed. The domain where you are inspecting from has to be paravirt as HVM domains are missing critical hypercalls. Furthermore, the linux kernel in the domU you are running libvmi in has to be at least 3.8 (or apply my patch https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=30d4b180e20c081f435143f8bc211c66a930608a). Lastly, Xenstore has its own access control mechanism so you either have to enable your domain to be able to access xenstore, or simply ignore xenstore (its only used to store the name of the VM anyway). You can compile LibVMI with the flag --without-xenstore for this reason.

I recommend reading http://wiki.xen.org/wiki/Xen_Security_Modules_:_XSM-FLASK.

--

You received this message because you are subscribed to the Google Groups "vmitools" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vmitools+u...@googlegroups.com.

To post to this group, send email to vmit...@googlegroups.com.

Visit this group at http://groups.google.com/group/vmitools.
For more options, visit https://groups.google.com/d/optout.

[Tawfiq Shah]

Hi Bryan and Tamas, 

Thank you so much for the resources. I'm definitely looking into XSM. You have given me lots of insight and helpful advice. Once I've done the project, I will post my paper for the Wiki for future users who want to use LibVMI and DomU.

Thankyou

Tawfiq

Sent from my iPhone

You received this message because you are subscribed to a topic in the Google Groups "vmitools" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vmitools/gyc9kqSExaI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vmitools+u...@googlegroups.com.

[Tawfiq Shah]

Hey guys

I was wondering how can one find out which of xen hypercalls does libvmi use in-order to extract the memory dump? I have installed xsm and i am trying to create an allow policy through the security class that will give a specific domU  access to other guesOs memory?

Thank you for the help

[Tamas Lengyel]

Just boot Xen with flask_enforcing=0 and use LibVMI from the domU. xl dmesg will show you which hypercalls would have been denied by your policy.

[Tawfiq Shah]

Ahhh yes thats a great idea Thank you so much for the help

--
You received this message because you are subscribed to a topic in the Google Groups "vmitools" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vmitools/gyc9kqSExaI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vmitools+u...@googlegroups.com.

[Tawfiq Shah]

Hi,
I've been running into an error and I'm not sure what to do.  I've been running LibVMI in DomU and here is the result that I get (note: domU is a debian VM):

./examples/process-list domuname

xc: error: Could not obtain handle on privileged command interface (2 = No such file or directory) : internal error
VMI_ERROR: Could not find VMM or file to use.
VMI_ERROR: Opening a live VMM requires root access. (Note
: I am root when I run this command)
VMI_ERROR: Failed to identify correct mode.
Failed to init LibVMI library.

I first thought that this may be blocked due to LibVMI's hypercalls in domU being restricted but when I checked xm dmesg | tail this is what I got:
root@tssltpmdev1:/home/tawfiq# xm dmesg | tail
(XEN) CPU1: Temperature above threshold
(XEN) CPU1: Running in modulated clock mode
(XEN) CPU1: Temperature above threshold
(XEN) CPU1: Running in modulated clock mode
(XEN) CPU2: Temperature above threshold
(XEN) CPU2: Running in modulated clock mode
(XEN) CPU0: Temperature above threshold
(XEN) CPU0: Running in modulated clock mode
(XEN) CPU1: Temperature above threshold
(XEN) CPU1: Running in modulated clock mode

As you can see there is no message of a hypercall being denied so I'm not sure why the command is failing.

I also tired the volatility
python vol.py -l vmi://domUname --profile=LinuxUbuntu1004x86 linux_pslist

note:LinuxUbuntu1004x86 is a profile I created for the domU as the image I used didn't have a predefined image profile.

I keep getting:
No suitable address space mapping found
Tried to open image as:
LimeAddressSpace: lime: need base
WindowsHiberFileSpace32: No base Address Space
WindowsCrashDumpSpace64: No base Address Space
WindowsCrashDumpSpace32: No base Address Space
AMD64PagedMemory: No base Address Space
JKIA32PagedMemory: No Base Address Space
JKIA32PagedMemoryPae: No Base Address Space
IA32PagedMemoryPae: Module disabled
IA32PagedMemory: Module disabled
FileAddressSpace: Location is not of file scheme

If anyone has any suggestions or point me in the right direction. Please let me know. Thanks

On Friday, April 11, 2014 8:01:30 PM UTC-5, Tawfiq Shah wrote:

Ahhh yes thats a great idea Thank you so much for the help

On Fri, Apr 11, 2014 at 2:33 AM, Tamas Lengyel <tamas.k...@gmail.com> wrote:

Just boot Xen with flask_enforcing=0 and use LibVMI from the domU. xl dmesg will show you which hypercalls would have been denied by your policy.

On Thu, Apr 10, 2014 at 8:16 PM, Tawfiq Shah <tawfi...@gmail.com> wrote:

Hey guys

I was wondering how can one find out which of xen hypercalls does libvmi use in-order to extract the memory dump? I have installed xsm and i am trying to create an allow policy through the security class that will give a specific domU  access to other guesOs memory?

Thank you for the help

On Monday, March 31, 2014 1:42:37 AM UTC-7, Tamas K Lengyel wrote:

Correct, it is possible to do domU to domU inspection, or even domU to dom0. You need at least Xen 4.3 with XSM enabled and a policy where such inspection is allowed. The domain where you are inspecting from has to be paravirt as HVM domains are missing critical hypercalls. Furthermore, the linux kernel in the domU you are running libvmi in has to be at least 3.8 (or apply my patch https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=30d4b180e20c081f435143f8bc211c66a930608a). Lastly, Xenstore has its own access control mechanism so you either have to enable your domain to be able to access xenstore, or simply ignore xenstore (its only used to store the name of the VM anyway). You can compile LibVMI with the flag --without-xenstore for this reason.

I recommend reading http://wiki.xen.org/wiki/Xen_Security_Modules_:_XSM-FLASK.

On Mon, Mar 31, 2014 at 4:37 AM, Bryan D. Payne <br...@thepaynes.cc> wrote:

I was wonder were you able to use libvmi in domU to access another domU memory? I am currently working on a similar issue where i want Domu to have the ability to access the memory of another domu and perform analysis without the use of Dom0.

I know that this is possible, but I believe it requires some newish versions of Xen and Linux to make it happen.  I believe Tamas and/or Steve were working on this previously, so perhaps they can chime in with the details.

-bryan

--
You received this message because you are subscribed to the Google Groups "vmitools" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vmitools+u...@googlegroups.com.
To post to this group, send email to vmit...@googlegroups.com.

Visit this group at http://groups.google.com/group/vmitools.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "vmitools" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vmitools+unsubscribe@googlegroups.com.

To post to this group, send email to vmit...@googlegroups.com.
Visit this group at http://groups.google.com/group/vmitools.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "vmitools" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vmitools/gyc9kqSExaI/unsubscribe.

To unsubscribe from this group and all its topics, send an email to vmitools+unsubscribe@googlegroups.com.

[Tamas Lengyel]

That's probably because you haven't loaded the xen_privcmd kernel-module in your domU.

[Tawfiq Shah]

thank you xen_privcmd kernel-module was the issue

--

You received this message because you are subscribed to a topic in the Google Groups "vmitools" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vmitools/gyc9kqSExaI/unsubscribe.

To unsubscribe from this group and all its topics, send an email to vmitools+u...@googlegroups.com.

To post to this group, send email to vmit...@googlegroups.com.
Visit this group at http://groups.google.com/group/vmitools.
For more options, visit https://groups.google.com/d/optout.

[Lisa Marlene]

Hi All

I am new to LibVMI and I'm working on a project to perform virtual machine introspection using KVM.  I had a question, is it possible to have the same setup in XEN in KVM?  For example, in KVM can we have LibVMI in one virtual machine access another virtual machine's raw memory  to perform malware analysis and  Does KVM have anything similar to XEN's XSM MAC policy?

 

Thanks so much for you time.

[Lisa Marlene]

On Thursday, June 7, 2012 12:22:36 PM UTC-5, Winai Wongthai wrote:

[Tamas Lengyel]

Theoretically it should be possible but technically I'm not sure. There is SELinux for the KVM side (which like XSM, was also developed by the NSA), but I don't really know if it could be used in this fashion.

--

You received this message because you are subscribed to the Google Groups "vmitools" group.

[Winai Wongthai]

Yes, one of my colleague installed LibVMI on KVM and did some introspection.  

Best,

Winai

--
You received this message because you are subscribed to a topic in the Google Groups "vmitools" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vmitools/gyc9kqSExaI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vmitools+u...@googlegroups.com.

[Winai Wongthai]

it works. He used Fedora, but he did not document what he did sorry.

Best,

Winai

[Tamas Lengyel]

From what you are saying, 'installed LibVMI on KVM and did some introspection', I'm not sure if that's exactly what is under discussion here. Did he have two virtual machines and performed VMI from one to the other?

[Winai Wongthai]

Hi, I am sorry if i made the confusion.

From the questions:

'is it possible to have the same setup in XEN in KVM?  For example, in KVM can we have LibVMI in one virtual machine access another virtual machine's raw memory' 

and 

'did he have two virtual machines and performed VMI from one to the other?'

The answer to these questions is yes we can. 

My friend got Fedora+KVM+libVMI in his desktop. Then, he used a manager or monitoring VM (dom0 in Xen) to access (with the helps by libVMI) the main memory of a monitored VM (domU in Xen) to get some information he wanted.

Best,

Winai

[Tamas Lengyel]

My friend got Fedora+KVM+libVMI in his desktop. Then, he used a manager or monitoring VM (dom0 in Xen) to access (with the helps by libVMI) the main memory of a monitored VM (domU in Xen) to get some information he wanted.

That is still somewhat misleading and confusing. KVM is a type-2 hypervisor, aka. hosted hypervisor. So yes, with KVM normally you would do VMI from the host which has the KVM kernel module loaded and is running in VMX root mode into the domain running in VMX non-root mode. While that looks somewhat the same as doing it from dom0 on Xen, it actually is quite different. Dom0 on Xen is running in VMX non-root mode, thus it really is just a guest. On KVM, the host machine is _not_ a virtual machine. In the context of doing domU to domU introspection means doing introspection when both guests are running in VMX non-root mode. What you have described is not that from the looks of it.

[Winai Wongthai]

Hi, 

Thank you very much to clarify that. 

I am really sorry for the confusion.

Best, 

Winai

On Mon, Jun 2, 2014 at 4:50 PM, Tamas Lengyel <tamas.k...@gmail.com> wrote:

My friend got Fedora+KVM+libVMI in his desktop. Then, he used a manager or monitoring VM (dom0 in Xen) to access (with the helps by libVMI) the main memory of a monitored VM (domU in Xen) to get some information he wanted.

That is still somewhat misleading and confusing. KVM is a type-2 hypervisor, aka. hosted hypervisor. So yes, with KVM normally you would do VMI from the host which has the KVM kernel module loaded and is running in VMX root mode into the domain running in VMX non-root mode. While that looks somewhat the same as doing it from dom0 on Xen, it actually is quite different. Dom0 on Xen is running in VMX non-root mode, thus it really is just a guest. On KVM, the host machine is _not_ a virtual machine. In the context of doing domU to domU introspection means doing introspection when both guests are running in VMX non-root mode. What you have described is not that from the looks of it.

--

[Lisa VanHooser]

Thank you so much for the advice on the works of KVM.  I will use the XEN architecture and use XSM for domU to donU introspection.

[Tawfiq Shah]

Hi

I manged to create create an XSM policy that would allow domU to domU introspection but when i tired to test to see if LibVMI was able to perform memory analysis  it errors out.

Note: i have tired it in Dom0 to DomU it worked perfectly so I know that the issue is not with the volatility memory profile that i created.

The DomU doing the introspection is a pv and the other DomU that is infected is a HVM

This try is using libVMI with out volatility
./examples/process-list kbeastRootKit
This the result I got

VMI_ERROR: Could not find a live guest VM or file to use.
VMI_ERROR: Opening a live guest VM requires root access.

VMI_ERROR: Failed to identify correct mode.
Failed to init LibVMI library.

This is using libVMI + volatility
python vol.py   --profile=LinuxUbuntu1004Kbeastx86 -l vmi://kbeastRootKit linux_check_modules

Volatility Foundation Volatility Framework 2.3.1
Module Name
-----------
VMI_ERROR: Could not find a live guest VM or file to use.
VMI_ERROR: Opening a live guest VM requires root access.

VMI_ERROR: Failed to identify correct mode.

No suitable address space mapping found
Tried to open image as:

 MachOAddressSpace: mac: need base

 LimeAddressSpace: lime: need base
 WindowsHiberFileSpace32: No base Address Space
 WindowsCrashDumpSpace64: No base Address Space

 HPAKAddressSpace: No base Address Space
 VirtualBoxCoreDumpElf64: No base Address Space
 VMWareSnapshotFile: No base Address Space

 WindowsCrashDumpSpace32: No base Address Space
 AMD64PagedMemory: No base Address Space

 IA32PagedMemoryPae: No base Address Space
 IA32PagedMemory: No base Address Space
 PyVmiAddressSpace - EXCEPTION: Init failed

 FileAddressSpace: Location is not of file scheme

 ArmAddressSpace: No base Address Space

i checked the xl dmeg | audit2alloq -w -a in dom0 to check if there was a xsm policy issue this what i got

could not run ausearch - "[Errno 2] No such file or directory"

Can anyone please point me out to a possible solution, I would greatly appreciate it.

Thank you so much for the help and i look forward for the response .

On Thursday, June 7, 2012 12:22:36 PM UTC-5, Winai Wongthai wrote:

[Tamas Lengyel]

Does your PV domain have the required kernel modules loaded? You should have xen-privcmd loaded at least. Running 'xl list' in the pv control domain should work in that case. Also, if you want to access the domain by name you also need to set the xenstore permissions accordingly, which are separate from the XSM policy (at least for now).

--

You received this message because you are subscribed to the Google Groups "vmitools" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vmitools+u...@googlegroups.com.

[Tawfiq Shah]

Thank you Tamas for your quick response.

I thought privcmd was only for linux version prior to 3.8 my pv is 3.11.0-26-generic but yes i have not added this.

is it possible to exaplain to me how would i install the xen-privcmd in my pv??
I saw the link https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=30d4b180e20c081f435143f8bc211c66a930608a but i am not sure how to apply it.
 
i typed xl list on the pv and it gave me  this response:
The program 'xl' is currently not installed.  You can install it by typing:
sudo apt-get install xen-utils-common

So i added it a  xen-utils-common and i got the following result after typing xl list
Can't find default version of xen utils, bailing out!

which makes sense as the xen packages were not installed in the pv.

for xenstore  i think ill just use the domain id to reference it.

Thank you so much for your help

[Tawfiq Shah]

Hi guys

I have installed all necessary modules in the pv  listed below:
 
xen_wdt                13525  0
xen_kbdfront           12797  0
xen_netback            41491  0
netxen_nic            115074  0
xen_blkback            37861  0 [permanent]
xen_tpmfront           13578  0
xenfs                  12978  0
xen_privcmd            13286  1 xenfs
xen_gntalloc           13804  0
xen_gntdev             19066  0
xen_evtchn             13033  0
xen_fbfront            17552  0
fb_sys_fops            12703  1 xen_fbfront
sysimgblt              12806  1 xen_fbfront
sysfillrect            12901  1 xen_fbfront
syscopyarea            12633  1 xen_fbfront
xen_pcifront           18849  0

but when ever i try running libVMI i keep getting the same error:

VMI_ERROR: Could not find a live guest VM or file to use.
VMI_ERROR: Opening a live guest VM requires root access.
VMI_ERROR: Failed to identify correct mode.
Failed to init LibVMI library.

the HVM i am trying to introspect is running.

I also tired xl list in the pv domain but i got the following error:
Is xenstore daemon running?
failed to stat /var/run/xenstored.pid: No such file or directory
cannot init xl context

does anyone have any suggestions on how I could go about ??

Thank you so much for your time

[Steven Maresca]

The crux of the earlier message from Tamas is that your introspection domU is unable to resolve name to domain ID at the moment.

He alluded to xenstore being the root of this problem: unless you have taken steps to manually adjust xenstore privileges via dom0 so that the domU can read the data of another domU, you must use the domid only and modify libvmi code accordingly.  LibVMI doesn't care very much about the VM name; it only uses it to resolve the domid and config data anyway. As long as you provide domid and the needed configuration information at initialization time, you need not modify xenstore permissions. It is then only necessary to use XSM to allow the tagged domU to read memory of the target domU.

Steve

--

[Tamas Lengyel]

The error 'failed to stat /var/run/xenstored.pid: No such file or directory' can be simply fixed by running 'touch /var/run/xenstored.pid'. If xl list works afterwards, you should be set to use LibVMI as well, with the caveat that Steve described earlier.

[Tawfiq Shah]

Hi guys

Thank you so much for all the responses and suggestions. But unfortunately I am unable to have domU perform memory analysis of another domU.

I am currently going through the libVMI source code to trace back the error. So i can understand the errors better. 

So far I have found a comment regarding xms  that the type should have the permission getdomaininfo. I have this permission already set in xsm.

I still get an error using xl in the pv, I am planning to rebuild my pv domain to ensure all xen modules are part of the built-in kernel and follow through the libVMI source code.

Do you suggest using an old version of libVMI??? 

Thank you 

[Steven Maresca]

Tawfiq,

No, the current version of LibVMI is usable in this fashion. It is likely there is a missing hypercall in the XSM policy, xenstore permission issues (which you can still avoid by not using domain names), or something of the sort.

It's worth asking: what is the kernel version being used in the introspection vm where libvmi is installed ?

Steve

[Tawfiq Shah]

Hi Steve

Thank you for responding to my post.  

The pv kernel is 3.13.0-32-generic. But i did not build it from source so it came with the default modules.

privcmd is a module so on the config file is =m but not =y. Do you suggest i create the pv from source to verify

that libVMI is not working due to the kernel?

I will go through the XSM policy again, but from the the source code i noticed on the comment that the domU having libVMI

should have the getdomaininfo. I have added this and i have looked at the xl dmesg with the avc and used audit2allow.

but i will deeper into it.

I also have xl working in the pv domain  

Looking at the source code and tracing back from the three errors:

VMI_ERROR: Could not find a live guest VM or file to use.
VMI_ERROR: Opening a live guest VM requires root access.
VMI_ERROR: Failed to identify correct mode.

Failed to init LibVMI library. 

are related from the driver_init_mode() function . For some reason its failing the xen test and the count is equal resulting to the first two errors.

 I have not been able to figure out why this its not detecting xen.

And the Failed to identify correct mode from the set_driver_type().

I am not using xenstore but using the domain id unless i am typing it wrong

./examples/process-list 2

where 2 is the id of the HVM domain i am introspecting. the domain doing the introspection is a pv domain.

Thank you all for the advice 

--
You received this message because you are subscribed to a topic in the Google Groups "vmitools" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vmitools/gyc9kqSExaI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vmitools+u...@googlegroups.com.

[Tamas Lengyel]

Keep in mind that most of the examples treat the input as a domain name regardless if its a number or not. So in your case, when you run

./examples/process-list 2, LibVMI will assume your domain's name is '2' and try to find the domid for it. The only example right now that takes either a domid or a name is the win-guid example (read its source to see how the config is passed via a GHashTable).

[Tawfiq Shah]

Ooo yes that's a good point I was under the impression that the name was taken and would be used to resolve the the domid but if the domid was already supplied it would set the name to null.

I will create a windows HVM and test it with  win-guid example.  Then I will try to understand it's source  and see how it takes the domid as an argument and modify the dump-memory to work with the domainId when the name is not used.

I had a quick question when i use the combination of volatility and libvmi i still get the same error, does the volatility section work the same as win-guid where it can recognize the domain id or is it like the other example where it takes the 2  as the name and not id????? 

Note: when i did this in dom0 it worked but where the 2 is i used the vm name

This is using libVMI + volatility

python vol.py   --profile=LinuxUbuntu1004Kbeastx86 -l vmi://2 linux_check_modules 

Volatility Foundation Volatility Framework 2.3.1
Module Name
-----------

VMI_ERROR: Could not find a live guest VM or file to use.
VMI_ERROR: Opening a live guest VM requires root access.
VMI_ERROR: Failed to identify correct mode.

No suitable address space mapping found
Tried to open image as:
 MachOAddressSpace: mac: need base
 LimeAddressSpace: lime: need base
 WindowsHiberFileSpace32: No base Address Space
 WindowsCrashDumpSpace64: No base Address Space
 HPAKAddressSpace: No base Address Space
 VirtualBoxCoreDumpElf64: No base Address Space
 VMWareSnapshotFile: No base Address Space
 WindowsCrashDumpSpace32: No base Address Space
 AMD64PagedMemory: No base Address Space
 IA32PagedMemoryPae: No base Address Space
 IA32PagedMemory: No base Address Space
 PyVmiAddressSpace - EXCEPTION: Init failed
 FileAddressSpace: Location is not of file scheme
 ArmAddressSpace: No base Address Space

Thank you

[Tamas Lengyel]

Yes, you are correct. The simple volatility interface works with the assumption that what you give is the name. You can however supply it with a domainid as well if you set -l vmi://domid/<domid> (in your case it would be -l vmi://domid/2). Hope that helps. I acknowledge this is a somewhat (totally) undocumented feature of pyvmiaddressspace..

[Tawfiq Shah]

Hey

Thanx allot for the response, this is really good to know. I tired it but unfortunately it still does not work for me :(.

Since i have xl toolstack working in the pv domain, i will just extract the dump using xl dump-core. once i have the file ill create the /etc/libvmi.conf file for operating system offests.

if that does not work ill try to set xenstore permissions so I can use the domain name.

Thank youu All for the advice. 

[Tawfiq Shah]

Hi all

I managed to change the xenstore permission to allow my pv domain to access the name. It is a simple command :
xenstore-chmod -r /local/domain/2/name   n0 r1

but i still get the same same error:

VMI_ERROR: Could not find a live guest VM or file to use.
VMI_ERROR: Opening a live guest VM requires root access.
VMI_ERROR: Failed to identify correct mode.

checking domain nameSET_DRIVER_TYPE_ERROR

do i need to change any other the xenstore permission for libvmi??

Thanx

[Tamas Lengyel]

Does 'xl list' work properly now in your secondary domain? Does it show the guests running and their names?

--

You received this message because you are subscribed to the Google Groups "vmitools" group.

[Tawfiq Shah]

Hi  Tamas

Thank you for the response, I have attached a screen shot of xl list -Z result in my pv secondary domain and the change in xenstore permission. domain id 1 is the pv and 2 is the infected hvm.

i checked the xl dmesg | grep avc to see if xsm was the issue nothing is displayed. i used xl dmesg in both dom0 and my pv domain.

I have gone through the source code and I am not able to identity the exact issue.

Thank you so much for the help

 

--
You received this message because you are subscribed to a topic in the Google Groups "vmitools" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vmitools/gyc9kqSExaI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vmitools+u...@googlegroups.com.

[Tamas Lengyel]

Allright, it should work then with using the domain name in the secondary control domain. Can you post the entire debug output?

Tamas

[Tawfiq Shah]

This is the output I got when I enabled VMI_DEBUG (in the debug.h)

LibVMI Version 0.11.0

VMI_ERROR: Could not find a live guest VM or file to use.
VMI_ERROR: Opening a live guest VM requires root access.
VMI_ERROR: Failed to identify correct mode.

Failed to init LibVMI library.

[Tamas Lengyel]

That can't be all the debug output. Are you sure you recompiled the entire library and/or used the newly compiled binaries?

[Tawfiq Shah]

yes i did, and thats all i got. is theenabled VMI_DEBUG (in the debug.h)re something else i of should have done to output the debug? I just

enabled VMI_DEBUG (in the debug.h)

 Below is my compile output just in case if i mist anything
 
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking whether gcc and cc understand -c and -o together... yes
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking whether build environment is sane... yes
checking for xs_read in -lxenstore... yes
checking xenstore.h usability... yes
checking xenstore.h presence... yes
checking for xenstore.h... yes
checking xs.h usability... yes
checking xs.h presence... yes
checking for xs.h... yes
checking for xc_interface_open in -lxenctrl... yes
checking for vcpu_guest_context_any_t... yes
checking for grep... (cached) /bin/grep
checking for lsmod... /sbin/lsmod
checking for ceil in -lm... yes
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for FUSE... yes
checking for bison... bison
Found yacc as bison.
checking for bison... (cached) bison
checking for lex... lex
Found lex as lex.
checking for flex... (cached) lex
checking lex output file root... lex.yy
checking lex library... -lfl
checking whether yytext is a pointer... yes
checking for GLIB... yes
checking for CHECK... yes
checking for JANSSON... yes
configure: creating ./config.status
config.status: creating tools/vmifs/Makefile
config.status: creating Makefile
config.status: creating libvmi.pc
config.status: creating libvmi/Makefile
config.status: creating libvmi/config/Makefile
config.status: creating examples/Makefile
config.status: creating tests/Makefile
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
-------------------------------------------------------------------------------
LibVMI is configured as follows. Please verify that this configuration
matches your expectations.

Host system type: x86_64-unknown-linux-gnu
Build system type: x86_64-unknown-linux-gnu
Installation prefix: /usr/local

Feature      | Option                    | Reason
-------------|---------------------------|----------------------------
Xen Support  | --enable-xen=yes          | yes
Xen Events   | --enable-xen-events=no    | no
KVM Support  | --enable-kvm=no           | kernel module is not loaded
File Support | --enable-file=yes         | yes
Shm-snapshot | --enable-shm-snapshot=no  | no
-------------|---------------------------|----------------------------

Tools        | Option                    | Reason
-------------|---------------------------|----------------------------
VMIFS        | --enable-vmifs=yes              | yes

Extra features
----------------------------------------------------------------------
Support of Rekall profiles: yes

If everything is correct, you can now run 'make' and (optionally)
'make install'.  Otherwise, you can run './configure' again.

[Tamas Lengyel]

In that case the only thing I can recommend is to add extra debug printouts into libvmi/driver/xen.c in the functions xen_get_domainid_from_name and xen_check_domainid. You need to find out where things go wrong there, ie., which xc_* call fails (xc_interface_open, xc_domain_getinfo,...).

[Steven Maresca]

Running xl dmesg in Dom0 will also provide indication for any hypercalls which are being denied by the current XSM policy

Steve

[Tawfiq Shah]

Hey guys

Thank you all for the suggestions.

@Steve yes i ran xl dmesg | grep avc and  xl dmesg | audit2allow in both my secondary domain and dom0 no denials logged.

@Tamas thans on the hint, I am going to started adding more debugs on the suggested files. 

But before I stated on adding the debugs from the initial messages I got:

VMI_ERROR: Could not find a live guest VM or file to use.
VMI_ERROR: Opening a live guest VM requires root access.

This suggests that libvmi is not able to detect if xen or kvm or a file is used I am not sure as to why this message would print. 

in the secondary domain i type in xen-detect I get:

Running in HVM context on Xen v4.3.

while in dom0 xen-detect I get:

Running in PV context on Xen v4.3

Is this right ??? should of my secondary domain also output Running in PV context on Xen v4.3??? 

Thank you

[Tamas Lengyel]

Ha, there is your issue. Your secondary control domain _can't_ be an HVM guest. I think I pointed out in the very first message I sent to this thread. HVM guest only have a subset of the hypercalls available and that won't work for LibVMI.

[Tawfiq Shah]

Thank you Tamas for all the help yes you did mention that it had to be a PV domain. I followed the Debian pv setup to create the pv domain and when xl list -Z work in the domain I assumed it was a pv.  I did not know with XSM a HVM domain can gain access to hypervisor information.

By any chance do you know any good resources i could use to create a pv using xl?

Thank you for all the help   

 

[Tamas Lengyel]

Here is a config that I know worked:

bootloader = 'pygrub'

seclabel='tamas_u:system_r:zazen_t'

vcpus       = '2'

memory      = '512'

root        = '/dev/xvda1 ro'

disk        = [ 'file:/share/vms/live_images/zazen.img,xvda,w' ]

name        = 'zazen-pv'

vif         = [ 'ip=10.2.2.2,mac=00:16:3E:90:59:38,bridge=xenbr0' ]

on_poweroff = 'destroy'

on_reboot   = 'restart'

on_crash    = 'restart'

[fang...@gmail.com]

Is the problem of implanting LibVMI into domU solved now?

在 2014年8月27日星期三 UTC+8上午6:21:50，Tawfiq Shah写道：

[Tamas K Lengyel]

It has been working from pv domUs for a couple years now. You just need xsm enabled on xen.

[Big Strong]

Yes, I do reapper what Tawfiq Shah did. By configure flask policy and xenstore, I can run xl in pv domU to list other running guests, just as follows:

$ sudo xl list
Name                                        ID   Mem VCPUs      State   Time(s)
(null)                                       0  8188     1     -b----     232.1
ubuntu-hvm                                   1  2048     1     -b----      14.6
ubuntu-pv2                                   2  1024     1     r-----      17.8
win7                                         4  1024     1     ------    2369.2

But when I tried to use libvmi examples, I still got errors like:

$ process-list win7
VMI_ERROR: Could not find a live guest VM or file to use.

VMI_ERROR: Opening a live guest VM requires root access.
VMI_ERROR: Failed to identify correct mode.
Failed to init LibVMI library.

While in dom0, it works ok. So why xl can list guest with names, but libvmi cannot find guest with these names?

The only example that not show this error is win-guid as it can run with domid, the output message is:

$win-guid domid 4
VMI_ERROR: --requesting PA [0x1040c7000] beyond max physical address [0xff000000]
VMI_ERROR:      paddr: 1040c6000, length 1000, vmi->max_physical_address ff000000
VMI_ERROR: create_new_entry failed

While in dom0, the error is :

$win-guid domid 4
VMI_ERROR: --requesting PA [0x1040c7000] beyond memsize [0x40043000]
VMI_ERROR:      paddr: 1040c6000, length 1000, vmi->size 40043000
VMI_ERROR: create_new_entry failed
VMI_ERROR: --requesting PA [0x79734000] beyond memsize [0x40043000]
VMI_ERROR:      paddr: 79733000, length 1000, vmi->size 40043000
VMI_ERROR: create_new_entry failed
VMI_ERROR: --requesting PA [0x8ff77000] beyond memsize [0x40043000]
VMI_ERROR:      paddr: 8ff76000, length 1000, vmi->size 40043000
VMI_ERROR: create_new_entry failed
VMI_ERROR: --requesting PA [0xadc54000] beyond memsize [0x40043000]
VMI_ERROR:      paddr: adc53000, length 1000, vmi->size 40043000
VMI_ERROR: create_new_entry failed
VMI_ERROR: --requesting PA [0xc948d000] beyond memsize [0x40043000]
VMI_ERROR:      paddr: c948c000, length 1000, vmi->size 40043000
VMI_ERROR: create_new_entry failed

So in order to run libvmi in PV domU, how should I configure the Xenstore? Should I issue all guests' priviledges to the domU where libvmi runs?

BTW, "xl dmesg |grep avc |grep audit2allow" in dom0 shows :

$ sudo xl dmesg |grep avc |audit2allow

#============= dom0_t ==============
allow dom0_t domU_t:domain setvcpuextstate;
allow dom0_t domU_t:domain2 cacheflush;
allow dom0_t domU_t:mmu updatemp;
allow dom0_t self:event { bind create };
allow dom0_t unlabeled_t:domain { setdomainmaxmem pause getdomaininfo getscheduler max_vcpus getvcpuextstate setaffinity getvcpuinfo destroy getaffinity unpause getaddrsize };
allow dom0_t unlabeled_t:domain2 { setscheduler set_max_evtchn set_cpuid settsc setclaim cacheflush };
allow dom0_t unlabeled_t:event { status create send };
allow dom0_t unlabeled_t:grant { setup map_write unmap map_read };
allow dom0_t unlabeled_t:hvm { setparam hvmctl trackdirtyvram gethvmc nested sethvmc pciroute getparam pcilevel irqlevel cacheattr };
allow dom0_t unlabeled_t:mmu { map_write stat adjust physmap map_read };
allow dom0_t unlabeled_t:shadow enable;
#============= domU_t ==============
allow domU_t dom0_t:domain getdomaininfo;
allow domU_t domU_t_self:domain getdomaininfo;
allow domU_t security_t:security check_context;
allow domU_t unlabeled_t:domain { getdomaininfo pause destroy };
allow domU_t unlabeled_t:hvm { gethvmc getparam };
allow domU_t unlabeled_t:mmu { stat map_read };
allow domU_t xen_t:xen readconsole;
#============= unlabeled_t ==============
allow unlabeled_t dom0_t:event send;
allow unlabeled_t self:event bind;
allow unlabeled_t self:hvm { setparam getparam };
allow unlabeled_t self:mmu { adjust physmap };

While in the domU where libvmi runs:

$ sudo xl dmesg | grep avc | audit2allow

#============= dom0_t ==============
allow dom0_t domU_t:domain setvcpuextstate;
allow dom0_t domU_t:domain2 cacheflush;
allow dom0_t domU_t:mmu updatemp;
allow dom0_t self:event { bind create };
allow dom0_t unlabeled_t:domain { setdomainmaxmem pause getdomaininfo getscheduler max_vcpus getvcpuextstate setaffinity getvcpuinfo destroy getaffinity unpause getaddrsize };
allow dom0_t unlabeled_t:domain2 { setscheduler set_max_evtchn set_cpuid settsc setclaim cacheflush };
allow dom0_t unlabeled_t:event { status create send };
allow dom0_t unlabeled_t:grant { setup map_write unmap map_read };
allow dom0_t unlabeled_t:hvm { setparam hvmctl trackdirtyvram gethvmc nested sethvmc pciroute getparam pcilevel irqlevel cacheattr };
allow dom0_t unlabeled_t:mmu { map_write stat adjust physmap map_read };
allow dom0_t unlabeled_t:shadow enable;

#============= domU_t ==============
allow domU_t dom0_t:domain getdomaininfo;
allow domU_t domU_t_self:domain getdomaininfo;
allow domU_t security_t:security check_context;
allow domU_t unlabeled_t:domain { getdomaininfo pause destroy };
allow domU_t unlabeled_t:hvm { gethvmc getparam };
allow domU_t unlabeled_t:mmu { stat map_read };
allow domU_t xen_t:xen readconsole;

#============= unlabeled_t ==============
allow unlabeled_t dom0_t:event send;
allow unlabeled_t self:event bind;
allow unlabeled_t self:hvm { setparam getparam };
allow unlabeled_t self:mmu { adjust physmap };

They are the same. So should I add these rules to flask policy so as to run the libvmi examples?

As to other examples, should I test them all so as to obtain errors and update the rules?

在 2015年9月1日星期二 UTC+8下午9:34:55，Tamas K Lengyel写道：

[Tamas K Lengyel]

I normally don't bother with XenStore at all (which is probably the root cause of your issue). You can configure LibVMI in the domU to build without xenstore by adding the --without-xenstore option to the configure script.

If your flask policy is in permissive mode you don't need to really do anything - all hypercalls are allowed from all domains. While you are setting things up you don't need to worry about it.

[Big Strong]

Yes, I've also tried to complie libvmi without-xenstore, the win-guid example is working with domid, but not work with dom name. Besides, other examples which need dom names as arguments still fail for "VMI_ERROR: Could not find a live guest VM or file to use". And " xl dmesg | grep avc | audit2allow" shows no result. I've digged into the example code and found the main problem is at function vmi_init ($vmi, flags, name), as the funtion need dom names as argument. How was it possible to initialize vmi without the names? Then I looked into win-guid and found it use vmi_init_custom which use a hash table as argument instead of name. So I modified other examples, replaced vmi_init with vmi_init_custom and they still cannot work. So I believe the conversion between domname and domid in vmi_init must be something wrong. After keep looking into it, I found it is the get_id_from_name_ptr function which returns the VMI_INVALID_DOMID ERROR. And the reasons for this error is the following three ones:

1. HAVE_LIBXENSTORE is not met

2. the name is NULL

3. there is no corresponding domid to the name.

In the thrid one, the function traverse the /local/domain/ directory of Xenstore to get the id set and read /local/domain/*/name to get the domname. By comparing the domname with name to be conversed, we can get the corresponding domid. The premise of this operation is that the pv domU where libvmi works should have priviledges of xenstore-ls and xenstore-read to the directory or path. And that's not met in my implementation. So I just use  "xenstore-chmod -r /local/domain -r n0 r1" to authorize domain 1 access the xenstore of other domains. Then the examples all works! But the authorization may be too croase as there are many unneeded accesses under /local/domain, what we need is only /local/domain/*/name. Anyway. Xenstore is needed to run these examples. But if you can initialize vmi without domain names and you don't need access other domains, then you don't need that, but I think that would be meaningless.

在 2015年9月9日星期三 UTC+8上午12:10:25，Tamas K Lengyel写道：

...

[Tamas K Lengyel]

Yes, the examples will all fail that take the domain name if xenstore doesn't allow you to query to the domain names. The --without-xenstore imitative just allows LibVMI to work purely with domain IDs instead.

--

[Muhammad Asif]

I have followed this link but still facing some problem like           Failed to init LibVMI library.
What issue can be guys.

[Fanny Dwargee]

So nowdays with Xen 4.11 the steps needed for doing DomU to DomU inspection would be:

1. Install Xen 4.11 on the host machine, e.g. Debian 9
   
2. Install PVH DomU with Debian 9
3. Install Xen 4.11 on the PVH DomU? Maybe just copy the Xen modules from the host? Or what I need to be able to compile LibVMI?
4. Install LibVMI on the PVH DomU
5. Fix permissions via FLASK to access the Xenstore in the host from the PVH DomU (only if you want to access other DomU by name)
6. Fix permissions via FLASK for the PVH DomU being able to access the memory from another DomU

I don't see clearly how to setup the PVH DomU to access the Xenstore of the host instead of the local Xenstore... and the same goes for running 'xl list' inside the PVH DomU

Can I start/stop DomUs from the PVH DomU?

Many thanks in advance

[Fanny Dwargee]

I'm sorry, I was referring to PV instead of PVH

[Fanny Dwargee]

Anyone?