Write APIs are not working on KVM

[Sofia Belikovetsky]

Hi guys, 

I wrote a test program to try out the write APIs and they do not work even though I get VMI_SUCCESS after the execution. 

I have two issues: 

First, when I try to extract EIP and ESP (on a 32 machine) I always get the same values - which is weird.

This is the code I'm using: 

    reg_t esp, eip, cr3;

    status_t retVal = vmi_get_vcpureg(vmi, &esp, RSP, 0);

    printf("After esp %8lx %d\n", (unsigned long) esp, retVal);

    retVal = vmi_get_vcpureg(vmi, &eip, RIP, 0);

    printf("After eip %8lx %d\n", (unsigned long) eip, retVal);

    vmi_pid_t current_pid = -1;

    retVal = vmi_get_vcpureg(vmi, &cr3, CR3, 0);

    printf("After cr3 %8lx %d\n", (unsigned long) cr3, retVal);

    current_pid = vmi_dtb_to_pid(vmi, cr3);

Second, when I try to write to memory, it does not change the memory. 

Code Sample: 

uint32_t value; 

addr_t test_addr = esp - 4; 

status_t result = vmi_read_32_va(vmi, test_addr, current_pid, &value);

printf("Reading the value: %8x with result %d\n", value, (int) result);

value +=1; 

result = vmi_write_32_va(vmi, test_addr, current_pid, &value);

printf("Value that was written %8x with result %d\n", value, (int) result); 

value = 0; 

result = vmi_read_32_va(vmi, test_addr, current_pid, &value);

printf("Read the value again: %8x with result %d\n", value, (int) result);

The value of both reads is the same. 

What am I doing wrong? (I have initialized with VMI_INIT_COMPLETE). 

Is it because I'm trying to write to the stack? 

Here is the output without debug: 

Process listing for VM ubuntu32 (id=5)

Beginning of the execute process

After esp c1a75f70 0

After eip c10588f5 0

After cr3 33ba1280 0

Reading the value: f71b34c0 with result 0

l_��4

Value that was written f71b34c1 with result 0

Read the value again: f71b34c0 with result 0

finished vm4

(I'm not sure why I get the nonsense in the middle)

The full output with the debug prints is attached. 

Thank you!

Sofia

[Tamas K Lengyel]

Is the VM paused while you are doing the writing?

> --
> You received this message because you are subscribed to the Google Groups
> "vmitools" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to vmitools+u...@googlegroups.com.
> To post to this group, send email to vmit...@googlegroups.com.
> Visit this group at https://groups.google.com/group/vmitools.
> For more options, visit https://groups.google.com/d/optout.

[Sofia Belikovetsky]

Hi Tamas, 

Yes, of course. I'm using the initialization sequence from the examples: 

  /* initialize the libvmi library */

    if (vmi_init(&vmi, VMI_AUTO | VMI_INIT_COMPLETE, name) == VMI_FAILURE) {

        printf("Failed to init LibVMI library.\n");

        return 1;

    }

    /* pause the vm for consistent memory access */

    if (vmi_pause_vm(vmi) != VMI_SUCCESS) {

        printf("Failed to pause VM\n");

        goto error_exit;

    } // if

Thanks, 

Sofia 

[Tamas K Lengyel]

So this is a bug of the KVM driver. When you read from a memory page
using the current KVM driver the whole page is getting cached by
LibVMI. When you do the write, it probably does get written into
memory, but when you are reading the value again you are reading it
from the cached version, not directly from memory. You will have to do
a cacheflush after the write to see the value written.

In the log you see
--MEMORY cache set 0x1a75000
...
and then when you do the read again:
--MEMORY cache hit 0x1a75000

A quick fix for it is to do a memory_cache_remove of the page in kvm_put_memory

On Wed, Oct 4, 2017 at 10:08 AM, Sofia Belikovetsky

[Sofia Belikovetsky]

Hi Tamas, 

I wouldn't have thought of this. 

I will try it! 

Thank you so much!!!

Sofia 

[Tamas K Lengyel]

Give this branch a shot:
https://github.com/tklengyel/libvmi/tree/kvm_cache_flush

[Sofia Belikovetsky]

Thanks! I will

[Sofia Belikovetsky]

Hi Tamas, 

Finally tried it out.

First, I got an ERROR that the virtual address I'm trying to remove from the cache is not aligned. 

So just to check this on a 32bit machine, I've performed an "AND" with 0x0ffff000 (and saw that it worked, the page was removed from the cache). 

However, I still get the same result... the value after the write is not changed (the same as it was before). 

In the debug output, you can see that the read after the write doesn't get a cache hit, so the data is fresh. 

Is there a chance that it is not writing? 

I'm attaching the new output file. 

Thank you! 

Sofia 

[Sofia Belikovetsky]

Hi All, 

Just wanted to update that the write function did not work because I was using the slow GDB access. I've started using the patch and added the changes that Tamas has suggested and it worked. 

Thank you for your help! 

Sofia 

[Steven Maresca]

Excellent, thank you for reporting back with the success Sofia!

Regards,

Steve

To unsubscribe from this group and stop receiving emails from it, send an email to vmitools+unsubscribe@googlegroups.com.

[Tamas K Lengyel]

Good, the fix is already merged to master so you might want to do a pull.

Tamas

[hitchar...@gmail.com]

Hi Sofia,

i just meet the same problem as you .However ,i did not use the GDB access ,i use the QEMU 2.8 which has been patched .Could you tell me how you deal with this problem and what the changes that Tamas has suggested.

Thanks !

在 2017年11月13日星期一 UTC+8下午3:20:28，Sofia Belikovetsky写道：

[Sofia Belikovetsky]

Hi, 

Using the QEMU 2.8 is the right way :) 

The solution that Tamas has suggested and added to the main branch is adding: 

memory_cache_remove(vmi, paddr);

to the function kvm_put_memory in kvm.c. Then, it will flush the cache and you'll see your change.

However, this change has not worked for me, I've added the following line instead:

memory_cache_remove(vmi, (phys_address >> vmi->page_shift) << vmi->page_shift);

because the cache stores pages and you need to find the right page to remove. 

I hope i'm not mistaken...

Hope it will help you! 

Sofia