UTM size

[Mriganka Chakravarty]

Hello,

I am trying to increase the size of the untrusted memory(around a GB). Could you please help me.

I tried increasing it here, but I end up getting a segment fault.

Thank you.

[Paul Heath]

Just had this discussion last week with Dayeol.

There is about a two meg limit on that space due to the allocation method used. I had to break stuff into chunks to get all I needed into the enclave.

--
You received this message because you are subscribed to the Google Groups "Keystone Enclave Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keystone-enclave-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keystone-enclave-forum/770e7869-37fd-4052-800d-b50a31b74a7cn%40googlegroups.com.

[Dayeol Lee]

Thanks Paul!

Yes, UTM allocation is relying on the buddy allocator, so 2MB is possibly the largest you can expect.

We may need some better error handling on the library side such that the kernel does not print out error messages when it fails to allocate.

Thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/keystone-enclave-forum/CAJ3fi33D6ewkVmYq2WHEuj06XJMWW5ef4VDBzP44%2B-%2BH2vx2eQ%40mail.gmail.com.

[Mriganka Chakravarty]

Is there any workaround to this limitation? I want to get around ocalls for a faster transfer :(

[Dayeol Lee]

To hack this around, you can modify the driver to use CMA for the UTM as well. 
Use `dma_alloc_coherent` (as in keystone-page.c:52) instead of `__get_free_pages` (as in keystone-page.c:101)

Note that even with that, your total allocation cannot exceed the total CMA reservation (see Linux config. probably it's about 1GB in our default config).

Hope that helps!

Dayeol

To view this discussion on the web visit https://groups.google.com/d/msgid/keystone-enclave-forum/e1bcd539-ba61-49cd-b0d9-327836d96d61n%40googlegroups.com.

[Dayeol Lee]

Sorry, I missed how UTM is handled in the security monitor.

The method I described won't work because the UTM should be always be naturally aligned to the power of two.

This is because we are handling UTM by using only the last PMP register (NAPOT encoding).

Because of that, I think having a larger UTM is not as trivial as I described.

How critical is it to have a large UTM for your transfer performance?

I think context switch is pretty cheap so switching every 2MB sounds not much of a problem to me.

[Mriganka Chakravarty]

I tried CMA. With CMA i am getting around 64 MB, after which the seg fault comes in. Its not mission critical, I am a student and kinda on an exploratory path. :)

[Dayeol Lee]

I'd just stick to 2MB buffer, if it's not critical.

Anything larger than 2MB allocated by CMA could be unaligned, which may cause an undefined behavior (or probably fail).

[rajesh sarvapalli]

Hey, I am facing the same situation.  dma_alloc_coherent returns me allocated memory, but  i am getting segmentation fault here: https://github.com/keystone-enclave/keystone-sdk/blob/5e4592cb1e50c731d6937ef5aba632c672ec2a6f/lib/host/src/keystone.cpp#L91

It looks like the page table runs out of pages, i guess.

Error Log: https://pastebin.com/6R9eWjH5 (I dont understand why am i getting segF at 0x0..0147)

Any help would be highly appreciated.

Thanks

[rajesh sarvapalli]

Hello friends. I am sorry to bug, but I really need to increase the UTM size for a research work that I am doing. Please help me if possible. I am very close to getting positive results, and am stuck at this for like a month.

Thank you.

[Dayeol Lee]

This seems to be really old SDK, so could you please try the latest version?

Also, what did you change? it's hard to tell by just looking at your error log.

To view this discussion on the web visit https://groups.google.com/d/msgid/keystone-enclave-forum/981b094b-af98-498f-bc10-7fda795cfbdcn%40googlegroups.com.

[rajesh sarvapalli]

Hi.

The exact problem is a seg-fault at this faulting line:

Control reaches the above line via  Keystone::init()----->loadUntrusted()----> allocPage()--->faulting line.

The fault occurs as __ept_walk_create() returns wrong pte_t* which basically contains the bad-address.

It occurs to me that it could be solved by increasing the space allocated to epm_free_list, but I might be very wrong.

If the solution to the problem comes at the cost of performance, then also its fine.

Changes I did are :-

In linux-keystone-driver/keystone-page.c

Instead of:

utm->ptr = (void*) __get_free_pages(GFP_HIGHUSER, order);

I used:

if (order <= MAX_ORDER)

    utm->ptr = (void*) __get_free_pages(GFP_HIGHUSER, order);

if (!utm->ptr) {

    printk(KERN_INFO "[driver] Buddy Allocator for UTM failed\n");

    #ifdef CONFIG_CMA

        utm->ptr = (void *) dma_alloc_coherent(keystone_dev.this_device,

                                                                    count << PAGE_SHIFT,

                                                                    &device_phys_addr,

                                                                    GFP_KERNEL | __GFP_DMA32);

        if(!device_phys_addr)

            return -ENOMEM;

    #endif

    if(!utm->ptr)

        return -ENOMEM;

}

printk(KERN_INFO "[driver] UTM INIT successful\n");      //I get this print, so I have memory

In test-runner.cpp

I have used:

params.setUntrustedSize(32*1024*(4096));

Please suggest possible solutions.

Thank you.

[rajesh sarvapalli]

Also, I did the same with the recent sdk, but got the same seg-fault again.

PS:  In the previous post, device_phys_addr is a local var, phys_addr_t device_phys_addr;

[Paul Heath]

I have tried to overcome the UTM size limit by breaking items up into chunks and sending them over one at a time, rebuilding 

things on the enclave side when the transfer process is completed. If this would be useful, I could repackage it a bit and 

share the code to anyone who could use it. My current code only needs matrices of doubles, but I will need to make changes 

to allow generic blobs of bytes. Let me know if this would be useful to anyone . 

Paul