Accessing shm@0 memory from Linux on APU

[jim....@gmail.com]

I'd like to be able to code my application directly to Posix APIs to mmap shared memory reserved in the device tree. How is it done?

I know libmetal does it but I would like to bypass libmetal. I know that I should be able to reverse engineer how to do it by using the debugger to step through a libmetal example, but I am optimistic that the solution is simple enough that someone here will be generous enough to share it.

I expect that the core sequence of calls is:

1. shm_open()
2. ftruncate()
3. mmap
4. mlock()

Writing code that uses those calls to allocate virtual memory at the desired address is easy, but the virtual memory does not map to the desired physical memory.

In the call to shm_open, I believe I will need to pass the name of a pseudo-device, and that the pseudo-device will be related to my device tree entry:

shm0: shm@0 {
compatible = "shm_uio";
reg = <0x0 0x40B80000 0x0 0x80000>;
};

This results in a directory in the /sys/bus/... filesystem:

/sys/bus/platform/devices/40b80000.shm

But that path is clearly not the path that I need to pass to shm_open().

Can anyone clue me in?

Thanks

[jim....@gmail.com]

Answering my own question, the following seems to work fine for me:

1. fd = open("/dev/mem", O_RDWR | O_SYNC) # instead of shm_open
2. virtAddress = mmap(0, size, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_LOCKED, fd, 0x40B80000)
3. close(fd)
4. mlock(virtAddress, size);

Note that 0x40B80000 is the address that I have reserved in the device tree as node shm@0. size is set to 0x80000, matching the size in shm@0.

[Jiaying Liang]

[Wendy] You can check the system/linux/device.c, it binds /sys/bus/platform/devices/xxxxx with platform generic UIO driver, and then it does mmap. Please note that with this method, it will map the memory as device memory (register memory) but not normal memory.

Best Regards,
Wendy
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "open-amp" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to open-amp+u...@googlegroups.com.
> To post to this group, send an email to open...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Jiaying Liang]

> -----Original Message-----
> From: open...@googlegroups.com [mailto:open...@googlegroups.com]
> On Behalf Of jim....@gmail.com
> Sent: Tuesday, October 23, 2018 4:39 PM
> To: open-amp <open...@googlegroups.com>

> Subject: [open-amp] Re: Accessing shm@0 memory from Linux on APU
>
> Answering my own question, the following seems to work fine for me:
>
> 1. fd = open("/dev/mem", O_RDWR | O_SYNC) # instead of shm_open
> 2. virtAddress = mmap(0, size, PROT_READ | PROT_WRITE, MAP_SHARED |
> MAP_LOCKED, fd, 0x40B80000)
> 3. close(fd)

[Wendy] can you use the memory after the fd is closed?

> 4. mlock(virtAddress, size);
[Wendy] Do you need mlock() in this case?

[Jim Lloyd]

Following up on this after more experimentation.

The code that I posted does allow the APU to gain access to a range of physical memory. The memory can be any memory excluded from use by Linux on the APU, including OCM memory. Most of my testing has used OCM, but some of my testing was with DDR memory that was reserved but not assigned to any device driver.

Wendy asked if the virtual memory is accessible after the file descriptor `fd` is closed. It works for me, and The Single Unix Specification man page for mmap has this note:

> The mmap() function adds an extra reference to the file associated with the
> file descriptor fildes which is not removed by a subsequent close() on that
> file descriptor. This reference is removed when there are no more mappings
> to the file.

Wendy also asked if the `mlock()` is needed. It may not be needed, but I haven't yet tested with it removed. I'm not motivated to remove it since the code is working fine.

But there is one hugely important additional change that is required to make my original code work fully, which is to modify the MPU caching attributes for the range of memory. I am currently doing this on the RPU side only using the `Xil_SetMPURegion()` function, setting the region attributes to `STRONG_ORDERD_SHARED | PRIV_RW_USER_RW`. I see that libmetal uses `NORM_SHARED_NCACHE | PRIV_RW_USER_RW` which might be a better choice. I'll experiment with that setting later.

Once the MPU region attributes are set, I am able to use gcc atomic operations on shared memory with results that seem to be correct in my testing so far.

You received this message because you are subscribed to a topic in the Google Groups "open-amp" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/open-amp/xh8gWvfiKZY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to open-amp+u...@googlegroups.com.

[garfy123]

Hi, I am trying to do something similar to what you are doing, that is, I want to share big size of data between Linux app on CPU 0 and baremetal app on CPU 1. Due to some reasons, I have to stay at 2015.4 and i do want to upgrade.

I am using RPMSG to transfer data between both CPUs but for large data transfer, I want to do via shared memory. The RPMSG will be used to inform the other CPU the pointer address of the shared memory and the size of the data tranfer. At the Linux size, I could use what you mentioned, including the changes i need to add at the device tree table.

Could you elaborate on the baremetal, how do i access the shared memory?

Maybe, Wendy can help too?

Sorry, I am really a newbie in this and I have been trying to search on the internet and forum but i couldn’t get any details.

Would appreciate some responses.

Thank you so much!