OpenAMP and reserved DDR

[snspa...@gmail.com]

Hello,

I am implementing a design on ZC702 EK with HDMI output based on Analog Devices design. I use openAMP for running linux on CPU0 and baremetal on CPU1. I have tested all the demo applications and everything works fine.

However I am trying to implement an image processing algorithm and I need a shared DDR region for passing image data. I am using reseved-memory nodes on my device tree and the I mmap() the area through /dev/mem, but my application sometimes crushes the system and I am forced to reboot. Is there something wrong on how I implement this or this way cannot work for data passing between processors at all?

This is my tree nodes that i add on analog devices device tree:

reserved-memory{
#address-cells = <0x1>
#size-cells = <0x1>
ranges;

sharedmem@0x3FE00000{
reg = <0x3fe00000 0x200000>
};
};

And this is how I open the memory area in my program:

#define DDR_SIZE (1024*1024*2)
#define DDR_BASE (0x3FE00000)

int mem_file;
void* ddr=NULL;

mem_file = open("/dev/mem", O_RDWR);

ddr = mmap(NULL, DDR_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, mem_file,DDR_BASE);

Thanks, Nikos.

[wendy...@xilinx.com]

Hi Nikos,

How you open /dev/mem and mmap looks fine.

However, as you mentioned your system crashes the system sometimes. Could you check the following:

* is your application + shared memory are covered in the reserved memory?

* do you use fixed shared memory or shared memory?

* from your mmap, you use the whole reserved memory as read and write. does it mean your application will also try to change the CPU1 executable memory? if no, you can split your reserved memory, the one for the firmware can be mapped as READ only if you want to access it for debugging.

* what's your resource table settings?

Best Regards,

Wendy

[snspa...@gmail.com]

Hello Wendy and thanks for the answer.
Maybe i wasn't clear enough, my remote application runs on the standard area for remoteproc, that is the first 256Mi. Here is the device tree entry:

remoteproc@0{
compatible = "xlnx,zynq_remoteproc";
reg = <0x00000000 0x10000000>;
firmware = "firmware";
vring0 = <15>;
vring1 = <14>;
};

To your questions:

* is your application + shared memory are covered in the reserved memory? *

If I understand correctly you mean if my application runs from the reserved memory? If so, then no, my CPU1 application runs at the first MBs of RAM and uses a pointer to 0x3FE00000 to read and write data.

* do you use fixed shared memory or shared memory? *

I am not sure what you mean here. You are talking about the mmap() parameter?

* what's your resource table settings? *
My resource table is the default for openAMP. This is what it looks like:

ELF_START 0x00000000
ELF_END 0x08000000
NUM_VRINGS 0x02
VRING_ALIGN 0x1000
RING_TX 0x08000000
RING_RX 0x08004000
VRING_SIZE 256

Some more info about the application: My linux app uses TX write to send signal to CPU1 to start reading data from DDR area. Then it waits for reply by read RX.
When CPU1 app is done it writes to the buffer and waits. Then CPU0 reads data from the DDR area and thats it. The area of DDR is hardcoded to BM application and the address is 0x3F300000.

Thank you for your time,
Nikos.

[Jiaying Liang]

Hello Nikos,

> -----Original Message-----
> From: open...@googlegroups.com [mailto:open...@googlegroups.com]
> On Behalf Of snspa...@gmail.com
> Sent: Friday, June 30, 2017 5:29 AM
> To: open-amp <open...@googlegroups.com>
> Cc: snspa...@gmail.com
> Subject: [open-amp] Re: OpenAMP and reserved DDR
>
> Hello Wendy and thanks for the answer.
> Maybe i wasn't clear enough, my remote application runs on the standard
> area for remoteproc, that is the first 256Mi. Here is the device tree entry:
>
> remoteproc@0{
> compatible = "xlnx,zynq_remoteproc";
> reg = <0x00000000 0x10000000>;
> firmware = "firmware";
> vring0 = <15>;
> vring1 = <14>;
> };
>
> To your questions:
>
> * is your application + shared memory are covered in the reserved memory?
> *
> If I understand correctly you mean if my application runs from the reserved
> memory? If so, then no, my CPU1 application runs at the first MBs of RAM
> and uses a pointer to 0x3FE00000 to read and write data.

[Wendy] It looks like you are using Linux kernel xilinx-v2016.4 or before.
You have put your kernel to start from 0x10000000 and you have already set the kernel/userspace 2/2 split.

>
> * do you use fixed shared memory or shared memory? *
> I am not sure what you mean here. You are talking about the mmap()
> parameter?

[Wendy] Yes, but looks like the memory you mmap() is only used for data sharing, which looks fine

>
> * what's your resource table settings? *
> My resource table is the default for openAMP. This is what it looks like:
>
> ELF_START 0x00000000
> ELF_END 0x08000000
> NUM_VRINGS 0x02
> VRING_ALIGN 0x1000
> RING_TX 0x08000000
> RING_RX 0x08004000
> VRING_SIZE 256

[Wendy] As I recalled, with xilinx Linux kernel xilinx-v2016.4 and before, kernel will parse
Resource table and initialize vrings before it starts the remote processor.
In this case, what's seen by CPU1 from resource table when it boots should be fine.
But please double check because you looks like using an older version of OpenAMP,
with this version, OpenAMP will not check the virtio READY bit before it parse the resource table,
And thus, it is possible that it will get the wrong value of vrings before remoteproc master (Linux kernel) sets it.

>
> Some more info about the application: My linux app uses TX write to send
> signal to CPU1 to start reading data from DDR area. Then it waits for reply by
> read RX.
> When CPU1 app is done it writes to the buffer and waits. Then CPU0 reads
> data from the DDR area and thats it. The area of DDR is hardcoded to BM
> application and the address is 0x3F300000.

[Wendy] Looks fine.

Best Regards,
Wendy

>
> Thank you for your time,
> Nikos.
>

> --
> You received this message because you are subscribed to the Google Groups
> "open-amp" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to open-amp+u...@googlegroups.com.
> To post to this group, send an email to open...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments. Delete this email message and any attachments immediately.

[vivozhang]

Any progress on this topic? did you make it work?

[Jiaying Liang]

Zynq is working for us, but I am not sure if it is working for Nikos.

 

Maybe Nikos can answer the question.

 

Best Regards,

Wendy

 

From: open...@googlegroups.com [mailto:open...@googlegroups.com] On Behalf Of vivozhang
Sent: Saturday, August 26, 2017 1:13 AM
To: open-amp <open...@googlegroups.com>
Cc: snspa...@gmail.com
Subject: [open-amp] Re: OpenAMP and reserved DDR

 

Any progress on this topic? did you make it work?

--

You received this message because you are subscribed to the Google Groups "open-amp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-amp+u...@googlegroups.com.

To post to this group, send email to open...@googlegroups.com.

[Jiaying Liang]

Hi Nikos,

 

Does your linux kernel have this patch:

https://github.com/Xilinx/linux-xlnx/commit/fd313e2095b873bb99b3942d5537fc8453dd94b1#diff-af0885ab56f27c1b199cc826e62f3d4c

 

Best Regards,

Wendy

 

 

From: Nikos Spanidis [mailto:snspa...@gmail.com]
Sent: Tuesday, September 12, 2017 10:18 AM
To: Jiaying Liang <jli...@xilinx.com>
Subject: RE: [open-amp] Re: OpenAMP and reserved DDR

 

Hello and sorry for the late answer,

 

I wasn't able to make it work for the application I wanted. Unfortunately because of my work I wasn't able to pursue it further so I don't have an answer. If anyone is willing to try it, maybe with a DMA driver for the reserved area, I am looking forward to see their implementation.

 

Thank you again for your time,

Nikos

[Jie Zhang]

Hello,

I am also trying use a large block of DDR as shared memory between baremetal and linux in with OpenAmp. However, i met following issues. 

1. When i write a block of different  data in to memory starting from 0x09000000 in baremetal CPU1 periodically, everything works fine. (Readback data looks ok)But when i read data from the same address in Linux in CPU0,  the data stay unchanged. I can get update data only when i stop CPU 1.

2, reverse the above, CPU0 write,  while CPU1 read, the same problem happens. 

Test Setup,

CPU 0, Linux,  0x10000000 to 0x3FFFFFFF,

CPU 1, baremetal, elf from 0x0001000 to 0x08000000,

Software,  vivado 2016.4

I know DDR is shared by CPUs, but the DDR reserved block  belongs to neither of CPUs. How cannot i get updated value in memory when i read it from the other CPU,  L2 cache?

Where can i get solution to this problem? 

Thank you.

Jie

Nikos

 

To unsubscribe from this group and stop receiving emails from it, send an email to open-amp+unsubscribe@googlegroups.com.

To post to this group, send email to open...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

 

--
You received this message because you are subscribed to a topic in the Google Groups "open-amp" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/open-amp/0a5B2AisE50/unsubscribe.
To unsubscribe from this group and all its topics, send an email to open-amp+unsubscribe@googlegroups.com.

[Jiaying Liang]

Hi Jie,

 

From: open...@googlegroups.com [mailto:open...@googlegroups.com] On Behalf Of Jie Zhang
Sent: Friday, November 10, 2017 3:16 AM
To: open...@googlegroups.com
Subject: RE: [open-amp] Re: OpenAMP and reserved DDR

 

Hello,

 

I am also trying use a large block of DDR as shared memory between baremetal and linux in with OpenAmp. However, i met following issues. 

 

1.     When i write a block of different  data in to memory starting from 0x09000000 in baremetal CPU1 periodically, everything works fine. (Readback data looks ok)But when i read data from the same address in Linux in CPU0,  the data stay unchanged. I can get update data only when i stop CPU 1.

[Wendy] it looks like you have cached your memory either in CPU0 or CPU1

You will need to map your memory as non-cacheable memory.

 

Best Regards,

Wendy

Thank you.

Jie

 

Nikos

 

To unsubscribe from this group and stop receiving emails from it, send an email to open-amp+u...@googlegroups.com.

To post to this group, send email to open...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "open-amp" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/open-amp/0a5B2AisE50/unsubscribe.

To unsubscribe from this group and all its topics, send an email to open-amp+u...@googlegroups.com.

To post to this group, send email to open...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

 

--

You received this message because you are subscribed to the Google Groups "open-amp" group.

To unsubscribe from this group and stop receiving emails from it, send an email to open-amp+u...@googlegroups.com.

[Jie Zhang]

Yes, i marked the shared block as non-cache memory, now it works as expected,  thank you. 

Jie

2017年11月11日 01:31，"Jiaying Liang" <jli...@xilinx.com>写道：

Hi Jie,

 

Thank you.

Jie

 

Nikos

 

To unsubscribe from this group and stop receiving emails from it, send an email to open-amp+unsubscribe@googlegroups.com.

To post to this group, send email to open...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "open-amp" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/open-amp/0a5B2AisE50/unsubscribe.

To unsubscribe from this group and all its topics, send an email to open-amp+unsubscribe@googlegroups.com.

To post to this group, send email to open...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "open-amp" group.

To unsubscribe from this group and stop receiving emails from it, send an email to open-amp+unsubscribe@googlegroups.com.

To post to this group, send email to open...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "open-amp" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/open-amp/0a5B2AisE50/unsubscribe.

To unsubscribe from this group and all its topics, send an email to open-amp+unsubscribe@googlegroups.com.