Shared buffers in AMP configuration

[Eric]

I am using OpenAMP on Zynq-7000 with Linux (linux-xlnx kernel 4.0.0) running on cpu0 and a bare metal application on cpu1. 

I have two 32MB shared memory buffers at 0x37E0_0000 and 0x39E0_0000 that cpu1 writes to and uses the rpmsg interface to communicate to cpu0 when there is new data.

I have the buffers defined as UIO devices using uio_pdrv_genirq driver like this in my device tree:

reserved-memory {
    #address-cells = <1>;
    #size-cells = <1>;
    ranges;
    
    shared_reserved: shared@37e00000 {
        reg = <0x37e00000 0x4000000>;
        no-map;
    };
};

buf1 { 
    compatible = "uio-pdrv";
    reg = <0x37e00000 0x2000000>;
};

buf2 { 
    compatible = "uio-pdrv";
    reg = <0x39e00000 0x2000000>;
};

cpu1app: cpu1app@0 {
    compatible = "xlnx,zynq_remoteproc";
    reg = < 0x1e000000 0x2000000 >;
    interrupt-parent = <&ps7_scugic_0>;
    interrupts = < 0 27 4   0 29 4   0 39 4   0 31 4 >;
    firmware = "cpu1app.elf";
    vring0 = <15>;
    vring1 = <14>;
    memory-region = <&shared_reserved>;  // not sure that this actually does anything
}; 

In my kernel parameters, I have "uio_pdrv_genirq.of_id=uio-pdrv" and this creates /dev/uio0 and /dev/uio1 for buf1 and buf2 above.  Then in a userspace application, I open and mmap the UIO devices to read the shared memory buffers.

I thought this configuration makes the shared buffers shareable and non-cacheable, but that doesn't seem to be the case because at times, cpu0 is not reading coherent data from cpu1.

I'm certain that the bare metal cpu1 application is marking the shared buffers as shareable, non-cacheable.  But I'm not sure that's the case on Linux/cpu0.  Should this configuration make the shared buffers shareable and non-cacheable on Linux side?

Some things I tried in hopes of fixing this:

1) I tried using the uio_dmem_genirq driver instead by using kernel parameter "uio_dmem_genirq.of_id=uio-pdrv", but that doesn't seem to work. The /dev/uio devices are not created at all.  And I'm not sure how I would make this driver use the static buffer addresses I want vs. letting it allocate memory dynamically.

2) I changed the cpu1app "reg" property to < 0x1E000000 0x1DE00000 >.  I'm not sure what this property is for exactly, but I made it include the shared memory buffers.  When I do this however, there is a Linux kworker task that utilizes >85% cpu and really bogs things down.  The shared buffer data seems to be coherent though, but I can't be sure that it is this property that's doing it or if it's because the CPU usage is affecting things a lot.  But this can't be a solution because the cpu usage renders the system unusable.

3) As a final desperate effort, I modified the zynq_remoteproc_probe() function in zynq_remoteproc.c to add this call to try to force the shared memory buffers to be coherent:
    dma_declare_coherent_memory(&pdev->dev, 0x37E00000, 0x37E00000, 0x40000000 - 0x37E0000, DMA_MEMORY_IO);

But the call fails. I'm not sure why, other than I'm giving it magic addresses out of nowhere from the driver's perspective.

Can anyone help me with the proper way to fix my configuration and make my shared memory buffers shareable and non-cacheable from Linux?  I know I am using a fairly old kernel and see a lot of changes in zynq_remoteproc since then, but cannot upgrade the kernel at this time.

Thank you for any help/suggestions.