Re: [PATCH 0/5] Add noncoherent platform support for vop driver

[Arnd Bergmann]

On Fri, Sep 25, 2020 at 9:27 AM Sherry Sun <sherr...@nxp.com> wrote:
>
> Change the way of allocating vring to support noncoherent platform for vop
> driver, and add some related dma changes to make sure noncoherent platform works
> well.

Could you describe why you are doing this? Are you using Intel MIC
devices on Arm hosts, or trying to reuse the code for other add-on
cards?

Note that we have a couple of frameworks in the kernel that try to
do some of the same things here, notably the NTB drivers and the
PCI endpoint support, both of which are designed to be somewhat
more generic than the MIC driver.

Have you considered using that instead?

Arnd

> Sherry Sun (5):
> misc: vop: change the way of allocating vring for noncoherent platform
> misc: vop: change the way of allocating used ring
> misc: vop: simply return the saved dma address instead of virt_to_phys
> misc: vop: set VIRTIO_F_ACCESS_PLATFORM for nocoherent platform
> misc: vop: mapping kernel memory to user space as noncached
>
> drivers/misc/mic/bus/vop_bus.h | 2 +
> drivers/misc/mic/host/mic_boot.c | 8 ++
> drivers/misc/mic/vop/vop_main.c | 51 +++++++++----
> drivers/misc/mic/vop/vop_vringh.c | 117 ++++++++++++++++++++----------
> 4 files changed, 125 insertions(+), 53 deletions(-)
>
> --
> 2.17.1
>

[Sherry Sun]

Hi Arnd,

> Subject: Re: [PATCH 0/5] Add noncoherent platform support for vop driver

>
> On Fri, Sep 25, 2020 at 9:27 AM Sherry Sun <sherr...@nxp.com> wrote:
> >
> > Change the way of allocating vring to support noncoherent platform for
> > vop driver, and add some related dma changes to make sure noncoherent
> > platform works well.
>
> Could you describe why you are doing this? Are you using Intel MIC devices
> on Arm hosts, or trying to reuse the code for other add-on cards?
>

We want to reuse the vop driver between two i.MX boards which are arm64 architecture. And in fact we have successfully verified it.

But the biggest problem we currently encounter is about the noncoherent memory.
Since the hardware device of our platform is not dma coherent,
when use the original way(__get_free_pages and dma_map_single, but without dma_sync_single_for_cpu/device) will meet errors.

Device pages/vring which are interact between ep and rc should use consistent memory without caching effects,
so allocate them by dma_alloc_coherent is a better way.

> Note that we have a couple of frameworks in the kernel that try to do some
> of the same things here, notably the NTB drivers and the PCI endpoint
> support, both of which are designed to be somewhat more generic than the
> MIC driver.
>
> Have you considered using that instead?
>
> Arnd
>

Sorry I don't much about NTB, but for PCI endpoint driver, we will use it for pci data interaction below the vop layer.

Best regards
Sherry