On later kernels 'comedi_config /dev/comedi0 ni_pcimio' does not work

[Anders Blomdell]

With comedi from https://github.com/Linux-Comedi/comedi (git tag 9792ce4) 'comedi_config /dev/comedi0 ni_pcimio'
later kernels (6.2.9, 6.3.7 and 6.4.6) fails with 'Buffer allocation failed' in dmesg but earlier kernels (6.0.15) works.

The problem is that the call to 'dma_alloc_coherent' (called from 'comedi_buf_alloc' fails to allocate
the needed buffer.

Looking at the source in the kernel tree shows very different code in 'comedi_buf_alloc', than in the standalone comedi.

Should I take this as a discouragement to use the out of tree version of comedi, and instead rebuild the kernel with
comedi support?

Best regards

/Anders
--
Anders Blomdell Email: anders....@control.lth.se
Department of Automatic Control
Lund University Phone: +46 46 222 8793
P.O. Box 118
SE-221 00 Lund, Sweden

[Anders Blomdell]

On 2023-08-25 17:10, 'Anders Blomdell' via Comedi: Linux Control and Measurement Device Interface wrote:
> With comedi from https://github.com/Linux-Comedi/comedi (git tag 9792ce4) 'comedi_config /dev/comedi0 ni_pcimio'
> later kernels (6.2.9, 6.3.7 and 6.4.6) fails with 'Buffer allocation failed' in dmesg but earlier kernels (6.0.15) works.
>
> The problem is that the call to 'dma_alloc_coherent' (called from 'comedi_buf_alloc' fails to allocate
> the needed buffer.
>
> Looking at the source in the kernel tree shows very different code in 'comedi_buf_alloc', than in the standalone comedi.

And the driver built with the kernel source works (6.4.11).

[Ian Abbott]

On 25/08/2023 16:10, 'Anders Blomdell' via Comedi: Linux Control and

Measurement Device Interface wrote:
> With comedi from https://github.com/Linux-Comedi/comedi (git tag
> 9792ce4) 'comedi_config /dev/comedi0 ni_pcimio'
> later kernels (6.2.9, 6.3.7 and 6.4.6) fails with 'Buffer allocation
> failed' in dmesg but earlier kernels (6.0.15) works.
>
> The problem is that the call to 'dma_alloc_coherent' (called from
> 'comedi_buf_alloc' fails to allocate
> the needed buffer.

It seems to be the use of `| __GFP_COMP` bit in `flags` parameter of
`dma_alloc_coherent()` that results in the allocation failure. This
changed in Linux commit ffcb75458460 ("dma-mapping: reject __GFP_COMP in
dma_alloc_attrs") from Linux kernel v6.2:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ffcb754584603adf7039d7972564fbf6febdc542

(Note: dma_alloc_coherent() calls dma_alloc_attrs().)

For now, I'll just avoid setting the `__GFP_COMP` flag. I'm not
entirely sure why it was set in the first place!

> Looking at the source in the kernel tree shows very different code in
> 'comedi_buf_alloc', than in the standalone comedi.

The Linux source version allocates the DMA buffer in a single lump so
that `dma_mmap_coherent()` can be used to mmap it. There still does not
seem to be an official way to mmap a buffer consisting of individually
allocated DMA coherent pages, so I was stuck with allocating it as one
big lump to keep it official.

Hopefully, we can still get away with using `remap_pfn_range()` on DMA
coherent pages in the standalone comedi, but it might not work on all
architectures. Feedback welcome!

> Should I take this as a discouragement to use the out of tree version of
> comedi, and instead rebuild the kernel with
> comedi support?

Some distro kernels include comedi support, e.g. Debian and Ubuntu.

--
-=( Ian Abbott <abb...@mev.co.uk> || MEV Ltd. is a company )=-
-=( registered in England & Wales. Regd. number: 02862268. )=-
-=( Regd. addr.: S11 & 12 Building 67, Europa Business Park, )=-
-=( Bird Hall Lane, STOCKPORT, SK3 0XA, UK. || www.mev.co.uk )=-

[Anders Blomdell]

Thanks a lot!

On 2023-08-29 13:28, Ian Abbott wrote:
> On 25/08/2023 16:10, 'Anders Blomdell' via Comedi: Linux Control and Measurement Device Interface wrote:
>> With comedi from https://github.com/Linux-Comedi/comedi (git tag 9792ce4) 'comedi_config /dev/comedi0 ni_pcimio'
>> later kernels (6.2.9, 6.3.7 and 6.4.6) fails with 'Buffer allocation failed' in dmesg but earlier kernels (6.0.15) works.
>>
>> The problem is that the call to 'dma_alloc_coherent' (called from 'comedi_buf_alloc' fails to allocate
>> the needed buffer.
>
> It seems to be the use of `| __GFP_COMP` bit in `flags` parameter of `dma_alloc_coherent()` that results in the allocation failure.  This changed in Linux commit ffcb75458460 ("dma-mapping: reject __GFP_COMP in dma_alloc_attrs") from Linux kernel v6.2:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ffcb754584603adf7039d7972564fbf6febdc542
>
> (Note: dma_alloc_coherent() calls dma_alloc_attrs().)
>
> For now, I'll just avoid setting the `__GFP_COMP` flag.  I'm not entirely sure why it was set in the first place!

That's what stopped me from proposing that change directly (but got to refresh my knowledge about linux DMA, long time since
I looked at at last time...)

>> Looking at the source in the kernel tree shows very different code in 'comedi_buf_alloc', than in the standalone comedi.
>
> The Linux source version allocates the DMA buffer in a single lump so that `dma_mmap_coherent()` can be used to mmap it.  There still does not seem to be an official way to mmap a buffer consisting of individually allocated DMA coherent pages, so I was stuck with allocating it as one big lump to keep it official.
>
> Hopefully, we can still get away with using `remap_pfn_range()` on DMA coherent pages in the standalone comedi, but it might not work on all architectures.  Feedback welcome!
>
>> Should I take this as a discouragement to use the out of tree version of comedi, and instead rebuild the kernel with
>> comedi support?
>
> Some distro kernels include comedi support, e.g. Debian and Ubuntu.

Will have to nag Fedora about this (and get comedilib back in again)