Atomic operations ...

[Jérôme Kieffer]

Hi,

I am a long-time cython user and I wonder how is the support for atomic
operation in Cython ?

My usecase is a fairly long initialization process (~ seconds)
which is single threaded since it is a "scatter" operation. All my
attempts to parallelized it have failed so far (either the processing
was slower than serial, or the memory consumption was ridiculous, or
the result were wrong). I would like to test using atomic ...

Since the code is widely used in production (40 projects depend on it)
and it has users will all major OS/hardware, this would requires at
minima a decent support under linux (gcc), mac (llvm) and windows
(msvc).

Last quesiion: Would you recommand to export the cython code to C or C++ ?

Thanks for your help

--
Jerome Kieffer

[da-woods]

Hi Jérôme,

Cython has no built-in support for atomics(*), but you can easily access
the C++ standard library support
(https://en.cppreference.com/w/cpp/atomic/atomic, exposed in Cython in
https://github.com/cython/cython/blob/master/Cython/Includes/libcpp/atomic.pxd).

That's widely supported by all major compilers on all major platforms.

C also has standard library support for atomics. Unfortunately:
* MSVC doesn't support them at all,
* We haven't wrapped it ourselves as a convenient cimport - (it should
be pretty simple though, so there's no reason you couldn't do it yourself).

So I recommend using the C++ standard library.

David

* sort of... there's one or two places we use them internally, but not
in a way that's exposed to users.

[Jérôme Kieffer]

Thanks David, I'll give it a try using C++ atomic.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/cython-users/5ddc12a4-75c6-459f-bc04-39226f11b8d9%40d-woods.co.uk.
>


--

[Peter Schay]

I'm not sure how relevant this is for your needs but at any rate, here is an interesting discussion about atomics in Python itself, and I have seen various commits involving work in this area, for all the freethreading work:

https://discuss.python.org/t/exposing-atomic-operations-tooling-in-the-public-c-api/39973/9

To view this discussion visit https://groups.google.com/d/msgid/cython-users/20241116183400.65ff2fd3%40antarctica.fournet.lan.

[Jérôme Kieffer]

Thanks Peter,

As David mentionned, there is a fairly good/portable API in C++ and
since Cython can export either to C or to C++, the later option is
rather appealing.

I can testify it since some Cython extentions in pyFAI got contribution
in (plain) C++ and switching from C to C++ complete cython extensions was
pain-less (maybe slightly slower compilation time, but really marginal).

> To view this discussion visit https://groups.google.com/d/msgid/cython-users/CALKyFjYV8dmhkCtmrPEpSgcBWCC6btt2JbbUex6%2BVB1yZeW0rg%40mail.gmail.com.


--