best practice for cython singleton extension class

[Gregor Thalhammer]

Hi all,

I experience some problems with my wrapper for AMDs open source GPU FFT library (clFFT),
https://github.com/geggo/gpyfft It seems that the clFFT library does not like to be initialized and released repeatedly. Therefor I would like to change my cython extension class (GpyFFT), which wraps access to the C library, to a singleton class. Some nice recipes I found for Python singletons that redefine __new__, seem not to work with cython, it refuses to compile ('use __cinit__ instead'). Can somebody recommend a best practice approach to realize a singleton class with cython?

Another question: For easy installation of the wrapper on Windows, I would like to add the external library (clFFT.dll) to the package directory. Any advice on a best practice? I found some hints to use install_clib from numpy.distutils, but it seems cython and numpy.distutils don't play well together, and some patching is required (https://github.com/matthew-brett/du-cy-numpy)

thanks
Gregor

[Stefan Behnel]

Gregor Thalhammer, 26.06.2014 13:21:

> I experience some problems with my wrapper for AMDs open source GPU FFT
> library (clFFT), https://github.com/geggo/gpyfft It seems that the
> clFFT library does not like to be initialized and released repeatedly.
> Therefor I would like to change my cython extension class (GpyFFT),
> which wraps access to the C library, to a singleton class. Some nice
> recipes I found for Python singletons that redefine __new__, seem not to
> work with cython, it refuses to compile ('use __cinit__ instead'). Can
> somebody recommend a best practice approach to realize a singleton class
> with cython?

Why is it linked to a class and not the module? If reinitialisation is not
supported, then just initialise it globally, either on module import or on
first use of the library.

How do you know when to do the cleanup? Wouldn't that best be done by an
atexit callback?

> Another question: For easy installation of the wrapper on Windows, I
> would like to add the external library (clFFT.dll) to the package
> directory. Any advice on a best practice? I found some hints to use
> install_clib from numpy.distutils, but it seems cython and
> numpy.distutils don't play well together, and some patching is required
> (https://github.com/matthew-brett/du-cy-numpy)

It looks like you're not using cythonize().

http://docs.cython.org/src/reference/compilation.html#compiling-with-distutils

Stefan

[Gregor Thalhammer]

Am 26.06.2014 um 13:34 schrieb Stefan Behnel <stef...@behnel.de>:

> Gregor Thalhammer, 26.06.2014 13:21:
>> I experience some problems with my wrapper for AMDs open source GPU FFT
>> library (clFFT), https://github.com/geggo/gpyfft It seems that the
>> clFFT library does not like to be initialized and released repeatedly.
>> Therefor I would like to change my cython extension class (GpyFFT),
>> which wraps access to the C library, to a singleton class. Some nice
>> recipes I found for Python singletons that redefine __new__, seem not to
>> work with cython, it refuses to compile ('use __cinit__ instead'). Can
>> somebody recommend a best practice approach to realize a singleton class
>> with cython?
>
> Why is it linked to a class and not the module? If reinitialisation is not
> supported, then just initialise it globally, either on module import or on
> first use of the library.

Thanks for the advice. That reinitializing is not possible is an experimental finding (after writing the wrapper), and possibly not intended. At initialization of the library you can define some configuration settings, therefor a class based approach seemed more suitable for me. If you are able to set options only once, that feels like a broken design.

I agree that linking the lifetime of the library to the extension module is probably the best solution, but I am looking for an alternative, that does not require changes to existing code using this wrapper.

>
> How do you know when to do the cleanup? Wouldn't that best be done by an
> atexit callback?

At the moment cleanup is done in __dealloc__ of the object wrapping the library (with Cython.Compiler.Options.generate_cleanup_code = 2)

>
>
>> Another question: For easy installation of the wrapper on Windows, I
>> would like to add the external library (clFFT.dll) to the package
>> directory. Any advice on a best practice? I found some hints to use
>> install_clib from numpy.distutils, but it seems cython and
>> numpy.distutils don't play well together, and some patching is required
>> (https://github.com/matthew-brett/du-cy-numpy)
>
> It looks like you're not using cythonize().
>
> http://docs.cython.org/src/reference/compilation.html#compiling-with-distutils
>

Thanks for the hint, I have been using `from Cython.Distutils import build_ext`.

> Stefan
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Chris Barker]

On Thu, Jun 26, 2014 at 8:31 AM, Gregor Thalhammer <gregor.t...@gmail.com> wrote:

Thanks for the advice. That reinitializing is not possible is an experimental finding (after writing the wrapper), and possibly not intended. At initialization of the library you can define some configuration settings, therefor a class based approach seemed more suitable for me. If you are able to set options only once, that feels like a broken design.

I agree that linking the lifetime of the library to the extension module is probably the best solution, but I am looking for an alternative, that does not require changes to existing code using this wrapper.

in pure python, you could use class attributes, so that all instances shared the same instance of the lib(and whatever config parameters, etc,) -- so need need for a true singleton.

not sure if you can do that with a cdef class, but worth a shot.

-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov