python setup.py sdist - include pxd files

639 views
Skip to first unread message

Robert Cimrman

unread,
Nov 29, 2011, 10:38:07 AM11/29/11
to cython-users
Hi,

I googled a bit how to properly make 'python setup.py sdist' include the pxd
files, and found not much. So I have solved my problem by including them
manually as data files, but I wonder - is there a cleaner way?

r.

Robert Bradshaw

unread,
Nov 29, 2011, 12:15:20 PM11/29/11
to cython...@googlegroups.com
That's probably the best way to do it; distutils doesn't know anything
about .pxd files.

Lisandro Dalcin

unread,
Nov 29, 2011, 1:34:50 PM11/29/11
to cython...@googlegroups.com

I usually add them to MANIFEST.in with a line like below:

recursive-include your_source_dir *.pyx *.px[di] *.h *.c


--
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

Robert Cimrman

unread,
Nov 29, 2011, 6:49:27 PM11/29/11
to cython...@googlegroups.com
On Tue, 29 Nov 2011, Lisandro Dalcin wrote:

> On 29 November 2011 14:15, Robert Bradshaw <robe...@math.washington.edu> wrote:
>> That's probably the best way to do it; distutils doesn't know anything
>> about .pxd files.
>>
>> On Tue, Nov 29, 2011 at 7:38 AM, Robert Cimrman <cimr...@ntc.zcu.cz> wrote:
>>> Hi,
>>>
>>> I googled a bit how to properly make 'python setup.py sdist' include the pxd
>>> files, and found not much. So I have solved my problem by including them
>>> manually as data files, but I wonder - is there a cleaner way?
>>>
>
> I usually add them to MANIFEST.in with a line like below:
>
> recursive-include your_source_dir *.pyx *.px[di] *.h *.c

That would work for me too, thanks!

r.

Robert Kern

unread,
Nov 30, 2011, 5:29:58 AM11/30/11
to cython...@googlegroups.com

If you want them as part of the installed package like Cython/Include/ does,
they should be included in the package_data. If they are just to be distributed
in the source tarball outside of the package itself, Lisandro's MANIFEST.in line
is appropriate.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Lisandro Dalcin

unread,
Nov 30, 2011, 11:46:23 AM11/30/11
to cython...@googlegroups.com
On 30 November 2011 07:29, Robert Kern <rober...@gmail.com> wrote:
> On 11/29/11 3:38 PM, Robert Cimrman wrote:
>>
>> Hi,
>>
>> I googled a bit how to properly make 'python setup.py sdist' include the
>> pxd
>> files, and found not much. So I have solved my problem by including them
>> manually as data files, but I wonder - is there a cleaner way?
>
>
> If you want them as part of the installed package like Cython/Include/ does,
> they should be included in the package_data.

And of course, installing *.pxds (and adding a get_include() in your
top package namespace, like NumPy does) is a VERY good idea. That way,
third party Cython code can fast access the internals of your cdef
classes.

> If they are just to be
> distributed in the source tarball outside of the package itself, Lisandro's
> MANIFEST.in line is appropriate.
>

BTW, It is also also appropriate for *.pyx sources.

Robert Kern

unread,
Nov 30, 2011, 5:09:37 PM11/30/11
to cython...@googlegroups.com
On 11/30/11 4:46 PM, Lisandro Dalcin wrote:
> On 30 November 2011 07:29, Robert Kern<rober...@gmail.com> wrote:

>> If they are just to be
>> distributed in the source tarball outside of the package itself, Lisandro's
>> MANIFEST.in line is appropriate.
>>
>
> BTW, It is also also appropriate for *.pyx sources.

Those *should* already be taken care of since they will be in the sources list
of Extensions.

Robert Cimrman

unread,
Dec 1, 2011, 4:29:19 AM12/1/11
to cython...@googlegroups.com
On Wed, 30 Nov 2011, Lisandro Dalcin wrote:

> On 30 November 2011 07:29, Robert Kern <rober...@gmail.com> wrote:
>> On 11/29/11 3:38 PM, Robert Cimrman wrote:
>>>
>>> Hi,
>>>
>>> I googled a bit how to properly make 'python setup.py sdist' include the
>>> pxd
>>> files, and found not much. So I have solved my problem by including them
>>> manually as data files, but I wonder - is there a cleaner way?
>>
>>
>> If you want them as part of the installed package like Cython/Include/ does,
>> they should be included in the package_data.
>
> And of course, installing *.pxds (and adding a get_include() in your
> top package namespace, like NumPy does) is a VERY good idea. That way,
> third party Cython code can fast access the internals of your cdef
> classes.
>
>> If they are just to be
>> distributed in the source tarball outside of the package itself, Lisandro's
>> MANIFEST.in line is appropriate.
>>
>
> BTW, It is also also appropriate for *.pyx sources.

Thanks, Lisandro and Robert, for your feedback.

My primary concern was to have the *.pxd files in the source tarball, not
so much their installation.

I am using numpy.distutils, and so use config.add_data_files() function, which
does the job, and also makes them to be installed along the compiled
modules. I have also (wrongly) added the *.pyx files in the same way,
but there is probably no reason for them to be installed?

Thanks!
r.

Robert Kern

unread,
Dec 1, 2011, 5:45:33 AM12/1/11
to cython...@googlegroups.com
On 12/1/11 9:29 AM, Robert Cimrman wrote:

> I am using numpy.distutils, and so use config.add_data_files() function, which
> does the job, and also makes them to be installed along the compiled modules. I
> have also (wrongly) added the *.pyx files in the same way, but there is probably
> no reason for them to be installed?

Not really, no.

Robert Bradshaw

unread,
Dec 1, 2011, 3:18:22 PM12/1/11
to cython...@googlegroups.com
On Thu, Dec 1, 2011 at 2:45 AM, Robert Kern <rober...@gmail.com> wrote:
> On 12/1/11 9:29 AM, Robert Cimrman wrote:
>
>> I am using numpy.distutils, and so use config.add_data_files() function,
>> which
>> does the job, and also makes them to be installed along the compiled
>> modules. I
>> have also (wrongly) added the *.pyx files in the same way, but there is
>> probably
>> no reason for them to be installed?
>
>
> Not really, no.

They can be handy for introspection (e.g. ?? in ipython).

- Robert

Lisandro Dalcin

unread,
Dec 1, 2011, 7:01:22 PM12/1/11
to cython...@googlegroups.com

Yes, but I never figured out how to hack Python or IPython to hook the
loading of installed *.pyx files. Have you ever get this working?

Robert Bradshaw

unread,
Dec 1, 2011, 7:03:57 PM12/1/11
to cython...@googlegroups.com
Works in Sage. (Not sure how many hacks that required...)

MinRK

unread,
Dec 1, 2011, 7:11:24 PM12/1/11
to cython...@googlegroups.com
On Thu, Dec 1, 2011 at 16:03, Robert Bradshaw <robe...@math.washington.edu> wrote:
Works in Sage. (Not sure how many hacks that required...)

I would love to know how that works, so we could get it into IPython proper (or Cython - whoever needs patching).

-MinRK

Robert Kern

unread,
Dec 2, 2011, 6:06:43 AM12/2/11
to cython...@googlegroups.com
On 12/2/11 12:11 AM, MinRK wrote:
>
>
> On Thu, Dec 1, 2011 at 16:03, Robert Bradshaw <robe...@math.washington.edu
> <mailto:robe...@math.washington.edu>> wrote:
>
> Works in Sage. (Not sure how many hacks that required...)
>
>
> I would love to know how that works, so we could get it into IPython proper (or
> Cython - whoever needs patching).

Sage has its own extensions to the inspect module. I haven't looked through all
of the details, but you can probably extract the Cython-specific hacks for
IPython's extensions to the inspect module.

http://hg.sagemath.org/sage-main/file/tip/sage/misc/sageinspect.py

Reply all
Reply to author
Forward
0 new messages