Reorganizing include files in Python source
Thomas Wouters
As I recall, you mentioned you wanted to reorganize the Python include files, separating 'public' ones from 'private' ones, at least to reduce the pollution in user-code. Have you thought any more about this yet? The reason I ask is that for Unladen Swallow, we want to make that distinction as well (all our new files so far are purely 'internal') and it would be good if we used the same scheme as CPython will (eventually) use :)
Right now I'm thinking we have three distinct cases: includefiles users may need to include directly (Python.h and perhaps a select few others), includefiles that Python.h may need (but probably shouldn't pollute the global include "namespace"), and includefiles that are just for CPython source (and tightly coupled extension modules) and which shouldn't even be installed by 'make install'. Actually, there's a fourth category that might be considered distinct from the third: .h files that are included by specific C files to provide implementations, like the thread_*.h files.
For the second category, a subdirectory seems ideal to me, although a Python-specific prefix on the names probably works fine too. For the third we probably want a separate subdirectory, say, 'internal', as that will make it easier to not install those files. Of course, we have to be really really certain those files are strictly internal. On the other hand, right now, we've been adding the private includefiles alongside their accompanying source file (meaning in Python/) which means there's no chance of them getting installed or being used by third-party code by accident. I personally don't like the amount of clutter that creates, though.)
Any thoughts on this issue?
--
Thomas Wouters <twou...@google.com>
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
"Martin v. Löwis"
> files, separating 'public' ones from 'private' ones, at least to reduce
> the pollution in user-code. Have you thought any more about this yet?
Thought - yes. Done anything - no.
> Right now I'm thinking we have three distinct cases: includefiles users
> may need to include directly (Python.h and perhaps a select few others),
> includefiles that Python.h may need (but probably shouldn't pollute the
> global include "namespace"), and includefiles that are just for CPython
> source (and tightly coupled extension modules) and which shouldn't even
> be installed by 'make install'.
My approach would be different: leave all the files as they are, but
instead hide parts of it in #ifdef blocks.
Of course, it might be also possible to reorganize the files, but I
would consider this secondary (for my objective - which is to provide
a stable ABI).
> For the second category, a subdirectory seems ideal to me, although a
> Python-specific prefix on the names probably works fine too. For the
> third we probably want a separate subdirectory, say, 'internal', as that
> will make it easier to not install those files.
Ok, so this is about pollution of the #include space. Is that really
a problem that needs to be solved?
If so, I think there should be exactly two header files that either
embedding applications or extensions modules might include:
Python.h, and pyconfig.h. If an application legitimately needs to
include anything else, I would consider that a bug (in Python.h)
For the files included from Python.h, either moving them to a
subdirectory, or renaming them sounds fine to me.
> Any thoughts on this issue?
I don't think the #include space is a tight resource. If we
learn about a conflict with a specific file, we could always
rename ours.
OTOH, for the stable ABI, the names of the header files don't
matter. What matters more is
a) what functions extension modules call - certain "internal"
functions should be hidden (e.g. support routines for
frames, code objects, and the like)
b) what structures they access (PyObject fields are public,
anything else should be private)
Regards,
Martin