git submodules as python subpackages

[Spencer Lyon]

We are trying to create a python package to do various forms of interpolation. The end goal is to have this package serve two purposes:

1. Be a standalone package on its own
2. Be included as a subpackage of a different python package.

For example, suppose the interpolation package is named interp (so that we could do from interp import foo in a python session) and suppose that the parent package of which interp will be a subpackage is called parent (so that we could do from parent.interp import foo).

My question is, how can we include interp as a submodule within the parent repository and maintain the desired package structure.

The ideal solution would be to include, as a submodule, only the directory within the interp repo that contains the actual source code. I don’t think it is possible to partially include a submodule or to have a submodule point at a particular directory of another repo.

Potential solutions that have been suggested thus far are:

• Have parent include interp as required dependency and structure parents namespace using __init__.py files. In this case the source of interp would not be included in parents repo at all (maybe a good thing, maybe not).
• Create a branch named submod in the interp repo that has it’s root set to mirror the source code directory of the master branch. We could then include the submod branch of interp as a submodule in parents source directory and al __init__.py imports should work. This could get messy as we not only have to manage the submodule, but also need to make sure that we keep the submod branch in sync with releases of the standalone interp package.
• Create a directory called submodules in the root of parents repo and symlink the necessary folders from there to parents source code directory.

While all functional, it seems that each of those options may be sub-optimal. Any suggestions on the best way to attack this problem?

​

[Pablo Winant]

Another potential solution could involve Python namespace packages. Python 3.3 includes them by default, and for older versions, `pkgutil` is part of the standard library. 

We could have a structure for parent like:

parent

- parent

      - normal_code

      - interp

            path manipulation

- interp [a submodule]

     - actual implementation of interp

The idea is to make it look like there is a regular python package, called parent, containing a subpackage interp, while the subpackage is actually located somewhere else.

--
You received this message because you are subscribed to the Google Groups "econforge" group.
To unsubscribe from this group and stop receiving emails from it, send an email to econforge+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Pablo Winant]

It looks like namespace packages serve a different purpose. They are used when code from different source base are supposed to belong to the same package (for instance if  parent.normal_code and parent.interp  are defined in two completely places. In some sense, it is the symmetric operation.

I have made some experiments with symlinks: they work just fine when one remains with Linux but but couldn't have it to work on Linux.

Currently, I'm leaning towards the package requirement option.

Pablo

To unsubscribe from this group and stop receiving emails from it, send an email to econforge+unsubscribe@googlegroups.com.