Managing Dependencies

[mattjmorrison]

Spencer, Toran and Myself started a discussion on Twitter that we then moved to email and I thought it would be a good idea to move it here so everyone can contribute. I'm also copying in Robert - we briefly talked about this at the last Pyowa meeting.

Here are a few questions that have come out of this discussion:

1. How do you manage changes to your code that also request changes to your dependencies?
   
2. What happens when you do not control the versioning of the change in your dependency? Do you fork it and maintain your own fork?
3. What tools do you use to help manage dependencies? (virtualenv, pythonbrew, buildout, pip, an internal pypi server, etc)
4. How do you build the distribution for your application? (tarballs, debian packages, etc)
5. How do you deliver your distribution to the target machine? (internal repositories, etc)

I think this would be an excellent topic for a meeting, but maybe we can flush out some ideas here on the mailing list to get a better feel for a more narrow focus for a meeting topic. 

For context you can read through the twitter discussion here: https://twitter.com/mattjmorrison/status/321984324297711617

Below is the email part of the conversation:

On Wed, Apr 10, 2013 at 9:10 AM, Spencer Herzberg wrote:

Not sure if you have covered pip in a previous meeting so I'm not sure if that would beneficial since there are lots of things online about using pip for pulling in dependencies.

I am more concerned/interested in developing on a large python project where you are using external dependencies from third-party (or internal) libraries that have their own release and packaging where you are needing to make modifications. So things like what do you do until a pull request is merged in and released. Do you package an internal version until then? Do you just use pip install -e? Also things like hosting your own pypi server and maybe how people are releasing to it. On a continuous build? Manually?

I am running into the above issues about waiting for a new package from a third party where I have requested a pull request. I have been using pip install -e from my branch with the fix, but I cannot do that any more because I am trying to build native debian packages for deployment (http://hynek.me/articles/python-app-deployment-with-native-packages/) and virtualenvs don't like when you try to move it to another directory when using editable deps because of the symlinking virtualenv does.

That's a lot for a meeting but what are your thoughts?

-Spencer

On Wed, Apr 10, 2013 at 8:58 AM, Matthew J. Morrison wrote:

Did either of you have a specific presentation in mind for this topic, or do you think this would be better served as a panel of experts asking/answering questions about how they currently tackle this problem?

-- 
Matthew J. Morrison

@mattjmorrison

mattjmorrison.com