mozsvc and dependencies

[Alexis Métaireau]

Hello folks,

I'm currently, with the help of tarek, doing my first steps with our way
of dealing with packaging, building Makefiles for stokenserver. While
doing that, I noticed that mozsvc had a bunch of dependencies I don't
care about.

It seems, from what I understood, that mozsvc is a package where we put
all parts that could be useful for others projects we are doing when
developing services.

Because it's not a tool on its own, what do you think about having the
setup.py not declaring its dependencies and letting the distributions
using mozsvc declaring them? This would allow to lower the number of
deps we are getting each time we want to work with mozsvc. Currently, a
python setup.py develop + pip freeze returns a quite long list with some
distributions that seem not necessary for all of us.

For instance, I'm getting the tokenlib, repoze.* libs, simplejson,
pyzmq, pygments and metlog. I also don't get why unittest2 is in the
deps since it's only useful for testing purposes (unless I'm missing
something).

After a quick chat with Tarek, it seems that this could be resolved by
putting all the hard deps (the one I listed, mainly) in a makefile (or a
dev-requirements.txt file), and only letting the "soft" ones in the
setup.py. This would allow us to run for instance the tests using make
test, but not having to fetch the whole world when building services.

What are your feelings about this? An other concerns? thoughts?

Alexis
_______________________________________________
Services-dev mailing list
Servic...@mozilla.org
https://mail.mozilla.org/listinfo/services-dev

[Alexis Métaireau]

cross posting to ops, that may also be interesting to them wrt rpm packages.

[Alexis Métaireau]

I've made the appropriate changes. See the changeset on github [1] for
the details. It should impact your projects so you have to put the
dependencies in them rather than letting them being managed by mozsvc
(So think about it when your tests will brake ;)

[1]
https://github.com/mozilla-services/mozservices/commit/ac2b39c259ac1f7e6599d79b96134e565ac69ebc

-- Alex

On 23/03/2012 17:55, Alexis Métaireau wrote:

[Tarek Ziade]

On 3/26/12 4:52 PM, Alexis Métaireau wrote:

I've made the appropriate changes. See the changeset on github [1] for the details. It should impact your projects so you have to put the dependencies in them rather than letting them being managed by mozsvc (So think about it when your tests will brake ;)

[1] https://github.com/mozilla-services/mozservices/commit/ac2b39c259ac1f7e6599d79b96134e565ac69ebc

also, some of the tools in our build process don't support setuptools extras notation like this one: " metlog-py[zeromqpub]"

fwiw I'd remove that and separate the zeromq backend in another project in case the goal here is to make it as an optional install

Cheers

[Rob Miller]

On Mon Mar 26 07:58:27 2012, Tarek Ziade wrote:
> also, some of the tools in our build process don't support setuptools
> extras notation like this one: " metlog-py[zeromqpub]"

which tools don't support this?

> fwiw I'd remove that and separate the zeromq backend in another
> project in case the goal here is to make it as an optional install

ugh, i *really* don't want to do this. it's an optional install, but so
many people are going to need it that it seems silly to make them
install a separate package. and the amount of code in that separate
package would be very very little.

-r

[Tarek Ziade]

On 3/26/12 8:05 PM, Rob Miller wrote:
> On Mon Mar 26 07:58:27 2012, Tarek Ziade wrote:
>> also, some of the tools in our build process don't support setuptools
>> extras notation like this one: " metlog-py[zeromqpub]"
>
> which tools don't support this?

Most new tools in fact (distutils2, Python itself) . Turns out this
feature was dropped in the post-setuptools era because the use case is
quite exotic. So I don't support this in our build-chain since I use d2.

>
>> fwiw I'd remove that and separate the zeromq backend in another
>> project in case the goal here is to make it as an optional install
>
> ugh, i *really* don't want to do this. it's an optional install, but
> so many people are going to need it that it seems silly to make them
> install a separate package. and the amount of code in that separate
> package would be very very little.

I don't know if it's silly, it depends on the use case really. If you're
saying "..so many people are going to need it.." , I am not sure to see
the point of using the [extras] feature in the first place.

Make it a plain feature and people will just depend on metlog ?

Now if you don't want people that may no use it to pull pyzmq, the way
we implemented plugins in mozsvc seem to be a good way to solve this.
They'd just need to install pyzmq after the fact.

e.g. in your plugin:

try:
import zmq
except ImportError:
raise "To use this backend you need X and Y and Z"

Cheers
Tarek

[Rob Miller]

We're already using conditional imports, although it's set up to not
blow up until the first time you actually try to use the sender, rather
than at import time:

https://github.com/mozilla-services/metlog-py/blob/master/metlog/senders/zmq.py

Given this, it might be okay to just remove the zmq dependency
altogether.

-r