other approaches to static resources package management
Jan-Jaap Driessen
I remember that you discussed static resource package management with
other people in the past. Care to join this discussion:
http://jw.n--tree.net/irclogs/fanstatic/%23fanstatic.2011-02-01.log.html
Cheers,
JJ
Martijn Faassen
I think Thomas should bring these up on the mailing list too, but:
"fanstatic requires Paste and webob.
This is a bit repelling if one's project already uses something else
(like e.g werkzeug)."
That's the "oh there's Zope in the package name so I won't use it!"
thing all over again. We supply WSGI middleware. We can't get much
more compatible than that.
What libraries we use to implement them should not be a major concern
of anyone This thing frustrates me, as it's impossible to please
everyone, ever. :(
(note that I do think it'd be good to eventually remove our dependency
on paste, as it contains a lot of random stuff that's partially
replaced by webob)
"Injecting stuff into html output. Looks a bit ugly at first glance,
but using the injector seems optional."
I think we're entirely open to providing alternatives and integrate
them into Fanstatic if it makes sense.
"fanstatic static files packages. static files packages that require
fanstatic should be named fanstatic.* not js.*"
That is technically impossible. fanstatic.js.* would be the closest we
can get, and that looks pretty nasty.
Anyway, we've already had a discussion about this, and we could
change, but we've staked a claim for now. :) I think staking a claim
here is actually worthwhile.
"Even better would be if those packages were generic, meaning:
a) do NOT require fanstatic (see toolkit universes issue), do not even
require anything special
b) just provide the static files and metadata in a generic, easy-to-use way
c) after doing a) and b), you can rightfully have js.* or even * as
PyPi package name (but you'll notice that e.g. "jquery" is already
taken by turbogears jquery stuff - wtf!)
d) this would save fanstatic developers/users as well as
non-fanstatic developers/users quite some work, because with less
specialisation of such packages more collaboration can happen. So you
don't need to create all those packages yourself, they WILL be there
on pypi already because someone else already did."
I think this would be a useful initiative.
A few comments, though:
Instead of implementing our own generic metadata for this, we should
consider whether we can't use common.js formats for this (package.json
and the like). I had some problems translating everything into
hurry.resource terms last time and not much luck seeing adjustments
happen, but they've made some further progress so who knows what time
will bring.
I also don't think we should be hosting these things on pypi, but I
guess there's lack of anything better. I.e. if we're going to go for
metadata, let's go all the way and be language-neutral.
That said, it's probably more pragmatic to just come up with our own
incompatible metadata format.
I'm disappointed we have "non-Fanstatic developers" in the Python
world we need to please. Seems like fragmentation is alive and well...
These are useful comments, but I have the sense that NIH is also
involved a bit here.
Regards,
Martijn
Thomas Waldmann
> This is a bit repelling if one's project already uses something else
> (like e.g werkzeug)."
>
> That's the "oh there's Zope in the package name so I won't use it!"
> thing all over again.
> We supply WSGI middleware.
I don't want middleware (at least not right now), I want static file
packages.
> What libraries we use to implement them should not be a major concern
> of anyone This thing frustrates me, as it's impossible to please
> everyone, ever. :(
packages.
Would require some dependency reorg for fanstatic.
fanstatic - (almost?) empty namespace package
fanstatic.pkg.whatever - static package for <whatever>, no
requirements (except maybe fanstatic)
fanstatic.ext.coolcode - some cool code doing something, may have
requirements
> (note that I do think it'd be good to eventually remove our dependency
> on paste, as it contains a lot of random stuff that's partially
> replaced by webob)
> "fanstatic static files packages. static files packages that require
> fanstatic should be named fanstatic.* not js.*"
> That is technically impossible. fanstatic.js.* would be the closest we
> can get, and that looks pretty nasty.
> ...
Or do we need to reinvent the trivial part of the wheel that we can't
get separately yet?
I am not keen on reinventing that part, but I rather would do that
than indirectly requiring stuff I don't need / don't want.
> Instead of implementing our own generic metadata for this, we should
> consider whether we can't use common.js formats for this (package.json
> and the like).
If it is something simple and doesn't require a bunch of other stuff,
it could be ok.
Our idea of just using __init__.py with some simple attributes was
that importing a module is as simple as it can get for a python
packaged bunch of static files.
> I also don't think we should be hosting these things on pypi, but I
> guess there's lack of anything better.
> I.e. if we're going to go for
> metadata, let's go all the way and be language-neutral.
realistic when even the python world obviously couldn't agree on some
simple thing for themselves yet? And what should that be? XML? JSON?
Plain text?
> That said, it's probably more pragmatic to just come up with our own
> incompatible metadata format.
> I'm disappointed we have "non-Fanstatic developers" in the Python
> world we need to please. Seems like fragmentation is alive and well...
we'ld like to share easy-to-use no-fat-added static packages.
> These are useful comments, but I have the sense that NIH is also
> involved a bit here.
inventing it and searched for existing solutions.
But I didn't find any simple pypi package that just provides the
static files I want, so I am using a script with wget and tar/unzip
right now.
I am not in a hurry and can wait for a while (if fanstatic developers
would consider moving in that direction in the near future), but i
need to get this stuff into a sane state in the next months, so some
statement about this would be useful so I don't wait for something
that won't happen.
Martijn Faassen
On Fri, Feb 11, 2011 at 3:33 AM, Thomas Waldmann <tw-p...@gmx.de> wrote:
> But I didn't find any simple pypi package that just provides the
> static files I want, so I am using a script with wget and tar/unzip
> right now.
What is a simple pypi package that just provides the static files you
want? How do you use it? How do you serve these files? Do you have
dependency issues to resolve? Should the static file packages provide
dependency information? Do you need to control the cache? Do you want
to serve minified versions automatically? Do you need to specify
dependencies on Javascript in Python code? How is the requirement on
WebOb related to any of this?
> I am not in a hurry and can wait for a while (if fanstatic developers
> would consider moving in that direction in the near future), but i
> need to get this stuff into a sane state in the next months, so some
> statement about this would be useful so I don't wait for something
> that won't happen.
The reasons why you can't use the middleware Fanstatic supplies are
still unclear to me. Could you elaborate further?
Concerning namespace reorganization, I don't like nested namespace
packages, and I avoid them. There's a particular use case Fanstatic
supports:
Fanstatic currently configures data on the Python level, specific to
Fanstatic. The primary benefit is that you can import directly from
these packages to declare dependencies in your own code, like this:
from js.jquery import jquery
... somewhere... : jquery.need()
We can't break that use case without good reasons.
But in general, I don't really like configuring things in Python code
as it locks you into a single solution - your argument too, I believe.
Innovation for Python packaging tools has been hindered by the
connection to distutils/setputools in setup.py, for instance. If that
had been static data files from the start it'd have been better. So
with that in mind, I'm looking to evolve Fanstatic in that direction
too.
If we're moving towards static data, we'd need to come up with some
clever approach to expose this information to Python so it can still
be imported.
> It's easy. Don't require any non-trivial stuff for the static file
> packages.
I've already been engaged on the CommonJS mailing list months ago.
Unfortunately the format at least then wasn't really suitable for
browser purposes (they're working on that), and my requests for
particular extensions that I knew Fanstatic needs fell on deaf ears.
It's possible I misunderstand what's possible with CommonJS however.
I'd be interested in hearing your thoughts.
With CommonJS packaging, you'd competely sidestep the whole PyPI issue
altogether in that case. Unfortunately however CommonJS infrastructure
is immature for the browser, and there's no equivalent to PyPI either.
Perhaps we can create a tool to wrap a CommonJS package for PyPI. :)
If you're going to go generic, it makes total sense: we're packaging
Javascript code after all. The ideal situation is that the developers
of the JS packages themselves supply the dependency information, just
like we expect Python developers to do today. If that can't be done,
then we'd want to be able for *any* Javascript developer to add this
information externally to the package. This is stil a very broad class
of people.
I'm not convinced that it makes sense to do this *as Python
developers*. We'd have to coordinate formats with others (with ideas
of their own) for no real gain to the Fanstatic effort, as we can
already wrap the packages for Fanstatic. If you're going to go
generic, I think you should go all the way, unless you can come up
with good reasons why not.
There are some potential reasons why not: it requires coordination
with more people, and CommonJS might not fully support our use cases.
We need to investigate further (and also RequireJS, which claims to be
based on CommonJS but takes care of browser issues).
> I am not in a hurry and can wait for a while (if fanstatic developers
> would consider moving in that direction in the near future), but i
> need to get this stuff into a sane state in the next months, so some
> statement about this would be useful so I don't wait for something
> that won't happen.
I'm not sure what you want. Please tell us first what you want, and why.
Regards,
Martijn