webassets or fanstatic?

417 views
Skip to first unread message

Sascha Gottfried

unread,
Mar 11, 2014, 1:39:12 PM3/11/14
to pylons-...@googlegroups.com
Hi All,
for an existing medium-sized pyramid application I now want to switch to static resource publishing / asset management.
Current JS dependencies are widely used stuff like jQuery, jQuery UI, jQuery DataTables, Highcharts/Highstocks but not more than 10 different libraries. 
I have fanstatic or webassets as possible candidates. Both provide pyramid integration. I found fanstatic earlier than webassets, that is why I started examining fanstatic first.
On PyPI I found https://pypi.python.org/pypi/js.jquery/ but there are no updates since a year. What does that mean? 
I am actually not able to find similar bundles made with webassets.

I want to go for the solution that is mature, but still under development and provides a lot of packages in PyPI?
What is your recommendation?



Thanks in advance
Best regards, Sascha




Randall Leeds

unread,
Mar 11, 2014, 5:05:24 PM3/11/14
to pylons-...@googlegroups.com
I started with fanstatic and switched to webassets.

I found that the js namespace python packages were not useful. They are often out of date, as you noticed, and to repackage everything you need as python is not sustainable to me.

I'm sure you can use fanstatic without that, though.

I liked webassets for other reasons I can't remember. I found the developer to be very responsive. Also, sontek, who maintains pyramid_webassets, is also very responsive and visible in the community. I've landed PRs there and have more to come.

I think the easiest way to manage assets for a pyramid application is to check them into your repo.

Lately I have been curious about nodeenv[1], though. With so many client-side js package repositories there might be a solution where you just use node to manage those dependencies, calling bower or npm from python, and then using webassets (or fanstatic) to include the downloaded assets.

To me, this is a problem that doesn't have a consensus solution in the community yet. It's ripe for blog posts from any who have a clear vision.


--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To post to this group, send email to pylons-...@googlegroups.com.
Visit this group at http://groups.google.com/group/pylons-discuss.
For more options, visit https://groups.google.com/d/optout.

Blaise Laflamme

unread,
Mar 11, 2014, 6:35:48 PM3/11/14
to pylons-...@googlegroups.com
I still prefer to use bower & grunt to manage my assets. Once you have installed npm and bower, you can easily make them isolated to your project using ree's buildout recipie and then have them updated manually by calling them or by updating the buildout.

I propose you to take a look at bower & grunt, they are well done and widely used... more than any python lib you can find for this job.

Jens W. Klein

unread,
Mar 12, 2014, 5:51:32 AM3/12/14
to pylons-...@googlegroups.com
On 2014-03-11 23:35, Blaise Laflamme wrote:
> I still prefer to use bower & grunt to manage my assets. Once you have
> installed npm and bower, you can easily make them isolated to your
> project using ree's buildout recipie and then have them updated manually
> by calling them or by updating the buildout.
>
> I propose you to take a look at bower & grunt, they are well done and
> widely used... more than any python lib you can find for this job.

Same here. I dont think any python lib will ever do the job more
efficient. Also grunt is awesome flexible and extensible. We had a
project where we deployed an angularjs based js/css/html thing as
dynamic pyramid website and as offline phonegap app. Easy with
bower+grunt, lots of work to rewrite all the steps needed in python.

Passing dependency management over to python creates more headache than
it solves. Once learned how to use Bower+Grunt it just works. Theres
good, already existing infrastructure providing nearly all the js
libraries and opensource communities maintaining it.

regards Jens
--
Klein & Partner KG, member of BlueDynamics Alliance

Sascha Gottfried

unread,
Mar 12, 2014, 9:24:24 AM3/12/14
to pylons-...@googlegroups.com
Thanks for sharing your experience. Obviously a lot of python developers (those adopting latest technologies) are shifting towards tools using the node.js/npm ecosystem to get this type of work done. If these developers were the maintainers of the various packages on PyPI using fanstatic/webassets, they will presumably not update these packages using latest versions available anymore. This could explain why I do not find a recent js.jquery version of PyPI.






--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/o5SB8r3jpUk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discuss+unsubscribe@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.



--
Mit freundlichen Grüßen
Sascha Gottfried

Gryphiusstr. 16
10245 Berlin
Mobil: 0177/3017020
Fon: 030/23464807

Randall Leeds

unread,
Mar 12, 2014, 2:34:34 PM3/12/14
to pylons-...@googlegroups.com
I still believe there is good utility to using fanstatic or webassets in the final step.

Build the assets with a node toolchain, serve them with pyramid_webassets or pyramid_fanstatic. The use is that having content hash in versions in query strings with far future headers is really great for caching and making sure you never serve stale assets or mixed old/new assets.


--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To post to this group, send email to pylons-...@googlegroups.com.

Sascha Gottfried

unread,
Mar 12, 2014, 4:27:49 PM3/12/14
to pylons-...@googlegroups.com
Thanks Randall,
that is absolutely true. I have this issue every time I deploy new versions and have it already as a backlog item. Thanks for pointing this out. I will give python solutions definitely a try before I switch to node.js ecosystem.


To unsubscribe from this group and all its topics, send an email to pylons-discus...@googlegroups.com.

To post to this group, send email to pylons-...@googlegroups.com.
Visit this group at http://groups.google.com/group/pylons-discuss.
For more options, visit https://groups.google.com/d/optout.

Jonathan Vanasco

unread,
Mar 12, 2014, 4:41:51 PM3/12/14
to pylons-...@googlegroups.com
After struggling with a decision for a while, I ended up just using fabric and building this into our deployment routine. 

Our pyramid app toggles different includes between Production and Testing (based on environment.ini variables):

- production 
   * uses compressed / minified / joined / filtered assets in /-dist
   * appends a release number as the query string to all assets ( to bust cache across releases )

- development
   * uses normal assets in /js /css
   * appends the datetime as the query string to all assets ( to bust cache across requests )

before joining and minifying the js files, they're heavily regexed/filtered.  a javascript flag is set to production, logger lines are commented out, and lots of other changes.  sometimes libraries are patched.

I also manage versioned assets like this:

  /js
  |-- /jquery
     |-- /active ( symlink to other folder in this directory )
     |-- /v1.8.1
     |-- /v1.8.5

the fabric deployment file is written to deploy off the /active symlink in each folder.  It occasionally needs to be rewritten/updated , as 3rd party libraries sometimes will change their file structure.  

a lot of the work just re-implements the html5boilerplate build tools ( originally in ant, now in node ).  html5boilerplate is installed into the virtualenv/source, and kept relatively up-to-date.  instead of running their commands natively, I use fabric as the exclusive interface.  that lets me move files around or do all the other work in Python.

It took about three hours to set up and has been really great.  Instead of having to deal with all sorts of evolving config files, everything stays in python.  

Randall Leeds

unread,
Mar 12, 2014, 4:45:20 PM3/12/14
to pylons-...@googlegroups.com
I feel like this is a great topic for a tutorial.

Next Python meetup project night here in San Francisco I might start a little scaffolding project to demonstrate some of this stuff.

Jonathan, if you have links to any code that'd be useful, could you drop some here?


--

Paul Winkler

unread,
Mar 12, 2014, 5:09:00 PM3/12/14
to pylons-...@googlegroups.com
FWIW there was a pretty good presentation at Pycon about webassets last year.
p.s. Hi Randall :)

Randall Leeds

unread,
Mar 12, 2014, 7:21:49 PM3/12/14
to pylons-...@googlegroups.com
Nice!

Hi, Paul :)

Steve Piercy

unread,
Mar 12, 2014, 8:53:51 PM3/12/14
to pylons-...@googlegroups.com

Randall,

Please look for me at tonight's SF Python presentation meet up at Yelp. I'll also be at next Wednesday's hack night for Pyramid in PyCharm. I'd like to see what you're doing.

--steve

Randall Leeds

unread,
Mar 13, 2014, 12:47:36 AM3/13/14
to pylons-...@googlegroups.com
Link to next week's hack night anywhere?
I'd totally love to work on something.

Whit Morriss

unread,
Mar 13, 2014, 2:32:16 AM3/13/14
to <pylons-discuss@googlegroups.com>
hacking in SF in the mission tomorrow morning if anyone is around.

-w

d. "whit" morriss
Systems Engineering / DevOpstetrics 





On Mar 12, 2014, at 9:47 PM, Randall Leeds <randal...@gmail.com>

Steve Piercy

unread,
Mar 13, 2014, 3:13:46 AM3/13/14
to pylons-...@googlegroups.com
It was announced tonight that the next SF Python Hack Night
would take place next Wednesday, March 19, at Uber. I heard
they just moved into the old Yelp HQ on Mission at Third Street,
in the same building as the Men's Wearhouse. They have not yet
posted details to the website, but it should go up in the next
day or two. If you join the group, you will be notified.
http://www.meetup.com/sfpython/

--steve


On 3/12/14 at 9:47 PM, randal...@gmail.com (Randall Leeds) pronounced:
------------------------
Steve Piercy, Soquel, CA

Steve Piercy

unread,
Mar 14, 2014, 5:09:54 PM3/14/14
to pylons-...@googlegroups.com
The next SF Python Hack Night has been announced:

Hope to see y'all there!

--steve
>>>> an email to pylons-discuss+unsubscribe@googlegroups.com.
>>>> To post to this group, send email to pylons-discuss@googlegroups.com.
>>>> Visit this group at http://groups.google.com/group/pylons-discuss.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>  --
>>> You received this message because you are subscribed to the Google Groups
>>> "pylons-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to pylons-discuss+unsubscribe@googlegroups.com.
>>> To post to this group, send email to pylons-discuss@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/pylons-discuss.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>--
>>You received this message because you are subscribed to the Google Groups
>>"pylons-discuss" group.
>>To unsubscribe from this group and stop receiving emails from it, send an
>>email to pylons-discuss+unsubscribe@googlegroups.com.
>>To post to this group, send email to pylons-discuss@googlegroups.com.

Sascha Gottfried

unread,
Oct 28, 2014, 6:16:52 AM10/28/14
to pylons-...@googlegroups.com
Today I found bowerstatic, successor of fanstatic, and want to highlight the history page of the docs. It is a great introduction into the topic and gives answers to some questions found in this thread.

>>>> an email to pylons-discus...@googlegroups.com.
>>>> To post to this group, send email to pylons-...@googlegroups.com.
>>>> Visit this group at http://groups.google.com/group/pylons-discuss.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>  --
>>> You received this message because you are subscribed to the Google Groups
>>> "pylons-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to pylons-discus...@googlegroups.com.
>>> To post to this group, send email to pylons-...@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/pylons-discuss.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>--
>>You received this message because you are subscribed to the Google Groups
>>"pylons-discuss" group.
>>To unsubscribe from this group and stop receiving emails from it, send an
>>email to pylons-discus...@googlegroups.com.
>>To post to this group, send email to pylons-...@googlegroups.com.

Sascha Gottfried

unread,
Oct 28, 2014, 10:51:46 AM10/28/14
to pylons-...@googlegroups.com
This is the announcement of the package from the author...
http://blog.startifact.com/posts/announcing-bowerstatic.html


--
You received this message because you are subscribed to a topic in the Google Groups "pylons-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pylons-discuss/o5SB8r3jpUk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pylons-discus...@googlegroups.com.

To post to this group, send email to pylons-...@googlegroups.com.
Visit this group at http://groups.google.com/group/pylons-discuss.
For more options, visit https://groups.google.com/d/optout.



--

Marc Rijken

unread,
Nov 5, 2014, 5:10:28 PM11/5/14
to pylons-...@googlegroups.com
I have made an integration of bowerstatic in Pyramid:
https://github.com/mrijken/pyramid_bowerstatic

It is not released yet, because it will need some testing first.

Marc

John Anderson

unread,
Nov 11, 2014, 10:42:45 AM11/11/14
to pylons-...@googlegroups.com

2014-10-28 15:51 GMT+01:00 Sascha Gottfried <sascha.g...@googlemail.com>:
> This is the announcement of the package from the author...
> http://blog.startifact.com/posts/announcing-bowerstatic.html
>
>
> On Tue, Oct 28, 2014 at 11:16 AM, Sascha Gottfried
> <sascha.g...@googlemail.com> wrote:
>>
>> Today I found bowerstatic, successor of fanstatic, and want to highlight
>> the history page of the docs. It is a great introduction into the topic and
>> gives answers to some questions found in this thread.
>>
>> Bowerstatic@readthedocs - History

On Wed, Nov 5, 2014 at 2:07 PM, Marc Rijken <ma...@rijken.org> wrote:
I have made an integration of bowerstatic in Pyramid:
https://github.com/mrijken/pyramid_bowerstatic

It is not released yet, because it will need some testing first.

Marc


Bower is a package manager, not as asset pipeline (bundler) system. In my experience getting the static assets that are required for my development and serving them isn't the important part. I don't see many people actually serving static assets from their WSGI application outside of development, at a minimum we use something like nginx to serve them and majority of the time we have them deployed to a CDN like fastly or akamai instead.

The important part of tools like webassets is that they can take a related list of assets (backbone, underscore, and zepto for example) and in develop serve them statically but for prod/deployment collect those static assets, minify them into a single file, and push them up to a CDN and generate the URL for the CDN.

As an example, if I declare the following bundle in my code:

Bundle(
  'js/zepto.js',
  'js/underscore.js',
  'js/backbone.js'
)

in development I want to see 3 static assets declare:

<script src="/static/js/zepto.js">
<script src="/static/js/underscore.js">
<script src="/static/js/backbone.js">

but when I'm deploying to production I want to say "collect my static assets and minify them" so that I can copy them up to my CDN, and then I expect a single URL in production:


My WSGI application will not have access to a javascript minifier or css minifier and it wont be serving any static assets itself.

Do you have a flow that solves this using bower + grunt, or are you shipping to production having your application server also serve up your static assets?

- John

 

Marc Rijken

unread,
Nov 11, 2014, 11:11:31 AM11/11/14
to pylons-...@googlegroups.com
My WSGI apps are serving the static assets. So there is no difference
in serving static assets between development and production.

I do use http://www.cloudvps.com/openstack/cdn-acceleration/, which is
a CDN which gets the assets from my apps. So my app serve the static
assets but only a small number of requests are actually served
directly by the app.

Note: minifying and bundling is not implement by bowerstatic yet. When
bowerstatic can do that automatically in the near future, static
assets are hassle free.

Marc

John Anderson

unread,
Nov 11, 2014, 12:45:54 PM11/11/14
to pylons-...@googlegroups.com
I still don't think just putting it behind a CDN as a cache quite solves the asset deployment issue. Using the same domain as your application is going to pass around all the cookies for that domain on every static asset request.  You have to be able to customize the root URL of static assets when it generates the URL in production.  This is why you see things like twimg.com and yimg.com, or just applications using akamai URLs directly.  It doesn't make much sense to use up application server workers (like gunicorn) to serve a CSS file, especially if you aren't using gevent because then you are going to have a static set of threads + processes that you've configured, especially since you rarely expose it directly and are usually sitting behind a proxy like nginx anyways.

Marc Rijken

unread,
Nov 11, 2014, 3:03:52 PM11/11/14
to pylons-...@googlegroups.com


Op 11 nov. 2014 18:45 schreef "John Anderson" <son...@gmail.com>:


>>
> I still don't think just putting it behind a CDN as a cache quite solves the asset deployment issue. Using the same domain as your application is going to pass around all the cookies for that domain on every static asset request.  You have to be able to customize the root URL of static assets when it generates the URL in production.  This is why you see things like twimg.com and yimg.com, or just applications using akamai URLs directly.  It doesn't make much sense to use up application server workers (like gunicorn) to serve a CSS file, especially if you aren't using gevent because then you are going to have a static set of threads + processes that you've configured, especially since you rarely expose it directly and are usually sitting behind a proxy like nginx anyways.
>

In my situation the CDN is behind a different domain name as the main site: cdn.example.com vs www.example.com. In bowerstatic you can define that the statics assets will have a different base than the regular pages.  Therefor cdn.example.com has no cookies. For me this is a nice solution, but will not be others.

Jonathan Vanasco

unread,
Nov 11, 2014, 8:47:21 PM11/11/14
to pylons-...@googlegroups.com

On Tuesday, November 11, 2014 3:03:52 PM UTC-5, Marc Rijken wrote:

In my situation the CDN is behind a different domain name as the main site: cdn.example.com vs www.example.com. In bowerstatic you can define that the statics assets will have a different base than the regular pages.  Therefor cdn.example.com has no cookies. For me this is a nice solution, but will not be others.

Just to be clear:

Do you mean that they're both on different subdomains?  ( cdn.example.com vs www.example.com )

That leading "www" subdomain is important.

On modern browsers, a cookie set on (or specified to) to "example.com" will be available to all subdomains "www.example.com", "cdn.example.com", "dev.www.example.com", etc

Earlier browsers required a trailing ".", but the RFC changed a while back.  

Marc Rijken

unread,
Nov 12, 2014, 3:13:21 AM11/12/14
to pylons-...@googlegroups.com
2014-11-12 2:47 GMT+01:00 Jonathan Vanasco <jvan...@gmail.com>:
>
> Just to be clear:
>
> Do you mean that they're both on different subdomains? ( cdn.example.com vs
> www.example.com )

Yes with www. But of course it can also be a different domain.

Jonathan Vanasco

unread,
Nov 12, 2014, 10:53:18 AM11/12/14
to pylons-...@googlegroups.com
Yes with www. But of course it can also be a different domain.

Great! Some people just don't realize that. Glad you do.

If you're really worried about theoretical optimal speeds, the approach I like is to use getting a dedicated domain that is 4-5 characters on a 2 letter extension (i.e. http://abcd.ef ).  You can still grab many for around $12/yr.  It covers the cookie issue and saves a couple bytes here and there. 
Reply all
Reply to author
Forward
0 new messages