Re: [Rails-core] Automatic asset paths prevent logical asset organization

[Ken Collins]

Hey Andrew,

I have always liked the automatic asset paths and thought they allowed the top level Rails applications as well as engine authors the ability to create namespaces via directories.

I read your whole email but kind of got lost. Is your main issue that creating a namespace via directories is a chore since it has to be done in 3 places?

Or a more concrete question. I am the author of the less-rails and the matching less-rails-bootstrap gems. I was very conscious about directory namespaces and made sure that my namespace for less-rails-bootstrap was "twitter/bootstrap". I am a firm believer that engines that deliver assets should follow this pattern. So the question, what would you suggest doing differently vs this setup?

https://github.com/metaskills/less-rails-bootstrap/tree/master/vendor/assets


- Ken

On Aug 4, 2012, at 12:50 PM, Andrew Vit wrote:

> In 2fe70c1 (last year), rails changed from a static list of assets.paths to automatically register any path under assets/*. (This was not reflected in Rails Guides so I pushed an update to docrails.)
>
> However, in my opinion, these automatic asset paths create confusion.
>
> The result is that the first-level directories under `(app|lib|vendor)/assets` get swallowed and if I put assets into a directory of my own naming, it will conflict with assets from another asset path. e.g.
>
> app/assets/images/icons/close.png
>
> vendor/assets/xyz/icons/close.png
>
> Here, the path segments "images" and "xyz" become asset path roots, so both of the above resolve as icons/close.png, and serve from /assets/icons/close.png. (If you didn't expect that, I should point out that the file from app will take precedence.)
>
> I don't know if I'm missing how this is expected to be used. I might expect app/assets/stylesheets, lib/assets/stylesheets, and vendor/assets/stylesheets to correspond and mask each other, but not a subdirectory of my own naming.
>
> Take for example a typical frontend widget I might want to download and install:
>
> foo/widget.js
> foo/styles.css
> foo/images/click.png
>
> If I want to keep this in a namespaced asset bundle instead of reorganizing it into separate javascripts/stylesheets/images directories, I can't just drop the folder into vendor/assets, as the asset paths will swallow the "foo" directory name, leaving the files naked in public/assets. In my manifests, these also automatically become available directly as "styles.css" and "widget.js" without a "foo/" prefix.
>
> I could arbitrarily put it into vendor/assets/javascripts/foo (or vendor/assets/whatever/foo), but that makes no sense: it's not all just javascripts, it's a whole package. Is it really the right solution to just nest things 2 levels down so as not to lose the subdirectory for namespacing?
>
> For vendor/assets packages, ideally I think the way to handle such packages is to define a "foo.js" and "foo.css" manifest to wrap the vendored package, and keep all its files scoped under its own directory, e.g.
>
> vendor/assets/foo.css # manifest file: //= require './foo/widget.css'
> vendor/assets/foo.js # manifest file: //= require './foo/widget.js'
> vendor/assets/foo/widget.js
> vendor/assets/foo/styles.css
> vendor/assets/foo/images/click.png
>
> (Alternately, the manifests could be index files inside the folder.)
>
> These would correspond to the public paths:
>
> /assets/foo/styles.css # not used directly when included through a manifest
> /assets/foo/widget.js # not used directly when included through a manifest
> /assets/foo/images/click.png
>
> This would make the package available in application.(css|js) as a simple "require foo"
>
> How best to configure rails for this? Should assets/(stylesheets|javascripts|images) be magically considered "global" or should the initializer for sub-paths under assets be dropped altogether, leaving it to the user to organize assets how they see fit?
>
> (Also, currently there's that somewhat confusing mismatch between app/assets and public/assets, where app/assets is organized into subdirectories for javascripts & stylesheets, and public is not -- defaulting to just assets as the config path instead of assets/* could fix that.)
>
> Are there any established usage patterns I'm missing, or should Rails consider something to organize this better by default?
>
> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/rubyonrails-core/-/dWp6EGpUJZUJ.
> To post to this group, send email to rubyonra...@googlegroups.com.
> To unsubscribe from this group, send email to rubyonrails-co...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/rubyonrails-core?hl=en.
>

[Andrew Vit]

Thanks Ken, I'll try to be more clear:

Asset gems often exist just to isolate those resources into a simple package to save everyone the trouble of reorganizing third-party files specially for the asset pipeline & maintaining that reorganization going forward. For some things the reorganization makes sense, and I won't disagree with the way you've packaged the less-bootstrap gem. +1 for namespacing under "twitter/bootstrap", but this isn't necessarily about that.

The issue is that for drop-in use of third-party assets (from plain folders, not gems), I think it's more logical to keep their files in one self-contained folder and namespace, without having to tease out and reorganize them. As a benefit, we can use a git submodule to update their whole package, and encapsulate access into it  using manifest files.

I think I'll end up settling on something like this:

    vendor/assets/images/      # empty

    vendor/assets/javascripts/ # empty

    vendor/assets/stylesheets/ # empty

    vendor/assets/packages/xyz/

    vendor/assets/packages/xyz/random.js

    vendor/assets/packages/xyz.js # manifest //= ./xyz/random.js

Using "packages" is there to nest everything one level down so I can isolate the "xyz" package into its own directory.

There doesn't seem to be any documentation or conventions for this so I wanted to open the discussion around how third-party assets should be handled for drop-in use.

My original point was that adding another folder of my own naming e.g. "assets/xyz" would shadow the standard folders: I don't think that's desired behaviour. I did update docrails to reflect how it currently works, but let's discuss if the docs were originally right and Rails currently isn't: https://github.com/lifo/docrails/commit/ad088f09

So, to be more concrete:

1. Should asset paths be limited to (images|javascripts|stylesheets) instead of * glob to prevent accidental shadowing? (Other asset paths can be added explicitly.)

2. If assets/* should remain, should Guides document how other directories in there are meant to be used? (e.g. just for adding extra file types like "flash")

3. Promote something like "vendor/assets/packages" as an optional convention, or bless it as a standard path for drop-in use of asset folders?

Andrew

[Rodrigo Rosenfeld Rosas]

I understand what you mean. For my gem oojspec and oojs I use the manifest approach you have described:

https://github.com/rosenfeld/oojspec/blob/master/app/views/oojspec/runner.html.erb#L8
https://github.com/rosenfeld/oojspec/blob/master/lib/assets/javascripts/oojspec.js.coffee
https://github.com/rosenfeld/oojspec/blob/master/vendor/assets/javascripts/buster/all.js.coffee

You'll notice I have submodule to third-party libraries (Buster.js):

https://github.com/rosenfeld/oojspec/tree/master/external/busterjs

Then I symlink the ones I'm interested in here (yes, I know this won't work on Windows, but I don't think I would get any contributions from Windows users anyway):

https://github.com/rosenfeld/oojspec/tree/master/vendor/assets/javascripts/buster


The same happens with oojs. Submodules:

https://github.com/rosenfeld/oojs/tree/master/resources

Linked assets:

https://github.com/rosenfeld/oojs/tree/master/vendor/assets/javascripts

Then the custom assets go into the lib directory instead of the vendor one:

https://github.com/rosenfeld/oojs/blob/master/lib/assets/javascripts/oojspec_helpers.js.coffee


But I do share your feeling that it should be easier to use third-library JavaScripts in Rails.

Maybe we could support something like AssetsDependencies similar to a Gemfile where you would specify the url to the assets or their repositories and what files should be required and in which order (or specify their dependencies) so that we could run some rake task to download them and write the manifest files. In that case we could add those files to the VCS or just do that dynamically before deploying like it happens with "rake assets:precompile".

I'd love to see something like that for Rails. Currently whenever I want to use some library with Rails I have to write my own gem when I don't find one, as it happened with this one:

https://github.com/rosenfeld/jquery-layout-rails

What do you think about the idea of Rails providing something like the AssetsDependencies DSL and a rake task as mentioned above?

Best,
Rodrigo.

[Andrew Vit]

Rodrigo, your symlink strategy is yet another interesting idea...

But I don't think reinventing Bundler for javascripts is really a solution to the problem. You can still wrap it in a gem and use Bundler for that.

[Rodrigo Rosenfeld Rosas]

This isn't about reinventing Bundler as they would have quite different
goals and strategy.

I already currently wrap my JS libraries/frameworks in gems when they
don't already exist but replicating every JS library as a gem containing
just the assets just doesn't feel right to me.

Also, while I spend about 10 minutes to vendor jQuery-layout as a gem, I
think it could take a single minute to do the same through a DSL like
I've described. Upgrading the library version would be even faster with
the DSL approach, specially if you're the gem's author.

Think about this. Suppose there are a thousand interesting JS
libraries/frameworks out there that it would worth to package as a gem.

Then suppose there are 5 Ruby web frameworks willing to extend their
frameworks with plugins packaged as a gem and all of them decide to
package the assets as a gem. Then you could possibly have 5,000 gems in
the Ruby world that actually contain no Ruby code at all, except for the
plugin glue code that could be different for each web framework.

All of that just to make it easier for users to use a JS library/framework.

[Andrew Vit]

On Monday, August 6, 2012 5:23:46 AM UTC-7, Rodrigo Rosenfeld Rosas wrote:

Then you could possibly have 5,000 gems in
the Ruby world that actually contain no Ruby code at all, except for the
plugin glue code that could be different for each web framework.

All of that just to make it easier for users to use a JS library/framework.

+1, I see this as a mild abuse of rubygems. There are already naming conflicts.