Comments please: Asset Scheme

[Rasmus Schultz]

I wrote a draft for a simple scheme for the inclusion of static assets in (Composer) packages.

https://gist.github.com/mindplay-dk/90507eb164e74bac7bbbf9abc97a04ee

Not submitting this or anything, just dumping it here to start a discussion :-)

Thoughts?

[Christopher Pitt]

Nice write-up. Do you think it would be possible to survey member projects (including recently departed projects, like Laravel), to determine things like the most common/preferred name for the webroot folder?

[Rasmus Schultz]

The webroot folder can have any name - a folder needs to be designated, I thought the spec was pretty clear on this point? :-)

On Oct 16, 2016 23:00, "Christopher Pitt" <cgp...@gmail.com> wrote:

Nice write-up. Do you think it would be possible to survey member projects (including recently departed projects, like Laravel), to determine things like the most common/preferred name for the webroot folder?

--
You received this message because you are subscribed to a topic in the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/php-fig/f4qtsS54mVY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to php-fig+unsubscribe@googlegroups.com.
To post to this group, send email to php...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/ad00f8f7-74f3-4a00-99aa-8b0909df7fa9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hari K T]

Thank you Rasmus Schultz for the nice write up.

After looking at it I think, it was something similar to what we had for Aura  https://github.com/friendsofaura/Aura.Asset_Bundle .

I recently added psr-7 support : https://github.com/harikt/psr7-asset ( Not saying it is perfect )

I like some of the good things you mentioned towards the end of "A component responsible for the delivery of vendor-supplied assets:"

https://gist.github.com/mindplay-dk/90507eb164e74bac7bbbf9abc97a04ee#12-asset-url-scheme

Also I have question : 

Why do you want the folder name to be named assets itself ? ( But I do agree it is good to have a folder not to expose other files ;-) )

What Aura.Asset_Bundle was using was mapping the folder with the package path. So it don't necessary to be in vendor folder you mentioned. It gives flexibility for the users to choose the path for their vendor specific css.

Eg : In our case you can alter the css / js behaviours of the vendor/a/b with your own if you are mapping to /web/project-specific-assets .

I do understand it needs some configuration though.

And thanks for bringing this up, and it will be nice feature to have for considering psr-7, psr-15 etc.

Thank you

Hari K T

You can ring me : +91 9388 75 8821

http://harikt.com , https://github.com/harikt , http://www.linkedin.com/in/harikt , http://www.xing.com/profile/Hari_KT

Skype  : kthari85

Twitter : harikt

--
You received this message because you are subscribed to the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to php-fig+unsubscribe@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/CADqTB_gt7%2BjPbRKUZ88-m%2Be3N1OanuC%3DOiMj2QNZ%3DfkqjtmDqg%40mail.gmail.com.

[Rasmus Schultz]

> Why do you want the folder name to be named assets itself ?

The folder has to have a name - "assets" seemed like the logical choice.

Perhaps what you're really wondering is, why a single folder and not a
map like in the Aura library?

Because it's simpler. A map would require more than a standard - it
would require at least a configuration file format specification,
and/or possible a library or interfaces.

Also, because we learned from doing something similar at work, that
being able to symlink vendored asset folders into the project's public
asset folder is really useful - it enables you to run an installation
script once, and the continue to add more files to the vendor
packages, since every file in the symlinked folder becomes
automatically available.

Another reason is that a map cannot be interpreted or resolved at
design-time, by the browser - things like source-maps of relative
paths fall apart, since the relative public URLs do not map directly
to physical files. This is perhaps the main reason we came up with
this approach - anything else has proven to be highly impractical to
work with. Having to run a build or deploy script between every change
is cumbersome.

Finally, the length or appearance of asset URLs is typically
completely irrelevant - about as irrelevant as the physical
file-system structure is to Composer packages. For the most part, no
person will ever see the URL of your CSS or JS files - wanting shorter
or neater URLs is purely vanity, it has no practical consequence.
Having a simple, predictable URL structure that prevents collissions,
is much more valuable than having pretty URLs.

Good question though, thanks for reviewing this :-)

>>> php-fig+u...@googlegroups.com.

>>> To post to this group, send email to php...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/php-fig/ad00f8f7-74f3-4a00-99aa-8b0909df7fa9%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "PHP Framework Interoperability Group" group.
>> To unsubscribe from this group and stop receiving emails from it, send an

>> email to php-fig+u...@googlegroups.com.

>> To post to this group, send email to php...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/php-fig/CADqTB_gt7%2BjPbRKUZ88-m%2Be3N1OanuC%3DOiMj2QNZ%3DfkqjtmDqg%40mail.gmail.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "PHP Framework Interoperability Group" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/php-fig/f4qtsS54mVY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

> php-fig+u...@googlegroups.com.

> To post to this group, send email to php...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/php-fig/CAESZFtK_vrVRKEVFHBENU2hdFxGEc8P%2Bk4HL2m2F3sB2PHRymg%40mail.gmail.com.

[David Négrier]

Hey Rasmus,

Interesting!

I have a few questions. It seems you are looking to keep the spec as simple as possible. But how do you plan to work with more complex "asset pipes" like minifying / concatenating / compiling LESS/SASS files...?

Also, @webmozart tackled the same problem in a very different way with Puli (http://docs.puli.io/en/latest/).

Have you had a look at it? Any reason not to use Puli?

++

David.

[Rasmus Schultz]

> Have you had a look at it? Any reason not to use Puli?

I have looked at it, and decided not to use it.

I had it in mind though, and I believe it could supported a standard
like this one, as an option, if they wanted to. (?)

It's no that there is anything wrong with it, per se - if what you
want is an asset management framework.

I don't though. I want something simple - something that doesn't marry
you to one vendor. A pattern that different vendors can implement very
easily, or even simple enough that a project can simply implement and
follow the pattern without using any library.

I just added a "meta" section to the document, it discusses some of
the reasons for keeping this simple:

https://gist.github.com/mindplay-dk/90507eb164e74bac7bbbf9abc97a04ee#meta

> how do you plan to work with more complex "asset pipes" like minifying / concatenating / compiling LESS/SASS files...?

Good question, I should discuss this in the meta section.

The short answer is, it's out of scope of this standard.

I had it in mind, of course - but basically any complex build system,
ranging from a compiler, up to complex pipelines with Grunt, WebPack,
and other build tools, ultimately result in a compiled, static asset.

In a sense, your SASS files aren't static assets - they require some
tools for processing, compilation, minification, etc.

The output of those tools are static assets.

So this standard deals with the delivery of the compiled, static assets only.

> https://groups.google.com/d/msgid/php-fig/0b271ca0-589b-4cfb-8c8b-61e080f1847a%40googlegroups.com.

[Sven Sauleau]

Hi,

It seems easier to discuss here.

Since this standard focus on publishing client-side assets. I think, assets pipeline and directory structure are out of scope for this standard.

I also commented the gist (https://gist.github.com/mindplay-dk/90507eb164e74bac7bbbf9abc97a04ee#gistcomment-1899260).

[Hari K T]

Hi Rasmus, 

Trying to write inline, so I don't miss your question.

> Why do you want the folder name to be named assets itself ?

The folder has to have a name - "assets" seemed like the logical choice.

Perhaps what you're really wondering is, why a single folder and not a
map like in the Aura library?

Because it's simpler. A map would require more than a standard - it
would require at least a configuration file format specification,
and/or possible a library or interfaces.

What we followed was just an array, no other format.

[

 'vendor/package/css/hello.css' => dirname(dirname(__DIR__)) . '/asset/css/test.css', 

 'vendor/package' => dirname(dirname(__DIR__)) . '/asset', 

]

 

Also, because we learned from doing something similar at work, that
being able to symlink vendored asset folders into the project's public
asset folder is really useful 

- it enables you to run an installation
script once, and the continue to add more files to the vendor
packages, since every file in the symlinked folder becomes
automatically available.

Symlink may be easy. I was not using symbolic link and the files were served via psr-7 response though. It lacks caching and stuffs like that for now.

 

Another reason is that a map cannot be interpreted or resolved at
design-time, by the browser - things like source-maps of relative
paths fall apart, since the relative public URLs do not map directly
to physical files. This is perhaps the main reason we came up with
this approach - anything else has proven to be highly impractical to
work with. Having to run a build or deploy script between every change
is cumbersome.

Ya, the approaches choosen to serve the files is a bit different. You follow symlink, we followed directly looking at filesystem and reading the contents and serving as psr-7 Response .

 

Finally, the length or appearance of asset URLs is typically
completely irrelevant - about as irrelevant as the physical
file-system structure is to Composer packages. For the most part, no
person will ever see the URL of your CSS or JS files - wanting shorter
or neater URLs is purely vanity, it has no practical consequence.
Having a simple, predictable URL structure that prevents collissions,
is much more valuable than having pretty URLs.

I have no trouble with the length of the url. Not sure if someone else asked the same.

[Fabien Potencier]

I disagree. I agree that nobody cares about the FS structure of the
Composer packages; but for assets, that's a bit different, security
wise. Having a direct correlation between the asset paths and the
package names means that you are leaking some interesting/"sensitive"
information for a potential hacker.

Fabien

[Rasmus Schultz]

> Having a direct correlation between the asset paths and the package names means that you are leaking some interesting/"sensitive" information for a potential hacker.

How so?

The only way I can see your vendor/package name as "sensitive
information", is if you have a very serious security problem somewhere
else - otherwise, exposing the vendor/package-name should not be a
concern. It's a name. It means nothing unless you exposed the vendor
folder to the public or something - in which case the problem wasn't
created here and shouldn't affect the design, IMO.

> https://groups.google.com/d/msgid/php-fig/05c76270-d682-7f65-d611-f37af67c1a8a%40gmail.com.

[Rasmus Schultz]

> What we followed was just an array, no other format.

The point is, this requires server-side bootstrapping. I can't add a
new assets without also writing code, so already here you've created a
dependency on some sort of server-side global bootstrapping facility.
I am to keep things simpler than that.

> You follow symlink, we followed directly looking at filesystem and reading the contents and serving as psr-7 Response .

Symlinks are optional - from the spec:

"This specification does not stipulate how this is implemented.
Examples include: proprietary web-server configuration files, e.g.
Apache, NGINX, or other web-server software, front controllers,
middleware, symbolic links created by installation scripts, or any
other means."

You can absolutely use the file-system and serve contents via PSR-7
response. (I don't plan to do that myself, as dispatching PHP just to
deliver a static asset is not necessary if all your static assets are
pre-baked and checked into source-control - we strictly view things
like compilation and serialization as design-time rather than run-time
tasks, which seems to make life easier for our front-end developers.
YMMV)

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "PHP Framework Interoperability Group" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/php-fig/f4qtsS54mVY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> php-fig+u...@googlegroups.com.
> To post to this group, send email to php...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/php-fig/CAESZFt%2BVK3ORC7pB-7iDd0j5-kZZ1TZ06ruoRmVzy5tswiy9Tg%40mail.gmail.com.

[Fabien Potencier]

On 10/17/16 07:54, Rasmus Schultz wrote:
>> Having a direct correlation between the asset paths and the package names means that you are leaking some interesting/"sensitive" information for a potential hacker.
>
> How so?

If assets are under twigphp/twig or symfony/symfony, it gives
information about which stack I'm using. I'm not saying that per se,
this is enough, but it gives some interesting hints.

[Rasmus Schultz]

Er, but, even if you mangled the filenames, all anyone would need to
do is checksum the contents of the files, and they'd known which
vendor libraries you're using, like, immediately.

There are blatantly simple ways to sniff most popular client and
server-side libraries. Hiding package names amounts to nothing more
than security by obscurity.

If you don't trust the packages you're using, or don't trust the
people using it, IMHO, you have much bigger (unrelated) problems...


On Mon, Oct 17, 2016 at 5:13 PM, Fabien Potencier

> https://groups.google.com/d/msgid/php-fig/9b8ba3b5-bfa3-72e1-7270-17fdad9ab73c%40gmail.com.

[Sven Sauleau]

Using this standard, people can know what packages you are using because of its predictable paths. Some packages are running server-side code as well as exposing public assets.

I said (in the comments of the gist) that exposing stuff is the responsibility of the developer. I’m sure some people will be confused and disclose informations they shouldn’t (as mention by Fabien).

[Fabien Potencier]

I'm not saying that we should not do it (and in fact, we are doing
something similar in Symfony), just that we should mention this
somewhere. It's important to mention any potential security issue (even
if it is small) so that developers can take a conscious decision.

[Rasmus Schultz]

A lot of people do a lot of things wrong. In my opinion, it's better to create simple things that are easy to learn to use correctly - as opposed to creating complex things that supposedly shield you from making mistakes. Often such things provide only a false sense of security - and usually you can break them and hurt yourself anyhow.

Most server side components have some kind of client side footprint - URLs or HTML class names etc which can be used to figure out what you're running.

I really don't think it's within the scope of this specification to teach OWASP 101? Package names are obviously revealed in the URLs - it's hardly a hidden detail, it's basically the whole concept.

In my opinion, there is no security problem inherent in this idea, unless you create one.

--
You received this message because you are subscribed to a topic in the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/php-fig/f4qtsS54mVY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to php-fig+unsubscribe@googlegroups.com.

To post to this group, send email to php...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/c2ba50bb-ad74-4675-8ccd-08cdd8a360a0%40googlegroups.com.

[David Négrier]

Hey Rasmus,

Another thing came to my mind. Your proposal looks awfully similar to Rob Loach's component system.
Have you had a look at it? => http://robloach.github.io/component-installer/
Any thoughts about that? Shouldn't we consider building on this existing example?

The difference is that Rob Loach has an implementation of a Composer installer instead. So rather than working on a specification, he directly created an implementation. Which is actually fine because you don't need a framework for your assets to be copied (they are copied/linked by the component-installer composer plugin that is a dependency of your asset package).
What would be the value of an Asset-PSR (if you want to build a PSR out of it), instead of Rob's direct implementation?

Best regards,
David.

Le lundi 17 octobre 2016 21:57:20 UTC+2, Rasmus Schultz a écrit :

A lot of people do a lot of things wrong. In my opinion, it's better to create simple things that are easy to learn to use correctly - as opposed to creating complex things that supposedly shield you from making mistakes. Often such things provide only a false sense of security - and usually you can break them and hurt yourself anyhow.

Most server side components have some kind of client side footprint - URLs or HTML class names etc which can be used to figure out what you're running.

I really don't think it's within the scope of this specification to teach OWASP 101? Package names are obviously revealed in the URLs - it's hardly a hidden detail, it's basically the whole concept.

In my opinion, there is no security problem inherent in this idea, unless you create one.

On Oct 17, 2016 9:30 PM, "Sven Sauleau" <sven.s...@gmail.com> wrote:

Using this standard, people can know what packages you are using because of its predictable paths. Some packages are running server-side code as well as exposing public assets.

I said (in the comments of the gist) that exposing stuff is the responsibility of the developer. I’m sure some people will be confused and disclose informations they shouldn’t (as mention by Fabien).

Le lundi 17 octobre 2016 16:43:25 UTC+9, Rasmus Schultz a écrit :

I wrote a draft for a simple scheme for the inclusion of static assets in (Composer) packages.

https://gist.github.com/mindplay-dk/90507eb164e74bac7bbbf9abc97a04ee

Not submitting this or anything, just dumping it here to start a discussion :-)

Thoughts?

--
You received this message because you are subscribed to a topic in the Google Groups "PHP Framework Interoperability Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/php-fig/f4qtsS54mVY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to php-fig+u...@googlegroups.com.

[Matthieu Napoli]

Hi Rasmus,

I was going to mention Puli too. Not because of all its features, but because it contains a very simple solution to a wider problem: locating resources.

Puli encourages to have a `res/` directory containing all resource files (just like `src/` contains all source files). Resources include assets, configs, templates, etc. Because the problem with everything that is not a source file is "how to locate it" (for source files we have the autoloader).

Once an application can locate assets, it can publish them, minify them, etc. The standard would avoid deciding what to do with assets (whereas your proposition only solves how to expose them, which is quite restrictive). The less the standard expects the better.

Matthieu

[Rasmus Schultz]

> Your proposal looks awfully similar to Rob Loach's component system.

With some major differences.

First, the URL scheme (base path) is variable - which means you need
server-side code (or server-generated code provoding client-side
bootstraping).

Also, this does more than just deliver assets - it explicitly
distinguishes CSS and JS from other assets, providing support for
serialization, which makes it an asset manager.

It also requires configuration, which would tie it (minimally) to a
specification, but more likely to an implementation. (or a
specification detailed enough that it's basically pseudo-code in
disguise.)

It's also based on the idea of explicitly packaging assets as separate
packages - whereas my proposal aims to allow for the inclusion of
assets into packages/projects. This means you have to set the package
"type" to "component" in your "composer.json" - which may force you in
some cases to package client and server-side dependencies separately,
which has implications in terms of dependency management, e.g. being
able to include my project's assets in the project, versus having to
version it as two separate packages and manage counter-breaking
changes between them, even for assets that are meaningless without the
server-side component, and vice-versa.

Rob's library looks nice. I aim for something simpler, that's all.

> https://groups.google.com/d/msgid/php-fig/38ad2ac4-ec1c-49b5-9988-1f5dcc9ee993%40googlegroups.com.

[Rasmus Schultz]

> I was going to mention Puli too. Not because of all its features, but because it contains a very simple solution to a wider problem: locating resources.

If you think Puli is simple, we can agree to disagree.

I make a sharp distinction between simple and easy - Puli aims to make
dependency management easy, but doing everything Puli does for assets
requires a complex system. It's not simple.

Don't get me wrong - the complexity in Puli is justified for all the
tasks it makes easy: mapping, locating, publishing, minifying,
serializing, etc.

The problem is, I don't have any of those requirements - so it solves
a bunch of problems that I don't have.

> your proposition only solves how to expose them, which is quite restrictive

From your point of view, this is restrictive, because you have all of
those requirements.

From my point of view, it's liberating, because I have none of those
requirements.

Or rather, I choose to address them at a different level - I don't
view tasks such as compiling, minifying, serializing, etc. as run-time
tasks at all. I view them strictly as design-time tasks - I perform
those tasks ahead of run-time, and ship the ready, published,
minified, serialized assets as pre-baked static files that I check
into source-control.

There is also the case where scripts receive no design-time processing
and are simply hand-written and checked in. For that scenario, having
any sort of run-time overhead or server-side complexity is completely
meaningless.

Standardizing for the simple case is simple.

If you wanted to standardize the far more complex case, that's going
to require an entirely different, far more complex standard.

> https://groups.google.com/d/msgid/php-fig/e4f0f008-ab9f-4592-8d6e-edab03812447%40googlegroups.com.

[Matthieu Napoli]

Ugh I didn’t say that :( What I said with more emphasis:

Not because of all its features,

(because I agree Puli is very complex)

but because it contains a very simple solution to a wider problem: locating resources.

(“contains” ≠ “is”)

You want to standardize a assets/ directory, I’m suggesting to standardize the res/ directory instead (which would contain assets).

And it’s not more complex, I would argue it’s even simpler because instead of creating discrepancies between different resources (assets, templates, config…) it would all have the same behavior.

The problem is, I don’t have any of those requirements - so it solves a bunch of problems that I don’t have.

What people do with assets is a real question. Either the PSR addresses the major use cases, or it avoids addressing any (both solutions are good but the last one is obviously simpler).

Matthieu

To view this discussion on the web visit https://groups.google.com/d/msgid/php-fig/CADqTB_jMYt_fM7Fk2JbFYwTsf%2B6LJR-0ykmwKi9gnCXjWT2A5g%40mail.gmail.com.

[Rasmus Schultz]

> I would argue it’s even simpler because instead of creating discrepancies between different resources (assets, templates, config…) it would all have the same behavior.

What "behavior"?

Public, static assets do precisely one thing - they get delivered to the client.

Server-side resources such as templates and configuration-files are
not public assets, they're something else entirely - why would you mix
those concerns?

This specification deals with the delivery of public, static assets -
the problem you're bringing up, server-side resources, is an entirely
separate problem. Yes, they're both files, and no, neither are PHP
class-files you can autoload, but that's about where the similarities
end.

One set of files are client-side dependencies, another set of files
server-side dependencies.

One affects behavior (configuration, templates), and the other does not.

One gets delivered to the client-side, and the other does not.

I don't see any practical reason to construe these needs into one
problem - apart from involving files, IMO, there is no real overlap
between these problem spaces.