Re: [jplatform] JDocument Registering JS Files for Easy Dependency Management

[Niels Braczek]

Am 07.09.2012 17:28, schrieb Donald Gilbert:

> What I would like to propose and gauge interest for is extending the
> JDocument class with the following methods:
> registerScript($name, $path, $dependencies = null, $overwrite = false);
> unregisterScript($name);

In general, I think dependency management is a good thing to do.
I would though prefer a kind of manifest file solution, so the
dependency description is separated from the source code.

Regards,
Niels

--
| http://barcamp-wk.de · 2. Barcamp Westküste Frühjahr 2013 |
| http://www.bsds.de · BSDS Braczek Software- und DatenSysteme |
| Webdesign · Webhosting · e-Commerce · Joomla! Content Management |
------------------------------------------------------------------

[Donald Gilbert]

Trying to understand your position. Could you elaborate a little on possibly setting script dependencies in a manifest?

What I can think of is maybe a field that basically states what scripts this component requires on which views. Something like

<views>

    <view name="list">

        <script name="jquery" src="{hosted CDN jquery src}" />

        <script name="datatables" src="{hosted CDN datatables src" />

    </view>

    <view name="detail">

       etc...

    </view>

</view>

Am I understanding that right? It seems to me that might be a little complicated (unless I'm way off base) to manage named dependencies across components.

[Niels Braczek]

Am 08.09.2012 15:34, schrieb Donald Gilbert:

> I'm not to keen on that approach, because you remove all ability to use CDN
> versions, if the builder just pulls the scripts and then compiles them.
> Especially since each view can have any number of different js libraries
> associated with it.

Doesn't matter. You may have a look at Composer, which uses JSON files.
See the composer file for Symfony 2.1 as an example.

It defines the requirements. These are independent from the view, the
scripts are used in. Maybe you get the idea with an example:

{
"name": "DatePicker",
"description": "Creates a Picker, which can be used for anything",
"require": {
"MooTools/Core/Element.Dimensions": "1.4.*",
"MooTools/Core/Fx.Tween": ">=1.4.0",
"MooTools/Core/Fx.Transitions": ">=1.4.1,<1.5-dev"
},
"provide": {
"Picker": "1.0.1-dev"
}
}

In your view, you just say you want to use the picker:

JHtml::_('behavior.picker', '>=1.0.0');

and the builder does all the rest. It's up to the builder to know (from
its configuration) where to get the required elements from.

[Niels Braczek]

Am 08.09.2012 17:25, schrieb Donald Gilbert:

> and theres the real issue... what does JHtml::_('behavior.whatever') even
> mean?

You want to use 'whatever' on the current page.

> Why are we involving JHtml?

Because JavaScript makes sense in HTML pages, but not on paper. Maybe
there are better places (JDocument coming in mind), but that's a
secondary question for now.

> I've never agreed with that approach, as
> it doesn't really make any sense symantically.

??

> How are you supposed to know
> what JHtml::_('behavior.whatever') even does if you a newcomer to the
> Joomla platform?

It does 'whatever'.

> The code doesn't explain itself just by reading.

Well, I think it does. It only hides any dependencies between all the
JavaScript files / class / frameworks, which is desirable.

> Following
> that logic, should we drop all JDocument::addScript and
> addScriptDeclaration methods in favor of the JHtml::script method?

Yes, that would be best. As mentioned, JDocument might be a better home
for this than JHtml.

> However, registering scripts to the Document head using
> JDocument::registerScript is very clear and self explaining.

The registerScript approach requires the developer to know the structure
and all dependencies of the JavaScript stuff. Changes in the
JS-framework's structure would require all views to be changed, that use
them. That's not ideal.

[Donald Gilbert]

The registerScript approach requires the developer to know the structure
and all dependencies of the JavaScript stuff. Changes in the
JS-framework's structure would require all views to be changed, that use
them. That's not ideal.

I don't think we are looking at this from the same angle. It doesn't require a developer to know anything except what their code requires to run. If I'm using something that requires jQuery, I add jQuery as a dependency. Using addScript to load in jQuery leads to multiple versions on the same page, wasting resources. The proposed code would function much like the current addScript method, which extension developers use to add scripts to the document for their views. Only instead of directly adding a script, they register a script with a name and src. Say I wanted to add jQuery Cycle to my view. That specific script depends on jQuery. Instead of adding both jQuery and then jQuery cycle, I would simply use this example code:

JDocument::getInstance()->registerScript('cycle', '{path to cycle}', 'jquery');

Then, the script gets registered to the document, and when the head is rendered, the jQuery library is loaded first, and then the jQuery Cycle library, since I declared the dependency.

You could also use this method to override a script src. Say I wanted to use the CDN hosted version of jQuery as a site developer or resolve conflicting/multiple versions. Well I would add a call to my themes index file like this

JDocument::getInstance()->registerScript('jquery', '{cdn hosted path}', null, true);

Then, every extension that had utilized the registerScript dependency of jquery would now load my specified version of jQuery instead of their own.

The main goal of this is the ability to easily override script src's for just this type of situation. It will help handle the "loading multiple version of jQuery" problem that we see time and time again in the forums and mailing lists.

[Niels Braczek]

Am 10.09.2012 07:32, schrieb Donald Gilbert:

> I wrote:

>> The registerScript approach requires the developer to know the structure
>> and all dependencies of the JavaScript stuff. Changes in the
>> JS-framework's structure would require all views to be changed, that use
>> them. That's not ideal.

> I don't think we are looking at this from the same angle.

Obviously.

> It doesn't
> require a developer to know anything except what their code requires to
> run. If I'm using something that requires jQuery, I add jQuery as a
> dependency. Using addScript to load in jQuery leads to multiple versions on
> the same page, wasting resources.

This only happens, because the developer is given a choice, where the
site builder should havce decided. It is *not* up to the extension or
template developer to decide, which script code should be loaded and
where from.

> Only instead of directly adding a
> script, they register a script with a name and src. Say I wanted to add
> jQuery Cycle to my view. That specific script depends on jQuery. Instead of
> adding both jQuery and then jQuery cycle, I would simply use this example
> code:
>
> JDocument::getInstance()->registerScript('cycle', '{path to cycle}',
> 'jquery');

Well, the behavior-like solution would say

JHtml::_('behavior.cycle');
or, let's say,
$document->behavior('cycle');

in an ideal world not even knowing if MooTools or jQuery is used on this
particular site.

> Then, the script gets registered to the document, and when the head is
> rendered, the jQuery library is loaded first, and then the jQuery Cycle
> library, since I declared the dependency.

As an extension developer, it is *impossible* to know, which other
extensions will be installed on the same site, and which problems that
may leed to. It is up to the site builder to decide, which script
versions are used, and if they are stored locally or not.

> JDocument::getInstance()->registerScript('jquery', '{cdn hosted path}',
> null, true);

See the problem? This line is written by the developer, but at least
{cdn hosted path} is decided by the site builder. The registerScript
approach enforces the site builder to change extension code, which
should not be necessary.

> Then, every extension that had utilized the registerScript dependency of
> jquery would now load my specified version of jQuery instead of their own.

I don't see that. Let's say one module uses

$doc->registerScript('jquery', 'media/js/jq-1.1.0.js', null, true);

a second uses

$doc->registerScript('jquery', 'media/js/jq-1.8.0.js', null, true);

and a third

$doc->registerScript('jquery', 'media/js/jq-1.4.0.js', null, true);

Who wins? Which version is used?

> The main goal of this is the ability to easily override script src's for
> just this type of situation. It will help handle the "loading multiple
> version of jQuery" problem that we see time and time again in the forums
> and mailing lists.

I'm afraid, it will not.

Regards,
Niels

BTW: Don't Cc: me. That leads to the 'private' conversation, we had,
since the list mail and the cc mail look identically, but one answers
the list, and the other you.

[Donald Gilbert]

> It doesn't
> require a developer to know anything except what their code requires to
> run. If I'm using something that requires jQuery, I add jQuery as a
> dependency. Using addScript to load in jQuery leads to multiple versions on
> the same page, wasting resources.

This only happens, because the developer is given a choice, where the
site builder should havce decided. It is *not* up to the extension or
template developer to decide, which script code should be loaded and
where from.

So it's the site builders job to decide what scripts get loaded from where. So I as a developer am not supposed to say what scripts my code requires?

 

> Only instead of directly adding a
> script, they register a script with a name and src. Say I wanted to add
> jQuery Cycle to my view. That specific script depends on jQuery. Instead of
> adding both jQuery and then jQuery cycle, I would simply use this example
> code:
>
> JDocument::getInstance()->registerScript('cycle', '{path to cycle}',
> 'jquery');

Well, the behavior-like solution would say

    JHtml::_('behavior.cycle');
or, let's say,
    $document->behavior('cycle');

in an ideal world not even knowing if MooTools or jQuery is used on this
particular site.

So I'm just supposed to call the cycle script without declaring where the src should come from or that it requires jQuery?

Does the solution you are referring to already exist? I'd love to look at some examples.

 

> Then, the script gets registered to the document, and when the head is
> rendered, the jQuery library is loaded first, and then the jQuery Cycle
> library, since I declared the dependency.

As an extension developer, it is *impossible* to know, which other
extensions will be installed on the same site, and which problems that
may leed to. It is up to the site builder to decide, which script
versions are used, and if they are stored locally or not.

I agree it's impossible to know, that's why we need this solution so that the site builder CAN decide what to use without changing code in extensions.

 

> JDocument::getInstance()->registerScript('jquery', '{cdn hosted path}',
> null, true);

See the problem? This line is written by the developer, but at least
{cdn hosted path} is decided by the site builder. The registerScript
approach enforces the site builder to change extension code, which
should not be necessary.

> Then, every extension that had utilized the registerScript dependency of
> jquery would now load my specified version of jQuery instead of their own.

I don't see that. Let's say one module uses

    $doc->registerScript('jquery', 'media/js/jq-1.1.0.js', null, true);

a second uses

    $doc->registerScript('jquery', 'media/js/jq-1.8.0.js', null, true);

and a third

    $doc->registerScript('jquery', 'media/js/jq-1.4.0.js', null, true);

Who wins? Which version is used?

Like CSS, the last one declared will be used. Since the document head is rendered last after all the extensions have had their say, declarations in the index.php would take precedence over all others.

 

> The main goal of this is the ability to easily override script src's for
> just this type of situation. It will help handle the "loading multiple
> version of jQuery" problem that we see time and time again in the forums
> and mailing lists.

I'm afraid, it will not.

I'm afraid, it will. (see, we can both make points without substantiating them :P)

 

Regards,
Niels

BTW: Don't Cc: me. That leads to the 'private' conversation, we had,
since the list mail and the cc mail look identically, but one answers
the list, and the other you.

I didn't CC you. You are the one you first sent me a direct email, just so you know. The 'private' conversation started when you failed to correctly reply to my second message (the third one on the list).

[Donald Gilbert]

I missed one.

> JDocument::getInstance()->registerScript('jquery', '{cdn hosted path}',
> null, true);

See the problem? This line is written by the developer, but at least
{cdn hosted path} is decided by the site builder. The registerScript
approach enforces the site builder to change extension code, which
should not be necessary.

Actually, this DOESN'T require site builder to change extension code. You change the code in your templates index (or from a panel in the admin, I don't care).

[Donald Gilbert]

Does anyone else have a stance on this? Is it a good idea or something that's needed? Gauging interest for an idea isn't really possible with only 2 participants, 1 for and 1 against.

[Niels Braczek]

Am 10.09.2012 15:27, schrieb Donald Gilbert:

> So it's the site builders job to decide what scripts get loaded from where.
> So I as a developer am not supposed to say what scripts my code requires?

Exactly. Because you can't know, which - if any - CDN the site owner
wants to use. As a developer, you only say, that you need the 'cycle'
function. In fact you shouldn't even bother, how it is implemented, but
just 'talk' to the API. Don't get me wrong: We'ree far from being there,
but that's where the journey should go.

> So I'm just supposed to call the cycle script without declaring where the
> src should come from or that it requires jQuery?

Yes, The dependencies of 'Cycle' have to be declared in context of that
behavior/plogin/you_name_it.

> Does the solution you are referring to already exist? I'd love to look at
> some examples.

These mechanisms are used in PHP frameworks, fx. Symfony (using
Composer). In your code, you say you want to use an 'HTTP Request', the
framework does the rest.

> I agree it's impossible to know, that's why we need this solution so that
> the site builder CAN decide what to use without changing code in extensions.

That's why the registerScript approach does not work. It is code, not
configuration.

>> Who wins? Which version is used?
>
> Like CSS, the last one declared will be used. Since the document head is
> rendered last after all the extensions have had their say, declarations in
> the index.php would take precedence over all others.

But then any source path or version declaration is useless.

>> I'm afraid, it will not.
>
> I'm afraid, it will. (see, we can both make points without substantiating
> them :P)

I thought, I explained, why the version problem will not be solved with
registerScript.

Regards,
Niels

[Niels Braczek]

So you want to load any scripts ever needed on all pages? I'd prefer
each single module or plugin to ask for the functionality it needs,
regardless of how that functionality is implemented.

Regards,
Niels

[Don Gilbert]

You don't load it on every page. Registering a script is not the same as addin a script to the head. Your simply registering the name and location o the script. After thinking about it, You would call something like $document->behavior(name) to actually enqueue the script for inclusion.

Sent from my iPhone

[pollen8]

would not advocating a something like Require.JS (http://requirejs.org/)  be a better solution?

[Niels Braczek]

Am 10.09.2012 19:24, schrieb Don Gilbert:

> You don't load it on every page. Registering a script is not the same
> as addin a script to the head. Your simply registering the name and
> location o the script. After thinking about it, You would call
> something like $document->behavior(name) to actually enqueue the
> script for inclusion.

Ok, so if I understand you right, the difference is, that registerScript
uses PHP to define the dependencies within the template code, where my
solution uses separate config files. Right?

So, the question is: Are script dependencies template specific? If they
are, I'm fine with the coupling with the template. Otherwise (and that's
what I see now), if the dependecies are installation specific, the
information belongs to a config file.

[Niels Braczek]

Am 12.09.2012 00:47, schrieb pollen8:

> would not advocating a something like* *Require.JS (http://requirejs.org/)
> be a better solution?*

Depends ;)
Donald's proposal puts the dependency information into the template code
(PHP).
Require.JS puts it into the JS code (leading to modification of used
scripts).

Let's elaborate a simple use case: Usage of a slider.
I'm going to describe, how it IMHO should be ideally.

Now there are different roles with different needs.

1. The extension developer.
The extension developer uses markup according to the API doc, where a
slider *could* be used, eg.
<div class="slider">
<div class="slide">...</div>
<div class="slide">...</div>
<div class="slide">...</div>
</div>
Of course, the extension works without JS.

2. The plugin developer
The plugin developer provides a JS, that grabs any element with a class
"slider" and presents the contained "slide" elements.
The plugin developer provides a manifest file declaring the
dependencies, so the system(builder) can resolve them.

3. The site builder
The site builder decides, *which* slider solution has to be used (and on
which pages). It is up to the site builder to provide base scripts, so
that the system can resolve the dependencies, so everything plays nice
together. This can be done with local files or CDN access.

Resume:
Only the site builder has the information, which things should play
together, which slider or lightbox solution is used. So only the site
builder is able to pick the right base files to satisfy all the plugins
used.
On the other hand, a site builder should never be enforced to touch
code. (I know: reality differs, but that's not how it should be).

Therfore, the dependency information
- does not belong to the template
- does not belong to a JS plugin
- does belong to the system configuration

[Donald Gilbert]

In a perfect world, that would be ideal.

However, how many extension developers do you know that build sliders that are js agnostic, leaving the choice of js framework up to the end user? I don't know of any. And I don't think that this is how end users want it. They want to be able to buy an extension, plug it in, and have it all work. That frame of mind (which IS the majority of what I see) kills this implementation.

Also, dependency information is not contained in the template in my proposal. You can do a final override within the template head if you desire, but it is not required. The extension developers would register all their scripts and dependencies using registerScript, and then when they are all ready, they would call $document->behavior('cycle') which would enqueue all the scripts that cycle depends on to the document head for rendering, and then when the head gets rendered, any calls to registerScript in the template index would be processed to overwrite any script src they desire. (although it is not required)

You are right on the point that our solutions in reality only differ on the point of where is dependency declared. You say in a config file, that at some point get's imported by the builder, and then processed (I'm assuming by PHP) to enqueue all the required scripts to the document head. I suggest declaring the script dependency in PHP, bypassing the config file. Would you say that's a fair assessment?

If so, what builder exists to import these config files and do that heavy lifting? I think the closest thing Joomla has to that is JDocument. But JDocument doesn't currently have the ability to sequence required scripts except - they are included in the order that addScript is called while a page is being rendered. I dare say that a registerScript method could be a stepping stone to this more ideal implementation wherein the config file holding the dependencies will be imported and then processed by the builder using the registerScript method. Then developers (eventually) have the choice to either directly declare their dependencies, or declare them in a config file.

Also, I don't like the term "behavior" for what we are doing. Maybe JDocument could have an enqueueScript function that takes a string name and includes the named script with any dependencies.

[Niels Braczek]

Am 12.09.2012 15:47, schrieb Donald Gilbert:

> In a perfect world, that would be ideal.
>
> However, how many extension developers do you know that build sliders that
> are js agnostic, leaving the choice of js framework up to the end user? I
> don't know of any.

Unfortunately, I don't neither. Nevertheless, everything we do should
get us nearer to the ideal, shouldn't it?

> And I don't think that this is how end users want it.
> They want to be able to buy an extension, plug it in, and have it all work.
> That frame of mind (which IS the majority of what I see) kills this
> implementation.

Not necessarily. The extension developer would provide the information,
that the extension supports sliders and tabs, eg.

supports: {
slider,
tabs
}

or, in a less ideal case, that the extension requires it

requires: {
slider,
tabs
}

The (Joomla!-) installer gives this information to the site builder, so
the site builder can decide, if he wants to install a slider plugin (in
case of 'supports') and which. Look at the PEAR installer or Composer
for an example for such a mechanism.

> Also, dependency information is not contained in the template in my
> proposal. You can do a final override within the template head if you
> desire, but it is not required. The extension developers would register all
> their scripts and dependencies using registerScript,

Ok so far. Some questions:

1. What happens, if one extension developer thinks, that "XYSlider"
(depending on jQuery 1.2) is so cool, another developer requires
"ABCSlider" (depending on jQuery 1.8), and a third wants "MooSlider"
(depending on MooTools 1.4)?

Currently, all three frameworks will be loaded, leading to three
different solutions for the same problem, and having negative impact on
the user experience (loading time and broken corporate identity).

With my approach, the extension developer simply does not have that
choice. He can just request a slider.

How would that be solved with your approach?

2. What happens, if a site builder wants to use "SuperSlider" instead of
the "UberSlider" he used until now?

Currently, the site builder does not have that choice.

With my approach, the sitebuilder just changes the slider entry in the
config file.

How would that be solved with your approach?

> You are right on the point that our solutions in reality only differ on the
> point of where is dependency declared. You say in a config file, that at
> some point get's imported by the builder, and then processed (I'm assuming
> by PHP) to enqueue all the required scripts to the document head. I suggest
> declaring the script dependency in PHP, bypassing the config file. Would
> you say that's a fair assessment?

Yes, that is it, but not all (see question above).

> If so, what builder exists to import these config files and do that heavy
> lifting?

Don't know - haven't searched for a thing like that until now. But there
are solutions for dependency resolution out in the wild (PEAR, Composer,
MooTools' packager), so that is feasible.

> I think the closest thing Joomla has to that is JDocument. But
> JDocument doesn't currently have the ability to sequence required scripts
> except - they are included in the order that addScript is called while a
> page is being rendered.

We agree on this being far from ideal. That's why we discuss alternaive
solutions.

> I dare say that a registerScript method could be a
> stepping stone to this more ideal implementation wherein the config file
> holding the dependencies will be imported and then processed by the builder
> using the registerScript method. Then developers (eventually) have the
> choice to either directly declare their dependencies, or declare them in a
> config file.

So this would be the workflow?

- JDocument (better: an HTML render class, but that's another story)
loads the config file(s).
- Extensions may call registerScript() to provide additional
information (the question about precedence has to be answered yet).
- Extensions declare their need using behavior().
- JDocument (renderer) resolves the dependencies and loads the needed
scripts. It of course uses caching and optionally precompiles the
script combination into a single (minified) file.

> Also, I don't like the term "behavior" for what we are doing. Maybe
> JDocument could have an enqueueScript function that takes a string name and
> includes the named script with any dependencies.

I think "behavior" is a very precise term for what we want here. We want
a piece of the markup to behave as a slider. To use scripts for that is
one possible solution, but some day it may be solved with CSS (see
DropDown menus for example).

[Donald Gilbert]

> In a perfect world, that would be ideal.
>
> However, how many extension developers do you know that build sliders that
> are js agnostic, leaving the choice of js framework up to the end user? I
> don't know of any.

Unfortunately, I don't neither. Nevertheless, everything we do should
get us nearer to the ideal, shouldn't it?

Yes, it should. 

That's a good question that I don't have a direct answer to, except that a site builder would be asking for conflicts if he knowingly implemented three different slider extensions.

 

2. What happens, if a site builder wants to use "SuperSlider" instead of
the "UberSlider" he used until now?

Currently, the site builder does not have that choice.

With my approach, the sitebuilder just changes the slider entry in the
config file.

How would that be solved with your approach?

Currently, I don't know of any slider extensions that use the same markup as another. So if you wanted to change, you would have to change the markup as well as the config entry. Although currently, the typical solution would be uninstall SuperSlider and install UberSlider. They take care of their own dependencies in their code, no config or markup changes necessary.

Exactly the process I was thinking. Precedence could be the config file if it exists.

 

> Also, I don't like the term "behavior" for what we are doing. Maybe
> JDocument could have an enqueueScript function that takes a string name and
> includes the named script with any dependencies.

I think "behavior" is a very precise term for what we want here. We want
a piece of the markup to behave as a slider. To use scripts for that is
one possible solution, but some day it may be solved with CSS (see
DropDown menus for example).

Looking at it from that perspective, it makes sense. I just couldn't figure out how the term behavior was applied to this, but it works with that explanation.

[Niels Braczek]

Am 12.09.2012 17:26, schrieb Donald Gilbert:

>> Unfortunately, I don't neither. Nevertheless, everything we do should
>> get us nearer to the ideal, shouldn't it?
>
> Yes, it should.

Ok, so we managed to close that door ;)

>> 1. What happens, if one extension developer thinks, that "XYSlider"
>> (depending on jQuery 1.2) is so cool, another developer requires
>> "ABCSlider" (depending on jQuery 1.8), and a third wants "MooSlider"
>> (depending on MooTools 1.4)?
>>
>> Currently, all three frameworks will be loaded, leading to three
>> different solutions for the same problem, and having negative impact on
>> the user experience (loading time and broken corporate identity).
>>
>> With my approach, the extension developer simply does not have that
>> choice. He can just request a slider.
>>
>> How would that be solved with your approach?

> That's a good question that I don't have a direct answer to, except that a
> site builder would be asking for conflicts if he knowingly implemented
> three different slider extensions.

The site builder did not mess with the different sliders - that was (in
this scenario) caused by three different extensions.

>> 2. What happens, if a site builder wants to use "SuperSlider" instead of
>> the "UberSlider" he used until now?
>>
>> Currently, the site builder does not have that choice.
>>
>> With my approach, the sitebuilder just changes the slider entry in the
>> config file.
>>
>> How would that be solved with your approach?
>
> Currently, I don't know of any slider extensions that use the same markup
> as another. So if you wanted to change, you would have to change the markup
> as well as the config entry. Although currently, the typical solution would
> be uninstall SuperSlider and install UberSlider. They take care of their
> own dependencies in their code, no config or markup changes necessary.

But using your model, the *extension* has a registerScript call for
"UberSlider" anywhere in its code.

We already have places with "normalized" markup - where Joomla provides
it ("modal" coming in mind). As with modal, *any* call to behavior() can
be accompanied with a selector to identify the elements involved, so no
markup change is needed here.

>> So this would be the workflow?
>>
>> - JDocument (better: an HTML render class, but that's another story)
>> loads the config file(s).
>> - Extensions may call registerScript() to provide additional
>> information (the question about precedence has to be answered yet).
>> - Extensions declare their need using behavior().
>> - JDocument (renderer) resolves the dependencies and loads the needed
>> scripts. It of course uses caching and optionally precompiles the
>> script combination into a single (minified) file.

> Exactly the process I was thinking. Precedence could be the config file if
> it exists.

Ok. That way, extensions can define (and deliver) a default set of
scripts, which can be overridden by the site builder using the config
file(s). The extension can be used out of the box, but the site builder
has the power.

I consider this being a step into the right direction.