Re: [jplatform] Fix for installing extensions directly from a dynamic URL

[Chad Windnagle]

I believe since this is in the legacy folder for the CMS you may need to make your PR to the CMS on github, and also create a joomlacode feature tracker item for testing. But I agree, it would be great to have dynamic install URLs.

On Sunday, January 13, 2013, Bakual wrote:

Hi

 

Currently in Joomla 3.0.2 you can't install an extension directly from an URL, if said URL is dynamically created using a tool like ARS or cTransifex. It only works if the URL is pointing to the real file or has a valid filename behind the latest slash. Stuff like 'www.domain.com/download.html?dlid=06d028362cf9256ac96619571aa0ed8c&jcompat=my.zip' doesn't work.

The fix is quite simple: The function which fetchs the package (JInstallerHelper::downloadPackage()) used to get the header data using stream_get_meta_data() and this got changed to JHttpFactory::getHttp(). But apparently the check for the filename in 'Content-Disposition' is still done in the old place.

 

I've done the PR (https://github.com/joomla/joomla-platform/pull/1795) in the platform to fix this. But since the installer got moved to libraries/legacy, I'm not even sure if the platform is the right place for it. However in CMS it's still in the /libraries/joomla tree.

 

I think it would be a rather important fix as a core function is broken in some cases.

--

---

Chad Windnagle s-go Consulting 607-330-2574 x103 607-229-6260 (Cell) Website Design - SEO - Video

Follow Me:   

[Donald Gilbert]

Dynamic install url's would pair nicely with the J!Lite extension search / install feature if were to make a sort of "Joomla Extension Store" where items could be purchased, at which point you put in your key, and are then able to install.

[Bakual]

The funny thing is that it was working in Joomla! 2.5. It just broke when the function to get the headers got changed to the new JHttpFactory class.

I'll open a trackeritem and a new PR in the CMS then then.

[Chad Windnagle]

Well I may be confused as to where the broken code is (if its a legacy class with a dependancy on the platform?), but at tr very least it sounds like a bug for the tracker.

I believe 3.0 and 2.5 have the language installer work done by Javier Gomez, so perhaps the bug was introduced through that? Ill see if he can chime in here.

[Bakual]

The current platform implementation in Joomla 3.0.2 has the installer class in /libaries/joomla, but the current platform trunk on GitHub has it in libraries/legacy. So it got moved sometimes between :-)

Maybe it even needs to be fixed in both repos.

[Bakual]

PR on CMS repo: https://github.com/joomla/joomla-cms/pull/649

[Bakual]

If someone could test or approve this PR(s) before Joomla 3.0.3 is released, that would be great. I think it's quite an important fix for a partially broken core function (one click extension updater).
There is already one successfull testing done, needs two more (at least that's my understanding).

 

PR on platform: https://github.com/joomla/joomla-platform/pull/1795

PR on CMS: https://github.com/joomla/joomla-cms/pull/649

TrackerItem: http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=29946

 

Thanks.

[Elin Waring]

Did you test and comment on the tracker? That is the only way that fixes get into the CMS.   If you want to know what is happening in the CMS you need to post on the CMS or bug squad mailing lists not the platform mailing list.

In the meantime it looks like it is RTC for the 3 branch but if people want a fix in 2.5 they need to do that separately.

Elin

On Monday, February 4, 2013 1:16:05 PM UTC-5, Nacho Brito wrote:

El domingo, 13 de enero de 2013 10:36:02 UTC+1, Bakual  escribió:
> Hi
>  
> Currently in Joomla 3.0.2 you can't install an extension directly from an URL, if said URL is dynamically created using a tool like ARS or cTransifex. It only works if the URL is pointing to the real file or has a valid filename behind the latest slash. Stuff like 'www.domain.com/download.html?dlid=06d028362cf9256ac96619571aa0ed8c&jcompat=my.zip' doesn't work.
>
> The fix is quite simple: The function which fetchs the package (JInstallerHelper::downloadPackage()) used to get the header data using stream_get_meta_data() and this got changed to JHttpFactory::getHttp(). But apparently the check for the filename in 'Content-Disposition' is still done in the old place.
>  
> I've done the PR (https://github.com/joomla/joomla-platform/pull/1795) in the platform to fix this. But since the installer got moved to libraries/legacy, I'm not even sure if the platform is the right place for it. However in CMS it's still in the /libraries/joomla tree.
>  
> I think it would be a rather important fix as a core function is broken in some cases.

Seems that it finally didn't make it to 3.0.3... Any plan for integrating the patch in 3.0.4? I also find this a rather critical issue...

[Bakual]

It's not broken in 2.5, only in 3.0. Since it's now ready to be commited, there is nothing left to fix :-)

It's just a pity it didn't went into 3.0.3.

[Amy Stephen]

There's got to be a better way. 3 patches for the same bug is not going to work well (as this example shows).

[Michael Babker]

So long as the Platform and CMS are separate projects, then it's going to be hard to come up with a better way short of automatically accepting all patches from the Platform into the CMS.  But, would you want to blindly merge in changes from your project's upstream without testing first to make sure the changes don't break your existing product, and don't cause unnecessary headache to your downstream?  The CMS is between a rock and a hard place honestly.  Maybe every accepted Platform patch can be automatically submitted into the CMS tracker for possible testing/inclusion (the technology is there, well, maybe not in Joomlacode *evil laugh*) then the CMS can decide to test and merge immediately or hold off depending on the change?

--
You received this message because you are subscribed to the Google Groups "Joomla! Platform Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-plat...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Bakual]

I think the process for the CMS needs some serious work anyway. If you really want to have contributes from many people, you need to streamline the process (JoomlaCode is just an outdated site and clunky as hell). And the whole process needs to be faster.

I have currently 5 PRs sitting around with relatively minor changes, some even only typos. They don't get merged for months. That just discourages people.

I agree that it's important to test changes. But do we really always need 3 tests for a simple typo? Or like in this case an oversight? If yes we seriously need more testers.

Asking the contributor to advertise his change and look for testers himself is not going to work. I tried it here, and it still took 4 months and not enough testers. And it's a broken core function.

I feel like I have less problems to get a patch into the platform than into the CMS. Somehow it works there without testers at all :-)

[Donald Gilbert]

I think it's sad that there is such a separation between the two systems. There are critical fixes in 3.0 that will never be back ported to 2.5.x, even though it would be stupidly trivial to do so. Case in point: https://github.com/joomla/joomla-cms/blob/2.5.x/libraries/joomla/form/rules/tel.php#L84 <-- this doesn't properly validate phone numbers. There was a fix added in platform 12.1, so why can't this fix be put into 2.5? Because the contribution process for the CMS is a joke, IMO. This is one of the reasons I contribute mostly to the platform, because the process is so much simpler.

[Amy Stephen]

It's a library. So, downstream users - like myself - need to take a release and *use it*. 

No one needs to stay up-to-date, in fact doing so is exactly what puts the application between a rock and a hard place. But, it's self-inflicted, not forced on any application. Trying to keep up makes it hard for application devs and sites to keep up and discourage the platform from revitalizing.

In fact, one of the major reasons for the platform project was to give the platform room to operate while cushioning the the CMS from change since they would take a version and stabilize on it. So, no, I don't expect the CMS to merge in the changes. I expect them to do as was planned, as is done with other libraries, and that is to take a release, use it. 

No one is chasing down PHPMailer or SimplePie. Do it like that, right?

It's been a year and a half now. Maybe it's time for leadership to decide - is the platform project something the project wants to invest in? If so, adapt the CMS workflow to the change. If support is not there to adapt the workflow, then put it back together. In my opinion, a clear decision in either direction is best. It stops the forking of the platform code inside of the Joomla project and brings sanity back to the workflow.  

Thanks for listening, Michael.

On Monday, February 4, 2013 11:23:31 PM UTC-6, Michael Babker wrote:

So long as the Platform and CMS are separate projects, then it's going to be hard to come up with a better way short of automatically accepting all patches from the Platform into the CMS.  But, would you want to blindly merge in changes from your project's upstream without testing first to make sure the changes don't break your existing product, and don't cause unnecessary headache to your downstream?  The CMS is between a rock and a hard place honestly.  Maybe every accepted Platform patch can be automatically submitted into the CMS tracker for possible testing/inclusion (the technology is there, well, maybe not in Joomlacode *evil laugh*) then the CMS can decide to test and merge immediately or hold off depending on the change?

From: Amy Stephen <amyst...@gmail.com>
Reply-To: <joomla-de...@googlegroups.com>
Date: Monday, February 4, 2013 11:18 PM
To: <joomla-de...@googlegroups.com>
Subject: [jplatform] Re: Fix for installing extensions directly from a dynamic URL

[Elin Waring]

Overall the CMS both moves more quickly than and tests b/c issues more thoroughly than the platform, but the platform does better automated tested and code review.  So the best process if possible is fix in platform followed by fix in CMS.  Sometimes however that is not going to reflect reality on the ground. Unit tests sadly are not really enough when it comes to use in an application so any changes need to go through another round of integration testing for the CMS. 

The best thing to do is to make sure that if you have a platform patch accepted that you want for downstream ie the CMS that you go ahead and submit a patch to the CMS, otherwise you have to wait until there is a platform sync that someone who isn't you will have to manage it. If it is your fix, then you are probably the best one to describe the problem it solves and how to test and to supply any necessary code for that.  On the other hand, if you don't mind waiting and taking your chances on timing then you can just wait for a sync.

If you have a fix that in your opinion  is urgent and absolutely must be fixed in the CMS without waiting for the the platform,  submit both places at the same time with an explanation (but be prepared to have the CMS one downgraded to not an emergency and don't take it personally if that happens).  If this is a regression due to a previous platform change, then please indicate that. A good thing with the pull tester is that even if a fix is in the platform backlog at least we have documented unit test results so no matter what, a platform pull request has to be part of any platform fix done in the CMS. 

As we saw with the 3.0.3 the CMS is trying to be a bit more aggressive about staying in sync with platform bug fixes and non b/c breaking improvements but this always must be done with extreme care. This makes getting things fixed easier and puts less pressure on the platform to fix something by a given release of the CMS. It also means that a much more careful job can be done assessing any potential issues since it is always better to do that in small manageable pieces rather than in huge chunks.  

No, the CMS does not expect its users to live with known and fixed bugs for the sake of some abstract bureaucratic principle.  Just like downstream applications of all types using all kinds of open source libraries it is completely free to modify at will if that suits the needs of its users or developers and to take any risks inherent in doing so.  Knowing that such things are happening is good motivation perhaps to keep that backlog getting processed and to be responsive to the needs of those developing on your library so that they can stay synced rather than having patched versions. (Making that more transparent is a reason I favor more frequent tagging but as Andrew has said the CMS can always say this is platform commmitnumber) This is always the challenge of open source. 

It's also good for the platform to avoid discouraging contributors by essentially putting all contributions into bitrot mode until unknown future dates when the CMS syncs since the vast majority of platform users are using the version distributed with the CMS.  I certainly write platform code with the intent that people will use it, not that it will be in a museum exhibit.  A repeat of that terrible situation which, formerly existed, is what we are trying to avoid by having two projects.

Elin

[Donald Gilbert]

All of those reasons is why I favor moving towards a component based framework (in the sense of Symfony Components, not what the CMS historically refers to as "components"). New code can be added and tested and easily integrated by downstream users. It would make it much more simple for the CMS to integrate things like Diana's OAuth work as well as JTwitter / JFacebook.

--

[Amy Stephen]

Don - you are absolutely right that the goal of separating out packages, making them individually testable and installable, is what will help.

Considering a goal is that *many* applications might use the platform, and all would benefit from direct bug reports, very quickly it becomes obvious patches would have to be shared with even more than three repositories if such a manual approach was considered prudent.

If waiting on the next quarterly release is too long, how about adding automation that will produce accurate information regarding all changes made to the platform so that all applications that use the platform have good information regarding the change and are able to make better choices on how best to proceed?

That should get the CMS better information while doing so in a way that can benefit everyone, and those community sharing fixes can do so knowing all downstream users benefit even if they only issue a PR at one repository.

Thanks.

[Nestor Ledon]

I'm all in for a modular design pattern. It apparently works. :D

[Matt Thomas]

+1

Best,

Matt Thomas
Founder betweenbrain™
Lead Developer Construct Template Development Framework
Phone: 203.632.9322
Twitter: @betweenbrain
Github: https://github.com/betweenbrain

Composed and delivered courtesy of Nexus 7.

On Feb 6, 2013 6:59 AM, "Nestor Ledon" <nesto...@gmail.com> wrote:

I'm all in for a modular design pattern. It apparently works. :D

[Andrew Eddie]

On 6 February 2013 21:59, Nestor Ledon <nesto...@gmail.com> wrote:

I'm all in for a modular design pattern. It apparently works. :D

Kind of but not really in terms of what is being discussed here. Try installing "Node 1" and one of the solar arrays individually on the Chinese space station. The ISS is a good example of tightly coupled dependancies built with a modular architecture (aka the Joomla CMS).

Regards,

Andrew Eddie

[Amy Stephen]

Now, that's funny, Andrew. Love it.

--

[Paul Orwig]

The ISS is a good example of tightly coupled dependancies built with a modular architecture (aka the Joomla CMS).

Can we TM that?

[Mark Dexter]

I would guess that in a space station "tightly coupled" is a very, very good thing! Mark

[Nestor Ledon]

How about my Orbital Design Pattern:

The core libraries (nucleus) and its files (quarks) will remain tightly coupled, yet light, portable, and extensible.

"Orbits" can be groups of modules or even other libraries dependent on the core.

So when making updates or changes to something, only it's orbit and direct descendants (if any) are concerned.

Eventually... we'll end up with string theory.

[Andrew Eddie]

On 7 February 2013 17:14, Nestor Ledon <nesto...@gmail.com> wrote:

How about my Orbital Design Pattern 

The core libraries (nucleus) and its files (quarks) will remain tightly coupled, yet light, portable, and extensible.

"Orbits" can be groups of modules or even other libraries dependent on the core.

So when making updates or changes to something, only it's orbit and direct descendants (if any) are concerned.

Getting better :) Can we get Schrödinger's cat in there somewhere, hehe (/me quietly hopes it is actually dead). 

 

Eventually... we'll end up with string theory.

Well, that's a different kettle of fish and currently has no basis in experimental science as far as I'm aware. That would be like the Mambo project still trying to convince people that Joomla is never going to make it :D

Regards,

Andrew Eddie

[Bakual]

Fixed in platform and CMS. Thanks.

Am Sonntag, 13. Januar 2013 10:36:02 UTC+1 schrieb Bakual: