The Future of Firefox support - let's discuss

[David Burns]

Selenium 2.27 is out the door! Let's all rejoice!

However, this release has shown that we need to revisit what we support on the Firefox side. The Firefox 3.6 CI job is very flaky and we keep building for it when it's not supported anymore. I say we stop this madness and support only the following:

Firefox N and Firefox N -1 where N is the current version in the release branch
Firefox ESR that is still supported by Mozilla. Currently Firefox 17 ESR is out but Firefox 10 ESR is still alive and won't be pulled for a couple months while companies update to 17.

The Why I suggest this:

Firefox 3.6 users were sent a forced upgrade, around Firefox 12 time which they could decline, so the numbers are small of real users!

My colleagues and I are working hard to get Marionette complete and into Firefox. FirefoxDriver is a "subset" of Marionette since Marionette can drive the browser chrome too. Marionette is in FirefoxOS and soon to be in Firefox Mobile and is Mozilla's answer to OperaDriver and ChromeDriver. As we get to a stage of transitioning over we need to be reigning in support for versions to what is in the Release branch, like we do with ChromeDriver. We have agreement in principle with our security team to get it into release builds of Firefox desktop and a plan for mobile. Once we are baked in, its release or nothing really. Marionette is also part of the Mozilla build and test cycle. You can see pushes with it being tested at https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=marionette

When is Marionette going to be complete? Well I am not sure but one of the blockers for 2.27 wouldn't have happened with Marionette since we are in the browser in a privileged area.

Let's discuss how we can support our community without making us do unnecessary work!

David

David Burns
URL: http://www.theautomatedtester.co.uk/

[Santiago Hernandez]

Thanks David.

I agree that maybe is not the best option to continue supporting FF 3.6 when we are on version 17 :)

Just my opinion...

Thanks!

 Santius

--
 
 

[Kevin Menard]

I'm probably one of the few still relying on FF 3.6 support because I do have customer demand for it.  Having said that, SSL support for FF 3.6 has been broken since Selenium 2.25 and I haven't had the time to bother fixing it.  It's not particularly fair for me to force that on others.

A counter-proposal of sorts may be to drop 3.6 but instead of just supporting ESR, also support ESR - 1.  But since we're just hitting the second ESR release, I have absolutely no data to either support or refute that suggestion.  It's more a gut feeling that 3.6 is in use because it's the "stable" release, so it's been sort of a de facto ESR - 1 up until now.

A third proposal is Mozilla stops breaking their API every six weeks :-)

--
Kevin

David Burns

Wednesday, December 05, 2012 4:04 PM

--
 
 

[Jan Kester]

I noticed that the 2.27.0 java driver on download site: http://code.google.com/p/selenium/downloads/detail?name=selenium-java-2.27.0.zip&can=2&q=
contains a ChangeLog with nothing about 2.27.0.

The official selenium website still shows 2.25.0 as latest version: http://seleniumhq.org/download/
and in changelog on the same side, it also only contains information up to 2.26.0.

Does 2.27.0 version have support for native events in FF17 ?

Regards, Jan Kester.

[David Burns]

Changing subject of email since question is totally unrelated to thread it was sent to.

Google Code is the canonical download site. We are trying to change the DNS on seleniumhq which none of the committers have access to. 

2.27 has all the relevant Firefox 17 support.

David

--
 
 

[Santiago Suarez Ordoñez]

I'm down with this plan and from a Sauce Labs stand point, this is something we can deal with. I think a *must* for the project to proceed this way is to have a clear outline and history of which versions each release supports as we go along. This way users out there interested in specific versions, will be able to look back download an older version of Selenium that provides a reasonable support for their platform.
This is what I'm thinking Sauce Labs will end up doing transparently for customers with such requests.
I don't want to introduce much additional work to our release effort, how do you guys bacon holders think we can make this happen easily?

Santi

--
 
 

[Santiago Suarez Ordoñez]

After sending this I realized of another caveat. With this policy in place, there's a possibility for a Firefox version to never get support by any Selenium version. This can happen simply if Selenium's releases get stuck long enough for Firefox to make 3 releases in a row without a Selenium release coming out. Under those conditions I think we would need to include such version (and wichever other comes out) in the list of supported for the very next Selenium that comes out.

A good way to prevent this from happening while no one's looking is for the documentation that I just proposed to have the following format:

+-------------------+------------------+
|Firefox version #  | Selenium version #
+-------------------+------------------+
|    14             |  2.26.0, 2.27.0  |
|    15             |  2.26.0, 2.27.0  |
|    16             |  2.26.0, 2.27.0  |
|    17             |  2.26.0, 2.27.0  |
|    18             |  2.27.0, 2.28.0  |
|    19             |  2.28.0, 2.29.0  |
|    20             |  2.29.0, 2.30.0  |
|    21             |  2.30.0, 2.31.0  |

This way it'd be easy to spot if a Firefox version comes out and no Selenium version has explicit support for it.
Do you guys think this is reasonable?

Santi

[Leah Klearman]

I made a suggestion at some point that there be a link that resolves to 'latest' on the download page, rather than having to remember to update the download webpage for each release.

Regards,

-Leah

--
 
 

[Alexei Barantsev]

+1 to Kevin.

I'm voting to provide support for latest, previous, latest ESR and previous ESR.

Regards,

-- 

Alexei Barantsev

Software-Testing.Ru

Selenium2.Ru

[Simon Stewart]

Counter-proposal:

Firefox N, N-1, ESR, ESR -1

I agree that 3.x support should be dropped now, though I'd like a Googler to weigh in on that to confirm that usage has dropped below the level we expect for a browser to be supported there.

Reasoning: most users will have N or won't have updated yet (N-1). The ESR releases are on corp desktops (probably) so supporting those is a Good Idea. Corps are bloody slow at updating things (which is why 3.x is still hanging on), so ESR-1 gives us the best possible coverage with those folks.

Simon

On Wed, Dec 5, 2012 at 1:04 PM, David Burns <david...@theautomatedtester.co.uk> wrote:

--
 
 

[Ross Patterson]

According to StatsCounter (http://gs.statcounter.com/chart.php?bar=1&statType_hidden=browser_version&region_hidden=ww&granularity=monthly&statType=Browser%20Version&region=Worldwide&fromMonthYear=2012-11&toMonthYear=2012-11&csv=1), for November 2012:

 

Browser	Version	Market Share % (Nov 2012)	% of FF (calculated from StatsCounter data)
Firefox	3.6	0.7%	3.1%
Firefox	10	0.43%	1.9%
Firefox	16	15.29%	68.4%
Firefox	17	1.04%	4.7%

 

So “N” (17) and 3.6 are darned close to equal, and 3.6 has more usage than ESR (10).

 

Ross

--
 
 

[David Burns]

Seeing as Firefox was released on November 30, I think that is not bad figures :)

David

David Burns
URL: http://www.theautomatedtester.co.uk/

--
 
 

[Daniel Wagner-Hall]

On 6 December 2012 19:10, Simon Stewart <simon.m...@gmail.com> wrote:

Counter-proposal:

Firefox N, N-1, ESR, ESR -1

I agree that 3.x support should be dropped now, though I'd like a Googler to weigh in on that to confirm that usage has dropped below the level we expect for a browser to be supported there.

Reasoning: most users will have N or won't have updated yet (N-1). The ESR releases are on corp desktops (probably) so supporting those is a Good Idea. Corps are bloody slow at updating things (which is why 3.x is still hanging on), so ESR-1 gives us the best possible coverage with those folks.

That all SGTM.  Just checked our numbers.  If we can avoid actively breaking support for a while that would be handy, but there's no reason for marionette to support ff36 going forwards.