The rules for the 1.8 branch were strigent:
* No interface may change after the initial 1.8 release (Firefox 1.5)
** This includes both IDL-declared interfaces as well as .h-declared
pseudo-interfaces
** If new functionality is necessary, make a new derived interface (see
nsIExtensionManager_1_8_BRANCH etc)
We broke the rules on purpose at least once during the 1.8.1 cycle, to
implement a new nsIContent function, IIRC.
I think that part of the heat about "blocking 1.9" would be lessened if the
API were not so set in stone the instant we release Firefox 3. Instead, I
think we should come up with a more reasonable balance that allows important
bugs to be fixed after the initial release even if these fixes may involve
interface changes. I think we should use the following rules for the 1.9 branch:
* interfaces declared in IDL will not be changed during the 1.9 cycle
** Exceptions to this rule may be made before the 1.9 release by annotating
an interface as explicitly not-stable for 1.9:
@status UNSTABLE-1.9: this interface may change during the 1.9 cycle
* Incompatible changes to the behavior of XUL are not allowed
* Incompatible changes to JS functions are not allowed
* pseudo-interfaces declared in .h files may change at any time, as long as
the IID is revised properly
I think that the rules I have proposed give a more reasonable balance
between change and stability: extensions written in JS/XUL will not break
between release, but extensions that use binary components may need to be
recompiled between releases. It also makes it much easier to backport some
kinds of fixes (mainly layout/content fixes) to the branch.
Of course, this policy wouldn't particularly help the particular bug with
mac menus that started this discussion.
--BDS
The failure mode here is basically that we change the interface, a user has a
binary component from an extension, the extension's maxVersion is too high (e.g.
1.9.0.*), the user gets crashes. The bug is that the extension set maxVersion
too high, but the user perception is that our app crashes.
Can we enforce a maxVersion == minVersion for any extension that actually uses a
pseudo-interface (or at least any extension that includes a binary blob, though
this might screw over extensions that use JS to talk to Gecko and have binary
blobs for 3rd party libraries)? That would alleviate some of the concerns about
crashes causing our users grief...
-Boris
They crash *if* they don't test that the IID still exist, and just
blindly used returned value without checking for NULL.
If you cut/paste existing code (or just model yourself on it), you don't
do that. They are few people doing binary extension, I think they are in
most case experienced enough not to fall into such a trap.
I even think in most case it's more convenient for them to have only one
source for the binary module across versions, often by using IID testing.
About downward compatibility, in many cases change in interface are only
to add new element, not to change any existing one. It would be great
when that's the only reason for the change that the old IID redirects to
a compatiblity bridge.
> Can we enforce a maxVersion == minVersion for any extension that
> actually uses a pseudo-interface (or at least any extension that
> includes a binary blob, though this might screw over extensions that use
> JS to talk to Gecko and have binary blobs for 3rd party libraries)? That
> would alleviate some of the concerns about crashes causing our users
> grief...
Please don't hurt extensions that do things correcly.
It is convenient to use only one binary blob and to test which IID is
present, in order to use the right one depending on the version.
Yes, of course. Historically, that's exactly what some of the popular ones do;
the resulting crashes were what led to the current branch policy.
> They are few people doing binary extension, I think they are in
> most case experienced enough not to fall into such a trap.
You're being too optimistic, sorry. Experience shows that they do in fact "fall
into such a trap". This is especially true for people developing on Windows,
apparently; I've seen claims that in the COM world once an interface is
published it does not change (perhaps only if Microsoft is doing the publishing,
though).
> Please don't hurt extensions that do things correcly.
How can we tell which ones those are?
-Boris
I just realized that what I really proposed was that an extension's maxVersion
be enforced to be no bigger than the version when it was installed. That way
when there is a minor update that revs IIDs the extension will become
incompatible (which is the case for the IID rev anyway, right)? You could still
use a single blob for "current and older" versions, just not for "current and
future".
-Borsi
Yes, if they include binary blobs (or if it's possible to detect that situation
if they have binary blobs that touch xpcom).
> This in return will hurt the community
> testing efforts a lot, either because once you rev a nightly version,
> this will hurt people or because some people will no longer download
> security release RCs.
I'd think binary blob extensions already have poor compat with nightlies and RCs...
> It will also result in people finding platform or app bugs only after a
> release is out and a extension has been updated, not beforehand, when a
> certain extension will trigger a bug in our code.
Do binary blob extensions often get updated as we go?
> So while your proposal sounds doable, I don't think that this is the road
> that you should be walking on.
Well, the other option is what we've been doing: no changes to .h
pseudo-interfaces on stable branches.
-Boris
> Can we enforce a maxVersion == minVersion for any extension that
> actually uses a pseudo-interface (or at least any extension that
> includes a binary blob
Well, we still wouldn't be changing interfaces (or, more generally, be
making any incompatible changes) in point releases, right? So a
maxVersion of 1.9.0.* would be fine. My assumption from what bsmedberg
wrote is that the rules would be different for a 1.9.1 release.
Addons.mozilla.org should already be enforcing limitation on how
forwards-compatible an extensions can be. EG, once FF3 released,
extensions for 3.0.* will be ok but 3.*, 3.1, 3.1.* would all be
prohibited. Similarly for other projects.
Of course, non-AMO hosted extensions can still shoot themselves in the
foot. I suppose we could hardcode maximum limitations in the product
itself, and update the limit as new versions ship. [Eg, rewinding
history, have Firefox 2.0.0.1 prohibit a maxVersion greater than
2.0.0.*, Firefox 3 and Firefox 2.0.0.12 would start allowing 3.0.*. And
eventually 3.1.* would be allowed for something based on Gecko 1.9.1]
...That sounds ugly and complicated, though.
Justin
I think you're reading things into what he said. He never said his proposal was
limited to 1.9.1.... If it were, I would be a lot happier with us running with it.
> Of course, non-AMO hosted extensions can still shoot themselves in the
> foot.
And do! ;)
> ....That sounds ugly and complicated, though.
All solutions to this problem that avoid the crash issue are largely to be ugly.
-Boris
I think this enforcement approach creates more problems than it solves.
The effect for extension developers and users is "Firefox update kills
the extension". Is that the effect you want? It does not seem to me that
this will make the "platform" look better, just less robust and more
insular. Any user of an extension frankly does not care if the platform
fails to crash if it also fails to load their extension.
In addition this club allows the uncommon case to drive away the common
case. The most likely outcome of a Gecko update on an extension is *no*
effect. Enforcement cause all updates to break all extensions when only
a few extensions may be affected. It reminds me of the secure update
restriction on FF3 addons: a huge impact at the wrong spot for a small
problem.
How about a "viral testing" approach instead? Rather than preventing a
user from loading a mis-matched version, invite them to load it for
testing. "Firebug 1.05 has not be marked as tested with Firefox 3.0;
Would you like to try it and report on your success or problems?" With a
URL pointing to Firebug issues/discussion list. The timid may flee; the
brave are warned.
John.
> Well, we still wouldn't be changing interfaces (or, more generally, be
> making any incompatible changes) in point releases, right? So a
> maxVersion of 1.9.0.* would be fine. My assumption from what bsmedberg
> wrote is that the rules would be different for a 1.9.1 release.
No. I am talking about point-releases, not 1.9.1. I don't think we should
do a 1.9.1, but that's a separate discussion.
--BDS
Note that we're just talking about restrictions on extensions that include
binary blobs, and better yet just extensions that include binary blobs that call
.h pseudo-interfaces, if we can detect them. No one is suggesting draconean
changes like this for JS-only extensions.
> In addition this club allows the uncommon case to drive away the common
> case. The most likely outcome of a Gecko update on an extension is *no*
> effect.
If we started changing pseudo-interfaces in dot-releases, this would not be the
case for extensions that use pseudo-interfaces.
> Enforcement cause all updates to break all extensions
Nothing is going to break JS-based extensions any more than they got broken by
the 1.8.0.x releases.
-Boris
> No. I am talking about point-releases, not 1.9.1.
Oh. Yikes. :-o
Incompatible changes in point releases sounds like a hard pill to
swallow... Though I don't have a good sense of how common binary
extensions are. End users won't care about the distinction, so unless
our developer outreach is excellent we'd take the heat for breaking
things. [Which then touches on the issue of people disabling security
updates, or switching to other browsers.]
It's also a little scary to be making a change like this so late in the
cycle, because there's no little (if any) time to mitigate the problem
for the people it affects [eg, by adding stable, scriptable interfaces
for their needs].
OTOH, maintenance of previous releases can be a big time sink, so it
does sound like a tempting tradeoff.
Justin
Could you please elaborate here?
IMHO we are mixing technical and marketing considerations on this issue.
I see plenty of reasons NOT to do Firefox 3.1, but my mind cannot
peacefully accept two libraries with the same version being even
slightly incompatible. What the versions are for, otherwise?
I know a solution. It is time to decouple Firefox and Gecko versions.
What prevents us from having these?
Firefox/Gecko
3.0.0/1.9.0
3.0.1/1.9.1
3.0.2/1.9.1 (non-Gecko related fix)
3.0.3/1.10.0 (security fix required interface change)
...
Users care about Firefox versions, developers care about toolkit
versions. JS extensions can have only Firefox MaxVersion (3.0.*), but
extensions, which ever touch an @UNFROZEN interface in native code, must
have two MaxVersion -- Firefox (3.0.*) and Toolkit (1.9.*).
Furthermore, I am not sure, that using @UNFROZEN from JS is so safe. In
my experience, interface change is usually *forced* by behavior change.
That means, pre-change-style usage of a changed interface may introduce
a security hole. So the above rule should probably be applied to JS code
as well.
This change can solve a range problems:
* add-on stability on updates;
* sharing gecko with SM, TB, etc.;
* .msi package.
--
Sergey Yanovich
My impression is that binary extensions are quite rare, but the ones
that do exist are generally more ambitious and "serious" than average.
FWIW, I think it would be tremendous help to offer hosting of
third-party extensions on Mozilla tinderboxes. This would enable the few
extension developers who use binary components to catch compatibility
problems early in the cycle (rather than having users contact them en
masse to tell them "Firefox 2.0.0.9 broke your extension"). To the
extent that developers are willing/able to include unit tests in their
build, they will get additional assurance that the upcoming point
release doesn't break their code.
If this is a non-starter due to personnel/hardware constraints (which I
would certainly understand) then perhaps more guidelines on how to set
up/operate a tinderbox? From what I've seen this is something of a black
art at present.
Matt
> FWIW, I think it would be tremendous help to offer hosting of
> third-party extensions on Mozilla tinderboxes. This would enable the few
> extension developers who use binary components to catch compatibility
> problems early in the cycle (rather than having users contact them en
> masse to tell them "Firefox 2.0.0.9 broke your extension"). To the
> extent that developers are willing/able to include unit tests in their
> build, they will get additional assurance that the upcoming point
> release doesn't break their code.
>
> If this is a non-starter due to personnel/hardware constraints (which I
> would certainly understand) then perhaps more guidelines on how to set
> up/operate a tinderbox? From what I've seen this is something of a black
> art at present.
I don't think we want to get into the build-hosting business at this point.
But I think we're willing to allow third-party tinderbox clients to send
reports to the Mozilla tinderbox server.
Setting up a tinderbox is a relatively simple process: just check out
mozilla/tools/tinderbox and follow the instructions. I can help if needbe on
irc, and ask you to write up a devmo doc for it.
It is also possible to set up buildbot so that final reports get sent to a
tinderbox reporter, but I don't personally know how to do this: you want to
talk to bhearsum.
--BDS
> IMHO we are mixing technical and marketing considerations on this issue.
> I see plenty of reasons NOT to do Firefox 3.1, but my mind cannot
> peacefully accept two libraries with the same version being even
> slightly incompatible. What the versions are for, otherwise?
What "two libraries with the same version"? I don't understand.
> I know a solution. It is time to decouple Firefox and Gecko versions.
> What prevents us from having these?
> Firefox/Gecko
> 3.0.0/1.9.0
> 3.0.1/1.9.1
> 3.0.2/1.9.1 (non-Gecko related fix)
> 3.0.3/1.10.0 (security fix required interface change)
What problem are you trying to solve? The problem we need to confront is:
can a minor/security update of Firefox caused some extensions to stop
working or crash? If so, how can we manage it so that users
1) don't crash
2) don't decline security updates in order to keep their extensions working
I don't see how separating gecko/firefox version numbers helps solve either
of these problems effectively.
> Users care about Firefox versions, developers care about toolkit
> versions. JS extensions can have only Firefox MaxVersion (3.0.*), but
> extensions, which ever touch an @UNFROZEN interface in native code, must
> have two MaxVersion -- Firefox (3.0.*) and Toolkit (1.9.*).
How do you enforce your rules?
--BDS
I've read your objection to 1.9.1 as an intention to keep 1.9.0 version
until mozilla 2 while reserving the ability to introduce incompatible
changes. In this case, firefox-3.0.3 from the example below should have
gecko 1.9.0.3, and we have two libxul libraries (1.9.0.0 and 1.9.0.3)
with the same 1.9.0 version, and this two libraries will be
incompatible. By incompatible I mean, that 1.9.0.0 cannot be substituted
by 1.9.0.3 without regressions -- crashes in 3.0.0 on 1.9.0.3, that
don't occur in 3.0.0 on 1.9.0.0.
>> I know a solution. It is time to decouple Firefox and Gecko versions.
>> What prevents us from having these?
>> Firefox/Gecko
>> 3.0.0/1.9.0
>> 3.0.1/1.9.1
>> 3.0.2/1.9.1 (non-Gecko related fix)
>> 3.0.3/1.10.0 (security fix required interface change)
>
> What problem are you trying to solve? The problem we need to confront
> is: can a minor/security update of Firefox caused some extensions to
> stop working or crash? If so, how can we manage it so that users
>
> 1) don't crash
> 2) don't decline security updates in order to keep their extensions working
>
> I don't see how separating gecko/firefox version numbers helps solve
> either of these problems effectively.
I've explicitly named problems, this solution addresses. I'll try to
explain my reasoning:
A) add-on stability on updates.
This is exactly 1) from above. Frankly, while I don't think, there is a
solution for 2) at all, I believe, that update induced extension crashes
are completely Mozilla's fault.
Min/MaxVersion of product is a totally misleading concept. Each add-on
should state EXACTLY with which gecko(=libxul) version, it is known to
work. After that, responsibility to track compatibility doesn't belong
to add-on developer, but to libxul vendor. If libxul in the next release
(major or minor) has the same version, add-on works, otherwise it
requires to be updated as well. No crashes allowed :P
B) sharing gecko with SM, TB, etc.
I am developing a XR application, and I manage to keep pace with the
trunk now. But I clearly see, that I will be in KaiRo shoes with
Mozilla2, unless something changes.
Min/MaxVersion is not such a big problem by itself, if both users and
developers are happy. But it has become a major hurdle for SM2, TB3, etc
to be released without forking gecko. Taking into consideration the cost
of forking for both sides, the whole issue deserves the most serious
attention.
C) .msi package
It may look related, but it is. Windows installer (MSI) requires all
versions of the same file (installed to the same location with the same
name) to be compatible. There are workarounds, but Windows installer
patches are only meaningful when used on mostly the same set of files.
This issue needs to be fixed, before Firefox can establish a solid
position as a corporate browser.
> How do you enforce your rules?
Agreed. The previous proposal is scrap, but dependency on a single
libxul version may work.
--
Sergey Yanovich
What does "same 1.9.0 version" mean? There has never been a Firefox release
where the minor version of gecko (now libxul) hasn't changed, so it has been
traditionally:
1.8 == FF 1.5
1.8.0.1 == FF 1.5.0.1
1.8.0.2 == FF 1.5.0.2
The same will hold true for 1.9:
1.9 == FF3
1.9.0.1 == FF3.0.1
1.9.0.2 == FF3.0.2
> incompatible. By incompatible I mean, that 1.9.0.0 cannot be substituted
> by 1.9.0.3 without regressions -- crashes in 3.0.0 on 1.9.0.3, that
> don't occur in 3.0.0 on 1.9.0.0.
That was the previous policy: *no* API changes (only additions) were
permitted in the 1.8.x series. I believe we should relax this policy and
explicitly allow changes for interfaces that are suitably internal.
>> What problem are you trying to solve? The problem we need to confront
>> is: can a minor/security update of Firefox caused some extensions to
>> stop working or crash? If so, how can we manage it so that users
>>
>> 1) don't crash
>> 2) don't decline security updates in order to keep their extensions
>> working
>>
>> I don't see how separating gecko/firefox version numbers helps solve
>> either of these problems effectively.
>
> I've explicitly named problems, this solution addresses. I'll try to
> explain my reasoning:
>
> A) add-on stability on updates.
>
> This is exactly 1) from above. Frankly, while I don't think, there is a
> solution for 2) at all, I believe, that update induced extension crashes
> are completely Mozilla's fault.
They are absolutely related questions: we cannot reasonably expect our users
to upgrade from FF3 to FF3.0.1 if their browser will crash as a result, or
if an extension that they depend on is suddenly unavailable.
The previous solution to this problem has been to make pretty far-reaching
compatibility promises for the lifetime of the 1.8.0.x branch, so that
extensions can safely say "I'm compatible with all releases in the 1.8.0.x
series" up front, rather than having to test at every minor release point.
--BDS
1.9.0.3 = {1.9.0}.3
{version}.release
>>> What problem are you trying to solve? The problem we need to confront
>>> is: can a minor/security update of Firefox caused some extensions to
>>> stop working or crash? If so, how can we manage it so that users
>>>
>>> 1) don't crash
>>> 2) don't decline security updates in order to keep their extensions
>>> working
>>>
>>> I don't see how separating gecko/firefox version numbers helps solve
>>> either of these problems effectively.
>>
>> I've explicitly named problems, this solution addresses. I'll try to
>> explain my reasoning:
>>
>> A) add-on stability on updates.
>>
>> This is exactly 1) from above. Frankly, while I don't think, there is
>> a solution for 2) at all, I believe, that update induced extension
>> crashes are completely Mozilla's fault.
>
> They are absolutely related questions: we cannot reasonably expect our
> users to upgrade from FF3 to FF3.0.1 if their browser will crash as a
> result, or if an extension that they depend on is suddenly unavailable.
My previous post contains a recipe to avoid after-update crashes
altogether. On second thought, it will work for the second part as well!
My own experience with extension crashes tells that users are unlikely
to decline an update, but are likely to revert it, if it is broken.
'Broken' in a wide sense pertains to browser + all installed extension.
So I was reverting, because I wanted my browser back, fast.
With correct libxul versions, the browser will always be working. It
will be *only* extensions, that are broken. And the extensions will be
gradually brought back on-line after update by their authors. Needless
to say, this will add up to both perceived and real security of firefox.
> The previous solution to this problem has been to make pretty
> far-reaching compatibility promises for the lifetime of the 1.8.0.x
> branch, so that extensions can safely say "I'm compatible with all
> releases in the 1.8.0.x series" up front, rather than having to test at
> every minor release point.
I will say in less pleasant words.
1.8.1 version was occupied by firefox 2.0, so there was no other option
as to keep version compatibility for 1.8.0, or face after-update crashes.
I don't think, it is a good idea to enter such a situation voluntarily.
--
Sergey Yanovich
This feels like a mischaracterization to me. For one thing, it's not
yet at all clear how suitable/unsuitable trunk Gecko will be for
embedders to release from. It would be very informative to talk to
folks who have done this sort of thing before (Songbird? ActiveState?
Joost?...) before assuming this to be a problem. Additionally, cutting
a private branch off the trunk (based either at the 1.9 tagging point or
somewhere significantly later than that) in order to gain stability and
the ability to take a few locally important Gecko fixes hardly qualifies
as forking, I don't think.
Dan
KaiRo made an emotional post to "Gecko/Toolkit/Core 1.9.1 cycle for
non-FF apps?" thread in this group, in which he went much further than
just cutting a private branch for SeaMonkey.
I guess, most embedders *implement* unfrozen interfaces. Since I am an
embedder exactly of this type, my case may be a representative one.
A sudden change in such an unfrozen interface, I implement, will require
corresponding adjustments in my code. So I cannot trust system-libxul in
a way I trust system-nspr. So I need to ship my own libxul with my app.
But I will need to cherry-security fixes, but not features from 1.9.0
branch. This is may part of the cost of changing interfaces without
changing library version.
I will also have considerably less time to work with upstream branch and
trunk code. Not a big deal, but "given enough eyeballs, all bugs are
shallow". With each pair of eyeballs off the trunk, all bugs become a
bit less shallow. That's Mozilla part of the cost.
Library versioning rules on both Linux and Windows are consistent with
my proposal, so libxul may improve OS integration level.
--
Sergey Yanovich
(Preamble: Songbird - XR-based app, trunk. Not an embedder, we are
totally into XUL.)
As a data point, at Songbird, we just check out the trunk once a week.
The built binary / headers go into SVN, so if anything majorly bad
happens we just revert and wait until it gets fixed (possibly attempting
to help). We go for a couple weeks without updates if we're nearing a
release. Yes, we're crazy - ideally we'd at least be smart enough to
checkout by date...
If an interface changes, yes, we will need to do whatever work necessary
to change our use of it to match as well. That sucks, but there's not
much we can do.
1.9 has been mostly good for us so far, probably because we have to use
frozen linkage (it doesn't really change all that much). For the most
part, we just use the widely used interfaces (DOM, observers, etc.), so
the risk isn't that bad. We do, of course occasionally hit things being
changed upstream, but are basically powerless to have input since we
only get notified when things break. For example, if the threading
stuff changes, we'll be in a world of pain (okay, more so).
We do ship our own XR, even on Linux - we need to apply a few patches.
This means we don't care much about binary compatibility, but source
compatibility is rather important. Interface frozenness doesn't have
much impact to our use (I see no markings about the stability of
nsIXPCScriptable, for example, but we're using it more because it's just
so darn useful).
<offtopic>
As fallout from this, currently my personal opinion of Moz2 is
"massively scary". XPCOMGC means we're screwed on threading (you don't
really want to be doing background work like scanning for metadata on
the main thread - no, we don't touch DOM on background threads); no more
binary components makes talking to external libraries (e.g. gstreamer (C
with macros), taglib (namespaced C++ / STL) ) look to be somewhere
between painful and impossible. But I haven't actually pulled the moz2
code.
</offtopic>
--
Mook
random guy
We released from the XR 1.9 because it was more suitable that 1.8 for
what we want to use and we expected it much earlier to come into release
than it takes. We released our Windows release with 1.9a6. Many bugs are
fixed since then. Important bugs especially on Mac are fixed since
1.9b2, so we can't release the Mac version of our product with an older
XR than 1.9b2, but excatly there a very powerful interface (support for
install.js in install manager) has gone, we depend on.
Our application is not an extension of FF but XR is the engine inside of
our application.
We do not build XR, we just download the nightlies and copy them into
our application to test, whether known bugs have gone and which new
problems occur.
So after I got 1.9b2 I was very happy to now succeed with many stuff on
Mac. When I wanted to release the first beta of our Mac version I did
also the QA for our update and deployment wizard, which depends on the
install manager, I detected, that this no longer works. Now I have a lot
of extra work to reimplement the old interface using download manager
and other components to get back the facility to install files to
arbitrary places.
So I think it is important to see XR not just as engine for FF but as
general purpose engine providing everything needed by FF but also stuff
not needed by FF but needed by other applications.
Stuff not needed by FF might be stored in separate jar files inside XR
not shipped with FF. I think you find a solution satisfying FF and
others too.