Ensuring that interface changes are documented for add-on compatibility
Paolo Amadini
nsIUnicharStreamLoader caused a severe issue in an add-on I'm
maintaining, in case a non-default setting was selected. Despite
the add-on has more than 50K users, probably they're still not enough
to provide meaningful testing in the Aurora and Beta channels, so the
issue was reported to me the day after the release and I had to rush
for providing a fix.
I know there exists a process for ensuring that changes that are likely
to affect add-ons are tracked, and automatic search patterns are added
to the tool that is used to mark add-ons hosted by Mozilla as
compatible or not.
However, it's not clear to me how this process works from start to end,
and if it's formalized at all. Can you describe it?
I'm asking not only to understand what happened with the change I
mentioned, that belongs to a past release, but also because I made an
interface change that will be released in Firefox 8 and it seems to
me it's not being tracked in any way, so either the process needs
improvement, or needs to be better communicated, since the people that
worked with me, or which I asked to, seem unaware of how it works.
Regards,
Paolo
jorgev
The process at the moment consists in me looking into all fixed bugs
done for a given Firefox release, in this case Firefox 6. I go through
this list of hundreds of fixes and try to identify those that could
affect extensions. Anything that appears to make any changes in an
XPCOM interface is investigated, and I look in a search tool that can
browse the source code of all add-ons on AMO, looking for matching add-
ons.
This process is far from perfect, and it has a number of failure
points:
* The bug fix list is incomplete. There may be fixes that don't have
the milestone set and won't appear in my queries. There are also
security-critical fixes that are hidden by default and also won't
appear there.
* Since the list is very long, I can only afford to look at the
summaries and guess which may affect add-ons. Clearly, there are some
of them that will fall through the cracks. We discussed creating a
diff tool that runs for all XPCOM interfaces, but that hasn't happened
yet.
* The add-on source search index is out of date, so some recent add-on
files are not indexed. There's a bug open for this.
Most importantly, we haven't created a good communication line between
Firefox engineers and the add-ons team. I do get informed about some
add-on breaking changes every so often, but the majority of them I
have to fish from all the bug fixes. We're also working on improving
this.
I'm currently looking into breaking changes for Firefox 8, so I would
appreciate it if you let me know which interface change you are
talking about.
Regards,
Jorge Villalobos
Add-ons Developer Relations Lead
L. David Baron
> * Since the list is very long, I can only afford to look at the
> summaries and guess which may affect add-ons. Clearly, there are some
> of them that will fall through the cracks. We discussed creating a
> diff tool that runs for all XPCOM interfaces, but that hasn't happened
> yet.
The hg command:
hg diff -rAURORA_BASE_20110705:AURORA_BASE_20110816 | grep "^diff.*idl$"
would give you all the interfaces that are in idl, which is probably
all that matters since extensions using C++ have to recompile
anyway.
-David
--
𝄞 L. David Baron http://dbaron.org/ 𝄂
𝄢 Mozilla Corporation http://www.mozilla.com/ 𝄂
Paolo Amadini
> The process at the moment consists in me looking into all fixed bugs
Thanks for the explanation, and for your help to add-on developers!
And yes, this doesn't look very scalable.
> * Since the list is very long, I can only afford to look at the
> summaries and guess which may affect add-ons. Clearly, there are some
> of them that will fall through the cracks.
> Most importantly, we haven't created a good communication line between
> Firefox engineers and the add-ons team.
What about asking patch authors and reviewers to set a keyword on the
bug if they think it can affect extensions, like we currently do with
"dev-doc-needed" for the MDN documentation?
It's not the solution to all the issues you mentioned, but it might help
reduce the chance that those changes fall though the cracks, even if
they're not IDL. And it's probably better than "e-mail Jorge" :-)
> I'm currently looking into breaking changes for Firefox 8, so I would
> appreciate it if you let me know which interface change you are
> talking about.
Sure, it's bug 591289 where we added a parameter to a method in
nsIDownloadHistory. This should only affect binary consumers.
Cheers,
Paolo
Sheppy
> With the release of Firefox 6, an undocumented change made to
> nsIUnicharStreamLoader caused a severe issue in an add-on I'm
> maintaining, in case a non-default setting was selected. Despite
> the add-on has more than 50K users, probably they're still not enough
> to provide meaningful testing in the Aurora and Beta channels, so the
> issue was reported to me the day after the release and I had to rush
> for providing a fix.
Looks like someone forgot to tag the corresponding bug with the dev-
doc-needed keyword to call it to the attention of the documentation
team.
Eric Shepherd
Developer Documentation Lead
Mozilla
http://www.bitstampede.com/
Justin Dolske
> The process at the moment consists in me looking into all fixed bugs
> done for a given Firefox release, in this case Firefox 6. I go through
> this list of hundreds of fixes and try to identify those that could
> affect extensions. Anything that appears to make any changes in an
> XPCOM interface is investigated, and I look in a search tool that can
> browse the source code of all add-ons on AMO, looking for matching add-
> ons.
Yikes.
I believe Dave Townsend wrote a little tool do generate a list of
interface (IDL) changes between releases. I know it's been used in the
past to catch interface changes that didn't rev the IID. I'd suggest
getting in touch with Dave to get ahold of that!
Justin
Justin Dolske
> What about asking patch authors and reviewers to set a keyword on the
> bug if they think it can affect extensions, like we currently do with
> "dev-doc-needed" for the MDN documentation?
Sure, but you still need a way to find the cases where people forget to
do that (and they will).
Justin
Gijs Kruitbosch
> On Thursday 2011-08-18 09:14 -0700, jorgev wrote:
>> * Since the list is very long, I can only afford to look at the
>> summaries and guess which may affect add-ons. Clearly, there are some
>> of them that will fall through the cracks. We discussed creating a
>> diff tool that runs for all XPCOM interfaces, but that hasn't happened
>> yet.
>
> The hg command:
>
> hg diff -rAURORA_BASE_20110705:AURORA_BASE_20110816 | grep "^diff.*idl$"
>
> would give you all the interfaces that are in idl, which is probably
> all that matters since extensions using C++ have to recompile
> anyway.
>
> -David
>
And what about public methods in browser.js, contentAreaUtils.js and other
browser JS files?
I'm not sure hg diff + grep is a reliable way to do this kind of checking,
either... :-(
Gijs
Paolo Amadini
>> What about asking patch authors and reviewers to set a keyword on the
>> bug if they think it can affect extensions, like we currently do with
>> "dev-doc-needed" for the MDN documentation?
>
> Sure, but you still need a way to find the cases where people forget to
> do that (and they will).
Of course, the keyword would be in addition to the other methods, like
checking interface and JSM differences during the Nightly -> Aurora
migration. The goal is to reduce the chances of changes falling though
the cracks, and not seek a bullet-proof solution.
But if a change not marked with the keyword is found using the
interfaces diff, I think it would be sensible not only to set the
keyword retroactively on the bug, but also to leave a comment saying
why the keyword was added, that is, the changes could affect add-on
compatibility. This way, I'm sure people will learn the process rapidly,
and will more rarely forget to set the keyword :-)
What I'm wondering is whether the existing "dev-doc-needed" keyword is
enough, or a separate keyword is needed.
Jorge, in your existing lists of bugs affecting compatibility in past
browser versions, how many bugs that affected add-ons compatibility
were not already marked with the "dev-doc-needed"/"dev-doc-complete"
keywords?
Maybe you need in any case to discriminate between the general
"dev-doc-needed" bugs and bugs specifically affecting compatibility?
Do you need to track patterns added to the add-on compatibility tool?
Cheers,
Paolo
Jorge Villalobos
I opened a new thread in here and in dev.planning about this. We have
defined a new keyword: addon-compat, since add-on compatibility is
somewhat orthogonal to documentation needs. I also listed some of the
most common add-on breaking changes, so that engineers are better aware
of what we're looking for.
- Jorge
Paolo Amadini
> I opened a new thread in here and in dev.planning about this. We have
> defined a new keyword: addon-compat, since add-on compatibility is
> somewhat orthogonal to documentation needs. I also listed some of the
> most common add-on breaking changes, so that engineers are better aware
> of what we're looking for.
That's excellent work. Thank you very much Jorge!
I'll review my open bugs and add the "addon-compat" keyword at once.
Paolo