Extension signing vs. alpha / beta extension versions

[Jonathan Kamens]

Under this plan, how will alpha / beta versions of
add-ons hosted on AMO be handled?

Right now, when I push a beta version of my Thunderbird
add-on to AMO, it is available immediately for users
who are in my beta channel (i.e., users who already
have an alpha or beta version installed) and pushed to
them the next time Thunderbird checks for updates.

What will the workflow be for beta versions once
extension signing is enforced?

(I am assuming all this extension signing stuff applies
to Thunderbird?)

[Jorge Villalobos]

On 2/10/15 1:22 PM, Jonathan Kamens wrote:
> Under this plan, how will alpha / beta versions of
> add-ons hosted on AMO be handled?

They will probably get the basic signing that non-AMO add-ons will get,
since they aren't typically code reviewed.

> Right now, when I push a beta version of my Thunderbird
> add-on to AMO, it is available immediately for users
> who are in my beta channel (i.e., users who already
> have an alpha or beta version installed) and pushed to
> them the next time Thunderbird checks for updates.
>
> What will the workflow be for beta versions once
> extension signing is enforced?
>
> (I am assuming all this extension signing stuff applies
> to Thunderbird?)

There are no plans to implement this on Thunderbird, since Thunderbird
extension malware hasn't been a problem in the past. So it shouldn't
make a difference if your extension is signed or not for it to work on
Thunderbird. Whether we automatically sign AMO extensions for
Thunderbird or we don't will probably depend on what's easier to
implement for AMO.

Jorge

[luck...@musites.com]

On Tuesday, 10 February 2015 20:25:48 UTC, Jorge Villalobos wrote:
> On 2/10/15 1:22 PM, Jonathan Kamens wrote:
> > Under this plan, how will alpha / beta versions of
> > add-ons hosted on AMO be handled?
>
> They will probably get the basic signing that non-AMO add-ons will get,
> since they aren't typically code reviewed.

It is important that beta versions can be distributed near-instantaneously so I hope that the proposed automated signing process will not be trying too many fancy code-profiling tricks - for example, the current AMO submission validation warning level is very easy to set off for innocuous reasons. I think relying on very basic checks and user-level blocks would be appropriate, at least to start with.

> There are no plans to implement this on Thunderbird, since Thunderbird
> extension malware hasn't been a problem in the past. So it shouldn't
> make a difference if your extension is signed or not for it to work on
> Thunderbird. Whether we automatically sign AMO extensions for
> Thunderbird or we don't will probably depend on what's easier to
> implement for AMO.

As an author of an add-on with a single version that can be installed to Firefox or Thunderbird I would strongly prefer a consistent behaviour in order to avoid causing users unnecessary concern about why the addon is unsigned when they install it to Thunderbird.

Overall though, I must say that I'm happy with the signing plan and look forward to it rolling out to all Firefox users.

Thanks,
Chris

[Jorge Villalobos]

On 2/10/15 3:38 PM, luck...@musites.com wrote:
> On Tuesday, 10 February 2015 20:25:48 UTC, Jorge Villalobos wrote:
>> On 2/10/15 1:22 PM, Jonathan Kamens wrote:
>>> Under this plan, how will alpha / beta versions of
>>> add-ons hosted on AMO be handled?
>>
>> They will probably get the basic signing that non-AMO add-ons will get,
>> since they aren't typically code reviewed.
>
> It is important that beta versions can be distributed near-instantaneously so I hope that the proposed automated signing process will not be trying too many fancy code-profiling tricks - for example, the current AMO submission validation warning level is very easy to set off for innocuous reasons. I think relying on very basic checks and user-level blocks would be appropriate, at least to start with.

We want to avoid blocking add-on distribution as much as possible, both
for beta versions and all non-AMO add-ons. We will definitely aim for
this process to be as quick and easy as possible.

>> There are no plans to implement this on Thunderbird, since Thunderbird
>> extension malware hasn't been a problem in the past. So it shouldn't
>> make a difference if your extension is signed or not for it to work on
>> Thunderbird. Whether we automatically sign AMO extensions for
>> Thunderbird or we don't will probably depend on what's easier to
>> implement for AMO.
>
> As an author of an add-on with a single version that can be installed to Firefox or Thunderbird I would strongly prefer a consistent behaviour in order to avoid causing users unnecessary concern about why the addon is unsigned when they install it to Thunderbird.

Since we're using the same signing system that already exists for
add-ons on both Firefox and Thunderbird, that shouldn't be a problem.
Signed add-ons will appear as signed on Thunderbird. There might be a
need to update the UI so the add-ons don't appear as signed by Mozilla,
but that should be a relatively minor issue.

> Overall though, I must say that I'm happy with the signing plan and look forward to it rolling out to all Firefox users.
>
> Thanks,
> Chris
>

Thanks,

Jorge

[Mike Connor]

On 10 February 2015 at 16:38, <luck...@musites.com> wrote:

> On Tuesday, 10 February 2015 20:25:48 UTC, Jorge Villalobos wrote:

> > On 2/10/15 1:22 PM, Jonathan Kamens wrote:
> > > Under this plan, how will alpha / beta versions of
> > > add-ons hosted on AMO be handled?
> >
> > They will probably get the basic signing that non-AMO add-ons will get,
> > since they aren't typically code reviewed.
>

> It is important that beta versions can be distributed near-instantaneously
> so I hope that the proposed automated signing process will not be trying
> too many fancy code-profiling tricks - for example, the current AMO
> submission validation warning level is very easy to set off for innocuous
> reasons. I think relying on very basic checks and user-level blocks would
> be appropriate, at least to start with.
>

If the validator is inconsistent and/or wrong, we should work to improve
that. However, as the automated scanning process is a key part of the
security/reliability story, I would expect that we're more likely to expand
the scanning process to be as effective as possible, vs. reducing the
coverage somehow.

-- Mike

[luck...@musites.com]

On Tuesday, 10 February 2015 22:25:13 UTC, Mike Connor wrote:
>
> If the validator is inconsistent and/or wrong, we should work to improve
> that. However, as the automated scanning process is a key part of the
> security/reliability story, I would expect that we're more likely to expand
> the scanning process to be as effective as possible, vs. reducing the
> coverage somehow.

Of course I agree that it would be good to improve the scanning coverage. My impression was that the code signing solution was important enough to ship ASAP and implementing a reliable Javascript code scanning process is very difficult/time consuming.

If code signing needs to ship before the code scanning test suite is reliable enough, I would prefer it if only 100% reliable tests are included in the initial scanning test suite, or that there is at least some kind of weighting given to each test such that (for example) an add-on developer with no history of malicious addons and a history of including n "suspicious" (but manually approved) test results in previous addon versions can successfully automatically publish a beta version with 2n "suspicious" test results.

For all I know, the warning messages I frequently see in AMO validation test results are unrelated to the scanning being proposed here but if they are related, I still think it's going to be challenging to reliably convert those tests from "an AMO reviewer should look at this before approving the add-on" to "this addon is definitely so unsafe that we won't even allow a beta version to be distributed".

Or perhaps I'm just underestimating the resources Mozilla can bring to bear on the problem of Javascript code analysis :-)

Thanks,
Chris

[Jorge Villalobos]

The current AMO validator has a few error tests, where the add-on is
blocked from being submitted, and many warning tests that just appear in
validation results for reviewers to check during the manual code review.

Since we're introducing a category of add-ons that won't be code
reviewed, that means we need to create a different set of rules for that
case. It's likely going to be a subset of current checks, and some
additions, which will be upgraded to be errors rather than warnings. The
exact tests that we will be running in these cases are still to be
determined.

Jorge