Are "Unlisted" Chrome extensions reviewed the same as "Public" ones?

[Sean Wilson]

Hi,

I have an unlisted version of my Chrome extension I use for beta testing and a public one for the production version. Recently I published several updates to my unlisted Chrome extension. All updates passed review.

A few weeks later, I pushed the recently accepted code as an update to the public Chrome extension and I got a "blue argon" rejection that singled out a snippet of code as possibly running remote code (a false positive I believe). I'm going to run the beta testing again once this is fix.

1. Why did the public update get rejected but not the unlisted one? Are public Chrome extensions reviewed more thoroughly than unlisted ones? Or maybe new automated checks were added between the times I was doing my uploads?

2. Does testing your upload as a beta before uploading to your public/production Chrome extension reduce the chances of your Chrome extension being removed if it gets flagged in review? I'd like to be as cautious as possible here as some of the stories on here about Chrome extensions being removed are scary.

(extension ID if someone wants to look into this: dagohlmlhagincbfilmkadjgmdnkjinl)

Thanks,

Sean Wilson

[Oliver Dunk]

Hi Sean,

Looking at your specific extension, it looks like you are using Firebase. There are some known issues where the SDK loads some remote code - we're looking at trying to make that easier but since we treat all code the same that is in scope for a Remote Hosted Code violation.

You might find this guide we recently published helpful. Under "What if you can't wait for a library update" there are some suggestions for ways to resolve this. I know that has worked in most cases for Firebase auth, other than a few edge cases when using certain auth mechanisms.

To answer your two questions, we try to be as consistent as possible in review. Having submitted an extension already as a beta doesn't influence the review verdict of another submission to your stable item since the reviews are done independently. However, it might help you to spot issues earlier. Something to note is that in extreme cases, we always give a warning before removing an item - you can learn more about the process here: https://developer.chrome.com/docs/webstore/review-process/

I hope that helps.

Oliver Dunk | DevRel, Chrome Extensions | https://developer.chrome.com/ | London, GB

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/aa6ff331-86b8-4c80-8b72-eabdd86bdc01n%40chromium.org.

[Sean Wilson]

Hi Oliver,

> To answer your two questions, we try to be as consistent as possible in review.

Thanks for looking into this. What confused me here is the Firebase related warnings look to be automated because the (non-recommended) fixes I found online suggest you could workaround them by blanking out some substrings e.g. URLs containing "api.js" and "gapi.js".

If these are automated warnings, why would my unlisted/beta update in December pass review but fail when uploaded as a public update? The potential fix required significant changes, which is going to require another phase of beta testing so it's a confusing and frustrating surprise.

> Something to note is that in extreme cases, we always give a warning before removing an item

I'm hearing about cases like this (which is related to warnings about Firebase also):

https://github.com/firebase/firebase-js-sdk/issues/7617#issuecomment-1734580890

"""However, the initial policy violation email states that if "...if the new draft fails to comply with our policies, both the draft and the existing store listing will be removed." We can't risk our extension being taken down but we also urgently need to update the extension."""

I obviously don't have all the details but I'm very worried about getting removal warnings like this and I'm trying to be as cautious as possible. Is it likely if I upload several potential Firebase fixes to an established extension in a row that get rejected I might get a warning back saying I only have one last chance until removal?

Along with this, my nightmare scenario for the future is if a buggy update gets accepted and then I'm unable to upload a fix quickly because of surprise rejections and/or up to 3 week review times. A way to rollback to a previously accepted update would help a lot here.

Thanks,

Sean Wilson

[Oliver Dunk]

If the new draft fails to comply with our policies, both the draft and the existing store listing will be removed.

Thanks for flagging this. I confirmed with the team that the text is inaccurate and we're going to look at getting it updated. To confirm - other than in extreme cases, you'll always be given a timeline within which to address any violations. You can submit as many drafts for review as needed within that time. Your published item will only be taken down if it is still in violation after the deadline.

What confused me here is the Firebase related warnings look to be automated because the (non-recommended) fixes I found online suggest you could workaround them by blanking out some substrings e.g. URLs containing "api.js" and "gapi.js".

These aren't automated - blanking out the URLs does help towards fixing this as it means the browser will no longer load remote scripts. However, tree-shaking the code to load scripts is also something I'd recommend since if that code exists but is unused it may also be flagged by a reviewer.

A way to rollback to a previously accepted update would help a lot here.

We're actively working on this and are hoping to launch something by June :)

Oliver Dunk | DevRel, Chrome Extensions | https://developer.chrome.com/ | London, GB

[Sean Wilson]

Hi Oliver,

Thanks, that helps! Rollbacks would really help with making more frequent releases practical. I think right now the only other option if a bad bug goes live would be to have a zipped/unpacked version of your extension on your own website that users could manually install while they wait for the fix.

> other than in extreme cases, you'll always be given a timeline within which to address any violations.

My concern here is if the timeline is something like 3 weeks and reviews can take up to 3 weeks, you might only get one chance to fix it even if you have a fix ready immediately?

It looks like new drafts/submissions can trigger removal deadlines as well https://developer.chrome.com/docs/webstore/review-process/ 'Existing published extensions are occasionally subject to review outside of the standard submission time review process...A warning is sent to the developer about the violation. The developer has a set amount of time to address the violation before the item will be taken down...In some cases, a violation detected in a submission may trigger a review of the published extension." 

I get that I'm considering worst case scenarios here but it's important to understand when you have users relying on your extension. The Firebase issue was nontrivial to fix for example (the treeshaking solution only works for some versions of Firebase, it can require build system changes/upgrades, retesting).

Thanks,

Sean Wilson

[Oliver Dunk]

Hi Sean,

I definitely appreciate the feedback. I would be very surprised if you ever ended up in a situation like this though - as you mentioned there are a lot of worst case assumptions and generally reviews are much faster.

If this did ever happen, we would certainly be willing to take a look.

Oliver Dunk | DevRel, Chrome Extensions | https://developer.chrome.com/ | London, GB

[Likely Logic]

Hi Sean, Oliver...

I've been following this thread with interest.

Not to hijack, but I find that my extension (with a fairly hefty codebase) has gone from 1 - 14 day review times a couple of years ago to under an hour as of Summer last year.

The only time it takes longer (maybe 1 - 2 days) is if there's a problem, I presume because it would get flagged for manual review.

Oliver, can you shed any light on why it might go through so quickly these days?

Cheers,

Dave

[Oliver Dunk]

I can't share any specifics, but glad to hear you've seen improvements there.

There is a combination of manual and human review, which can help to keep things moving, and in general we try to find places where we can make changes to speed things up.

Oliver Dunk | DevRel, Chrome Extensions | https://developer.chrome.com/ | London, GB