Why "Pending Review" when its set to "Published to Testers"

[Michael Cann]

Hi,

We have a CI system that builds and publishes our extension to a dev extension every time we push the code. 

Unfortunately the process fails when an extension is "Pending Review". 

I understand why we need a review process for live extensions, but is there a reason we need it for extensions that are "Published to Testers"? The only people that can see the extension are on the testers list..

Cheers,

Mike

[Thierry Régagnon]

We have the same problem as Mike's team.

In our company, we use the "Private / Trusted testers" distribution to have an Extension for our QA team.

I don't understand why Private extensions requires such a long review. Particularly because changing the publication of an Extension (switching to Unlisted or Public) will trigger a review.

[Simeon Vincent]

All extensions submitted to the web store go through review, trusted testers included. Publishing to testers may be faster, but it's not a get out of jail free card. If you're only distributing the extension internally, you may want to consider using enterprise policies to force install extensions and self-hosting inside your org. See Managing Extensions in Your Enterprise for more. While it's possible to do this for free and without using any Google services, it's kinda a pain. I'd recommend considering using Chrome Browser Cloud Management to make this easier. CBCM also offers a free tier.

Cheers,

Simeon - @dotproto

Extensions Developer Advocate

[Michael Cann]

Hi Simeon,

I appreciate you sending me those links but man that does seems like a really hard way to go about things. Why not just change your policy so things work as we would expect. The term "Published to Testers" implies that its designed for a limited specific group of testers and not the public thus shouldnt need review. Why have this status at all if it just acts the same way as public publishing? 

As far as I am aware we can develop mobile apps for IOS and Android without having to go through review for every single build, why not the same for chrome extensions?

Thanks for all your hard work by the way, I realise you are massively under-staffed and are working as hard as you can. 

[Simeon Vincent]

The term "Published to Testers" implies that its designed for a limited specific group of testers and not the public thus shouldnt need review. Why have this status at all if it just acts the same way as public publishing? 

While there is some review for trusted testers, it's different than the public publishing. I don't know exactly how it's different because a) I can't publicly discuss the internals of the review process so b) I make a point of not learning too much in the first place lest I slip up.

To modify your second question, "why does trusted testers have any review?" Well, we're still distributing the extension, even if it's to a narrowly scoped group. As such, we don't want to, you know, send known malware to people. I honestly don't have a good bearing on how long tester review takes, so I'd love to hear more about your experiences.

As far as I am aware we can develop mobile apps for IOS and Android without having to go through review for every single build, why not the same for chrome extensions?

Boy howdy, I don't want to touch the discrepancy between Google Play, Apple's App Store, and the Chrome Web Store with a 10 foot pole. Speaking 100% from a personal point of view, the only thing I'd point out is that it took years for those ecosystems to get good release tooling, and that was despite having significantly larger development communities and revenue impact on their respective companies. 

Simeon - @dotproto
Extensions Developer Advocate

[Michael Cann]

Well, we're still distributing the extension, even if it's to a narrowly scoped group. As such, we don't want to, you know, send known malware to people.

Ye well to be honest I think thats really being over-cautious. Us the developers choose the people that can and cannot access the extension and then the tester has to manually choose to install the extension, IMO there is a contract there that the tester is fully aware of, its pretty much the same as just zipping up the extension, uploading it to FTP and telling people to go ahead download it and install it manually.. but its a whole bunch more work to do it that way.

I honestly don't have a good bearing on how long tester review takes, so I'd love to hear more about your experiences.

Days minimum, really this is annoying when we are wanting quick turn-around of features (CI / CD).

If you think its going to be impossible to change this rule please can you change it so that if an extension is "Pending Review" and we submit another one (via CI for example) then it simply replaces the release that is pending review, or is queued up?

Boy howdy, I don't want to touch the discrepancy between Google Play, Apple's App Store

Ye I realise that we are in the dark ages somewhat but with more and more browsers adopting Chrome extensions as the defacto way to do extensions in chromium then IMO the pressure is on you guys to improve the developer experience. Even tho there are less people and less money, its certainty not insignificant!

[馮韋元Francois Devatine]

Just wondering if sharing a zip file to the team would maybe help speed things up?

Extensions don't need to be on the chrome store to be distributed. But I agree that if it's more than just 3 or 4 technical people, then having it on the chrome store is quite helpful for faster install.

[Michael Cann]

Ye thats what we are currently having to do. I have a build process on my CI that zips then sends to slack to tell the team to test, but obviously its not ideal as its a manual process rather than letting chrome just auto-update it.

[Doug Jodrell]

I honestly don't have a good bearing on how long tester review takes, so I'd love to hear more about your experiences.

It can take hours or days. There is very little consistency, and absolutely no visibility into how long it'll take.
This is quite a roadblock in our release process because the extension needs to be coordinated with other aspects of a product release.
Our latest production release was pending review for several days before being rejected because the description (which hadn't changed) wasn't detailed enough.

That's another thing; the pass/rejection decision seems arbitrary. 

Some ideas for improvement:

• show us our position in the review queue.
• give us the ability to pre-validate a package so we can control the timing of a release.
• allow us to update a package that is pending review (without loosing our place in the queue) or let us queue up new versions for review so that our internal development process doesn't stall out while a release is pending review.

[Doug Jodrell]

@Michael Cann

We have a CI system that builds and publishes our extension to a dev extension every time we push the code. 
Unfortunately the process fails when an extension is "Pending Review". 

We updated our scripts to only publish if the manifest version is greater than the store version.
The release is triggered when a new manifest version is committed.
It's still not ideal because it requires a manual effort to check the store version, but it's better than having all our build fail while a review is in progress.
I can share the relevant snippets from our config if you are interested.

[Michael Cann]

We updated our scripts to only publish if the manifest version is greater than the store version.

To be honest we have something similar. Each build has a unique auto incrementing build number so every single push to git produces a new version where the first two digits are controlled by us the developers and the last one is the build number e.g. v1.11.153 then v1.11.154 etc 

We changed our build to:

- yarn produce:staging || true

Which means even if the publish part of that fails due to pending review it wont fail the whole build, then the next step after that we zip it up and send it to slack basically circumventing the whole store... sigh...