MV3 service worker broken after auto-update

[Kyle Edwards]

I've had a hard time keeping users on the latest version of my MV3 extension with chrome's default auto-update behavior - as a result I've been playing around with a "forced auto-update" approach, as outlined in the extension docs. It was a simple addition to the service worker -  it reloads + installs the latest version of my extension as soon as the browser downloads the updated package: 

```

chrome.runtime.onUpdateAvailable.addListener((details) =>

  chrome.runtime.reload()

);

```

I've seen this exact block of code in other well established extensions in the wild, so I'm lead to believe it's a reasonable / safe approach to take.

I've pushed this change to my "trusted testers" beta channel, and most of the time it works like a charm.  For a couple of users something goes wrong and the service worker stops loading all together - ie all message passing stops working, the user can't launch the service worker inspector.  This behavior even persists through chrome restarts.  The only two ways to fix the service worker are 1) to push another update that overwrites the "bad" update (it's odd the issue is intermittent with users). or 2) have the user remove / re-install the new version.

The users with broken service workers are on the same chrome version as users that experience no issues.  No errors appear in chrome://extensions and content scripts continue work as expected (...except for message passing).  

Has anyone else experienced this behavior with MV3 service workers - with or without ?  Could this be an MV3 bug?  

Cheers,
Kyle

[Kyle Edwards]

There was a confusing typo in my last paragraph - it should be: "...with or without auto-update**?"

[wOxxOm]

Assuming there's no conditional global code at the start of the script that may produce an unhandled exception, definitely sounds like a bug. To investigate more try opening any extension page (or just chrome-extension://*******/manifest.json), then open devtools -> Application -> Service Worker, and see what's different there.

[Simeon Vincent]

The behavior you describe sounds reminiscent of a service worker (SW) failing to register at installation time, but in the current versions of the stable, beta, and canary channels there will at least be aa (minimally helpful) error logged in the extension's error view (chrome://extensions/?errors=<ID>).  The really odd part is the intermittent nature of the problem. 

Are you doing anything with serviceWorker.register or serviceWorkerRegistration.update? I'm asking as the extension platform handles SW registration for you, so you should not need to manually modify the root scope SW, and doing so may lead to unexpected results.

Assuming it is a worker registration issue, is there anything in your SW that could conditionally throw on the first execution of the script? If so, that may account for the behavior you're seeing. You can also ask affected users to take a look at chrome://serviceworker-internals/. You should be able to search for the extenisn's ID on this page; if it is not present, the SW is not registered.

Another thing you may want to try is to wrap your SW in a try/catch. The least disruptive way to do this is to introduce a try/catch wrapper script as your background context. Quick example…

Old manifest

{

...

"background": {

"service_worker": "background.js"

}

...

}

New manifest

{

...

"background": {

"service_worker": "wrapper.js"

}

...

}

wrapper.js

try {

importScripts("background.js");

} catch (err) {

console.error(err);

}

This also makes it easy to remove from your project when Chrome addresses the underlying issues that necessitate its use.

Simeon - @dotproto

Chrome Extensions DevRel

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/f20a4033-e48e-41fe-88ab-91f82270c443n%40chromium.org.

[Kyle Edwards]

Circling back as the issue has persisted - our user base is growing, making updates increasingly painful.  We have 1000+ users and ~1% of them are impacted each release.  It's a seemingly random distribution - it hardly(or never) affects the same user twice.   The extension is rendered completely useless; you can't even inspect the service worker at chrome://extensions/.  No error is shown, and a browser(or computer) restart doesn't help.  The only solution is to manually reinstall the extension.

As a reminder, the issue started after moving to MV3, and included a change for forced update (a documented pattern we've seen other popular extensions take advantage of), as well as error reporting with Sentry.

Here's the published extension: https://chrome.google.com/webstore/detail/brighthire/mbokiighkhdopgedihndpibkincpcgan

@wOxxOm or @Simeon - if you're curious, here's the service worker source (~200 lines):
https://gist.github.com/kltdwrds/adc956705147d32aada0e967c0a92bce

As per your suggestion, I added the SW wrapper try..catch, but have yet to actually captured an exception. 

Appreciate all your help with extension development + community support!

Cheers,
Kyle

[Simeon Vincent]

Thanks for following up on this. I was already planning to follow up with a couple engineers to discuss the extension service worker lifecycle and update flow later this week. Since this seems very related, I'll make sure to pass along your report.

Simeon - @dotproto

Chrome Extensions DevRel

[wOxxOm]

Try checking navigator.serviceWorker.getRegistrations() after the update in a normal extension page such as the popup or a post-update notification page and if the returned array is empty or has more than one entry or the entry's parameters differ from the expected ones, ask the user to reinstall the extension and send a bug report to you with the info.

[Kyle Edwards]

Thanks all, we're working on reproducing the issue in our beta channel so we can capture a bug report.

@simeon feel free to tap me for more context, you can contact me on this thread or reach me at ky...@brighthire.ai.

@w0xx0m I think I follow, and am working with our internal beta users to capture the output from getRegistrations(). Please check my understanding:

1. Extension service worker update fails
2. Open a "normal extension page" - our extension doesn't have a popup, or a post-update notification page. We'll use chrome-extension://opnlcljcbpodfkedodibhbnegdkhappp/manifest.json
3. Open dev inspector on the extension page
4. Run `await navigator.serviceWorker.getRegistrations()`
   
5. Expand the array and take a screenshot (I've had no luck with console log / json stringify on this object)

Thanks again for your help.

Cheers,
Kyle

[wOxxOm]

Yeah, this will work. Ask them to expand the entries in the array and probably show them a gif how this all should work and look like.

[Vladimir Yankovich]

We faced a similar problem. Colleagues, tell me, did anyone find a solution? I am interested in how to restart SW from any tab, I have access to all hosts.

On Friday, November 12, 2021 at 3:19:53 PM UTC+3 vladimira...@gmail.com wrote:

help me figure out the site etherscan.io I am the owner , it is unclear how to withdraw funds from the exchange , I ask for help , you will receive a percentage of the profit if you figure it out

четверг, 11 ноября 2021 г. в 00:22:06 UTC+3, wOxxOm:

[Валера Киселев]

When i refresh dev extension or stop and run prod extension some time service worker die at all. When i close and open browser worker doesn't run and any listeners inside worker doesn't run it too. It tried register worker manually. Fore example:

override.html 

<!DOCTYPE html> <html lang="en"> <head>...<head> <body> ... <script defer src="override.js"></script> <body> <html>

// override.js 

navigator.serviceWorker.getRegistrations().then((res) => { 

  for (let worker of res) { 

  console.log(worker) 

  if (worker.active.scriptURL.includes('background.js')) { 

  return 

 } } 

 navigator.serviceWorker .register(chrome.runtime.getURL('background.js')) 

 .then((registration) => { 

  console.log('Service worker success:', registration) 

 })

.catch((error) => { 

  console.log('Error service:', error) 

 }) 

})

This solution partially helped me but it does not matter because i have to register worker on different tabs. May be somebody know decision. I will pleasure.

среда, 17 ноября 2021 г. в 02:11:08 UTC+3, yankovic...@gmail.com:

[Валера Киселев]

I created an issue, I hope the Chromium team will find a solution - https://bugs.chromium.org/p/chromium/issues/detail?id=1271154

среда, 17 ноября 2021 г. в 15:06:14 UTC+3, Валера Киселев:

[Kyle Edwards]

Update - we've reproduced the update issue in beta by releasing an update with no changes.

I was one of two beta users (out of 12) who experienced the issue - I was able to collect the debug output suggested by @w0xx0m (the other user's output was the same as mine):

No service worker registrations when visiting an extension page (manifest.json)

No inspector is launched when clicking "service worker":

Observations

We track users' current extension version by external message polling from our main application (extension responds with `chrome.runtime.getManifest().version`)  What's interesting is, every user with the issue is on the latest version - meaning an updated service worker received/responded to at least on message. This is new info: What I've been describing as an "update error" (since we tend to hear several reports of the issue after an update) might be more accurately described as a "service worker crash" or "service worker init issue" - because in a small number of cases it's silently unregistered after successfully updating.

I understand the timeline as follows:

1. Update published
2. Browser receives update
3. Chrome triggers `chrome.runtime.onUpdateAvailable`, which calls: `chrome.runtime.reload()` (documented here, here's our code)
4. Extension updated (new service worker registered?)
5. New service worker handles external message, loads manifest, responds with latest version number
6. Something happens? Service worker kaput :( user reports issue
7. `navigator.serviceWorker.getRegistrations()` returns an empty array, service worker inspector doesn't launch
8. User must manually re-install, or wait for our next update to be pushed very painful
   (we've also seen at least one case of fixing a broken service worker by flipping the enable / disable switch, but we recommend users re-install)

- For further debugging, here is our beta manifest.json

- The exact same codebase is deployed to our production extension

- Here's the service worker source: https://gist.github.com/kltdwrds/adc956705147d32aada0e967c0a92bce

Please let me know how I can help - we can't afford to continue using mv3 with this issue and we're even evaluating a move back to mv2. That's a move we would really like to avoid, given its sunsetting soon.

Appreciate the continued support!
Kyle

[Andreas Gruber]

I could reproduce the issue with our extension. It turns out that when you inspect chrome://serviceworker-internals/ we end up with TWO extension service workers and none of them works. The old one is ACTIVATED, but it does not work anymore because of chrome.runtime.reload() and the new one is INSTALLED and in WAITING WORKER state, but not activated.

Switching the extension off and on again, made it work again.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/44dad861-4098-4ab3-846c-75768a29ab73n%40chromium.org.

[Vladimir Yankovich]

All of these cases look like serious bugs in production.

@Simeon, help your early MV3 adopters :) Tell me, how can we speed up solving this problem? 

[Simeon Vincent]

Thank you all for sharing your observations and data gathering here. I'm passing along your most recent updates to the folks working on this part of the platform now.

Vladimir, if we can find a reliable set of steps to reproduce the issue, that would be ideal. Failing that, though, I suspect that the kind of data gathering Валера, Kyle, and Andreas have done in their most recent posts may help to the platform engineers understand and track down the issue. I'll follow up here if the extensions platform engineers have any other suggestions or requests.

Simeon - @dotproto

Chrome Extensions DevRel

[Cuyler Stuwe]

I’m just here to ask for an extension to the MV2 lock deadline, given that we’re only a couple months away from being forced to submit new extensions as MV3, and we can’t even guarantee basic things like being able to reliably update and run MV3 extensions for all users.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/CAFY0HLOfMrC353Wz95sTjaAMJo1_W1%3Dr6Ffo1YrqOOfAGdnUPA%40mail.gmail.com.

[Vladimir Yankovich]

In fact, it's very simple. 

1. Install any MV3 extension with service worker (SW) in developer mode. 

2. Make sure SW running through the console by running
//navigator.serviceWorker

    .getRegistrations()

    .then((res) => console.log(res))

3. refresh the extension several times with the button (look screenshot) 

4. Verify that the serviceWorker is broken by running 

//navigator.serviceWorker in the console

    .getRegistrations()

    .then((res) => console.log(res))

5. If SW is alive, repeat steps 3 and 4. Sooner or later (sooner rather sooner :) it will die, although it should be registered 100% of the time.  

[Vladimir Yankovich]

@Cuyler Stuwe, On the one hand I agree with you. On the other hand, postponing the end of MV2 support can relax the Chrome development team and personally for me it will be bad news :) That is why I am in favor of making an effort together to make MV3 fully ready as soon as possible.

[Andreas Gruber]

We could resolve the WAITING WORKER issue, by telling all content scripts to remove the iframe (part of the extension) that we have injected and to remove all listeners. After that we call runtime.reload() in the extension service worker and it seems that the old service worker is de-registered, and the new one gets registered.

[Валера Киселев]

Source code of this sample you find there https://github.com/kiselyou/extension-mv3-demo

Video to reproduce: https://take.ms/9ewbP

[Andreas Gruber]

Can anyone confirm if chrome.runtime.onUpdateAvailable gets called in MV3? I put a console.log statement inside the function, but I can never see it being logged.

[Miguel Espinoza]

Hi all, I just got a chance to catch up with this long thread. I reported the issue back in Aug, but never got traction. Glad to see there's finally some traction.

I can provide steps to reproduce with code and video (I hope this helps the extension team):

• bug report link: https://bugs.chromium.org/p/chromium/issues/detail?id=1242225
  
• Github repo with steps to repro: https://github.com/miguelespinoza/-v3-service-worker-update
• Loom Videos:
  • Walkthrough how to repro: https://www.loom.com/share/e5b12be75c64483f82a9f148e1026b63
  • Walkthrough of the codebase: https://www.loom.com/share/4c99022583d04d699dadf8764eb3ecca

A short summary:

I identified this issue before my launch, so decided to downgrade to MV2. I'm hoping this issue could be addressed soon to revert to MV3.

For my extension, this issue was caused by loading iFrames, I'm not familiar with the relationship between iFrames and service-workers, but this combination (reproed in the video) causes servicer-worker to break.

----

I hope this provides the information necessary to fix the issue, I'm happy to test new builds of Chrome to verify once this issue has been fixed

[Vladimir Yankovich]

Oh dear, it seems that the problem described by Valera and the problem described by Miguel are two different problems. I don't know who is responsible for MV3 development priorities, but it seems that these bugs must have the highest priority. 

[Kyle Edwards]

Agree with yankovic...@gmail.com - these look like two different but related service worker issues.  I can confirm that our specific issue is easily reproduced in @v.a.ki...@gmail.com's video: https://take.ms/9ewbP

+1 @mig...@ensopi.com - I'm also happy to help test chrome dev builds. 

Any timeline estimate for fix would be helpful when evaluating our alternatives - ie. MV2 downgrade or releasing a new MV2 for our users in the time being.

Kyle

[Simeon Vincent]

Any timeline estimate for fix would be helpful when evaluating our alternatives - ie. MV2 downgrade or releasing a new MV2 for our users in the time being. - Kyle

No timeline on a fix as of yet, but I'm actively discussing both issues raised here with the team. 

 

Simeon - @dotproto

Chrome Extensions DevRel

[Andreas Gruber]

In our case it really seems to be the iframes that we inject into some web pages. When we then add event listeners for the Service Worker 

* install

* activate

and for the Extension 

* chrome.runtime.onInstalled

We get the following order (when no iframe is injected):

1. SW install

2. chrome.runtime.onInstalled

3. SW activate

which makes sense.

When an iframe is still active, we only get a chrome.runtime.onInstalled event, while the service worker from the previous extension version stays active and does not update!

When the iframes are removed, the service worker will eventually update, sometimes resulting in the case that no service worker is registered at all and the extension is completely unresponsive. 

Clicking the inspect view on the service worker does not work, since there is none.

[Ceof1]

Thanks Simeon. 

Regarding your earlier post

>I was already planning to follow up with a couple engineers to discuss the extension service worker lifecycle and

>update flow later this week.

Not wanting to distract from this high priority issue, but there are also these two other open issues, 

- Issue 1152255 (open since November 2020) https://bugs.chromium.org/p/chromium/issues/detail?id=1152255

- Issue 1189678 (open since March 2021)  https://bugs.chromium.org/p/chromium/issues/detail?id=1189678

that affects an extension developed by my team and going by the discussion threads on both, extensions developed by other teams, which also appear to be Service Worker Lifecycle related.

If there is a discussion amongst the engineers on Service Worker Life cycle, could these be discussed also? 

Thanks in advance.

[Ceof1]

Hi @Simeon

Regarding my question below, my team has an Mv2 extension which we've published to the Chrome Web Store. We are doing our planning for 2022. The Mv3 upgrade of that extension is being planned for next year.

When we attempted to upgrade to Mv3 earlier this year we found we were blocked by this issue    https://bugs.chromium.org/p/chromium/issues/detail?id=1189678

which is an Extension Service Worker issue.

We haven't seen an update from Chromium team on the issue since September, the update was that it was been worked on.

My team is growing increasingly concerned about this, particularly given the reminder email we received from the Chrome Web Store, reminding us of the timeline for the phasing out of Mv2 extensions.  We'd appreciate feedback on the issue reported by https://bugs.chromium.org/p/chromium/issues/detail?id=1189678 , so that we can factor it into our planning

Thanks in advance.

[Vladimir Yankovich]

Santa give us an actual roadmap for Christmas with specific timelines and priorities for the development of the Google Chrome extension platform. God, this is a technology project, not a creative improvisation. Why so much obscurity and uncertainty? 

Yes, we're still shocked that you took away our background page. But let's start small, with transparency and responsiveness. I think that's the most important thing we, the community of Chrome extension developers, are lacking right now.

God, this silence in the Chromium bugtracker is exhausting. Just respond to our posts more than once a month and preferably with specific answers and deadlines, not by adding new silent people to CC :)

[Pratyush Raj]

Have been seeing this problem but not only after updating but also when it is installed for few days and randomly happens to some of our users.

I am not able to reproduce it constantly, but mainly happens when laptop wake from sleep and chrome was open or system is in low memory.

Our users have been complaining again and again about it. This has made our extension worthless.

[Simeon Vincent]

I am not able to reproduce it constantly, but mainly happens when laptop wake from sleep and chrome was open or system is in low memory.

If possible, please provide some additional data in order to help us understand how to reproduce the problem.

• What operating system are you using? Please include name and the build/version number.
  
• Does the length of the sleep matter? If so, how long does the device need to be asleep?
• Have you noticed any other trends in the user reports you've received?

Simeon - @dotproto

Chrome Extensions DevRel

[Vladimir Yankovich]

Simeon, could you share with the developer community the current status of this issue?

In my opinion, we are dealing with a critical bug in the stable version of the platform. The problem has been known for more than 2 months. At the same time, I only know that we all together accumulate information about the reproduction of the error. And what about fixing it? 

[Simeon Vincent]

Vladimir, there are several issues being discussed in this thread, so I'm not exactly sure which one you're referring to. For the moment I'm going to assume you mean crbug.com/1271154, "MV3 service worker broken after auto-update and manual refresh".

In short, I caught up with the eng team to discuss extension service worker startup issues last week and an engineer is investigating 1271154 now. We do not have an ETA on a fix, but this is a priority issue for us.

Simeon - @dotproto

Chrome Extensions DevRel

[Vladimir Yankovich]

Simeon, thank you very much, that's exactly what I wanted to hear: the task at hand, you share our position that this is a critical error. 

Have a good holiday, colleagues! May all the bugs remain in the past year :) 

[Shu]

Hello,

What should I advise to users, maybe there is something else they can do. If they update the script, sometimes it fails and SW doesn't work, re-install doesn't help (as they say), only if I insist and they reinstall the browsers it is fixed then. I hate this problem and lost many customers for this reason. i can't reproduce this and have few reports like this per day (from over 30 000 customers)

If I press the Script Icon in the Top corner of google chrome, to open up the script window it does#t open the script.

Nothing is happening then.

It’s google chrome Version 96.0.4664.110 (Offizieller Build) (x86_64) on Mac Catalina 10.15.7