Chrome bug affecting ~3-5% of users of at least 4 popular extensions. Help fix it?

411 views
Skip to first unread message

Jói Sigurdsson

unread,
Jun 10, 2022, 5:14:38 AMJun 10
to Chromium Extensions
Hi folks,

This is a follow-up from this older thread. After 8 weeks of myself and much of my team working to reproduce or otherwise figure out this Chrome bug, and try for workarounds in our extension to avoid it, I wrote up what we've learned so far and asked folks to help. My company is also posting two $4K bug bounties, one for a reproduction of the issue and the other for a fix.

Here is the write-up: A Chrome bug affects 3-5% of users of 5+ popular extensions. Help hunt it down!

If you can help in any way, please do. In particular I'm looking to connect with the authors of the extensions mentioned in the post (LastPass, Screencastify, Norton Password Manager, Mendeley Web Importer, CrxMouse Chrome Gestures, Save to Pocket and Mute Tab, as well as Google's own two extensions on the list) to make sure they are aware of the registered issue and can chime in there with any further details.

Cheers,
Jói

Deco

unread,
Jun 10, 2022, 10:39:29 AMJun 10
to Jói Sigurdsson, Chromium Extensions
Interesting, based on the readup and the information provided from the timing reports, this seems like there is a race condition of some sort triggering for the setTimeout(..., 0) function which Chrome uses to cipher JS tasks which take too long to process, the bug's seemingly random nature of appearing makes this a difficult bug to test as Chrome will automatically break down tasks, itself, without user intervention. 

Chrome version 102 has support for relaxing this timing clamp, you can find it under #unthrottled-nested-timeout, in theory, if this is the culprit of what is being triggered, then relaxing the clampdown should significantly reduce the occurrences of the error reporting.

Cheers,
Decklin

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/46fe91bc-49e5-4f28-8129-15d70eb78d4dn%40chromium.org.

PhistucK

unread,
Jun 10, 2022, 10:44:10 AMJun 10
to Deco, Jói Sigurdsson, Chromium Extensions
If I am not mistaken, since it has reportedly only started recently (a few versions back), I doubt throttling/clamping (which I believe have been there for years) are the culprit.

PhistucK


amit...@gmail.com

unread,
Jun 23, 2022, 11:37:11 AMJun 23
to Chromium Extensions, PhistucK, Jói Sigurdsson, Chromium Extensions, decklin...@gmail.com
Incredible writeup @Jól. I hope this Chrome bug will be prioritized and fixed!
Our users are experiencing the "extension stopped working" issue as well. We're using manifest version 2.
The browser action stops responding, no errors are seen in inspect background page while the extension is enabled but "off", and the only way we found to recover from that status is to disable and re-enable the extension (which is not something most users will do).

Jói Sigurdsson

unread,
Jun 23, 2022, 11:48:08 AMJun 23
to Chromium Extensions, amit...@gmail.com, PhistucK, Jói Sigurdsson, Chromium Extensions, decklin...@gmail.com
Thanks Amit! Bringing more attention to the bug should help the Chrome team prioritize it higher.

> The browser action stops responding, no errors are seen in inspect background page while
> the extension is enabled but "off", and the only way we found to recover from that status is
> to disable and re-enable the extension (which is not something most users will do).

Sounds like exactly the same as what's happening to our users. The worst thing is that there's no way to tell them what's going on since we're not getting anything (so it just looks like they're no longer using the system)... so they simply assume it's broken and uninstall it :/ 

Here's hoping we get through this together!!

Cheers,
Jói

Jói Sigurdsson

unread,
Jun 29, 2022, 4:55:42 AMJun 29
to Chromium Extensions, Jói Sigurdsson, amit...@gmail.com, PhistucK, Chromium Extensions, decklin...@gmail.com
Hi folks, if you know any capable Chromium freelancers who might be interested, we have now posted a job on Upwork for help tracking this issue down. Whoever is selected will be paid their hourly rate and will also remain eligible for the two $4K bounties we posted, at least until they've done $5K worth of work:

https://www.upwork.com/freelance-jobs/apply/Chromium-consultant-expert-look-elusive-timing-bug_~012592f6132a748e95/

Cheers,
Jói

hortius tossinou9

unread,
Jun 29, 2022, 10:03:39 AMJun 29
to Jói Sigurdsson, PhistucK, amit...@gmail.com, Chromium Extensions, decklin...@gmail.com

Thank you but I didn't understand anything!

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extensions+unsub...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/6848729f-4bc6-493d-84fc-4ecd3ca932fcn%40chromium.org.

amit...@gmail.com

unread,
Jul 3, 2022, 12:22:48 PMJul 3
to Chromium Extensions, hort...@gmail.com, PhistucK, amit...@gmail.com, Chromium Extensions, decklin...@gmail.com, Jói Sigurdsson
Well done Jói. 
Did anyone apply for your Upwork bounty? If not I will try to share this opportunity on some facebook groups.
Did you hear back from any of the major extensions that are affected by this issue? or from the Chromium team itself?

Jói Sigurdsson

unread,
Jul 4, 2022, 5:15:08 AMJul 4
to Chromium Extensions, amit...@gmail.com, hort...@gmail.com, PhistucK, Chromium Extensions, decklin...@gmail.com, Jói Sigurdsson
Hi Amit, I have somebody working on the Upwork job I posted, but I'm up for having one or two more people working on it if they have the right skillset, so I think it'd be great if you share it.

We did not hear back from any of the popular extensions affected :/

rdevlin@ from the Chromium team has responded a few times on the issue but there's been no other communication aside from what's visible on the issue tracker :/ 

Cheers,
Jói

Jói Sigurdsson

unread,
Jul 6, 2022, 3:35:09 PMJul 6
to Chromium Extensions, Jói Sigurdsson, amit...@gmail.com, hort...@gmail.com, PhistucK, Chromium Extensions, decklin...@gmail.com
The issue being discussed in this thread has achieved its 15 minutes of fame with coverage in The Register: https://www.theregister.com/2022/07/05/subtle_chromium_bug_extensions/

The reporter reached out to me yesterday wanting to send some questions but unfortunately I was out sick and didn't see their message until too late, but I think it's good coverage nonetheless and hopefully will help bring attention to the issue.

In other news we have tried again to reach out to all of the companies mentioned in the article via their support channels to see if we can get them to chime in whether this seems to be the same bug that is affecting their users. So far only one response of the "works on my computer" variety :)

Cheers,
Jói

Uladzimir Yankovich

unread,
Jul 7, 2022, 4:19:49 PMJul 7
to Chromium Extensions, Jói Sigurdsson, amit...@gmail.com, hort...@gmail.com, PhistucK, Chromium Extensions, decklin...@gmail.com
As far as I understand, we're talking about a problem we discovered back in November of last year - https://bugs.chromium.org/p/chromium/issues/detail?id=1271154. When we found it, I was sure the extension platform team would mark it as a critical bug, but nothing seemed to have happened. Too bad...

Jói Sigurdsson

unread,
Jul 8, 2022, 9:26:51 AMJul 8
to Chromium Extensions, yankovic...@gmail.com, Jói Sigurdsson, amit...@gmail.com, hort...@gmail.com, PhistucK, Chromium Extensions, decklin...@gmail.com
You are correct, it's either the same bug with the same underlying cause as the issue you mentioned (crbug.com/1271154), or a very similar bug for MV2 extensions, details on the MV2 bug at crbug.com/1316588

Simeon Vincent

unread,
Jul 13, 2022, 1:31:00 PMJul 13
to Jói Sigurdsson, Chromium Extensions, yankovic...@gmail.com, amit...@gmail.com, hort...@gmail.com, PhistucK, decklin...@gmail.com
Just tossing in my 2 cents, but I don't think crbug.com/1271154 and crbug.com/1316588 are the same issue. The former seems to be related to how the extension platform handles unregistering an old service and registering a new one during an extension update while the latter describes a more general failure to dispatch events to background pages. IMO both are critical issues.

IMO the slow progress on crbug.com/1316588 isn't so much an issue of prioritization as the fact that it's incredibly difficult to investigate this issue when we cannot reproduce it. To that end, I want to acknowledge and thank you, Jói, for the amount of effort you've put into trying to track this down. 

Simeon - @dotproto
Chrome Extensions DevRel


Jói Sigurdsson

unread,
Jul 14, 2022, 5:59:02 AMJul 14
to Chromium Extensions, Simeon Vincent, Chromium Extensions, yankovic...@gmail.com, amit...@gmail.com, hort...@gmail.com, PhistucK, decklin...@gmail.com, Jói Sigurdsson
Thank you Simeon!

I agree it's not a given that crbug.com/1271154 and crbug.com/1316588 are related, but the reason I feel like there's still a possibility they are, is that the MV2 issue seems to only reproduce when there's a lot of the background page becoming inactive, then re-activating, which feels like it might possibly be related to the MV3 issue which as you mentioned is related to registering a new service worker. Also, in our testing and repro cases (see comments #48, 49, 50, 52), the MV3 issue would reproduce not only during an extension update but also randomly without any change incoming, just like the MV2 issue.

> IMO both are critical issues.

I agree. Any chance you might be able to convince folks to bump the priority on crbug.com/1316588 to P1? It's currently marked as a P2.

You're totally right that it's hard to fix when it's so hard to reproduce (I have been trying now for several days on a release-with-logging build of Chromium), but that shouldn't affect the priority rating, I don't think, as I believe it is affecting tons of users.

In any case, thank you Simeon, for your ongoing support of this community of extension developers. I don't know where we'd be without you!!

Cheers,
Jói

amit...@gmail.com

unread,
Jul 28, 2022, 5:47:19 PMJul 28
to Chromium Extensions, j...@crankwheel.com, Simeon Vincent, Chromium Extensions, yankovic...@gmail.com, amit...@gmail.com, hort...@gmail.com, PhistucK, decklin...@gmail.com
Hi @Simeon,
Thanks for helping with this. I'm joining Jói's question about potentially prioritizing the related Chrome issue to P1.
We're losing users to this bug every day, and obviously so do other extension developers which aren't necessarily aware of the reason.
If there's anything else we can do to support, let us know.
Thank you
Amit

PhistucK

unread,
Jul 28, 2022, 7:11:43 PMJul 28
to amit...@gmail.com, Chromium Extensions, j...@crankwheel.com, Simeon Vincent, yankovic...@gmail.com, hort...@gmail.com, decklin...@gmail.com
Note that setting the priority to 1 will not magically generate an easy reproduction, this is why it cannot be 1.

PhistucK

Amit Rosner

unread,
Jul 28, 2022, 7:46:46 PMJul 28
to PhistucK, Chromium Extensions, j...@crankwheel.com, Simeon Vincent, yankovic...@gmail.com, hort...@gmail.com, decklin...@gmail.com
Could be. I honestly don't know how these priorities are defined. In my opinion, difficult reproduction is a secondary factor to the number of users that suffer from unavailable service when it comes to setting priorities. Perhaps the 3,000+ currently assigned priority 1 issues are more important.

It's constantly happening on my laptop. How may I help debugging?

PhistucK

unread,
Jul 29, 2022, 4:45:29 AMJul 29
to Amit Rosner, Chromium Extensions, j...@crankwheel.com, Simeon Vincent, yankovic...@gmail.com, hort...@gmail.com, decklin...@gmail.com
If you are able to build Chromium with the added-logging patch provided in the issue (or maybe Jói can provide you with an already built binary), you can run it and attach the logs to the issue once it happens.

PhistucK

Jói Sigurdsson

unread,
Jul 29, 2022, 5:09:48 AMJul 29
to PhistucK, Amit Rosner, Chromium Extensions, Simeon Vincent, yankovic...@gmail.com, hort...@gmail.com, decklin...@gmail.com
I do have a pre-built binary for macOS with the logging, that I can share. I am traveling today but tomorrow morning I could send a link to download the pre-built macOS files if that works for you Amit - you can respond to me privately to let me know (joi@ my company name dot com).


Jói Sigurdsson / Founder & CEO 
j...@crankwheel.com

CrankWheel 
+1 (877) 753-2945
Dalshraun 1, Hafnarfjordur 220, Iceland 
http://crankwheel.com/

Reply all
Reply to author
Forward
0 new messages