What are the execution time limits for the service worker in Manifest V3?

[ErikRothoff]

My extension is an RSS reader that periodically polls for new posts in feeds that the user has added. From my understanding, MV3 will require this to be setup using chrome.alarms, which can at most be executed every 1 minute. 

So for each "runloop" iteration we need to fire off a bunch of requests. Say the user has 30 feeds, and some servers are responding slowly (sometimes it can be as slow as 1 minute to fetch a URL). Will the service worker be killed by Chrome before the slow requests are finished?  How long can it execute before being killed in that case?

If not, could I create a "persistent" background page by making a GET request to our server and just sleep(10000000) so the service worker isn't killed? (I'm not saying this is a good idea, I just want to understand how the service worker works). I take it this is not the case since WebSockets won't be allowed in service workers (which is really sad for our user case).

[wOxxOm]

The service worker will unload regardless of pending fetch() requests in approximately 15 seconds. The exact duration isn't fixed though so don't rely on that.

This problem was mentioned right after the initial announcement of ManifestV3 but we never got a *reasonable* response, only a bunch of pep talk. I don't know how service workers are implemented internally but the fact that Chrome terminates them even with pending network requests looks to me like they are *inherently* broken. The same was the case with event pages though.

This workflow is another victim of ManifestV3's unsubstantiated claim that there's no need for persistent background scripts. Unfortunately, the switch to service workers is a done deal which means any talk about persistence will be ignored or dismissed. The practical workarounds would be the same I mentioned in firebase thread:

1. A kinda lame workaround is to open one port to any tab's content script and keep it open (indefinitely or until the user signs out of firebase if you support such an action). This will prevent unloading of the service worker, which is a documented feature. You can use the port's onDisconnect event to repeat the process with a different random tab. When there are no connectable tabs (i.e. you can't connect to an empty new tab as it doesn't run content scripts in modern Chrome) you'll either have to let it sleep or open a new dummy tab just to have a port connection, which is why this workaround is kinda lame.
   
   
2. An even more lame workaround is to have a dedicated tab, which you can try to make less lame by tricking the users into believing this tab is useful: display some info about your extension, live graphs, indicators, stats, pulse. You can set the tab's state to "pinned". You can also use "beforeunload" event listener to prevent the tab from being closed accidentally. Also show something like "If you don't like having this tab open please star the bug #### to convince Chromium team to find a better solution".
   

[ErikRothoff]

So a service worker won't be killed if a port is open, but will if a request is being made? That sounds more like a bug then a feature? Has this been confirmed by the Simeon and the team?

[wOxxOm]

This is how event pages always worked, and service workers are based on them so apparently it's not a bug, but a feature to ensure that service workers won't run for too long. However, it still doesn't make sense in case of extensions because unlike sites, there'll be just a few extensions knowingly installed by users so they shouldn't be treated like rogue actors.

[wOxxOm]

Difference is, an extension messaging port is treated like an external "hook" that keeps the worker hanging, whereas the network requests initiated by the worker are treated as its internal kitchen which can be collapsed any time just like any other pending stuff in a worker (setTimeout or a Promise, for example).

[Simeon Vincent]

The service worker will unload regardless of pending fetch() requests in approximately 15 seconds. The exact duration isn't fixed though so don't rely on that. - wOxxOm

This isn't quite right. Per the Service Worker Security FAQ, Chrome will terminate a service worker after it "has been idle for 30 seconds. Chrome also detects long-running workers and terminates them. It does this if an event takes more than 5 minutes to settle, or if the worker is busy running synchronous JavaScript and does not respond to a ping within 30 seconds."

In other words, while you can't call sleep() to extend your lifetime, you have a window of time for requests to settle and to process their responses. Also for what it's worth, as developers build MV3 extensions I plan to keep an eye out for service worker life cycle issues that may be indicative of platform gaps. 

I take it this is not the case since WebSockets won't be allowed in service workers - Erik

Service workers can open WebSocket connections, they just won't extend the worker's lifetime. One of the demos on my backlog is to create a simple Node.js server and extension that uses push notifications to wake the extension, WebSocket to sync data after wake, and then allow the extension to go back to sleep. 

Has this been confirmed by the Simeon - Erik

This made me laugh as it sounds like my first name became my job title ;)

Unfortunately, the switch to service workers is a done deal which means any talk about persistence will be ignored or dismissed. - wOxxOm

As a bit of unsolicited feedback, wOxxOm, my main frustrations with your communication style are your tendency to malign those you disagree with and to assign motive to their actions. I don't mean to tone police; you have every right to be frustrated and to vehemently object, but you do not have the right to cast aspersions on others.

The fact that we haven't been convinced that long-lived pages are the right solution for the extension platform does not mean that we ignore or dismiss feedback. I promise you that I personally read and consider as much discussion on this topic as I can and regularly surface these discussions with the extensions team. Besides documentation, I think this one of the most frequent topics that I discuss with extensions engineering and leadership. And I expect it will continue to be. That's in no small part because you continue to bring it up and we continue to value your insight. To be candid, I have half a dozen other things I need to be doing right now but I've spent more time on this email that I would like to admit because I want to work with you rather than to continue to butt heads.

Your feedback unambiguously states that there are things you can do today that would not be possible in the exact same way in the future. I don't think there's much disagreement there, but that's where communication seems to break down. In our view persistent background pages are untenable, so we need to figure out how to move forward. Service workers have drawbacks, yes, but they also have benefits that, frankly, we value more highly than you do. As we discuss cases that currently necessitate a persistent context, it seems that you conclude persistent background pictures are the only solution whereas we want to understand the problem space so we can find solutions that work with an event based model.

Cheers,

Simeon - @dotproto

Developer Advocate for Chrome Extensions

--
You received this message because you are subscribed to the Google Groups "Chromium Extensions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-extens...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/fc07ffea-5683-4e12-a1ce-71997724990en%40chromium.org.

[wOxxOm]

>  This isn't quite right. Per the Service Worker Security FAQ, Chrome will terminate a service worker after it "has been idle for 30 seconds 

15 seconds was the actual measured time. Either way, it's ignoring the pending requests which is the important point. 

> you have a window of time for requests to settle and to process their responses

An arbitrary inflexible duration, which isn't sufficient for unpredictable real world conditions as shown in this case. MV3 also doesn't provide an official way to postpone unloading the worker, only the workaround via extension messaging port. And stop saying it's for performance reasons because it's not: restarting the worker is much more costly than keeping an idle process for a minute.

> As a bit of unsolicited feedback, wOxxOm, my main frustrations with your communication style are your tendency to malign those you disagree with and to assign motive to their actions. I don't mean to tone police; you have every right to be frustrated and to vehemently object, but you do not have the right to cast aspersions on others.

Your frustrations, huh. You're new here. Imagine any extension author who has been waiting for something meaningful to happen with the extension API for the last ~5 years. For example, for the team to fix all those reported bugs, to add the features we asked. Of those hundreds reports only a couple are processed each year, but the whole bulk is just left unattended and ignored, periodically some reports are archived because it becomes apparent that no one will deal with them. They're dying and our trust is vanishing.

Now what happened instead is that they're breaking the entire platform by removing the persistent pages with a cherry on top: the blocking webRequest is also removed for non-forced-installed extensions. We the extension authors don't benefit from that. Our users don't benefit from that. You and the team ludicrously exaggerate the bad impact of persistent pages without providing any real measurements performed globally (like these).

The only good things added would be Promises (not yet added) and declarativeNetRequest (could be added to MV2).

> The fact that we haven't been convinced that long-lived pages are the right solution for the extension platform does not mean that we ignore or dismiss feedback. 

But the team effectively does ignore/dismiss it in the end. This decision was made 6 years ago or even earlier without any tangible proof based on global metrics. No one in the team has even tried to listen AFAICT.

> I promise you that I personally read and consider as much discussion on this topic as I can and regularly surface these discussions with the extensions team.

Stop taking it personally. You're a cool person and all that, but I don't see what you can change. The team isn't all that bubbly image you painted for me. They're primarily practical people so there are things they just can't do without the team being expanded several times e.g. from a couple of people currently to 5 or 10 people.

> Service workers have drawbacks, yes, but they also have benefits that, frankly, we value more highly than you do. 

That's a part of the problem the team creates: the documents/promos they've released are inflating the benefits of service workers while exaggerating the bad effects of persistent background pages. We the extension authors are not retarded. We are not sheep in need of herding. Show us the facts and global measurements or just stop selling the point of improved performance and admit the performance will be almost the same or negligibly better in some cases, negligibly worse in others (here you could mention what to do to avoid that in detail), and rarely much worse (here you need to list the known cases and say what you're doing to solve that). None of the documents say that. Fix that or finally admit that no one actually cares. None of the related reports on crbug get any meaningful attention anymore, and judging by the 5+ trend outlined above they might never get any.

> it seems that you conclude persistent background pictures are the only solution whereas we want to understand the problem space so we can find solutions that work with an event based model.

Any programmer who's not an idealist knows that when performance is concerned one needs measurements, statistics, or a proof-of-concept before making any change, especially a breaking one. The chromium team did neither of those judging by the fact we didn't see the data. The fact that you and the team are unable to even admit the possibility of a case where your assumption is completely wrong is also telling you're either idealistic or just arrogantly thinking your experience/understanding is superior. There are real-life examples shown over the last year here in the groups (uBlock Origin, Tampermponkey/Violentmonkey, Stylus/Stylish) that work much better with a persistent background script, which should be obvious to anyone, who's actually looking at what those extensions are doing. Their authors aren't posting on crbug though because the extensions will be broken by other changes in MV3. There are many other examples I gave that you've dismissed as being theoretical but they are based on real life cases that I observed for the last ~5 years on StackOverflow where I helped on literally thousands of questions that comprise quite a lot of various practical cases.

[wOxxOm]

>  Also for what it's worth, as developers build MV3 extensions I plan to keep an eye out for service worker life cycle issues that may be indicative of platform gaps  

This thread shows such an issue. Now is your chance to show us what you can do about it. A vague promise so far, which doesn't amount to anything, and then you veered to talk about your frustrations with me that could be seen as an attempt to avoid admitting the case in question is indeed exposing such a gap in the platform.

[wOxxOm]

Ah, almost missed one.

>  you do not have the right to cast aspersions on others.  

That wasn't what I thought I was doing. My only goal was to show the end result, the fact, as it appears from the point of view of a long-term extension developer who isn't an applauding groupie on a corporate event. I blame my spotty knowledge of the English language, not my native one. Still, as the saying goes, the road to Hell is paved with good intentions, so intentions don't matter if the observed results don't match. This applies both to me and the team. I'll work on that because my goal was always to help the team as I've been participating on crbug for several years (bisecting the regressions, testing the reports, verifying something is a bug or working-as-intended) so that a team member who is assigned would spend less time investigating.

[ErikRothoff]

wOxxOm, I appreciate your frustrations but I also feel that Simeon is here to listen and we need to take advantage of that to have a constructive dialog. I'm here to try to understand the limitations of the service worker, and understand how we might need to work around them to get our app working as our users expect it.

My main concern that I want to solve is: How do we update a bunch of URLs in the background at a regular interval, do some processing, then sleep? The following quote sounds hopeful:

> This isn't quite right. Per the Service Worker Security FAQ, Chrome will terminate a service worker after it "has been idle for 30 seconds. Chrome also detects long-running workers and terminates them. It does this if an event takes more than 5 minutes to settle, or if the worker is busy running synchronous JavaScript and does not respond to a ping within 30 seconds."

So if we setup a chrome.alarm to wake regularly, load the URLs from the database, then call `fetch` on them and then do some processing (parse RSS/JSON, etc), store results in IndexedDB, show notifications etc, then sleep. We don't have to worry about Chrome killing the SW if a request takes 30 seconds? Since that's technically not "idle"? It's more if the SW is mining bitcoin for 30 seconds thus not responsive to outside events?

> Also for what it's worth, as developers build MV3 extensions I plan to keep an eye out for service worker life cycle issues that may be indicative of platform gaps. 

English is not my main language, so not really sure what this means... :) Do you mean "platform gaps" as in, gaps in functionality caused by MV3 vs MV2? In that case it sounds very hopeful and I really want to continue having an open dialog.

I realise I could try to build this to see what happens, but I tried for a couple of hours and couldn't get the background SW to run at all so was getting a bit worried.

[wOxxOm]

> to have a constructive dialog 

That'd be awesome because so far it didn't happen and doesn't even look like it's about to happen. There's no any actual progress on chromium git repo or crbug.com for all of our feedback and efforts to be constructive in the most painful areas such as the blocking webRequest and a persistent background script. So far, we're only given explanations not based on any tangible data why we should rewrite our code, which effectively looks arrogant and condescending. One pleasant exception is the way the developer of declarativeNetRequest handled user feedback during the last year even though that's not strictly a part of MV3.

> So if we setup a chrome.alarm to wake regularly

No, chrome.alarms are throttled to one wake-up per minute so the worker will be dead by then.

>  We don't have to worry about Chrome killing the SW if a request takes 30 seconds? Since that's technically not "idle"?

No, it *is* idle. You can make a recording in devtools profiler and see what happens for yourself when the code makes a single request and then prints the results. There will be one event for the request, then nothing for 30 seconds (effectively "idle" state), then another event to print the results but in your case (30 seconds or a minute of network delay) the worker will be dead by then.

> Chrome also detects long-running workers and terminates them. It does this if an event takes more than 5 minutes to settle, or if the worker is busy running synchronous JavaScript and does not respond to a ping within 30 seconds."

I see a lot of confusion coming from the fact that most people don't understand what the JavaScript event loop is, how it works, what an "event" is, what a "task" is, what "idle" means, what "running" means. In the case we're looking at, there will be no event longer than a few milliseconds, there will be no synchronous JS running for more than a few millliseconds, so this quote is irrelevant here.

[wOxxOm]

> How do we update a bunch of URLs in the background at a regular interval, do some processing, then sleep?

Both your practical workarounds are listed in my first comment. As for theoretically possible solutions targeted at this case specifically, it'd be nice if someone from the team acknowledges the problem first because in this thread we didn't see such an indication so maybe you should open an issue on htttps://crbug.com where the discussions are more focused.

[Simeon Vincent]

15 seconds was the actual measured time. Either way, it's ignoring the pending requests which is the important point. 

I just verified with a service worker engineer that the 30 second and 5 minute lifespans should behave as described in that doc. If you haven't already, I'd appreciate it if you could file a bug. 

And stop saying it's for performance reasons because it's not: restarting the worker is much more costly than keeping an idle process for a minute.

I'm not sure what you're referring to; I didn't say anything about performance in my response. 

This decision was made 6 years ago or even earlier without any tangible proof based on global metrics. No one in the team has even tried to listen AFAICT.

Thanks for that link. This is the first time I've seen that comment or any evidence that that persistent background page depreciation was on the radar that long ago. I knew in abstract that moving away from persistent pages was being discussed well before I joined, but I didn't realize just how far back that went.

Show us the facts and global measurements or just stop selling the point of improved performance and admit the performance will be almost the same or negligibly better in some cases, negligibly worse in others (here you could mention what to do to avoid that in detail), and rarely much worse (here you need to list the known cases and say what you're doing to solve that). None of the documents say that. Fix that or finally admit that no one actually cares

I hear you; we (I) have overemphasized performance when discussing extension service workers. I acknowledged in another thread last week (I think it was last week?), but I'm happy to say it again to make my position more clear. Moving to service workers is in no small part motivated by reducing maintenance costs and bringing the extension platform into better alignment with the web platform. 

Our current migration documentation is a bare minimum MVP. I'm actually in the process of scoping out documentation changes and updates in the lead up to MV3 reaching stable. I agree with your assessment that we should better set expectations and document mitigation strategies. Add some notes to my outline to that effect. 

The fact that you and the team are unable to even admit the possibility of a case where your assumption is completely wrong is also telling you're either idealistic or just arrogantly thinking your experience/understanding is superior. 

I'm a little surprised at this take; I feel like I've stressed that we're targeting the middle of the bell curve and that there are cases that will be negatively impacted. If others agree with your assessment I'll redouble my efforts to communicate clearly and with empathy.

To your broader point in this paragraph, though, I am excited about the in-memory storage API (930235) and a small part because I anticipate it will help mitigate some of the issues you've raised around ephemerality. And as we continue to discuss it, I'll continue to advocate for the use case. 

Imagine any extension author who has been waiting for something meaningful to happen with the extension API for the last ~5 years.

As you know firsthand, the past five years have been an incredibly trying time for the Chrome extension platform. There is a period of disinvestment during which, as you've mentioned yourself, the extension platform team was basically one guy. It's hard to make progress when you're that understaffed. That's the context for why so little has happened during that time.

But we're not there anymore. To the best of my knowledge the Chrome team started to course correct shortly before I joined. In fact, my role was created because the team was investing more engineering resources in the extension platform. While I can't share the team's current headcount, I think you would be pleasantly surprised with the team's current size.

I'll work on that because my goal was always to help the team as I've been participating on crbug for several years (bisecting the regressions, testing the reports, verifying something is a bug or working-as-intended) so that a team member who is assigned would spend less time investigating.

Your efforts and insights are genuinely appreciated. Thank you for all the work you've done on crbug, StackOverflow, and here. You've helped a lot of people from new extension developers to Chromium engineers. 

P.S. I had no idea that English wasn't your first language!

My main concern that I want to solve is: How do we update a bunch of URLs in the background at a regular interval, do some processing, then sleep?

The options that come to mind are push notifications (web notifications or chrome.gcm), chrome.alarms, and periodic background sync. 

English is not my main language, so not really sure what this means... :) Do you mean "platform gaps" as in, gaps in functionality caused by MV3 vs MV2? In that case it sounds very hopeful and I really want to continue having an open dialog.

Sorry for the unusual phrasing. Yes, gaps in functionality. These gaps may be caused by the MV3 migration, or they may be features that would open up new possibilities for extension developers. 

I realise I could try to build this to see what happens, but I tried for a couple of hours and couldn't get the background SW to run at all so was getting a bit worried.

Are you using Chrome Stable or Chrome Canary? MV3 is currently only available in Canary. I'm hoping in the next couple weeks to start making example Manifest V3 extensions; I'll be sure to post here when I get some samples merged into Chromium.

> So if we setup a chrome.alarm to wake regularly

No, chrome.alarms are throttled to one wake-up per minute so the worker will be dead by then.

The flow Erik described should work. The expected lifetimes are 30 seconds of continuous execution (e.g. a while(true) loop) or five minutes of asynchronous work. If you're observing behaviors other than this please open a bug. 

Cheers,

Simeon - @dotproto

Developer Advocate for Chrome Extensions

[Michael Cann]

Some great discussions here guys and although they are getting a little heated in parts I can sympathise with both w0xx0m and Simeon.

At the risk of muddying this discussion I just wanted to throw my two cents in.

I have been developing chrome extensions for almost a decade and have developed over a dozen extensions, even to the point that our business is currently dedicated to building a chrome extension game. 

I too have been frustrated by the lack of attention from google directed towards the extension platform over the reccent number of years but I am glad that we at least have one form of communication in the form of Simeon to advocate on our behalf. 

Im not going to lie, Manifest V3 will be a major pain in the butt for me and my team. BattleTabs relies on a persistant background page to setup a websocket socket connection to the server which we use to push and pull data. Not having a background page and that single websocket connection is really going to cause us headaches and could fundementally change the way our game works. 

I do however understand Google's rationale for doing this. I can imagine that if every extension has a persistant background page and they are all consuming memory any resources then it could rapidly degrade the browser experience. If performance is one of the major driving factors for the change then im wondering if some sort of "memory limits" or other heuristics could be applied to find "bad actor" extensions and kill them if they are behaving badly?

[wOxxOm]

30 seconds now, per the specification, so apparently I measured incorrectly the first time as I did it by counting in my mind. Guess I'm just that slow...

[ابو الحكم حكمي‎]

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-extensions/0add322a-9704-4fba-bd44-c75b96f91c0fn%40chromium.org.

[Any Name]

HI.

Have you already found a solution for this problem?

Thanks!

On Wednesday, September 9, 2020 at 12:44:08 PM UTC+3 ErikRothoff wrote:

[Taylor Dawson]

@Michael Cann 

How did you end up solving this issue?

[Robbi]

I have something to say too,

nothing that I haven't said in my previous post in other threads.

@Simeon Vincent says: > "Chrome will terminate a service worker after it has been idle for 30 seconds".

Now I would ask @Simeon Vincent: "Please can you explain what you mean by "has been idle" after having a look first to these 2 already confirmed bugs?"

Issue 1371876: The extension service worker internal timer is not restarted by any event (of any type)

Issue 1305369: MV3 extension is shutdown while waiting on fetch request after a very short amount of time

In my opinion these two bugs (the first would be enough since it also includes the second one) should be in the "hall of fames" of known-issues page

It seems to me, however, that you want to turn a deaf ear.

[Robbi]

The car is starting up (it would seem)

I've seen some change in docs: #sw-fixed-lifetime