Avoiding UI blocking and using web workers

[David Hoff-Vanoni]

I have a plugin that performs updates on items in a loop which is started from a Zotero.Notifier event. I've noticed that, while my plugin is running through this loop, it appears to lock up the UI. I assume this happens because this is all running on the main JavaScript thread. Is that correct?

I'm considering moving this code to a web worker, and I had some questions around this.

• First, is using a worker a valid approach for this scenario?
• Do I need to do anything special to access the `Zotero` object from a worker script?
• Would it be problematic to update items (using `saveTx`) from a worker?
• @Emiliano, I see you use `ChromeWorker` in BBT. Is there a reason we need to use `ChromeWorker` instead of the standard `Worker`?

I'd appreciate any guidance on this. Thanks!

[Abe Jellinek]

The UI lock-up is probably because you’re running the loop on the main thread, yeah, but unfortunately you probably can’t access the Zotero object (or save changes to items, or anything like that) from a worker. It just doesn’t work that way - the Zotero object lives on the main thread and isn’t designed to be used from workers or made available to workers. Similar situation to the DOM not being available in workers.

What you can do is run calculations from your worker thread, then use async messaging to send the results back to the main thread, which can modify and save the items. That won’t be especially helpful if your big bottleneck is the item.save() call itself, though.

Workers aside, make sure you’re batching your item saves into as few transactions as possible - in other words, don’t use saveTx().

On Jan 30, 2024, at 1:10 AM, David Hoff-Vanoni <da...@vanoni.dev> wrote:

I have a plugin that performs updates on items in a loop which is started from a Zotero.Notifier event. I've noticed that, while my plugin is running through this loop, it appears to lock up the UI. I assume this happens because this is all running on the main JavaScript thread. Is that correct?

--
You received this message because you are subscribed to the Google Groups "zotero-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zotero-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/bfc1f67f-af0d-477f-9d45-d2a1c623f719n%40googlegroups.com.

[volatile static]

We cannot access `Zotero` in WebWorker directly. However, we can define a proxy object in Worker, which communicates to the main thread via HTTP requests.
I've realized an EndPoint to do so.

[Abe Jellinek]

It’s a bad idea to add an HTTP endpoint that blindly evals any code that’s sent to it. Especially since the endpoint evals with full chrome privileges, has CORS disabled (Access-Control-Allow-Origin: *), and lacks even a basic level of authentication. I see that you’re only doing this when your plugin is running in dev mode, but it absolutely isn’t something that should be recommended to others. Any plugin adding an endpoint like this is inherently compromising the security of the machines that it runs on.

As Emiliano said, you can use ChromeWorkers and postMessage. Define a simple message interface on the main thread side (e.g. ‘updateItem’ takes an item ID and an object containing modified fields, makes the modifications, and saves the item) and call it from your worker.

--
You received this message because you are subscribed to the Google Groups "zotero-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zotero-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/36df6b00-1f18-419c-bd50-1de2b68864d6n%40googlegroups.com.

[Emiliano Heyns]

Wrt chromeworkers vs we workers, chromeworkers have the same system access as the main thread. Webworkers are very tightly sandboxed. 

You received this message because you are subscribed to a topic in the Google Groups "zotero-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zotero-dev/idY0PmyUrk4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zotero-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/D8472827-1AEF-4CC9-90AA-044CE752CDB3%40berkeley.edu.

[volatile static]

Thank you for the heads up, Abe! I have strictly applied that EndPoint in my testing environment. Excuse me if it caused any misguided by appearing to advise others. 

[David Hoff-Vanoni]

Thanks for the input, everyone!

Upon further investigation, I've narrowed down the cause of the UI blocking to calls to `Zotero.QuickCopy.getContentFromItems`. For each item, I call this function up to two times (depending on the user's configuration)—once to generate an item citation and again to generate a bibliography entry. Each call to the function causes a noticeable interaction delay.

Unrelated to my plugin, I now realize this is the same cause of the delay I experience when dragging an item in Zotero. Interestingly, I only notice the delay if the selected Quick Copy Format is a "citation style." If I instead choose an "export format," the interaction delay goes away. (See example in this screen recording.) Looking at the Quick Copy code, I imagine this is related to export formats using a callback whereas citation styles appear to run synchronously.

Based on what I've heard from y'all, this sounds like something I wouldn't be able to directly offload to a worker since it requires the Zotero object. I'm not very familiar with the citation system. Is there any way I can more efficiently obtain citations and bibliography entries for items? Can I somehow use the CSL engine from a worker?

Any suggestions are greatly appreciated!

[Emiliano Heyns]

It technically can be offloaded to a worker, since zotwro just calls citeproc to get that output, and citeproc can be ran in a worker. Will take a bit of setting up though. 

You received this message because you are subscribed to a topic in the Google Groups "zotero-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zotero-dev/idY0PmyUrk4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zotero-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/cd455458-44b8-4d8c-87e9-6bd6ac03cdd3n%40googlegroups.com.

[Emiliano Heyns]

But if all you want is to give the UI a breather while you do this, you can just call `await Zotero.Promise.delay(1)` between calls to `quickCopy`.

[David Hoff-Vanoni]

Thanks for the suggestions, Emiliano!

I don't think adding a delay is really what I'm looking for. I essentially already have that because my calls to generate citations alternate with calls to  await fetch() (among other things), and that seems to be providing a UI breather.

I'm curious about running citeproc in a worker. Do you know of any good resources/references for how to do that, or is the Zotero codebase the best place to look?

To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/99d8acaf-6578-4e01-bfd8-dd024a3a5c0an%40googlegroups.com.

[Emiliano Heyns]

There's not much to it -- citeproc-js is pure javascript and uses dependency injection for interaction with the environment, so it will just run in a worker as long as you prep the data to its requirements.

Personally I always opt for bundling with esbuild, in which case you can just use https://www.npmjs.com/package/citeproc. I bundle to an IIFE, in which case you need a trick like https://github.com/retorquere/zotero-better-bibtex/blob/master/esbuild.js#L178 to expose stuff to the global scope, but esbuild is so ridiculously fast it's unnoticeable I'm bundling twice.

[Emiliano Heyns]

This is a scaffold you could use: https://gist.github.com/retorquere/b765166b9b280c08a25b13e94d3406ce

you will have to retrieve the style and the locale either by `fetch`ing them from a private endpoint or by postMessage'ing them into the worker.

[volatile static]

Another example for setting up a Worker. It's simple enough, I think.