XUL overlays vs React modifications

[Emiliano Heyns]

Continuing the discussion from https://github.com/zotero/zotero/pull/1636 here -- BBT currently modifies the Zotero UI at a number of spots which I will have to carry over where possible to the brave new React world in future Zoteros. I currently make these changes:

chrome://zotero/content/exportOptions.xul: I modify a checkbox label added by one of the displayOptions from the translator so it can be localized, and I add behavior (onclick stuff) to make sure "Keep updated" cannot be selected when "Export attachments" is.

chrome://zotero/content/preferences/preferences.xul to add BBTs preferences. I realize this is uncommon and that other extensions provide their own prefs screen. I'm not super-committed to keeping them there although my personal preference is for preferences to be centralized in some way, either by adding them to the Zotero prefs or by having some kind of common infra for extensions preferences to make the user experience more uniform.

chrome://zotero/content/itemPane.xul: to patch in the citekey display field. This used the modify the XBL in Zotero 4 but now just adds it to the pane and actively (through onload) moves it to the appropriate spot. Also monkey-patches ZoteroItemPane.viewItem so that I can update the citekey field.

chrome://zotero/content/zoteroPane.xul: adds menu items to menu_ToolsPopup, zotero-collectionmenu, zotero-itemmenu, and an extra column to zotero-items-columns-header. Monkey-patches buildCollectionContextMenu on the current pane to enable/disable the menu items added in the xul overlay and to add context-dependent behavior (depending on what collection was active), serializePersist to make sure the BBT data is not persisted when BBT was just uninstalled (it would really screw up the layout if I left it there)

I also use the overlay on zoteroPane to kickstart BBT.

[Adomas Venčkauskas]

I'll keep this thread bookmarked for when we have more details about it (or possibly when something changes due to xul element reactification in Zotero). One thing to note is that we will be rewriting our preferences completely with HTML in the XUL version of Zotero, so that will certainly need attention in some not too distant future.

[Emiliano Heyns]

That and the collections view, right? I also overlay stuff in the collections view to add menu items.

Is there a more general plan for a plugin system for ui and behavior? Firefox came with one and that simplified things. TBH I'm a little scared of the job ahead of porting BBT to the electron/react platform. The switch from Z4 to Z5 took me about 6 months, and while the asyncification bits will likely carry over, and it will be great to get rid of webpack, moving away from XUL looks to me like it'll be more disruptive than the Z4 - Z5 migration. It'd be nice to be able to mentally prepare for the switchover, and read up on the prospective plugin system.

[Adomas Venčkauskas]

On Friday, 17 May 2019 12:47:41 UTC+3, Emiliano Heyns wrote:

That and the collections view, right? I also overlay stuff in the collections view to add menu items.

The menu system isn't changing for the reactified collection view, so no or very minor changes will be required. 

Is there a more general plan for a plugin system for ui and behavior? Firefox came with one and that simplified things. TBH I'm a little scared of the job ahead of porting BBT to the electron/react platform. The switch from Z4 to Z5 took me about 6 months, and while the asyncification bits will likely carry over, and it will be great to get rid of webpack, moving away from XUL looks to me like it'll be more disruptive than the Z4 - Z5 migration. It'd be nice to be able to mentally prepare for the switchover, and read up on the prospective plugin system.

We want to make sure we support the Zotero plugins with as little modifications in React as possible. We will be including a plugin frameworkm. All of the main Zotero class interfaces will remain unchanged, we are also shimming a non-trivial part of XUL native modules and keeping their APIs as close as possible to the XUL ones.

Hopefully the changes to make the plugin run in the React world will be limited to changing how the plugins are packaged, some of the header/config files and minor adjustments to where the plugins attach to Zotero logic/UI.

[Emiliano Heyns]

WRT packaging -- are you guys thinking of npm-hosted modules? Aside the plugin format, the FF base also included a facility for installing extensions and keeping them updated of course.

[Dan Stillman]

On 5/18/19 4:20 AM, Emiliano Heyns wrote:
> WRT packaging -- are you guys thinking of npm-hosted modules? Aside
> the plugin format, the FF base also included a facility for installing
> extensions and keeping them updated of course.

We haven't decided, but you'll certainly be able to install plugins
through the UI, and they'll auto-update.

[Diego de la Hera]

Hi all! I'm starting to develop a Wikicite plugin for Zotero and I'm trying to make sure I make the best decisions possible so it can easily be adapted as Zotero continues moving away from Mozilla's XUL/XPCOM to React/Electron. I'm following this and other threads here, as well as some Github issues and pull requests regarding this process (endeavour!).

I share Emiliano's concern that it would be useful to have some guidelines for extension developers, although I'm happy and relieved to know that it is your idea to try and make sure that Zotero plugins will be supported with as little modifications as possible.

As recommended here, I'm using Will Shanks' Zutilo as an example of a bootstrap plugin. But I understand that it was a XUL-overlay plugin originally, so it still uses XUL where it could use HTML/React instead (for example, in the preferences window). Is there another plugin you can recommend that is also ready for future Zotero development paths?

In the plugin I'm developing I plan to have a separate preferences window that uses HTML/React instead of changing the Zotero preferences window (which still uses XUL). On the other hand, I wanted to add an extra tab to the item pane, but it still uses XUL, and I understand it will be reactified soon (BTW, have you started working on this? Is there somewhere already where I can follow this development?). Would you suggest using a separate HTML/React popup instead?

Regarding the item tree, I don't plan to change it with the plugin. I understand its reactification is being discussed in pull request 1827. Is there an estimate of when it will be merged?

Finally, the developer of Zutilo has suggested some time ago to "keep your [plugin] core code cleanly separated from anything specific to Firefox" and to "think carefully about how you use the Zotero and Firefox api's, possibly putting those into wrapper functions so it is easy to change them when Zotero moves to Electron". However, Dan said the idea is to "shim Mozilla XPCOM interfaces (e.g., nsIFile) and/or Zotero functions (e.g., Zotero.File.getContentsAsync()) so that we can avoid rewrites". Then, would you recommend using these wrapper functions as Will suggested? If yes, do you have examples of how to do this?

Thank you very much for reading this far, and for all the effort you are constantly putting into this amazing project!

Best,

Diego

[Dan Stillman]

On 11/16/20 5:46 PM, Diego de la Hera wrote:

As recommended here, I'm using Will Shanks' Zutilo as an example of a bootstrap plugin. But I understand that it was a XUL-overlay plugin originally, so it still uses XUL where it could use HTML/React instead (for example, in the preferences window). Is there another plugin you can recommend that is also ready for future Zotero development paths?

I don't know of any relevant plugins off-hand — but HTMLification/Reactification is still a work in progress, so we haven't really given much guidance on this. The most important thing is to avoid XUL overlays, since those will be removed as soon as we move to Firefox 78 or later (which we've made some progress on recently). XUL should certainly be avoided if possible, but some XUL elements are replaced by equivalent Custom Elements in Firefox 78, so not everything would be a problem.

In the plugin I'm developing I plan to have a separate preferences window that uses HTML/React instead of changing the Zotero preferences window (which still uses XUL). On the other hand, I wanted to add an extra tab to the item pane, but it still uses XUL, and I understand it will be reactified soon (BTW, have you started working on this? Is there somewhere already where I can follow this development?). Would you suggest using a separate HTML/React popup instead?

Using a separate HTML window for your prefs is a good idea. If you want to add a new item pane tab now, it'd have to be XUL, so no point fretting about that — most of this stuff isn't hard to convert later if necessary. (On the item pane tab, the biggest thing I'd say is that, if every plugin did that, people with multiple plugins would quickly run out of space. We've considered adding a Connections (or similar) tab that plugins could use for this sort of thing.)

Regarding the item tree, I don't plan to change it with the plugin. I understand its reactification is being discussed in pull request 1827. Is there an estimate of when it will be merged?

Pretty soon.

Finally, the developer of Zutilo has suggested some time ago to "keep your [plugin] core code cleanly separated from anything specific to Firefox" and to "think carefully about how you use the Zotero and Firefox api's, possibly putting those into wrapper functions so it is easy to change them when Zotero moves to Electron". However, Dan said the idea is to "shim Mozilla XPCOM interfaces (e.g., nsIFile) and/or Zotero functions (e.g., Zotero.File.getContentsAsync()) so that we can avoid rewrites". Then, would you recommend using these wrapper functions as Will suggested? If yes, do you have examples of how to do this?

Well the Zotero functions are wrapper functions themselves, so, e.g., you should definitely use Zotero.File stuff if possible instead of using nsIFile directly. For stuff Zotero doesn't provide wrappers for, prefer Mozilla's JS-style stuff over XPCOM — OS.File over nsIFile, Services.* stuff for getting services over getService(), etc. — since that stuff is more likely to remain compatible with later Firefox versions.

[Emiliano Heyns]

On Tuesday, November 17, 2020 at 1:26:16 AM UTC+1 Dan Stillman wrote:

On 11/16/20 5:46 PM, Diego de la Hera wrote:

As recommended here, I'm using Will Shanks' Zutilo as an example of a bootstrap plugin. But I understand that it was a XUL-overlay plugin originally, so it still uses XUL where it could use HTML/React instead (for example, in the preferences window). Is there another plugin you can recommend that is also ready for future Zotero development paths?

I don't know of any relevant plugins off-hand — but HTMLification/Reactification is still a work in progress, so we haven't really given much guidance on this. The most important thing is to avoid XUL overlays, since those will be removed as soon as we move to Firefox 78 or later (which we've made some progress on recently). XUL should certainly be avoided if possible, but some XUL elements are replaced by equivalent Custom Elements in Firefox 78, so not everything would be a problem.

See this worries me. BBT uses XUL among other things to hook into menus and the item pane. I still have no idea how that would work when Reactified. I was very slow to react to the Z4 -> Z5 change, and between too many non-starter workarounds while I had my head in the sand on asyncification, that took me 6 months to complete, and I don't have as much time available as I did then.

[adoma...@gmail.com]

I think for as long as we stay with the Mozilla platform we will be using their native menu system, since HTML/React based menus are clunky in many ways. There are no changes planned for that on our side and should not affect plugins either. Dan might be able to say more about the Item Pane, although I'm not sure we'll be able to tell much until we actually have the finished HTML version and see what are the problems and potential solutions to them when extending via a plugin.

[Emiliano Heyns]

I also use it to modify right-click popup menus, and the middle pane that lists the items

[adoma...@gmail.com]

On Tuesday, 17 November 2020 at 09:56:57 UTC+2 Emiliano Heyns wrote:

I also use it to modify right-click popup menus, and the middle pane that lists the items

All menus whether the ones in the menu bar or right-click ones are not changing. The item tree will be changing, but that's a separate can of worms from the context menus and we'll address that once we get there. I assume we will need to perform some modifications to our code to ensure plugins are able to properly extend it once we begin merging that. You can also try the PR yourself, and we can start talking about the specifics of what is needed for BBT and how we can accommodate that. Might be better suited for a different thread though.

[Emiliano Heyns]

Are there builds that incorporate this?

[adoma...@gmail.com]

On Tuesday, 17 November 2020 at 10:30:14 UTC+2 Emiliano Heyns wrote:

Are there builds that incorporate this?

Not yet since this isn't merged to master.  

[Diego de la Hera]

Thank you very much for your detailed reply!