Parsing xul

[Emiliano Heyns]

Can someone tell me what I'm doing wrong here? If I remove the xmlns= attribute it parses, but I see no reason why that would make a difference, and I need the namespace.

const xml = `
<overlay id="zotero-better-bibtex-preferences" xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">
</overlay>`

Components.utils.import('resource://gre/modules/Services.jsm')
const parser = Components.classes['@mozilla.org/xmlextras/domparser;1'].createInstance(Components.interfaces.nsIDOMParser)
const xul = parser.parseFromString(xml, 'text/xml')
xul.documentElement.textContent

[Abe Jellinek]

Where are you adding this code? Make sure you construct and use the DOMParser from a system context: XPCOM code, a script loaded from an overlay, something like that. DOMParser will refuse to construct XUL documents when invoked from web content, eval()’d scripts, and elsewhere.

(I’m assuming this is in Zotero 6. Different answer in Zotero 7.)

--
You received this message because you are subscribed to the Google Groups "zotero-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zotero-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/d86a4986-6912-4a3b-bd18-8be2dc69ba01n%40googlegroups.com.

[Emiliano Heyns]

In a xpcom plugin (BBT to be precise, loaded in an overlay) and in the js console under the developer options.

Instantiating the parser isn't a problem, when I remove the namespace declaration it parses just fine.

You received this message because you are subscribed to a topic in the Google Groups "zotero-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zotero-dev/x7piNncQkkU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zotero-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/54273347-AAB3-4A2D-9038-FFDCA97BC524%40berkeley.edu.

[Abe Jellinek]

Works fine for me from the console in Z6 if I replace the Components.classes[…].newInstance() stuff with just plain new DOMParser() (which I didn’t think worked in Z6, but go figure):

const xml = `

<overlay id="zotero-better-bibtex-preferences" xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">

</overlay>`

const parser = new DOMParser();

const xul = parser.parseFromString(xml, 'text/xml’);

xul.documentElement.outerHTML

newInstance() replaces the system principal (which allows XUL document construction) with a null principal (which doesn’t). Not sure why.

In Z7, it’ll become:

// ...

const parser = new DOMParser();

parser.forceEnableXULXBL();

const xul = parser.parseFromSafeString(xml, 'text/xml’);

xul.documentElement.outerHTML

To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/CAHOSGO4ojrYzOmuRsNuX2_rQVvFFxJietJ4cSJ0q1aXHHTbGyA%40mail.gmail.com.

[Emiliano Heyns]

That gets me closer. I'm trying to dynamically load what was previously an overlay using

  // boring bits

  const prefwindow = win.document.querySelector('prefwindow#zotero-prefs')
  if (!prefwindow) return log.error('prefs.start: prefwindow not found')

  let xml = Zotero.File.getContentsFromURL('chrome://zotero-better-bibtex/content/Preferences.xul')
  const url = xml.match(/<!DOCTYPE window SYSTEM "([^"]+)">/)[1]
  const dtd: Record<string, string> = dtdparser.parse(Zotero.File.getContentsFromURL(url))
  for (const [key, value] of Object.entries(dtd)) {
    xml = xml.replace(new RegExp(`&${key};`, 'g'), escapeHtml(value))
  }

  // interesting bit starts here

  const parser = new DOMParser

  const xul = parser.parseFromString(xml, 'text/xml')

  const prefpane = xul.querySelector('prefpane')
  const id: string = prefpane.getAttribute('id') + '2' // eslint-disable-line @typescript-eslint/restrict-plus-operands
  log.debug('prefs id=', id)
  if (win.document.querySelector(`prefpane#${id}`)) return // already loaded

  log.debug('prefpane: strip load')
  prefpane.removeAttribute('load')
  log.debug('prefpane: append')
  prefwindow.appendChild(prefpane) // nope
  log.debug('prefpane: appended')

The top isn't really interesting, it just makes sure I have valid XML content. Everything works fine until the line commented with "nope", that gets me

JavaScript error: chrome://global/content/bindings/tabbox.xml, line 250: TypeError: children[i].getAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/tabbox.xml, line 250: TypeError: children[i].getAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 236: TypeError: val.setAttribute is not a function
JavaScript error: chrome://global/content/bindings/tabbox.xml, line 250: TypeError: children[i].getAttribute is not a function
JavaScript error: chrome://global/content/bindings/menulist.xml, line 229: TypeError: oldval.removeAttribute is not a function

but I can't find those bits of code anywhere in BBT or in Zotero.

[Abe Jellinek]

Something that isn’t an Element is being appended as a child of a tabbox.

Try using the devtools DOM inspector or just printing out win.document.documentElement.outerHTML after the “nope” line. (I’m assuming that line itself succeeds but causes errors in listeners elsewhere, so code after it should be able to run.) Should be able to see if something wrong has been appended to the tabbox.

If the DOM looks fine, maybe (dubiously) it’s an ownership problem. Try Document.importNode() - something like const prefpane = win.document.importNode(xul.querySelector('prefpane'), true); instead of what you’re doing now.

To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/71a2c7e0-301c-44c8-a122-ae5004980acdn%40googlegroups.com.

[Emiliano Heyns]

On Wednesday, November 9, 2022 at 9:17:44 PM UTC+1 Abe Jellinek wrote:

Something that isn’t an Element is being appended as a child of a tabbox.

How is this even possible? I'm loading a working XUL overlay through DOMParser -- how can a DOMParser return things that aren't Elements?

 

Try using the devtools DOM inspector or just printing out win.document.documentElement.outerHTML after the “nope” line.

https://gist.github.com/c47c1311117a606450569ce4374d2d1a

I don't see anything strange, and xmllint doesn't complain

(I’m assuming that line itself succeeds but causes errors in listeners elsewhere, so code after it should be able to run.)

Yep, it does, but the pane doesn't show.

 

Should be able to see if something wrong has been appended to the tabbox.

If the DOM looks fine, maybe (dubiously) it’s an ownership problem. Try Document.importNode() - something like const prefpane = win.document.importNode(xul.querySelector('prefpane'), true); instead of what you’re doing now.

If I do that the loaded XUL doesn't appear in the outerHTML (I also don't see the error messages)

[Emiliano Heyns]

My mistake, I had an error. With importNode it does show in the outerHTML, and the same error messages appear. But the extra pane doesn't load.

[Emiliano Heyns]

Is the only way to add the UI with createElement calls? It's a fair bit of UI I'm adding. I'd prefer to keep that as XUL.

[Emiliano Heyns]

Plain DOMParser seems to reliably error out in Win32 Zotero.

[Emiliano Heyns]

This is sort of a showstopper for me in my transition to bootstrapped. I need to be able to add my preferences to the prefs pane.

[Dan Stillman]

It's not clear to me what you're trying here. Can you post a full example of what you're doing?

In Zotero 6, you have to use `Components.interfaces.nsIDOMParser` in XPCOM code. `new DOMParser()` should work in window code. In Zotero 7, you can use `new DOMParser()` anywhere (but there's also a new built-in hook to add preference panes).

[Dan Stillman]

The specifics of this aside, though, one thing you might consider is creating a make-it-red 1.1–style plugin [1] that is simultaneously an overlay plugin for Zotero 6 and a bootstrapped plugin for Zotero 7. Then you can keep your overlays for now and do things programmatically for Z7, where, for example, we have a hook for adding a preference pane).

[1] https://github.com/zotero/make-it-red

--
You received this message because you are subscribed to the Google Groups "zotero-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zotero-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/01000184b3863930-e9f631d4-32f0-49ea-abeb-9148e5697767-000000%40email.amazonses.com.

[Emiliano Heyns]

Step by step then: this fails to parse unless I change the `xmlns:xul` to anything but `http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul"`. Ran in the "Run javascript" console.

const xul = `<?xml version="1.0"?>
<?xml-stylesheet href="chrome://global/skin/" type="text/css"?>
<?xml-stylesheet href="chrome://global/skin/global.css"?>
<?xml-stylesheet href="chrome://zotero-platform/content/preferences.css"?>
<?xml-stylesheet href="chrome://browser/skin/preferences/preferences.css"?>
<?xml-stylesheet href="chrome://zotero/skin/preferences.css"?>
<?xml-stylesheet href="chrome://zotero/skin/zotero.css"?>
<?xml-stylesheet href="chrome://zotero-platform/content/overlay.css"?>
<?xml-stylesheet href="chrome://zotero-better-bibtex/skin/preferences.css"?>
<?xml-stylesheet href="chrome://zotero-better-bibtex/skin/error-report.css" type="text/css"?>
<!DOCTYPE window SYSTEM "chrome://zotero-better-bibtex/locale/zotero-better-bibtex.dtd">
<xul:overlay id="zotero-better-bibtex-preferences" xmlns:xul="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul" xmlns:html="http://www.w3.org/1999/xhtml" xmlns:bbt="http://retorque.re/zotero-better-bibtex/">
</xul:overlay>
`

const parser = Components.classes['@mozilla.org/xmlextras/domparser;1'].createInstance(Components.interfaces.nsIDOMParser)

parser.parseFromString(xul, 'application/xml').documentElement.outerHTML

[Dan Stillman]

I wouldn't have any idea how DOMParser handles a XUL overlay. That's not something I would particularly expect to work, and I'm not totally sure why you're trying this.

A bootstrapped extension generally creates UI programmatically or loads a document in a custom window. That would generally be what one should expect to do when converting to a bootstrapped extension.

If you have an existing overlay for something that you know will take markup in Zotero 7 — which is the case for preferences — then, as I say, you're probably better off just loading the extension as an overlay for Zotero 6. You can pass markup to the new prefpane hook for Zotero 7.

Maybe there's a way to do what you're trying to do, but I don't think that's something we'll be able to help with.

[Emiliano Heyns]

OK, I've taken another approach, and I'm now executing

  const prefwindow = win.document.querySelector('prefwindow#zotero-prefs')

  const dtd = {

    "better-bibtex.Preferences.prefpane.better-bibtex": "Better BibTeX",

    ...

  }

  makeUI(win.document, prefwindow, dtd)

where makeUI is https://gist.github.com/4eb1411a46094fc34f60017b50f13989

This runs without error, but no UI is shown.

[Emiliano Heyns]

The resulting DOM looks like this: https://gist.github.com/5cc32b3b1fb71ef9621676c2b1705c5d . Looking at that, I would have expected to see the BBT prefs pane twice.

No idea what I'm doing wrong here. Is there anything about the prefs pane that would make that I can't construct a UI that way? That would be a blocking issue for BBT to go restartless.

[Emiliano Heyns]

I really want to make a start at moving BBT to restartless, but not being able to create the prefs UI is a real issue for me. Is there something else I could try?

[Dan Stillman]

On 12/8/22 4:58 AM, Emiliano Heyns wrote:
> I really want to make a start at moving BBT to restartless, but not
> being able to create the prefs UI is a real issue for me. Is there
> something else I could try?

I answered that — you can keep the extension as an overlay extension for
Zotero 6 and use the new pref hooks in Zotero 7.

[Emiliano Heyns]

So there's no sensible way to move an extension to bootstrapped for Zotero 6? I struggled immensely with parallel development when I had to transition to async, so this time I'd much prefer to make as much as possible Z7-friendly (if not downright compatible) so the final push for Z7 compat will be managable, working from a single code base throughout.

[Dan Stillman]

On 12/8/22 2:50 PM, Emiliano Heyns wrote:
> So there's no sensible way to move an extension to bootstrapped for
> Zotero 6? I struggled immensely with parallel development when I had
> to transition to async, so this time I'd much prefer to make as much
> as possible Z7-friendly (if not downright compatible) so the final
> push for Z7 compat will be managable, working from a single code base
> throughout.

I'm not sure why you think that. We provided an example of a plugin that
works 1) as a hybrid plugin loading mostly the same code or 2) as a
bootstrap-only plugin compatible with both versions. You're asking about
one specific integration point that most other plugins don't use and
that is replaced by an official hook in Zotero 7. I haven't looked into
your examples, and there's almost certainly a way to make it work as
bootstrapped, but if you're having trouble with it I'm pointing out that
you can always use the hybrid model and use an overlay in a few places
while using the same code for most of the plugin.

[Emiliano Heyns]

Ah, I see. I'd prefer to go all-bootstrapped, but that's apparently not possible for overlaid prefs, but I could use the hybrid, src-1.1 jumping-off point to remove all overlays except the prefs, and then when Z7 is released, move to bootstrapped. Correct?

I had planned to move to the src-1.2 model, but I can't go without the prefs. Can I just move my prefs to a separate xul file with its own prefwindow, and load that using window-watcher openWindow? That is possible from a bootstrapped extension, right? That would allow me to go straight to bootstrapped. I'm looking to declutter the BBT startup process, and I worry that the hybrid model would complicate the startup rather than simplifying it.

How does the hybrid plugin signal to Z7 it wants to be loaded as bootstrapped? I had expected to find that in the install.rdf

[Dan Stillman]

On 12/8/22 3:49 PM, Emiliano Heyns wrote:
> Ah, I see. I'd prefer to go all-bootstrapped, but that's apparently
> not possible for overlaid prefs, but I could use the hybrid, src-1.1
> jumping-off point to remove all overlays except the prefs, and then
> when Z7 is released, move to bootstrapped. Correct?

Well, again, I suspect there's a way to make the prefs work, but yes,
that hybrid approach is what I'm suggesting.

> I had planned to move to the src-1.2 model, but I can't go without the
> prefs. Can I just move my prefs to a separate xul file with its own
> prefwindow, and load that using window-watcher openWindow? That is
> possible from a bootstrapped extension, right?

Sure — you'd add it as an option in the Tools menu like various other
plugins. And for Z7 (where prefwindow doesn't exist) you could use to
the new hook.

> How does the hybrid plugin signal to Z7 it wants to be loaded as
> bootstrapped? I had expected to find that in the install.rdf

To Z7? Z7 only supports bootstrapped. Zotero 6 uses install.rdf if it's
available rather than loading bootstrap.js.

[XY Wong]

I write a prefpane register for Zotero 6, using the exact same parameters as the Zotero 7's Zotero.PreferencePanes.register.

Developers only need to maintain one prefs.xhtml under Zotero 7's requirements and it runs exactly the same on Zotero 6.

See zotero-addon-template/README.md at bootstrap · windingwind/zotero-addon-template (github.com)

[Emiliano Heyns]

It says "Note that <preferences> element is deprecated. Please use the full pref-key in the elements' preference attribute.". Does that mean no more hidden preferences?

[Abe Jellinek]

No. <preference> elements gave local IDs to preference keys. They were dubiously useful because each preference was usually only used once per prefpane, and their IDs ended up being shortened versions of the preference keys anyway. They had no bearing on support for hidden preferences.

(I’ll document that on the Z7 for Developers wiki page.)

On Dec 30, 2022, at 8:51 AM, Emiliano Heyns <emilian...@iris-advies.com> wrote:

It says "Note that <preferences> element is deprecated. Please use the full pref-key in the elements' preference attribute.". Does that mean no more hidden preferences?

--

You received this message because you are subscribed to the Google Groups "zotero-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zotero-dev+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/3aa34662-3ea8-46c0-83e3-45d5734fbc00n%40googlegroups.com.

[Abe Jellinek]

See https://www.zotero.org/support/dev/zotero_7_for_developers#preference_tags_and_preference_binding.

[Emiliano Heyns]

Yeah that's what I had meant to say -- this means you can only introduce a preference when it has a visible control in the UI, and the default values are gone. If that's how it is going to be in Z7 I will have to adjust, but I used both features, being able to specify a preference without a visible control, and being able to specify a default value for it. I use the pref pane as a source of documentation for my preferences, so I will need to update that.

[XY Wong]

I use defaults/preferenses/defaults.js to set default values to preference settings. However, these default preferences do not work the first time a bootstrapped plugin is installed. They are not set until Zotero is restarted. Any suggestions?

[Emiliano Heyns]

That's one of the things I used the defaults in the prefs pane for - I parsed those out and use those as the value if the preference returned undefined. But that's not something I would generally recommend to do.

It would be easy enough to read and eval the defaults.js at startup with a user_pref that sets the pref if undefined, we could add that to the lib.

[XY Wong]

That would be great. Maybe:

    Services.scriptloader.loadSubScript(

      `${rootURI}/ defaults/preferenses/defaults.js`,

      {

        pref: (key: string, value: string | boolean | number) => {

          typeof Zotero.Prefs.get(key) === "undefined" &&

            Zotero.Prefs.set(key, value);

        },

      }

    );

[Dan Stillman]

On 12/31/22 12:09 AM, XY Wong wrote:
> I use defaults/preferenses/defaults.js to set default values to
> preference settings. However, these default preferences do not work
> the first time a bootstrapped plugin is installed. They are not set
> until Zotero is restarted. Any suggestions?

We'll fix that.

[XY Wong]

Glad to know! Thank you very much.

BTW, would this fix come to Zotero 6, or only Zotero 7?

[Emiliano Heyns]

How does Zotero infer the preference type in the new markup, now that it's no longer specified in the <preference...> element? And does the new markup already work in Z6?

On Friday, December 30, 2022 at 5:32:55 PM UTC+1 Abe Jellinek wrote:

[Abe Jellinek]

How does Zotero infer the preference type in the new markup, now that it's no longer specified in the <preference...> element?

It doesn't need to. Make sure you’re setting defaults of the correct type for all your preferences and it’ll work fine.

(I see that you’re currently generating defaults files based on your <preference> tags. I would really recommend that you switch to a different approach. That said, it won't harm anything if you keep your <preference> tags in for the time being in order to parse them during the build process without using them at runtime - they're just no-ops now.)

And does the new markup already work in Z6?

No, unfortunately. This will certainly not be the only change you’ll have to make in your panes between Z6 and Z7, though. The switch from pure XUL to HTML-and-XUL and the new appearance will require other parts of your pane markup to diverge. I know that it’s inconvenient to have to switch to new markup, but the old ID-based system was unnecessary, redundant, and made panes more difficult to maintain.

To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/4b65060e-11c6-426d-a594-866aa18b0a6dn%40googlegroups.com.

[Emiliano Heyns]

On Thursday, January 5, 2023 at 4:11:51 AM UTC+1 Abe Jellinek wrote

(I see that you’re currently generating defaults files based on your <preference> tags. I would really recommend that you switch to a different approach. That said, it won't harm anything if you keep your <preference> tags in for the time being in order to parse them during the build process without using them at runtime - they're just no-ops now.)

Are they going to remain no-ops in Z7?

 

If I have to, I will switch, but currently all knowledge and docs of my prefs is centralised in the preference pane. Does defaults.js tolerate comments? I could try switching to jsdoc for documenting the prefs.

No, unfortunately. This will certainly not be the only change you’ll have to make in your panes between Z6 and Z7, though. The switch from pure XUL to HTML-and-XUL and the new appearance will require other parts of your pane markup to diverge. I know that it’s inconvenient to have to switch to new markup, but the old ID-based system was unnecessary, redundant, and made panes more difficult to maintain.

But if I set the IDs to the full preference names, this would work for both Z6 and Z7 right?