LibreOffice Plugin Wire Protocol: Incomplete documentation

[Michael Jungnickl]

Steps:
Start a word processor. 
Load a document with existing citations, or add citations. 
Now you can open the Document Preferences dialog. 
The last setting is a button "Switch to a Different Word Processor". 
If you click to the button and confirm all changes, it exports a convertion document. 
If you now open an other word processor, load the document, you can import it with the Zotero Refresh command. 

I am currently develop the import only. 
While doing this, I found a unknown command, and unknown arguments. 
This happens when "Refresh" is done with the converted document. 

Unknown argument: 

Document_getDocumentData is called twice. 

At first it returns "ZOTERO_TRANSFER_DOCUMENT". 

At second it returns the preferences as usual. 

Unknown command: 

Document_importDocument

In: ["Document_importDocument",[2,"ReferenceMark"]]

This is command, documentID, format 

Out: true or false

This declares if we support the format ReferenceMark ?

Links: 
https://www.zotero.org/support/dev/client_coding/libreoffice_plugin_wire_protocol
https://www.zotero.org/support/kb/moving_documents_between_word_processors 

[Adomas Venčkauskas]

May I ask which word processor are you developing the plugin for?

On Monday, 7 September 2020 11:44:18 UTC+3, Michael Jungnickl wrote:

Unknown argument: 

Document_getDocumentData is called twice. 

At first it returns "ZOTERO_TRANSFER_DOCUMENT". 

At second it returns the preferences as usual. 

Yes, it seems this isn't documented. When a document is converted to the "export" form, the first line of the document contains "ZOTERO_TRANSFER_DOCUMENT". If the first line of the document contains "ZOTERO_TRANSFER_DOCUMENT", then Document_getDocumentData must return that string, i.e. "ZOTERO_TRANSTER_DOCUMENT", which signals to Zotero that a document import must be initiated. The first thing Zotero will then do is call Document_importDocument.

Unknown command: 

Document_importDocument

In: ["Document_importDocument",[2,"ReferenceMark"]]

This is command, documentID, format 

Out: true or false

This declares if we support the format ReferenceMark ?

The parameters for this command are the documentID and a field type to which the import citation links should be converted to. The export document is structured such that all "Zotero relevant text" is set to hyperlinks that point to https://www.zotero.org/*. Document_importDocument has to scan the document for all these links. The document preferences string will be at the end of the document and prefixed with `DOCUMENT_PREFERENCES `, regular Zotero citations will be prefixed with `ITEM CSL_CITATION `, and the bibliography field will be prefixed with `BIBL `. Document_importDocument only needs to convert these pieces of text to their relevant counterparts - doc prefs need to be stored where the doc processor would regularly store them (and remove them from text), and citation/bibl hyperlinks need to be converted to their equivalent of Fields/ReferenceMarks in a given processor. Note that you will need to remove the `DOCUMENT_PREFERENCES ` prefix from the doc prefs string, but do not need to modify the link text at all for citations/bibl. You will also need to remove the "ZOTERO_TRANSFER_DOCUMENT" line at the start of the doc.

Also see:

- Zotero tests doc plugin dummy comments https://github.com/zotero/zotero/blob/e8dd1f7824992608b83bbccd99e257c094fd2d98/test/tests/integrationTest.js#L124-L147

- Import/Export code in LibreOffice https://github.com/zotero/zotero-libreoffice-integration/blob/abac20076ef011737f699656f64f7a5dd1733394/build/source/org/zotero/integration/ooo/comp/Document.java#L156-L234

- Import/Export code in Google Docs https://github.com/zotero/zotero-google-docs-integration/blob/4a0f525a64024e6303f4b666743d1073f90db400/src/apps-script/Code.js#L523-L587

- Import/Export code in Win Word https://github.com/zotero/zotero-word-for-windows-integration/blob/master/build/zoteroWinWordIntegration/document.cpp#L816-L910

- Import/Export code in Mac Word https://github.com/zotero/zotero-word-for-mac-integration/blob/master/build/src/document.m#L938-L1085

Feel free to update the Zotero dev wiki regarding this. Hope it helps!

Adomas

[Michael Jungnickl]

Thank you very much for your reply. 

>  May I ask which word processor are you developing the plugin for? 

It's TextMaker, part of SoftMaker Office. SoftMaker develops it in Nuremberg / Germany since 30 years.  

Links: 
https://www.softmaker.com/en/
https://www.freeoffice.com/en/

***

I already implemented parsing of the url links and replacing it with fields / footnotes or endnotes. 
However it has to be improved now. 

Michael 

[Michael Jungnickl]

Since I finished the conversion source code for TextMaker, I like to do a little bit of the documentation here. 

This could be verified by the Zotero team and updated on the webpage. 
See https://www.zotero.org/support/dev/client_coding/libreoffice_plugin_wire_protocol

Document_importDocument

Parameters: 
documentID
fieldType like "ReferenceMark"

Returns: 
BOOL wether the fieldType is supported 

Document_exportDocument

Parameters: 
Integer documentID
String fieldType like "ReferenceMark"
String heading paragraph text 

Returns:
null 

Document_setBibliographyStyle

Parameters: 
Integer  documentID
Integer  first line indent
Integer  left line indent
Integer  line spacing
Integer  paragraph spacing after the paragraph
Array (Integer)  tabulator indent 
Integer  number of tabulators (0, 1) 

Examples: 

// [1, -720, 720, 240, 0, [], 0]

// [1, -720, 720, 240, 0, [1], 720]

All arguments in 1440 units per inch in LibreOffice 6.3. 

Returns: 
null

Important remarks about how plain / formatted strings are handled: 

Field_setText
Zotero can set the text in RTF format, or in plain text. 
The wordprocessor must create a plain text string from the RTF string. 

Field_getText
Always return the plain text string in this command ! 

[adoma...@gmail.com]

Thanks, I've updated the wiki page with some modifications and clarifications. See the importDocument return parameter, which you didn't interpret quite right.

Also Field_setText parameter is either plaintext or rtf, as specified by the isRich parameter. If it's a RTF string the provided formatting must also be set (i.e. any italics, bolds, superscript, etc). Field_getText should only ever return plaintext which is used to warn users when they modified the Zotero inserted citation. Most doc processors will edit/reinterpret the richtext supplied to them, if it even allows inspecting parts of doc as RTF instead of just plaintext in the first place, so plaintext is used since it's reliably consistent.

Great work on the integration plugin!

Adomas

[adoma...@gmail.com]

So I've just given a shot testing the plugin and it works great, well done! A couple of issues:

1. All Zotero plugins allow to edit field text. This is both a plus and a minus. In some cases users may find it easier to do dirty final edits to citations this way, which is great. The minus is that Zotero will prompt the user about modified citation text if it needs to update a given citation, which presents a rather confusing dialog which allows to either keep the user modifications and stops the citation from being automatically updated by Zotero or discards user edits and updates the citation. This can cause great confusion especially if the edits occur by accident. Nonetheless, we allow edits in all our plugins and for consistency's sake, especially if Zotero users decide to switch to TextMaker from somewhere else you should consider supporting citation editing in-text.

2. You don't seem to correctly apply parameters specified by Zotero via the Document_setBibliographyStyle. E.g. when inserting a Bibliography with APA style on Ubuntu with TextMaker 2021 the bibliography is not indented at all. You may want to see what output LibreOffice, Word and Google Docs produces for various styles. This is actually a major issue and you should prioritize fixing it.

Adomas

[Michael Jungnickl]

Thank you for fixing the wiki page. This makes programming easier, and avoid bugs. 

I read the changes: 
The command Document_setBibliographyStyle  parameter #4 is called  entrySpacing.  
But when I test it with Libre Office it's the space after the paragraph. 

Don't worry about the  Document_setBibliographyStyle  command. 
I finished my work yesterday. It's currently  only available for the testers. 

>  All Zotero plugins allow to edit field text 

At first I implemented a flat model for Zotero. 

Later it was split into three parts:  
- Preparation 
- Zotero messages 
- Applying of changes to the document

The Zotero messages were moved into a thread. 
Waiting for data shouldn't block TextMaker itself. 
All data of the communication is cached within a temporary document. 
This means changed document preferences and changes to fields.  

Finally after the communication with Zotero is finished, it's all applied to the document. 
Then the fields, footnotes, endnotes or the bibliography are created, edited or deleted. 

>  So I've just given a shot testing the plugin and it works great, well done! 

Zotero is integreated into TextMaker. The source code is compiled to it. 
But we use conditions like CAP_ZOTERO in C/C++.
Therefore it's possible  to configure different releases. 

The Zotero application still needs the Libre Office plugin. 

Testing: 

I used Libre Office as sample implementation. 

Furthermore RawCap and Wireshark were useful to debug the communication of Zotero and Libre Office.  
RawCap can log internal data from the IP 127.0.0.1 address. 

[adoma...@gmail.com]

On Tuesday, 13 October 2020 at 14:39:23 UTC+3 Michael Jungnickl wrote:

I read the changes: 
The command Document_setBibliographyStyle  parameter #4 is called  entrySpacing.  
But when I test it with Libre Office it's the space after the paragraph. 

I guess we could be more specific, since each bibliographic entry will be on a separate paragraph, so entrySpacing does coincide with paragraph spacing. 

 

>  All Zotero plugins allow to edit field text 

At first I implemented a flat model for Zotero. 

Later it was split into three parts:  
- Preparation 
- Zotero messages 
- Applying of changes to the document

The Zotero messages were moved into a thread. 
Waiting for data shouldn't block TextMaker itself. 
All data of the communication is cached within a temporary document. 
This means changed document preferences and changes to fields.  

Finally after the communication with Zotero is finished, it's all applied to the document. 
Then the fields, footnotes, endnotes or the bibliography are created, edited or deleted. 

What I meant here was that the citation/field text in other Word processors with Zotero integration are editable by the user. In TextMaker the cursor jumps right over the whole text of the citation, or you can select it in full. You might want to consider making the field text for Zotero citations editable.

Adomas

 

[Michael Jungnickl]

>  What I meant here was that the citation/field text in other Word processors with Zotero integration are editable by the user. In TextMaker the cursor jumps right over the whole text of the citation, or you can select it in full. You might want to consider making the field text for Zotero citations editable. 

TextMaker only supports fields, footnotes and endnotes for Zotero. 

The fields are intentionally not editable. They can only selected / deleted completely. 

Footnotes and endnotes uses fixed numbers.  The text is editable.
The citations are also cached in a table within the document. 
Changing the text doesn't change the cached citations data. 

[Adomas Venčkauskas]

On Fri, 16 Oct 2020 at 11:00, Michael Jungnickl <jung...@softmaker.de> wrote:

>  What I meant here was that the citation/field text in other Word processors with Zotero integration are editable by the user. In TextMaker the cursor jumps right over the whole text of the citation, or you can select it in full. You might want to consider making the field text for Zotero citations editable. 

TextMaker only supports fields, footnotes and endnotes for Zotero. 

The fields are intentionally not editable. They can only selected / deleted completely. 

Footnotes and endnotes uses fixed numbers.  The text is editable.
The citations are also cached in a table within the document. 
Changing the text doesn't change the cached citations data. 

Right, so what I wrote in the previous messages was that field text for Zotero citations is editable in every other editor that supports citing with Zotero and Zotero users have an expectation to be able to edit their citations in-text. You do not need to perform any special handling or update any citation data for this to work correctly, Zotero will handle that.