Storing structured data for items via plugin

[David Hoff-Vanoni]

Hi all,

I'm working on a plugin, and I'd like to store some structured data for each item. I'm not yet sure of the exact structure, but I think it'll be something along the lines of an array of objects with UUID and timestamp properties—nothing too complicated.

I found some discussion around this in this thread from many years ago which mentioned managing a separate database file or storing data in notes. I'd prefer to not have to manage my own database since my data will be relatively simple, and I'd like to take advantage of Zotero's built-in syncing capabilities. It seems like notes may be a decent fit.

It was also mentioned that, "giving plugins a clean way to store data is definitely something we want to support." Has there been any progress in this regard?

Do y'all have any recommendations for the best way to approach this? Are notes the best available option? And if so, is there a recommendation for how best to store structured data in the HTML content of notes?

I'd appreciate any suggestions that anyone has.

Many thanks!

David

[Dan Stillman]

On 7/1/23 9:02 PM, David Hoff-Vanoni wrote:

I'm working on a plugin, and I'd like to store some structured data for each item. I'm not yet sure of the exact structure, but I think it'll be something along the lines of an array of objects with UUID and timestamp properties—nothing too complicated.

I found some discussion around this in this thread from many years ago which mentioned managing a separate database file or storing data in notes. I'd prefer to not have to manage my own database since my data will be relatively simple, and I'd like to take advantage of Zotero's built-in syncing capabilities. It seems like notes may be a decent fit.

It was also mentioned that, "giving plugins a clean way to store data is definitely something we want to support." Has there been any progress in this regard?

We're actually working on this at the moment. Hope to have something in a few weeks.

- Dan

[David Hoff-Vanoni]

That's exciting! Looking forward to hearing about it.

[Emiliano Heyns]

Where will it be announced when it drops?

[David Hoff-Vanoni]

@Dan, are there any updates on this? I'd love to start thinking about how I could improve my plugin with this functionality.

Thanks!

[Emiliano Heyns]

Would this plugin data (optionally I suppose) be part of Zotero sync?

[David Hoff-Vanoni]

Hi Dan, sorry to bug you about this again, but I was curious if a plugin data storage API is still on the roadmap?

It's been great to see all the new plugin APIs y'all are making available in Zotero 7, and I'd love to see data storage become another one. Combined with the recently-released item pane API, I'd be able to significantly improve the UX of my plugin.

Thanks!

[Bjoern Hassler]

Hi David,

just to say that I'd be interested in this feature too. Thanks for raising it again.

Björn

[David Hoff-Vanoni]

Hi Zotero devs,

Now that Zotero 7 is released, I was wondering if this feature might be making its way onto the roadmap. Do y'all have any update?

My plugin currently stores its data in link attachments on items—including embedded notes on those attachments. I just came across PR #4748: Add option to migrate embedded attachment notes to item notes, and I'm worried that a tool to mass migrate embedded notes might blow away my plugin's data.

If this feature isn't coming any time soon, is there a better recommendation for storing item data since embedded attachment notes are being deprecated?

Thanks!

David

[XY Wong]

For now, plugins are supposed to use the extra field to store item-related data. (https://www.zotero.org/support/kb/item_types_and_fields#citing_fields_from_extra)

The embedded attachment note is deprecated, so yes, it's not recommended to use that.

[David Hoff-Vanoni]

Is there a recommended approach for using the Extra field in a way that won't conflict with other plugins or the user's own use of it? I worry that using the Extra field is even more fragile than my current approach.

In the documentation you sent, I see the Extra field can be used for CSL data using the format `CSL Variable: Value` with each variable on a separate line. Would I have a separate line for my plugin and do something like `Plugin Name: JSON string` with all the data I need inside that JSON string?

[Emiliano Heyns]

That's what I do. I have unilaterally claimed

tex.Thing: Value

bibtex.Thing: Value

biblatex.Thing: Value

and also

tex.Thing = Value

bibtex.Thing = Value

biblatex.Thing = Value

which has different semantics for my plugin