preparing for 5.0

[Dan Stillman]

We're aiming to release a developer preview of Zotero 5.0 within the
next couple weeks, to be followed soon after by a public beta.

While we're not totally done moving things around, if you develop a
Zotero extension, now would be an excellent time to begin updating it to
work with 5.0, if you haven't yet. I'm happy to answer any questions you
have about the changes, which are, let's just say, extensive.

If you've been following along with development on the master branch,
you may have seen both Promisification [1] and Deasyncification [2]. The
former has been a long process of switching to asynchronous database and
(to a lesser extent) file access, which involved converting much of the
code base to use promises via ES6 generators. The latter involved
reverting or moving some of those changes as we discovered functionality
that couldn't be made to work asynchronously in the Mozilla framework
(most notably drag-and-drop Quick Copy). The upshot of these two
endeavors is that Zotero now reads most data asynchronously at startup
but keeps many data layer retrieval methods unchanged, while all
operations that require writes to the database return promises.

We took the opportunity to make a lot of other changes, but hopefully in
the service of a much cleaner and more logical code base. 5.0 also has a
massively expanded test suite (600+ tests and counting), which should
greatly improve Zotero's stability and make it easier to development
for. Overall, the changes in 5.0 should provide for a much smoother user
experience and lay the groundwork for some major, long-awaited changes,
such as an improved data model.

Thanks for your patience with these changes, and let me know if you have
any questions.

- Dan

[1] https://github.com/zotero/zotero/issues/518
[2]
https://github.com/zotero/zotero/commit/daf4a8fe4db46ed29de2babed79610a2522821df

[Will S]

Is this the right place to watch to know when the preview and beta are released and where to download them?

[Dan Stillman]

On 3/23/16 7:19 AM, Will S wrote:
> Is this the right place to watch to know when the preview and beta are
> released and where to download them?

Yep — we'll post the preview to this list when it's ready for testing.

For development purposes, you can of course run current 5.0 code at any
time by checking out the master branch and setting up a source
installation in a dev profile:

https://www.zotero.org/support/dev/source_code#working_with_git_repositories

[Ephrim Khong]

For those running their own server: are there any incompatibilties in
the API used between client and server? Essentially, can I continue to
run the rather old version of the server with the new 5.0 client?

Thanks,
- Eph

[Dan Stillman]

On 3/23/16 9:42 AM, Ephrim Khong wrote:
> For those running their own server: are there any incompatibilties in
> the API used between client and server? Essentially, can I continue to
> run the rather old version of the server with the new 5.0 client?

No, the client will only work with the latest server code, so you or
someone else will need to get the new code working in order to run 5.0.

However, one of the goals of the new API syncing architecture is making
it simpler to set up the server. Classic sync is still supported in the
latest server version (for old clients), but if the server is just used
with 5.0 it won't be necessary to set up those parts.

Once classic sync is phased out (which we'd like to do pretty quickly
after 5.0), we may be able to provide a bit more official support for
external installations, which would include getting the code into a more
easily installable state.

[jl]

Thanks for the update! Not sure how I will manage the transition for zotfile considering time constraints but I will see. Any easy way to get a list of methods that are now promise-based or have changed in significant ways? As a rough rule, the data retrieval methods remain unchanged and writes are promise based? Here is a (probably incomplete) list of methods used in zotfile based on a quick regex search.

Best!

Zotero.Attachments.getFileBaseNameFromItem()

Zotero.Attachments.getStorageDirectory()    

Zotero.Attachments.importFromFile()         

Zotero.Attachments.linkFromFile()           

Zotero.Browser.createHiddenBrowser()        

Zotero.Browser.deleteHiddenBrowser()        

Zotero.Collections.get()                    

Zotero.DB.beginTransaction()                

Zotero.DB.commitTransaction()               

Zotero.debug()                              

Zotero.File.createDirectoryIfMissing()      

Zotero.File.getExtension()                  

Zotero.File.getValidFileName()              

Zotero.getStorageDirectory()                

Zotero.getZoteroDirectory()

Zotero.getString()                          

Zotero.HTTP.doGet()

Zotero.Item()

Zotero.Items.get()                          

Zotero.ItemTypes.getLocalizedString()       

Zotero.ItemTypes.getName()                  

Zotero.launchFile()                         

Zotero.logError()                           

Zotero.Notifier.registerObserver()          

Zotero.Notifier.unregisterObserver()        

Zotero.Prefs.get()                          

Zotero.Search()                             

Zotero.Searches.getAll()                    

Zotero.Sync.Storage.setSyncedHash()         

Zotero.Sync.Storage.setSyncState()          

Zotero.Tags.getID()                         

Zotero.Tags.getName()                       

Zotero.Utilities.htmlSpecialChars()         

Zotero.Utilities.levenshtein()

Zotero.Utilities.removeDiacritics()         

Zotero.Utilities.text2html()

Zotero.Utilities.trimInternal()  

ZoteroPane.getSelectedLibraryID()

ZoteroPane.getSelectedCollection()

ZoteroPane.getSelectedItems()

ZoteroPane.getSelectedSavedSearch()

ZoteroPane.onCollectionSelected()

ZoteroPane.collectionsView.selectLibrary()

ZoteroPane.itemsView.selectItems()

ZoteroPane.itemsView.saveSelection()

[jl]

ps: And chance that there is a methods to chance from imported to linked attachment in 5.0 that does not corrupt the database (without just creating a new attachment item)?

On Tuesday, March 22, 2016 at 3:44:39 PM UTC-5, Dan Stillman wrote:

[Dan Stillman]

On 3/23/16 4:19 PM, jl wrote:
> Thanks for the update! Not sure how I will manage the transition for
> zotfile considering time constraints but I will see. Any easy way to
> get a list of methods that are now promise-based or have changed in
> significant ways?

No, sorry, we weren't quite as diligent as we might have been at keeping
track of the changes — the changes were just too extensive — though in
many cases we used different functions or had the old ones throw/warn
when passed outdated parameters. But a simple grep/ack -C 2 for the
function name will be enough to tell quickly in most cases — the ones
that return promises will generally now either be wrapped in
Zotero.Promise.coroutine() or say "@return {Promise}" on the comment
line above.

If we make additional function changes between now and 5.0 Final we'll
be sure to document them in the commits, so keep an eye on the commit log.

> As a rough rule, the data retrieval methods remain unchanged
> and writes are promise based?

Yes, though some of the auxiliary retrieval methods that are based on
SQL queries (e.g., Zotero.Item.prototype.getBestAttachments()) we
haven't yet deasyncified or can't be (e.g.,
Zotero.Search.prototype.search()), and file-based retrieval methods
generally are now promise-based or were replaced with async versions
(e.g., getFile -> getFilePathAsync(); a synchronous getFilePath() exists
but it doesn't do a file existence check, since that's an async op).

Various file methods also now work with regular string paths, following
Mozilla's OS.File approach, instead of nsIFiles.

> Here is a (probably incomplete) list of methods used in zotfile based
> on a quick regex search.

Some quick annotations off the top of my head:

> Zotero.Attachments.getFileBaseNameFromItem()
> Zotero.Attachments.getStorageDirectory()

Unchanged

> Zotero.Attachments.importFromFile()
> Zotero.Attachments.linkFromFile()

Return promises

> Zotero.Browser.createHiddenBrowser()
> Zotero.Browser.deleteHiddenBrowser()

Unchanged

> Zotero.Collections.get()

Unchanged, thanks to deasyncification

> Zotero.DB.beginTransaction()
> Zotero.DB.commitTransaction()

Replaced with promise-based executeTransaction(function* () { ... })

> Zotero.debug()

Unchanged

> Zotero.File.createDirectoryIfMissing()

Unchanged, but should probably return a promise

> Zotero.File.getExtension()
> Zotero.File.getValidFileName()

> Zotero.getStorageDirectory()
> Zotero.getZoteroDirectory()

Still return nsIFiles, but you can add .path to use with OS.Path.join()
instead of using nsIFile .append(). Going the other direction,
Zotero.File.pathToFile() will convert a path to an nsIFile. Many methods
now take either (e.g., Zotero.File.getContentsAsync()).

> Zotero.getString()
> Zotero.HTTP.doGet()

Use promise-based Zotero.HTTP.request() instead

> Zotero.Item()
> Zotero.Items.get()
> Zotero.ItemTypes.getLocalizedString()
> Zotero.ItemTypes.getName()
> Zotero.launchFile()
> Zotero.logError()
> Zotero.Notifier.registerObserver()
> Zotero.Notifier.unregisterObserver()
> Zotero.Prefs.get()
> Zotero.Search()

All unchanged. Note that you may see Zotero.Items.getAsync() and similar
in some places, but those just haven't been reverted since deasyncification.

> Zotero.Searches.getAll()

Returns a promise, but that should be deasyncified.

> Zotero.Sync.Storage.setSyncedHash()
> Zotero.Sync.Storage.setSyncState()

Replaced with attachmentSyncedHash and attachmentSyncState properties on
Zotero.Item.prototype, and require separate item save()/saveTx().

> Zotero.Tags.getID()
> Zotero.Tags.getName()
> Zotero.Utilities.htmlSpecialChars()
> Zotero.Utilities.levenshtein()
> Zotero.Utilities.removeDiacritics()
> Zotero.Utilities.text2html()
> Zotero.Utilities.trimInternal()

Unchanged

> ZoteroPane.getSelectedLibraryID()
> ZoteroPane.getSelectedCollection()
> ZoteroPane.getSelectedItems()
> ZoteroPane.getSelectedSavedSearch()

Unchanged

> ZoteroPane.onCollectionSelected()
> ZoteroPane.collectionsView.selectLibrary()

Return promises, for complicated reasons

> ZoteroPane.itemsView.selectItems()
> ZoteroPane.itemsView.saveSelection()

Unchanged

[Dan Stillman]

On 3/23/16 4:30 PM, jl wrote:
> ps: And chance that there is a methods to chance from imported to
> linked attachment in 5.0 that does not corrupt the database (without
> just creating a new attachment item)?

No, afraid not, sorry. There are different checks on the different link
modes, so it would require changes to both client and server, and I
think dealing with, say, a sync conflict between a linked attachment and
a stored attachment would get messy.

[Will S]

I started working on upgrading my plugins (mainly adding item.saveTx() to the end of functions that were broken in 5.0), but I haven't worked with promises much before. I'm hoping this is an easy question. I have a few functions that copy an item and then modify it. The way they work in 4.0 is by calling ZoteroPane.duplicateSelectedItem() and then ZoteroPane.getSelectedItems() to get the new item to modify. With 5.0, duplicateSelectedItems is asynchronous so getSelectedItems finds the old selected item still selected and that one gets modified by the rest of my function (and then it ends up getting duplicated). I tried putting my modifications (with the call to getSelectedItems() inside it) into a function called modifyItem and changing "ZoteroPane.duplicateSelectedItem()" to "ZoteroPane.duplicateSelectedItem().then(modifyItem)" but modifyItem item was still getting and modifying the old item. I also tried changing "ZoteroPane.duplicateSelectedItem()" to "yield ZoteroPane.duplicateSelectedItem()" but that gave an error about a return value from an anonymous generator. Is there syntax I can use to keep using ZoteroPane.duplicateSelectedItem() in this way, or do I need to clone the item with lower level calls? Or is there a different problem (do I need to include Bluebird in my plugin?)?

[Dan Stillman]

(Sorry, didn't see this earlier.)

So, first, it's almost always better to use yield than .then(), but to
use yield 1) you have to be in an ES6 generator (i.e., function*, not
function — the latter with yield produces an old Mozilla-style
generator, which won't work) and 2) that generator has to be wrapped
with Zotero.Promise.coroutine() (which is just Bluebird's coroutine()).

duplicateSelectedItem() doesn't return until the new item is selected,
so that should work. It'd be better to avoid using getSelectedItems()
here anyway, though, so I've modified duplicateSelectedItem() to just
return the new Zotero.Item.

And just to confirm, is that definitely the method you want to use? That
is, is the behavior you want here to duplicate the currently selected
item and add the new item to the currently selected collection? (If not,
you'd want to use Zotero.Item.prototype.clone() directly.)

[Will S]

Thanks, Dan! I changed that section of the function to the following and it works:

if (zoteroVersion.split('.')[0] < 5) {
    // XXX: Legacy 4.0
    ZoteroPane.duplicateSelectedItem();
    zitems = this.getSelectedItems();
    var sectionItem = zitems[0];
    modifyNewItem(this, sectionItem)
} else {
    Zotero.Promise.coroutine(function* (context) {
        let sectionItem = yield ZoteroPane.duplicateSelectedItem()
        modifyNewItem(context, sectionItem)
    })(this)
}

I guess it would look cleaner if I didn't try to maintain backwards compatibility (then I could just wrap the calling function in coroutine()).

I want the item added to the collection and selected, so I think duplicateSelectedItem is good here.

[Frank Bennett]

Thanks Dan, this is really good to hear.

After the move to async was announced last year, I took a diff against master and worked through a bunch of patches to adapt Juris-M to the async code. Much to my surprise, I was able to get it working in a reasonable amount of time. It wasn't nearly as tough as I had feared---and the Bluebird stack traces were much more helpful than ordinary JS exceptions in Firefox. Given the scale of changes in master since that initial foray, I'll be discarding the initial work, but it was a valuable learning experience and built confidence.

If I understand correctly, the plan will be to run a beta of Zotero 5.0 for some time, and then retain legacy sync for a few months after the official 5.0 release. That should work out well for us here: teaching term is about to begin, so I'll plan to start work on a 5.0 version of Juris-M in July, to complete during the summer break. If it works out, Juris-M users will be able to move over smoothly to the new sync architecture before the old method is discontinued.

Looking forward to working with 5.0!

Frank

[Ephrim Khong]

On 23.03.2016 20:26, Dan Stillman wrote:
> On 3/23/16 9:42 AM, Ephrim Khong wrote:
>> For those running their own server: are there any incompatibilties in
>> the API used between client and server? Essentially, can I continue to
>> run the rather old version of the server with the new 5.0 client?
>
> No, the client will only work with the latest server code, so you or
> someone else will need to get the new code working in order to run 5.0.

Thanks for the heads up! Were there any changes to the server's data
scheme or only to the sync part? In other words, do you think that
after merging and pushing all current changes to the production server,
can we just run the new server code on the old DB and it will work
(probably automatically update the schema where required), or will there
be some manual DB transitions necessary?

Thanks,
- Eph

[Dan Stillman]

On 4/6/16 5:19 AM, Ephrim Khong wrote:
> Thanks for the heads up! Were there any changes to the server's data
> scheme or only to the sync part? In other words, do you think that
> after merging and pushing all current changes to the production
> server, can we just run the new server code on the old DB and it will
> work (probably automatically update the schema where required), or
> will there be some manual DB transitions necessary?

Depends how old your version is, but there are probably some schema
changes required, and you'll have to manually apply them. Whenever such
changes are made, they're indicated with "[schema change]" in the commit
message — if you're running your own server, you definitely want to
following along with those. Sometimes the message includes the exact SQL
commands to run (as is the case for the last one, a year ago [1]) and
sometimes for simpler changes you have to look at the diff for the .sql
files in the 'misc' directory.

[1]
https://github.com/zotero/dataserver/commit/021b378065374b222192f8f422ad130db14bf98c

[jl]

I started working on a zotfile version for zotero 5 a little and have a question about changes in `libraryID`. It used to be the case that libraryID is null for the local library and only defined for group libraries. Is libraryID now always 1 for the local library? So can I use item.libraryID == 1 to see whether an item is in a local or in a group library?

Did anything change with the Zotero protocol handlers? It used to be the case that 'zotero://.../[LIB ID]_[KEY]/...' referred to the item with key KEY in library LIB ID. LIB ID was 0 for the local library and the actual library ID for other libraries. Is that still the case? 

Thanks!

[Dan Stillman]

On 4/10/16 7:14 PM, jl wrote:
> I started working on a zotfile version for zotero 5 a little and have
> a question about changes in `libraryID`. It used to be the case
> that libraryID is null for the local library and only defined for
> group libraries. Is libraryID now always 1 for the local library? So
> can I use item.libraryID == 1 to see whether an item is in a local or
> in a group library?

It's 1, but it's better to do item.libraryID ==
Zotero.Libraries.userLibraryID. Note, too, there are other library types
(feeds, publications) and will likely be more in the future, so it might
be better to do item.library.getType() == 'user'.

(I just added .library — before that you would've had to do
Zotero.Libraries.get(item.libraryID).getType() == 'user'.)

> Did anything change with the Zotero protocol handlers? It used to be
> the case that 'zotero://.../[LIB ID]_[KEY]/...' referred to the item
> with key KEY in library LIB ID. LIB ID was 0 for the local library and
> the actual library ID for other libraries. Is that still the case?

The protocol handlers still need a bit more work in 5.0, but the URLs
are now modeled after the web API, so this would a URL for a user
library item:

zotero://report/library/items/report.html?itemKey=N44LYNTL

And a group URL would be:

zotero://report/groups/151256/report.html?itemKey=N44LYNTL

(The user library isn't /users/:userID like in the web API because there
isn't necessary a userID locally, though /users/:userID could be made to
work after the user had synced.)

Various other parameters from the API will be supported (e.g.,
'sort=itemType&direction=desc'). (Eventually I'd like the client to
actually serve a subset of the web API over HTTP, allowing external
programs to work with either the Zotero servers or the local client.)

Old URLs are probably broken, but we can restore those before 5.0 ships.

(Also, the client no longer gets global libraryIDs, since those aren't
exposed via the API, so local libraryIDs will be DB-local going forward.
That's why the URLs are changing to mirror the web API, with global
group IDs.)

[Sebastian Karcher]

> The protocol handlers still need a bit more work in 5.0, but the URLs
> are now modeled after the web API, so this would a URL for a user
> library item:
>
> zotero://report/library/items/report.html?itemKey=N44LYNTL
>
> And a group URL would be:
>
> zotero://report/groups/151256/report.html?itemKey=N44LYNTL

Don't mean to derail, but I just want to say that that's awesome -- I
had just meant to write up a feature request for exactly this. With
attachments URIs now stable across synced devices, I think it should be
possible to use open annotation tools like hyptothes.is for them.

[Dan Stillman]

On 4/10/16 7:53 PM, Dan Stillman wrote:
> it might be better to do item.library.getType() == 'user'.

Sorry, I meant item.library.libraryType == 'user'. (I was thinking of
Zotero.Libraries.getType(libraryID), which is now deprecated.)

[jl]

Thanks for all the info! Very helpful. Couple of follow ups:

1) I am having problems with the new select URL. In this example, the report url works but select gives the error "Selected items not found" (line 853 in zotero-protocol-handler.js). Maybe I am just using the wrong URL for select though.

<p><a href="zotero://report/library/items/report.html?itemKey=JHYDCRBD">report</a></p>

<p><a href="zotero://select/library/items/report.html?itemKey=JHYDCRBD">select</a></p>

2) I think it would be good to maintain compatibility with the old URL just because some people might use them.

3) It would be great if 5.0 also fixes this issue "Opening zotero links (zotero://...) also opens an empty window" (https://github.com/zotero/zotero/issues/498)

[Dan Stillman]

On 4/11/16 11:23 AM, jl wrote:
> Thanks for all the info! Very helpful. Couple of follow ups:
>
> 1) I am having problems with the new select URL. In this example, the
> report url works but select gives the error "Selected items not found"
> (line 853 in zotero-protocol-handler.js). Maybe I am just using the
> wrong URL for select though.
>
> <p><a
> href="zotero://report/library/items/report.html?itemKey=JHYDCRBD">report</a></p>
> <p><a
> href="zotero://select/library/items/report.html?itemKey=JHYDCRBD">select</a></p>

There shouldn't be a "/report.html" in zotero://select links.

(We also can probably remove it in report links too. It was there
originally so that Firefox's built-in save functionality chooses a
proper filename when saving a report, but I'm not sure reporting saving
works in Standalone at all right now, and if we add that ability
ourselves we might have more control of the filename. And we might be
able to get it to save as "items.html" anyway, which would be fine.)

[Dan Stillman]

OK, I've removed "/report.html" for reports. Firefox saves as "Zotero
Report.html" now anyway.

[Sebastian Karcher]

Following up on this -- is there any reason why (for an attachment)
zotero://select/library/items?itemKey=4IRKVFFX"
and
zotero://report/library/items?itemKey=4IRKVFFX
work, but
zotero://attachment/library/items?itemKey=4IRKVFFX
fails with "attachment ID not an integer"? I was really hoping for this
to work for attachments in particular.

[Dan Stillman]

On 5/6/16 12:47 AM, Sebastian Karcher wrote:
> Following up on this -- is there any reason why (for an attachment)
> zotero://select/library/items?itemKey=4IRKVFFX"
> and
> zotero://report/library/items?itemKey=4IRKVFFX
> work, but
> zotero://attachment/library/items?itemKey=4IRKVFFX
> fails with "attachment ID not an integer"? I was really hoping for this
> to work for attachments in particular.

Issue created: https://github.com/zotero/zotero/issues/992

[jl]

Just to be clear on this: There is no synchronous way to check whether an attachment file exists in Zotero's 5.0 API, right? Zotero.Item.fileExists() is now async and uses Zotero.Item.getFilePathAsync(). There is a synchronous version of  getFilePathAsync called Zotero.Item.getFilePath but that only returns false if the path is invalid not if the file doesn't exist (subtle difference between getFilePath and getFilePathAsync). Finally, there is Zotero.Item.fileExistsCached but my impression is that shouldn't be used for operations on files.

var atts = Zotero.Items.get(attIDs)

.filter(att => att.isAttachment() && !att.isTopLevelItem())

            .filter(att => att.getFile())

[jl]

Incomplete post, sorry. I wanted to add that I often use this

var atts = Zotero.Items.get(attIDs)

      .filter(att => att.isAttachment() && !att.isTopLevelItem())

      .filter(att => att.fileExists())

But I guess I have to work around that because generator functions don't work with arrow functions so that yield att.fileExists() doesn't work. Of course, I can fall back on nsIFile stuff but that seems short sided. 

[Marielle Volz]

Ages ago there was some talk of the item types and fields changing for 5.0 (this stuff http://aurimasv.github.io/z2csl/typeMap.xml) - is this still the case? Are there docs/code corresponding to this anywhere yet? Should we file bugs and tag for 5.0 when things are odd (like ISBN being a valid field for videoRecording but not film)?

On Tue, Mar 22, 2016 at 8:44 PM, Dan Stillman <dsti...@zotero.org> wrote:

We're aiming to release a developer preview of Zotero 5.0 within the next couple weeks, to be followed soon after by a public beta.

While we're not totally done moving things around, if you develop a Zotero extension, now would be an excellent time to begin updating it to work with 5.0, if you haven't yet. I'm happy to answer any questions you have about the changes, which are, let's just say, extensive.

If you've been following along with development on the master branch, you may have seen both Promisification [1] and Deasyncification [2]. The former has been a long process of switching to asynchronous database and (to a lesser extent) file access, which involved converting much of the code base to use promises via ES6 generators. The latter involved reverting or moving some of those changes as we discovered functionality that couldn't be made to work asynchronously in the Mozilla framework (most notably drag-and-drop Quick Copy). The upshot of these two endeavors is that Zotero now reads most data asynchronously at startup but keeps many data layer retrieval methods unchanged, while all operations that require writes to the database return promises.

We took the opportunity to make a lot of other changes, but hopefully in the service of a much cleaner and more logical code base. 5.0 also has a massively expanded test suite (600+ tests and counting), which should greatly improve Zotero's stability and make it easier to development for. Overall, the changes in 5.0 should provide for a much smoother user experience and lay the groundwork for some major, long-awaited changes, such as an improved data model.

Thanks for your patience with these changes, and let me know if you have any questions.

- Dan

[1] https://github.com/zotero/zotero/issues/518
[2] https://github.com/zotero/zotero/commit/daf4a8fe4db46ed29de2babed79610a2522821df

--
You received this message because you are subscribed to the Google Groups "zotero-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zotero-dev+...@googlegroups.com.
To post to this group, send email to zoter...@googlegroups.com.
Visit this group at https://groups.google.com/group/zotero-dev.
For more options, visit https://groups.google.com/d/optout.

[Sebastian Karcher]

On 05/23/2016 02:42 PM, Marielle Volz wrote:
> Ages ago there was some talk of the item types and fields changing for
> 5.0 (this stuff http://aurimasv.github.io/z2csl/typeMap.xml) - is this
> still the case? Are there docs/code corresponding to this anywhere
> yet? Should we file bugs and tag for 5.0 when things are odd (like

> ISBN being a valid field for videoRecording but not film)?0
this will be after 5.0 (hopefully very soon after) for logistical reasons.
There's an (informal) collection of issues -- some relating to CSL, not
Zotero -- here: https://github.com/avram/zotero-bits/issues/
feel free to add. (FWIW, I think the ISBN choice seems right to me --
films that aren't "published as a DVD/video" can't have an ISBN).

> <mailto:zotero-dev%2Bunsu...@googlegroups.com>.

> To post to this group, send email to zoter...@googlegroups.com

> <mailto:zoter...@googlegroups.com>.

> Visit this group at https://groups.google.com/group/zotero-dev.
> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "zotero-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to zotero-dev+...@googlegroups.com

> <mailto:zotero-dev+...@googlegroups.com>.

> To post to this group, send email to zoter...@googlegroups.com

> <mailto:zoter...@googlegroups.com>.

[Dan Stillman]

That's all correct. There's no guarantee how long file access will take, so it shouldn't block the main thread with a synchronous call. One approach would be to do Zotero.Promise.all(items.map(item => item.fileExists())) (or maybe Zotero.Promise.map(items, item => item.fileExists()) [1]) followed by the synchronous filter() with fileExistsCached().


[1] http://bluebirdjs.com/docs/api/promise.map.html

[Dan Stillman]

Or, sorry, you could just use Bluebird's promise-based filter():

http://bluebirdjs.com/docs/api/promise.filter.html

[jl]

Thanks! Super helpful! And the correct way to check whether an attachment is a pdf would be `item.attachmentContentType == "application/pdf"`, right?

[Dan Stillman]

On 5/24/16 7:41 AM, jl wrote:
> Thanks! Super helpful! And the correct way to check whether an
> attachment is a pdf would be `item.attachmentContentType ==
> "application/pdf"`, right?

Yes, that's right.

[jl]

Can you say something about the difference between item.save() and item.saveTx()? Also does it make sense to minimize the item.save[Tx]() calls? So should I try to first make all changes to tags, title etc and then call save?

[Dan Stillman]

On 5/28/16 2:31 PM, jl wrote:
> Can you say something about the difference between item.save()
> and item.saveTx()?

They're exactly the same, except item.save() requires a DB transaction
to be open already and saveTx() opens one itself (and therefore can't be
called within an existing transaction, since the second transaction will
try to wait for the other, causing a deadlock). Generally you'd use the
former if you're modifying multiple items at once, or a combination of
different objects (e.g., creating a collection and adding some items to
it), and you'd use the latter for making a change to a single item.

> Also does it make sense to minimize the item.save[Tx]() calls? So
> should I try to first make all changes to tags, title etc and then
> call save?

Yes, definitely. Transactions are expensive, and there's also other
overhead to saving objects. So you want to do things in batches as much
as possible. And due to the way the Zotero web API works, most data in
5.0 — tags, collections, relations — is now set as a property of a
Zotero.Item, which means that most changes can be made with a single
save call.

The one exception here is that, if you're modifying a large number of
objects, doing everything in a single transaction isn't ideal.
Transactions are serialized, which means that new transactions will
queue until the current transaction finishes. This won't hang the UI in
5.0, but it could mean that clicking on something wouldn't appear to
have an effect until the current transaction finishes.

[jl]

Thanks!

So for single items, best practice would be

item.addTag(tag);

item.setField('title', title);

yield item.saveTx()

and for multiple items (this is the part I am not sure about: 

yield Zotero.DB.executeTransaction(function* () {

     for (let itemID of itemIDs) {

           let item = yield Zotero.Items.getAsync(itemID);

           item.addTag(tag);

           item.setField('title', title);

           yield item.save();

     }

});

Zotero.DB.executeTransaction is preferable unless itemIDs is large.

[Dan Stillman]

On 5/29/16 11:33 AM, jl wrote:
> Thanks!
>
> So for single items, best practice would be
>
> item.addTag(tag);
> item.setField('title', title);
> yield item.saveTx()
>
> and for multiple items (this is the part I am not sure about:
>
> yield Zotero.DB.executeTransaction(function* () {
> for (let itemID of itemIDs) {
> let item = yield Zotero.Items.getAsync(itemID);
> item.addTag(tag);
> item.setField('title', title);
> yield item.save();
> }
> });
>
> Zotero.DB.executeTransaction is preferable unless itemIDs is large.

Yes, that's right.

Another thing to clarify: for operations in a library that the user has
necessarily already interacted with (e.g., because this was triggered by
some UI function for a selected item), instead of 'yield
Zotero.Items.getAsync()' you can just use 'Zotero.Item.get()', which
will be faster. Post-deasyncification, once a library is viewed by the
user, all item data will have been loaded. getAync() is still useful if
it might be operating on an item in a library that hasn't been loaded.

[jl]

Another question about item.eraseTx() (might not be 5.0 specific though).

Any reason why item.eraseTx() changes the selection and is there a way to avoid that? Its pretty annoying for some of zotfile's functionality. Here are the steps to reproduce it:

1. Select Zotero item with at least two attachments

2. run code

```

var item = ZoteroPane.getSelectedItems()[0];

var att = Zotero.Items.get(item.getAttachments()[0]);

att.eraseTx()

```

Now Zotero changes the selection to the remaining attachment. I guess that makes sense when the deleted att was selected but it doesn't really make sense when something else was selected.