Clearing the library

[Emiliano Heyns]

As part of my tests I wish to clear the library between runs. I currently run the code below, but if there are some 3.4k items int my library, it times out

Zotero.Promise.coroutine(function*() {

  return (yield Zotero.DB.executeTransaction(function*() {

    var collection, i, items, len, ref;

    items = (yield Zotero.Items.getAll(Zotero.Libraries.userLibraryID, false, true, true));

    yield Zotero.Items.erase(items);

    yield Zotero.Items.emptyTrash(Zotero.Libraries.userLibraryID);

    ref = Zotero.Collections.getByLibrary(Zotero.Libraries.userLibraryID, true);

    for (i = 0, len = ref.length; i < len; i++) {

      collection = ref[i];

      yield collection.erase();

    }

  }, {

    waitTimeout: 300000

  }));

})();

The error console says "Timestamp: 8/16/17, 10:13:08 PM Error: operation timed out"; the debug log says

(1)(+0005822): Timed out waiting for transaction xVwL8AD5

(1)(+0000010): Timed out waiting for transaction wFfqcMu5

The only extensions I have installed are

1. MozRepl (to execute the code -- of there are other ways to execute chrome code in Zotero I'd love to know)
2. Zotero LibreOffice Integration
3. Zotero Word for Mac Integration (this is on a Mac)

I've generated a debug report under D553718995. If there is another way to efficiently clear the library that's also OK of course. I already tried calling erase on the library but as it's the user library it's not allowed.

[Dan Stillman]

On 8/16/17 10:28 PM, Emiliano Heyns wrote:
> As part of my tests I wish to clear the library between runs.
>

> […]

>
> If there is another way to efficiently clear the library that's also
> OK of course.

This is what we use:

https://github.com/zotero/zotero/blob/d1de8b751a3b6594d18de404d18c461669a5cf0b/test/content/support.js#L563
https://github.com/zotero/zotero/blob/d1de8b751a3b6594d18de404d18c461669a5cf0b/components/zotero-service.js#L219

[Emiliano Heyns]

I tried translating that to this code, but it's not returning, what am I doing wrong?

Zotero.Promise.coroutine(function*() {

  // other stuff

  var db;

  db = Zotero.DataDirectory.getDatabase();

  return (yield Zotero.reinit(Zotero.Promise.coroutine(function*() {

    yield OS.File.remove(db);

  }), false, {}).then(function() {

    return Zotero.Schema.schemaUpdatePromise;

  }));

  // more stuff

})();

 

[Emiliano Heyns]

Wait, this would also reload and reinit all translators, right? As I'm doing mostly translator tests, this would mean there'd be 10-30 seconds between each test case (that's how long the translator init seems to take on Circle), which would really slow my tests down. Can't I do something like "delete from items" and tell Zotero to clear the items cache?

[Emiliano Heyns]

More specifically: should the code below do the job, and how do I tell Zotero to drop the collections/items cache (in toto is OK, there will not be other libraries in my tests):

Zotero.Promise.coroutine(function*() {

  yield Zotero.DB.executeTransaction(function*() {

    yield Zotero.DB.queryAsync("DELETE FROM itemData WHERE itemID in (SELECT itemID from items WHERE libraryID=?)", [Zotero.Libraries.userLibraryID]);

    yield Zotero.DB.queryAsync("DELETE FROM itemDataValues WHERE valueID NOT IN (SELECT valueID from itemData)");

    yield Zotero.DB.queryAsync("DELETE FROM items WHERE libraryID = ?", [Zotero.Libraries.userLibraryID]);

    yield Zotero.DB.queryAsync("DELETE FROM deletedItems WHERE itemID NOT IN (SELECT itemID FROM items)");

    yield Zotero.DB.queryAsync("DELETE FROM collections WHERE libraryID = ?", [Zotero.Libraries.userLibraryID]);

    /*

      notify Zotero that items for Zotero.Libraries.userLibraryID must be uncached

      notify Zotero that collections for Zotero.Libraries.userLibraryID must be uncached

     */

  });

})(); 

[Emiliano Heyns]

On Wednesday, August 16, 2017 at 10:44:35 PM UTC+2, Dan Stillman wrote:

The script I used times out, but the Zotero client manages to do it without timeouts (albeit not as fast as I thought it'd be). Move to trash is pretty fast, and empty trash at least finishes without errors. What code would I use to replicate this process?

[Emiliano Heyns]

On Friday, August 18, 2017 at 11:23:48 AM UTC+2, Emiliano Heyns wrote:

On Wednesday, August 16, 2017 at 10:44:35 PM UTC+2, Dan Stillman wrote:

On 8/16/17 10:28 PM, Emiliano Heyns wrote:
> As part of my tests I wish to clear the library between runs.
>
> […]
>
> If there is another way to efficiently clear the library that's also
> OK of course.

This is what we use:

https://github.com/zotero/zotero/blob/d1de8b751a3b6594d18de404d18c461669a5cf0b/test/content/support.js#L563
https://github.com/zotero/zotero/blob/d1de8b751a3b6594d18de404d18c461669a5cf0b/components/zotero-service.js#L219

I have this working now, but it takes ~ 50 seconds to delete 3.5k references on a fairly beefy machine. This would make a full reset including translator reinit the faster option but I haven't gotten that to work yet. My tests don't require the deletion of large numbers of references often (only one test does), so this is semi-managable. If there's a quicker way by direct deletes on the database and then telling Zotero to drop the cache but which does not require a translators reinit (which would kill my tests), I'm very much interested.

  var chunk, items;

  items = (yield Zotero.Items.getAll(Zotero.Libraries.userLibraryID, false, true, true));

  while (items.length) {

    chunk = items.splice(0, 100);

    yield Zotero.Items.erase(chunk);

  }