How does the cache work with multiple tabs/windows open?

[Jeremy Scheff]

https://github.com/google/lovefield/blob/master/docs/dd/04_cache.md says:

Lovefield has an in-memory row cache, which is conceptually a big map of row ids to rows (and that is why Lovefield has unique row ids across the board). Currently the cache is a "dumb" cache: it contains exact duplica of what are persisted in the IndexedDB. The reason for doing that is to workaround IndexedDB inefficiency of handing bulk I/O, as described in backstore section. By caching all rows in memory, Lovefield avoids any additional round-trip required to load data from IndexedDB, with the price of memory usage.

I've thought about the same thing for the same reason, but never got around to implementing it because I thought it would be a pain to handle synchronization when the user opens my app in multiple windows. Does Lovefield have any solution for that?

[Arthur Hsu]

There's not a good solution so far.

https://github.com/google/lovefield/blob/master/docs/spec/03_life_of_db.md#32-multi-process-connection

The common practice will be having a ServiceWorker or a background page that handles all Lovefield operations, and the other tabs/windows postMessage to that ServiceWork or background page to perform DB operations.

IndexedDB people are discussing a change notification spec, which will help in cross-process synchronization. I don't have any idea when will that spec settle, though.

[nbru...@gmail.com]

Does this mean that, if I'm using Lovefield, I need to go out of my way to make sure that the user does not open two different tabs to my website?  If I don't do that, I would expect to see data saved in one tab not returned in queries run in the other tab?

I would have to somehow detect if there are multiple tabs connecting to the same database and force the user to close one of them?

According to caniuse, Service Workers are not fully supported in any browser, so I'm not really excited about trying to use that approach.

[dpa...@chromium.org]

Correct. As explained on Lovefield's spec, data saved in one tab, will not be seen by the other tab, because the other tab's in memory data structures (cache, indices) will not be up-to-date. This is a limitation caused by the fact that there is no easy way to observe Indexed DB. There has been discussion about adding a way to observe IDB (see proposal at https://github.com/WICG/indexed-db-observers), but it is unclear when/if this will happen, and it can't help us as of now.

Besides the workarounds you already mentioned (limiting DB connections to 1, or using a service worker), you could consider using a Firebase backing store instead of IDB (see https://github.com/google/lovefield/blob/master/docs/spec/02_data_store.md#22-persistence). This would result in all your tabs getting updated (tab1 updates Firebase server via a Lovefield query, Firebase server notifies all Firebase instances about changes, in-memory data structures get updated across tabs), at the expense on not supporting offline cases (at least not for writing).

[nbru...@gmail.com]

Thanks for the response.  I have a follow-up question for you.

If I'm comfortable with the two tabs not sharing any information after the initialization of each, can I trust that they won't?  From reading the docs, it sounds like the entire database is copied into RAM before you can make any queries.  So it sounds like the two tabs will essentially live completely separately from each other once each of their databases is initialized.  This is probably fine for my use case (or at least for some of my use cases).

The thing I'm worried about is that some of the data saved from one tab might become available to the other, but not all of it.  For example, if I have two tables, A and B, and there is a foreign key relationship between them, then it would be bad if tab X saved data to both tables, and then when tab Y executed a query, it got the new stuff from table A but not from table B.  Can I be sure nothing like that will happen?

[dpa...@chromium.org]

If you are using Indexed DB and you are planning to have two separate tabs write to the DB, then you will run to problems (I'll explain further below).

You are right that the entire DB is placed in an in-memory cache on startup.  If you are only doing READ_ONLY queries, you will not be interacting with IDB 

after startup and therefore there will not be any issues.

Once you start issuing READ_WRITE queries, things break. Imagine the following scenario.

1) Tab1 adds a record in TableA with primary key 100.

2) Tab2 attempts to add a record in TableA with the same primary key. Normally (if both queries happened in the same tab), Lovefield would detect the constraint violation (PK must be unique), and would throw an appropriate exception. Because Tab2's in-memory indices are not updated after 1, Lovefield will not detect the violation and will attempt to write to IDB. You have just entered undefined/unspecified behavior. If you want to dive deeper in this undefined behavior, here is what will happen.

Lovefield will (erroneously) succeed writing the 2nd row with the same primary key in IDB. The DB is corrupted now (persisted data violate constraints). Next time you open a new tab and attempt to initialize a DB connection, you will always get a PK violation error, with no way to recover.

This was just one example of many that will corrupt the DB in a multi-connection case. In short, it is not safe to make any assumptions when using multiple DB connections. I suggest you rethink the following things about your app.

1) Do you really need persistence? There are many use cases where an app wants to leverage Lovefield's SQL queries, but does not need persistence. If that's the case, MEMORY_DB is the right choice, not INDEXED_DB.

2) Do you need to allow multiple tabs writing to the DB? Consider using webworker/serviceworker (I have not tried using Lovefield in a service worker myself, but others have tried with varying degrees of success).

3) Could you have multiple DB connections but limit to only have one tab writing to the DB, and everybody else just reading? In that case you would not corrupt the DB, but you would get obsolete results in all tabs (except the one that writes).

I understand that none of the above sounds very appealing, but unfortunately that is the state of things with regards to multiple DB connections.

Hope that helps

[nbru...@gmail.com]

That helps a whole lot.  Thanks for the detailed explanation.