Comments on the proposed DataStore API

[JOSE MANUEL CANTERA FONSECA]

Hi there,

My comments to https://wiki.mozilla.org/WebAPI/DataStore

1. The current API proposal assumes that records are going to be indexed
by integer key but I think that assumption is not going to be always
useful / true. A more generic approach would be to have string keys as any
data type could be converted to a string.

2. One of the use cases to which this API would apply is FB Contacts Data
Store. The issue is that in the current Gaia implementation the FB Data is
indexed by different fields, particularly uid and telephone number. Thus,
we would need the ability to support multiple indexes in the Data Store.
Thus the DataStore interface would need some kind of mechanism to support
the creation / declaration of indexes and additional interfaces would be
needed to allow to query the indexes by key.

3. I'm missing a mechanism that would allow to obtain a cursor for all the
records in the DataStore.

These gaps in the proposed API make wonder if it wouldn't more simple to
simply add an extension to the indexedDB API that would allow applications
to create shared indexedDBs. With this proposal it would be pretty simple
to support all the use cases that indexedDB currently supports, providing
more flexibility.

Many thanks,

All the best


________________________________

Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
http://www.tid.es/ES/PAGINAS/disclaimer.aspx

[Mounir Lamouri]

On 02/05/13 14:24, JOSE MANUEL CANTERA FONSECA wrote:
> Hi there,
>
> My comments to https://wiki.mozilla.org/WebAPI/DataStore
>
> 1. The current API proposal assumes that records are going to be indexed
> by integer key but I think that assumption is not going to be always
> useful / true. A more generic approach would be to have string keys as any
> data type could be converted to a string.
>
> 2. One of the use cases to which this API would apply is FB Contacts Data
> Store. The issue is that in the current Gaia implementation the FB Data is
> indexed by different fields, particularly uid and telephone number. Thus,
> we would need the ability to support multiple indexes in the Data Store.
> Thus the DataStore interface would need some kind of mechanism to support
> the creation / declaration of indexes and additional interfaces would be
> needed to allow to query the indexes by key.
>
> 3. I'm missing a mechanism that would allow to obtain a cursor for all the
> records in the DataStore.
>
> These gaps in the proposed API make wonder if it wouldn't more simple to
> simply add an extension to the indexedDB API that would allow applications
> to create shared indexedDBs. With this proposal it would be pretty simple
> to support all the use cases that indexedDB currently supports, providing
> more flexibility.

Hi Jose,

I think there is a misunderstanding with the purpose of this API and it
is mostly because this wiki page wasn't meant to be shared very widely
but it ended up pasted here and there (and I'm definitely faulty for that).

The DataStore API isn't actually meant to be used as a storage mechanism
but as a storage sharing mechanism. That means that a Facebook
application will unlikely use this DataStore to save its contacts but
will mirror its internal IndexedDB with this DataStore. Then, a Contacts
application will show Facebook contacts and internal contacts in the
same UI, it will try to keep in sync with the Facebook Contacts and
System Contacts DataStores and have an internal representation of those
stores merged a way or another.
For example, we could definitely imagine that the Contacts applications
will merge contacts with the same name and simply add a Facebook button
when this contact is also in Facebook. In that case, this application
will not fully copy the Facebook Contacts DataStore but just check
name/handle association.

The reason why we ended up with that design is mostly for reasons you
pointed in your email: indexing is mostly application dependant and even
if the Facebook application indexes contacts on some columns,
applications using the Facebook contacts might not want to index the
data the same way. Also, some applications will actually take those data
and merge them with others.
For example, a Messaging application might store the last message, the
contact name and picture and other information to improve loading
performances. That requires to merge some information between the System
Contacts store and the System Messages store.

Why not exposing IndexedDB instead of this raw format? We didn't spend
too much time on this but my feeling would be that, for System
DataStores, that would require the System to have a copy of its storage
using IndexedDB while it might be in another format for real. A simple
format is less hard for that. Also, even for application provided
DataStores, you could imagine that an application is actually not using
IndexedDB for its storage mechanism (nowadays, in the Firefox OS
context, this isn't much realistic but we should try to see wider than
that).
There might also be difficulties to handle some features we want the
DataStore to be able to do but Jonas Sicking or Ben Turner would know
more about this.

Does these explanations make sense and help understanding the use cases
of that API?

Thanks,
--
Mounir

[JOSE MANUEL CANTERA FONSECA]

El 03/05/13 15:42, "Mounir Lamouri" <mou...@lamouri.fr> escribió:

I would like to know how that mirroring could be implemented without
duplicating the data between the DataStore and the internal app database
(indexedDB or whatever). Apparently with the current API design that
cannot be done and it seems the data store would be another "table" but
not a view over the internal app "tables".

>Then, a Contacts
>application will show Facebook contacts and internal contacts in the
>same UI, it will try to keep in sync with the Facebook Contacts and
>System Contacts DataStores and have an internal representation of those
>stores merged a way or another.
>For example, we could definitely imagine that the Contacts applications
>will merge contacts with the same name and simply add a Facebook button
>when this contact is also in Facebook. In that case, this application
>will not fully copy the Facebook Contacts DataStore but just check
>name/handle association.
>
>The reason why we ended up with that design is mostly for reasons you
>pointed in your email: indexing is mostly application dependant and even
>if the Facebook application indexes contacts on some columns,
>applications using the Facebook contacts might not want to index the
>data the same way.

Thus, if a particular application needs different indexes that would mean
that such an application would need to get a snapshot of the DataStore and
index such data accordingly in its own indexedDB, right?

>Also, some applications will actually take those data
>and merge them with others.
>For example, a Messaging application might store the last message, the
>contact name and picture and other information to improve loading
>performances. That requires to merge some information between the System
>Contacts store and the System Messages store.

When you talk about System Contacts store and System Messages store, I can
guess you plan to expose Contacts or Messages also as DataStores, and that
there will be application data stores and System Data Stores? Am I right?

>
>Why not exposing IndexedDB instead of this raw format? We didn't spend
>too much time on this but my feeling would be that, for System
>DataStores, that would require the System to have a copy of its storage
>using IndexedDB while it might be in another format for real. A simple
>format is less hard for that. Also, even for application provided
>DataStores, you could imagine that an application is actually not using
>IndexedDB for its storage mechanism (nowadays, in the Firefox OS
>context, this isn't much realistic but we should try to see wider than
>that).
>There might also be difficulties to handle some features we want the
>DataStore to be able to do but Jonas Sicking or Ben Turner would know
>more about this.

I would be happy to know more about that

>
>Does these explanations make sense and help understanding the use cases
>of that API?

Yes, they have been helpful but please note my comments

>
>Thanks,
>--
>Mounir
>_______________________________________________
>dev-webapi mailing list
>dev-w...@lists.mozilla.org
>https://lists.mozilla.org/listinfo/dev-webapi