Apps using indexedDB

[Fabrice Desre]

tl;dr: if your app doesn't use indexedDB, you're fine. If it does,
please read.

Back in the early days, when we had no way to set application
permissions from their manifest (don't look at me like if was 90 years
old, this has changed only last August) we granted permissions by
default to preloaded apps using a hardcoded list. Archeologists can
refer to
https://mxr.mozilla.org/mozilla-central/source/b2g/chrome/content/shell.js?rev=d3b21ac4dbe6#53
for details.

Now, if you want to use more than 5MB of data with indexedDB, you need
to ask the "storage" permission in your manifest. As far as I can tell,
only the system app does that currently. Because of what looks like a
bug in the indexedDB quota management, this has not prevented apps to
work so far, but Ben is fixing that so manifests of relevant apps must
be updated as soon as possible.

thanks!

Fabrice
--
Fabrice Desr�
b2g team
Mozilla Corporation

[Julien Wajsberg]

Will this need to be fixed on all branches, including v1.0.1 ?


Fabrice Desre <fab...@mozilla.com> a écrit :

>Fabrice Desré
>b2g team
>Mozilla Corporation
>_______________________________________________
>dev-gaia mailing list
>dev-...@lists.mozilla.org
>https://lists.mozilla.org/listinfo/dev-gaia

[Fabrice Desre]

On 03/29/2013 05:53 PM, Julien Wajsberg wrote:
> Will this need to be fixed on all branches, including v1.0.1 ?

yes.

[David Flanagan]

I've heard that blobs stored in indexedDB are stored in separate files.
Do those blobs count toward the quota?

If so, Gallery and Music will need the storage permission (for photo and
album art thumbnails). The Video app should probably also have it to be
safe, though users are much less likely to exceed 5mb of video thumbnails.

David

[Fabrice Desre]

On 03/29/2013 08:13 PM, David Flanagan wrote:
> I've heard that blobs stored in indexedDB are stored in separate files.
> Do those blobs count toward the quota?

I'll let Ben answer that.

> If so, Gallery and Music will need the storage permission (for photo and
> album art thumbnails). The Video app should probably also have it to be
> safe, though users are much less likely to exceed 5mb of video thumbnails.

I see no reason to not ask for the permission if you use indexedDB
indeed. We should really be on the safe side.

[David Flanagan]

On 3/29/13 11:59 PM, Ben Turner wrote:
>>> I've heard that blobs stored in indexedDB are stored in separate
>>> files.
>>> Do those blobs count toward the quota?

> Yep!

>
>> I see no reason to not ask for the permission if you use indexedDB

> Basically, if you don't request the "storage" permission you get 5mb of indexedDB space before you'll start getting random quota-error exceptions thrown at you. It's really, really hard to guess how much you can squeeze into 5mb of indexedDB (actually, SQLite) storage, so the rule of thumb should definitely be to request the permission if you use indexedDB whatsoever.
>
> Oh, and if we have any localStorage holdouts, please be warned: One day soon we'll have localStorage tied into the same quota mechanism so localStorage + indexedDB will be subject to the same 5mb limit. It's probably best to go ahead and request the "storage" permission now :)
>
> Happy Hacking,
> -Ben
Do we need to make these permission changes on v1-train, or just on master?

[Fabrice Desre]

On 03/30/2013 12:11 AM, David Flanagan wrote:

> Do we need to make these permission changes on v1-train, or just on master?

We need these changes on master, 1.0.1 and v1-train.

[Ben Turner]

[DANIEL JESUS COLOMA BAIGES]

Please, request tef? on any bug that fixes this issue. I'll mark them as
tef+ so they are uplifted to 1.0.1.

>_______________________________________________
>dev-gaia mailing list
>dev-...@lists.mozilla.org
>https://lists.mozilla.org/listinfo/dev-gaia

________________________________

Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at:
http://www.tid.es/ES/PAGINAS/disclaimer.aspx

[Anthony Chung]

Can someone check me here on what qa needs to know to test this? I'm hearing this global uplift affects core apps that request storage > 5 Mb:

-system
- gallery
- music

And anything exceeding the 5mb will throw quota exceptions in logcat.

Am I missing anything?

Lastly, please drop the bug in here tracking this.

Thanks,
Tony

[Jordano Francisco (UK)]

As Fabrice suggested, we should apply this change to any app using
indexedDB.

In the case of communications app we should add it as long as we are
storing information from FB in indexedDB.
Another app that comes to my mind is cost control.

But as Anthony suggest better to track it on a bug :)

Cheers!
F.

This electronic message contains information from Telefonica UK, Telefonica Europe or Telefonica Digital which may be privileged or confidential. The information is intended to be for the use of the individual(s) or entity named above. If you are not the intended recipient be aware that any disclosure, copying distribution or use of the contents of this information is prohibited. If you have received this electronic message in error, please notify us by telephone or email.


Switchboard: +44 (0)113 272 2000
Email: feed...@o2.com

Telefonica UK Limited 260 Bath Road, Slough, Berkshire SL1 4DX Registered in England and Wales: 1743099. VAT number: GB 778 6037 85
Telefonica Europe plc 260 Bath Road, Slough, Berkshire SL1 4DX Registered in England and Wales: 05310128. VAT number: GB 778 6037 85
Telefonica Digital Limited 260 Bath Road, Slough, Berkshire SL1 4DX Registered in England and Wales: 7884976. VAT number: GB 778 6037 85

[Ben Turner]

And we found the problem first in the Browser app.

[Stefan Arentz]

On 2013-03-30, at 4:39 PM, Anthony Chung <tch...@mozilla.com> wrote:

> Can someone check me here on what qa needs to know to test this? I'm hearing this global uplift affects core apps that request storage > 5 Mb:
>
> -system
> - gallery
> - music
>
> And anything exceeding the 5mb will throw quota exceptions in logcat.
>
> Am I missing anything?

communications, as it stores images for Facebook contacts?

S.

[Tim Chien]

The experimental JSZhuyin IME uses IndexedDB too.

> _______________________________________________
> dev-gaia mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-gaia
>

--
Tim Guan-tin Chien, Senior Front-end Dev., Mozilla Corp. (Taiwan)

[David Flanagan]

On 3/30/13 2:33 PM, Jordano Francisco (UK) wrote:
> As Fabrice suggested, we should apply this change to any app using
> indexedDB.

asyncStorage (shared/js/async_storage.js) is based on indexedDB, so any
app that uses that also uses indexedDB.

>
> In the case of communications app we should add it as long as we are
> storing information from FB in indexedDB.
> Another app that comes to my mind is cost control.
>
> But as Anthony suggest better to track it on a bug :)
>
> Cheers!
> F.
>

> On 30/03/2013 20:39, "Anthony Chung" <tch...@mozilla.com> wrote:
>
>> Can someone check me here on what qa needs to know to test this? I'm
>> hearing this global uplift affects core apps that request storage > 5 Mb:
>>
>> -system
>> - gallery
>> - music
>>
>> And anything exceeding the 5mb will throw quota exceptions in logcat.
>>
>> Am I missing anything?
>>

>> Lastly, please drop the bug in here tracking this.
>>
>> Thanks,
>> Tony
>>
>> On Mar 30, 2013, at 12:22 PM, Ben Turner <btu...@mozilla.com> wrote:
>>
>>>>> I've heard that blobs stored in indexedDB are stored in separate
>>>>> files.
>>>>> Do those blobs count toward the quota?
>>> Yep!
>>>
>>>> I see no reason to not ask for the permission if you use indexedDB
>>> Basically, if you don't request the "storage" permission you get 5mb of
>>> indexedDB space before you'll start getting random quota-error
>>> exceptions thrown at you. It's really, really hard to guess how much you
>>> can squeeze into 5mb of indexedDB (actually, SQLite) storage, so the
>>> rule of thumb should definitely be to request the permission if you use
>>> indexedDB whatsoever.
>>>
>>> Oh, and if we have any localStorage holdouts, please be warned: One day
>>> soon we'll have localStorage tied into the same quota mechanism so
>>> localStorage + indexedDB will be subject to the same 5mb limit. It's
>>> probably best to go ahead and request the "storage" permission now :)
>>>
>>> Happy Hacking,
>>> -Ben

>>> _______________________________________________
>>> dev-gaia mailing list
>>> dev-...@lists.mozilla.org
>>> https://lists.mozilla.org/listinfo/dev-gaia
>> _______________________________________________
>> dev-gaia mailing list
>> dev-...@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-gaia
>

[Mª ANGELES OTEO MARTINEZ]

Bugs for Cost Control https://bugzilla.mozilla.org/show_bug.cgi?id=856557
and Communications https://bugzilla.mozilla.org/show_bug.cgi?id=856538
already resolved and landed in Master. They are also flagged as tef+.

Regards
Maria

El 01/04/13 21:13, "David Flanagan" <dfla...@mozilla.com> escribió:

[David Flanagan]

I've filed bug 856798 to cover all apps that use either of my mediadb or
async_storage libraries.

David

On 4/1/13 1:39 PM, M� ANGELES OTEO MARTINEZ wrote:
> Bugs for Cost Control https://bugzilla.mozilla.org/show_bug.cgi?id=856557
> and Communications https://bugzilla.mozilla.org/show_bug.cgi?id=856538
> already resolved and landed in Master. They are also flagged as tef+.
>
> Regards
> Maria
>

> El 01/04/13 21:13, "David Flanagan" <dfla...@mozilla.com> escribi�:

> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra pol�tica de env�o y recepci�n de correo electr�nico en el enlace situado m�s abajo.

[David Flanagan]

On 4/1/13 2:01 PM, David Flanagan wrote:
> I've filed bug 856798 to cover all apps that use either of my mediadb
> or async_storage libraries.
>

This has been fixed. But it looks like browser, calendar and homescreen
all still need the storage permission, too. Cc'ing Ben, Andrew, and Vivien.

David

The following apps now have the storage permission:

$ grep -l '"storage"' *apps/*/manifest.webapp
apps/camera/manifest.webapp
apps/clock/manifest.webapp
apps/communications/manifest.webapp
apps/costcontrol/manifest.webapp
apps/email/manifest.webapp
apps/fm/manifest.webapp
apps/gallery/manifest.webapp
apps/music/manifest.webapp
apps/settings/manifest.webapp
apps/sms/manifest.webapp
apps/system/manifest.webapp
apps/video/manifest.webapp
test_apps/test-container/manifest.webapp
test_external_apps/mochitest/manifest.webapp

Apps using indexedDB:

$ grep -l indexedDB *apps/*/js/*.js
apps/browser/js/places.js
apps/calendar/js/db.js
apps/calendar/js/mozalarm_shim.js
apps/camera/js/camera.js
apps/clock/js/alarm.js
apps/clock/js/alarmsdb.js
apps/homescreen/js/state.js
apps/system/js/window_manager.js
test_apps/image-uploader/js/credentials_db.js

[Yuan Xulei(袁徐磊)]

The pinyin IME of the keyboard app, though not enabled by default, requires an indexDB more than 5 MB to store IME dictionary.

[David Flanagan]

On 4/1/13 9:21 PM, Yuan Xulei(袁徐磊) wrote:
> The pinyin IME of the keyboard app, though not enabled by default,
> requires an indexDB more than 5 MB to store IME dictionary.

I'm pretty sure that (for now at least) the keyboard runs as part of the
system app. And the system app has the storage permission already, so
we're okay there.

[Mounir Lamouri]

On 30/03/13 00:41, Fabrice Desre wrote:
> Now, if you want to use more than 5MB of data with indexedDB, you need
> to ask the "storage" permission in your manifest. As far as I can tell,
> only the system app does that currently. Because of what looks like a
> bug in the indexedDB quota management, this has not prevented apps to
> work so far, but Ben is fixing that so manifests of relevant apps must
> be updated as soon as possible.

Could be a stupid question but why don't we allow more space to instaled
applications by default compared to regular web content?

--
Mounir

[David Flanagan]

On 4/1/13 4:40 PM, David Flanagan wrote:
> On 4/1/13 2:01 PM, David Flanagan wrote:
>> I've filed bug 856798 to cover all apps that use either of my mediadb
>> or async_storage libraries.
>>
> This has been fixed. But it looks like browser, calendar and
> homescreen all still need the storage permission, too. Cc'ing Ben,
> Andrew, and Vivien.
>
> David

I've filed:

https://bugzilla.mozilla.org/show_bug.cgi?id=857174 for Calendar and
have assigned it to James.
https://bugzilla.mozilla.org/show_bug.cgi?id=857177 for Browser and have
assigned it to Ben
https://bugzilla.mozilla.org/show_bug.cgi?id=857178 for Homescreen and
have assigned it to Cristian

All that needs to be done, I think, is to add "storage":{} to the
permissions list in manifest.webapp

Because these are urgent but trivial bugs to fix, I've taken the liberty
of assigning them to module owners.

Daniel Coloma has already marked these as tef+.

David

[Jonas Sicking]

I want to stick with the general rule of thumb that apps only get the
same permissions as websites unless they enumerate things in the
"permissions" property.

That will soon mean that they get 0MB of permanent storage, but that
by default they use temporary storage. At that point we'll also make
applications that enumerate "storage" in the permissions property will
get unlimited permanent storage and will use permanent storage by
default.

/ Jonas