File System API changes for Chrome 13

8,247 views
Skip to first unread message

Eric Bidelman

unread,
Jun 1, 2011, 8:20:01 PM6/1/11
to Chromium HTML5
This is to give everyone a heads up on changes that have recently gone
into Chromium nightlies that may affect the HTML5 File System API.

These changes are in line with recent updates to the spec
[http://dev.w3.org/2009/dap/file-system/file-dir-sys.html#uniformity-
of-interface],
and will be in Chrome 13.

Here are the important changes:

1) The implementation now obfuscates paths on disk, so you can't see
your filesystem under the profile directory any more. This is a
security-driven change, but also allows us to make the API more
powerful and portable. The files are still there, but their names are
now just consecutive numbers, and the directories aren't visible at
all. To view your directory for debugging purposes, open a window on
[your filesystem object].root.toURL() [e.g.
filesystem:http://www.google.com/temporary/].

All existing user data is automatically moved over into the new system
on the first filesystem access after upgrade. There should be
absolutely no user-visible effect of this, but if anything goes wrong,
please let me know ASAP.

Known issues:

* There is a bug in the directory listing displayed in that all file
sizes show up as 0. If you click on a file, you'll still get the real
contents, though.
* The code that prevents certain illegal moves [moving a directory
into its parent without changing its name, moving a directory into one
of the directories it contains] just does path-based checks. If you
try to copy/move from e.g. the temporary to the persistent filesystem,
it just looks at the path without the filesystem type specifier, and
may disallow otherwise-legal moves. In most cases this can be worked
around with an extra rename.

2) The filesystem API is now case-sensitive [and of course
case-preserving]. Previously it was case-insensitive, but
case-preserving.

The new version of the spec also says that previously-restricted
filenames [e.g. COM1, files ending with spaces, etc] will now be
allowed. That will come, but isn't yet implemented. Expect it in
Chrome 14.

Please don't hesitate to post if you have any questions, or if you see
any odd behavior.

Jordon Wii

unread,
Jun 2, 2011, 11:56:50 PM6/2/11
to Chromium HTML5
Any updates on the quota API? I thought I heard that was going into
Chrome 13 as well.

Kinuko Yasuda

unread,
Jun 3, 2011, 6:41:56 AM6/3/11
to Jordon Wii, Chromium HTML5
Hi Jordon,

Yes we have updates on the quota API in M13.
I think we'll make more proper announcement or some Web page to
explain the API, but here's a quick update.

The quota API is not standardized (yet) and is subject to change, but
in M13 we will be releasing following API:
--------------------------------------------------------------------
Window implements QuotaStorageEnvironment;

[NoInterfaceObject]
interface QuotaStorageEnvironment {
// Note: the attribute name is prefixed by 'webkit'
readonly attribute StorageInfo webkitStorageInfo;
};

interface StorageInfo {
const unsigned short TEMPORARY = 0;
const unsigned short PERSISTENT = 1;

// Queries the current quota and how much data is stored for the host.
void queryUsageAndQuota(
unsigned short storageType,
optional StorageInfoUsageCallback successCallback,
optional StorageInfoErrorCallback errorCallback);


// Requests a new quota. Requesting a larger quota may require user's
// explicit permission via UI prompting / infobar.
void requestQuota(
unsigned short storageType,
unsigned long long newQuotaInBytes,
optional StorageInfoQuotaCallback successCallback,
optional StorageInfoErrorCallback errorCallback);
};

[NoInterfaceObject, Callback=FunctionOnly]
interface StorageInfoUsageCallback {
void handleEvent(unsigned long long currentUsageInBytes,
unsigned long long currentQuotaInBytes);
};

[NoInterfaceObject, Callback=FunctionOnly]
interface StorageInfoQuotaCallback {
void handleEvent(unsigned long long grantedQuotaInBytes);
};

[NoInterfaceObject, Callback=FunctionOnly]
interface StorageInfoErrorCallback {
void handleEvent(DOMCoreException error);
};
--------------------------------------------------------------------

In short, you'll be able to get the current quota and usage for your
origin/host by calling:

webkitStorageInfo.queryUsageAndQuota(
webkitStorageInfo.TEMPORARY, // or PERSISTENT
usageCallback,
errorCallback);

And you'll be able to request a new quota by calling:

webkitStorageInfo.requestQuota(
webkitStorageInfo.TEMPORARY, // or PERSISTENT
quotaCallback,
errorCallback);

The data stored in TEMPORARY filesystem or Web SQL DB are charged as
'TEMPORARY' storage data,
while data stored in PERSISTENT filesystem will be charged as
'PERSISTENT' storage data.
We are going to add appcache and IndexedDB under the same quota
management, but it may not happen in M13.

By default webapps/sites will be given some modest TEMPORARY storage quota and
*zero* PERSISTENT quota (except for apps/extensions that have
'unlimitedStorage' permission).

Any attempts to store more data than the granted quota will fail with
QUOTA_EXCEEDED_ERR.

Webapps can request a larger quota than the current quota but it may
require an explicit user permission
and chrome may show an infoBar for prompting.
(Or the request may be silently rejected and the current quota or
smaller quota might be returned.)

Important note: application data stored in TEMPORARY storage (e.g.
data stored in TEMPORARY
filesystem or in SQL DB) may be deleted at the browser's discretion
when the local disk's available space
is getting tight. When the browser deletes temporary data stored for
an origin it will delete
the entire data for the origin, so that it won't corrupt app data in
an unexpected way.

As a known limitation, the usage reported by the quota API may not
match up precisely with the
size of the user data, as each storage may need some extra bytes to
store their meta data or
there might be some time lag until updates are reflected to the quota API.

Kinuko

> --
> You received this message because you are subscribed to the Google Groups "Chromium HTML5" group.
> To post to this group, send email to chromiu...@chromium.org.
> To unsubscribe from this group, send email to chromium-html...@chromium.org.
> For more options, visit this group at http://groups.google.com/a/chromium.org/group/chromium-html5/?hl=en.
>
>

Jordon Wii

unread,
Jun 3, 2011, 9:27:50 AM6/3/11
to Chromium HTML5
Wow, thanks for all the info Kinuko. Is LocalStorage counted under
either of the Persistent or Temporary quotas, or is it separate?
> > To post to this group, send email to chromium-ht...@chromium.org.
> > To unsubscribe from this group, send email to chromium-html5+unsubscr...@chromium.org.

Kinuko Yasuda

unread,
Jun 3, 2011, 11:07:11 AM6/3/11
to Jordon Wii, Chromium HTML5
Hi Jordon,

On Fri, Jun 3, 2011 at 10:27 PM, Jordon Wii <jord...@gmail.com> wrote:
> Wow, thanks for all the info Kinuko.  Is LocalStorage counted under
> either of the Persistent or Temporary quotas, or is it separate?

No I don't think we have a plan to add LocalStorage under the new
quota management. It will likely keep having 5MB upper limit.

> To post to this group, send email to chromiu...@chromium.org.
> To unsubscribe from this group, send email to chromium-html...@chromium.org.

Paul Irish

unread,
Jun 6, 2011, 5:37:28 PM6/6/11
to Chromium HTML5, Jordon Wii, Kinuko Yasuda
Just wanted to add a few more details (via Michael Nordman) around Quota Management, some overlapping with what Kinuko already said..


Temporary Storage.

For the FileSystem and the SQLDatabase system, there is no longer a 5M limit. Instead there is a shared pool of disk space that sites can collectively consume. A single site can consume up to 20% of the pool. As usage of the TEMP pool approaches the limit for the pool as a whole (1G in M13), least recently used data will be reclaimed. No new api, permissions, or prompts are needed to store resources in the shared TEMP pool. AppCache and IndexedDB will also use the TEMP pool in M14.

There is a new api that allows sites to queryUsageAndQuota() which can be used to see how much of the TEMP pool you're using and return how much you can expect to use in total.

HostedApps and Extensions with 'unlimited' storage rights are not subject the the constraints of the TEMP pool and are not subject to eviction.


Persistent Storage

Only the FileSystem supports PERSISTENT storage. There is a new api that sites should use, requestQuota(), prior to utilizing a PERSISTENT file system. This api puts up an infobar prompt on first use. PERSISTENT storage is not subject to eviction, nor is there a fixed size pool which is drawn from. The queryUsageAndQuota() api can also be used to see how much PERSISTENT data is being used and what quota has been granted.

HostedApps and Extensions with the 'unlimited' storage right are grandfathered in to this as well without user prompts.


TL;DR

M13 - just FileSystem (TEMP & PERSISTENT) and WebSQL (TEMP only) are under QuotaManagment
M14 - AppCache and IndexedDB will be added (TEMP only)
Local Storage will remain capped at 5m and we have no plans to rope into the quota mgmt system. 


_
Paul
Chrome Developer Advocate

Anton Podviaznikov

unread,
Jun 7, 2011, 4:56:17 PM6/7/11
to Chromium HTML5
So IndexedDB will be stored in the Temporary storage?

As I remember data from temporary storage can be removed at any time
by browser. So data in IndexedDB(or whole DB) can be removed by
browser.

Am I correct?

Best regards,

Anton Podviaznikov

On Jun 6, 11:37 pm, Paul Irish <paulir...@google.com> wrote:
> Just wanted to add a few more details (via Michael Nordman) around Quota
> Management, some overlapping with what Kinuko already said..
>
> *Temporary Storage.*
>
> For the FileSystem and the SQLDatabase system, there is no longer a 5M
> limit. Instead there is a shared pool of disk space that sites can
> collectively consume. A single site can consume up to 20% of the pool. As
> usage of the TEMP pool approaches the limit for the pool as a whole (1G in
> M13), least recently used data will be reclaimed. No new api, permissions,
> or prompts are needed to store resources in the shared TEMP pool. AppCache
> and IndexedDB will also use the TEMP pool in M14.
>
> There is a new api that allows sites to queryUsageAndQuota() which can be
> used to see how much of the TEMP pool you're using and return how much you
> can expect to use in total.
>
> HostedApps and Extensions with 'unlimited' storage rights are not subject
> the the constraints of the TEMP pool and are not subject to eviction.
>
> *Persistent Storage*
>
> Only the FileSystem supports PERSISTENT storage. There is a new api that
> sites should use, requestQuota(), prior to utilizing a PERSISTENT file
> system. This api puts up an infobar prompt on first use. PERSISTENT storage
> is not subject to eviction, nor is there a fixed size pool which is drawn
> from. The queryUsageAndQuota() api can also be used to see how much
> PERSISTENT data is being used and what quota has been granted.
>
> HostedApps and Extensions with the 'unlimited' storage right are
> grandfathered in to this as well without user prompts.
> *
> *
> *
> *
> *TL;DR*

Kinuko Yasuda

unread,
Jun 8, 2011, 4:17:47 AM6/8/11
to Anton Podviaznikov, dgr...@chromium.org, Chromium HTML5
+dgrogan, hoping that he could answer better for your question.


On Wed, Jun 8, 2011 at 5:56 AM, Anton Podviaznikov <podvia...@gmail.com> wrote:
So IndexedDB will be stored in the Temporary storage?

As I remember data from temporary storage can be removed at any time
by browser. So data in IndexedDB(or whole DB) can be removed by
browser.

Am I correct?

I thought we're going to / eventually want to have two tiers of storage, temporary (evictable) and permanent for IndexedDB too.
 

Anton Podviaznikov

unread,
Jun 8, 2011, 2:53:06 PM6/8/11
to David Grogan, Kinuko Yasuda, Chromium HTML5
David,
 
Can you please explain why was made decision to put IndexedDB and WebSQL into Temporary storage?
I'm just interesting what was the motivation.

What are the cases for using IndexedDB if it can be deleted any time?
I don't see problem with AppCache because AppCache can be any time retrieved from server in background.

But with this change to IndexedDB and WebSQL it would be much harder to implement few types of applications.

Previously it has no server at all. So IndexedDB was the  only way to store data about songs, artists etc.
And now I need to store these data on server-side and than repopulate IndexedDB when it will be deleted?
This can have very bad impact on user experience because it'll probably will take some time to get all information about all songs from server (or one more time parse that data from music files).

If I understood planning changes correctly it may have very bad impact on some types of apps (in my case it would be even better to move away from IndexedDB).

Just wanted to tell you about impact of such decision and here your thoughts about it.

Best regards,
Anton Podviaznikov


On 8 June 2011 19:38, David Grogan <dgr...@chromium.org> wrote:


On Wed, Jun 8, 2011 at 1:17 AM, Kinuko Yasuda <kin...@chromium.org> wrote:
+dgrogan, hoping that he could answer better for your question.


On Wed, Jun 8, 2011 at 5:56 AM, Anton Podviaznikov <podvia...@gmail.com> wrote:
So IndexedDB will be stored in the Temporary storage?

Yes.
 

As I remember data from temporary storage can be removed at any time
by browser. So data in IndexedDB(or whole DB) can be removed by
browser.
 
Am I correct?

You are.  I think it's important to note that if the quota system does evict your origin's indexeddb data, it will also evict your origin's appcache data, temporary filesystem data, websql data, etc.
 

I thought we're going to / eventually want to have two tiers of storage, temporary (evictable) and permanent for IndexedDB too.

Also correct, though the timeline is unclear.
 




--
Best regards,

Anton Podviaznikov

Michael Nordman

unread,
Jun 8, 2011, 3:14:24 PM6/8/11
to Anton Podviaznikov, David Grogan, Kinuko Yasuda, Chromium HTML5
(Really) This time sending with the 'right' email addr... (really) sorry for the spam.

On Wed, Jun 8, 2011 at 12:06 PM, Michael Nordman <mich...@google.com> wrote:
This is very much a work in progress. The state of affairs in m13 or m14 is not the final resting place for this feature.

On Wed, Jun 8, 2011 at 11:53 AM, Anton Podviaznikov <podvia...@gmail.com> wrote:
David,
 
Can you please explain why was made decision to put IndexedDB and WebSQL into Temporary storage?
I'm just interesting what was the motivation.

We intend to provide a means to create PERSISTENT IndexedDBs and WebSQLDBs in the future. It's just not done yet. For the time being the only storage type that has an API to put data in the PERSISTENT bucket is the FileSystem.
 

What are the cases for using IndexedDB if it can be deleted any time?

Apps+Extensions continue to be protected from deletion via the 'remove all cookies' dialog. Furthermore, Apps+Extensions that set the 'unlimitedStorage' right are also not subject to quota based eviction.

So there are means in place to satisfy a variety of use cases.

 
I don't see problem with AppCache because AppCache can be any time retrieved from server in background.

 
This is not entirely true... what if you happen to be offline?
 
But with this change to IndexedDB and WebSQL it would be much harder to implement few types of applications.
 
Previously it has no server at all. So IndexedDB was the  only way to store data about songs, artists etc.
And now I need to store these data on server-side and than repopulate IndexedDB when it will be deleted?
This can have very bad impact on user experience because it'll probably will take some time to get all information about all songs from server (or one more time parse that data from music files).

If I understood planning changes correctly it may have very bad impact on some types of apps (in my case it would be even better to move away from IndexedDB).

Just wanted to tell you about impact of such decision and here your thoughts about it.
 
I'd suggest relying on the additional privileges granted to HostedApps in the m13/m14 timeframe.

Anton Podviaznikov

unread,
Jun 8, 2011, 3:35:15 PM6/8/11
to Chromium HTML5
Thank you for explanations.

Now I understood. Apps/extensions with 'unlimitedStorage' will not be
affected to these changes. All these changes take affect for regular
web sites/apps that will use AppCache, IndexedDB, WebSQL.

Thank you one more time for clarification.

Best regards,

Anton Podviaznikov

On Jun 8, 9:14 pm, Michael Nordman <micha...@chromium.org> wrote:
> (Really) This time sending with the 'right' email addr... (really) sorry for
> the spam.
>
> On Wed, Jun 8, 2011 at 12:06 PM, Michael Nordman <micha...@google.com>wrote:
>
>
>
>
>
>
>
> > This is very much a work in progress. The state of affairs in m13 or m14 is
> > not the final resting place for this feature.
>
> > On Wed, Jun 8, 2011 at 11:53 AM, Anton Podviaznikov <
> > podviazni...@gmail.com> wrote:
>
> >> David,
>
> >> Can you please explain why was made decision to put IndexedDB and WebSQL
> >> into Temporary storage?
> >> I'm just interesting what was the motivation.
>
> > We intend to provide a means to create PERSISTENT IndexedDBs and WebSQLDBs
> > in the future. It's just not done yet. For the time being the only storage
> > type that has an API to put data in the PERSISTENT bucket is the FileSystem.
>
> >> What are the cases for using IndexedDB if it can be deleted any time?
>
> > Apps+Extensions continue to be protected from deletion via the 'remove all
> > cookies' dialog. Furthermore, Apps+Extensions that set the
> > 'unlimitedStorage' right are also not subject to quota based eviction.
>
> > So there are means in place to satisfy a variety of use cases.
>
> >> I don't see problem with AppCache because AppCache can be any time
> >> retrieved from server in background.
>
> > This is not entirely true... what if you happen to be offline?
>
> >> But with this change to IndexedDB and WebSQL it would be much harder to
> >> implement few types of applications.
>
> > E.x. I created music player (
> >>https://chrome.google.com/webstore/detail/lhjjnnhpfomdbcgnmkejhacbcdp...
> >> ).
> >> Previously it has no server at all. So IndexedDB was the  only way to
> >> store data about songs, artists etc.
> >> And now I need to store these data on server-side and than repopulate
> >> IndexedDB when it will be deleted?
> >> This can have very bad impact on user experience because it'll probably
> >> will take some time to get all information about all songs from server (or
> >> one more time parse that data from music files).
>
> >> If I understood planning changes correctly it may have very bad impact on
> >> some types of apps (in my case it would be even better to move away from
> >> IndexedDB).
>
> >> Just wanted to tell you about impact of such decision and here
> >> your thoughts about it.
>
> > I'd suggest relying on the additional privileges granted to HostedApps in
> > the m13/m14 timeframe.
>
> >> Best regards,
> >> Anton Podviaznikov
>
> >>>>> To post to this group, send email to chromium-ht...@chromium.org.
> >>>>> To unsubscribe from this group, send email to
> >>>>> chromium-html5+unsubscr...@chromium.org.
> >>>>> For more options, visit this group at
> >>>>>http://groups.google.com/a/chromium.org/group/chromium-html5/?hl=en.
>
> >> --
> >> Best regards,
>
> >> Anton Podviaznikov
>
> >>  --
> >> You received this message because you are subscribed to the Google Groups
> >> "Chromium HTML5" group.
> >> To post to this group, send email to chromium-ht...@chromium.org.
> >> To unsubscribe from this group, send email to
> >> chromium-html5+unsubscr...@chromium.org.

David Grogan

unread,
Jun 8, 2011, 3:58:35 PM6/8/11
to Kinuko Yasuda, Anton Podviaznikov, Chromium HTML5
I also sent from the wrong address, hopefully this is the right one.

On Wed, Jun 8, 2011 at 10:38 AM, David Grogan <dgr...@chromium.org> wrote:
On Wed, Jun 8, 2011 at 1:17 AM, Kinuko Yasuda <kin...@chromium.org> wrote:
+dgrogan, hoping that he could answer better for your question.


On Wed, Jun 8, 2011 at 5:56 AM, Anton Podviaznikov <podvia...@gmail.com> wrote:
So IndexedDB will be stored in the Temporary storage?

Yes.
 

As I remember data from temporary storage can be removed at any time
by browser. So data in IndexedDB(or whole DB) can be removed by
browser.
 
Am I correct?
You are.  I think it's important to note that if the quota system does evict your origin's indexeddb data, it will also evict your origin's appcache data, temporary filesystem data, websql data, etc.
 
I thought we're going to / eventually want to have two tiers of storage, temporary (evictable) and permanent for IndexedDB too.
Also correct, though the timeline is unclear.
 
 
Best regards,

Ramin Mahmodi

unread,
Dec 1, 2013, 9:06:27 PM12/1/13
to chromiu...@chromium.org


در پنجشنبه 2 ژوئن 2011، ساعت 4:50:01 (UTC+4:30)، Eric Bidelman نوشته:
کروم ... مارو گرفتی  ها .... اصلآ تو  html5  باز نمیشه ... تا از جاوا میایم  بیرون .. همش  قفل میشه . دیگه  تا html رو نبندیم همه چی  بسته اس  ها  

Jenn Cook

unread,
Oct 17, 2017, 11:15:17 PM10/17/17
to Chromium HTML5
I'm just curious which one am I? M13? M12? M14? I will reiterate...i honestly don't have problems helping you "get noticed", but why can't you stop redirecting and using all my accounts? I can't go into a few sites because my name is flagged. So you guys are the one's doing this, for over 6 months. I still haven't gone "there" yet. Why can't I be an active part instead of the old lady that your team hacked? I could be useful. Just tired of losing my devices.

Jenn

Jenn Cook

unread,
Oct 18, 2017, 2:29:41 AM10/18/17
to Chromium HTML5


On Tuesday, October 17, 2017, Jenn Cook <jennc...@gmail.com> wrote:
I'm just curious which one am I?  M13? M12? M14? I will reiterate...i honestly don't have problems helping you "get noticed", but why can't you stop redirecting and using all my accounts? I can't go into a few sites because my name is flagged. So you guys are the one's doing this, for over 6 months. I still haven't gone "there" yet. Why can't I be an active part instead of the old lady that your team hacked? I could be useful. Just tired of losing my devices.

Jenn

--
You received this message because you are subscribed to the Google Groups "Chromium HTML5" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-html...@chromium.org.

To post to this group, send email to chromiu...@chromium.org.
Visit this group at https://groups.google.com/a/chromium.org/group/chromium-html5/.
For more options, visit https://groups.google.com/a/chromium.org/d/optout.
Reply all
Reply to author
Forward
0 new messages