Fwd: Proposal : Making Local Storage quota value configurable

[santosh mahto]

Hi chromium-dev, blink-dev

In chromium the value of local storage quota per origin is hard coded to 10 *1024 * 1024 (10MB)

(https://code.google.com/p/chromium/codesearch#chromium/src/content/common/dom_storage/dom_storage_types.h&l=22)

               const size_t kPerStorageAreaQuota = 10 * 1024 * 1024; 

In our case(low end device)  we want to limit the quota per origin due to storage limitation to avoid overflow.

So I would like to make it configurable ( either by content api or comamnd line arg) if agreed upon.

So I  would like to know view upon this proposal.

Regards

sam...@cisco.com

[Michael Nordman]

It's already smallish, you want to make it smaller? What did you have in mind, a flag to set it down to 5M from 10M per origin? 

I think there is blink/content api to selectively allow/disallow access to localstorage for a particular origin (WebContentSettingsClient::allowStorage()). Chrome has that wired up to the it's CookieSetting:: IsSettingCookieAllowed. Would allowing all but a select few work for you?

Logic to evict stale unused data might hlep. Coalescing the <n> sqlite databases into a single leveldb could get rid of some overhead too, each empty localstorge db takes around 3K bytes. Would a cap on total available for use system wide make sense for your use case?  These would be larger changes though.

[Santosh]

The idea here is to make quota per origin value configurable, I mean we can set value using command line arg or may be using some content API.

What my concern is with 10M/5M, its not hard to  overflow the disk(use http://www.filldisk.com/) space in DESKTOP even. So low end devices could

be very easy target where these scenario are not guaranteed to be gracefully handled. 

If it could be set using command line/ content Api to some value( decided based on device storage capacity) then we can provide protection & stability to 

low end device to large extent.

I can provide the patch  if agreement happens on this.

[Michael Nordman]

So the use case is general purpose web browser on a device with limited resources. How much storage is on the device? I don't think changing the limit from 10 to 5 really solves the problem. Limits based on eTLD might be a better way to address the filldisk.com issue specifically. With proactive eviction of lru data, we could put a configurable upper bound on how much space is used for localStorage in aggregate.

[Chris Palmer]

On Fri, Oct 16, 2015 at 6:28 PM, Michael Nordman <mich...@chromium.org> wrote:

So the use case is general purpose web browser on a device with limited resources. How much storage is on the device? I don't think changing the limit from 10 to 5 really solves the problem. Limits based on eTLD might be a better way to address the filldisk.com issue specifically. With proactive eviction of lru data, we could put a configurable upper bound on how much space is used for localStorage in aggregate.

I agree with michaeln.

[Santosh]

I  was planning to implement local storage parameter configurable and also handle the lightweight LRU cleanup. 

I could see some issue related to disk overflow, So I need to know that if any work has been done in this area or not.

And it would be great if I would be prior informed of any design decision, policy, etc.

As an Implementation I am going by  overriding LocalStorageDatabaseAdapter  to create new database adapter which will take care of  validation and cleanup. XXLocalStorageDatabaseAdapter will keep track of current usage by db files.

I will upon design doc for review once ready.

Thanks

santosh

[Michael Nordman]

Hi Santosh,

If you're interested in contributing code, great!

Nothing's actively going on that you don't see in the tree or in CLs under review. I'm not sure I follow the reasoning behind putting this logic in the 'adapter' class, that class is pure shimware and is lacking knowlege of more than one origin's data?

> And it would be great if I would be prior informed of any design decision, policy, etc.

One thing to be mindful of is the latent desire to replace the backing store for local storage. Currently, each origin's data is kept in a separate sqlite db. The latent desire is...

1) to consolidate the multitude of dbs into one (more space efficient)

2) to switch from sqlite to leveldb (to reduce our dependency of that lib)

... making it more like session storage which uses a single leveldb per storage partition.

Logic to prune based on dates and storage pressure should be agnostic with regard to the backing store. So heads up... the LocalStrorageAdapter class might not be the best place for it since that class sole purpose in life is to abstract away the differences between local vs session storage backends.