IndexedDB Blob "brain transplants" to magically convert memory-backed Blobs to IndexedDB-file-backed Blobs without re-reading db after write?

[Andrew Sutherland]

It's my understanding that our current Blob/File implementation is both
conceptually and implementation-wise immutable.

That is, if I do the following, I create a memory-backed Blob:
var hugeArray = new Uint8Array(128 * 1024 * 1024);
var memoryBackedBlob = new Blob([hugeArray]);

We are able to persist to disk via DeviceStorage:
navigator.getDeviceStorage('sdcard').addNamed(memoryBackedBlob,
'file.big'); // => DOMRequest

And IndexedDB:
trans.objectStore('store').put({ myBlob: memoryBackedBlob });

But in both cases, the existing handles to memoryBackedBlob will remain
memory-backed, despite the availability of an exact duplicate of the
data in the reference-counted IndexedDB Blob-handling subsystem that can
provide the same semantics. (Note that from a resource-perspective,
although the spec provides for blob.close(), we do not yet implement it:
https://bugzilla.mozilla.org/show_bug.cgi?id=1048325. Also, I've
proposed allowing Blobs to take ownership of ArrayBuffers on
https://www.w3.org/Bugs/Public/show_bug.cgi?id=28496 but I think it may
not have needinfo nags, since :sicking has never responded.)

Could we consider having IndexedDB brain-transplant the backing store
for the Blob (someday)? I ask this because I'm writing new
IndexedDB-using code to do a write followed by an immediate read-back
again, and it seems to me that this is the behaviour that everyone
wants. And I expect it's also the behaviour that is either explicitly
or implicitly assumed. (I doubt most people think about it.)

The semantics seem clear-cut for IndexedDB. DeviceStorage is a little
trickier unless we assume a linux-style FS with our File implementation
holding an open file handle so that even if the file is deleted the data
continues to exist. (Although arguably if we ensure that new
Blob([memoryBackedBlob]) prior to the save does not experience a
brain-transplant, then the deleted-DeviceStorage case could be viewed as
poor hygiene where some other code with a reference to memoryBackedBlob
called close() on it.)

I realize from a complexity perspective, this is not entirely trivial.
I believe the benefit outweighs the cost for lower-memory mobile
devices, although I also believe this would be an enhancement for down
the road once existing standards are implemented and stable.

The outcome I'm looking for here is "yay, file the enhancement bug",
"nay, there are spec reasons this doesn't work and this is a bad idea",
or "yay, good idea, file an enhancement bug, but we also need to fix the
spec first too."

Andrew

[Jonas Sicking]

On Fri, Jun 5, 2015 at 2:09 PM, Andrew Sutherland
<asuth...@asutherland.org> wrote:
> It's my understanding that our current Blob/File implementation is both
> conceptually and implementation-wise immutable.
>
> That is, if I do the following, I create a memory-backed Blob:
> var hugeArray = new Uint8Array(128 * 1024 * 1024);
> var memoryBackedBlob = new Blob([hugeArray]);
>
> We are able to persist to disk via DeviceStorage:
> navigator.getDeviceStorage('sdcard').addNamed(memoryBackedBlob, 'file.big');
> // => DOMRequest
>
> And IndexedDB:
> trans.objectStore('store').put({ myBlob: memoryBackedBlob });
>
> But in both cases, the existing handles to memoryBackedBlob will remain
> memory-backed, despite the availability of an exact duplicate of the data in
> the reference-counted IndexedDB Blob-handling subsystem that can provide the
> same semantics. (Note that from a resource-perspective, although the spec
> provides for blob.close(), we do not yet implement it:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1048325.

I hope this is not the only reason that this is needed? Surely
implementing .close() is no harder than implementing the other thing
you are asking for here.

Have you reached out to the platform team about getting .close()
implemented? It should be quite trivial.

> Also, I've proposed
> allowing Blobs to take ownership of ArrayBuffers on
> https://www.w3.org/Bugs/Public/show_bug.cgi?id=28496 but I think it may not
> have needinfo nags, since :sicking has never responded.)

I commented there. This seems like a no-brainer to me as long as we
can come up with a decent syntax. Can you propose a good syntax? I bet
we can get it quickly added to the Blob spec.

> Could we consider having IndexedDB brain-transplant the backing store for
> the Blob (someday)?

For writing stuff to IndexedDB I would imagine this wouldn't be too
much work. Though not trivial. We already have support for keeping
Blob data around if there are JS Blob objects referencing it, even if
the data is removed from the database.

For DeviceStorage this is harder. Not impossible but we'd have to
create some form of copy-on-write scheme.

But yes, definitely file a bug for IndexedDB. I'll defer to dhylands
regarding what we should do, if anything, for the DeviceStorage spec.

/ Jonas

[Andrea Marchesini]

>
> Have you reached out to the platform team about getting .close()
> implemented? It should be quite trivial.

I had a working patch for blob.close() and I think bug 1048325 is still
assigned to me.
I'm happy to propose a new patch soon if we agree we want to have this
implemented.

b

[Jonas Sicking]

We definitely want .close() implemented. I didn't realize there were
arguments against?

/ Jonas