Binary data in ExternalAsciiStringResource

[ryan dahl]

Hello -

I am trying to store binary in a string by using an
ExternalAsciiStringResource. There is a problem with values over 127.
It seems this might be solved a some cast somewhere inside of V8? Or
is it more complicated than that? The problem is shown with the
attached C++ program.

It seems arbitrary binary data inside a string is possible as
demonstrated by this program:

for (var i = 0; i < 256; i++) {
var s = "'\\" + i.toString(8) + "'";
S = eval(s);
print(s + " " + JSON.stringify(S) + " " + S.charCodeAt(0));
if(S.charCodeAt(0) != i) throw "mismatch";
}

It would be nice to be able to use ExternalAsciiStringResource in this way.

[Erik Corry]

2009/9/8 ryan dahl <coldre...@gmail.com>:

> Hello -
>
> I am trying to store binary in a string by using an
> ExternalAsciiStringResource. There is a problem with values over 127.

You are breaking the contract by putting non-ASCII in an ASCII string.
There are multiple places in the code where we assume that this is
not done.

What you are looking for is Blob support, which is not done:
http://code.google.com/p/v8/issues/detail?id=270

If you know you always have an even number of bytes then you can use
16bit strings. This will probably work fairly well.

> It seems this might be solved a some cast somewhere inside of V8? Or
> is it more complicated than that? The problem is shown with the
> attached C++ program.
>
> It seems arbitrary binary data inside a string is possible as
> demonstrated by this program:
>
>  for (var i = 0; i < 256; i++) {
>  var s = "'\\" + i.toString(8) + "'";
>  S = eval(s);
>  print(s + " " + JSON.stringify(S) + " " + S.charCodeAt(0));
>  if(S.charCodeAt(0) != i) throw "mismatch";
>  }
>
> It would be nice to be able to use ExternalAsciiStringResource in this way.
>
> >
>

--
Erik Corry, Software Engineer
Google Denmark ApS. CVR nr. 28 86 69 84
c/o Philip & Partners, 7 Vognmagergade, P.O. Box 2227, DK-1018
Copenhagen K, Denmark.

[ryan dahl]

On Tue, Sep 8, 2009 at 8:20 PM, Erik Corry<erik....@gmail.com> wrote:
> What you are looking for is Blob support, which is not done:
> http://code.google.com/p/v8/issues/detail?id=270

It seems this would be a simpler way importing binary data into
javascript. Is there any fundamental reason why it can't be done this
way? If so, what would be involved in making an
"ExternalBinaryStringResource" class?

[Erik Corry]

2009/9/9 ryan dahl <coldre...@gmail.com>:

I don't think I can add much to
http://groups.google.com/group/v8-dev/browse_thread/thread/dba28a81d9215291/ece2b50a3b4022c

[ryan dahl]

> I don't think I can add much to
> http://groups.google.com/group/v8-dev/browse_thread/thread/dba28a81d9215291/ece2b50a3b4022c

The difference would be that the object would be inside a string
instead of an opaque object. For things like HTTP upload requests,
where one needs to parse both ascii headers and an arbitrary binary
body it's much nicer to use a string.

Although Christian's Blob proposal would provide another way of
exposing binary data to the user, it seems best suited to be used as
method of allocating space inside V8 for storage of data for external
objects. Which would also be nice but not for HTTP upload requests.

Though it's totally slow, and uses twice the necessary space, I'm
experimenting with something like:

len = recv(fd, buf, 4096, 0); // receive data from a socket
const unsigned char *cbuf = static_cast<const unsigned char*>(buf);
uint16_t twobytebuf[len];
for (size_t i = 0; i < len; i++) {
twobytebuf[i] = cbuf[i];
}
Local<String> chunk = String::New(twobytebuf, len);

Which is, just pack the binary data into the first byte of a uint16_t.
This works nicely for the upload use-case.

It seems to me (and correct me if I'm wrong, please) that
ExternalAsciiStringResource provides a way to give one-byte strings to
V8 without copying data, albeit restricted to 8-bit clean data.
Couldn't something like that work for any byte value?

Thanks