[ANN] view-buffer

[Brandon Benvie]

This is a cool little tool I've been meaning to make for a while. It combines the features of Buffers, TypedArrays, and DataView into one useful interface by using Proxies to provide an amorphous surface. A ViewBuffer can start out using the same parameters as a regular Buffer, or it can wrap an existing buffer. The primary way to set and get data from it is using the numbered indexes like Buffers have or arrays. The difference is that you can set the "view" that the data is accessed through.

https://github.com/Benvie/view-buffer  

Note: this isn't tested at all and isn't currently targeted at production use. In npm it's set to require node >=0.7. It also requires running with --harmony_proxies and --harmony_collections (or --harmony).

simple creation:

var buffer = ViewBuffer(10);

buffer [3] = 100;

{ view: 'Uint8',

  bytes: 10,

  length: 10,

  '0': 0,

  '1': 0,

  '2': 0,

  '3': 100,

  '4': 0,

  '5': 0,

  '6': 0,

  '7': 0,

  '8': 0,

  '9': 0 }

Here's where we go flipmode

buffer.view = 'uint32';

{ view: 'Uint32',

  bytes: 10,

  length: 2,

  '0': 1677721600,

  '1': 0 }

buffer.view = 'uint16';

{ view: 'Uint16',

  bytes: 10,

  length: 5,

  '0': 0,

  '1': 25600,

  '2': 0,

  '3': 0,

  '4': 0 }

All of the functions that a ViewBuffer has work using its current view to it's basically automated translation. On top of the numeric types there's also some string ones and binary (individual bits). The strings are interesting. When you write a string it'll interpret as ascii and one byte per character by default even if the current view is set to another size.

buffer.write('viewbuffer')

//oops

{ view: 'Uint16',

  bytes: 10,

  length: 5,

  '0': 26998,

  '1': 30565,

  '2': 30050,

  '3': 26214,

  '4': 29285 }

And switching to a string type view automatically translates the values

buffer.view = 'ascii';

{ view: 'Ascii',

  bytes: 10,

  length: 10,

  '0': 'v',

  '1': 'i',

  '2': 'e',

  '3': 'w',

  '4': 'b',

  '5': 'u',

  '6': 'f',

  '7': 'f',

  '8': 'e',

  '9': 'r' }

And with bytesize-agnostic functions that makes things like this work:

var ucs2 = ViewBuffer('ucs2', 20);

buffer.copy(ucs2);

{ view: 'Ucs2',

  bytes: 20,

  length: 10,

  '0': 'v',

  '1': 'i',

  '2': 'e',

  '3': 'w',

  '4': 'b',

  '5': 'u',

  '6': 'f',

  '7': 'f',

  '8': 'e',

  '9': 'r' }

And back to ascii

ucs2.view = 'ascii';

{ view: 'Ascii',

  bytes: 20,

  length: 20,

  '0': 'v',

  '1': '\u0000',

  '2': 'i',

  '3': '\u0000',

  '4': 'e',

  '5': '\u0000',

  '6': 'w',

  '7': '\u0000',

  '8': 'b',

  '9': '\u0000',

  '10': 'u',

  '11': '\u0000',

  '12': 'f',

  '13': '\u0000',

  '14': 'f',

  '15': '\u0000',

  '16': 'e',

  '17': '\u0000',

  '18': 'r',

  '19': '\u0000' }

[Jann Horn]

On Sun, Mar 25, 2012 at 06:57:20AM -0700, Brandon Benvie wrote:
> This is a cool little tool I've been meaning to make for a while. It
> combines the features of Buffers, TypedArrays, and DataView into one useful
> interface by using Proxies to provide an amorphous surface. A ViewBuffer
> can start out using the same parameters as a regular Buffer, or it can wrap
> an existing buffer. The primary way to set and get data from it is using
> the numbered indexes like Buffers have or arrays. The difference is that
> you can set the "view" that the data is accessed through.
>
> https://github.com/Benvie/view-buffer
>

> *Note: this isn't tested at all and isn't currently targeted at production

> use. In npm it's set to require node >=0.7. It also requires running with

> --harmony_proxies and --harmony_collections (or --harmony).*
>
> *simple creation:*

> var buffer = ViewBuffer(10);
> buffer [3] = 100;
> { view: 'Uint8',
> bytes: 10,
> length: 10,
> '0': 0,
> '1': 0,
> '2': 0,
> '3': 100,
> '4': 0,
> '5': 0,
> '6': 0,
> '7': 0,
> '8': 0,
> '9': 0 }
>
> Here's where we go flipmode
> buffer.view = 'uint32';
> { view: 'Uint32',
> bytes: 10,
> length: 2,
> '0': 1677721600,
> '1': 0 }

How fast is it? It sounds relatively inefficient...

Also, what if the 32-bit numbers in your buffer aren't aligned at 4?

[Brandon Benvie]

It uses a DataView for actual data manipulation which does aligned writing. I don't have stats, I'll get some now that I've settled on a better implementation of the property dispatcher. Reading and writing itself should be as fast as a DataView is, there might be a small overhead in the dispatch of indexes but I'm not sure.

I'm also considering switching to an implementation that will use instances of the specific ArrayBuffer type since they have seen more optimization than DataView has, but they have no built in support for aligned writing and simply use system endianness so it might not be a net gain in many cases. It should at least be worth it for 1 byte types though.

[Brandon Benvie]

I just updated it so that there shouldn't be any problems on indexed lookup either now. I'll see about getting some performance stats.

[Brandon Benvie]

Oh I just reread your question. The length displayed is rounded down to the nearest whole unit displayable. If you have a 4 byte type and 2 bytes left then it will not show those two bytes, and it will have a property "excess" set to 2, so you know there is some not showing.