Working with bits and bytes
240 views
Skip to first unread message
Gen
May 1, 2012, 10:11:42 AM5/1/12
to General Dart Discussion
Hi,
as I am trying to implement my own cross platform messaging system,
I would like to have a comfortable and efficient way to handle bits
and bytes.
In particular I would like to have methods for these purposes:
1)
Converting between an integer and a corresponding little endian list
of bytes.
Maybe with an optional ordering of the bits per byte.
2)
Converting between an integer and a corresponding big endian list of
bytes.
Maybe with an optional ordering of the bits per byte.
3)
Converting between a double and 2 integers; the significand and the
exponent.
If possible with a third parameter to indicate the base (.e.g. 2, 8 or
10) of the exponent.
4)
Accessing the bits of a byte using a 0 based offset.
as I am trying to implement my own cross platform messaging system,
I would like to have a comfortable and efficient way to handle bits
and bytes.
In particular I would like to have methods for these purposes:
1)
Converting between an integer and a corresponding little endian list
of bytes.
Maybe with an optional ordering of the bits per byte.
2)
Converting between an integer and a corresponding big endian list of
bytes.
Maybe with an optional ordering of the bits per byte.
3)
Converting between a double and 2 integers; the significand and the
exponent.
If possible with a third parameter to indicate the base (.e.g. 2, 8 or
10) of the exponent.
4)
Accessing the bits of a byte using a 0 based offset.
Message has been deleted
Gen
May 1, 2012, 1:06:57 PM5/1/12
to General Dart Discussion
Does somebody has interest in such methods ?
If not then there will not by any standard implementation of these
methods; I guess.
Hence my question if there are ways to implement these methods using
available or soon available Dart methods ?
If not then there will not by any standard implementation of these
methods; I guess.
Hence my question if there are ways to implement these methods using
available or soon available Dart methods ?
Marcel Batista
May 1, 2012, 1:18:22 PM5/1/12
to Gen, General Dart Discussion
Hey Gen,
Guess you could use one the ArrayBufferView subinterfaces? I didn't see any way of setting/getting bits though...
Gen
May 1, 2012, 1:36:37 PM5/1/12
to General Dart Discussion
Thanks for the hint.
But I believe that the "htlml" library as not available on the server
side.
On 1 Mai, 19:18, Marcel Batista <batista.marc...@gmail.com> wrote:
> Hey Gen,
>
> Guess you could use one the ArrayBufferView subinterfaces? I didn't see any
> way of setting/getting bits though...
>
> http://api.dartlang.org/html/ArrayBufferView.html
>
> Marcel
>
> 2012/5/1 Gen <gp789...@gmail.com>
But I believe that the "htlml" library as not available on the server
side.
On 1 Mai, 19:18, Marcel Batista <batista.marc...@gmail.com> wrote:
> Hey Gen,
>
> Guess you could use one the ArrayBufferView subinterfaces? I didn't see any
> way of setting/getting bits though...
>
> http://api.dartlang.org/html/ArrayBufferView.html
>
> Marcel
>
William Hesse
May 1, 2012, 1:43:10 PM5/1/12
to Marcel Batista, Gen, General Dart Discussion
They can be implemented in Dart using bitwise and arithmetic
operations. If the performance is not good,
a benchmark using them can be submitted to the Dart team, and they
will be improved.
A possible problem is that some shift and bitwise operations have
slightly different semantics when compiled to
JavaScript than when run on the VM.
A good starting point would be to make a library implementing them
using shift (>>), bitwise and (&),
and bitwise or (|). For example, the low byte of x can be extracted with
int b = x & 0xFF;
x = x >> 8.
Note that this may cause problems with negative integers.
--
William Hesse
Software Engineer
whe...@google.com
Google Denmark ApS
Frederiksborggade 20B, 1 sal
1360 København K
Denmark
CVR nr. 28 86 69 84
If you received this communication by mistake, please don't forward it
to anyone else (it may contain confidential or privileged
information), please erase all copies of it, including all
attachments, and please let the sender know it went to the wrong
person. Thanks.
operations. If the performance is not good,
a benchmark using them can be submitted to the Dart team, and they
will be improved.
A possible problem is that some shift and bitwise operations have
slightly different semantics when compiled to
JavaScript than when run on the VM.
A good starting point would be to make a library implementing them
using shift (>>), bitwise and (&),
and bitwise or (|). For example, the low byte of x can be extracted with
int b = x & 0xFF;
x = x >> 8.
Note that this may cause problems with negative integers.
William Hesse
Software Engineer
whe...@google.com
Google Denmark ApS
Frederiksborggade 20B, 1 sal
1360 København K
Denmark
CVR nr. 28 86 69 84
If you received this communication by mistake, please don't forward it
to anyone else (it may contain confidential or privileged
information), please erase all copies of it, including all
attachments, and please let the sender know it went to the wrong
person. Thanks.
Gen
May 1, 2012, 2:08:34 PM5/1/12
to General Dart Discussion
Dart has dynamically sized integers and I guess that programmers
should not assume a certain integer implementation except maybe in
Ansi C.
Do you know the integer value limits within which shift and bitwise
operations are guaranteed to work as expected ?
Furthermore possible semantic differences between client and server
side are quite discouraging.
Would could be the problems ?
What is the problem with negative integes ? The sign bit ?
Are low level operations using bits and bytes considered as intended
part of the Dart platform standard or will they remain hacks for at
least some months ?
On 1 Mai, 19:43, William Hesse <whe...@google.com> wrote:
> They can be implemented in Dart using bitwise and arithmetic
> operations. If the performance is not good,
> a benchmark using them can be submitted to the Dart team, and they
> will be improved.
> A possible problem is that some shift and bitwise operations have
> slightly different semantics when compiled to
> JavaScript than when run on the VM.
>
> A good starting point would be to make a library implementing them
> using shift (>>), bitwise and (&),
> and bitwise or (|). For example, the low byte of x can be extracted with
> int b = x & 0xFF;
> x = x >> 8.
> Note that this may cause problems with negative integers.
>
> On Tue, May 1, 2012 at 7:18 PM, Marcel Batista
>
>
>
>
>
>
>
>
>
> <batista.marc...@gmail.com> wrote:
> > Hey Gen,
>
> > Guess you could use one the ArrayBufferView subinterfaces? I didn't see any
> > way of setting/getting bits though...
>
> >http://api.dartlang.org/html/ArrayBufferView.html
>
> > Marcel
>
> > 2012/5/1 Gen <gp789...@gmail.com>
should not assume a certain integer implementation except maybe in
Ansi C.
Do you know the integer value limits within which shift and bitwise
operations are guaranteed to work as expected ?
Furthermore possible semantic differences between client and server
side are quite discouraging.
Would could be the problems ?
What is the problem with negative integes ? The sign bit ?
Are low level operations using bits and bytes considered as intended
part of the Dart platform standard or will they remain hacks for at
least some months ?
On 1 Mai, 19:43, William Hesse <whe...@google.com> wrote:
> They can be implemented in Dart using bitwise and arithmetic
> operations. If the performance is not good,
> a benchmark using them can be submitted to the Dart team, and they
> will be improved.
> A possible problem is that some shift and bitwise operations have
> slightly different semantics when compiled to
> JavaScript than when run on the VM.
>
> A good starting point would be to make a library implementing them
> using shift (>>), bitwise and (&),
> and bitwise or (|). For example, the low byte of x can be extracted with
> int b = x & 0xFF;
> x = x >> 8.
> Note that this may cause problems with negative integers.
>
> On Tue, May 1, 2012 at 7:18 PM, Marcel Batista
>
>
>
>
>
>
>
>
>
> <batista.marc...@gmail.com> wrote:
> > Hey Gen,
>
> > Guess you could use one the ArrayBufferView subinterfaces? I didn't see any
> > way of setting/getting bits though...
>
> >http://api.dartlang.org/html/ArrayBufferView.html
>
> > Marcel
>
Bob Nystrom
May 4, 2012, 1:33:19 PM5/4/12
to Gen, General Dart Discussion
On Tue, May 1, 2012 at 11:08 AM, Gen <gp78...@gmail.com> wrote:
Do you know the integer value limits within which shift and bitwise
operations are guaranteed to work as expected ?
This is not my area of expertise in the language, but I believe the native VM implementation promises arbitrary-sized integers (limited by your machine's memory, I suppose). The JS implementation compiles down to normal JS numbers, which specify:
8.5 The Number Type
The Number type has exactly 18437736874454810627 (that is, 264−253+3) values, representing the doubleprecision 64-bit format IEEE 754 values as specified in the IEEE Standard for Binary Floating-Point Arithmetic, except that the 9007199254740990 (that is, 253−2) distinct "Not-a-Number" values of the IEEE Standard are represented in ECMAScript as a single special NaN value. (Note that the NaN value is produced by the program expressionNaN.) In some implementations, external code might be able to detect a difference between various Not-a-Number values, but such behaviour is implementation-dependent; to ECMAScript code, all NaN values are indistinguishable from each other.
In other words, I believe you have 53 bits of integer precision.
Furthermore possible semantic differences between client and server
side are quite discouraging.
Would could be the problems ?
The obvious problem is that your code will do different things on client and server. It may overflow and lose precision on JS but not on the native VM. This is definitely a serious area of concern for us and something we talk about frequently. So far, we haven't come down and made a decision about how the problem will be resolved.
What is the problem with negative integes ? The sign bit ?
I'm not sure about this one.
Are low level operations using bits and bytes considered as intended
part of the Dart platform standard or will they remain hacks for at
least some months ?
Yes, you should be able to work at the level of bits and bytes in Dart. Cryptography and working with raw binary byte streams are important parts of web programming. Dart, being fundamentally dynamically typed and not having a range of fixed-precision integer types probably will never be as natural at those tasks as say C or Go, but it should still be doable without too much difficulty.
Cheers!
- bob
Stephen Adams
May 4, 2012, 2:04:03 PM5/4/12
to Bob Nystrom, Gen, General Dart Discussion, Florian Loitsch
In JavaScript, bit operations (& | ^ << >> >>>) are restricted to a 32-bit range.
Usually this range is signed, i.e. [-(2^31), 2^31) but for some operations is unsigned [0, 2^32) ( >>> has an unsigned 32 bit result, e.g. -1 >>> 0 == 4294967295)
The safe range where int values behave properly when compiling to JavaScript would be the intersection of the two, i.e. 31 bit unsigned, [0, 2^31)
We are experimenting with making the safe range [0, 2^32) - this change was recently made to the new dart2js compiler (r2728) but not the old one ('frog') that is currently the default.
Seth Ladd
May 4, 2012, 2:10:36 PM5/4/12
to Stephen Adams, Bob Nystrom, Gen, General Dart Discussion, Florian Loitsch
FYI here's the bug (recently closed) http://code.google.com/p/dart/issues/detail?id=2725
Reply all
Reply to author
Forward
0 new messages