Using Macros for bitwise and bit-shift operations.

[Guillaume Belrose]

Hi all, 

For the past few weeks, I've been working on an Akka-based network application which processes RTP packets. Part of the work involves extracting binary level information (i.e. read 3 bits at offset 9 and put it in variable x, etc..). Although this is not very difficult, you have to write bitwise and bit shift operations. So I came up with something I called the BitWrangler, a set of helper methods to make my life easier. Given a sequence of bytes (stored in an Akka ByteString), you can read a boolean value at a given offset within that sequence, or you can extract an integer value at a given bit offset and with a given length. For example.

import akka.util.ByteString
object SimpleTest extends App {
  import org.kafecho.bitwrangler.macros.BitWrangler
  val data = ByteString(0x40, 0xFF) // Essentially 0010 0000 1111 1111 
  println(BitWrangler.boolean(data, 2)) // Read the 3rd bit value 
  println(BitWrangler.int(data, 7, 3)) // Read the value 011 
}

The BitWrangler is implemented as a set of Scala macros which generate an AST for the bitwise and bit-shift operations based on information known at compile time. 

I've checked in the project in Gibhub (https://github.com/kafecho/BitWrangler) , hopefully it will be useful to some.

Guillaume.

[Paul Phillips]

On Wed, Apr 17, 2013 at 2:47 AM, Guillaume Belrose <kaf...@gmail.com> wrote:

The BitWrangler is implemented as a set of Scala macros which generate an AST for the bitwise and bit-shift operations based on information known at compile time. 

There is currently a value class penalty in that it inhibits constant folding. If this were remedied, I think you wouldn't need to use macros at all. Here is what I mean:

final class Value (val x: Int) extends AnyVal {

  @inline def & (other: Value): Value = new Value(x & other.x)

}

object Foo {

  final val int1   = 0x00111

  final val int2   = 0x11100

  // 0: sipush        273

  // 0: ldc           #20                 // int 69888

  def f1 = int1 & int2

  // 0: sipush        256

  // 3: ireturn

  final val value1 = new Value(0x00111)

  final val value2 = new Value(0x11100)

  // 0: aload_0

  // 1: getfield      #22                 // Field value1:I

  // 0: aload_0

  // 1: getfield      #24                 // Field value2:I

  def f2 = value1 & value2

  //  0: getstatic     #31                 // Field Value$.MODULE$:LValue$;

  //  3: aload_0

  //  4: invokevirtual #33                 // Method value1:()I

  //  7: aload_0

  //  8: invokevirtual #35                 // Method value2:()I

  // 11: invokevirtual #39                 // Method Value$.$amp$extension:(II)I

  // 14: ireturn

}

[martin odersky]

--
You received this message because you are subscribed to the Google Groups "scala-language" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-languag...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

I agree. It would be really nice if value class instances over constants were treated as constants. I recently ran into another restriction: Patterns of value classes are not treated as constants, so matching with guaranteed @switch does not work for them. But to resolve that it will mean we have to promote some of the value class distinctions into typer and pattern matcher.

Cheers

 - Martin

--

Martin Odersky

See you at Scala Days 2013 in NYC!

June 10th - June 12th

www.scaladays.org

[Guillaume Belrose]

@paul, @martin, thanks for the feedback. 

I am not sure I understand your points, so let me ask another question. 

Assuming you have the following:

val data = ByteString(0xFF,0xFF,0xFF)

BitWrangler.int(data,5,19) // Read 19 bits from the 5th bit in the byte string.

The macro, according to the Scala compiler, is expanded as: SimpleExample.this.data.apply(0).$amp(7).$less$less(3).$bar(SimpleExample.this.data.apply(1).$amp(255)).$less$less(8).$bar(SimpleExample.this.data.apply(2).$amp(255)) 

So are you saying that assuming Value classes that support constant foldings, the Scala compiler and the JVM could make the non-macro code as efficient as the macro-generated version? 

Many thanks, 

Guillaume.

[Jeff Olson]

On Wednesday, April 17, 2013 9:39:59 AM UTC-5, Paul Phillips wrote:

There is currently a value class penalty in that it inhibits constant folding. If this were remedied, I think you wouldn't need to use macros at all.

I would love to see this implemented. I've been cooking up a set of macros to work around this problem but direct compiler support would be so much better!

-Jeff

[Paul Phillips]

On Thu, Apr 18, 2013 at 5:11 AM, Guillaume Belrose <kaf...@gmail.com> wrote:

So are you saying that assuming Value classes that support constant foldings, the Scala compiler and the JVM could make the non-macro code as efficient as the macro-generated version? 

Arrays aren't constants and can't be folded (I'm assuming that's what ByteString is) but I think we can do better than the macro version anyway. More importantly we can make it possible to express the operation in a much safer way. But I have no time to elaborate at the moment.