first class byte literals (no hints required)

76 views
Skip to first unread message

Doug Tangren

unread,
Dec 28, 2011, 11:22:53 PM12/28/11
to scala-i...@googlegroups.com
I'm not sure it this is the right list to be posting to but I found myself this evening hankering for a byte literal in scala that didn't need type hints to tell the compiler I want a byte, i.e. 

val b: Byte = 1

or 

val b = 1.toByte

I wanted to get some thoughts appending to section 1.3.1 of http://www.scala-lang.org/docu/files/ScalaReference.pdf (Numeric literals) with a new literal specifically for bytes. Something like

byteNumeral ::= ‘0’ ‘b’ hexDigit {hexDigit}

I think there's something like this in java7 and I know I've seen talk of it in other languages [1] too so it's not that off the rails of an idea.

Thanks to the the wonders of github, I have now xray specs into what's going on under the covers in scala's compiler and, if one were to add this byte literal type to the compiler, it looks like it would happen round around these parts


https://github.com/scala/scala/blob/master/src/compiler/scala/tools/nsc/ast/parser/Scanners.scala#L847-915

I don't think I have britches big enough to fork scala and try this myself yet, but I wanted to get some commentary from the list on the general idea first.

Is having an explicit literal for bytes worth implementing or is it too rare of a usecase? What brought this up was defining constants in binary protocols, the memcache binary protocol [2] in this instance. Having a byte literal for these constants instead of lots of n.toByte and :Byte = ... typing would make my fingers happier patrons of scala's campground.


-Doug Tangren
http://lessis.me

Erik Osheim

unread,
Dec 28, 2011, 11:54:38 PM12/28/11
to scala-i...@googlegroups.com
On Wed, Dec 28, 2011 at 11:22:53PM -0500, Doug Tangren wrote:
> I don't think I have britches big enough to fork scala and try this myself
> yet, but I wanted to get some commentary from the list on the general idea
> first.
>
> Is having an explicit literal for bytes worth implementing or is it too
> rare of a usecase? What brought this up was defining constants in binary
> protocols, the memcache binary protocol [2] in this instance. Having a byte
> literal for these constants instead of lots of n.toByte and :Byte = ...
> typing would make my fingers happier patrons of scala's campground.

Hi Doug,

Many people have wished for nicer syntax for specifying Byte and Short
literals (I know I have). And I don't think the relevant changes to the
compiler are that hard--I'm confident that I or someone else could
create a patch to add them.

But the last time that I remember someone (Paul Phillips) bringing up a
new literal syntax for Byte/Short Martin was strongly against it. See
this email thread:

http://thread.gmane.org/gmane.comp.lang.scala.internals/1858/focus=1867

I'm not saying you shouldn't push for this, but be aware that it has
been discussed before.

-- Erik

P.S. If anyone wants to chime in about how nice a literal syntax for
Byte/Short would be, and/or how much the current numeric coercions
suck, please feel free. :)

Paul Phillips

unread,
Dec 28, 2011, 11:58:46 PM12/28/11
to scala-i...@googlegroups.com


On Wed, Dec 28, 2011 at 8:54 PM, Erik Osheim <er...@plastic-idolatry.com> wrote:
But the last time that I remember someone (Paul Phillips) bringing up a
new literal syntax for Byte/Short Martin was strongly against it.

That's true, and he still would be I expect, but the context was a million miles away: at the time we were still on the crazy train of "5l != 5" and I was trying to find a way to avoid that world (such as making different numerics incomparable, which made literal syntax seem more urgent.)

Paul Phillips

unread,
Dec 29, 2011, 12:02:36 AM12/29/11
to scala-i...@googlegroups.com
On Wed, Dec 28, 2011 at 8:22 PM, Doug Tangren <d.ta...@gmail.com> wrote:
I'm not sure it this is the right list to be posting to but I found myself this evening hankering for a byte literal in scala that didn't need type hints to tell the compiler I want a byte, i.e. 

val b: Byte = 1

One thing you can do is:

scala> def b(x: Byte): Byte = x
b: (x: Byte)Byte

scala> val b0 = b(55)
b0: Byte = 55

I wanted to get some thoughts appending to section 1.3.1 of http://www.scala-lang.org/docu/files/ScalaReference.pdf (Numeric literals) with a new literal specifically for bytes.

Unless martin pulls one of his occasional reversals, it's unlikely.

I don't think I have britches big enough to fork scala and try this myself yet, but I wanted to get some commentary from the list on the general idea first.

The code part would be very easy for you once you got past the compiler intimidation.   But the code part usually isn't the hard part.

Doug Tangren

unread,
Dec 29, 2011, 12:08:34 AM12/29/11
to scala-i...@googlegroups.com
in my case I can't do something like

val b = 0x81

and compare that with a value of 0x81 fetched from a java.nio.ByteBuffer. I have the same issue you hinted at above because my 0x81 != to the ByteBuffer's 0x81. Mine is really 129 (the int value), not the byte value of -127 until an add explicit type annotations or call toByte on it. Runtime debugging whoo!


Doug Tangren

unread,
Dec 29, 2011, 12:10:39 AM12/29/11
to scala-i...@googlegroups.com

-Doug Tangren
http://lessis.me


On Thu, Dec 29, 2011 at 12:02 AM, Paul Phillips <pa...@improving.org> wrote:


On Wed, Dec 28, 2011 at 8:22 PM, Doug Tangren <d.ta...@gmail.com> wrote:
I'm not sure it this is the right list to be posting to but I found myself this evening hankering for a byte literal in scala that didn't need type hints to tell the compiler I want a byte, i.e. 

val b: Byte = 1

One thing you can do is:

scala> def b(x: Byte): Byte = x
b: (x: Byte)Byte

scala> val b0 = b(55)
b0: Byte = 55


Thanks. This will save me some keystrokes but it feels like just another form of hinting at the numeric type I want.

Erik Osheim

unread,
Dec 29, 2011, 12:11:50 AM12/29/11
to scala-i...@googlegroups.com
On Wed, Dec 28, 2011 at 09:02:36PM -0800, Paul Phillips wrote:
> One thing you can do is:
>
> scala> def b(x: Byte): Byte = x
> b: (x: Byte)Byte

Similarly, you can (ab)use implicits to get something a bit closer to
the 13b syntax with:

scala> implicit def xyz(i:Int) = new { def b = i.toByte; def s = i.toShort }
xyz: (i: Int)java.lang.Object{def b: Byte; def s: Short}

scala> (1.b, 2.s, 3)
res0: (Byte, Short, Int) = (1,2,3)

If you don't sweat the implicits or the object allocations then this
almost starts to look nice...

-- Erik

Paul Phillips

unread,
Dec 29, 2011, 12:28:53 AM12/29/11
to scala-i...@googlegroups.com


On Wed, Dec 28, 2011 at 9:11 PM, Erik Osheim <er...@plastic-idolatry.com> wrote:
 scala> implicit def xyz(i:Int) = new { def b = i.toByte; def s = i.toShort }
 xyz: (i: Int)java.lang.Object{def b: Byte; def s: Short}

The key difference being that my version will only accept valid bytes, whereas this one will happily chop off 24 bits. (Which may well be what is desired, but it bears pointing out.)

scala> def b(x: Byte): Byte = x
b: (x: Byte)Byte

scala> b(5000)
<console>:10: error: type mismatch;
 found   : Int(5000)
 required: Byte
              b(5000)
                ^

scala> 5000.b
res6: Byte = -120

Doug Tangren

unread,
Dec 29, 2011, 1:12:09 AM12/29/11
to scala-i...@googlegroups.com
Thanks guys.

Would it be overkill to go through the sip processes [1]? That may give more people a chance to weigh in without the idea getting buried on a mailing list. 




-Doug Tangren
http://lessis.me

Todd Vierling

unread,
Dec 29, 2011, 1:07:58 PM12/29/11
to scala-i...@googlegroups.com
On Thursday, December 29, 2011 1:12:09 AM UTC-5, softprops wrote:
Would it be overkill to go through the sip processes [1]?

That would definitely be the best way to go about it.

Prior art: There isn't such a syntax in jdk7 AFAIK; the latest proposal for byte/short literals dates to 2009 (though the gdocs link in this post still works): http://mail.openjdk.java.net/pipermail/coin-dev/2009-March/000905.html

That said, I do find the ideal of byte/short literals a little "weird". The reason I actually prefer explicit type hinting is that Byte and Short types are somewhat strange at the JVM level compared to Int and Long. (See the "scala numeric meta-sip" thread over on scala-language for details.)

Jack Viers

unread,
Dec 30, 2011, 8:04:49 AM12/30/11
to scala-internals
What is wrong with x.asInstanceOf[Byte]?

√iktor Ҡlang

unread,
Dec 30, 2011, 8:16:29 AM12/30/11
to scala-i...@googlegroups.com
On Fri, Dec 30, 2011 at 2:04 PM, Jack Viers <jack....@t8webware.com> wrote:
What is wrong with x.asInstanceOf[Byte]?

Casting in Scala is like cursing in Church. ;-)
 



--
Viktor Klang

Akka Tech Lead
Typesafe - Enterprise-Grade Scala from the Experts

Twitter: @viktorklang

Paul Phillips

unread,
Dec 30, 2011, 3:01:18 PM12/30/11
to scala-i...@googlegroups.com


2011/12/30 √iktor Ҡlang <viktor...@gmail.com>
On Fri, Dec 30, 2011 at 2:04 PM, Jack Viers <jack....@t8webware.com> wrote:
What is wrong with x.asInstanceOf[Byte]?

Casting in Scala is like cursing in Church. ;-)

asInstanceOf on primitives doesn't even rise to the level of cursing in church.

scala> 5.asInstanceOf[Long]
res0: Long = 5

scala> (5: AnyVal).asInstanceOf[Long]
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
	at scala.runtime.BoxesRunTime.unboxToLong(Unknown Source)
	at .<init>(<console>:8)

scala> class A1 { def f[T <: Int](x: T) = x.toLong }                    
defined class A1

scala> class A2 { def f[T <: Int](x: T) = x.asInstanceOf[Long] }
defined class A2

scala> (new A1) f 5
res1: Long = 5

scala> (new A2) f 5
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
	at scala.runtime.BoxesRunTime.unboxToLong(Unknown Source)

scala> class A3[T] { def f(x: T) = x.asInstanceOf[Long] }
defined class A3

scala> class A4[@specialized T] { def f(x: T) = x.asInstanceOf[Long] }
defined class A4

scala> (new A3[Int]) f 5
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
	at scala.runtime.BoxesRunTime.unboxToLong(Unknown Source)
scala> (new A4[Int]) f 5
res1: Long = 5

√iktor Ҡlang

unread,
Dec 30, 2011, 3:08:05 PM12/30/11
to scala-i...@googlegroups.com
On Fri, Dec 30, 2011 at 9:01 PM, Paul Phillips <pa...@improving.org> wrote:


2011/12/30 √iktor Ҡlang <viktor...@gmail.com>
On Fri, Dec 30, 2011 at 2:04 PM, Jack Viers <jack....@t8webware.com> wrote:
What is wrong with x.asInstanceOf[Byte]?

Casting in Scala is like cursing in Church. ;-)

asInstanceOf on primitives doesn't even rise to the level of cursing in church.

Schrödinger told us of the dangers of putting things into boxes.
 

Eugene Burmako

unread,
Jan 1, 2012, 8:49:23 AM1/1/12
to scala-internals
Hi guys! Martin's recent string interpolation proposal provides
generic syntax for various external domain-specific languages:
https://docs.google.com/document/d/1NdxNxZYodPA-c4MLr33KzwzKFkzm9iW9POexT9PkJsU/edit?hl=en_US.

The proposal mentions using this syntax to implement string
interpolation (s"hello, $s") and code quasiquotations (c"$x + $y").
However, the domain of possibilities is much wider. For example, we
can write regexes and not struggle with escapes (since quasiquotes do
not interpret escape characters). Also, one could try to embed HTML
and JS with deep integration into the host Scala program (e.g.
transparent, but typesafe references to variables in lexical scope).

The very same approach can be used to implement pretty syntax for byte
literals. For example, b"0a" would stand for 10 of type byte.

In case you wonder about performance implications. There won't be any
if you use a macro to implement the "b" quasiquote provider. Here's
how it works. This macro will take some representation of a quasiquote
(say, a tree that contains a constructor of new Quasiquote("0a")).
Then during macro expansion, it will extract the "0a" string literal
from the AST, parse it and emit the AST you need, e.g.
Select(Literal(Constant(10)), newTermName("toByte")). The resulting
AST will be inlined into the callsite, so there won't be any overhead
in comparison with writing the code manually.

Exact details of the quasiquoting API are still in flux. This is
something I've been actively working during the last week, and in a
few days I'm going to present a prototype and some refinements to the
SID. Stay tuned.

On Dec 30 2011, 9:08 pm, √iktor Ҡlang <viktor.kl...@gmail.com> wrote:
> On Fri, Dec 30, 2011 at 9:01 PM, Paul Phillips <pa...@improving.org> wrote:
>
> > 2011/12/30 √iktor Ҡlang <viktor.kl...@gmail.com>
>
> >> On Fri, Dec 30, 2011 at 2:04 PM, Jack Viers <jack.vi...@t8webware.com>wrote:
>
> >>> What is wrong with x.asInstanceOf[Byte]?
>
> >> Casting in Scala is like cursing in Church. ;-)
>
> Typesafe <http://www.typesafe.com/> - Enterprise-Grade Scala from the
> Experts
>
> Twitter: @viktorklang

Jon Steelman

unread,
Jan 1, 2012, 3:01:27 PM1/1/12
to scala-i...@googlegroups.com
Hi Eugene,

Does this mean the proposal will support multiple levels of nesting?

Thanks,
Jon

Eugene Burmako

unread,
Jan 1, 2012, 5:41:29 PM1/1/12
to scala-internals
Hi Jon,

In my view, there are no conceptual obstacles to allowing multiple
levels of quasiquotes (just a few days ago we had a discussion about
multi-level code quasiquotes). However, there are some syntactic trade-
offs to be made.

Basically there are two design options w.r.t. nesting: 1) allow nested
quotes in single-line quotes (e.g. s"foo${s"bar"}"), 2) only allow
nested quotes in multi-line quotes (e.g. s"""foo${s"bar"}"""). The
latter is more in line with the look of vanilla string literals,
however the former supports arbitrary levels of nesting.

At the moment sorting this out is more of a low-priority task, since
some fundamental thingies (e.g. matching against code quasiquotes)
aren't yet in place. However, nesting is definitely on my task list
(and I even wrote a unit test for that =)).

Cheers,
Eugene

On Jan 1, 9:01 pm, Jon Steelman <jon.steel...@gmail.com> wrote:
> Hi Eugene,
>
> Does this mean the proposal will support multiple levels of nesting?
>
> Thanks,
> Jon
>
>
>
> On Sun, Jan 1, 2012 at 8:49 AM, Eugene Burmako <xeno...@gmail.com> wrote:
> > Hi guys! Martin's recent string interpolation proposal provides
> > generic syntax for various external domain-specific languages:
>
> >https://docs.google.com/document/d/1NdxNxZYodPA-c4MLr33KzwzKFkzm9iW9P...
> > .

Doug Tangren

unread,
Jan 1, 2012, 6:28:38 PM1/1/12
to scala-i...@googlegroups.com
On Sun, Jan 1, 2012 at 5:41 PM, Eugene Burmako <xen...@gmail.com> wrote:
Hi Jon,

In my view, there are no conceptual obstacles to allowing multiple
levels of quasiquotes (just a few days ago we had a discussion about
multi-level code quasiquotes). However, there are some syntactic trade-
offs to be made.

Basically there are two design options w.r.t. nesting: 1) allow nested
quotes in single-line quotes (e.g. s"foo${s"bar"}"), 2) only allow
nested quotes in multi-line quotes (e.g. s"""foo${s"bar"}"""). The
latter is more in line with the look of vanilla string literals,
however the former supports arbitrary levels of nesting.

At the moment sorting this out is more of a low-priority task, since
some fundamental thingies (e.g. matching against code quasiquotes)
aren't yet in place. However, nesting is definitely on my task list
(and I even wrote a unit test for that =)).



Thanks for following up with the macro pointers guys. I'm not very familiar with macros in scala. Is http://scalamacros.org/ a good place to start looking? It looks interesting. How does it differ from a compiler plugin where you can already manipulate streams of terms?

Eugene Burmako

unread,
Jan 1, 2012, 7:44:08 PM1/1/12
to scala-internals
Yep, so far scalamacros.org is the only resource on macros for Scala.
There's not so much content, but the main points are covered. After I
defense my semester project I'm looking forward to updating the site
with the info about recent developments.

Speaking of the difference between macros and compiler plugins. Macros
are less powerful (macro expansion only happens in Typers, there is no
way to customize the compilation pipeline), but are much more
lightweight (writing a "macro def" function is all that it takes to
write a macro).

On Jan 2, 12:28 am, Doug Tangren <d.tang...@gmail.com> wrote:
> On Sun, Jan 1, 2012 at 5:41 PM, Eugene Burmako <xeno...@gmail.com> wrote:
> > Hi Jon,
>
> > In my view, there are no conceptual obstacles to allowing multiple
> > levels of quasiquotes (just a few days ago we had a discussion about
> > multi-level code quasiquotes). However, there are some syntactic trade-
> > offs to be made.
>
> > Basically there are two design options w.r.t. nesting: 1) allow nested
> > quotes in single-line quotes (e.g. s"foo${s"bar"}"), 2) only allow
> > nested quotes in multi-line quotes (e.g. s"""foo${s"bar"}"""). The
> > latter is more in line with the look of vanilla string literals,
> > however the former supports arbitrary levels of nesting.
>
> > At the moment sorting this out is more of a low-priority task, since
> > some fundamental thingies (e.g. matching against code quasiquotes)
> > aren't yet in place. However, nesting is definitely on my task list
> > (and I even wrote a unit test for that =)).
>
> Thanks for following up with the macro pointers guys. I'm not very familiar
> with macros in scala. Ishttp://scalamacros.org/a good place to start
Reply all
Reply to author
Forward
0 new messages