Pre-SIP: Extending case classes to have type members for each field

[Mark Kegel]

When working with shapeless you'll often find yourself referring to the names of fields using Symbol literals. Shapeless does this by implicitly converting a Symbol to an internal type, called Witness, which has a refinement that specifies the Singleton type of the Symbol literal. 

For example, let's say we had a case class Person with the following definition:

case class Person(name:String, occupation:Option[String])

Then we could refer to the fields, assuming we projected our Person class into shapeless using LabelledGeneric, as 'name and 'occupation, respectively. For example, maybe we have a function named swizzle that does something interesting, but requires the field to be an Option:

swizzle[Person]('occupation)

This mechanism is extremely handy and typesafe. For example, if you refer to a field 'foo in the context of a Person, then the code will fail to compile. Alternatively, if Person changes "name" to "fullName", then any code referencing 'name will now fail to compile signaling that code needs to be updated.

While handy, this mechanism is actually too powerful for the problem at hand, and I think the community would benefit if we had access to one that was more targetted. *Please keep in mind: I'm not proposing that shapeless or this mechanism be dropped, only that we get access to second more limited piece of functionality.*

The core issue is that 'foo can refer to any field named "foo" in any case class, when in the examples above, what I really needed was some way to specifically refer to fields within some specific class.

The most appropriate mechanism I can think of for doing this would be to extend case classes so that they have type members with names identical to the fields of the case class. Thus Person would become:

case class Person(name:String, occupation:Option[String]) {

  // Inserted by the compiler...

  type name = ...

  type occupation = ... 

}

Going back to our swizzle function, then we might write:

swizzle[Person#occupation]()

The field reference is now entirely contained within the type signature.

The major benefit this addition would have is that IDEs and other tools would now have a direct link to where in your codebase fields are being referenced "by name", and these symbols can now be linked back to their declarations by the IDE. This also means that refactoring is now safe, so that when you rename "name" to "fullName" your refactoring tools can find and update Person#name to Person#fullName with relative ease. While this might be possible with shapeless, it's ad-hoc.

As to the actual type of each field, they should presumably be some kind of singleton type, whose inhabitant is both instantiable and includes a Lens. Ideally, the types would be constructed so that, for example, swizzle could be written such that it would only accept references to fields in the Person case class, or references to fields of a certain type T, but in any case class, or some combination of these constraints.

I'm looking forward to hearing what folks have to say!

[Alex Cruise]

This reply doesn't address your proposal in a substantial way, but I do want to try to trigger a discussion, since it's a topic I enjoy. 

I really like the idea of method/field references but in order to qualify for a language feature, IMO it needs to be generalized pretty hard.

Long, long ago, on the original Lift wiki, I wrote a proposal called "Scala Record Update Syntax" but last time I looked (a few weeks ago) I was unable to find a surviving copy. 

In any case, it would be good to look at other examples of lens support, and steal from the best and/or compare and contrast. :)

Maybe one of these? 

https://hackage.haskell.org/package/fclabels

https://github.com/julien-truffaut/Monocle

Finally, this topic is probably better addressed to the dotty-internals list rather than scala-debate per se, since the likelihood of new language features appearing over there is dramatically higher than here. :)

-0xe1a

--
You received this message because you are subscribed to the Google Groups "scala-debate" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-debate+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mark Kegel]

Last time I checked the Scala SIP docs said to post to this group first for new potential language changes. You may be correct, though, that I would get more involvement on the dotty internals list.

We could discuss which Lens library to use, but I think that's less critical than just agreeing that whatever the type of the type members is, the inhabitant of that type should have some sort of Lens available. Beyond a Lens, what other pieces of functionality should this inhabitant have? Or maybe Lenses could be provided via implicit resolution of some type class from that inhabitant? Then it could be pluggable that way. Actually if that worked, all the pieces of functionality could be implemented that way...

It was suggested in another private message that this proposal could be prototyped using a macro. I like that idea for getting an initial idea of what kinds of issues might crop up, and certainly we could then argue less about which lens library gets used since it could just be configurable :-)

What would be the long-term pros/cons of having this particular feature built-in vs bolted-on (either via macro or compiler plugin)?

For me, I feel like the types of fields are as fundamental as say TypeTag. I also worry about competing implementations, etc.

Mark

To unsubscribe from this group and stop receiving emails from it, send an email to scala-debate...@googlegroups.com.

[Scala Process]

Hi Mark,

Actually, there are two things here: (a) if you want to include this feature into scalac, this should go into the dedicated mailing list scala-sips; (b) f you want to integrate this feature into Dotty, you should indeed forward it to the dotty mailing list or gitter channel. However, note that your feature depends on the type projection, which is not sound and is going to be removed in Dotty (https://github.com/lampepfl/dotty/issues/1050).

I forwarded your proposal idea to the shapeless gitter channel five days ago, in case Shapeless users have something to say. As you have already introduced your idea here, let's discuss it here and see if more people find this feature interesting. To get more technical people on board, what about writing an email in scala-sips forwarding to this thread?

On a side note, the SIP Committee prefers that features that can be implemented as libraries stay this way. Adding a feature to the compiler has some overhead and important maintenance costs. This is just a preference, of course, and there may be exceptions.

A small tip: if you like your idea and want to have some fun, try to implement it with macros (http://docs.scala-lang.org/overviews/macros/overview.html) or a compiler plugin (a good example is boxer: https://github.com/retronym/boxer). From your small proposal description, I would guess that macros should be enough for this use case.

Generally, if you can convince your fellow Scala developers that your idea is useful in real life, you can surely convince the SIP Committee ;)

Have a nice day!

- Jorge

[Mark Kegel]

Jorge, 

I wanted to gauge reaction to the feature here first, as suggested in the SIP docs the last time I read them. Maybe that advice has changed...

I will forward this suggestion off to the dotty lists and post in the gitter channel. I wouldn't mind having the feature in scalac but I'm not as invested in that possibility.

I was under the impression that type projection would remain, but in a limited capacity (e.g. accessing Java inner classes). The rule was that the prefix must be stable and explicit class names should be stable. Has this changed recently?

As for shapeless, well that was where I first asked publicly about adding this feature. To be honest, Miles is happy with the shapeless solution and no one else really perked up. I still think this is a reasonable feature to add to the language.

I'm not sure I agree that this feature is best implemented as a library/macro. The simple reason is that the whole point of this feature is that it is language wide, so that even when you are interfacing with other people's code (OPC) you don't have to worry about whether or not these type members exist, you just know they will. If you only have your own code annotated that doesn't help much when you are working with a case class outside your control.

I do like adding this feature as a compiler plugin, if that's a possibility. Can already compiled case class definitions be updated to contain these type members? If not, then we're back at the same problems when interfacing with OPC.

Mark

[Scala Process]

Hey Mark,

Yes, the first step is to gauge some attention from the Community, so you did the right thing. I just wanted to give you some early feedback.

To be honest, the status of type projections in Dotty is unclear, but last time I heard they were going to be removed. You would be better off asking in the Gitter channel, but I don't know until which point the Dotty team has already made a decision.

I'm not saying this is best implemented as a library/macro. I was just letting you know that doing so may be a good idea for convincing the SIP Committee, getting some people use it in real life and have some fun. As you have already shown your interest in reaching out the Dotty team, I will not comment on the SIP process nor the Committee anymore. But I want to emphasize that the general feeling is that if a feature can live out of the compiler, people usually lean towards this option.

The way you're describing the feature, you can hardly do it with macros without introducing an annotation that wraps the classes you want to use the type projection on. With a compiler plugin, you could modify the symbols associated with source and binary code, and create the type projections inside them. Your plugin should be running before typer, since your feature wouldn't compile at all without first introducing the type projections in the class definitions. If you go this route, you won't need a concrete annotation, but beware that you will only be able to use untyped trees. If you need some type information, go with macros (that are run while typechecking). Other folks may have better approaches, these are just quick ideas of possible implementations, without putting too much thought into it.

I'm not aware of any fundamental limitation that could hinder your use case, but it may very well exist. As Dotty does not yet have the infrastructure for compiler plugins, you could try implementing it in scalac, just to play around with the idea.

Cheers,

--

      Jorge Vicente Cantero

      Scala Center Engineer, EPFL

You received this message because you are subscribed to a topic in the Google Groups "scala-debate" group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/scala-debate/9txFNIMUt9w/unsubscribe.

To unsubscribe from this group and all its topics, send an email to scala-debate...@googlegroups.com.

[Matthew Pocock]

Hi Mark,

The idea of creating some accessible 'type' to identify a field is IMHO a Very Good Thing. The details are a bit messy, as there is often not obviously a 1:1 mapping between a field type and a field. Once you add in class hierarchies, or abstractions over dumb data by shape, not just inheritance, you need a more general idea of a field-as-type. For example:

sealed trait Foo {

  type Bar

  def value: Bar

}

case class  FooAtInt(value: Int) {

  type Bar = Int

}

case class FooAtString(value: String) {

  type Bar = String

}

You have two fields here called `value`:

  field[value, FooAtInt]: FooAtInt => Int

  field[value, FooAtString]: FooAtString => String

Or alternatively, using lenses, and staging the `value` lookup:

  fieldFor[value].from[FooAtInt] : Lens[FooAtInt, Int].

Not suggesting this is the exactly correct API. But this at least gives you the staging to decouple the field type from the concrete datatype providing that field, and the concrete return type for that field at that data type.

To make this system nice to use, we need to generate all the boilerplate for the field types, and for either asserting or inferring substitutability of field accessors/lenses over datatype families.

Matthew

--
You received this message because you are subscribed to the Google Groups "scala-debate" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scala-debate+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dr Matthew Pocock

Turing ate my hamster LTD

mailto: turingate...@gmail.com

Integrative Bioinformatics Group, School of Computing Science, Newcastle University

mailto: matthew...@ncl.ac.uk

gchat: turingate...@gmail.com

msn: matthew...@yahoo.co.uk

irc.freenode.net: drdozer

skype: matthew.pocock

tel: (0191) 2566550

mob: +447535664143