Is DRY code generation from SQL db via macros a pipe dream or a real hope?

[Marc Siegel]

Hi Stephan and other SLICK folks,

I just watched this video of the ScalaDays talk on the future of Scala macros:

http://scalamacros.org/news/2013/05/18/half-a-year-in-macro-paradise.html

Clearly, there is a potential future where SLICK may evolve one of the key

features of Rails, which attracted so many followers and attention in the 00's:

a DRY mapping between code models and SQL schema.

Is it a pipe dream to imagine that SLICK may evolve to automatically generate

table classes and row case classes at compile time via macros?

It seems like an extremely compelling feature for the Play framework -- one

that could have an disproportionately large effect on platform evangelism. 

Thanks for any insight,

-Marc

--

Marc Siegel

Team Lead

TIM Group

[Marc Siegel]

Hi all,

If it helps get an answer -- we are Typesafe customers. Would appreciate a quick answer to this question?

Thanks,

-Marc

[Erik Post]

Hi Marc,

That feature is called 'type providers', in case you feel like googling it. I think it's planned, but I'm not sure about the timeframe.

Cheers,

Erik

[Marc Siegel]

Thanks Erik!

Google revealed to me the branch "topics/type-providers", and a Scala Types interview from January, among other results. I see now that Type Providers currently compiles with Macro Paradise, so am I correct in understanding that this branch cannot land in master until the necessary features from Macro Paradise make into the mainline Scala compiler?

Thanks,

-Marc

[Erik Post]

Hi Marc, 

I think that's correct. I believe the original plan was for Slick's type providers to depend on type macros, but as far as I know that feature has been scrapped. Instead, it will be implemented in terms of macro annotations, which I *think* are somewhere in paradise. Please do take this with a grain of salt, as it's mostly hearsay.

Cheers,

Erik

[Pedro Furlanetto]

Adding to Erik's comments, also with a nice handful of salt: there a compiler plugin that adds most macro paradise's functionality to 2.10. No idea how this would play with the type-providers branch. 

--
 
---
You received this message because you are subscribed to the Google Groups "Slick / ScalaQuery" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalaquery+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Marc Siegel]

Thanks for this info guys.

Speaking as someone who 

  (a) has a team using ScalaQuery/Slick "in the trenches", and 

  (b) had teams using Rails/ActiveRecord in the past, 

I think it may merit consideration to promote the proposal for Slick Type Providers as to the community, long before it's finalized for a release.

This could allow interested community members (such as us) a chance to review early and to provide some feedback on the proposed mechanisms, before they are set in stone. If organized as a set of HowTo's (c.f. Rails Guides, such as http://guides.rubyonrails.org/association_basics.html), the community could also help edit these docs, and perhaps identify missing usage patterns to discuss and cover. This way, by the time the feature is released, it would already have common usage patterns documented, and have a body of practice in the community already.

I wonder if such a guide could be put up at slick.typesafe.com/guides/proposed-type-providers.html, and take pull requests via github?

Thanks for all,

[virtualeyes]

Type Providers aren't coming until after Scala 2.11 release, which is to say, not soon -- this is from Slick presentation at Scala Days; if things have changed I haven't heard/seen anything...

[Christopher Vogt]

Hi Marc,

type providers are indeed coming. We have two implementations at the
moment, one based on type macros, one based on source-code generation.
Macros are just nicely compiler-integrated ast-generators and can be
replaced by classical source-code-generation. Integration of type macros
into main stable scala is currently perceived as too risky and may never
happen. Annotation macros will probably come, but it is not certain yet.

Source-code-generation-based type-providers will likely ship with
upcoming Slick 2.1 later this year. For now, they will only allow
limited control over the structure of the generated source code.

An alternative approach with more control would be using your own code
generator or the one from my migrations tool prototype, which is very
flexible and uses Slick's schema reverse-engineering features. We do not
officially release or maintain it atm, but it is small enough to be
copied and pasted into your code base (~250 LOC).

Code:
https://github.com/cvogt/migrations/tree/master/src/main/scala/scala/slick/jdbc

Usage:
https://github.com/cvogt/migrations/blob/master/src/main/scala/SampleCodegen.scala

Real world application:
https://bhudgeons.telegr.am/blog_posts/slick-tool-for-db-schema-generation .

Chris
--
Slick Team

[Pedro Furlanetto]

There is the slick-macros tweeted today https://github.com/ebiznext/slick-macros.

[Stefan Zeiger]

On 2013-08-20 12:57, Christopher Vogt wrote:

Hi Marc,

type providers are indeed coming. We have two implementations at the
moment, one based on type macros, one based on source-code generation.
Macros are just nicely compiler-integrated ast-generators and can be
replaced by classical source-code-generation. Integration of type macros
into main stable scala is currently perceived as too risky and may never
happen. Annotation macros will probably come, but it is not certain yet.

Source-code-generation-based type-providers will likely ship with
upcoming Slick 2.1 later this year.

I assume you mean 2.0. 2.1 is scheduled for early next year and will hopefully include macro-based type providers if we get a Scala release with annotation macros by that time. Note that the current type-providers branch in Slick is based on type macros. This will probably not make it into any Slick release.

--
Stefan Zeiger
Slick Tech Lead
Typesafe - Build Reactive Apps!
Twitter: @StefanZeiger

[Marc Siegel]

Thank you all for taking the time to answer this question! So it seems clear that Type Providers are coming.

My concern, in terms of adoption (for us) and evangelism (for others), seems to still stand unanswered: What is the story for how the user community will begin to early-adopt and document how to develop in practice with this technology? And if there is no story for that yet, then what do you think of my earlier proposal (for a pre-release Rails guides inspired effort)?

I fear it may not work out well for Slick adoption, if the initial Type Provider feature delivered (via code generation in 2.0, apparently) is not extremely well pre-vetted, pre-adopted, and pre-documented by the community, in other words, as there would be too much risk in investing in the "first draft". 

(Apologies to all if I've just missed the community engagement on this!)

Regards,

-Marc

[Stefan Zeiger]

On 2013-08-20 19:07, Marc Siegel wrote:

Thank you all for taking the time to answer this question! So it seems clear that Type Providers are coming.

My concern, in terms of adoption (for us) and evangelism (for others), seems to still stand unanswered: What is the story for how the user community will begin to early-adopt and document how to develop in practice with this technology? And if there is no story for that yet, then what do you think of my earlier proposal (for a pre-release Rails guides inspired effort)?

I fear it may not work out well for Slick adoption, if the initial Type Provider feature delivered (via code generation in 2.0, apparently) is not extremely well pre-vetted, pre-adopted, and pre-documented by the community, in other words, as there would be too much risk in investing in the "first draft".

That risk always exists, no matter what you call the first version, and I don't expect the first version to solve all the problems and provide a perfect API. That said, the API is pretty small since the code generator is merely a tool that you run as part of your build process. The only API is the one you use for implementing custom name and type mappings. By the time the "cool" part (macro-based type providers) hits master, we should have enough experience with the code generator.

We will release this in a milestone build as soon as it's been merged into master. A chapter in the manual might have to wait until the next milestone. I don't really like the idea of wiki-style community documentation because it's not versioned along with the code. We tried that with the github wiki for ScalaQuery and then for Slick but abandoned it in favor of a proper user manual that lives inside the main repo. The process for changing this documentation is the same as for code: Create a pull request. Previewing documentation changes locally is not quite as convenient as testing code changes where all you need is Java and sbt, but the overhead is low. You need to install Python and Sphinx, which is straight-forward and can be done in a few minutes (even on Windows). Then "sbt make-site" will build the manual.