Scala's not-so-Simple Build Tool

[Ben Hutchison]

With every passing month I get less fond of SBT.

I find myself wondering why, with all the technology and skill poured
into SBTs creation, it often seems so hard to do anything new &
unfamiliar.

I find myself wondering why SBT 0.7 and SBT 0.10 needed to be so
incompatible, and why SBT 0.10+ needed to invent a new dialect of
Scala (that IDEs dont understand), just to write a config file.

And every time I go to the docs, to find out how to do something
mundane like change the target compilation directory, I find myself
wondering why I get a 14 page lecture on SBTs "System of Programming"
just to edit my build config.

This video is illustrative of the SBT focus: SBT Introduction &
Cookbook [http://www.youtube.com/watch?v=vED2LMbdFDc]. Jump to some
random point in it and you'll learn that "Initialize[_] is an
Applicative Functor", and everything else you probably didn't want to
know about SBT's internals. Its a nicely put together video - for SBT
Hackers - but that isn't most people.

And where *do* you learn how to change the target compilation directory?

By scrapping around on stack overflow I guess, swapping tidbits with
other users, or by reading the source code, of course!
[https://groups.google.com/forum/?fromgroups#!topic/simple-build-tool/0qhCyBgsmdU]

-Ben

[Jason Zaugg]

On Fri, Jul 13, 2012 at 1:49 PM, Ben Hutchison <brhut...@gmail.com> wrote:
> I find myself wondering why SBT 0.7 and SBT 0.10 needed to be so
> incompatible, and why SBT 0.10+ needed to invent a new dialect of
> Scala (that IDEs dont understand), just to write a config file.

SBT 0.7 modeled extensibility with the template method pattern;
extends a base class and override methods. This was both fragile and
rigid. You could model testSources and sources, but had no easy way to
setup a compilation chain for, say, integration test code.

IntelliJ understands it; just associate .sbt with Scala files in a
IntelliJ project created by the sbt-idea plugin. You get
autocompletion, error highlighting.

I tend to use and recommend the .scala build configuration; .sbt
config is just a sugar on top of that.

> And every time I go to the docs, to find out how to do something
> mundane like change the target compilation directory, I find myself
> wondering why I get a 14 page lecture on SBTs "System of Programming"
> just to edit my build config.
>
> This video is illustrative of the SBT focus: SBT Introduction &
> Cookbook [http://www.youtube.com/watch?v=vED2LMbdFDc]. Jump to some
> random point in it and you'll learn that "Initialize[_] is an
> Applicative Functor", and everything else you probably didn't want to
> know about SBT's internals. Its a nicely put together video - for SBT
> Hackers - but that isn't most people.
>
> And where *do* you learn how to change the target compilation directory?

You can learn how things fit together with `inspect`. [1]

> inspect target
[info] Setting: java.io.File = C:\code\analytics\analytics-utils\target
[info] Description:
[info] Main directory for files generated by the build.
[info] Provided by:
[info] {file:/C:/code/analytics/}analytics-utils/*:target
[info] Dependencies:
[info] analytics-utils/*:base-directory
[info] Reverse dependencies:
[info] analytics-utils/*:cross-target
[info] analytics-utils/*:clean-files
[info] analytics-utils/*:history
...

set target <= baseDirectory.apply(bd => bd / "my-target')

An enhanced version of inspect is provided by sbt-inspectr [2], and
will be part of the standard SBT 0.12.

Stick with it. Ask questions on the mailing list, and contribute
configuration snippets to the examples on the Wiki [3]

Or, if you're more comfortable with Maven, know that the incremental
recompilation engine from SBT has ported to maven-scala-plugin by
Typesafe. [4]

-jason

[1] https://sbtwiki.backchat.io/Inspecting%20Settings
[2] https://github.com/eed3si9n/sbt-inspectr
[3] https://sbtwiki.backchat.io/Quick%20Configuration%20Examples
[4] https://groups.google.com/forum/?fromgroups#!topic/maven-and-scala/51EFqZjSocw

[Andy Kitchen]

I can't agree enough Ben. SBT is a woeful beast. My question is not if it should die, but how to kill it.

Sent from my iPhone

> --
> You received this message because you are subscribed to the Google Groups "Melbourne Scala User Group" group.
> To post to this group, send an email to scala...@googlegroups.com.
> To unsubscribe from this group, send email to scala-melb+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scala-melb?hl=en-GB.
>

[Ben Hutchison]

On Sat, Jul 14, 2012 at 10:43 AM, Andy Kitchen <kitche...@gmail.com> wrote:
> I can't agree enough Ben. SBT is a woeful beast. My question is not if it should die, but how to kill it.

Well, the reason most software tools "die" is because something better
comes along. If you want to kill it, you might start by proposing a
better alternative: Maven? Ant? .sh files? Gradle? Buildr?

Personally, I don't think SBT should "die". I do however, think that
its not particularly Simple any more, that it's a bit over-engineered,
and that the documentation focuses on SBT internals & architecture
topics over end-user functionality.

-Ben

[Ken Scambler]

A fork of 0.7?

[Tony Morris]

The difficult problem is that of dependency management, which flows on
to build tools and libraries on top.

We have an in-house haskell library that resolves this dependency
management with some compromises (it is NP-complete in general), but we
use it for JavaScript and Ruby projects.

The only reason we continue to use sbt is because it provides an
essential feature: incremental compilation. However, it has also been
proposed that we put together a scala compiler written with haskell that
provides many more features above scalac, including incremental compilation.

If only we had more people who were willing to step up...

--
Tony Morris
http://tmorris.net/

[Ben Hutchison]

Ouch! Surely it's not that dire. As Tony notes, SBT has some awesome technology inside.

It might just need someone with an end-user-product focus to put a more friendly surface on it, write docs for mere mortals, and smooth some rough spots like webapp support.

Ben

[Jem]

That sounds like the ticket. Additionally, remove the word 'simple' from the name. Surely that was the touch of death.

[Toby Corkindale]

On 13 July 2012 21:49, Ben Hutchison <brhut...@gmail.com> wrote:
> With every passing month I get less fond of SBT.
>
> I find myself wondering why, with all the technology and skill poured
> into SBTs creation, it often seems so hard to do anything new &
> unfamiliar.
>
> I find myself wondering why SBT 0.7 and SBT 0.10 needed to be so
> incompatible, and why SBT 0.10+ needed to invent a new dialect of
> Scala (that IDEs dont understand), just to write a config file.
>
> And every time I go to the docs, to find out how to do something
> mundane like change the target compilation directory, I find myself
> wondering why I get a 14 page lecture on SBTs "System of Programming"
> just to edit my build config.

I found SBT near-incomprehensible, and then they changed everything
once I had figured out enough to get by.

Now I just use good old GNU Make.

It works well, it provides partial dependency-based recompilation, and
the rules it uses are simply defined and understandable. It's also
compatible with almost everything, so you can use it to build
multi-lingual projects.

[Korny Sietsma]

From memory, some of the "awesome technology inside" was nigh-unreadable code; I vaguely understood this code 18 months ago when I was trying to get sbt 0.9 (I think?) to work, but now it's almost totally incomprehensible*:

http://harrah.github.com/xsbt/latest/sxr/Aggregation.scala.html

It's possible this is documented further in tests somewhere, but I dug for a bit and couldn't find much in the way of unit tests for sbt.   I might just have missed them, but they didn't seem to be in any of the obvious places.

- Korny

* of course, I haven't really used scala much in the last 12 months, so maybe I'm just too rusty in general!

Kornelis Sietsma  korny at my surname dot com http://korny.info
"We do not quit playing because we grow old, we grow old because we quit playing" - O.W. Holmes

[Ben Hutchison]

I'm not sure if readability to the casual reader is, or should be, a goal for SBTs internal code. The scala compiler and scalaz aren't easily readable either!

The build scripts users write and maintain do need to be readable of course, and there I have some concerns.

SBT does have unit tests, but they are spread around in odd places. For some reason, it doesn't follow it's own convention for project layout.

Ben

[Korny Sietsma]

I consider code readability to _always_ be a goal of good software.

- Korny

[Travis Dixon]

Always _a_ goal, but in sbt I'd rate the usage docs as a higher priority.

Readable code can be a decent fall back for when docs don't show you how to do something or when you're stuck on an edge case, but it's not what I want to start with when using a library/tool

[Tony Morris]

I think Ben's point is that quite often many people confuse "code is readable" with "I claim this code is more readable to me (when often, if the claim is tested, it is in fact not)." I often hear some programmers make claims about their more readable code and it turns out that (when tested) not only do I disagree, but so do they and refuse to admit it.

There are ways of measuring code readability without appealing to the person.

-- 
Tony Morris
http://tmorris.net/

[Toby Corkindale]

On 17 July 2012 14:26, Korny Sietsma <ko...@sietsma.com> wrote:
> I consider code readability to _always_ be a goal of good software.

How do you feel about the effect of Scala on code, in general? Does
Scala encourage readable code?

-T

[Jem]

Rich Hickey says in this talk, "I find German unreadable. But that's because I don't know German". 

I'm not fussed about how SBT is under the covers, because I am a user, not a contributor. This 'customer' distinction is important. As a user, I'm happy to let it be written in any language, in any style. However, the API and docs need to be in a language and at a level that the user can comprehend. v0.7 was like that, 0.10 is not, IMO. It's good to give a stretch target; to nudge the average skill level of the users up a notch. Is there a bridge to take me from 0.7 to 0.10?  The gap seems too large - especially if clever clogs like Ben have complaints.

[Tony Morris]

I agree that Hickey is right in his point that claims of readability/unreadability are easily undermined. I just claim that he is being generous here. Claims of readable/unreadable can almost always be even further undermined with a swish. His point assumes that were he to come to learn German, then this would somehow affect readability -- in my experience, it's not even then.

If readable/unreadable is to have any useful meaning, I'll stick with a metric that does not appeal to the person. This exists.

[Andrew Conway]

> How do you feel about the effect of Scala on code, in general? Does Scala encourage readable code?

I have very strong but mixed feelings on this. Scala has mixed effects on code (compared, to, say, Java).

Scala can express many things concisely and in an easy to read manner with less obscuring what is going on. The ability to easily do things many different ways makes it possibly to write more readable code than Java **IF YOU ARE WORKING HARD TO DO SO**

However the same flexibility can very easily lead to PERL style illegibility.

Incidentally, while the Scala collection classes are much better than Java's in many ways, they do lend theirself to long, hard to read lines. What I used to do in a page of incomprehensible Java, I now do in one long line of incomprehensible Scala. I guess that reduces the number of incomprehensible lines... but...

I think opinions may vary on which of the following is more legible (please forgive typos) :

var sumFred = 0
for (s<-things) sumFred+=s.fred

val sumFred = things.map{_.fred}.sum

val sumFred = things.foldLeft(0){_+_.fred}

val sumFred = things :\ (0){_+_.fred}

val sumFred = things.foldLeft(0){(cum,s)=>cum+s.fred}

Implicit functions can improve legibility: eg to convert a collection to an array, Scala uses implicit functions to avoid type erasure problems:

c.toArray  

whereas in Java one needs

c.toArray(new Fred[c.size()])

In this case Scala is much more readable that Java. However, you can make some very hard to read code with implicit functions if you are not careful (and even if you are). You have to be pretty good at reading Scala to read the Lift source code (but once you are, it is amazingly powerful).

BTW: I rapidly gave up on sbt and use simple old ant and manual library dependency management.

Regards,

Andrew.

[Toby Corkindale]

On 17 July 2012 15:04, Jem <jem.m...@gmail.com> wrote:
> Rich Hickey says in this talk, "I find German unreadable. But that's because
> I don't know German".

But I could argue that German is much easier to learn than, say,
Chinese, due to its use of a restricted character set, lack of
required inference in grammar, and tendency to build complex words by
directly combining simple words, thus reducing the required memorised
vocabulary.

In particular, Scala libraries seem to heavily use arbitrary
combinations of symbols for declared methods, which I suspect will be
retrospectively seen as a poor decision.

-T


--
Turning and turning in the widening gyre
The falcon cannot hear the falconer
Things fall apart; the center cannot hold
Mere anarchy is loosed upon the world

[Jem]

>  I rapidly gave up on sbt and use simple old ant and manual library dependency management 

That's precisely what I do. Manual (3rd party) dependency especially useful in environments where disk is cheap and proxy servers are unforgiving. (i.e. every place I have ever worked).

--
You received this message because you are subscribed to the Google Groups "Melbourne Scala User Group" group.

To view this discussion on the web, visit https://groups.google.com/d/msg/scala-melb/-/o2p0-y-7TxQJ.

[Korny Sietsma]

Definitely, in general Scala code is more readable than (say) Java, mostly as it gets rid of a whole lot of verbosity.

Terser code _tends_ to be more readable than more verbose code, but it also packs more information into a smaller space, so people have to be more careful about things like meaningful names, relevant comments, and good test cases.  For some reason many people get into a language like Scala or Clojure or Haskell and start going wild with inline functions and operator overloading and single-letter variable names, producing powerful but totally obtuse code.  Also some people seem to do less unit testing in functional languages, which means you lose some of the documentation you get from a good set of unit tests.

But yes, overall I'd much prefer reading code in any of these languages than wading through hundreds of lines of Java or C# bloat.

- Korny

--
You received this message because you are subscribed to the Google Groups "Melbourne Scala User Group" group.
To post to this group, send an email to scala...@googlegroups.com.
To unsubscribe from this group, send email to scala-melb+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/scala-melb?hl=en-GB.

[Ben Hutchison]

On Tue, Jul 17, 2012 at 3:30 PM, Toby Corkindale <to...@dryft.net> wrote:
> But I could argue that German is much easier to learn than, say,
> Chinese, due to its use of a restricted character set, lack of
> required inference in grammar, and tendency to build complex words by
> directly combining simple words, thus reducing the required memorised
> vocabulary.

Straying completely off topic, can the advantages of German/European-
over Chinese- script be reduced to mathematics?

26^5 > 2000^2

That is, recombining a smaller alphabet of symbols into longer chains
yields more total combinations than combining a larger alphabet into
shorter chains.

-Ben

[Jem]

Importantly 

>  Also some people seem to do less unit testing in functional languages 

The stronger the type system, the more the compiler is testing for you. e.g. There's no need to check if the object is like a valid phone number, if you actually have a PhoneNumber and not a String.

But you are right, there is always a place for diligence when it comes to TDD. It takes some time to learn how to best approach testing (and code style) when the paradigm is unfamiliar.

[Tony Morris]

On 17/07/12 15:30, Toby Corkindale wrote:
> On 17 July 2012 15:04, Jem <jem.m...@gmail.com> wrote:
>> Rich Hickey says in this talk, "I find German unreadable. But that's because
>> I don't know German".
> But I could argue that German is much easier to learn than, say,
> Chinese, due to its use of a restricted character set, lack of
> required inference in grammar, and tendency to build complex words by
> directly combining simple words, thus reducing the required memorised
> vocabulary.
>
> In particular, Scala libraries seem to heavily use arbitrary
> combinations of symbols for declared methods, which I suspect will be
> retrospectively seen as a poor decision.
>
> -T
>
>

Maybe so, but identifier names are not correlated with readability.

[Toby Corkindale]

I strongly disagree, but I think we've had this argument before, and
may just have to agree to disagree.

[Tony Morris]

I find "agreeing to disagree" to be dishonest to I do not accept. I just
think you're wrong -- not even you believe that when under test.

We can put it aside though. I do not recall the argument, but I do know
there are lot of people who are under the illusion that they believe
this proposition.

[Ken Scambler]

I also strongly agree with Toby, on the basis that structure and types can tell me "what" a piece of code is doing, but naming is better equipped to tell me "why".

(Further assuming that readability has something to do with being able to tell what code does and why it does it)

What are the arguments against?

[Toby Corkindale]

On 17 July 2012 16:01, Tony Morris <tonym...@gmail.com> wrote:
> On 17/07/12 15:56, Toby Corkindale wrote:
>> On 17 July 2012 15:52, Tony Morris <tonym...@gmail.com> wrote:
>>> On 17/07/12 15:30, Toby Corkindale wrote:
>>>> On 17 July 2012 15:04, Jem <jem.m...@gmail.com> wrote:
>>>>> Rich Hickey says in this talk, "I find German unreadable. But that's because
>>>>> I don't know German".
>>>> But I could argue that German is much easier to learn than, say,
>>>> Chinese, due to its use of a restricted character set, lack of
>>>> required inference in grammar, and tendency to build complex words by
>>>> directly combining simple words, thus reducing the required memorised
>>>> vocabulary.
>>>>
>>>> In particular, Scala libraries seem to heavily use arbitrary
>>>> combinations of symbols for declared methods, which I suspect will be
>>>> retrospectively seen as a poor decision.
>>>>
>>>> -T
>>>>
>>>>
>>> Maybe so, but identifier names are not correlated with readability.
>> I strongly disagree, but I think we've had this argument before, and
>> may just have to agree to disagree.
>>
>
> I find "agreeing to disagree" to be dishonest to I do not accept. I just
> think you're wrong -- not even you believe that when under test.

Yes, I know you think I'm wrong. Just as I think you are wrong. Thus,
the only thing we agree upon is that we don't agree.
It is my attempt to politely avoid starting an argument with someone
whom I consider it pointless to argue with.

I realise you feel strongly about this issue, and I am sure your way
of working *does* work well for you. However you must have realised by
now that the world is full of people who are not identical to you.
If most programmers found it easier to read and write software
consisting entirely of bizarre symbols, then A+ would be the dominant
language.

> We can put it aside though. I do not recall the argument, but I do know
> there are lot of people who are under the illusion that they believe
> this proposition.

It's neither a proposition nor an illusion, but yeah, we can put it
aside. Please.

[Ishaaq Chandy]

On 17 July 2012 15:23, Tony Morris <tonym...@gmail.com> wrote:

If readable/unreadable is to have any useful meaning, I'll stick with a metric that does not appeal to the person. This exists.

Such a metric exists??? Don't leave us hanging - point me to information I can read! Note that there is no sarcasm implied here - I am genuinely surprised by your statement and would like to know more.

I do understand that there are metrics like the Law of Demeter and others like it that can attempt to gauge the quality of code, but to my mind these are ultimately weak heuristics that are quite easily broken by a sufficiently crafty (or stupid, more often for that matter) human programmer. In my experience the correlation between programs that adhere to these rules and those that are readable is definitely not one to one.

I've always felt that readability is at least partly subjective. Perhaps your definition of the word "readability" differs from mine?

Ishaaq

[Tony Morris]

Where types fail, quantified algebraic properties do the rest in that they do everything types do, except provide proof.

More to the point, names have never had any bearing, for either of us. It's an illusion, one that I understand exists, but is so utterly uninviting to me and my peers, for adverse practical reasons.

[Travis Dixon]

On Tue, Jul 17, 2012 at 3:43 PM, Korny Sietsma <ko...@sietsma.com> wrote:

 people have to be more careful about things like meaningful names

This bit is harder than I ever thought it could be though.

You get a whole spectrum from giving large names to indexes to giving your most important business model's an arbitrary letter.

Worst of all, the entire spectrum is completely subjective (although opinions are not evenly distributed along that spectrum).

I tend to find though, that the closer I'm getting to code that actually does anything, the less important variable names are, and the further away I get from code that does things, the more meaningful I need it to be.

Just an observation I found interesting

[Travis Dixon]

On Tue, Jul 17, 2012 at 5:22 PM, Ishaaq Chandy <ish...@gmail.com> wrote:

On 17 July 2012 15:23, Tony Morris <tonym...@gmail.com> wrote:

If readable/unreadable is to have any useful meaning, I'll stick with a metric that does not appeal to the person. This exists.

Such a metric exists??? Don't leave us hanging - point me to information I can read! Note that there is no sarcasm implied here - I am genuinely surprised by your statement and would like to know more.

seconded, link please