Prioritize the de-conflation of pattern matching and the standard library
Shelby
mailing list.
https://issues.scala-lang.org/browse/SUGGEST-18
Attempting to bump up the priority of this suggestion. Can anyone
comment on how extensive this change would be?
The relevant technical discussion is in the above link. Below I want
to argue the case for prioritization.
With Ceylon 1.0 shooting for release this year (first alpha released
already), Scala might benefit from more experimentation with creating
easy-to-use libraries that will be competitive and best matched to the
mainstream.
This conflation of pattern matching (one of Scala's brilliant
features) and the standard library, is a potential point of
significant competitive weakness, as Gavin King is a library master
and afaics (in the Ceylon language design FAQ) is already
demonstrating his pragmatism for making user friendly design choices.
I don't know how long it will take them to challenge Scala, but I
don't dismiss the potential exponential speed of a ramping community
from of huge pool of excited Java developers (against I assume a dozen
or so hard-core Typesafe developers).
An example of that pragmatism, I assume Ceylon is going to avoid the
type inference binary compatibility problem of Scala, by allowing only
type inference only where the impacts are local:
http://ceylon-lang.org/documentation/1.0/introduction/#type_aliases_and_type_inference
http://www.slideshare.net/mircodotta/managing-binary-compatibility-in-scala
I assume Gavin will make other clever design decisions like this
(corner cases he is missing?), and the Scala community needs to be
free to compete without having to carry the baggage of the standard
library (i.e. any mistakes that become apparent over time, and the
first being that it is apparently monolithic, since the pattern match
feature of the compiler is conflated) around in everything. I heard in
the Paul Philips interview that there are already weird corner cases
in the standard library.
So I am asking if we perhaps need to divert some our priority
attention from type theoretic focus (e.g. efforts to make the current
compiler perfectly sound), and deal with other bigger picture
pragmatic issues like first? Eric Raymond (Cathedral & Bazaar open
source manifesto) claims that large numbers random monkeys banging on
the open bazaar of code will beat the closed cathedral of experts by
orders of magnitude (and this is apparently being proven in the real
world results of open source vs. closed source). So I am pondering
that maybe we need to prioritize the issues of openness, i.e. de-
conflation in this case.
Exhibits:
http://www.catb.org/esr/comscore/ (open source Android vs. closed
source iPhone)
http://ceylon-lang.org/documentation/1.0/faq/#when_will_it_be_ready
http://blog.joda.org/2011/11/scala-feels-like-ejb-2-and-other.html
(see my Modularity, Concurrency, Complexity comments)
http://blog.joda.org/2011/11/scala-ejb-2-feedback.html
http://stackoverflow.com/questions/1722726/is-the-scala-2-8-collections-library-a-case-of-the-longest-suicide-note-in-hist
http://grahamhackingscala.blogspot.com/2011/12/will-typesafe-address-scala-criticisms.html
Paul Phillips
against I assume a dozen or so hard-core Typesafe developers
Daniel Sobral
joke with a simple smiley at the end. The lack of said smiley, and the
fact the the linked suggestion is serious enough, leaves me utterly
confused. Anyway, here's my input.
On Mon, Mar 5, 2012 at 20:45, Shelby <she...@coolpage.com> wrote:
> Per Soc's comment, I am belatedly posting this suggestion to the
> mailing list.
>
> https://issues.scala-lang.org/browse/SUGGEST-18
>
> Attempting to bump up the priority of this suggestion. Can anyone
> comment on how extensive this change would be?
That would break a lot of code to absolutely no advantage that I can
see. Specifically:
1. One needs to implement three methods to define a Seq (iterator,
apply and length), so the argument that it is hard holds no water.
2. No example of a situation where a something might want to define a
non-standard collection and return it through unapplySeq was given,
and I, for one, can't think of one.
> With Ceylon 1.0 shooting for release this year (first alpha released
> already), Scala might benefit from more experimentation with creating
> easy-to-use libraries that will be competitive and best matched to the
> mainstream.
I'd put more faith in that with Java compatibility was a stated goal
of Ceylon. Last time I looked, it was explicitly not a given.
> An example of that pragmatism, I assume Ceylon is going to avoid the
> type inference binary compatibility problem of Scala, by allowing only
> type inference only where the impacts are local:
>
> http://ceylon-lang.org/documentation/1.0/introduction/#type_aliases_and_type_inference
> http://www.slideshare.net/mircodotta/managing-binary-compatibility-in-scala
If only type inference was the major cause of binary compatibility
problems! Type inference is a smallish detail, which is simply a
matter of carefulness.
> I assume Gavin will make other clever design decisions like this
> (corner cases he is missing?), and the Scala community needs to be
> free to compete without having to carry the baggage of the standard
> library (i.e. any mistakes that become apparent over time, and the
> first being that it is apparently monolithic, since the pattern match
> feature of the compiler is conflated) around in everything. I heard in
> the Paul Philips interview that there are already weird corner cases
> in the standard library.
I assume, I heard... can't you can actually come with facts, such as
"I wrote a program to do X, and had the trouble Y"?
> So I am asking if we perhaps need to divert some our priority
> attention from type theoretic focus (e.g. efforts to make the current
> compiler perfectly sound), and deal with other bigger picture
> pragmatic issues like first? Eric Raymond (Cathedral & Bazaar open
And here's the part where you threw away any credibility you might
have. The biggest effort spent all through 2.9x has been on *Eclipse
IDE support*. Yes, some individuals have been working on "type
theoretic stuff" -- though nothing of the like you mention -- but it's
been so far from the main focus that it hasn't even appeared anywhere
but papers and conferences -- certainly not the compiler itself. Other
efforts were given to documentation, to having simple and efficient
installers, to getting the source control on github so that all may
contribute more easily, to making the sites look good, to getting an
integrated offering of tools to go together with the language, etc.
So, basically, for the past year and a half Scala has been nothing but
pragmatic, which means that you have absolutely no knowledge
whatsoever about what is actually being done. An *interested*
outsider, one who had bothered to give even a cursory look at what's
happening, would have a much better chance of gaining support. If for
no other reason, you'd at least look like you know what you were
talking about, which, right now, you are not.
> source manifesto) claims that large numbers random monkeys banging on
> the open bazaar of code will beat the closed cathedral of experts by
> orders of magnitude (and this is apparently being proven in the real
> world results of open source vs. closed source). So I am pondering
Note that Eric Raymond extrapolated that from a single example, and
ignoring counter examples to his theory. There's absolutely no
evidence or study that gives credence to what he said.
> that maybe we need to prioritize the issues of openness, i.e. de-
> conflation in this case.
And what's that to do with what Eric said, no matter whether he's
right or wrong?
>
> Exhibits:
>
> http://www.catb.org/esr/comscore/ (open source Android vs. closed
> source iPhone)
iPhone still beats Android by a huge margin. I know: I have an
Android, my wife has an iPhone, and I don't want to migrate to iPhone
due for personal reasons. But I'm under no illusions of how much
better the iPhone is.
> http://ceylon-lang.org/documentation/1.0/faq/#when_will_it_be_ready
Show me the money, as they say. That's just a wishlist.
> http://blog.joda.org/2011/11/scala-feels-like-ejb-2-and-other.html
Coming from someone who never used Scala and actually complains about
stuff like where the type is related to the variable name.
> http://blog.joda.org/2011/11/scala-ejb-2-feedback.html
Oh, him again!
Well, it's 2.9.1, and Scala 2.8 collections haven't stopped anyone so far.
> http://grahamhackingscala.blogspot.com/2011/12/will-typesafe-address-scala-criticisms.html
That one I haven't read, but, honestly, I'm not much in the mood to do
so either.
--
Daniel C. Sobral
I travel to the future all the time.
Shelby
> I find this whole text so divorced from reality that I'd take it as a
Daniel, thank you for the detailed response. I will take some time to
research your points before I reply.
I am very interested, although my level of experience with Scala is no
where near yours. I am merely trying to raise some thoughts and see if
they hold up. I hope I can come to understand better, and see clearly
if you are correct or not.
I am not here to spread FUD. Please see my 3 deeply reasoned comments
at the bottom of the page of that person who is criticizing Scala:
http://blog.joda.org/2011/11/scala-feels-like-ejb-2-and-other.html
Also I am not wanting to be a superstar here, nor step on anyone's
toes. I am here to try to fit it and be humble. Please feel free to
continue to tell me frankly where I am overstepping or not
sufficiently researched.
Adriaan Moors
cheers
Shelby
> That would break a lot of code
that is independent from the standard library would I assume break
much code.
Couldn't this be averted with compile time option and/or an
annotation? Code which wants to use `Seq` with the pattern matcher
could continue to do so.
> to absolutely no advantage that I can
> see. Specifically:
>
> 1. One needs to implement three methods to define a Seq (iterator,
> apply and length), so the argument that it is hard holds no water.
http://www.scala-lang.org/api/current/index.html#scala.collection.Seq
> 2. No example of a situation where a something might want to define a
> non-standard collection and return it through unapplySeq was given,
> and I, for one, can't think of one.
collections library, e.g. based on category theory:
http://stackoverflow.com/questions/1722726/is-the-scala-2-8-collections-library-a-case-of-the-longest-suicide-note-in-hist/7397388#7397388
[...]
> I'd put more faith in that with Java compatibility was a stated goal
> of Ceylon. Last time I looked, it was explicitly not a given.
some cases:
http://codahale.com/downloads/email-to-donald.txt
http://grahamhackingscala.blogspot.com/2011/12/will-typesafe-address-scala-criticisms.html
> If only type inference was the major cause of binary compatibility
> problems! Type inference is a smallish detail, which is simply a
> matter of carefulness.
didn't type a changed method signature.
From Odersky's post about binary compatibility, apparently the main
extra burden on binary compatibility in Scala are changes in traits.
My current thinking about modularity, is that traits should never
change, so that mainly leaves type inference (of binary compatibility
issues unique to Scala and not those shared with most languages):
http://blog.joda.org/2011/11/scala-feels-like-ejb-2-and-other.html?showComment=1330675016809#c4293420055426595433
[...]
> And here's the part where you threw away any credibility you might
> have. The biggest effort spent all through 2.9x has been on *Eclipse
> IDE support*.
some changes were needed in the compiler to make an IDE work well.
I am writing about the present, not the past.
> Yes, some individuals have been working on "type
> theoretic stuff" -- though nothing of the like you mention -- but it's
> been so far from the main focus that it hasn't even appeared anywhere
> but papers and conferences -- certainly not the compiler itself.
of the type system. I assume that could go on for years to perfect
all.
I agree that is useful. I am pondering if such perfection can take
temporary backseat in priority to more deeply rooted, fundamental
issues that may have wider impact.
For me so far, those might be (not sure):
* lack of first-class union type
* inability to use the pattern matching with sequences without forcing
the standard library.
> Other
> efforts were given to documentation, to having simple and efficient
> installers, to getting the source control on github so that all may
> contribute more easily, to making the sites look good, to getting an
> integrated offering of tools to go together with the language, etc.
interested in Scala.
> So, basically, for the past year and a half Scala has been nothing but
> pragmatic, which means that you have absolutely no knowledge
> whatsoever about what is actually being done.
Adriaan might be a key implementator on virtual pattern matcher and on
any future union type, and for example I read a bug report where he is
stumped for days on some obscure typing soundness bug. Perhaps my
sample size was too small on where the priorities are in the
*PRESENT*.
> An *interested*
> outsider, one who had bothered to give even a cursory look at what's
> happening, would have a much better chance of gaining support. If for
> no other reason, you'd at least look like you know what you were
> talking about, which, right now, you are not.
collections (tries, etc), concurrency and parallelism libraries, etc..
I have sense there is a lot of being thrown at the standard
collections, and I am asking whether some effort should be thrown at
de-coupling the standard library from the compiler.
> > source manifesto) claims that large numbers random monkeys banging on
> > the open bazaar of code will beat the closed cathedral of experts by
> > orders of magnitude (and this is apparently being proven in the real
> > world results of open source vs. closed source). So I am pondering
>
> Note that Eric Raymond extrapolated that from a single example, and
> ignoring counter examples to his theory. There's absolutely no
> evidence or study that gives credence to what he said.
Firefox
Android
etc..
> > that maybe we need to prioritize the issues of openness, i.e. de-
> > conflation in this case.
>
> And what's that to do with what Eric said, no matter whether he's
> right or wrong?
culture of a dozen people or so, on the entire POTENTIAL community
(thus which we may never attain).
I think modularity is very important. That is why I think `trait` is a
brilliant feature of Scala. I wish a light-weight trait was employed
for pattern matching instead of a heavy-weight one that carries much
of the standard library in with it.
> > Exhibits:
>
> >http://www.catb.org/esr/comscore/(open source Android vs. closed
>
> iPhone still beats Android by a huge margin.
iPhone has 29% marketshare in USA, Android nearly 50%. Android came
from behind with only 5% and raced past iPhone.
The numbers are even more lopsided in the developing markets, which
are more price and feature-diversity sensitive.
[...]
>
> >http://ceylon-lang.org/documentation/1.0/faq/#when_will_it_be_ready
>
> Show me the money, as they say. That's just a wishlist.
For a language, marketshare is critical.
> >http://blog.joda.org/2011/11/scala-feels-like-ejb-2-and-other.html
>
> Coming from someone who never used Scala and actually complains about
> stuff like where the type is related to the variable name.
allegations about qualifications.
[...]
> Well, it's 2.9.1, and Scala 2.8 collections haven't stopped anyone so far.
[...]
Paul Phillips
> I had the impression (even before his comments today) for example that
> Adriaan might be a key implementator on virtual pattern matcher and on
> any future union type, and for example I read a bug report where he is
> stumped for days on some obscure typing soundness bug. Perhaps my
> sample size was too small on where the priorities are in the
> *PRESENT*.
The "obscure typing soundness" bugs, and other bugs whose relevance
may not be obvious to you, do not live in a separate universe where
they have no bearing. Everything is intertwined. The luxury of
seeing soundness bugs as unimportant is a viewpoint which you can
possess only because we do not.
Stan Campbell
Adriaan Moors
I assume that could go on for years to perfect all.
Francois
I was about to answer on that really point, but that answer is just great :)
Just a note for Shelby: the type system is here to guard me from errors. If I can't trust the type system, what you are asking for, I will have to spend MUCH time in overlly unnecessary tests and/or obscure, difficults bug to trace - time better spend correcting business bugs, adding features, etc. That's one of the main reason why I use Scala, and not Java or Gosu.
As a really 'pragmatic'[1] user who (try to) earn money thanks to Scala, ignoring unsoudness would be a clear signal to make me wonder if really Scala worth it.
[1] I really dislike that words, because most of the time, it's just an argument of authority - as it is here, my prafmatism is not yours, and you can't argue why mine is not OK.
-- Francois ARMAND http://fanf42.blogspot.com http://www.normation.com
Daniel Sobral
> On Mar 6, 9:32 am, Daniel Sobral <dcsob...@gmail.com> wrote:
>
> [...]
>
>> That would break a lot of code
>
> Replacing everywhere the use of `Seq` with a simple trait interface
> that is independent from the standard library would I assume break
> much code.
>
> Couldn't this be averted with compile time option and/or an
> annotation? Code which wants to use `Seq` with the pattern matcher
> could continue to do so.
The code which calls the pattern matcher is not the code which defines
the methods the pattern matcher will call, which I consider a big
problem with this approach. In the end, of course, yes, one could have
a compiler switch so that two completely different (and incompatible)
implementations could be selected from. From my biased wording, I
think one can easily see I do not consider this a good idea. :-)
>> to absolutely no advantage that I can
>> see. Specifically:
>>
>> 1. One needs to implement three methods to define a Seq (iterator,
>> apply and length), so the argument that it is hard holds no water.
>
> I see dozens of methods and multiple inheritance:
>
> http://www.scala-lang.org/api/current/index.html#scala.collection.Seq
Yes, so what? Here, let's write one on the REPL:
scala> class X extends Seq[Int] {
| def length = 2
| def apply(n: Int) = n match { case 0 => 1 case 1 => 2 }
| def iterator = Iterator(1, 2)
| }
defined class X
There, three methods.
>> 2. No example of a situation where a something might want to define a
>> non-standard collection and return it through unapplySeq was given,
>> and I, for one, can't think of one.
>
> I (or others) might want to implement an entirely orthogonal
> collections library, e.g. based on category theory:
>
> http://stackoverflow.com/questions/1722726/is-the-scala-2-8-collections-library-a-case-of-the-longest-suicide-note-in-hist/7397388#7397388
>
> [...]
And using Seq with pattern matching prevents you from doing so how,
exactly? Remember that unapplySeq is not defined on the collection
class, but on a companion object. All the companion object needs to do
is convert what it receives into a Seq.
>
>> I'd put more faith in that with Java compatibility was a stated goal
>> of Ceylon. Last time I looked, it was explicitly not a given.
>
> Some claim that Java interopt may be more pain than it is worth in
> some cases:
>
> http://codahale.com/downloads/email-to-donald.txt
> http://grahamhackingscala.blogspot.com/2011/12/will-typesafe-address-scala-criticisms.html
True, but how can people start "creating easy-to-use libraries" (your
words) that will NOT be compatible with Java? What use it will be to
the "ramping community from of huge pool of excited Java developers"?
>
>
>> If only type inference was the major cause of binary compatibility
>> problems! Type inference is a smallish detail, which is simply a
>> matter of carefulness.
>
> But type inference binary breakage is opaque, because programmer
> didn't type a changed method signature.
Yes, and it can be fixed by having the programmer declare the return
type of his functions. That's something a code style tool can handle.
>> Yes, some individuals have been working on "type
>> theoretic stuff" -- though nothing of the like you mention -- but it's
>> been so far from the main focus that it hasn't even appeared anywhere
>> but papers and conferences -- certainly not the compiler itself.
>
> I perused some recent bug reports, and see effort to fix corner cases
> of the type system. I assume that could go on for years to perfect
> all.
Well, if there are bug reports about what you call "corner cases" of
the type system, and no bug reports about replacing Seq in unapplySeq
with something else, I think that should tell you something about
what's pragmatic and what's not. Bug reports arise from bad things
happening to code that is actually being used.
> I agree that is useful. I am pondering if such perfection can take
> temporary backseat in priority to more deeply rooted, fundamental
> issues that may have wider impact.
Oh, sure. I can see how that's going to play out. "Sorry, Twitter. I
know you are having trouble because some valid code you wrote is not
compiling as expected, but we decided that's a corner case, so we'll
put it in the back-burner. Meanwhile, I'm sure you'll be excited to
know that the next version of Scala will let you replace the "Seq" in
"unapplySeq" with an entirely different class of your own choosing!"
>
> For me so far, those might be (not sure):
>
> * lack of first-class union type
Oh-hoh! Yes, I'd like union types too, but how many of Tiobe's top-20
languages (http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html)
have them? How many Java programmers would even know how to use or
what to do with them?
> * inability to use the pattern matching with sequences without forcing
> the standard library.
And I'm yet to see a single use case of that.
>> So, basically, for the past year and a half Scala has been nothing but
>> pragmatic, which means that you have absolutely no knowledge
>> whatsoever about what is actually being done.
>
> I had the impression (even before his comments today) for example that
> Adriaan might be a key implementator on virtual pattern matcher and on
> any future union type, and for example I read a bug report where he is
> stumped for days on some obscure typing soundness bug. Perhaps my
> sample size was too small on where the priorities are in the
> *PRESENT*.
One developer is certainly too small, so it's no surprise that your
results are skewed when you pick one whose specialty and research is
Scala's type system. But let's not get that in the way. The soundness
bug stumping Adriaan, afaik, was related to the virtual pattern
matcher. The virtual pattern matcher, on the other hand, does this:
1. Replaces the current pattern matcher, which has many serious bugs,
with one that doesn't. It will have its own share of bugs, sure, but,
for now, it already solves a bunch of bugs that have, for instance,
gotten in the way of Yammer. I think getting pattern matching to
actually work beyond trivial cases is pretty pragmatic, and of far
surpassing importance than changing the return type of unapplySeq.
2. Turns pattern matching into method calls to a collection that can
then be replaced, just like for-comprehensions! If you think replacing
"Seq" in "unapplySeq" brings flexibility, sure you can appreciate the
gains in flexibility from being able to replace pattern matcher's
whole engine!
>
>> An *interested*
>> outsider, one who had bothered to give even a cursory look at what's
>> happening, would have a much better chance of gaining support. If for
>> no other reason, you'd at least look like you know what you were
>> talking about, which, right now, you are not.
>
> Additionally I have been aware there is a lot of effort on immutable
> collections (tries, etc), concurrency and parallelism libraries, etc..
Actually, ctries are mutable, not immutable. Yes, the developer whose
orientation and research is concurrency and parallelism is doing that.
There are others as well. You don't consider concurrency and
parallelism a pragmatic issue?
Also, RedBlack was replaced and someone wrote a mutable AVL so that
mutable TreeSet and TreeMap could be created. The interesting thing
here, however, is that these efforts were *not* from EPFL/TypeSafe,
but from people working on other stuff who actually felt the need to
do so. These are pragmatic by definition: they needed it, so they
wrote it.
>
> I have sense there is a lot of being thrown at the standard
> collections, and I am asking whether some effort should be thrown at
> de-coupling the standard library from the compiler.
Well, guess what, Scala collections are a big deal. Writing a new
alternative collection to the one written just two years ago would be
a waste of effort: if not the new library, certainly the effort spent
two years ago. Let others create entirely new collection libraries.
People have done so in Java, so what's stopping anyone in Scala? And
just as Scala has conversions to and from Java libraries, yes, people
will have need to provide conversions to and from Scala library. Which
you'll then be able to use when returning something with unapplySeq.
>
>> > source manifesto) claims that large numbers random monkeys banging on
>> > the open bazaar of code will beat the closed cathedral of experts by
>> > orders of magnitude (and this is apparently being proven in the real
>> > world results of open source vs. closed source). So I am pondering
>>
>> Note that Eric Raymond extrapolated that from a single example, and
>> ignoring counter examples to his theory. There's absolutely no
>> evidence or study that gives credence to what he said.
>
> Linux (pretty much every where except desktops)
> Firefox
> Android
> etc..
Err, yes? Do you mean quality, which is what Eric said, or popularity?
Since you didn't actually show any data whatsoever, I'll assume from
your example latter on that you mean popularity -- which is NOT what
Eric said. As far as popularity is concerned, Windows is more popular
than Linux, and Internet Explorer is more popular than Firefox. Linux
is used in a bunch of other places because it is free. Which is all
irrelevant, because Eric was talking about quality, not popularity.
Please, do not even bother to answer this point if you are not going
to mention a peer reviewed scientific study to back the argument.
Anecdotes are not data.
>
>> > that maybe we need to prioritize the issues of openness, i.e. de-
>> > conflation in this case.
>>
>> And what's that to do with what Eric said, no matter whether he's
>> right or wrong?
>
> Forcing the standard library, means in this case forcing one library
> culture of a dozen people or so, on the entire POTENTIAL community
> (thus which we may never attain).
Forcing the standard library? You are much mistaken! If you replace
the standard library with a completely different one than has a
scala.collection.Seq of your choice, I'm pretty sure the compiler will
happily use it.
>
> I think modularity is very important. That is why I think `trait` is a
> brilliant feature of Scala. I wish a light-weight trait was employed
> for pattern matching instead of a heavy-weight one that carries much
> of the standard library in with it.
>
>> > Exhibits:
>>
>> >http://www.catb.org/esr/comscore/(open source Android vs. closed
>> > source iPhone)
>>
>> iPhone still beats Android by a huge margin.
>
> Did you look at the charts?
>
> iPhone has 29% marketshare in USA, Android nearly 50%. Android came
> from behind with only 5% and raced past iPhone.
Popularity, Eric was talking about quality.
>
> The numbers are even more lopsided in the developing markets, which
> are more price and feature-diversity sensitive.
>
>
> [...]
>
>>
>> >http://ceylon-lang.org/documentation/1.0/faq/#when_will_it_be_ready
>>
>> Show me the money, as they say. That's just a wishlist.
>
> iPhone got the money, but lost the marketshare.
>
> For a language, marketshare is critical.
That's not what I meant. What I meant is that the thing in that faq DO
NOT EXIST. Period. The existing beta does not implement what they are
promising. Code that does not exist is no parameter of comparison to
code that exists.
>
>
>> >http://blog.joda.org/2011/11/scala-feels-like-ejb-2-and-other.html
>>
>> Coming from someone who never used Scala and actually complains about
>> stuff like where the type is related to the variable name.
>
> I would rather analyze the substance, than dismiss based on
> allegations about qualifications.
If I found substance to his arguments, I'd do so. Since he doesn't
know Scala, his arguments have no substance. I'm not going to repeat
here the analysis others have done already.
>
> [...]
>
>> Well, it's 2.9.1, and Scala 2.8 collections haven't stopped anyone so far.
>
> Sure?
Sure. If not, show me the evidence. You are the one trying to get
others to do a lot of work to accomplish something you deem desirable,
so you have the burden of proving it is so.
Shelby
> may not be obvious to you, do not live in a separate universe where
> they have no bearing. Everything is intertwined. The luxury of
> seeing soundness bugs as unimportant is a viewpoint which you can
> possess only because we do not.
eventually fix them all.
My point is that there may (or may not) be priorities that are more
urgent in the short-term.
For example (and not the only one or even the main one), a union type
would enable one to design a different sort of standard library. The
Option trait would disappear, and heterogeneous collections would be
typed-checked instead of subsuming to Any (unityped) and losing
runtime safety.
I assume we want to build a system where everything is not
intertwined, and want to migrate towards modularity without breaking
existing things.
Shelby
> errors. If I can't trust the type system, what you are asking for, I
> will have to spend MUCH time in overlly unnecessary tests and/or
> obscure, difficults bug to trace - time better spend correcting business
> bugs, adding features, etc. That's one of the main reason why I use
> Scala, and not Java or Gosu.
>
> As a really 'pragmatic'[1] user who (try to) earn money thanks to Scala,
> ignoring unsoudness would be a clear signal to make me wonder if really
> Scala worth it.
I understand that and have used similar reasoning myself. However, the
type system didn't protect me from being able to test whether a
collection of `Int` contains a `String`:
http://stackoverflow.com/questions/8360413/selectively-disable-subsumption-in-scala-correctly-type-list-contains
Whereas, a union type would fix that.
The more general point being that any type system is never complete,
else it isn't Turing-complete.
Thus the line of reasoning that says if we can make the existing type
system perfect, then we will have gained perfect protection, ignores
the basic facts about the universe:
http://copute.com/docs/Event.html#Theory
That is why I argue for being attuned to the big picture priorities
and not bury our heads.
Paul Phillips
> I assume we want to build a system where everything is not
> intertwined, and want to migrate towards modularity without breaking
> existing things.
"Everything is intertwined" was intended as a general observation that
often the right path to fixing bugs involves fixing other bugs. It
was not code for "everything is excessively coupled."
Shelby
[...]
> build process (slowly, but steadily),
> faster compilation,...
I wonder if no implementation in traits (separate out mixins) and then
parallel compilation might help, but I haven't delved into that idea
further yet. I see SBT does parallel compilation now.
Fits with my ideas about modularity and binary compatibility (c.f.
prior comments and links in this thread).
> Unfortunately for us (since we're academics after all), this has not left
> much time for "truly theoretic"
> work (though ECOOP recently accepted Lukas's paper on the effect system
> he's working on, and
> Greg&Nada will present the work they did on embedding JavaScript in Scala
> using Tiark's lightweight modular staging methodology).
> As you can see, even our theoretic work is firmly grounded in reality.
theory (c.f. your virtual pattern matcher below).
> Concerning your thoughts on the pattern matcher, here's some more
> information on the new pattern matcher I've been working on
> for the last couple
> months:https://groups.google.com/d/msg/scala-internals/k9_4yp2RS90/H_r6gUeGQ3kJ
nested extractors.
Category theory helps us generalize.
>
> In short, it should support most of your use case, either right now (it's
> in the milestones under the flag -Yvirtpatmat),
> or in 2.11 (for 2.10 we're aiming for feature parity with the old pattern
> matcher, minus a lot the latter's long-standing bugs).
indicated prior:
https://issues.scala-lang.org/browse/SUGGEST-18?focusedCommentId=55241&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-55241
I have seen some discussion that structural types have some problems.
I also don't like them philosophically (what is the type theory for
matching on method names?). I would have preferred lightweight traits
(nominal types), perhaps one method signature per trait and no
implementation. But that wouldn't co-exist with the existing code
(perhaps could make it a separate compile option or annotation though).
Shelby
> "Everything is intertwined" was intended as a general observation that
> often the right path to fixing bugs involves fixing other bugs. It
> was not code for "everything is excessively coupled."
'intertwined'. I was thinking that if we get a union type, then if
parties create new orthogonal libraries utilizing it, we could be
thinking about how to de-couple from the standard library, so we don't
have to break it when we can otherwise put breaking changes into
orthogonal libraries.
Shelby
> The code which calls the pattern matcher is not the code which defines
> the methods the pattern matcher will call, which I consider a big
> problem with this approach. In the end, of course, yes, one could have
> a compiler switch so that two completely different (and incompatible)
> implementations could be selected from. From my biased wording, I
> think one can easily see I do not consider this a good idea. :-)
automatically from typing information and interleave within the same
`match`. Compiler switch becomes unnecessary.
> >> to absolutely no advantage that I can
> >> see. Specifically:
>
> >> 1. One needs to implement three methods to define a Seq (iterator,
> >> apply and length), so the argument that it is hard holds no water.
>
> > I see dozens of methods and multiple inheritance:
>
> >http://www.scala-lang.org/api/current/index.html#scala.collection.Seq
>
> Yes, so what? Here, let's write one on the REPL:
>
> scala> class X extends Seq[Int] {
> | def length = 2
> | def apply(n: Int) = n match { case 0 => 1 case 1 => 2 }
> | def iterator = Iterator(1, 2)
> | }
> defined class X
>
> There, three methods.
also carries with it implementation. Thus afaics, the keyword
`extends` means the new library is not orthogonal to the standard
library. For example, the new library may wish for everything to be a
functor in a category theory sense, which may conflict with that pre-
existing implementation in `Seq`.
> >> 2. No example of a situation where a something might want to define a
> >> non-standard collection and return it through unapplySeq was given,
> >> and I, for one, can't think of one.
>
> > I (or others) might want to implement an entirely orthogonal
> > collections library, e.g. based on category theory:
>
> > [...]
>
> And using Seq with pattern matching prevents you from doing so how,
> exactly?
> Remember that unapplySeq is not defined on the collection
> class, but on a companion object. All the companion object needs to do
> is convert what it receives into a Seq.
[...]
> True, but how can people start "creating easy-to-use libraries" (your
> words) that will NOT be compatible with Java? What use it will be to
> the "ramping community from of huge pool of excited Java developers"?
for something that they can grasp. Amortize interopt pain with Java of
a more coarse-grained level.
For example, union types are superior to the boxing in an `Option`.
Avoiding silent subsumption and not being able to add a `Man` to a
collection of `Woman` without calling a method which explicitly
subsumes the union type of the members (not to `Any` but to the
union), is apparently more correct from a type theoretic sense and
also makes more common sense and doesn't leave the user bewildered.
I am not saying that Scala's Java compatibility is a waste. I am
talking about priorities, and I don't know whether Ceylon's assertion
is true, that their design gains coherency by giving up some
compatibility. Nevertheless, union type appears to be popular already.
Everyone who understands what it can do, seems to want it.
> >> If only type inference was the major cause of binary compatibility
> >> problems! Type inference is a smallish detail, which is simply a
> >> matter of carefulness.
>
> > But type inference binary breakage is opaque, because programmer
> > didn't type a changed method signature.
>
> Yes, and it can be fixed by having the programmer declare the return
> type of his functions. That's something a code style tool can handle.
problem the way Ceylon is, by forcing it always with the compiler in
the case the method signature has non-local exports. Why should I have
to run a separate tool to tell me the same thing the compiler can?
My alternative idea is to not put implementation in interfaces,
separate out into mixins, then never allow functions to reference non-
interfaces. Then don't change interfaces without changing their name.
So then you've get modular versioning on a fine-grained scale.
> Well, if there are bug reports about what you call "corner cases" of
> the type system, and no bug reports about replacing Seq in unapplySeq
> with something else, I think that should tell you something about
> what's pragmatic and what's not. Bug reports arise from bad things
> happening to code that is actually being used.
so then we can't get bug reports on them.
Isn't that a strawman.
> Oh, sure. I can see how that's going to play out. "Sorry, Twitter. I
> know you are having trouble because some valid code you wrote is not
> compiling as expected, but we decided that's a corner case, so we'll
> put it in the back-burner. Meanwhile, I'm sure you'll be excited to
> know that the next version of Scala will let you replace the "Seq" in
> "unapplySeq" with an entirely different class of your own choosing!"
Scala: Can't fix. It is correct from a type theoretic view with
subsumption.
> > For me so far, those might be (not sure):
>
> > * lack of first-class union type
>
> Oh-hoh! Yes, I'd like union types too, but how many of Tiobe's top-20
> languages (http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html)
> have them? How many Java programmers would even know how to use or
> what to do with them?
Hoare, the creator of `null`, said `null` was his $billion mistake.
I read that Ceylon is the first language to fix it. I hope Scala
remains the most advanced typing language on the JVM.
[...]
> One developer is certainly too small, so it's no surprise that your
> results are skewed when you pick one whose specialty and research is
> Scala's type system. But let's not get that in the way. The soundness
> bug stumping Adriaan, afaik, was related to the virtual pattern
> matcher. The virtual pattern matcher, on the other hand, does this:
>
> 1. Replaces the current pattern matcher, which has many serious bugs,
> with one that doesn't. It will have its own share of bugs, sure, but,
> for now, it already solves a bunch of bugs that have, for instance,
> gotten in the way of Yammer. I think getting pattern matching to
> actually work beyond trivial cases is pretty pragmatic, and of far
> surpassing importance than changing the return type of unapplySeq.
cost. Efficiently pay down an investment in the future.
> 2. Turns pattern matching into method calls to a collection that can
> then be replaced, just like for-comprehensions! If you think replacing
> "Seq" in "unapplySeq" brings flexibility, sure you can appreciate the
> gains in flexibility from being able to replace pattern matcher's
> whole engine!
and start modularizing things, so we don't end up like Java's monolith
of libraries (a massive intractable problem I read about recently).
> >> An *interested*
> >> outsider, one who had bothered to give even a cursory look at what's
> >> happening, would have a much better chance of gaining support. If for
> >> no other reason, you'd at least look like you know what you were
> >> talking about, which, right now, you are not.
>
> > Additionally I have been aware there is a lot of effort on immutable
> > collections (tries, etc), concurrency and parallelism libraries, etc..
>
> Actually, ctries are mutable, not immutable. Yes, the developer whose
> orientation and research is concurrency and parallelism is doing that.
recent research paper.
> There are others as well. You don't consider concurrency and
> parallelism a pragmatic issue?
(cost prohibitive) to correct later.
> Also, RedBlack was replaced and someone wrote a mutable AVL so that
> mutable TreeSet and TreeMap could be created. The interesting thing
> here, however, is that these efforts were *not* from EPFL/TypeSafe,
> but from people working on other stuff who actually felt the need to
> do so. These are pragmatic by definition: they needed it, so they
> wrote it.
they also want to integrate with libraries orthogonal and modular from
the standard library, without losing pattern matching.
> > I have sense there is a lot of being thrown at the standard
> > collections, and I am asking whether some effort should be thrown at
> > de-coupling the standard library from the compiler.
>
> Well, guess what, Scala collections are a big deal. Writing a new
> alternative collection to the one written just two years ago would be
> a waste of effort: if not the new library, certainly the effort spent
> two years ago.
How can you know that?
> Let others create entirely new collection libraries.
> People have done so in Java, so what's stopping anyone in Scala? And
> just as Scala has conversions to and from Java libraries, yes, people
> will have need to provide conversions to and from Scala library. Which
> you'll then be able to use when returning something with unapplySeq.
[...]
> >> Note that Eric Raymond extrapolated that from a single example, and
> >> ignoring counter examples to his theory. There's absolutely no
> >> evidence or study that gives credence to what he said.
>
> > Linux (pretty much every where except desktops)
> > Firefox
> > Android
> > etc..
>
> Err, yes? Do you mean quality, which is what Eric said, or popularity?
> Since you didn't actually show any data whatsoever, I'll assume from
> your example latter on that you mean popularity -- which is NOT what
> Eric said. As far as popularity is concerned, Windows is more popular
> than Linux
significantly less popular on servers and embedded.
>, and Internet Explorer is more popular than Firefox.
http://www.w3schools.com/browsers/browsers_stats.asp
Also IE doesn't run on Android, which I think has sold more units than
PCs. This perhaps explains the rise of Chrome.
http://techcrunch.com/2012/02/17/study-windows-web-browsing-market-share-down-nearly-10-over-past-6-months/
> Linux
> is used in a bunch of other places because it is free.
advantages of the Inverse Commons that Eric explained in The Magic
Cauldron. Second Linux is more popular on servers and embedded because
it is more reliable and because of the technical/support benefits of
open source.
> Which is all
> irrelevant, because Eric was talking about quality, not popularity.
evident by his excitement about the Smartphone Wars, where Linux has
finally become the most popular end user OS in the world.
[...]
> >> > that maybe we need to prioritize the issues of openness, i.e. de-
> >> > conflation in this case.
>
> >> And what's that to do with what Eric said, no matter whether he's
> >> right or wrong?
>
> > Forcing the standard library, means in this case forcing one library
> > culture of a dozen people or so, on the entire POTENTIAL community
> > (thus which we may never attain).
>
> Forcing the standard library? You are much mistaken! If you replace
> the standard library with a completely different one than has a
> scala.collection.Seq of your choice, I'm pretty sure the compiler will
> happily use it.
standard library. Also you would need to know about all dependencies
that the compiler was depending on, which means your library would
have to track code changes in the compiler. That sounds like the
antithesis of modularity.
The point of declaring interfaces, is declaring the invariants. A
nominal name is not an invariant.
[...]
> > iPhone got the money, but lost the marketshare.
>
> > For a language, marketshare is critical.
>
> That's not what I meant. What I meant is that the thing in that faq DO
> NOT EXIST. Period. The existing beta does not implement what they are
> promising. Code that does not exist is no parameter of comparison to
> code that exists.
challenge Scala.
Perhaps Scala can do a few things early to make sure Ceylon has no
substantial advantages. If Scala waits until Ceylon releases, Ceylon
may have those advantages for a significant period of time.
I am not saying to tilt at every vaporware windmill, but perhaps there
are some key things that we need to do any way, because they make
enormous and broad sense. Or maybe not. But we are discussing it to
get our reasoning clear and open.
[...]
> If I found substance to his arguments, I'd do so. Since he doesn't
> know Scala, his arguments have no substance. I'm not going to repeat
> here the analysis others have done already.
w.r.t. to community.
I found several points of substance in his arguments:
1. Scala's subsumption is a semantic hole and a runtime typing hole.
2. Scala is in some respects a write-only language, because there are
too many ways to express the same thing (e.g. 9 ways to write a
function) and because of design choices to make DSLs blend with the
language. I am not proposing we change this in Scala. I may (or may
not) create a language (Copute) that compiles to Scala, which is
cleaner (and more limited), but still we would be coding in Scala for
a significant portion.
3. Scala is missing the ability to declare functions pure.
4. Scala's community is academic-centric, thus so are some of the
design decisions, priorites, and style of dialogue.
> >> Well, it's 2.9.1, and Scala 2.8 collections haven't stopped anyone so far.
>
> > Sure?
>
> Sure. If not, show me the evidence.
> You are the one trying to get
> others to do a lot of work to accomplish something you deem desirable,
> so you have the burden of proving it is so.
I didn't force anyone to do anything. I came here to discuss
priorities. And I didn't say that I wouldn't do something.
Btw, I still respect you and hope we will eventually come to
understand each other's perspectives on this issue.
Shelby
> > Well, guess what, Scala collections are a big deal. Writing a new
> > alternative collection to the one written just two years ago would be
> > a waste of effort: if not the new library, certainly the effort spent
> > two years ago.
>
> Disagree on both points.
disagree that the effort on the libraries (new or recent) would be
wasted.
Ittay Dror
Rex Kerr
On Mar 8, 12:25 am, Daniel Sobral <dcsob...@gmail.com> wrote:That is one way to attack the problem. I would prefer to attack the
> >> If only type inference was the major cause of binary compatibility
> >> problems! Type inference is a smallish detail, which is simply a
> >> matter of carefulness.
>
> > But type inference binary breakage is opaque, because programmer
> > didn't type a changed method signature.
>
> Yes, and it can be fixed by having the programmer declare the return
> type of his functions. That's something a code style tool can handle.
problem the way Ceylon is, by forcing it always with the compiler in
the case the method signature has non-local exports. Why should I have
to run a separate tool to tell me the same thing the compiler can?
Because forcing boilerplate reduces productivity in important use cases, such as scripting. Right now, I use Scala for the vast majority of my scripting tasks specifically because it doesn't make me pedantically repeat easily-inferred types just to try to help me catch errors that might crop up in enterprise-scale projects. I can type out a couple of files quickly, do some stuff in the REPL, and bingo!--it works, and with type safety.
When a feature is not a net win everywhere, such as requiring more redundant boilerplate to try to protect people from themselves, then I favor using external tools to provide the scolding and hand-holding rather than building them in. (Or by having them as compiler options or a compiler plugin, if there's sufficient time to add that feature to the compiler and it's sufficiently general-purpose.)
My alternative idea is to not put implementation in interfaces,
separate out into mixins, then never allow functions to reference non-
interfaces. Then don't change interfaces without changing their name.
So then you've get modular versioning on a fine-grained scale.
This sounds like a refactoring nightmare waiting to happen. Care to explain in more detail how this is helpful?
> If I found substance to his arguments, I'd do so. Since he doesn't
> know Scala, his arguments have no substance. I'm not going to repeat
> here the analysis others have done already.
Afaics, that is myopia and an example of what he is talking about
w.r.t. to community.
I found several points of substance in his arguments:
1. Scala's subsumption is a semantic hole and a runtime typing hole.
Not sure what you mean by "hole". It is correct, but union types would make it potentially better. Having invariant collections available for those cases where it's the right thing to do would also be nice. In the meantime, declaring the result type once it matters (e.g. method return value) is a good strategy to use to make sure you don't have unwanted changes in collection type.
3. Scala is missing the ability to declare functions pure.
This is true, but...
4. Scala's community is academic-centric, thus so are some of the
design decisions, priorites, and style of dialogue.
...this is exactly the opposite point. Pure functions are a much bigger deal academically than elsewhere. I happen to think that there's nothing wrong with an academic-centric point of view to some extent; what is the point of an academic treatment at all if not to think more deeply about the theoretical structure of programming and come up with solutions that you might not stumble across if driven by immediate pragmatism? But one still needs to blend pragmatic concerns with theoretical elegance, and it's far from clear to me that compiler-enforced pure functions are anywhere near the top of that list for the overwhelming majority of programming projects.
I'd like, eventually, to have some sort of compiler-assisted tag scheme which one could use for annotating function purity and/or units and/or IO and/or other stuff (which IMO is fundamentally much more powerful than the clunky way this is done with monads). But I would understand if someone dismissed as not serious a blog that made a big deal about a lack of compiler-enforced this-is-a-pure-function annotations, given the other important issues on the table.
--Rex
Sciss
[...]
> For example, union types are superior to the boxing in an `Option`.
They are not superior. They are more appropriate in some situations and less in others.
- Option is a common interface, i.e. you can do map, flatMap. I don't think that A | None could do that without pimping.
- The union type are useful for type safe data structures which need to distinguish sub types which still have an _intersection_ of common interface.
To give an example : I have a 'horizontal and vertical' tree like structure which distinguishes left and top elements from inner elements. I ended up having the following traits:
Branch, BranchLike, Child, ChildBranch, Empty, Leaf, LeafOrEmpty, Left, LeftBranch, LeftChild, LeftChildOption, LeftNonEmpty, LeftNonEmptyChild, LeftTopBranch, Next, NextOption, NonEmpty, NonEmptyChild, RightBranch, RightChild, RightChildOption, RightNonEmptyChild, RightTopBranch, TopBranch -- a real pita obviously, all to get a nice compile time type check for the whole structure.
[...]
> Hoare, the creator of `null`, said `null` was his $billion mistake.
>
> I read that Ceylon is the first language to fix it. I hope Scala
> remains the most advanced typing language on the JVM.
that's none sense. you can fix NPRs in scala if you want to. if you want safety from possible null returns from java libraries, wrap them appropriately. simple.
i think most bugs with respect to val initializations have been fixed. as far as my own projects are concerned, i haven't seen NPEs in a while.
[...]
> I found several points of substance in his arguments:
>
> 1. Scala's subsumption is a semantic hole and a runtime typing hole.
>
> 2. Scala is in some respects a write-only language, because there are
> too many ways to express the same thing (e.g. 9 ways to write a
> function) and because of design choices to make DSLs blend with the
> language. I am not proposing we change this in Scala. I may (or may
> not) create a language (Copute) that compiles to Scala, which is
> cleaner (and more limited), but still we would be coding in Scala for
> a significant portion.
no problem with that, create a new language. Scala will always be embraced because of its bazaar approach, other people will use Python, etc.
> 3. Scala is missing the ability to declare functions pure.
there are multiple options. there is a compiler plugin being developed for @pure, there is the uniqueness and borrowing extension, there is ongoing research into a scala effect systems, there is actors, there is scala-stm, etc. not just the io monad. it boils down to your responsibility to design the system in a consistent way.
>
> 4. Scala's community is academic-centric, thus so are some of the
> design decisions, priorites, and style of dialogue.
one of the biggest plus points for Scala IMO. that's why languages like haskell or scala will survive in the high-level realm; system languages like C will survive in the low level realm, because they are fast, not because they are particularly pretty or academically informed. those that are neither won't have a sustainable development (ruby, groovy, perl, ...). they are well founded and they are constantly improved based on actual research instead of messing around with them. see previous point for an example. scala has the big advantage over other languages such as mozart that it stands on two legs, it is also commercially supported now. try to make a list with other languages having these two legs...
best, .h.h.
Shelby
> you don't need to copy. just create 'class X extends Seq' that implements
> the three methods in terms of your data collection. e.g, 'def length = this.length'.
> you have an overhead of creating a new instance instead of returning 'this', but i think this is not really an issue.
in a new companion instance by reference, where the new instance
extends Seq.
The implementation in Seq is complete, if the implementation for the
three methods are provided.
That may work as a special case to interopt with the existing pattern
matcher, but afaics companion objects defeat subtyping, thus in
general will break modularity of composition. Synonomous with
Adriaan's effort to virtualize (to user land) what the pattern
matching engine does, I am thinking we should also virtualize the
types that Extractors can return, as long they adhere to a minimum set
of structural members or nominal trait(s).
Shelby
[...]
> > > type of his functions. That's something a code style tool can handle.
>
> > That is one way to attack the problem. I would prefer to attack the
> > problem the way Ceylon is, by forcing it always with the compiler in
> > the case the method signature has non-local exports. Why should I have
> > to run a separate tool to tell me the same thing the compiler can?
>
> Because forcing boilerplate reduces productivity in important use cases,
> such as scripting. Right now, I use Scala for the vast majority of my
> scripting tasks specifically because it doesn't make me pedantically repeat
> easily-inferred types just to try to help me catch errors that might crop
> up in enterprise-scale projects. I can type out a couple of files quickly,
> do some stuff in the REPL, and bingo!--it works, and with type safety.
dependencies, but does that make binary compatibility not matter?
> When a feature is not a net win everywhere, such as requiring more
> redundant boilerplate to try to protect people from themselves, then I
> favor using external tools to provide the scolding and hand-holding rather
> than building them in. (Or by having them as compiler options or a
> compiler plugin, if there's sufficient time to add that feature to the
> compiler and it's sufficiently general-purpose.)
then it is a formalism of the language (separate compilation requires
more boilerplate). Then your build can specify if compiling for
separate compilation or not.
> > My alternative idea is to not put implementation in interfaces,
> > separate out into mixins, then never allow functions to reference non-
> > interfaces. Then don't change interfaces without changing their name.
> > So then you've get modular versioning on a fine-grained scale.
>
> This sounds like a refactoring nightmare waiting to happen. Care to
> explain in more detail how this is helpful?
original name and interface.
When programming for separate compilation, i.e. modularity and wide-
area (enterprise or wider, i.e. whole internet) collaboration, then an
interface needs to be a static thing. If you need to change it, give
it a new version, which is just as easily accomplished by creating a
new copy with a new name. The original interface and name do not
disappear.
Implementation should never be in the interface. Want me to try to
justify that?
[...]
>
> > 1. Scala's subsumption is a semantic hole and a runtime typing hole.
>
> Not sure what you mean by "hole".
1. I can ask if a `Man` exists in a collection of only `Woman`. Type
system does not complain due to implicit subsumption to Any.
2. I can add a `Man` to a collection of `Woman`, I get a collection
implicitly subsumed to `Any`, without afaik any possible way to get a
warning, unless I make the collection invariant. I then have a runtime
typing hole, because I can downcast an `Any` to anything with `match`
and cause an exception to be thrown.
In nutshell, afaics the lack of union type means Scala is a unityped
(i.e. dynamic and not static) language in this case.
> It is correct, but union types would
> make it potentially better. Having invariant collections available for
> those cases where it's the right thing to do would also be nice. In the
> meantime, declaring the result type once it matters (e.g. method return
> value) is a good strategy to use to make sure you don't have unwanted
> changes in collection type.
parameterization.
> > 3. Scala is missing the ability to declare functions pure.
>
> This is true, but...
>
> > 4. Scala's community is academic-centric, thus so are some of the
> > design decisions, priorites, and style of dialogue.
>
> ...this is exactly the opposite point. Pure functions are a much bigger
> deal academically than elsewhere. I happen to think that there's nothing
> wrong with an academic-centric point of view to some extent; what is the
> point of an academic treatment at all if not to think more deeply about the
> theoretical structure of programming and come up with solutions that you
> might not stumble across if driven by immediate pragmatism? But one still
> needs to blend pragmatic concerns with theoretical elegance,
shift some so less theoretic programmers are able to grasp Scala
easily. And probably I lean towards documentation alone won't be
sufficient for that goal.
Any way, I only said I empathized with the goal in this sense of 80/20
rule, but I didn't agree that Scala is completely off course, as he
implied.
OTOH, maybe Scala is best to be positioned (where it is) as a language
somewhere between Haskell's very obtuse academic focus and say for
example Ceylon's Java "average Joe" focus.
> and it's far
> from clear to me that compiler-enforced pure functions are anywhere near
> the top of that list for the overwhelming majority of programming projects.
> I'd like, eventually, to have some sort of compiler-assisted tag scheme
> which one could use for annotating function purity and/or units and/or IO
> and/or other stuff (which IMO is fundamentally much more powerful than the
> clunky way this is done with monads). But I would understand if someone
> dismissed as not serious a blog that made a big deal about a lack of
> compiler-enforced this-is-a-pure-function annotations, given the other
> important issues on the table.
in my comments on his blog.
Kevin Wright
Implementation should never be in the interface. Want me to try to
justify that?
Seth Tisue
> Because forcing boilerplate reduces productivity in important use cases,
> such as scripting. Right now, I use Scala for the vast majority of my
> scripting tasks specifically because it doesn't make me pedantically repeat
> easily-inferred types just to try to help me catch errors that might crop up
> in enterprise-scale projects. I can type out a couple of files quickly, do
> some stuff in the REPL, and bingo!--it works, and with type safety.
+1. Couldn't agree more.
"Scalable" should mean small too, not big only.
--
Seth Tisue | Northwestern University | http://tisue.net
lead developer, NetLogo: http://ccl.northwestern.edu/netlogo/
Shelby
> I perused some recent bug reports, and see effort to fix corner cases
> of the type system. I assume that could go on for years to perfect
> all.
>
> I agree that is useful. I am pondering if such perfection can take
> temporary backseat in priority to more deeply rooted, fundamental
> issues that may have wider impact.
>
> For me so far, those might be (not sure):
>
> * lack of first-class union type
intuition might not be totally off base.
http://groups.google.com/group/scala-language/msg/4092bcb0ffd513be
It might end up that implementing union type fixes the type system,
and some intractable bugs melt away. That is just conjecture and I
haven't related any specific bugs.
> * inability to use the pattern matching with sequences without forcing
> the standard library.
from modularity is exponential in the installed base and derivative
dependent libraries, and thus intractable to recover from later.
Ittay Dror
object MyCollection {
def unapplySeq[A](col: MyCollection[a]) = Some(SeqWrapper(this))
}
where SeqWrapper extends Seq an implements the 3 required methods by
delegating to MyCollection's methods.
ittay
Shelby
I re-iterate that afaik in general, wrapper delegation breaks
subtyping.
Delegation for Java: API or Language Extension?, G Kniesel
"a series of recent theoretical papers claim that a type-safe
combination of delegation with subtyping, let along dynamic
delegation, is impossible ([1],[5],[6])"
Using Prototypical Objects to Implement Shared Behavior in Object
Oriented Systems, Henry Lieberman
http://web.media.mit.edu/~lieber/Lieberary/OOP/Delegation/Delegation.html
"3. Inheritance implements sets, delegation implements prototypes"
Dynamic Object-Based Inheritance with Subtyping, G Kniesel
http://roots.iai.uni-bonn.de/research/darwin/downloads/darwinDiss.pdf
"wrappers ... fall short ... operations need to be applied to the
composite object, rather than to the component object alone ..."
Ittay Dror
Shelby wrote:
On Mar 8, 4:51 pm, Ittay Dror <ittay.d...@gmail.com> wrote:On Mar 8, 12:16 am, Shelby <she...@coolpage.com> wrote:On Mar 8, 4:26 am, Ittay Dror <ittay.d...@gmail.com> wrote:you don't need to copy. just create 'class X extends Seq' that implements the three methods in terms of your data collection. e.g, 'def length = this.length'. you have an overhead of creating a new instance instead of returning 'this', but i think this is not really an issue.So if I understand correctly, you are saying wrap the current instance in a new companion instance by reference, where the new instance extends Seq.The implementation in Seq is complete, if the implementation for the three methods are provided.That may work as a special case to interopt with the existing pattern matcher, but afaics companion objects defeat subtyping, thus in general will break modularity of composition. Synonomous with Adriaan's effort to virtualize (to user land) what the pattern matching engine does, I am thinking we should also virtualize the types that Extractors can return, as long they adhere to a minimum set of structural members or nominal trait(s).class MyCollection[A].... object MyCollection { def unapplySeq[A](col: MyCollection[a]) = Some(SeqWrapper(this))
} where SeqWrapper extends Seq an implements the 3 required methods by delegating to MyCollection's methods.Agreed, and that is what I wrote that I assumed you meant. I re-iterate that afaik in general, wrapper delegation breaks subtyping.
Shelby
http://www.ibm.com/developerworks/java/library/j-cq04256/#N10215
I believe that section is making the point that instability increases
as abstraction decreases, and that when you must lower abstraction at
least try to stay as close as possible to the main line for that
tradeoff.
I don't think the section is claiming that reducing abstraction is
desirable.
Putting implementation in the interface is both Afferent and Efferent
coupling. In my limited experimentation thus far, I have yet to find a
case where it is necessary if one is designing from a clean slate.
There seems to be always a way to restructure the design into a set of
abstract interfaces. I am interested in counter examples.
The catch seems to be distributed systems, i.e. separate compilation.
Then we can't always restructure the design optimally. But I don't
think that is an argument for not structuring for abstraction in all
the cases that we can. It will then reduce the cases where we can't
later on.
Kevin Wright
On Mar 8, 7:42 am, Kevin Wright <kev.lee.wri...@gmail.com> wrote:
I read the "Distance from the main" section:> On 7 March 2012 23:06, Shelby <she...@coolpage.com> wrote:
>
> > Implementation should never be in the interface. Want me to try to
> > justify that?
>
> That's just a rule. Like many rules, it exists to be bent :)
>
> In practice, things are a little more complicated, and there's a whole
> sliding scale of exactly how stable/abstract things ought to be:
>
> http://www.ibm.com/developerworks/java/library/j-cq04256/
http://www.ibm.com/developerworks/java/library/j-cq04256/#N10215
I believe that section is making the point that instability increases
as abstraction decreases, and that when you must lower abstraction at
least try to stay as close as possible to the main line for that
tradeoff.
I don't think the section is claiming that reducing abstraction is
desirable.
Shelby
> > Because forcing boilerplate reduces productivity in important use cases,
> > such as scripting. Right now, I use Scala for the vast majority of my
> > scripting tasks specifically because it doesn't make me pedantically repeat
> > easily-inferred types just to try to help me catch errors that might crop up
> > in enterprise-scale projects. I can type out a couple of files quickly, do
> > some stuff in the REPL, and bingo!--it works, and with type safety.
>
> +1. Couldn't agree more.
>
> "Scalable" should mean small too, not big only.
signal.
If the tradeoff is between disallowing type inference for all
externally accessible interface signatures, and a tool to check for
changes to such signatures between two versions, the latter allows
less verbosity while still providing a check for binary compatibility
breakage between versions.
Ceylon's claim is that such exported signatures should be self-
documenting, so the consumer of the interface doesn't have to infer
the signature from the implementation.
This is also related to the issue of conflating interface and
implementation. And also to the issue of code being hard to read, i.e.
if a consumer of an interface has to understand the implementation in
order to know the interface signature.
Thus I just can't see the need for the external tool. Interfaces
requiring binary compatibility (wide-area, separate compilation,
composition) should not require the consumer of them to study the
implementation. Rather it seems we need is an annotation or compiler
switch to say whether files are for wide-area separate compilation or
for local-area development (small teams) scripting.
Rex Kerr
On Mar 8, 5:10 am, Rex Kerr <icho...@gmail.com> wrote:
[...]
> > > Yes, and it can be fixed by having the programmer declare the returnBy 'scripting' in this context you mean you always recompiling all the
> > > type of his functions. That's something a code style tool can handle.
>
> > That is one way to attack the problem. I would prefer to attack the
> > problem the way Ceylon is, by forcing it always with the compiler in
> > the case the method signature has non-local exports. Why should I have
> > to run a separate tool to tell me the same thing the compiler can?
>
> Because forcing boilerplate reduces productivity in important use cases,
> such as scripting. Right now, I use Scala for the vast majority of my
> scripting tasks specifically because it doesn't make me pedantically repeat
> easily-inferred types just to try to help me catch errors that might crop
> up in enterprise-scale projects. I can type out a couple of files quickly,
> do some stuff in the REPL, and bingo!--it works, and with type safety.
dependencies, but does that make binary compatibility not matter?
I mean, "Why are you even bothering me with thinking about "recompiling all the dependencies" and "binary compatibility"?". You're looking at it from an almost entirely unhelpful perspective.
I want something done, now. I type out the minimal amount code that does it. I hit enter. It is done.
This is the essence, to me, of scripting. Whether or not dependencies are recompiled is _entirely_ irrelevant except inasmuch as it helps with things being "done, now" and lets me "type out the minimal amount of code".
Likewise with binary compatibility, except inasmuch as if I've already typed it out before, I want it to work again without much fiddling.
For example, suppose I have seven dozen directories with several hundred files each, all named according to some scheme, and I notice that whoever created them made a systematic naming error, "_N2_" should be "_XJ30_". Do I think about recompiling dependencies and binary compatibility? No, I type
(new File(".")).listRecursive(_.getName contains "_N2_").foreach{
val g = new File(f.getParentFile, f.getName.replace("_N2_","_XJ30_"))
println(g.getPath)
}
and check to make sure the files I noticed are getting renamed. Then I switch the println to
f.renameTo(g)
and the problem is fixed. 90 seconds later or whatever, and I've fixed dozens of typos across a complex directory tree. (Thanks in part to the listRecursive method that I've enriched File with.)
My prior summary was slightly unclear. I don't mean discard the
> > My alternative idea is to not put implementation in interfaces,
> > separate out into mixins, then never allow functions to reference non-
> > interfaces. Then don't change interfaces without changing their name.
> > So then you've get modular versioning on a fine-grained scale.
>
> This sounds like a refactoring nightmare waiting to happen. Care to
> explain in more detail how this is helpful?
original name and interface.
When programming for separate compilation, i.e. modularity and wide-
area (enterprise or wider, i.e. whole internet) collaboration, then an
interface needs to be a static thing. If you need to change it, give
it a new version, which is just as easily accomplished by creating a
new copy with a new name. The original interface and name do not
disappear.
Agreed. This is relatively uncontroversial, if the usage is high enough. See Java. Note, however, that there is a downside: you end up with a horrible mess of stupid interfaces that piles of people are using because it was that way before, and/or they have the nice names.
There's a balance to be had, which depends on the breadth of adoption, the magnitude of the flaws in the previous interface, the availability of alternate names, and the manpower available to ensure that there is a working useful implementation of something that adheres to that interface.
Implementation should never be in the interface. Want me to try to
justify that?
Yes, please. It sounds like you are saying, "Interfaces should only expose things upon which there are no useful non-trivial computations possible or desirable."
[...]
> > I found several points of substance in his arguments:As I mentioned before:
>
> > 1. Scala's subsumption is a semantic hole and a runtime typing hole.
>
> Not sure what you mean by "hole".
1. I can ask if a `Man` exists in a collection of only `Woman`. Type
system does not complain due to implicit subsumption to Any.
2. I can add a `Man` to a collection of `Woman`, I get a collection
implicitly subsumed to `Any`, without afaik any possible way to get a
warning, unless I make the collection invariant. I then have a runtime
typing hole, because I can downcast an `Any` to anything with `match`
and cause an exception to be thrown.
In nutshell, afaics the lack of union type means Scala is a unityped
(i.e. dynamic and not static) language in this case.
Okay, I understand the point. I wouldn't call that a "hole" precisely, but rather "an unavoidable consequence of allowing an object-oriented model". If you are complaining that Scala allows object oriented programming, yes, indeed, it's got "holes".
I don't see how you can do that and not force invariant type
> It is correct, but union types would
> make it potentially better. Having invariant collections available for
> those cases where it's the right thing to do would also be nice. In the
> meantime, declaring the result type once it matters (e.g. method return
> value) is a good strategy to use to make sure you don't have unwanted
> changes in collection type.
parameterization.
Agreed (or violate LSP). So have invariant type parameterization. Maybe some people will have to drop in an extra type annotation to get the collection of appropriate type to begin with.
--Rex
Shelby
> On 8 March 2012 10:33, Shelby <she...@coolpage.com> wrote:
> > I don't think the section is claiming that reducing abstraction is
> > desirable.
>
> I am. Having abstraction (in the form of an artificial class/interface
> split) when it isn't needed is just pure boilerplate. This problem is
> worse in java where you're forced to use a new file for each interface, so
> the proliferation of needless interfaces just clutters your source folder
> and makes it harder in IDEs to drill down from a method invocation to where
> it's defined.
implementation is orthogonal to the interface?
The interface can be generated automatically from the class
declaration. It is the `mixin` (my proposal) that must never be in the
interface (as is the case with `trait`, which I am thinking is a
design error).
This relates to my prior post about binary compatibility, in that if a
class is going to be consumed for wide-area separate compilation, then
the compiler should enforce that the interface is not inferred from
the implementation.
Apparently, Gavin King and I think alike on this one (although there
are many decisions he has that I hate, such as putting VERY LONG
signatures before the variable name and verbose english boilerplate).
> Of course... abstracting over functions and the composition of functions
> (as opposed to abstracting over classes) is a different kettle of fish
> entirely, and is very much to be encouraged.
abstraction over classes vs. functions are duals.
Shelby
> In the general case, yes, but why is it important here? SeqWrapper just delegates calls.
breaking subtyping has unseen future costs.
Daniel Sobral
I'm sorry, but doesn't the paper say that *automatic* wrappers with
subtyping is impossible? All objects are wrappers, so to say wrappers
and subtyping is impossible is to deny the existence of object
orientation.
Francois
> Hi Francios,
(Francois)
> [...]
> The more general point being that any type system is never complete,
> else it isn't Turing-complete.
>
Scala type system is Turing-complete, see:
http://apocalisp.wordpress.com/2010/06/08/type-level-programming-in-scala/
(or just Google for it)
Thanks,
--
Francois ARMAND
http://fanf42.blogspot.com
http://www.normation.com
Shelby
leverage them in future, other than just the compiler's pattern
matching. Perhaps some of these hypothetical libraries would also be
able to offer features where the extractor is on functor. If the
functor was achieved via subtyping (as I have shown how to do, unlike
Scalaz), then the extractors employing wrappers would not interopt.
Shelby
On Mar 8, 9:02 pm, Francois <fan...@gmail.com> wrote:
> On 07/03/2012 18:02, Shelby wrote:
>
> > Hi Francios,
>
> (Francois)
>
> > [...]
> > The more general point being that any type system is never complete,
> > else it isn't Turing-complete.
>
writing that, that I had misspoken, and decided not to correct it
(hoping readers would realize what I meant). What I meant was that for
the language to be Turing-complete, no type system will ever be
complete. I had in mind dependent typing, e.g. Epigram which is not
Turing-complete.
http://en.wikipedia.org/wiki/Turing_completeness#Non-Turing-complete_languages