[SIP 18] New SIP: Modularizing language features
martin odersky
language features under an import flag. I expect we'll have an
interesting discussion about this one.
So, please, fire away!
Here's the link to the SIP:
https://docs.google.com/document/d/1nlkvpoIRkx7at1qJEZafJwthZ3GeIklTFhqmXMvTX9Q/edit
Cheers
-- Martin
Jon Steelman
Heather Miller
Rex Kerr
import language.{postfixOps, structuralTypes, higherKinded, existentials}
Instead, I propose that, if there is a significant demand for limiting language features, that we flip the sign: instead of requiring import language.structuralTypes to get structural types, import language.disable.structuralTypes if you want them to go away. If you're in a development environment with strict coding rules anyway, you probably will have half a screen of stuff at the top of every file anyway (copyright notice, imports of a bunch of libraries, etc.), so one more line doesn't really hurt that much.
Features that really are experimental could be kept as an import language.enable._ so that people are alerted that they shouldn't trust how they behave. For example, macros would be a good candidate here since they're not ironed out. Furthermore, some things that are compiler flags could be moved to language.enable (such as continuations), where they arguably more sensibly belong.
(Actually, even more sensibly, they'd be annotations on the file.)
--Rex
Tom Switzer
+1. Well said.
martin odersky
> I'm unable to comment within the document. It is View only unlike prior
> SIPs.
That was an oversight. i just turned comments on.
-- Martin
martin odersky
> The big problem with this proposal is that it threatens to add a bunch of
> boilerplate to every single existing Scala file and program and REPL
> session. With all these things off by default, I would argue that Scala is
> severely curtailed; why make everyone in every file and script type in
> "import language._" in order to enable the features that make you use Scala
> instead of something else? And then, you get not only the features of a
> present-day typical Scala distribution, but early versions of features that
> are not yet fixed; if you want "full power but stable" Scala, you end up
> with quite a mouthful:
> import language.{postfixOps, structuralTypes, higherKinded, existentials}
>
(as opposed to specialized) Scala use. To take the Scala compiler with
100K lines as an example: As far as I know, there are no uses of
postfix ops, there is one use of structural types (and we are trying
to get rid of it), there are no uses of higher-kinded types, and any
use of complex existentials, if it exists, I would deem a mistake. So
from that perspective, yes, I would value having to import features
explicitly so that I am warned from using these things accidentally.
For less experienced people I believe the need is even stronger. If
you really have a Scala style that uses all four of these features
regularly then you would be best off just redefining your scalac
command to enable all features by default.
Cheers
-- Martin
Debasish Ghosh
Debasish Ghosh
http://manning.com/ghosh
Twttr: @debasishg
Blog: http://debasishg.blogspot.com
Code: http://github.com/debasishg
Sean Parsons
http://seantparsons.blogspot.co.uk/2012/01/compiler-flags-in-scala.html
Daniel Spiewak
Ok, maybe that was too strong. Honestly, I don't see anything positive coming out of a change like this. As Rex pointed out, it just creates import/build-config boilerplate for, well, everyone. Even for the few who don't use these features (knowingly or unknowingly), these switches have no beneficial effect. You are hiding a language feature that these people haven't seen. By definition, this will be a completely invisible change for a few people and a significantly detrimental change for a majority.
It's also worth noting that many people who use these features do so without knowing it. Existentials and structural types are a particularly strong example of this, but type constructors are also quite common. People who know what these features entail will have already turned them on, while those who don't know will receive a mysterious warning/error on seemingly-innocuous code. Consider the sheer volume of Scala snippets and explanations which will be instantly invalidated.
On a more philosophical level, "hiding" language features like this only serves to fragment the community. It makes life very difficult in venues like mailing lists and Stackoverflow, where users will not only need to specify their language (Scala) but their language "level" (Nerfed Edition). By definition, newcomers to the language will not understand this kind of fragmentation, given that no other language has anything quite like this, and will thus be extremely confused when correct answers to seemingly-simple questions (like extending collections) lead to warnings, or worse, errors.
Pragmatically, I have to ask the question: who are we trying to benefit with this change, and is it such a large benefit as to merit inconveniencing the largest (and most vested) segment of your user-base?
Daniel
Bernd
It has been argued if this is necessary - it's a double edged sword. Removing tools that are a good solution for a given problem will lead to suboptimal solutions. But there is also the perspective that tools used unwisely are also a problem.
The first case will most probably lead to working solutions which are suboptimal and can be enhanced. That's daily business. No big deal.
The second case might lead to tangled or cryptic code, bumpy compiler rides and maybe failure... and another blog post that complains about "scala the complex".
I for my part don't have a problem with an additional line in my source to turn on things I need.
But it gives us the argument "if you turn THAT on, you better know what to do with it". That's better than saying "If you use SCALA, you better know what to do with it".
If scala wants to go mainstream it has to appeal to mainstream developers. Some of the switches certainly are a step towards this end.
Greetings
Bernd
Sean Parsons
I guess it's because of the recurrent complaints of the "unmanageable scala {complexity/power}".
It has been argued if this is necessary - it's a double edged sword. Removing tools that are a good solution for a given problem will lead to suboptimal solutions. But there is also the perspective that tools used unwisely are also a problem.
The first case will most probably lead to working solutions which are suboptimal and can be enhanced. That's daily business. No big deal.
The second case might lead to tangled or cryptic code, bumpy compiler rides and maybe failure... and another blog post that complains about "scala the complex".
I for my part don't have a problem with an additional line in my source to turn on things I need.
But it gives us the argument "if you turn THAT on, you better know what to do with it". That's better than saying "If you use SCALA, you better know what to do with it".
If scala wants to go mainstream it has to appeal to mainstream developers. Some of the switches certainly are a step towards this end.
Derek Wyatt
There's no value in explicitly requiring a user to load the gun before shooting it. If they want to use Macros, they're going to use them... all you're really doing is forcing them to put in some boilerplate first. It doesn't stop them from using it, it just makes using it annoying.
Flip the sign and now you can "protect" your code against unwanted features (if even that's a desirable thing, which I'm not convinced on).
I love Scala, and I think its creators (all of them) are brilliant... but this is just plain weird.
Daniel Spiewak
I think the answer to this is a firm "no". The features disabled don't appear in many, or indeed most, examples of complex Scala source code. For example, you can do some crazy, crazy things using libraries like Scalaz without ever touching higher-kinded types, existentials or similar in your client code. Obviously the library will use these features, but there's nothing we can do about this (see also: collections). Maybe an even better example are the knots you can tie yourself in with trait linearization and really inheritance in general. Oh, and let's not forget about implicits! These are features that yield code that is empirically complex, but of course they are far too intrinsic to be removed or disabled.
People (read: not PLT enthusiasts) don't care about the complexity (or simplicity) of the language. All they care about is the complexity (or simplicity) of the code that the language carrots them to write. If we want to address the "Scala is too complex" claims, it should be based on concrete examples of that complexity (code, not vague feature lists) and very careful root-cause investigation. In the end, addressing such issues at the language level is very nearly a lost cause. The implications of any language change are far too vast and cross-cutting; and the issues at hand are far too subtle to be resolved by so blunt an approach as disabling language features whole-sale.
Daniel
Rex Kerr
On Sat, Mar 17, 2012 at 10:07 PM, Rex Kerr <ich...@gmail.com> wrote:
> The big problem with this proposal is that it threatens to add a bunch of
> boilerplate to every single existing Scala file and program and REPL
> session...if you want "full power but stable" Scala, you end up
> with quite a mouthful:
> import language.{postfixOps, structuralTypes, higherKinded, existentials}
>
I believe that depends how important you find these features for daily
(as opposed to specialized) Scala use.
It's really hard to use higher kinded types and complex existentials _by accident_, and if they're rarely used, then the turn-them-on/off feature seems a waste of time to me; it won't have much of an impact in any direction. There may be an argument for postfix ops and structural types, if people commonly run into trouble with them that cannot be fixed in any more sensible way than turning them off.
So I still stand by my "boilerplate" critique.
--Rex
P.S. FWIW, I can't escape higher kinded types given the collections library design unless I'm content to do nothing generic. Maybe other people have a different experience.
Channing Walton
I really don't like this idea.
1. Developers and teams can decide whether they want to use features of the language without these flags.They can do so in ways which are much more nuanced than the broad brush strokes these flags offer.
2. Once a feature requires an explicit option to turn it on it may never be turned on in many conservative organisations. The decision will be taken away from developers/teams by architects and technical managers who don't really understand what they are doing and operate from fear caused by their own ignorance.
3. What a sad message this is about the sophistication and intelligence of developers, even patronising, although I hesistate to use such a strong word.
I am working with people in enterprise environments that find these more sophisticated parts of the language challenging. But they really enjoy how scala is teaching them new things way beyond what they currently know (above Java usually). Its exciting for them! Please don't let noisy detractors cause these kinds of changes to the language that many (quieter) developers enjoy.
Channing
tpolecat
Simon Ochsenreither
a few notes from my first expression:
- “language” could lead to confusion for Java developers, because of java.lang
- Would we still have -Xmacro? Or would import language.macro replace it?
- How would this relate to Predef? Wouldn't the SIP amount to something like a different Predef where more or less is brought into the scope?
- Any chance to add any2stringadd and the numeric conversions from integer types to floating point to the enum, e. g. all the arcane Java compat stuff?
- Some of the stuff feels like it could be easier solved by not putting it in a place where it is available by default. E.g. “scala.Dynamic needs import language.dynamic”. Why not put it somewhere like scala.lang.Dynamic to require an explicit import?
- postfixOps is scary .... we switch the Parser based on an import?
- structuralTypes looks interesting. So without it new Object { val member = 1 } will generate the same anonymous class, but won't have the structural type? In the end I think performance will be on par when Scala will start targeting Java >= 7, so it feels like an optimization we won't need in the long term. Which platform would suffer from it?
- higherKinded ... looks more like we need python's import from __future__.
- In general: How to decide in the future if something should end up as language.foo or as a compiler switch? Is it completely arbitrary?
- -F ... we already -X, -Y why not use one of them?
Frankly, I'm not sure if we really need that. Maybe helpful in some corporate setting so that superiors can more easily restrict the allowed language features “no imports from language allowed”, but I think just adding an IDE option or a compiler flag would be simpler.
This feels a bit like a PR stunt for the “Scala is so complex” crowd, but imho it will backfire and the mere existence of language.* will serve to proof their point.
Maybe we could have some -Xscary compiler flag instead? Which just emits warnings about the usage of these features without splitting the community into multiple groups ranging from “no language.* allowed”, “only postfixOps” to “no restrictions”.
Thanks and bye,
Simon
Channing Walton
On Saturday, 17 March 2012 19:48:40 UTC, Martin wrote:
Josh Suereth
Honestly, the language support for higher kinded types is useful enough to do higher-order abstractions, but still too painful for the code to be readable. I think improving higher-kinded types is a far better solution to removing them. NOW, as stated in the SIP, if the idea is to deprecate the current higher kinded types in favor of a new system that works better, I'm all for that. However, I think that new system needs to be (a) in place, or (b) spec'd before we take such a move against higher-kinded types. I know a *lot* of my personal software directly or indirectly relies on them.
Chapin, Peter @ VTC
In general I think modularization of large things (for some definition of “large”) is good. Scala is growing rapidly and new features are being added or proposed at a steady rate. What will happen to Scala if this continues indefinitely? Features will start interacting in surprising ways causing an ever increasing number of “gotchas” that programmers will have to worry about. The problem is increased when compatibility with legacy code is thrown into the mix.
Modularization would allow teams to take some control over the expansion of the language. “Feature X is cool, but we don’t need it. We won’t turn it on and we won’t get any ugly surprises due to some subtle backward-incompatible aspect of feature X. Plus the compiler can optimize our code better if it knows it doesn’t have to deal with feature X.”
It’s not the complexity of Scala now that concerns me; it’s the complexity of Scala twenty years from now that I’m worried about. What language can endure continuous growth without eventually falling over under its own weight (think C++)? Perhaps Scala can if it supports a language modularization feature.
As an aside Ada has a feature that allows programmers to selectively turn off certain features (pragma Restriction). It isn’t used much but it is used, for example, to simplify the language for purposes of static analysis or to reduce the runtime system requirements, etc. There is definitely value in it, although I notice that Ada’s approach is to present the whole language by default and allow the program to contract it when appropriate.
Peter
martin odersky
<ghosh.d...@gmail.com> wrote:
> What's the major force towards this ? Is it only to present a smaller
> surface area to newcomers in Scala ?
>
be explicit about in every codebase I collaborate in. For instance, if
I accidentally use a structural type I want to be warned about it,
mostly because I care about performance. Or, if someone I collaborate
with uses postfix operators I also want to see that upfront. And so
on. So I think the flags will be valuable for most Scala codebases,
with typical exceptions being DSLs.
The main feature left out are implicits. Implicits are probably
misused more often than any other part of Scala, but requiring them to
be put wholesale under a flag would mean we think all type-class
oriented programming is a special case, and that's absurd. So we'd
need a crisp definition what constitutes a problematic implicit
(implicit conversions, maybe?) I could not convince myself of a good
criterion so far, so I left them out.
Cheers
-- Martin
Jean-Laurent Huynh
On Saturday, March 17, 2012 6:37:01 PM UTC-7, Martin wrote:
On Sat, Mar 17, 2012 at 11:13 PM, Debasish Ghosh
On Saturday, March 17, 2012 6:37:01 PM UTC-7, Martin wrote:
On Sat, Mar 17, 2012 at 11:13 PM, Debasish Ghosh
Andreas Flierl
martin odersky
Imports are inspired from Python where things like
from Future import ...
work well. Imports have the advantage that doc comments can be
attached to the enumeration values, so it's immediately clear where to
look for documentation about a language feature.
Cheers
- Martin
Eugene Yokota
For instance, if I accidentally use a structural type I want to be warned about it,
mostly because I care about performance. Or, if someone I collaborate
with uses postfix operators I also want to see that upfront.
martin odersky
Macros are so easily abused that they definitely belong behind a flag,
even once they are no longer new.
> (4) Dynamic typing is dangerous to type safety, so it belongs behind an
> opt-in "I know better" "asInstanceOf" kind of flag. Open the box at your
> own risk.
>
> If these are the *core* reasons for the changes, I can buy solutions to
> those problems. However, here's my thoughts:
>
> (1) I think Structural types disable/enabled might not go far enough for #1.
> When i want to disable reflection I want to disable any use of it,
> including getClass or isInstanceOf runtime calls.
I think these are actually available on all platforms.
> (2) I'd rather wait until we have a new higher-kinded system spec'd out and
> proven before we try to deprecate what we have. They're too useful and
> indirectly used to hide behind a flag.
But we do not deprecate them! I think if there is a chance that we
change some aspects of them in the future it's better to put them
under a flag now, so that people can avoid them if they want a
guarantee that their code will last forever without changes.
> (3) Totally fine with new features having flags to enable before they
> stabilize (like CPS transforms)
> (4) Not sure this should default off or on, but the idea of a flag for
> dynamic typing is great IMHO. If I want guaranteed type safety (and the
> niceties, like refactoring that come with it) in a project, I want this.
>
I think flags should always be opt-in, not opt-out. "Keep simple
things simple and complicated things possible"
Cheers
- Martin
martin odersky
> Color me horrified.
>
> Ok, maybe that was too strong. Honestly, I don't see anything positive
> coming out of a change like this. As Rex pointed out, it just creates
> import/build-config boilerplate for, well, everyone. Even for the few who
> don't use these features (knowingly or unknowingly), these switches have no
> beneficial effect. You are hiding a language feature that these people
> haven't seen. By definition, this will be a completely invisible change for
> a few people and a significantly detrimental change for a majority.
>
codebases I contribute to, these features are very rarely used. To a
point, where I would welcome a warning if they are indeed used. So I
believe it will be an invisible change for the majority, a welcome
change for most people on the borderline, and a small nuisance for a
few.
> It's also worth noting that many people who use these features do so without
> knowing it. Existentials and structural types are a particularly strong
> example of this, but type constructors are also quite common. People who
> know what these features entail will have already turned them on, while
> those who don't know will receive a mysterious warning/error on
> seemingly-innocuous code. Consider the sheer volume of Scala snippets and
> explanations which will be instantly invalidated.
>
That's a good point. I actually think it's by far preferable if people
who do not know about using these features are notified. That way they
can look up the documentation for the feature, learn about it, and
then make a conscious decision whether they want to use it.
> On a more philosophical level, "hiding" language features like this only
> serves to fragment the community. It makes life very difficult in venues
> like mailing lists and Stackoverflow, where users will not only need to
> specify their language (Scala) but their language "level" (Nerfed Edition).
> By definition, newcomers to the language will not understand this kind of
> fragmentation, given that no other language has anything quite like this,
> and will thus be extremely confused when correct answers to seemingly-simple
> questions (like extending collections) lead to warnings, or worse, errors.
>
I don't think this will lead to fragmentation. These are very specific
features. For most discussions about the language it would actually be
very beneficial to rule out from discussion dispatch via dynamic, just
to pick an example. Where people do talk about specific features, it
will become common understanding to assume the import.
> Pragmatically, I have to ask the question: who are we trying to benefit with
> this change, and is it such a large benefit as to merit inconveniencing the
> largest (and most vested) segment of your user-base?
>
I already responded to Debashish on that.
Cheers
- Martin
martin odersky
<channin...@gmail.com> wrote:
> If all one has to do is import something, then those that want to use those
> features will do so, those that don't, will not. How is that different from
> simply having all the features 'turned on' and letting people write the code
> they want to? You can't use those feature by accident! Or can you?
>
slipped in by accident.
Cheers
- Martin
> Is the intention to enable automated checking of more sophisticated
> techniques so that they can be re-written in a more basic way, in teams
> where a decision has been made not to use certain features?
>
>
> On Saturday, 17 March 2012 19:48:40 UTC, Martin wrote:
>>
>> Here's a new SIP which proposes to put some new and existing advanced
>> language features under an import flag. I expect we'll have an
>> interesting discussion about this one.
>> So, please, fire away!
>>
>> Here's the link to the SIP:
>>
>>
>> https://docs.google.com/document/d/1nlkvpoIRkx7at1qJEZafJwthZ3GeIklTFhqmXMvTX9Q/edit
>>
>> Cheers
>>
>> -- Martin
--
Martin Odersky
Prof., EPFL and Chairman, Typesafe
PSED, 1015 Lausanne, Switzerland
Tel. EPFL: +41 21 693 6863
Tel. Typesafe: +41 21 691 4967
martin odersky
> I think it's important to separate the complexity of Scala from the
> complexity it allows you to generate. The truth is that when people
> complain about Scala being complex, they're almost always complaining about
> the complexity of the code Scala allows one to write. Working under the
> assumption that this feature is intended to address the "Scala is too
> complex" claims (which seems like a reasonable assumption, given this SIPs
> origin), we should be measuring its merits against the benchmark of "will it
> carrot or require developers to create simpler Scala code".
>
> I think the answer to this is a firm "no". The features disabled don't
> appear in many, or indeed most, examples of complex Scala source code. For
> example, you can do some crazy, crazy things using libraries like Scalaz
> without ever touching higher-kinded types, existentials or similar in your
> client code. Obviously the library will use these features, but there's
> nothing we can do about this (see also: collections). Maybe an even better
> example are the knots you can tie yourself in with trait linearization and
> really inheritance in general. Oh, and let's not forget about implicits!
> These are features that yield code that is empirically complex, but of
> course they are far too intrinsic to be removed or disabled.
>
library constructs which can lead to complicated usages (postfix ops
are the only exception). That's a conscious decision. Demanding the
same from users of such libraries would be too cumbersome, I think.
> People (read: not PLT enthusiasts) don't care about the complexity (or
> simplicity) of the language. All they care about is the complexity (or
> simplicity) of the code that the language carrots them to write. If we want
> to address the "Scala is too complex" claims, it should be based on concrete
> examples of that complexity (code, not vague feature lists) and very careful
> root-cause investigation. In the end, addressing such issues at the
> language level is very nearly a lost cause. The implications of any
> language change are far too vast and cross-cutting; and the issues at hand
> are far too subtle to be resolved by so blunt an approach as disabling
> language features whole-sale.
>
Well, they are not disabled. Just brought out in the open. I am pretty
sure that the concept of explicit feature imports will have a net
beneficial effect on large Scala projects. We can discuss about the
details what should go under a flag and what shouldn't. Those a much
more tentative in my mind than the idea itself.
Cheers
-- Martin
Daniel Spiewak
It's true that the "import tax" of the proposal is typically levied on
library constructs which can lead to complicated usages (postfix ops
are the only exception). That's a conscious decision. Demanding the
same from users of such libraries would be too cumbersome, I think.
This is exactly my point. The features which enable the most complex uses aren't even touched by the import tax. In order of significance, I have seen the most complexity in client code enabled by the following features:
- Traits
- Leaky linearization semantics (e.g. null vals)
- Path-dependent types (even concrete, non-existential ones)
- Implicit resolution (conversions are confusing, but rarely complex)
- Arcane applications of var (particularly when intersected with function values)
Existential types are probably a more valid point here. I find them ridiculously useful, but I doubt many people use them beyond Java interop. On the plus side, I've never seen a truly complex use of existential types (i.e. where the feature introduced complexity above and beyond the problem domain), but that could be just because it's generally impossible to get a usage of existential types to even compile without really knowing what you're doing. In short, existential types allow significant complexity in client code, but they're only ever used in cases where such complexity is unavoidable. So, the language is doing its job as it stands.
Daniel
Daniel Spiewak
I believe my development experience is different than yours then. In
codebases I contribute to, these features are very rarely used. To a
point, where I would welcome a warning if they are indeed used. So I
believe it will be an invisible change for the majority, a welcome
change for most people on the borderline, and a small nuisance for a
few.
Having to mechanically insert a bulky import statement at the top of every single file is not a "small" nuisance. (regarding bulky, note that "import language._" is insufficient since I don't want things like macros and dynamic by default)
That's a good point. I actually think it's by far preferable if people
who do not know about using these features are notified. That way they
can look up the documentation for the feature, learn about it, and
then make a conscious decision whether they want to use it.
On what grounds would they be making that decision? I can virtually guarantee that the thought process will go as follows:
1. Ask question on Stackoverflow
2. <wait 30 seconds>
3. Read answer by Daniel Scobral, posted 20 seconds ago (page refresh delay)
4. Paste snippet into code base
5. Receive giant scary warnings about some thingy that you don't recognize
6. Google "some thingy" (or possibly ask again on Stackoverflow)
7. Get directed to detailed technical explanation of "some thingy"
8. While skimming the 30 page PDF, encounter first instance of \inferrule*
9. Panic
I don't think the net result here will be enlightenment, nor do I believe this change will encourage people to make an informed decision about certain language features. Rather, I think the majority of people will hit the scary warning, skim some scary documentation, and decide that it's probably easier to use asInstanceOf. In other words, it is short-circuiting the learning process, not encouraging its progression.
I don't think this will lead to fragmentation. These are very specific
features. For most discussions about the language it would actually be
very beneficial to rule out from discussion dispatch via dynamic, just
to pick an example. Where people do talk about specific features, it
will become common understanding to assume the import.
Experienced Scala developers will simply turn everything on. Discussions they carry out, articles they post and talks they present will all make this assumption. Incoming Scala developers (who don't know about the magical switches) will be using a literally different language until they graduate into the elite group.
Currently, we're in a situation where experienced Scala developers understand and leverage certain features that appear magical to newcomers. Thus, experienced Scala developers have an understanding that newcomers haven't acquired yet (as a rule). This SIP would put us in a position where not only do experienced Scala developers have additional understanding, they are also in position of the secret keys for unlocking the hidden doors to this magical kingdom of understanding. It's throwing barriers in the path of those attempting to fully learn the language, and barriers always lead to fragmentation.
Daniel
Paul Phillips
> Could we get a compiler switch to turn that feature off? That's actually
> something I could get behind…
Yeah, I have a whole list of stuff I'd like to turn off, and this SIP
misses my list completely. Auto-coercions between primitives,
any2stringadd, auto-tupling...
The only thing I can see which I have any enthusiasm for is developing
some mechanism to control reflection. But this would not be a
satisfactory mechanism.
We'd be so much better off putting effort into something like checkstyle.
Miles Sabin
> We'd be so much better off putting effort into something like checkstyle.
Oh yes ...
Cheers,
Miles
--
Miles Sabin
tel: +44 7813 944 528
gtalk: mi...@milessabin.com
skype: milessabin
g+: http://www.milessabin.com
http://twitter.com/milessabin
http://underscoreconsulting.com
http://www.chuusai.com
martin odersky
>
>> I believe my development experience is different than yours then. In
>> codebases I contribute to, these features are very rarely used. To a
>> point, where I would welcome a warning if they are indeed used. So I
>> believe it will be an invisible change for the majority, a welcome
>> change for most people on the borderline, and a small nuisance for a
>> few.
>
>
> Having to mechanically insert a bulky import statement at the top of every
> single file is not a "small" nuisance. (regarding bulky, note that "import
> language._" is insufficient since I don't want things like macros and
> dynamic by default)
As the SIP states, import language._ would do nothing; you need
explicit imports. But come on, how many of these do you propose to use
in a single source file at the same time?
>>
>> That's a good point. I actually think it's by far preferable if people
>> who do not know about using these features are notified. That way they
>> can look up the documentation for the feature, learn about it, and
>> then make a conscious decision whether they want to use it.
>
>
> On what grounds would they be making that decision? I can virtually
> guarantee that the thought process will go as follows:
>
> 1. Ask question on Stackoverflow
> 2. <wait 30 seconds>
> 3. Read answer by Daniel Scobral, posted 20 seconds ago (page refresh delay)
> 4. Paste snippet into code base
> 5. Receive giant scary warnings about some thingy that you don't recognize
> 6. Google "some thingy" (or possibly ask again on Stackoverflow)
> 7. Get directed to detailed technical explanation of "some thingy"
> 8. While skimming the 30 page PDF, encounter first instance of \inferrule*
> 9. Panic
Well, that depends how we structure the docs, no?
>
> I don't think the net result here will be enlightenment, nor do I believe
> this change will encourage people to make an informed decision about certain
> language features. Rather, I think the majority of people will hit the
> scary warning, skim some scary documentation, and decide that it's probably
> easier to use asInstanceOf. In other words, it is short-circuiting the
> learning process, not encouraging its progression.
>
>>
>> I don't think this will lead to fragmentation. These are very specific
>> features. For most discussions about the language it would actually be
>> very beneficial to rule out from discussion dispatch via dynamic, just
>> to pick an example. Where people do talk about specific features, it
>> will become common understanding to assume the import.
>
>
> Experienced Scala developers will simply turn everything on. Discussions
> they carry out, articles they post and talks they present will all make this
> assumption. Incoming Scala developers (who don't know about the magical
> switches) will be using a literally different language until they graduate
> into the elite group.
I guess I am an experienced Scala programmer :-) and I can guarantee
you I will turn on nothing by default. I know where I need to use
macros and higher-kinded types, and am totally prepared to declare
that explicitly.
Cheers
- Martin
Daniel Spiewak
Well, they are not disabled. Just brought out in the open.
Forgot to reply to this in my earlier email. Throwing out a warning is almost the same as disabling them. Issuing an error (as proposed in the SIP) is definitionally the same as disabling them.
Here's my proposal. There are certain features like macros and dynamic that the whole community (well, almost the whole community) agrees should be off by default. Maybe this is because these features haven't stabilized yet (like continuations), or maybe it's because these features have far-reaching and surprising implications. These features should be off by default and enabled by a feature switch (like -Fenable-dynamic).
There are other features that have been identified by some people as being problematic or dangerous (e.g. structural types and existentials) but are not generally accepted to be net negatives. Likely, this situation is because these features are "advanced", or perhaps have surprising implications, but have a generally neutral or positive impact on most development. These features should be on by default and disabled by a feature switch (like -Fdisable-existentials).
I don't think anyone has complained about the off-by-default status of dynamic or macros. Making the other features (higher kinds, existentials, structural types) on by default with a disabling switch gives people the flexibility to opt out at a project level if they really want to prevent the use of some feature. More granular control could be implemented in the same way that -enable-assertions is handled by the JVM itself (i.e. at the package or compilation unit level).
No magical pragma or import overloading would be added.
I think this proposal actually satisfies all of the cases and objections voiced so far. Most significantly, it avoids stratifying the Scala language into "levels" based on apparent advancement of certain features. Perhaps more significantly, this would be an opt-in system. Nothing would need to change for people who aren't interested in the feature. Only those who really care would have to put in any effort. (pay for what you use)
Daniel
Daniel Spiewak
+1 to the checkstyle solution.
Daniel
martin odersky
> On Sat, Mar 17, 2012 at 7:49 PM, Daniel Spiewak <djsp...@gmail.com> wrote:
>> Could we get a compiler switch to turn that feature off? That's actually
>> something I could get behind…
>
> Yeah, I have a whole list of stuff I'd like to turn off, and this SIP
> misses my list completely. Auto-coercions between primitives,
> any2stringadd, auto-tupling...
Feel free to add to the list! Auto-tupling is a good candidate, I
completely overlooked that. any2stringadd, I would love to get rid of,
but that will need to be at some point in the future, now is
premature.
>
> The only thing I can see which I have any enthusiasm for is developing
> some mechanism to control reflection. But this would not be a
> satisfactory mechanism.
>
> We'd be so much better off putting effort into something like checkstyle.
I was thinking a lot about it, but: checkstyle has not materialized,
and will not materialize in foreseeable time. And the reason is the
same as this proposal: it's probably too hard to develop a consensus
what should go in it. And, anyway, the current proposal does not
preclude a checkstyle later.
Cheers
- Martin
martin odersky
>> Well, they are not disabled. Just brought out in the open.
>
>
> Forgot to reply to this in my earlier email. Throwing out a warning is
> almost the same as disabling them. Issuing an error (as proposed in the
> SIP) is definitionally the same as disabling them.
How is requiring an explicit import the same as disabling a feature?
>
> Here's my proposal. There are certain features like macros and dynamic that
> the whole community (well, almost the whole community) agrees should be off
> by default. Maybe this is because these features haven't stabilized yet
> (like continuations), or maybe it's because these features have far-reaching
> and surprising implications. These features should be off by default and
> enabled by a feature switch (like -Fenable-dynamic).
>
OK, that's something we agree on then. I still like the import
solution much better then the switch.
> There are other features that have been identified by some people as being
> problematic or dangerous (e.g. structural types and existentials) but are
> not generally accepted to be net negatives. Likely, this situation is
> because these features are "advanced", or perhaps have surprising
> implications, but have a generally neutral or positive impact on most
> development. These features should be on by default and disabled by a
> feature switch (like -Fdisable-existentials).
>
I don't like explicit disabling. It violates the principle that simple
things should be kept simple. And it complicates the machinery
considerably. Now you'd have to explain how to resolve an enabling
import that clashes with a disabling flag. I don't want to get into
that!
Cheers
- Martin
Dan Rosen
The features controlled by flags are enabled in the whole scope where the import statement is effective.
macros can lead to code that is hard to debug and understand.
Even where reflection is available, structural type dispatch can lead to surprising performance degradations.
Dynamic member selection can undermine static checkability of programs.
Several DSLs written in Scala need [postfix operator] notation.
Eugene Yokota
Daniel Spiewak
As the SIP states, import language._ would do nothing; you need
explicit imports. But come on, how many of these do you propose to use
in a single source file at the same time?
So, not only are we overloading import, we're also giving the overloaded import wildcard a silently different semantic? This is complexity from a linguistic standpoint.
Actually, most of the time when I'm using existentials, I'm also using type constructor polymorphism. Not always, but mostly. I'm always using type constructor polymorphism when I use structural types, simply because I primarily use structural types for the type lambda mechanism. As for postfix operators, I know the style guide says not to use them, but I must confess that I do make use of them on a fairly regular basis, generally without thinking about it. So, at a minimum, the header of all of my source files would have:
import language.{postfix, existentials, higherKinds}
Given the cognitive speed bump that an error on a "disabled feature" would cause, I would probably fall into the habit of just turning stuff on by default and forgetting about it.
Well, that depends how we structure the docs, no?
Scala doesn't have the greatest track record when it comes to documentation for beginners. :-) Granted, things have improved exponentially with Heather at the helm. It would take some fantastic writing to explain the full potential of existential types in such a way that a complete newcomer would not glaze over and/or run in terror. Not impossible, but very hard.
I guess I am an experienced Scala programmer :-) and I can guarantee
you I will turn on nothing by default. I know where I need to use
macros and higher-kinded types, and am totally prepared to declare
that explicitly.
It's a bit like a penalty for using those features. I want to be able to use Scala without any impingement on my train of thought. Remembering to inject a magical import in order to use a language feature is a significant added overhead, given that import is a mechanism for enabling library features.
I can understand the desire to make the language safer and carrot people to use less "surprising" features. I'm not sure this accomplishes that task though. What I am sure it accomplishes is tremendous annoyance in the upper tier of Scala developers, as evidenced by all of the rancor this has stirred up both here and on Twitter.
Daniel
martin odersky
<simon.och...@googlemail.com> wrote:
> Hi Martin,
>
> a few notes from my first expression:
>
> “language” could lead to confusion for Java developers, because of java.lang
> Would we still have -Xmacro? Or would import language.macro replace it?
-Xmacro would be replaced.
> How would this relate to Predef? Wouldn't the SIP amount to something like a
> different Predef where more or less is brought into the scope?
No I think it is more helpful to bring everything into scope but issue
errors or warnings
if a flag is required.
> Any chance to add any2stringadd and the numeric conversions from integer
> types to floating point to the enum, e. g. all the arcane Java compat stuff?
I believe that would affect way too many programs. I'd love
invalidating any2stringadd but we need at least have string
interpolation as an alternative in place and then wait until virtually
all code uses that instead of + on strings. That will take a long
time. Numeric conversions: It's one of the areas where we have decided
not to reinvent the wheel and just take what Java has.
> Some of the stuff feels like it could be easier solved by not putting it in
> a place where it is available by default. E.g. “scala.Dynamic needs import
> language.dynamic”. Why not put it somewhere like scala.lang.Dynamic to
> require an explicit import?
Because it would also affect indirect inheritance.
trait MyDynamic extends Dynamic { def applyDynamic(name: String) ... }
trait Dispatcher extends MyDynamic // needs import language.dynamic to work.
> postfixOps is scary .... we switch the Parser based on an import?
No, we just flag postfix ops after the parser with warnings.
> structuralTypes looks interesting. So without it new Object { val member = 1
> } will generate the same anonymous class, but won't have the structural
> type? In the end I think performance will be on par when Scala will start
> targeting Java >= 7, so it feels like an optimization we won't need in the
> long term. Which platform would suffer from it?
I know of Scala-GWT, but that's only experimental. I thought first
Android as well, but that seems to have reflection. So maybe the
problem is not as bad as I thought.
Cheers
- Martin
Paul Phillips
>
> I was thinking a lot about it, but: checkstyle has not materialized,
> and will not materialize in foreseeable time. And the reason is the
> same as this proposal: it's probably too hard to develop a consensus
> what should go in it.
What's great about checkstyle is that you don't need any consensus.
It's configurable. Turning off the language isn't configuring, it's
splintering.
The thing is, I am more certain than I would like to be (yes, I would
genuinely prefer to be less certain) that none of this will help with
the problem it's presumably trying to address. I think we should
measure ideas like this by comparing "time wasted" to "time saved" and
"confusion added" to "confusion subtracted" and "busywork created" to
"busywork avoided". The wrong side of those scales is going to look
like it just landed on jupiter. And not just for the people who
utilize the features being targeted. For everyone.
Daniel Spiewak
How is requiring an explicit import the same as disabling a feature?
The explicit import isn't an import at all, but a compiler pragma in the syntactic guise of an unrelated language feature. As with any compiler pragma, the net effect is to change the behavior of the compiler, and thus, the language.
If a feature is disabled by default, then it is not in the language (by definition). This is why Haskell discussions separate "Haskell" from "Haskell with GHC extensions".
I don't like explicit disabling. It violates the principle that simple
things should be kept simple. And it complicates the machinery
considerably. Now you'd have to explain how to resolve an enabling
import that clashes with a disabling flag. I don't want to get into
that!
I would remove the import mechanism entirely. Pragma was evil back in the days of C. Introducing it to Scala, especially when disguised as an orthogonal feature, is just begging for pain (like the pain of disambiguating compiler switches vs imports).
I think the compiler switch is still keeping things simple. In fact, I think it keeps things simpler than the import mechanism, which introduces a very weird and non-orthogonal feature into the core language semantics. A compiler switch enables teams to make and control decisions about these features at a high level. It's much easier to police your build.sbt than it is to police an entire codebase, grepping for a construct that uses the same syntax as another (very commonly-used) one. Additionally, compiler switches are generally understood to affect the semantics of the compiler. That is, after all, their purpose. Thus, they fit precisely within the template concept that already exists. Using compiler switches doesn't introduce any complexity, but using an "import" does.
How does the explicit "import" interact with scoping? Can we have certain typer features activated for part of our file but not another part? What if higher kinds are enabled inside a method but not outside? What error do we get if features "leak" through type inference? There's just a whole host of questions that arise when you start introducing language features – you must know this better than I! Compiler switches are the simpler mechanism, by far.
Daniel
martin odersky
> On Sat, Mar 17, 2012 at 8:14 PM, martin odersky <martin....@epfl.ch> wrote:
>>
>> I was thinking a lot about it, but: checkstyle has not materialized,
>> and will not materialize in foreseeable time. And the reason is the
>> same as this proposal: it's probably too hard to develop a consensus
>> what should go in it.
>
> What's great about checkstyle is that you don't need any consensus.
> It's configurable. Turning off the language isn't configuring, it's
> splintering.
>
Scala usage for general projects is. There's no prejudice against
controlled features except saying and documenting: watch out, here are
some complications, make sure you understand them before you decide to
use the feature. So it's actually the opposite of splintering.
> The thing is, I am more certain than I would like to be (yes, I would
> genuinely prefer to be less certain) that none of this will help with
> the problem it's presumably trying to address. I think we should
> measure ideas like this by comparing "time wasted" to "time saved" and
> "confusion added" to "confusion subtracted" and "busywork created" to
> "busywork avoided". The wrong side of those scales is going to look
> like it just landed on jupiter. And not just for the people who
> utilize the features being targeted. For everyone.
Individual features can be discussed. I am 100% certain that Dynamic
and macros need a feature import. I would dearly like postfix ops to
be under the same regime. I gave reasons for putting in the other
three, but am open to discussions.
Cheers
-- Martin
martin odersky
>
>> How is requiring an explicit import the same as disabling a feature?
>
>
> The explicit import isn't an import at all, but a compiler pragma in the
> syntactic guise of an unrelated language feature. As with any compiler
> pragma, the net effect is to change the behavior of the compiler, and thus,
> the language.
>
> If a feature is disabled by default, then it is not in the language (by
> definition). This is why Haskell discussions separate "Haskell" from
> "Haskell with GHC extensions".
>
I disagree. As long as a feature is specified in the SLS it is in the
language. The fact that it requires an explicit import does not change
that.
>> I don't like explicit disabling. It violates the principle that simple
>> things should be kept simple. And it complicates the machinery
>> considerably. Now you'd have to explain how to resolve an enabling
>> import that clashes with a disabling flag. I don't want to get into
>> that!
>
>
> I would remove the import mechanism entirely. Pragma was evil back in the
> days of C. Introducing it to Scala, especially when disguised as an
> orthogonal feature, is just begging for pain (like the pain of
> disambiguating compiler switches vs imports).
>
> I think the compiler switch is still keeping things simple. In fact, I
> think it keeps things simpler than the import mechanism, which introduces a
> very weird and non-orthogonal feature into the core language semantics. A
> compiler switch enables teams to make and control decisions about these
> features at a high level. It's much easier to police your build.sbt than it
> is to police an entire codebase, grepping for a construct that uses the same
> syntax as another (very commonly-used) one. Additionally, compiler switches
> are generally understood to affect the semantics of the compiler. That is,
> after all, their purpose. Thus, they fit precisely within the template
> concept that already exists. Using compiler switches doesn't introduce any
> complexity, but using an "import" does.
>
> How does the explicit "import" interact with scoping? Can we have certain
> typer features activated for part of our file but not another part?
I do not see why not.
> What if
> higher kinds are enabled inside a method but not outside? What error do we
> get if features "leak" through type inference? There's just a whole host of
> questions that arise when you start introducing language features – you must
> know this better than I! Compiler switches are the simpler mechanism, by
> far.
I believe this is explained in the SIP. It's not yet implemented, so
there might still be surprises here. But a priori I think it's doable.
Cheers
- Martin
Paul Phillips
> Numeric conversions: It's one of the areas where we have decided
> not to reinvent the wheel and just take what Java has.
Nobody wants to reinvent this wheel! They want to TURN THE WHEEL OFF.
Daniel Spiewak
I disagree. As long as a feature is specified in the SLS it is in the
language. The fact that it requires an explicit import does not change
that.
Ok, I'll buy that. It seems you're viewing the explicit import not as a pragma, but as a lexically scoped flag. Still, horrifying to contemplate from a formal perspective, but not as horrifying as I initially thought.
For the record, the overloading of import is still really problematic, precisely because overloading anything tends to introduce a lot of complexity and cognitive burden. By definition, it requires one to consciously consider whether this construct refers to concept A or concept B. The proposed "language import" is a very different concept from a library import and has very different effects. Lumping it under the same syntactic construct seems…wrong, to put it mildly.
I do not see why not.
Weren't you the one telling me that the typer is already too complex? ;-)
I believe this is explained in the SIP. It's not yet implemented, so
there might still be surprises here. But a priori I think it's doable.
I still think leaking is going to be a serious problem. At the very least, it will need to be resolved in a somewhat arbitrary fashion, creating confusion and frustration for those attempting to use the feature. A compiler switch is much clearer in this area; there is absolutely no uncertainty about where and how its effects will be applied.
Daniel
Daniel Sobral
> The big problem with this proposal is that it threatens to add a bunch of
> boilerplate to every single existing Scala file and program and REPL
> session. With all these things off by default, I would argue that Scala is
> severely curtailed; why make everyone in every file and script type in
> "import language._" in order to enable the features that make you use Scala
> instead of something else? And then, you get not only the features of a
> present-day typical Scala distribution, but early versions of features that
> are not yet fixed; if you want "full power but stable" Scala, you end up
> with quite a mouthful:
> import language.{postfixOps, structuralTypes, higherKinded, existentials}
>
> Instead, I propose that, if there is a significant demand for limiting
> language features, that we flip the sign: instead of requiring import
> language.structuralTypes to get structural types, import
> language.disable.structuralTypes if you want them to go away. If you're in
> a development environment with strict coding rules anyway, you probably will
> have half a screen of stuff at the top of every file anyway (copyright
> notice, imports of a bunch of libraries, etc.), so one more line doesn't
> really hurt that much.
>
> Features that really are experimental could be kept as an import
> language.enable._ so that people are alerted that they shouldn't trust how
> they behave. For example, macros would be a good candidate here since
> they're not ironed out. Furthermore, some things that are compiler flags
> could be moved to language.enable (such as continuations), where they
> arguably more sensibly belong.
>
> (Actually, even more sensibly, they'd be annotations on the file.)
I'm fully with Rex here. I've made a number of comments on the
proposal, but just seeking clarification on the exact mechanics of
various parts. But I don't like it. Yes, there are people who'd prefer
to limit the extent to which the language is available, but they can
easily add disable flags to their builds. I think Rex proposal of
having stable features enabled by default and experimental disabled by
default strikes a good balance.
--
Daniel C. Sobral
I travel to the future all the time.
Daniel Spiewak
I'm fully with Rex here. I've made a number of comments on the
proposal, but just seeking clarification on the exact mechanics of
various parts. But I don't like it. Yes, there are people who'd prefer
to limit the extent to which the language is available, but they can
easily add disable flags to their builds. I think Rex proposal of
having stable features enabled by default and experimental disabled by
default strikes a good balance.
In case it was lost in my flurry of emails, I am in favor of this proposal as well. Unstable (or generally-accepted as "dangerous") features should be off by default with flags to enable. Stable (and generally-accepted as "safe") features should be on by default with flags to disable. any2stringadd gets burned in effigy until such time as it can be flag-disabled.
Daniel
Daniel Sobral
> On Sat, Mar 17, 2012 at 10:07 PM, Rex Kerr <ich...@gmail.com> wrote:
>> The big problem with this proposal is that it threatens to add a bunch of
>> boilerplate to every single existing Scala file and program and REPL
>> session. With all these things off by default, I would argue that Scala is
>> severely curtailed; why make everyone in every file and script type in
>> "import language._" in order to enable the features that make you use Scala
>> instead of something else? And then, you get not only the features of a
>> present-day typical Scala distribution, but early versions of features that
>> are not yet fixed; if you want "full power but stable" Scala, you end up
>> with quite a mouthful:
>> import language.{postfixOps, structuralTypes, higherKinded, existentials}
>>
> (as opposed to specialized) Scala use. To take the Scala compiler with
> 100K lines as an example: As far as I know, there are no uses of
> postfix ops, there is one use of structural types (and we are trying
> to get rid of it), there are no uses of higher-kinded types, and any
> use of complex existentials, if it exists, I would deem a mistake. So
> from that perspective, yes, I would value having to import features
> explicitly so that I am warned from using these things accidentally.
Well, here you have me a bit. Yes, I'd prefer to disable postfix
methods, but adding that line to all specs files will be annoying.
Yes, I'd prefer to disable structural types, mostly to prevent
performance issues due to carelessness. And, yes, I'd like to disable
existentials, because people who try to use them are usually people
who shouldn't, because they don't know enough. However, the most
common use of existentials under this proposal is not caught. But...
this is my laundry list, my personal preferences. I don't see any
reason to burden others with it, and I'm not at all sure I should.
Daniel Sobral
> Yup. +1 here.
>
> There's no value in explicitly requiring a user to load the gun before
> shooting it. If they want to use Macros, they're going to use them... all
> you're really doing is forcing them to put in some boilerplate first. It
> doesn't stop them from using it, it just makes using it annoying.
>
> Flip the sign and now you can "protect" your code against unwanted features
> (if even that's a desirable thing, which I'm not convinced on).
Oh, yes, it is. A structural type, in particular, can ruin your day
performance-wise. It has happened on the standard library.
Daniel Sobral
> (4) Dynamic typing is dangerous to type safety, so it belongs behind an
> opt-in "I know better" "asInstanceOf" kind of flag. Open the box at your
> own risk.
> (4) Not sure this should default off or on, but the idea of a flag for
> dynamic typing is great IMHO. If I want guaranteed type safety (and the
> niceties, like refactoring that come with it) in a project, I want this.
And, on that note, I'd add asInstanceOf to number 4. It can cause just
as much trouble as dynamic types, and it is much more likely to be
used by beginners (because of Java). Pattern matching can deal with
most use cases.
Daniel Sobral
> On Sat, Mar 17, 2012 at 7:49 PM, Daniel Spiewak <djsp...@gmail.com> wrote:
>> Could we get a compiler switch to turn that feature off? That's actually
>> something I could get behind…
>
> Yeah, I have a whole list of stuff I'd like to turn off, and this SIP
> misses my list completely. Auto-coercions between primitives,
> any2stringadd, auto-tupling...
Yes, yes and yes. However, isn't any2stringadd a library feature? Or,
if you are refering to String+Any, wouldn't Any+String still be in the
library? I ask this just because I want it completely gone, not just
halfway. :-) Particularly now that we might well get string templates.
>
> The only thing I can see which I have any enthusiasm for is developing
> some mechanism to control reflection. But this would not be a
> satisfactory mechanism.
>
> We'd be so much better off putting effort into something like checkstyle.
And yes, but between a tool someone might write someday and something
here and now, I understand why that proposal might not be particularly
attractive to Mr. O. But the presentation compiler ought to be helping
build such tools.
Daniel Sobral
Sounds good to me. Should I bring the fuel or the matches?
RYAN RICHT
Our project, the first Scala project here, has now internally gained substantial support. We make use of all of these proposed "modularized" features, and certainly don't need the stigma of them being "dangerous," "difficult to maintain," "too complicated," "too slow," "impossible to hire for," or anything else.
The stigma associated with using Scala at a large, traditionally Java firm is sufficiently burdensome without this SIP. Please don't frustrate adoption further by implementing this SIP. This isn't helping by making Scala simpler, it is making it scarier to use the very features that make Scala high value (+1 on all Daniel Spiewak's comments).
Given the volume of highly negative feedback on this SIP, and years of complaining about more pressing (and high profile) issues like performance optimization of loops, is this *really* the most important SIP to pursue right now given limited resources?
Thank you for a great language,
ryan richt
Paul Phillips
> Yes, yes and yes. However, isn't any2stringadd a library feature? Or,
> if you are refering to String+Any, wouldn't Any+String still be in the
> library? I ask this just because I want it completely gone, not just
> halfway. :-) Particularly now that we might well get string templates.
I don't draw that distinction when it's arbitrary. If something is
auto-imported into every scope and either impossible to avoid or close
enough, then those are its relevant qualities. (Technically String.+
is implemented with compiler magic, but could have been an implicit
conversion; Any.+(String) is an implicit conversion, but could have
been compiler magic.)
Debasish Ghosh
On Sat, Mar 17, 2012 at 11:13 PM, Debasish GhoshDefinitely not. The selected features are ones which I would prefer to
<ghosh.d...@gmail.com> wrote:
> What's the major force towards this ? Is it only to present a smaller
> surface area to newcomers in Scala ?
>
be explicit about in every codebase I collaborate in. For instance, if
I accidentally use a structural type I want to be warned about it,
mostly because I care about performance. Or, if someone I collaborate
with uses postfix operators I also want to see that upfront. And so
on. So I think the flags will be valuable for most Scala codebases,
with typical exceptions being DSLs.
The main feature left out are implicits. Implicits are probably
misused more often than any other part of Scala, but requiring them to
be put wholesale under a flag would mean we think all type-class
oriented programming is a special case, and that's absurd. So we'd
need a crisp definition what constitutes a problematic implicit
(implicit conversions, maybe?) I could not convince myself of a good
criterion so far, so I left them out.
Cheers
-- Martin
Debasish Ghosh
http://manning.com/ghosh
Twttr: @debasishg
Blog: http://debasishg.blogspot.com
Code: http://github.com/debasishg
Erik Osheim
made very similar points. I send this to echo them and add support.)
From my point of view, this seems like something that could be
addressed with finer-grained control over generating warnings, as well
as options to turn warnings into errors.
For instance, if I want to be warned about a certain feature (because
it's "slow" or "dangerous" or whatever) I can enable warnings which let
me know when the thing occurs. If I *really* don't want this feature
(for instance I am a manager at Acme Boxes, Inc), I can make those
warnings (or maybe all warnings) become compile errors. My coders might
hate this but it would enforce the standard.
Meanwhile, it leaves the rest of us alone (or possibly requires us to
add a few compiler flags to build.sbt). I already have a bunch of these
(like -deprecated) so I wouldn't necessarily mind adding some more to
be able to use e.g. higher-kinded types.
This won't solve the problem that many of us in the community are
writing code that is perceived as "confusing" or "complex" or
what-have-you. But if the intention is to effect a real change in the
kind of code the community is writing I think the SIP is trying to
address a "social problem" with technology, which is often not a great
idea.
Basically, I am sympathetic to the idea that given developers (or
teams) will want to be able to limit their "feature exposure" but I
oppose mandating per-file-imports by default.
-- Erik
Dan Rosen
tl;dr
Jean-Laurent Huynh
> Features that really are experimental could be kept as an import
> language.enable._ so that people are alerted that they shouldn't trust how
> they behave.
If using macros only requires
import language.macros
It could give the illusion that the feature is stable. The compiler
flag is an effective deterrent that should be there until something is
no longer experimental.
--Jean-Laurent
On Mar 17, 2:07 pm, Rex Kerr <icho...@gmail.com> wrote:
> The big problem with this proposal is that it threatens to add a bunch of
> boilerplate to every single existing Scala file and program and REPL
> session. With all these things off by default, I would argue that Scala is
> severely curtailed; why make everyone in every file and script type in
> "import language._" in order to enable the features that make you use Scala
> instead of something else? And then, you get not only the features of a
> present-day typical Scala distribution, but early versions of features that
> are not yet fixed; if you want "full power but stable" Scala, you end up
> with quite a mouthful:
> import language.{postfixOps, structuralTypes, higherKinded, existentials}
>
> language features, that we flip the sign: instead of requiring import
> language.structuralTypes to get structural types, import
> language.disable.structuralTypes if you want them to go away. If you're in
> a development environment with strict coding rules anyway, you probably
> will have half a screen of stuff at the top of every file anyway (copyright
> notice, imports of a bunch of libraries, etc.), so one more line doesn't
> really hurt that much.
>
> Features that really are experimental could be kept as an import
> language.enable._ so that people are alerted that they shouldn't trust how
> they behave. For example, macros would be a good candidate here since
> they're not ironed out. Furthermore, some things that are compiler flags
> could be moved to language.enable (such as continuations), where they
> arguably more sensibly belong.
>
> (Actually, even more sensibly, they'd be annotations on the file.)
>
Francois Armand
On Sunday, 18 March 2012 04:04:21 UTC+1, Paul Phillips wrote:
On Sat, Mar 17, 2012 at 7:49 PM, Daniel Spiewak <djsp...@gmail.com> wrote:
> Could we get a compiler switch to turn that feature off? That's actually
> something I could get behind…Yeah, I have a whole list of stuff I'd like to turn off, and this SIP
misses my list completely. Auto-coercions between primitives,
any2stringadd, auto-tupling...
The only thing I can see which I have any enthusiasm for is developing
some mechanism to control reflection. But this would not be a
satisfactory mechanism.
We'd be so much better off putting effort into something like checkstyle.
Trond Olsen
Last time some criticism hit Scala you responded with the Scala levels which I felt was a rushed response. I feel the same here. This is not something, at a whim, that should be put into 2.10 on such a short notice just to counter criticism of Scala. I don't think it'll help at all. Does macros really exert such a heavy toll on the complexity budget of Scala to warrant this?
Francois Armand
Oh yes please !
Francois Armand
On Sunday, 18 March 2012 02:03:26 UTC+1, Peter C. Chapin wrote:
In general I think modularization of large things (for some definition of “large”) is good. Scala is growing rapidly and new features are being added or proposed at a steady rate. What will happen to Scala if this continues indefinitely?
Features will start interacting in surprising ways causing an ever increasing number of “gotchas” that programmers will have to worry about. The problem is increased when compatibility with legacy code is thrown into the mix.
Well, start to modularize the standard library, and not the language ? And hidding features is not removing them, you (or the Scala dev) will still have to take care of that. Better deprecating and actually removing features.
Modularization would allow teams to take some control over the expansion of the language. “Feature X is cool, but we don’t need it. We won’t turn it on and we won’t get any ugly surprises due to some subtle backward-incompatible aspect of feature X. Plus the compiler can optimize our code better if it knows it doesn’t have to deal with feature X.”
But in that case, why not just use some kind of checkstyle ? Far more intrusive for the vast majority, and it allows each team to fine tune what they really want.
It’s not the complexity of Scala now that concerns me; it’s the complexity of Scala twenty years from now that I’m worried about. What language can endure continuous growth without eventually falling over under its own weight (think C++)? Perhaps Scala can if it supports a language modularization feature.
Again, evolving the language to keep it simple is really different than hidding features. If ithat is the goal of the proposal, then better thinking deprecated and removing these feature. And at least, that would send a clear signal about what is not considered "the Scala way".
As an aside Ada has a feature that allows programmers to selectively turn off certain features (pragma Restriction). It isn’t used much but it is used, for example, to simplify the language for purposes of static analysis or to reduce the runtime system requirements, etc. There is definitely value in it, although I notice that Ada’s approach is to present the whole language by default and allow the program to contract it when appropriate.
Yeah... And we see how well it works, even in extremelly controled environment like, say, Ariane 5 dev ?
What generally happen is that these things are quickly turned on a priori and forgotten. And that quite different than a checkstyle, where team policy are checked at each compilation.
From: scala...@googlegroups.com [mailto:scala-sips@googlegroups.com] On Behalf Of Channing Walton
Sent: Saturday, March 17, 2012 20:26
To: scala...@googlegroups.com
Subject: Re: [SIP 18] New SIP: Modularizing language features
If all one has to do is import something, then those that want to use those features will do so, those that don't, will not. How is that different from simply having all the features 'turned on' and letting people write the code they want to? You can't use those feature by accident! Or can you?
Ivan Porto Carrero
Francois Armand
I do liked your essay, and second most of it!
Thank you to have articulated what I was struggling to write for hours (the social/technology aspect, chaing several goals with one bullet, the importance of documentation and explanation...)
So thanks again !
Eugene Burmako
Sean Parsons
On Saturday, 17 March 2012 19:48:40 UTC, Martin wrote:
Here's a new SIP which proposes to put some new and existing advanced
language features under an import flag. I expect we'll have an
interesting discussion about this one.
So, please, fire away!Here's the link to the SIP:
https://docs.google.com/document/d/1nlkvpoIRkx7at1qJEZafJwthZ3GeIklTFhqmXMvTX9Q/edit
Cheers
-- Martin
On Saturday, 17 March 2012 19:48:40 UTC, Martin wrote:
Here's a new SIP which proposes to put some new and existing advanced
language features under an import flag. I expect we'll have an
interesting discussion about this one.
So, please, fire away!Here's the link to the SIP:
https://docs.google.com/document/d/1nlkvpoIRkx7at1qJEZafJwthZ3GeIklTFhqmXMvTX9Q/edit
Cheers
-- Martin
Miles Sabin
> As a tool to allow stylistic choices to be made by groups or companies,
> options in individual files would have to be enforced by some external
> factor as it is. If this SIP is to provide guidance for a project, then why
> is it defined per file? How would anyone know without grepping the code base
> that I've turned on structural types in some .scala file deep within the
> code base?
Bang on ... even with the SIP-18 proposals in place you would _still_
need an additional external tool to verify that the expected
restrictions were in force.
Cheers,
Miles
--
Miles Sabin
tel: +44 7813 944 528
gtalk: mi...@milessabin.com
skype: milessabin
g+: http://www.milessabin.com
http://twitter.com/milessabin
http://underscoreconsulting.com
http://www.chuusai.com
martin odersky
> Very recently I've been doing some internal Scala training, so I'm quite
> sensitive to the complexity argument myself and if I had to add "You'll need
> to use this import statement, that isn't really an import statement, to use
> this feature." it would have zero benefit to those people learning Scala at
> all and would likely confuse people new to the language even more.
>
in Python. People don't have a problem with them.
> As a tool to allow stylistic choices to be made by groups or companies,
> options in individual files would have to be enforced by some external
> factor as it is. If this SIP is to provide guidance for a project, then why
> is it defined per file?
You can define it also globally per compiler option.
> How would anyone know without grepping the code base
> that I've turned on structural types in some .scala file deep within the
> code base?
I assume that code is at least cursorily reviewed. So an explicit
import will be noticed.
> For an existing example of why such a feature is terrible one
> need only look at the Option Strict statement from Visual Basic
> .Net: http://msdn.microsoft.com/en-us/library/zcd4xwzs(v=vs.80).aspx I
> personally lost a day due to someone changing that setting in just one file
> in a large project, because it's almost action at a distance, especially for
> users that have their editors fold up imports.
>
I agree that Open Strict is bad, because it changes the semantics of a
language. SIP 18 does not generally do that, it just gives warnings
which go away when you add an import. Or, almost. The structural types
inference would change the semantics and that is indeed something we
have to be careful about.
> While there are only 6 options currently, is there any guidance around what
> should and shouldn't be included in the future? If there are say 15 (which
> seems optimistically low) in 10 years that sounds like a combinatorial
> nightmare for the compiler, tools like IDEs and end users.
>
I would like to keep the options low. I do not see any combinatorial effects.
> For library developers (and therefore the whole ecosystem) I can see a few
> problems and burdens that it creates:
> 1.) Libraries will have to work with as few features as possible to allow
> them to be as widely used as possible. This in turn creates complexity over
> whether or not a library needs to be split into parts and/or how the
> requirements are communicated. As a result the benefit of said features is
> minimised as the pressure is for them to be never used, creating a downward
> pressure spiral between the need and the use of a feature.
And, if you ask me, that's a good thing! Type system surface is a cost
in maintainability and understandability, which is all too happily
ignored by some developers. I won't name names but I have seen several
large codebases that made me rip my hairs out for their needless
complexity. And yes, these tended to use the features I would like to
flag.
> 2.) Supporting all the new users that go "I tried your sample code and it
> didn't work." as a result of those features not being enabled.
An import statement belongs into documentation just like other imports
would. Besides, if the statement was missing, users would get a
warning or an error that explained to them that they have to turn the
import on. Nothing mysterious about it. Then when the import is on,
they can hyerlink on the imported flag to get more documentation about
the feature and its risks.
> 3.) Testing that libraries work correctly with and without various features
> being enabled, just in case turning on feature X causes some unexpected
> result. This one is terrifying to me, having just recently written a JSON
> library, it would be onerous to add tests for some arbitrary combinations.
>
That's a good point, in in light of that point I retract my proposal
to make type inference for structural types be controlled by a flag.
So absence of a flag would only warn about a feature, it would never
change behavior.
> At the core of what I think about this, the answer to complexity is _never_
> more complexity.
>
I think that's wrong. For instance, normal import statements add
complexity to a language, yet they avoid the complexity of a global
namespace that gets too large. In fact, I think in most evolved
systems we contain complexity with complexity. Take guards of
chainsaws as an example.
Cheers
- Martin
martin odersky
>> I disagree. As long as a feature is specified in the SLS it is in the
>> language. The fact that it requires an explicit import does not change
>> that.
>
>
> Ok, I'll buy that. It seems you're viewing the explicit import not as a
> pragma, but as a lexically scoped flag. Still, horrifying to contemplate
> from a formal perspective, but not as horrifying as I initially thought.
>
> For the record, the overloading of import is still really problematic,
> precisely because overloading anything tends to introduce a lot of
> complexity and cognitive burden. By definition, it requires one to
> consciously consider whether this construct refers to concept A or concept
> B. The proposed "language import" is a very different concept from a
> library import and has very different effects. Lumping it under the same
> syntactic construct seems…wrong, to put it mildly.
I think we disagree here. If we want to put it in the language (as
opposed to a lint or checkstyle tool), then we should control it with
real syntax, not with pragmas or switches (even though I'd allow
switches as an alternative). And import will work very well for that,
I am convinced.
>
>> I do not see why not.
>
>
> Weren't you the one telling me that the typer is already too complex? ;-)
>
Yes, because of existentials and higher-kinded types and lubs. This is
trivial by comparison. :-)
Cheers
- Martin
martin odersky
>
>> How is requiring an explicit import the same as disabling a feature?
>
>
> The explicit import isn't an import at all, but a compiler pragma in the
> syntactic guise of an unrelated language feature. As with any compiler
> pragma, the net effect is to change the behavior of the compiler, and thus,
> the language.
>
As responded to other mails, I agree we have to be careful not to
change the behavior of the language, except for requiring the flag.
I believe explicit imports will work pragmatically much better than
the alternatives. Python has shown this.
> If a feature is disabled by default, then it is not in the language (by
> definition). This is why Haskell discussions separate "Haskell" from
> "Haskell with GHC extensions".
>
>> I don't like explicit disabling. It violates the principle that simple
>> things should be kept simple. And it complicates the machinery
>> considerably. Now you'd have to explain how to resolve an enabling
>> import that clashes with a disabling flag. I don't want to get into
>> that!
>
>
> I would remove the import mechanism entirely. Pragma was evil back in the
> days of C. Introducing it to Scala, especially when disguised as an
> orthogonal feature, is just begging for pain (like the pain of
> disambiguating compiler switches vs imports).
>
> I think the compiler switch is still keeping things simple. In fact, I
> think it keeps things simpler than the import mechanism, which introduces a
> very weird and non-orthogonal feature into the core language semantics. A
> compiler switch enables teams to make and control decisions about these
> features at a high level. It's much easier to police your build.sbt than it
> is to police an entire codebase, grepping for a construct that uses the same
> syntax as another (very commonly-used) one. Additionally, compiler switches
> are generally understood to affect the semantics of the compiler. That is,
> after all, their purpose. Thus, they fit precisely within the template
> concept that already exists. Using compiler switches doesn't introduce any
> complexity, but using an "import" does.
>
> How does the explicit "import" interact with scoping? Can we have certain
> typer features activated for part of our file but not another part? What if
> higher kinds are enabled inside a method but not outside? What error do we
> get if features "leak" through type inference? There's just a whole host of
> questions that arise when you start introducing language features – you must
> know this better than I! Compiler switches are the simpler mechanism, by
> far.
>
The SIP is explicit about this: Higher kinded types: only explicit
definitions are affected. Existential types: inferred types are also
affected.
-- Martin
Paul Phillips
On Saturday, March 17, 2012, Erik Osheim wrote:
From my point of view, this seems like something that could be
addressed with finer-grained control over generating warnings, as well
as options to turn warnings into errors.
-Xfatal-warnings Fail the compilation if there are any warnings.-Xlint Enable recommended additional warnings.
martin odersky
> As someone alluded to, this would indeed substantially complicate adoption and usage at large firms, like Monsanto (views are my own, not Monsanto's).
>
> Our project, the first Scala project here, has now internally gained substantial support. We make use of all of these proposed "modularized" features, and certainly don't need the stigma of them being "dangerous," "difficult to maintain," "too complicated," "too slow," "impossible to hire for," or anything else.
Frankly, if you do indeed use all of these features, that would be a
red flag for me. You'd have to convince me about the long-term
maintainability of your project. An explicit import is a good nail on
which to hang the argument for the inclusion of a feature.
> The stigma associated with using Scala at a large, traditionally Java firm is sufficiently burdensome without this SIP. Please don't frustrate adoption further by implementing this SIP. This isn't helping by making Scala simpler, it is making it scarier to use the very features that make Scala high value (+1 on all Daniel Spiewak's comments).
>
> Given the volume of highly negative feedback on this SIP, and years of complaining about more pressing (and high profile) issues like performance optimization of loops, is this *really* the most important SIP to pursue right now given limited resources?
>
Don't worry. Loops will be optimized in 2.10.
Cheers
- Martin
Sébastien Bocq
> On Sat, Mar 17, 2012 at 11:56 PM, Daniel Spiewak <djsp...@gmail.com> wrote:
>> Color me horrified.
>>
>> Ok, maybe that was too strong. Honestly, I don't see anything positive
>> coming out of a change like this. As Rex pointed out, it just creates
>> import/build-config boilerplate for, well, everyone. Even for the few who
>> don't use these features (knowingly or unknowingly), these switches have no
>> beneficial effect. You are hiding a language feature that these people
>> haven't seen. By definition, this will be a completely invisible change for
>> a few people and a significantly detrimental change for a majority.
>>
> I believe my development experience is different than yours then. In
> codebases I contribute to, these features are very rarely used. To a
> point, where I would welcome a warning if they are indeed used. So I
> believe it will be an invisible change for the majority, a welcome
> change for most people on the borderline, and a small nuisance for a
> few.
>
My experience match yours. I think the SIP will make the language more
friendly for newcomers and motivate advanced programmers not to overly
use language features where they are not strictly needed.
I'd emit a reservation on restricting the explicit usage of
higher-kinded types because they are to me a core feature of the
language and using them teaches you a lot. if you do it wrong, if you
don't understand what you're doing or if you try to abuse them, your
code will become quickly too complex or difficult to compile. So, I
don't see the necessity to disable them by default. If it is to hide
that they sometime pushes type inference to its limit, then it is for
the wrong reason. And by the time you reach that point you probably
have already a good understanding of what you're doing anyway.
One other small note. The SIP says: "Features that existed in Scala
2.9 and that are now controlled by a feature flag will trigger a
warning when they are used where the flag is not imported. In a future
major Scala version that warning might be turned into an error.". Why
limit this to features of Scala 2.9 only? I would make this a
requirement for any feature flag. Without this, newcomers might not
even know that what they want is possible and that there is actually a
name for it. I'm speaking out of personal experience with pragmas in
Haskell ... it doesn't help with the learning curve.
--
Sébastien
martin odersky
clearly state the remedy, that you can make the error go away by
importing such and such feature.
-- Martin
Stuart Roebuck
Sciss
I am strongly opposed to this SIP. While its motivation is to make Scala 'less complex', it achieves quite the opposite. It re-uses an existing keyword, `import`, and "overloads" it with new meaning. It doesn't even adhere to the "normal" behaviour of `import`: "Wildcard imports from language have no effect".
The SIP complicates the language by toggling the addition or removal of keywords from the language. And this on a per file basis. It kind of states itself what it should be: "The values ... are called feature flags" -- so, there is a well defined mechanism for that, which is compiler flags.
I rarely use a specific language feature just in a single source code file, so the need to put an import statement into each and every file arises, whereas it would be transparent as a compiler flag. I don't see much difference between enabling or disabling macros, and doing the same with continuations.
The selection of features is highly heterogeneous. Higher kinded types are an essential feature of a functional programming language, switching it off doesn't make any sense. I am curious why it appears in the list, while implicits do not appear in it. They are also an essential feature of the language, providing the type class analogy in Scala. Disabling the _creation_ of higher kinded types doesn't make any sense, as you will still be using them as soon as you write `List( 1, 2, 3 )`. So while intending to make things less complex, you create a confusion by trying to hide a feature while it still is there.
I don't think there are many serious libraries out there which make no use of implicits and higher kinded types. This is after all why we use Scala and not Java. Hiding those features under an import tax is just a form of castration.
The list is heterogeneous for another reason: Most features described are _explicit_ features, you don't accidentally write a macro or mixin the Dynamic trait. You don't accidentally define a higher kinded function. On the other hand, there is one case which indeed _does_ occur accidentally, and that is structural typing. The example you give:
val v = new Object { val member = 1 }
I think most people have run into this by mistake. But then again, to hide that under an import is making it just more difficult to read -- you would need to look up the source code to see whether a particular declaration is meant as a structural type or not. Another thing is that you are creating a language construct to hide something which has to do with behind-the-scenes performance issues, which may well change in the future -- there may well be a point where using a structural type is not a penalty at all. A language statement should correspond to meaning and not implementation or hardware details.
There are two well known better solutions to protect from accidentally writing structural types:
(1) Rely on the IDE highlighting -- this is a non intrusive form, and I use it quite a bit. I don't know the Eclipse plug-in that well, but the Scala plug-in for IDEA has a powerful set of warning highlights which address typical problems. So it would be really easy to add a warning highlight against accidental usage of structural typing.
(2) Just as you have `@tailrec` as a means to check positively for tail recursion, or `@switch` to make sure a match is optimised, you could have a "negative" or "overriding" annotation (so, more like `@unchecked`!) to say that you _do_ want a structural type. You would have something like
val v: @structural = new Object { val member = 1 }
This is the only place where `@structural` would be needed, because it is the only place where you can have structural types by accident. Consider `def test( abortable: { def abort(): Unit })` instead -- there is no way that the type was introduced accidentally, thus no need for `@structural`.
A statement like the above without `@structural` would produce a compiler warning or error.
And lastly, for the postfix syntax. I use it rarely, so I cannot really comment on it. But disabling it by default means that every user of a particular DSL needs to import a language feature to use the DSL. Doesn't make much sense to me, either.
Summary: Please keep things simple. This SIP doesn't make things any simpler, it just adds more verbosity, patronises the user and nowhere contributes to the goal of making Scala more accessible by Java people.
Best, .h.h.
martin odersky
> I read the SIP 18, here is my opinion...
>
> I am strongly opposed to this SIP. While its motivation is to make Scala 'less complex', it achieves quite the opposite. It re-uses an existing keyword, `import`, and "overloads" it with new meaning. It doesn't even adhere to the "normal" behaviour of `import`: "Wildcard imports from language have no effect".
>
concept of a named import in the spec. So we'd need to say:
Feature X is enabled if the member X of the scala.language object is
available in scope via a named import. That's pretty easy.
> The SIP complicates the language by toggling the addition or removal of keywords from the language. And this on a per file basis. It kind of states itself what it should be: "The values ... are called feature flags" -- so, there is a well defined mechanism for that, which is compiler flags.
Technically, keywords would remain keywords independently of flags.
The only exception is macro, and that only for a transitory period of
one major release. We can't claim macro as un unconditional keyword
right now because code might break. In fact, it was the idea of the
import syntax that convinced me in the end that the proposal might
fly. I think it would be much more problematic if it relied on flags
only because then you guide what's a language rule by an
extra-language feature.
>
> I rarely use a specific language feature just in a single source code file, so the need to put an import statement into each and every file arises, whereas it would be transparent as a compiler flag. I don't see much difference between enabling or disabling macros, and doing the same with continuations.
The current proposal would let you write compiler flags as an alternative.
>
> The selection of features is highly heterogeneous. Higher kinded types are an essential feature of a functional programming language, switching it off doesn't make any sense. I am curious why it appears in the list, while implicits do not appear in it. They are also an essential feature of the language, providing the type class analogy in Scala. Disabling the _creation_ of higher kinded types doesn't make any sense, as you will still be using them as soon as you write `List( 1, 2, 3 )`. So while intending to make things less complex, you create a confusion by trying to hide a feature while it still is there.
>
Not at all. I think we all agree that List(1, 2, 3) is perfectly fine.
It required some serious engineering in the factories to make it
happen. After all that work, I would not mind at all by showing the
secret source of that engineering with an
import language.higherKinded
in the few files that actually define higher-kinded types in the
library. But using these higher-kinded types should be a conscious
decision which should be documented IMO. The SIP gives other reasons
as well.
> I don't think there are many serious libraries out there which make no use of implicits and higher kinded types. This is after all why we use Scala and not Java. Hiding those features under an import tax is just a form of castration.
>
I think that's put a bit too strongly. All that's required is be
explicit about it. Nothing is taken away.
-- Martin
Jorge Ortiz
Paul Phillips
> having a type be inferred as structural would almost always be undesirable.
>
> Likewise, types inferred as existential types are almost never what you
> want.
As someone who has experimented heavily with modifications to type
inference, I have to say you are wrong on both counts. There is
nothing quite so edifying as implementing a change to what is inferred
and then seeing where the compiler fails to compile itself. And
that's on a single source base which makes no effort to exercise every
bit of the dance floor. Just because you do not intentionally expose
existentials as principal types, that doesn't mean you don't depend on
their being inferred in intermediate expressions.
> To give you an idea of the magnitude of the problem, the
> textual representation of the inferred type was 225kb. Yes, kilobytes.
I just sent a 285 kb inferred type out which no amount of warning
about existentials could have prevented.
ARKBAN
the word "import" was good or bad, so I decided to see if translating
the discussion from English to Scala might help me:
What we have today:
/** Mimics the work of the "import" statement */
class Importer {
def import( path:Seq[String] ) {
if ( isFullQualifiedPath( path ) ) addAllClasses( path )
else addSingleClass( path )
}
}
def doImports( importer: Importer ) {
importer.import( Seq( "scala", "collections", "immutable" ) )
importer.import( Seq( "scala", "collections", "mutable", "HashSet" ) )
}
doImports( new Importer )
What SIP-18 is proposing
/** Mimics the work of the "import" statement */
trait Importer {
def import( path:Seq[String] )
}
/** Same as previous implementation of Importer */
trait ClassImporter {
def import( path:Seq[String] ) {
if ( isFullQualifiedPath( path ) ) addAllClasses( path )
else addSingleClass( path )
}
}
trait LanguageImporter {
def import( path:Seq[String] ) {
if ( isFullQualifiedPath( path ) ) throw new
IllegalArgumentException("wildcards disallowed by SIP")
else addSingleLanguageFeature( path )
}
}
def doImports( importer: Importer ) {
importer.import( Seq( "scala", "collections", "immutable" ) ) //
fails for LanguageImporter
importer.import( Seq( "scala", "collections", "mutable", "HashSet" )
) // fails for LanguageImporter
importer.import( Seq( "language", "macros" ) ) // fails for
ClassImporter AND LanguageImporter (SIP disallows wildcards)
importer.import( Seq( "language" ) ) // fails for ClassImporter
}
doImports( new ClassImporter )
doImports( new LanguageImporter )
(My translation is not perfect and its surely not the code that exists
in the compiler; creating production-ready code was not the intent of
this exercise for me.)
The new design has two different Importer implementations that are
incompatible. If someone saw this in code they would say that the
implementations and the design violates the Liskov substitution
principle. The LanguageImporter specifically violates the rule about
strengthening preconditions -- wildcards are disallowed by the SIP.
For me this exercise made it clear to me that reusing "import" for class
importing and language features is confusing. I hope this exercise is
useful to others (perhaps I've got it all backward and someone else's
code will show that side).
ARKBAN
P.S.: I imagine those authoring IDE plugins are also going to struggle
with two different behaviors for the same keyword depending on the context.
martin odersky
have now added the precise formulation that's needed to the SIP. It
does not touch on the semantics of imports at all; there are no two
kinds of import statements.
Cheers
- Martin
martin odersky
> On Sun, Mar 18, 2012 at 6:44 AM, Jorge Ortiz <jorge...@gmail.com> wrote:
>> having a type be inferred as structural would almost always be undesirable.
>>
>> Likewise, types inferred as existential types are almost never what you
>> want.
>
> As someone who has experimented heavily with modifications to type
> inference, I have to say you are wrong on both counts. There is
> nothing quite so edifying as implementing a change to what is inferred
> and then seeing where the compiler fails to compile itself. And
> that's on a single source base which makes no effort to exercise every
> bit of the dance floor. Just because you do not intentionally expose
> existentials as principal types, that doesn't mean you don't depend on
> their being inferred in intermediate expressions.
>
>> To give you an idea of the magnitude of the problem, the
>> textual representation of the inferred type was 225kb. Yes, kilobytes.
>
> I just sent a 285 kb inferred type out which no amount of warning
> about existentials could have prevented.
>
> http://permalink.gmane.org/gmane.comp.lang.scala/25959
I believe we can get rid altogether of those monstrosities in the
future IF we start curtailing existential types now.
Cheers
-- Martin
ARKBAN
It does not touch on the semantics of imports at all; there are no two kinds of import statements.
Cheers - Martin
martin odersky
lt's something else. The semantics of the import is not changed at
all, and there is no side effect. The _visibility of certain imported
identifiers_ then drives features. A feature flag is a proxy for the
availability of that feature. Honestly, I fail to see how it could get
any clearer than this.
- Martin
Paul Phillips
> Do you have some examples of that? Would be good to see them.
If we don't infer refinements, we turn type members into even
second-classier citizens. See appended. Also, constructs like this
one - which can be very handy, reflection be damned - would cease to
function entirely:
scala> implicit def imbueWithPowers(xs: List[Int]) = new { def bippy =
"I'm bippy!" }
imbueWithPowers: (xs: List[Int])Object{def bippy: String}
scala> Nil.bippy
res2: String = I'm bippy!
When it comes to existentials, I have seen it many times but I don't
have any prefab examples. The pattern matcher is often a major
participant, but not always. The usual manifestation of failure is
something failing to typecheck which used to, due to it no longer
conforming to bounds, because the evidence that it did conform to
bounds was being propagated by one of the less obvious mechanisms,
which was impacted in some subtle way by whatever innocuous-seeming
change I was investigating.
As a general rule, I have noticed that people quite freely take
advantage of very delicate arrangements in the type system - because
they don't know that they're doing so, of course. If something works,
it works! I feel the same way. But it means that in most cases people
don't even know what it is that they depend on, so the degree to which
they can articulate their needs is limited.
And I'm all for making these things more transparent, as I always have
been; I simply want to do that without taxing the most dedicated
users.
// type parameters vs. type members
class X1 ; class X2 extends X1 ; class X3 extends X2
trait A {
type T
def f: T
}
class A1 extends A {
type T = X2
def f = new X2
}
class A2 extends A {
type T = X3
def f = new X3
}
trait B[T] { def f: T }
class B1 extends B[X2] { def f = new X2 }
class B2 extends B[X3] { def f = new X3 }
object Test {
// List[A{def f: X2; type T >: X3 <: X2}]
def f1 = List(new A1, new A2)
// List[B[_ >: X3 <: X2]]
def g1 = List(new B1, new B2)
}
ARKBAN
From a pure language perspective I see now that there is no
side-effect. From a developer experince there is a behavior change,
call it either a direct-effect or side-effect or whatever. That is my
concern with reusing the word "import", I can't think of anyway to
explain it to others except for "but when you import from language..."
Honestly, I fail to see how it could get
any clearer than this.
I'm sorry that I am causing frustration for you. I'm really not trying
to be difficult. I'm trying to understand the distinction so that when I
lead the transition at my company to Scala 2.10 (or which ever version
contains this SIP), I can explain to my co-workers so they understand.
If I screw up and explain the concept poorly I leave many people with a
poorer impression of Scala, and that's not something I want to do.
>
> - Martin
I hope I've made it clear that I think a different keyword should be
used. That was all I was aiming for. I was hoping the Scala code
translation exercise would help with that but I don't think it did, we
seemed to get bogged down in a discussion about the technical nuances.
ARKBAN
Sciss
so what do you anticipate that happens in a developer's brain -- do you really think they ask themselves: "Oh, wait a second. Higher kinded types are an advanced language feature. Lets evaluate, if they are really necessary here, and it justifies me adding import.higherKindedTypes" --- seriously. I think you make a decision on what the purpose of a library or class is, and that determines whether you need HK or not. I don't think anyone needs the extra 'reminder'.
Erik Osheim
> First of all: this is a tempest in a teacup. The cries of pain from
> "expert" Scala users that they are being unduly punished ring hollow.
> Presumably, expert users of Scala are competent users of their build system
> and can trivially add compiler flags to their heart's content.
I largely agree with your email, but this is not a totally correct
characterization.
It seems like we're not talking about compiler flags but adding import
statements, right? Editing build.sbt (or whatever) once is a lot
different than possibly having to add imports to N files. If we're
talking about more fine-grained warning/error options for scalac then
it's a different kettle of fish.
Quibbles aside, your observation that linter is the place to try to
address these issues is a good one.
-- Erik
Sciss
Which are the things that make Scala more complex? I very much doubt that the SIP actually comes near its proclaimed target.
Example 1: Higher-kinded types (for me) are much simpler than type members and path dependent types. I struggle a lot with the latter when designing a library.
Example 2: Controlling side-effects is much more difficult that implicits. These are the real 'implicit' problems. Addressing the latter instead of the former just shows that the SIP is motivated by pressure from Java adopters which do not care about side effect and thus don't see it as complexity, while it actually is.
Example 3: Symbols in method names ( |@|, <<, >>=, <*>, ... ). The sbt, scalaz, and lift phenomen. More difficult to understand than postfix syntax.
...
So to make the language simpler, in my opinion, other things need to be addressed. If you manage to make the developer think about how side effects occur and are handled, much more is achieved than making them get annoyed and developing a reflex of adding `import language.{higherKinded, implicitConversions, allowSymbolicMethodNames}` at the top of each new file they create. (i'm going to have that in my IDEA new-file-template i guess).
Seth Tisue
> Experienced Scala developers will simply turn everything on.
Nonsense. I am an experienced Scala developer. And like Martin, I
would rarely enable even one of these features, let alone all of them.
--
Seth Tisue | Northwestern University | http://tisue.net
lead developer, NetLogo: http://ccl.northwestern.edu/netlogo/
Daniel Spiewak
> For the record, the overloading of import is still really problematic,I think we disagree here. If we want to put it in the language (as
> precisely because overloading anything tends to introduce a lot of
> complexity and cognitive burden. By definition, it requires one to
> consciously consider whether this construct refers to concept A or concept
> B. The proposed "language import" is a very different concept from a
> library import and has very different effects. Lumping it under the same
> syntactic construct seems…wrong, to put it mildly.
opposed to a lint or checkstyle tool), then we should control it with
real syntax, not with pragmas or switches (even though I'd allow
switches as an alternative). And import will work very well for that,
I am convinced.
The overloaded import is a pragma, it's just disguised as something else. Worse, the thing it's disguised as has very different semantics from the thing it is. So, the mental box into which imports are slotted has to take on another dimension of wrinkles. A few people have already pointed out that the analogy to Python's import from Future isn't really accurate, since Python imports already have very different semantics from Scala's, semantics more in line with the sort of behavior import from Future provides.
If we must have syntax for this (a conclusion I still strongly disagree with), it should be something that reflects the uniqueness of the construct. In other words, we shouldn't overload something that's already in the language. Nothing in the language behaves anything like what you're proposing, therefore new syntax is required if syntax is to be employed.
Daniel Spiewak
Frankly, if you do indeed use all of these features, that would be a
red flag for me. You'd have to convince me about the long-term
maintainability of your project. An explicit import is a good nail on
which to hang the argument for the inclusion of a feature.
The Precog codebase uses all of the features you propose to flag (except for dynamic and macros). We're pretty dependent on them, as a matter of fact. The Novel Vibe code base (may it reset in peace) used type constructor polymorphism and existentials (no structural types). The gll-combinators codebase uses type constructor polymorphism and existentials, as does the anti-xml code base. Maybe I just carry complexity with me; I don't know, but code that I work on always seems to leverage these features, and always for good reasons.
Daniel
Daniel Spiewak
Which are the things that make Scala more complex? I very much doubt that the SIP actually comes near its proclaimed target.
As far as I can tell, the point of this SIP is not to address the "Scala is too complex" meme. Early posts in the conversation assumed that it was and soundly trounced its effectiveness in that direction. This SIP seems more about explicitly controlling the use of more advanced and (arguably) less fundamental features in the language.
Daniel
martin odersky
>
>> > For the record, the overloading of import is still really problematic,
>> > precisely because overloading anything tends to introduce a lot of
>> > complexity and cognitive burden. By definition, it requires one to
>> > consciously consider whether this construct refers to concept A or
>> > concept
>> > B. The proposed "language import" is a very different concept from a
>> > library import and has very different effects. Lumping it under the
>> > same
>> > syntactic construct seems…wrong, to put it mildly.
>>
>> I think we disagree here. If we want to put it in the language (as
>> opposed to a lint or checkstyle tool), then we should control it with
>> real syntax, not with pragmas or switches (even though I'd allow
>> switches as an alternative). And import will work very well for that,
>> I am convinced.
>
>
> The overloaded import is a pragma, it's just disguised as something else.
> Worse, the thing it's disguised as has very different semantics from the
> thing it is. So, the mental box into which imports are slotted has to take
> on another dimension of wrinkles. A few people have already pointed out
> that the analogy to Python's import from Future isn't really accurate, since
> Python imports already have very different semantics from Scala's, semantics
> more in line with the sort of behavior import from Future provides.
>
access low-level system-oriented features. At the time everybody
really liked that. So, I still think import is the right way to
express this. The beauty of import is also that you can scope it
anyway you like. For instance, say you want to admit 3 reflexive calls
in a method but would otherwise like to be warned about
them. You can write:
def foo(x: { def a(); def b(); def c() }) {
import language.reflect
x.a()
x.b()
x.c()
}
I find that very elegant!
- Martin
Sciss
the SIP says it wants to make the use of these features "more explicit" -- but how is
import language.higherKindedTypes
trait MyStructure[ A ]
more explicit than
trait MyStructure[ A ]
? more verbose, yes, but not more explicit.
best, -h.h.-
Rex Kerr
in a method but would otherwise like to be warned about
them. You can write:
def foo(x: { def a(); def b(); def c() }) {
import language.reflect
x.a()
x.b()
x.c()
}
I find that very elegant!
I find that bizarre. First, the structural type appears _before the import_. So either it's a backwards reference, or people have to explicitly import a second feature to make the only possible valid use of the first (imported) feature, doubling the boilerplate needed for no reason.
Second, isn't this what annotations are often used for? Now we have two entirely different sets of annotations--ones with @whatever and ones with import language.whatever. It is almost as if you are proposing a certain type of implicit annotation, except it's not called an annotation (or an implicit) and uses entirely different syntax. But it acts just like an implicit annotation (e.g. @unchecked automatically applied to all places where it is valid).
I'm not sure this SIP adequately generalizes the concept of what it's trying to achieve.
--Rex