I am moving this from scala-user to scala-debate upon request. First, a bit of context (copied):
However, as I've thought more about it, I've realized that what enterprises need is _controllable_ complexity. If you have 2,000 software engineers using Scala (which is what we all hope for, right?), then some of them _will_ use the most complex features because they think it's cool, regardless of corporate guidelines.
[...]
Given, than , I refer back to the (implicit question) of my original post: does it make sense to allow the enterprise to control the degree of complexity of Scala that is used? (I think it does)
I understand that such a topic may be part of the scala language discussions. If so, apologies, I have not seen it.
-- Francois ARMAND http://fanf42.blogspot.com http://www.normation.com
Given, than , I refer back to the (implicit question) of my original post: does it make sense to allow the enterprise to control the degree of complexity of Scala that is used? (I think it does)I understand that such a topic may be part of the scala language discussions. If so, apologies, I have not seen it.Ken
On Nov 19, 2011 9:56 AM, "Kenneth McDonald" <ykke...@gmail.com> wrote:
>
> I am moving this from scala-user to scala-debate upon request. First, a bit of context (copied):
>
> However, as I've thought more about it, I've realized that what enterprises need is _controllable_ complexity. If you have 2,000 software engineers using Scala (which is what we all hope for, right?), then some of them _will_ use the most complex features because they think it's cool, regardless of corporate guidelines.
>
> Next, to put this into my own personal perspective: Until recently, I dismissed arguments that Scala was too complex on the grounds that "you can always use as simply a more productive Java".
>
> More recently, I have seen arguments that convinced me that a significant problems with Scala was that _the possibility of using it an an unduly complex manner restricted its corporate applicability_. I see this as a highly valid point. (Large) corporations need to expect that engineers are, to some degree, interchangeable. I think even Google must have that approach these days. It should be expected that an engineer can be dumped into another engineers' job and be able, int not too long, to pick up on what is going on.
>
> I don't believe this is possible with "full Scala". For example, look at Tony M. and (in addition) scalaz. How many engineers could pick up on code engineered in that universe? Very, very few.
Hi! Look at me!
You keep saying this, yet you still do not recognise that the problem is not with me or scalaz, but you.
Plenty of us can get along just fine with it, and we do exactly that, right now, at this moment. It has about 10 or more willing contributors and many of them write stuff that I do not comprehend. Want to know what I about it? That's right, I set about coming to terms with comprehending that very thing. Why might I do this? Because that very thing is often *useful*. Your "corporate guidelines" are antithetical to utility, and every scalaz committer soon learns that the only criteria for inclusion is usefulness, not bureacratic garbage; quite the contrary.
You keep using "what Ken fails to comprehend" as some kind of synonym for complexity or obtusity. It is so blatantly transparent that you are inventing excuses for that which you have developed an irrational anxiety response and I will call you out on it each time you burden me,personally with your personal issues. Please pull your head out of the sand, for your own benefit. You might even come to the point where you are able to comprehend that actually scalaz is one of the most easy to comprehend libraries out there, just like others who have bothered to acquaint themselves with the subject matter. Do you think I have scalaz memorised or something?
You can stop looking at me now and start looking at yourself.
Finding the optimal ratio of complexity to power is always a goal in
language or library design,
personally, I'm a bit clueless on how Scala's complexity could be
reduced without reducing it's power,
I am sure there are much more capable minds than mine thinking about this.
I am certain however that improving the tools can go a long way to
reduce the overall complexity that
Scala can bring to a project. Auto completion for example in an IDE
can be a powerfull educational tool,
when a user stalls the mouse pointer on a val and the IDE shows the
type, that is a huge complexity
reducer, that is just the begining, if an IDE could show a visual hint
at places in the code where
implicit conversions occur, and provide a link to the conversion function,
that would have a huge educational potential, and the complexity would
in effect have been reduced, without changing anything in the language.
Improvements in other tools could also go a long way, Scaladoc could
be made more explanatory,
for example with implicit params, showing a link to implicit val or
def declarations that applies within
the given library. SBT is another area, it needs to be as simple as
possible because it's the first tool
that a Scala new commer will use.
There is also accidental complexity, unstable or insufficiently
documented tools, etc...
All this can be addressed right now without even debating, not that
debating isn't usefull,
but if someont wishes to reduce the complexity of Scala for new
comers, there are plenty
of things that can be done by just helping improve tools in the ecosystem.
Cheers
ML
2011/11/18 Tony Morris <tmo...@tmorris.net>:
I'm still trying to decide if I think this argument is reasonable, or rationalization.
I mean, C++ was fabulously complex for its day, full of features that you didn't want the average programmer going near, but the world didn't end - a little education and coding guidelines sufficed just fine. (Ditto for Ada 95, but admittedly that didn't achieve such reach.)
I understand the argument reasonably well. But history suggests to me that it is probably more catastrophizing than anything else - a problem mostly solveable by coding guidelines...
While I dismiss the option to disable scala features I see the beauty in being
able to "spot" "critical" sections somehow.
What I am missing currently is a survey giving some basic facts about what
furthers understanding and what blocks understanding if someone is about to
read some given section of code. "Critical" would be anything that makes it
unecessarily hard to understand or follow a concept / solution that is
implemented (the "what it does").
That's not to be mixed with the complexity of a concept/solution in itself. If
I don't understand the concept/solution (the "why") in itself I won't be able
to figure out the meaning of code implementing that concept/solution in any
language.
Existing code metrics (for different languages) try to capture aspects of this
"what it does" problem (and there has been some related research) - but I am
not aware of any good code metrics for scala.
If we had such code metrics we could tag the code and try to align metric
signatures with "comprehension effort". This would give enterprises a valuable
instrument to spot source parts which should be reworked.
I think that would help...
Greetings
Bernd
I am moving this from scala-user to scala-debate upon request. First, a bit of context (copied):However, as I've thought more about it, I've realized that what enterprises need is _controllable_ complexity. If you have 2,000 software engineers using Scala (which is what we all hope for, right?), then some of them _will_ use the most complex features because they think it's cool, regardless of corporate guidelines.Next, to put this into my own personal perspective: Until recently, I dismissed arguments that Scala was too complex on the grounds that "you can always use as simply a more productive Java".More recently, I have seen arguments that convinced me that a significant problems with Scala was that _the possibility of using it an an unduly complex manner restricted its corporate applicability_.
I see this as a highly valid point. (Large) corporations need to expect that engineers are, to some degree, interchangeable. I think even Google must have that approach these days. It should be expected that an engineer can be dumped into another engineers' job and be able, int not too long, to pick up on what is going on.
I don't believe this is possible with "full Scala". For example, look at Tony M. and (in addition) scalaz. How many engineers could pick up on code engineered in that universe? Very, very few.
I know from experience that given enough people, developers who are completely incompetent in a technology will nonetheless use the extreme abilities of that technology. For example, I once spent several nasty months fixing the "code" of a "developer" who knew virtually nothing about C++, but who nonetheless decided she could overload operators at her discretion. Do you have any idea how difficult it is to understand code where "+" means "accumulate into a queue"?
Given, than , I refer back to the (implicit question) of my original post: does it make sense to allow the enterprise to control the degree of complexity of Scala that is used? (I think it does)
I am moving this from scala-user to scala-debate upon request. First, a bit of context (copied):However, as I've thought more about it, I've realized that what enterprises need is _controllable_ complexity. If you have 2,000 software engineers using Scala (which is what we all hope for, right?), then some of them _will_ use the most complex features because they think it's cool, regardless of corporate guidelines.Next, to put this into my own personal perspective: Until recently, I dismissed arguments that Scala was too complex on the grounds that "you can always use as simply a more productive Java".More recently, I have seen arguments that convinced me that a significant problems with Scala was that _the possibility of using it an an unduly complex manner restricted its corporate applicability_. I see this as a highly valid point. (Large) corporations need to expect that engineers are, to some degree, interchangeable. I think even Google must have that approach these days. It should be expected that an engineer can be dumped into another engineers' job and be able, int not too long, to pick up on what is going on.
I don't believe this is possible with "full Scala". For example, look at Tony M. and (in addition) scalaz. How many engineers could pick up on code engineered in that universe? Very, very few.
I know from experience that given enough people, developers who are completely incompetent in a technology will nonetheless use the extreme abilities of that technology. For example, I once spent several nasty months fixing the "code" of a "developer" who knew virtually nothing about C++, but who nonetheless decided she could overload operators at her discretion. Do you have any idea how difficult it is to understand code where "+" means "accumulate into a queue"?
Given, than , I refer back to the (implicit question) of my original post: does it make sense to allow the enterprise to control the degree of complexity of Scala that is used? (I think it does)
I understand that such a topic may be part of the scala language discussions. If so, apologies, I have not seen it.Ken
I assume the code was something like: "q + e". Do you think it would
be easier to understand the code if it was "q.plus(e)" or "q.add(e)"?
--
Daniel C. Sobral
I travel to the future all the time.
On Nov 20, 2011 4:44 AM, "Josh Suereth" <joshua....@gmail.com> wrote:
>
>
>
> On Fri, Nov 18, 2011 at 6:56 PM, Kenneth McDonald <ykke...@gmail.com> wrote:
>>
>> I am moving this from scala-user to scala-debate upon request. First, a bit of context (copied):
>>
>> However, as I've thought more about it, I've realized that what enterprises need is _controllable_ complexity. If you have 2,000 software engineers using Scala (which is what we all hope for, right?), then some of them _will_ use the most complex features because they think it's cool, regardless of corporate guidelines.
>>
>> Next, to put this into my own personal perspective: Until recently, I dismissed arguments that Scala was too complex on the grounds that "you can always use as simply a more productive Java".
>>
>> More recently, I have seen arguments that convinced me that a significant problems with Scala was that _the possibility of using it an an unduly complex manner restricted its corporate applicability_. I see this as a highly valid point. (Large) corporations need to expect that engineers are, to some degree, interchangeable. I think even Google must have that approach these days. It should be expected that an engineer can be dumped into another engineers' job and be able, int not too long, to pick up on what is going on.
>>
> This mindset has led to the greatest atrocities in engineering management. No two engineers are the same. We all have different capabilities, and the features we choose to work on in a code base reflects that. Making sure engineers can maintain code is *different* than treating them like replacable cogs.
>
>
>>
>> I don't believe this is possible with "full Scala". For example, look at Tony M. and (in addition) scalaz. How many engineers could pick up on code engineered in that universe? Very, very few.
>
>
> I completely disagree with the assumption that FP style code can't be engineered by many. Scalaz does itself a disservice in two ways:
>
> (1) Non-existent documentation. This is *just as bad* as that old legacy codebase we're all forced to maintain with 1k's of lines of non-documented junk code
I truly find it amazing that this is repeated so readily given the mountains of documentation out there. It is as if we could choke the world with even more documentation, just so we could increase the occurrence of repetition of this statement.
Aside from the fact that scalaz makes much more rigorous *machine checked* type system guarantees about the behaviour of functions than a typical scala program/library/user does, and aside from the mountain of papers out there documenting exactly the concepts implemented, and aside from the runnable usage examples, and aside from the articles written by people who switched off their blinkers, learned the material and wrote about it, and aside from machine run specifications, then what documentation are you alluding to?
There doesn't exist a scala library out there with more than 10% of the documentation of scalaz. This fact coupled with the ferver with which the contrary is so readily protested fascinates me endlessly.
It's simply cognitive blindness; I can think of no better hypothesis matching observations in reality. Where do you think the many people who overcome this illusory hurdle got their learning material from? That question is not rhetorical; think about the answer carefully.
> (2) Re-inventing itself every revision. Scalaz is tough because it changes so frequently. Luckily 6.x series has been maintained for a while and stable-ish.
It didn't reinvent itself. It is quite stable. You might be referring to the need to adjust to changes and contentions in scala itself. Scalaz aims to be *useful* so if scala decides to change its type inference slightly, or perhaps order of initialisation, scalaz must adjust accordingly in order to maintain the thesis. This is aside from all the contentions that arise from inherent properties of scala. Many have tried and fizzled attempting to resolve all of these contentions. Nobody likes shaving yaks, but it must be done as a consequence sometimes, *including the occasions when others choose to deny the existence of these consequences*.
>
> Assuming the Scalaz represents functional programming in scala on the whole is bogus.
It doesn't at all, but if you are functional programming, you're going to reinvent a good deal of scalaz. Denialists have very few tactics around this fact, which is itself refreshing.
Or it could be that the documentation is not in the correct place.
> Aside from the fact that scalaz makes much more rigorous *machine checked*
> type system guarantees about the behaviour of functions than a typical scala
> program/library/user does, and aside from the mountain of papers out there
Yeah, well, most people expect documentation to be in English, not in
type declarations. In fact, most people consider type declarations to
be "source code", and define "documentation" to be "not source code".
Disagree all you want with this perspective, but don't come
complaining when people act like people do.
> documenting exactly the concepts implemented, and aside from the runnable
> usage examples, and aside from the articles written by people who switched
Runnable examples? Mmmmm... that sounds suspiciously like "source
code". See above.
> off their blinkers, learned the material and wrote about it, and aside from
Blogs exist where documentation fails.
> machine run specifications, then what documentation are you alluding to?
Well, open http://scalaz.github.com/scalaz/scalaz-2.9.1-6.0.2/doc/index.html#scalaz.Identity.
See those 16 out of 69 methods that have some text in English below
it? Well, that's one thing people call "documentation", and I'll let
the numbers speak for themselves.
Another example of things people call "documentation" is this:
http://download.oracle.com/javase/6/docs/technotes/guides/collections/overview.html.
You might think this is bad for some reason, for all I know. If that's
the case, then just substitute "something bad" for "documentation"
whenever people talk about it. But _don't_ substitute "source code"
for "documentation".
Given what COBOL looks like, I'm pretty sure COBOL programmers thought
all other languages exceedingly weird and complex. I don't know how
many of them ever transitioned to another language, but that's
irrelevant -- new programmers learned new programming languages. Java
has quite a lot of COBOL potential: it might well outlive all of us,
their programmers might not ever switch to another language, but it
may well decrease in importance until few people would ever consider
learning it.
The complexity to *new programmers* is a much more important metric
than the complexity to existing programmers, and, yet, it is always
the latter that people talk about. Go figure.
and they use foldLeft just to calculate a max of an iterable
On Nov 20, 2011 7:49 AM, "Daniel Sobral" <dcso...@gmail.com> wrote:
>
> On Sat, Nov 19, 2011 at 18:42, Tony Morris <tmo...@tmorris.net> wrote:
> >
> >> (1) Non-existent documentation. This is *just as bad* as that old
> >> legacy codebase we're all forced to maintain with 1k's of lines of
> >> non-documented junk code
> >
> > I truly find it amazing that this is repeated so readily given the mountains
> > of documentation out there. It is as if we could choke the world with even
> > more documentation, just so we could increase the occurrence of repetition
> > of this statement.
>
> Or it could be that the documentation is not in the correct place.
It could be; but I think there are more likely explanations.
>
> > Aside from the fact that scalaz makes much more rigorous *machine checked*
> > type system guarantees about the behaviour of functions than a typical scala
> > program/library/user does, and aside from the mountain of papers out there
>
> Yeah, well, most people expect documentation to be in English, not in
> type declarations. In fact, most people consider type declarations to
> be "source code", and define "documentation" to be "not source code".
No they don't. You have a selection bias. So do I.
Watch this. Most people are absolutely adamant that your statement is wrong. That's because it is. I was able to make this claim by restricting myself to people who understand the subject matter at hand. Of course they see your statement is wrong. To most people, it is bleedingly obvious!
More seriously, I agree there are a lot of people who believe your statement. They are wrong. What now? Are you offering to join the agenda of educating those who seek it out? If not what? There are many people who have overcome this very exact hurdle; it's tough work though.
> Disagree all you want with this perspective, but don't come
> complaining when people act like people do.
I'm well aware of the popular views. There is an enormous amount of documentation in English out there; to be, clear, a FUCKING OVERWHELMING amount that I doubt I will get through in my life time.
>
> > documenting exactly the concepts implemented, and aside from the runnable
> > usage examples, and aside from the articles written by people who switched
>
> Runnable examples? Mmmmm... that sounds suspiciously like "source
> code". See above.
Yes, runnable examples are extremely useful by virtue of their being machine checked.
>
> > off their blinkers, learned the material and wrote about it, and aside from
>
> Blogs exist where documentation fails.
Bollocks. Blogs exist because people learn and want to share it with others who do more than pay lip service to the goal of learning.
>
> > machine run specifications, then what documentation are you alluding to?
>
> Well, open http://scalaz.github.com/scalaz/scalaz-2.9.1-6.0.2/doc/index.html#scalaz.Identity.
> See those 16 out of 69 methods that have some text in English below
> it? Well, that's one thing people call "documentation", and I'll let
> the numbers speak for themselves.
What if many people are wrong? It's not as if there aren't countless examples of many people sharing the same mistake, surviving only by patting each other on the back, confirming the perpetuity of the mistake.
>
> Another example of things people call "documentation" is this:
> http://download.oracle.com/javase/6/docs/technotes/guides/collections/overview.html.
> You might think this is bad for some reason, for all I know. If that's
> the case, then just substitute "something bad" for "documentation"
> whenever people talk about it. But _don't_ substitute "source code"
> for "documentation".
>
Its not bad; its just clumsy and error-prone, and subsequently inefficient. There are far better ways of achieving comprehension of a subject using oh I don't know, maybe COMMUNICATION SKILLS or something. Oh wait I forgot; I don't have those; I was born, with this understanding and everyone else is excluded from sharing it. Cheap stab sorry, but a bit of introspection is desperately in order around here.
I very much agree.
> What I am missing currently is a survey giving some basic facts about what
> furthers understanding and what blocks understanding if someone is about to
> read some given section of code. "Critical" would be anything that makes it
> unecessarily hard to understand or follow a concept / solution that is
> implemented (the "what it does").
>
> That's not to be mixed with the complexity of a concept/solution in itself. If
> I don't understand the concept/solution (the "why") in itself I won't be able
> to figure out the meaning of code implementing that concept/solution in any
> language.
>
> Existing code metrics (for different languages) try to capture aspects of this
> "what it does" problem (and there has been some related research) - but I am
> not aware of any good code metrics for scala.
>
> If we had such code metrics we could tag the code and try to align metric
> signatures with "comprehension effort". This would give enterprises a valuable
> instrument to spot source parts which should be reworked.
>
> I think that would help...
>
> Greetings
> Bernd
I believe you are very much thinking in the right direction Bernd.
For millennium (is that the right word?) there has been the notion of
artisan masters and apprentices. Apprentices would study, train and work
under the supervision of the masters until they themselves would become
masters. Often the apprentices would do much or most of the real work,
but the masters would routinely inspect their work and intervene or
guide when necessary.
It can be difficult doing serious code reviews all the time to make sure
that novices get the guidance they need or that mavericks are reined in
when necessary. If we had tools that could help automate this sort of
review we could make it easier for the masters to check on the work of
the apprentices.
Cheers, Eric
You are very correct Martin, there have been many threads that
negatively criticize the complexity of Scala - and many have been
unproductive or counterproductive.
Sorry to read you see this thread as negative - I do not see it that way
at all. You are absolutely correct that many people are happy with Scala
just the way it is. What I believe this thread is attempting to do is to
constructively satisfy those people who do find Scala too daunting and
offer solutions to make it less daunting for them.
I do not believe this thread is designed to negatively criticize Scala
for being too complicated, I think many of us accept Scala for what it
is. The problems of Scala complexity should not always be problems to be
disputed, they should also be problems to embrace as challenges to
solve. I believe this thread is designed to find ways to safely include
those people, especially the undecided or cautious, into the community
and culture of Scala.
>
> Second, I believe we have come to realize that complexity and power
> are pretty much aligned for Scala.
> Take away the complexity and you take away the power. Here I mean
> power in the sense: "Number of concepts that you can describe
> precisely using your language". I want to get away from the common
> argument that all Turing complete languages are equivalent in power
> anyway.
One good step I have seen lately is moving the narrative away from
trying to reduce or take away the complexity of Scala or any of its
power. I really like the idea of tools to help us best use the language.
In the early days of C the compiler diagnostics were a joke, but someone
wrote lint and it helped many people make better use of C. These days I
am truly amazed at the quality of the diagnostics of the Java compilers
and the IDEs - these automated tools make Java even more accessible to
more programmers, and even less complex. I am often amazed at how well
Eclipse can guide me into proper use of Java or best practices.
If there are people who are smart enough to invent languages like Scala
- there are surely people smart enough to imagine, design and build
tools to make Scala more accessible to more people. If you are smart
enough and experienced enough to ride Scala 'bareback' so to speak, more
power to you, but it does not mean we cannot also invest in better
saddles and stirrups for the rest of us.
>
> Third, if you look at other languages, there is always a set of "best
> practices" developed that come with it and that are used as
> guidelines. Certainly Java has a rich culture of best practices. And
> that cultures changes over time, as developments in Java EE show.
No disagreement here.
>
> Since Scala's enterprise use is still young, best practices have still
> a some way to develop and stabilize.
Truly. As we go along can we not find some way to automate the support
of those best practices?
> This is your chance! If you put out some clean libraries with easy-to
> use APIs you have a great chance to influence the future best
> practices for Scala, as long as they are still in flux. I wish we'd
> concentrate more on these aspects than wringing our hands.
Clean libraries and APIs alone do not solve the problems. Also, I do not
see us wringing our hands any longer, I think this thread is actually
going somewhere productive and should be encouraged.
>
> Can best practices be influenced by tools, such as style checkers?
> Maybe. It's certainly something we want to pursue. But it would be
> silly to delegate responsibility for forming best practices to these
> tools only.
I totally agree. Airbus tried to over automate the cockpit and the
result was the autopilot overrode the real pilots and landed the
aircraft in a forest at an airshow. These days there is a lot of effort
to further automate things in the cockpit, but it is generally accepted
now that the human pilot is ultimately in charge. Rather than making
human pilots redundant, we try to make them better pilots.
I am advocating giving the software developers, senior software
developers and software architects the tools to monitor and assess the
code quality and complexity, and maybe event suggest better ways to do
things. I have found findbugs exceedingly useful on many occasions and
have even used it to analyze our code base and report back to our team
my concerns about some of the dicey practices I found in the code base.
Let's not limit Scala's power and promise by restricting features -
rather let's build tools to help us make better use of Scala.
>
> [Cross-posted to scala-user, because it relates to quite a few threads
> there as well]
>
> -- Martin
>
Cheers, Eric
...and (some) strong programmers will always find a way to give other
programmers more rope and cause to hang themselves.
We don't need to 'nerf' Scala (or anyone), we just need to give people
less experienced with Scala more tools to better deal with the Scala we
have now.
Cheers, Eric
Recently while trying to answer this: https://twitter.com/#!/dibblego/status/130796120107532289
I looked at the code for MA(http://scalaz.github.com/scalaz/
scalaz-2.9.1-6.0.2/doc.sxr/scalaz/MA.scala.html)
I found that there is foldMap and foldMapDefault. The signature
difference between the two is the use of Foldable versus Traverse.
I've indirectly used Foldable and Traverse before, though I'm no
expert, I get a sense of what they can do.
Some testing showed that they yield the same result (on this
particular instance).
scala> val list = List(101,102, 103)
list: List[Int] = List(101, 102, 103)
scala> list.foldMap(i => (i, 1))
res1: (Int, Int) = (306,3)
scala> list.foldMapDefault(i => (i, 1))
res2: (Int, Int) = (306,3)
Why use one versus the other one? The signature doesn't tell me that
when it works on either case. It looks like most scala collections can
be lifted/converted into Traverse and Foldable. I then took a detour
and searched for foldMap and foldMapDefault and haskell. But that
didn't answer my question.
So I had to dig into what the code does, but for foldMapDefault it
wasn't obvious because I don't know what Const does (an enigmatic:
"Used to generate Phantom Applicative Functors and categories from a
Monoidal type A". Ok in restrospect it sorts of make sense but it's
hard, because Const is not informative as a name). Eventually I got a
sense of Const from http://scalaz.github.com/scalaz/scalaz-2.9.1-6.0.2/doc.sxr/scalaz/example/WordCount.scala.html#859028
But I feel that a bit of documentation like "use foldMap when abc over
foldMapDefault" or vice versa would be helpful and save time. Also
known issues like this could be documented as well:
scala> (1 to 10000).toList.foldMap(i => (i, 1))
res4: (Int, Int) = (50005000,10000)
scala> (1 to 10000).toList.foldMapDefault(i => (i, 1))
[stack overflows]
Overall, learning to use scalaz is challenging yet rewarding. But I
think that lack of documentation makes it harder than necessary.
--Jean-Laurent
On Nov 19, 6:02 pm, Tony Morris <tmor...@tmorris.net> wrote:
> On Nov 20, 2011 7:49 AM, "Daniel Sobral" <dcsob...@gmail.com> wrote:
>
[snip]
> > Yeah, well, most people expect documentation to be in English, not in
> > type declarations. In fact, most people consider type declarations to
> > be "source code", and define "documentation" to be "not source code".
>
> No they don't. You have a selection bias. So do I.
>
> Watch this. Most people are absolutely adamant that your statement is
> wrong. That's because it is. I was able to make this claim by restricting
> myself to people who understand the subject matter at hand. Of course they
> see your statement is wrong. To most people, it is bleedingly obvious!
>
> More seriously, I agree there are a lot of people who believe your
> statement. They are wrong. What now? Are you offering to join the agenda of
> educating those who seek it out? If not what? There are many people who
> have overcome this very exact hurdle; it's tough work though.
[snip]
Well, the rest of the world don't agree with you. C++ is still a
commonly used language, and yet to be replaced in many areas. And with
C++11 coming around it's still evolving and improving.
I'm not saying C++ is a nice language to work with, far from it, but it
gets the job done.
> My point is, I do not want to see Scala end up where C++ is now - the
> mess that it is. At the time C++ was the state of the art and I even
> passionately advocated it for a while. What I truly hope for is that the
> Scala community learn from the mistakes that were made with C++ and try
> to avoid them. Many of the mistakes that were made with C++ were not
> even the language design per se, rather they were the immaturity of the
> culture and community that was using and then abusing C++.
Hmmm, kinda hard to control the entire community. I mean you can decide
what to put in the stdlib (and now the Typesafe stack), but you can't
control what developers will use your language for (and that's a good
thing IMO). If there is a language feature to use/abuse, developers will
do so.
Btw, what mistakes in the C++ world are you referring to?
> I feel this argument is reasonable as there are people with valid
> concerns about the complexity of Scala and it serves us all well to
> address those concerns constructively rather than dismiss or invalidate
> them. It may seem to you that some of us are catastrophizing C++, but to
> me C++ is a very real catastrophe of what happens when we ignore these
> very issues.
I don't see C++ as a catastrophe, it's a result of compromises (for
example C compatibility), some good ideas (for example generic
programming) and some bad ones (for example templates). Scala is quite
similar, but with less bad ideas :)
/Jesper Nordenberg
On 2011-11-19 6:31 AM, martin odersky wrote:You are very correct Martin, there have been many threads that negatively criticize the complexity of Scala - and many have been unproductive or counterproductive.
Some remarks:
First, the thread and many like it seems too negatve to me. Scala is used in a large and growing number of enterprises, some of them with more than 100 devs working with it. These people just get on with the job (and love it for the most part); they don't find Scala's ``complexity'' too daunting.
Sorry to read you see this thread as negative - I do not see it that way at all. You are absolutely correct that many people are happy with Scala just the way it is. What I believe this thread is attempting to do is to constructively satisfy those people who do find Scala too daunting and offer solutions to make it less daunting for them.
One good step I have seen lately is moving the narrative away from trying to reduce or take away the complexity of Scala or any of its power. I really like the idea of tools to help us best use the language. In the early days of C the compiler diagnostics were a joke, but someone wrote lint and it helped many people make better use of C. These days I am truly amazed at the quality of the diagnostics of the Java compilers and the IDEs - these automated tools make Java even more accessible to more programmers, and even less complex. I am often amazed at how well Eclipse can guide me into proper use of Java or best practices.Second, I believe we have come to realize that complexity and power are pretty much aligned for Scala.
Take away the complexity and you take away the power. Here I mean power in the sense: "Number of concepts that you can describe precisely using your language". I want to get away from the common argument that all Turing complete languages are equivalent in power anyway.
If there are people who are smart enough to invent languages like Scala - there are surely people smart enough to imagine, design and build tools to make Scala more accessible to more people. If you are smart enough and experienced enough to ride Scala 'bareback' so to speak, more power to you, but it does not mean we cannot also invest in better saddles and stirrups for the rest of us.
No disagreement here.
Third, if you look at other languages, there is always a set of "best practices" developed that come with it and that are used as guidelines. Certainly Java has a rich culture of best practices. And that cultures changes over time, as developments in Java EE show.
Truly. As we go along can we not find some way to automate the support of those best practices?
Since Scala's enterprise use is still young, best practices have still a some way to develop and stabilize.
Clean libraries and APIs alone do not solve the problems.
This is your chance! If you put out some clean libraries with easy-to use APIs you have a great chance to influence the future best practices for Scala, as long as they are still in flux. I wish we'd concentrate more on these aspects than wringing our hands.
Also, I do not see us wringing our hands any longer, I think this thread is actually going somewhere productive and should be encouraged.
I totally agree. Airbus tried to over automate the cockpit and the result was the autopilot overrode the real pilots and landed the aircraft in a forest at an airshow. These days there is a lot of effort to further automate things in the cockpit, but it is generally accepted now that the human pilot is ultimately in charge. Rather than making human pilots redundant, we try to make them better pilots.
Can best practices be influenced by tools, such as style checkers? Maybe. It's certainly something we want to pursue. But it would be silly to delegate responsibility for forming best practices to these tools only.
I am advocating giving the software developers, senior software developers and software architects the tools to monitor and assess the code quality and complexity, and maybe event suggest better ways to do things. I have found findbugs exceedingly useful on many occasions and have even used it to analyze our code base and report back to our team my concerns about some of the dicey practices I found in the code base.
Let's not limit Scala's power and promise by restricting features - rather let's build tools to help us make better use of Scala.
On Nov 20, 2011 2:55 PM, "Kevin Wright" <kev.lee...@gmail.com> wrote:
>
> Oh absolutely! It also doesn't help that many of our current "high profile" libraries are particularly heavy on symbolic method names. Things are improving in at least two notable libraries though:
>
> - Akka has culled its symbols down to essentially just ! and ?
Worth noting there is that both those methods have a nominal alternative (tell/ask)
Exactly my point! Heck, I'm a guy who want applicative functor syntactic sugar like for-expressions provide for monadic thing. Scalaz is *useful*. It's unnecessarily hard to learn the simmer concepts. I think this can be improved, and I hate when people point to this as a form of complexity.
On Nov 20, 2011 9:01 AM, "Anwar Rizal" <anri...@gmail.com> wrote:My approach to scalaz is actually quite different. After using quite a while Scala, I realize that standard library has some limits.
For example, I need to have map that works with Either, or map that works correctly with java collection (scala.collection.JavaConversions sucks :-). I also discovered later applicative functor through
some(3) |@| some(5) {_ + _ }
then,
either Right(4) |@| Left("Error') {_ + _} === Left("Error")
without necessarily understanding the whole world of functor or applicative.
Those examples are useful and damn simple and I'm sure we can find a lot of pearl out there in scalaz without having to understanding each and every type class existing out there. Scalaz validation and monoid look to enter to this category too (easy to use).
I don't 100% agree even with "Scalaz uses very advanced topics". Scalaz is indeed implemented using this advanced topic, but one does not need to understand the implementation to work with Scalaz.
That said, the beginners like me don't find those things easily. Often, I had to go to the codes to discover those things.
Maybe a better documentation or more centralized tutorial will be helpful. It's always more comfortable to have the tutorial in one place than have to wander around a lot of blogs that you're not sure about their quality.
- AnwarOn Sun, Nov 20, 2011 at 1:44 PM, Josh Suereth <joshua....@gmail.com> wrote:Jason -
I think, as always, you're doing some amazing work on scalaz. The most recent scalaz7 refactorings will provide far more utility, but I think need more documentation for folks to understand.
I'm willing to help write scalaz Docs. Hell, I put a category theory chapter in my book.
My point, for others, is not to use scalaz as a bastion of how complex scala is. Scalaz uses very advanced topics with little internal documentation, so people assume that this means the library is as complex as the learning curve. I think sbt 0.10 is in a similar vein. Those of us who *have* learned the concepts see how simple things are to use, but learning is difficult.
Sbt just got a users guide. I'm more than willing to help scalaz with documentation. I've been watching scalaz7 with excitement and anticipation.
Maybe in Scalaz7 we can add a good amount of learning material.
On Nov 20, 2011 4:09 AM, "Jason Zaugg" <jza...@gmail.com> wrote:
On Sat, Nov 19, 2011 at 7:44 PM, Josh Suereth <joshua....@gmail.com> wrote:
I completely disagree with the assumption that FP style code can't be engineered by many. Scalaz does itself a disservice in two ways:(1) Non-existent documentation. This is *just as bad* as that old legacy codebase we're all forced to maintain with 1k's of lines of non-documented junk code.(2) Re-inventing itself every revision. Scalaz is tough because it changes so frequently. Luckily 6.x series has been maintained for a while and stable-ish.Assuming the Scalaz represents functional programming in scala on the whole is bogus.
Hi Josh,Let me address your second point.Scalaz 6.x was structurally the same as 5.x. The changes were additive [1] and were largely source compatible.The 5.x series differed from 4.x in order to take advantage of the type inference abilities in Scala 2.8.0.However, Scalaz 7 will mark a change in design. This is required to help us address three challenges:1) reconciling the tension between type classes and inheritance (both type classes inheriting from each other, and the classified types inheriting from each other),2) organizing implicits to avoid ambiguity using the relatively limited tools available to influence implicit search3) providing library functionality a-la-carte, without requiring (although still supporting) the "uber" import of Scalaz._ or even the use of implicit views, while still maximising type inference.We don't take this choice lightly. Firstly, it represents a huge job for us to move things around, but despite this we've tried out three distinct approaches to help us prove the new design. I'm really excited about the solution we've have adopted. I've started to document it's design principles [2] [3]. I plan to write this up more thoroughly, as I think it's of general interest to anyone designing and using type classes in Scala.I'm also mindful of ways to make the library more learnable: providing examples around 'standalone' usage of type classes to avoid the distraction of the 'syntactic' implicit conversions; consistently making methods like flatMap / ap / foldRight directly available on data structures, rather than only via the related type class instance, and modularizing the library to limit the size of scalaz-core.In recognition of the breaking nature of this release, we will maintain the 6.x series on a bug-fix basis well into 2012.Could we use a bit more ScalaDoc? Sure! Personally, I don't think that your proposed documentation for Functor (from another thread) added *that* much value, when weighed against the time needed to write it, and extra maintenance cost (automatic refactoring tools don't work yet for Scaladoc). But you might notice that documentation *is* being added in some places (e.g. DList [4]), and we will continue in this style.I'm cross-posting to [scalaz]; I suggest we move any followup questions over there. You can also email me directly with questions or suggestions.-jason
Am 20.11.2011 11:20, schrieb martin odersky:
> That's having put /: and :\ as aliases for foldLeft and foldRight into
> collections. I thought I had good reasons to do so. In particular
>
> (zero /: xs) (op)
>
> keeps the relative order of the zero and the list xs in the expansion of
> the fold:
>
> zero op x_1... op x_n
>
> Whereas
>
> xs.foldLeft(zero)(op)
>
> reverses them.
I'm not very polyglott in programming languages, and my first
impression, when I saw them was OMG! I'll never get that! But soon I saw
a presentation on video - not sure by whom, which explained the /: as
part from a tree:
op /
/ \ /
op xs(1) / (xs (1))
/ \ /
init xs(0) / (xs (0))
(I hope it will survive the email layout.)
It was a very easy to remember and beautiful explanation, and to my own
surprise, I understood it very fast and learned it in half an hour.
In contrast, I don't know till today which of /: and :\ is 'foldLeft'
and which is foldRight. Okay - the shown example is 'foldLeft', which I
only remember indirectly, because in most cases I use /: and I realized,
most of the times, people use foldLeft, and that both match.
Another observation:
When I first stumbled over the arrow in C++: foo-> bar (x); it raised my
curiosity. After knowing, what it meant, and how to use it, it was a
pleasure to use something new which looked a little mysterious to
newbies. Yes - this is a little bit childish, but aren't we all a little
bit childish?
I guess it is a double-edged sword: it can attract curiosity, if you use
exotic constructs or method names, and it can distract the fearfull
(which shouldn't mean that I am courageous) or more conservative people.
- --
Tsch���--->...Stefan
- ---------------------------
Don't visit my homepage at:
http://home.arcor-online.net/hirnstrom
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk7JHSYACgkQQeATqGpDnRoxegCZAWAwslNS7uPrMgW+HAiXsI4s
NycAoJoNRwfXJMXe+zFPOlg1eRiQ0yVY
=KOTI
-----END PGP SIGNATURE-----
On Nov 20, 3:25 pm, Kevin Wright <kev.lee.wri...@gmail.com> wrote:
> So, in order to avoid disagreement over the semantic meaning of
> "documentation". Would it it fair to say everyone agrees that scalaz lacks
> a centralised set of *tutorials* explaining the concepts involved and some
> real-world use cases?
>
> That's certainly a cause I could get behind...
>
> On 20 November 2011 14:16, Josh Suereth <joshua.suer...A@gmail.com> wrote:
>
>
>
>
>
>
>
> > Exactly my point! Heck, I'm a guy who want applicative functor syntactic
> > sugar like for-expressions provide for monadic thing. Scalaz is
> > *useful*. It's unnecessarily hard to learn the simmer concepts. I think
> > this can be improved, and I hate when people point to this as a form of
> > complexity.
> >> On Sun, Nov 20, 2011 at 1:44 PM, Josh Suereth <joshua.suer...@gmail.com>wrote:
>
> >>> Jason -
>
> >>> I think, as always, you're doing some amazing work on scalaz. The most
> >>> recent scalaz7 refactorings will provide far more utility, but I think need
> >>> more documentation for folks to understand.
>
> >>> I'm willing to help write scalaz Docs. Hell, I put a category theory
> >>> chapter in my book.
>
> >>> My point, for others, is not to use scalaz as a bastion of how complex
> >>> scala is. Scalaz uses very advanced topics with little internal
> >>> documentation, so people assume that this means the library is as complex
> >>> as the learning curve. I think sbt 0.10 is in a similar vein. Those of
> >>> us who *have* learned the concepts see how simple things are to use, but
> >>> learning is difficult.
>
> >>> Sbt just got a users guide. I'm more than willing to help scalaz with
> >>> documentation. I've been watching scalaz7 with excitement and
> >>> anticipation.
>
> >>> Maybe in Scalaz7 we can add a good amount of learning material.
> >>> On Nov 20, 2011 4:09 AM, "Jason Zaugg" <jza...@gmail.com> wrote:
>
> >>>> On Sat, Nov 19, 2011 at 7:44 PM, Josh Suereth <joshua.suer...@gmail.com
> >>>>https://github.com/scalaz/scalaz/commit/7b8ba0fe3cf51ad721e7fd8502167...
No, but they are important. Not so much from the user perspective butto set an example to others how code and APIs should be structured and formulated. For instance, you know one thing I have come to regret?That's having put /: and :\ as aliases for foldLeft and foldRight into collections. I thought I had good reasons to do so. In particular(zero /: xs) (op)keeps the relative order of the zero and the list xs in the expansion of the fold:zero op x_1... op x_nWhereasxs.foldLeft(zero)(op)reverses them. And, if we talk just about this library, I still think these reasons are valid. But as an example for other libraries it was the wrong choice. It gave an implicit encouragement to use symbolic operators in libraries where they are not necessary. And, I think the current set of core libraries uses too many symbolic operators (and, to be clear, I am not talking about scalaz here, that's on a different planet altogether).
>
> - Akka has culled its symbols down to essentially just ! and ?Worth noting there is that both those methods have a nominal alternative (tell/ask)
Some remarks:
First, the thread and many like it seems too negatve to me. Scala is used in a large and growing number of enterprises, some of them with more than 100 devs working with it. These people just get on with the job (and love it for the most part); they don't find Scala's ``complexity'' too daunting.
Second, I believe we have come to realize that complexity and power are pretty much aligned for Scala.
Take away the complexity and you take away the power. Here I mean power in the sense: "Number of concepts that you can describe precisely using your language". I want to get away from the common argument that all Turing complete languages are equivalent in power anyway.
Third, if you look at other languages, there is always a set of "best practices" developed that come with it and that are used as guidelines. Certainly Java has a rich culture of best practices. And that cultures changes over time, as developments in Java EE show.
Since Scala's enterprise use is still young, best practices have still a some way to develop and stabilize.
This is your chance! If you put out some clean libraries with easy-to use APIs you have a great chance to influence the future best practices for Scala, as long as they are still in flux. I wish we'd concentrate more on these aspects than wringing our hands.
>
> - Akka has culled its symbols down to essentially just ! and ?Worth noting there is that both those methods have a nominal alternative (tell/ask)
As someone who has used (and written :-( ) APIs that attempt to satisfy everyone by providing multiple symbols for the same operation, I disagree rather strongly with this approach. IMHO, Akka should have gone with "tell" and "show".
Alternatively, if they had gone with "!" and "?", I would have been less happy, but would not consider it a significant problem since these symbols will be omnipresent in Akka-based code, so the the cost of learning them is very low compared to their utility.
But providing both unnecessarily increases the amount of info a programmer must know, and more importantly, raises that prospect that a programmer who has used Akka in a "consistent" environment might encounter the other usage occasionally and be temporarily stymied by it.
It's not a huge issue, this is not horrible design. But I thing in the net, it's negative.
Ken
Eric Kolotyluk skrev 2011-11-20 04:16:
However, as far as I am concerned C++ is now a dead language.
Well, the rest of the world don't agree with you.
C++ is still a commonly used language, and yet to be replaced in many areas. And with C++11 coming around it's still evolving and improving.
I'm not saying C++ is a nice language to work with, far from it, but it gets the job done.
My point is, I do not want to see Scala end up where C++ is now - the
mess that it is. At the time C++ was the state of the art and I even
passionately advocated it for a while. What I truly hope for is that the
Scala community learn from the mistakes that were made with C++ and try
to avoid them. Many of the mistakes that were made with C++ were not
even the language design per se, rather they were the immaturity of the
culture and community that was using and then abusing C++.
Hmmm, kinda hard to control the entire community.
I mean you can decide what to put in the stdlib (and now the Typesafe stack), but you can't control what developers will use your language for (and that's a good thing IMO).
If there is a language feature to use/abuse, developers will do so.
Btw, what mistakes in the C++ world are you referring to?
I feel this argument is reasonable as there are people with valid
concerns about the complexity of Scala and it serves us all well to
address those concerns constructively rather than dismiss or invalidate
them. It may seem to you that some of us are catastrophizing C++, but to
me C++ is a very real catastrophe of what happens when we ignore these
very issues.
I don't see C++ as a catastrophe, it's a result of compromises (for example C compatibility), some good ideas (for example generic programming) and some bad ones (for example templates).
Scala is quite similar, but with less bad ideas :)
/Jesper Nordenberg
brilliant points..... "We want people using Java, Groovy, Clojure, JRuby et al, to be able to use Akka..." and this precisely should be the attitude for Scala, i.e "We want people using Java, Groovy, Clojure, JRuby et al, to be able to use Scala" ...
cheers
On Sun, Nov 20, 2011 at 5:55 AM, Eric Kolotyluk <eric.ko...@gmail.com> wrote:
On 2011-11-19 6:31 AM, martin odersky wrote:You are very correct Martin, there have been many threads that negatively criticize the complexity of Scala - and many have been unproductive or counterproductive.
Some remarks:
First, the thread and many like it seems too negatve to me. Scala is used in a large and growing number of enterprises, some of them with more than 100 devs working with it. These people just get on with the job (and love it for the most part); they don't find Scala's ``complexity'' too daunting.
Sorry to read you see this thread as negative - I do not see it that way at all. You are absolutely correct that many people are happy with Scala just the way it is. What I believe this thread is attempting to do is to constructively satisfy those people who do find Scala too daunting and offer solutions to make it less daunting for them.
I am absolutely for discussing the best way to make Scala more approachable for newcomers and large teams. I just wanted to stress that we should not start with a too negative point of view of the status quo. Scala is already making nice progress among these groups. But of course we should search ways to help it further along.
One good step I have seen lately is moving the narrative away from trying to reduce or take away the complexity of Scala or any of its power. I really like the idea of tools to help us best use the language. In the early days of C the compiler diagnostics were a joke, but someone wrote lint and it helped many people make better use of C. These days I am truly amazed at the quality of the diagnostics of the Java compilers and the IDEs - these automated tools make Java even more accessible to more programmers, and even less complex. I am often amazed at how well Eclipse can guide me into proper use of Java or best practices.Second, I believe we have come to realize that complexity and power are pretty much aligned for Scala.
Take away the complexity and you take away the power. Here I mean power in the sense: "Number of concepts that you can describe precisely using your language". I want to get away from the common argument that all Turing complete languages are equivalent in power anyway.
If there are people who are smart enough to invent languages like Scala - there are surely people smart enough to imagine, design and build tools to make Scala more accessible to more people. If you are smart enough and experienced enough to ride Scala 'bareback' so to speak, more power to you, but it does not mean we cannot also invest in better saddles and stirrups for the rest of us.
On the small scale, I believe you are right. I am pretty confident these tools will come for Scala. But on the large scale, I do not see tools criticizing systems architecture just yet.
No disagreement here.
Third, if you look at other languages, there is always a set of "best practices" developed that come with it and that are used as guidelines. Certainly Java has a rich culture of best practices. And that cultures changes over time, as developments in Java EE show.
Truly. As we go along can we not find some way to automate the support of those best practices?
Since Scala's enterprise use is still young, best practices have still a some way to develop and stabilize.
Yes, but that's a second step. We need to develop them first!
Clean libraries and APIs alone do not solve the problems.
This is your chance! If you put out some clean libraries with easy-to use APIs you have a great chance to influence the future best practices for Scala, as long as they are still in flux. I wish we'd concentrate more on these aspects than wringing our hands.
No, but they are important. Not so much from the user perspective butto set an example to others how code and APIs should be structured and formulated. For instance, you know one thing I have come to regret?That's having put /: and :\ as aliases for foldLeft and foldRight into collections. I thought I had good reasons to do so. In particular
(zero /: xs) (op)
keeps the relative order of the zero and the list xs in the expansion of the fold:
zero op x_1... op x_n
Whereas
xs.foldLeft(zero)(op)
reverses them. And, if we talk just about this library, I still think these reasons are valid. But as an example for other libraries it was the wrong choice. It gave an implicit encouragement to use symbolic operators in libraries where they are not necessary. And, I think the current set of core libraries uses too many symbolic operators (and, to be clear, I am not talking about scalaz here, that's on a different planet altogether).
Also, I do not see us wringing our hands any longer, I think this thread is actually going somewhere productive and should be encouraged.
That's great!
I totally agree. Airbus tried to over automate the cockpit and the result was the autopilot overrode the real pilots and landed the aircraft in a forest at an airshow. These days there is a lot of effort to further automate things in the cockpit, but it is generally accepted now that the human pilot is ultimately in charge. Rather than making human pilots redundant, we try to make them better pilots.
Can best practices be influenced by tools, such as style checkers? Maybe. It's certainly something we want to pursue. But it would be silly to delegate responsibility for forming best practices to these tools only.
I am advocating giving the software developers, senior software developers and software architects the tools to monitor and assess the code quality and complexity, and maybe event suggest better ways to do things. I have found findbugs exceedingly useful on many occasions and have even used it to analyze our code base and report back to our team my concerns about some of the dicey practices I found in the code base.
Let's not limit Scala's power and promise by restricting features - rather let's build tools to help us make better use of Scala.
+100 here.
Cheers
-- Martin
I think we just need better ideas on how to make Scala more attractive
and accessible to people who fear the complexity of Scala, rather than
persuade them Scala is not too complex or otherwise invalidate their fears.
I just noticed this thread.
On Nov 20, 5:55 am, Kevin Wright <kev.lee.wri...@gmail.com> wrote:
> Oh absolutely! It also doesn't help that many of our current "high
> profile" libraries are particularly heavy on symbolic method names. Things
> are improving in at least two notable libraries though:
>
> - Akka has culled its symbols down to essentially just ! and ?
> - scalaz is near the end of a rewrite that will make it far easier to use
> non-symbolic equivalents for many concepts
>
Hey, you forgot ScalaTest, which I would point out was designed with
the philosophy of: *avoid any operators that users are not already
experts in.* So ScalaTest originally just had ===, which means
equality, < for less than, > for greater than, <= for less than or
equals, and >= for greater than or equals. At some point I did add one
non-obvious operator, the - in FreeSpec, because after much searching
for a way to avoid it, it was the best solution. In 2.1, which is
coming next year, I'll be adding a few more operators. One is +/- as
an alias for plusOrMinus. This one has been requested by multiple
users over time, and I've done several informal surveys asking people
if they can guess its meaning, and people always have. So I decided to
add that one. I'm also adding &&, ||, and ! for boolean-like
expressions with the triple equals operator, and they will mean
exactly what they look like they mean. Lots of new features are
coming, but no other operators than these few that pass the "obvious
test."
I have been bummed that so many non-obvious operators have been
showing up in Scala APIs. The trouble is that although obvious
operators makes code easier to read (a + b is nicer than a add b), non-
obvious operators makes code harder for casual users to understand,
because they have to look up what all the meanings are. And a lot of
users are casual, in that they don't use your library often enough to
remember the symbols for whatever reason. And to Java programmers, at
least, it makes Scala code look cryptic.
Moreover I would claim that 99% of the time, if you work at it, you
can find a better alternative to a non-obvious operator. Compare Specs
tables to ScalaTest tables. Specs uses lots of operators:
http://etorreborre.github.com/specs2/guide/org.specs2.guide.Matchers.html#DataTables
ScalaTest doesn't use any:
http://www.scalatest.org/user_guide/table_driven_property_checks#testingMutables
You just don't need non-obvious operators most of the time. There's a
better way if you look for it.
The feeling I get sometimes from all this is that we're a bunch of
really smart, passionate Scala experts designing libraries for each
other, rather than for the mainstream. Because we can certainly learn
and understand each others cryptic operators, and maybe even enjoy the
conciseness of the expressions we can make out of them. But the
mainstream developer is not that interested. It's not to say they
aren't smart. They may be super-smart, but just passionate about
designing airplane wings or getting their startup off the ground or
something else rather than becoming experts in our libraries. They're
just not that into us.
Bill
Hi Kevin,
I just noticed this thread.
> The problem isn't the terms themselves � the problem is whether or not they
> encode something that will be useful later on. Are the abstractions useful
> � if they are then they will get used and the names will get association
> due to reuse.
>
> I think Perl has a lot to answer for here though. Everyone automatically
> says "Perl has symbols, Perl was bad, confusing, unmaintainable � Ergo:
> Symbols are bad".
>
> I am not saying I agree with indiscriminate usage of symbols, only that
> usage of symbols is not in itself a bad thing, certainly not "The price
> paid in terms of readability is extremely high and indefensible.".
>
> On 21 November 2011 15:49, Dinesh V <vis...@gmail.com> wrote:
>
>> Usage of non-obvious symbols and operators is one of the most significant
>> factors leading to all this perception of complexity. Perhaps the single
>> most important aspect holding back Scala adoption in my mind.
>>
>> Cryptic operators may save key strokes. The price paid in terms of
>> readability is extremely high and indefensible.
>>
>>
--
Tony Morris
http://tmorris.net/
This is such an overstated straw-man argument.
The answer of course is, it depends. Some things have clear and well understood alphanumeric names. Sometimes they do not at all, and make more sense as "operators". Consider:(3.some |@| 4.some |@| 5.some) { _ + _ + _ }what would you call |@| ? it combines arguments into an ApplicativeBuilder, so maybe applicativeBuild?(3.some applicativeBuild 4.some applicativeBuild 5.some) { _ + _ + _ }This is clearly longer, and I'd argue less readable (some will say less correct as well).
On Nov 20, 11:55 pm, Jed Wesley-Smith <jed.wesleysm...@gmail.com>
wrote:
I'd agree applicativeBuild is less readable than the |@| form once you
learn the meaning, and Haskell programmers probably already recognize
it. But to non-Haskell programmers it will look cryptic until they
learn it (if they learn it). But I'd point out the difference between |
@| and the operators in specs or sbt is that the latter operators are
just made up, invented out of thin air. The operators in specs have no
meaning in the testing domain, just as the operators in sbt have no
meaning in the build domain. To learn those operators, you have to
become an expert in that library and that knowledge does not benefit
you outside the library. To learn |@| by contrast, you need to become
an expert in something larger than scalaz, and you gain more than just
learning how to use scalaz. That's the difference.
Another example is domain-specific libraries for scientists. Say I'm
asked to write a library for a bunch of bio-informaticians who are
working in a certain area. Those folks may have some symbols they use
to communicate precisely. All are familiar with them. They use them in
the papers they write for each other, etc. I think it is likely that
using similar symbols in the API you make for them could make the code
easier for everyone to write and read. Sure, when a new programmer
gets hired who hasn't worked in bio-informatics before, that person
will have some symbols to learn, which at first will look cryptic to
them. But in learning about those, the new programmer isn't just
learning about the API, they are learning about the corner of bio-
informatics they are now working in. They are learning about the
domain, and that's going to help them be productive in that
environment.
The key is, do those symbols already have meaning in some domain and
are your users (or at least most of them) already experts in those
symbols. If not, then try and find a different way to say it that
doesn't use operators. For example, in specs you compose matchers with
^^, which is a made up operator. In ScalaTest you say "compose", which
is longer to type. But that's not actually a ScalaTest method.
ScalaTest matchers are Function1s, and so that's the compose method on
Function1. So if you don't know about compose, you'll need learn that,
but in the process you'll learn about function composition in Scala
(and in functional programming) in general, whereas with ^^ you just
learn something that has meaning only in specs.
Bill
" But there's a huge difference between a library and a language."
may be I am not as smart as you are...but I do understand that there is a huge difference between a library and a language .... there is certain level of arrogance in Scala community, that is beyond me.... my comments were strictly with respect to the kind of attitude that we should have ...and they were not against Scala. I totally agree with @Russ
cheersArif
cheers
Arif
On Nov 20, 12:04 am, Adam Jorgensen <adam.jorgensen...@gmail.com>
wrote:
> Ugh, all this debate about complexity.
>
> Weaker programmers will always find a way to hang themselves.
>
> Nerfing Scala is not going to fix that.
>
> Better to nerf the weak programmers than the language.
This is such an overstated straw-man argument.
Any symbol's utility is entirely derived from from its cognitive associativity. If you see a symbol and relate it to a useful concept it is a useful symbol to you. + is useful because almost everyone learns addition at an early age, and the association is obvious. Reusing it for string concatenation complicates matters though as this is not a commutative operation. Note that overloading is the problem here and not the use of a non-ascii symbol.
If I use a different symbol, say "zusatz", have I improved anything? Clearly I have not, unless I am German. For all non-German speakers this has no associativity and is less useful from a universal understandability point of view.
However, there are many things in computing and math that are not so universally understood. Some things that are very specific to certain domains, or even sub-domains. Is it necessary that symbols within these domains adopt a strict latin alpha-numeric symbolic structure?
The answer of course is, it depends. Some things have clear and well understood alphanumeric names. Sometimes they do not at all, and make more sense as "operators". Consider:
(3.some |@| 4.some |@| 5.some) { _ + _ + _ }
what would you call |@| ? it combines arguments into an ApplicativeBuilder, so maybe applicativeBuild?
(3.some applicativeBuild 4.some�applicativeBuild�5.some) { _ + _ + _ }
This is clearly longer, and I'd argue less readable (some will say less correct as well).
Now, the first time a non-haskell background Scala newbie comes across this they are going to be confused. They are going to wonder what the hell is this |@| thing and they'll spend some time researching it. Later on though they might wonder how they are going to do combine three Option[A]s and an (A, A, A) => B to get an Option[B] and remember that surprisingly useful |@| thing.
The problem isn't the terms themselves � the problem is whether or not they encode something that will be useful later on. Are the abstractions useful � if they are then they will get used and the names will get association due to reuse.
I think Perl has a lot to answer for here though. Everyone automatically says "Perl has symbols, Perl was bad, confusing, unmaintainable � Ergo: Symbols are bad".
I am not saying I agree with indiscriminate usage of symbols, only that usage of symbols is not in itself a bad thing, certainly not "The price paid in terms of readability is extremely high and indefensible.".
On 21 November 2011 15:49, Dinesh V <vis...@gmail.com> wrote:
Usage of non-obvious symbols and operators is one of the most significant factors leading to all this perception of complexity.� Perhaps the single most important aspect holding back Scala adoption in my mind.�Cryptic operators may save key strokes.� The price paid in terms of readability is extremely high and indefensible.
As someone who has dived in to some Scala code before and thought 'WTF?' I know how unsettling cryptic symbolics can be. I often ran into this same problem with C++ because of operator overloading.
Is it possible to build tools into the IDE to help the reader understand what they are seeing?
I mainly use Eclipse and I really like how when I hover the mouse over various parts of the code I get javadoc tool-tips. One thing I really like about these tool-tips is you can click to keep them open, re-size them, click links and go back and forth. The main thing is by integrating the javadoc with the tool-tips it can really lessen the WTF factor considerably.
Can something similar be done in Scala IDEs? Maybe even going beyond the scaladoc to some mechanism the helps people unfamiliar with the domain, or particular library/API style to better fathom what things mean?
Cheers, Eric
On 2011-11-20 11:55 PM, Jed Wesley-Smith wrote:
This is such an overstated straw-man argument.
Any symbol's utility is entirely derived from from its cognitive associativity. If you see a symbol and relate it to a useful concept it is a useful symbol to you. + is useful because almost everyone learns addition at an early age, and the association is obvious. Reusing it for string concatenation complicates matters though as this is not a commutative operation. Note that overloading is the problem here and not the use of a non-ascii symbol.
If I use a different symbol, say "zusatz", have I improved anything? Clearly I have not, unless I am German. For all non-German speakers this has no associativity and is less useful from a universal understandability point of view.
However, there are many things in computing and math that are not so universally understood. Some things that are very specific to certain domains, or even sub-domains. Is it necessary that symbols within these domains adopt a strict latin alpha-numeric symbolic structure?
The answer of course is, it depends. Some things have clear and well understood alphanumeric names. Sometimes they do not at all, and make more sense as "operators". Consider:
(3.some |@| 4.some |@| 5.some) { _ + _ + _ }
what would you call |@| ? it combines arguments into an ApplicativeBuilder, so maybe applicativeBuild?
(3.some applicativeBuild 4.some applicativeBuild 5.some) { _ + _ + _ }
This is clearly longer, and I'd argue less readable (some will say less correct as well).
Now, the first time a non-haskell background Scala newbie comes across this they are going to be confused. They are going to wonder what the hell is this |@| thing and they'll spend some time researching it. Later on though they might wonder how they are going to do combine three Option[A]s and an (A, A, A) => B to get an Option[B] and remember that surprisingly useful |@| thing.
The problem isn't the terms themselves – the problem is whether or not they encode something that will be useful later on. Are the abstractions useful – if they are then they will get used and the names will get association due to reuse.
I think Perl has a lot to answer for here though. Everyone automatically says "Perl has symbols, Perl was bad, confusing, unmaintainable – Ergo: Symbols are bad".
I am not saying I agree with indiscriminate usage of symbols, only that usage of symbols is not in itself a bad thing, certainly not "The price paid in terms of readability is extremely high and indefensible.".
On 21 November 2011 15:49, Dinesh V <vis...@gmail.com> wrote:
Usage of non-obvious symbols and operators is one of the most significant factors leading to all this perception of complexity. Perhaps the single most important aspect holding back Scala adoption in my mind.Cryptic operators may save key strokes. The price paid in terms of readability is extremely high and indefensible.
On Mon, Nov 21, 2011 at 8:42 AM, Eric Kolotyluk <eric.ko...@gmail.com> wrote:
As someone who has dived in to some Scala code before and thought 'WTF?' I know how unsettling cryptic symbolics can be. I often ran into this same problem with C++ because of operator overloading.
Note that Scala doesn't have operator overloading, the problem we're discussing is that methods can be named using arbitrary UTF characters.
Is it possible to build tools into the IDE to help the reader understand what they are seeing?
IDE's already support this, it's called Javadoc :-)
I'm sorry, but how is this a problem? Even remotely? Symbols are used
everywhere, in all kinds of fields. Where does it say that symbols
cannot be used in a coherent manner that will aid the description of
logic or a solution?
This is purely a familiarity problem.
> IDE's already support this, it's called Javadoc :-)
> But the problem with symbols is much deeper than this. For example, suppose
> you come across |@| in Scalaz code and you have never studied Haskell, much
> less applicatives. Your first reflex will be to google it. Searching for |@|
> alone yields no result (unsurprisingly) but "scalaz |@|" shows more promise.
> However, the first two links point to Scalaz' main project page, and none of
> them even contain "|@|" anywhere.
>
> This result should be obvious if you know how Google search works (it
> probably simply discarded |@| form the query) but it's one of the main
> problems with symbols: they're hard to look up.
> Google code search is of no help either, by the way.
> You're pretty much dead in the water at this point.
> So your only hope is to dive into the Scalaz code. Finding the definition of
> |@| is pretty easy, but take a look at the source... Yup, not a single
> comment, and a strong warning at the top telling you not to use it directly.
> To make things worse, "applicative builders" don't exist in Haskell, so even
> if you decided to bite the bullet and educate yourself in Haskell (never a
> bad idea, but not everybody has the time), walking the ladder from functors
> to applicatives and finally to monads will be a long and arduous climb.
It might be a climb, but the end results speak for themselves.
Additionally, there is a "culture" with programming these days that
the IDE seems to do more than the developer. This is truly concerning.
An IDE is nothing more than a tool (as is everything else) you need to
use it correctly. you can be just as productive with Scaladoc and a
text editor. Perhaps the issues are with how things are defined,
rather with how people expect that new and important concepts that
need to be understood require a little effort?
But the problem with symbols is much deeper than this. For example, suppose you come across |@| in Scalaz code and you have never studied Haskell, much less applicatives. Your first reflex will be to google it. Searching for |@| alone yields no result (unsurprisingly) but "scalaz |@|" shows more promise. However, the first two links point to Scalaz' main project page, and none of them even contain "|@|" anywhere.
and thusa |@| b
would be less ideal than one of
ApF(a,b)
(a,b) toApF
in most cases.
On Nov 22, 2011 3:54 AM, "Rex Kerr" <ich...@gmail.com> wrote:
>
> 2011/11/21 Cédric Beust ♔ <ced...@beust.com>
>>
>>
>> But the problem with symbols is much deeper than this. For example, suppose you come across |@| in Scalaz code and you have never studied Haskell, much less applicatives. Your first reflex will be to google it. Searching for |@| alone yields no result (unsurprisingly) but "scalaz |@|" shows more promise. However, the first two links point to Scalaz' main project page, and none of them even contain "|@|" anywhere.
>
>
> This is not an argument against well-chosen symbolic methods (i.e. operators), but it is an argument in favor of fanatic efforts at documentation given that some of the usual approaches do not work. Cryptic anything is bad, not just symbolic method names. (Thus, poorly-chosen method names are bad, whether it is << instead of print, or dividedBy instead of /.)
>
> |@| is a bit of a questionable case.
It also has nothing to do with haskell. It doesn't appear in haskell anywhere and exists solely for purposes that are specific to scala.
That's twice now that someone has decided to appeal to haskell unwarranted on the nature of |@|, which is an obvious indication that there has been no effort whatsoever to gather any understanding of the subject matter. In my mind that is an immediate forfeiture and it is no wonder this discussion goes in circles.
It also has nothing to do with haskell. It doesn't appear in haskell anywhere and exists solely for purposes that are specific to scala.
That's twice now that someone has decided to appeal to haskell unwarranted on the nature of |@|, which is an obvious indication that there has been no effort whatsoever to gather any understanding of the subject matter.
To make things worse, "applicative builders" don't exist in Haskell, so even if you decided to bite the bullet and educate yourself in Haskell (never a bad idea, but not everybody has the time), walking the ladder from functors to applicatives and finally to monads will be a long and arduous climb.
Oh look it's mister honest.
On Nov 21, 2:36 pm, etorreborre <etorrebo...@gmail.com> wrote:
> Hi Bill,
>
> I just want to point out that "^^" in specs2 was not completely taken out
> of thin air. I actually took the inspiration from the Parsers combinators
> library<http://www.scala-lang.org/api/current/scala/util/parsing/combinator/P...>,
> because this is how you transform a Parser[T] into a Parser[U]. The analogy
> here is similar, where you transform a Matcher[T] into a Matcher[U]. So if
> you learn the specs2 way, you will also get the idea for Parsers. "compose"
> would not fit here because specs matchers are not Function1s.
>
That's interesting. I hadn't made that connection with the parser
combinators "eyebrows" operator, but I see it now that you mention
it.
> About DataTables, I do agree that they're heavy on syntax but they fit *my*purpose which is to get a visually tabular notation for my examples. Which
> drives me to say that I wrote specs/specs2 first and foremost to learn
> Scala, and to give myself a library which I enjoy using. As it so happens,
> other people enjoy using it too, and I'm very dedicated to supporting them
> with whatever silly bug / sick feature I impose upon them. But then I think
> we should accept that one size does not fit all. There is room for
> different tastes in the ecosystem and we should just be grateful that
> Scala, as a language, gives us several ways of expressing ourselves. I love
> specs/specs2 because that's how I like it and I also love ScalaTest because
> that gives other users the possibility to use something different without
> having to push me in a direction where I might not want to go. I also love
> it because you gave me ideas for specs2 and because you're pushing hard to
> make sure that ScalaTest is not breaking client code and this is a lesson
> for every library maintainer.
>
> It would be a sad world if we only had ScalaTest and not specs2, Scalatra
> and not Unfiltered, and so on just because we have to please the majority.
>
I wholeheartedly agree. Sorry, I'm not trying to say we should all do
things the same way, just that a Scala API designer can, if they try,
usually find a way to avoid a made-up, non-obvious operator. I
highlighted specs because I also wrote a test framework and could
point out specific examples of alternatives to using non-obvious
operators. If I'd written a build system I might have picked on sbt. I
think those <+= etc symbols in sbt will make it harder for casual
users to figure out sbt build files, and I expect a great many users
of build tools are casual, i.e., non-expert-in-the-build-tool ones.
But it is hard for me to say for sure what might be an alternative way
to avoid those operators given I haven't ever tried to create a build
tool. I haven't walked in those shoes. Maybe there aren't any
satisfying alternatives to those operators. I myself did put one non-
obvious operator in ScalaTest, the dash in FreeSpec:
http://www.scalatest.org/scaladoc/1.6.1/#org.scalatest.FreeSpec
I thought long and hard to find a way around that, but there just
wasn't one that was as satisfying, so in it went even though it was
non-obvious.
Anyway, the I think the Scala community has definitely benefited from
having multiple test frameworks, web frameworks, etc., with different
design goals and designer tastes and visions to chose from. We inspire
each other and serve different niches. But still I think the community
of Scala library designers in general has been way too loose on the
operator front lately, and I think that lessens the approachability of
Scala to the masses to some extent. They aren't the end of the world,
but I think Scala would be more approachable if we used fewer
operators.
Bill
As Eric said - one size doesn't fit all. I suppose heavy operator use will be
here to stay. Scala gives this option - people will use it.
What I can imagine to ease this situation for those who prefer descriptive
names (the "casual user"): if we had a possibility to add a descriptive alias
(e.g. as annotation) which could be used by IDEs to support hoover help for
those operators. This could help even in the absence of documentation.
Maybe it would be enough just to add the "long name" as the first word in the
docs to enable a quick "ah yes - that was it".
At least everybody delivering public libraries should be aware that operators
while intended to speed up or tidy up source code can achieve the opposite.
Just my 5cents
Greetings
Bernd
Ka-Ching :-)
Applause!
There are many situations where I find myself the 'casual user' - such
as, when I have to find and fix a defect that has been assigned to me in
someone else's code, or when I am trying to learn about some new library
or other technology and I am looking at the code for the first time
trying to reverse engineer it in my mind.
While a tight team of expert Scala programmers is the ideal situation,
in a large enterprise legacy code is the norm - at least in my
experience. Legacy code usually results in people other than the authors
working on the code, in many cases the authors have long since moved on
to somewhere else and are usually inaccessible.
I really don't want to put a crimp in anyone's coding style, but it
would help enormously if the IDE could help reveal the intent of the
original authors in situations where it is not obvious.
Cheers, (the other) Eric
On Nov 21, 10:16 pm, Naftoli Gugenheim <naftoli...@gmail.com> wrote:
> Another idea: maybe the convention should be that all symbolic methods
> should be a simple alias for an alphabetic method.
>
> def ^%[A,B](x: A, y: B, z: Int) = fgh(x,y,z)
>
Perhaps a better alias might be:
def ^%[A,B](x: A, y: B, z: Int) = wtf(x,y,z)
I like Bernd's idea of some kind of scaladoc notation that can be used
for hover help over operator methods. That seems like it could help a
lot. Wait, don't they already pop up scaladoc?
Bill
This is not an argument against well-chosen symbolic methods (i.e. operators), but it is an argument in favor of fanatic efforts at documentation given that some of the usual approaches do not work. Cryptic anything is bad, not just symbolic method names. (Thus, poorly-chosen method names are bad, whether it is << instead of print, or dividedBy instead of /.)
--Rex
Good philosophy.
> So ScalaTest originally just had ===, which means
> equality,< for less than,> for greater than,<= for less than or
> equals, and>= for greater than or equals. At some point I did add one
> non-obvious operator, the - in FreeSpec, because after much searching
> for a way to avoid it, it was the best solution. In 2.1, which is
> coming next year, I'll be adding a few more operators. One is +/- as
> an alias for plusOrMinus. This one has been requested by multiple
> users over time, and I've done several informal surveys asking people
> if they can guess its meaning, and people always have. So I decided to
> add that one. I'm also adding&&, ||, and ! for boolean-like
Well said.
I would add, that for any software developer use whatever style you like
just so long as you know who your audience is. If you are part of a
small clique of experts in something, go with whatever works for you.
However, if you want to foster broad appeal of your APIs or libraries,
favor clarity and familiarity over concise expressiveness.
I would not even be opposed to going beyond the ASCII set of characters
and using some math or Greek symbols where they are exceptionally clear
- such as ≠ ≤ ≥ but there needs to be an easy/obvious way for people
without unicode or UTF-8 support to interoperate, and I am not sure we
are really there yet.
Cheers, Eric
The confusion over operator overloading was my fault as I was comparing
my experiences with operator overloading in C++ (not being able to
recognize what was going on) against my experiences with Scala where
there can be many new operators (that can also make it hard to recognize
what is going on).
I would disagree and say that the IDE should do more than the developer,
and that they still don't do enough. The more the IDE does for me, the
more I can focus my thoughts and intent of solving the problem at had,
the less I can worry about the language, libraries/APSs, etc.
>
> An IDE is nothing more than a tool (as is everything else) you need to
> use it correctly. you can be just as productive with Scaladoc and a
> text editor. Perhaps the issues are with how things are defined,
> rather with how people expect that new and important concepts that
> need to be understood require a little effort?
Again I disagree. To me using Scaladoc and a text editor is similar to
using a horse and buggy, where as using and IDE is like using a car
which can get me where I need to be faster.
Lately I have been writing some Scala code in Eclipse, and I am
constantly reminded that the Scala editor is like a Model T Ford
compared to the SUV that the Java editor is. From my perspective, the
Java editor in Eclipse significantly reduces the complexity of Java for
me - the complexity of the process of writing code.
Cheers, Eric