- 2.11:
- specialisation
- reflection
- optimizer
- non-urget api changes (even if backwards compatible)
- spec changes/updates/clarifications
We should also apply the surgical knife in 2.11 to extracting the most easily separable elements of the library into separate and optional libraries. Right now scala-library.jar is over 7 Mb. It could half that size without most people noticing anything had gone missing.The most easily separable pieces, not to mention the ones in the worst state of repair, are:- xml- collections viewsFrom memory these would in combination recapture between 1 and 1.5 Mb.Another easily separable piece (not in poor repair to my knowledge but also not yet in wide use) are the parallel collections, which again from memory would recapture about 1 Mb.
- The JSON classes feel a bit like they were meant as a code example for the parser combinator library. Compared to hand-tuned libraries, they seem to be too slow for any serious usage. There has to be a reason why it is not used in the wild. I guess it would make sense to ask those Scala projects which use other JSON libraries and investigate why they don't use the one shipping with Scala.
- There are many great third-party parser combinator libraries out there and I think the ecosystem could benefit from more competition here. Not bundling one with the standard lib could level the playing field and increase innovation, especially with macros around the corner.
On Thu, Dec 13, 2012 at 9:12 AM, Simon Ochsenreither <simon.och...@gmail.com> wrote:
- The JSON classes feel a bit like they were meant as a code example for the parser combinator library. Compared to hand-tuned libraries, they seem to be too slow for any serious usage. There has to be a reason why it is not used in the wild. I guess it would make sense to ask those Scala projects which use other JSON libraries and investigate why they don't use the one shipping with Scala.
I may be misremembering but I thought we agreed long ago to deprecate and remove the json library. I'm displeased to see that it's not even deprecated.
- There are many great third-party parser combinator libraries out there and I think the ecosystem could benefit from more competition here. Not bundling one with the standard lib could level the playing field and increase innovation, especially with macros around the corner.
I agree on this as well - I was holding back a little in case enumerating too many items might activate any automatic resistance systems and interfere with my extracting the largest pieces.
(These would join the ranks of scala-swing.jar and scala-reflect.jar)
Candidates for removal:
Looking at the packages above, there is one thing which becomes increasingly clear and which I'd like to repeat once again:
Putting stuff in scala.util is—in 99% of the cases—a clear code smell.
Continuing https://github.com/scala/scala/pull/1903 as suggested by Adriaan.
I think there are basically two categories and appropriate actions we'd have to consider:
- Broken stuff (which has alternatives) which we want to get rid of: first deprecation, then removal
- Stuff which works, but is not high-quality/popular enough to justify it shipping with the main jar: modularization (put into separate jar)
I'm wondering if there is a third category where we would both deprecate and modularize something. I think the only result would be to get deprecated stuff out of the main jar ahead of the removal phase. But if this was possible, would there be any case where we wouldn't use this possibility by default?
As far as I see proposed candidates for modularization are:
- scala.sys
- collection views
- parser combinators
(These would join the ranks of scala-swing.jar and scala-reflect.jar)
Candidates for removal:
- scala.actors
- scala.util.parsing.json
- scala.util.grammar
- scala.util.regexp
- scala.util.automata (BerrySethiGlushkov and friends (If they are still required for XML, at least make them private. No one should ever see these classes))
- small parts of scala.xml, e. g. the demo applications
Looking at the packages above, there is one thing which becomes increasingly clear and which I'd like to repeat once again:
Putting stuff in scala.util is—in 99% of the cases—a clear code smell.
Er, why? To make it extra-awkward in order to use one or two tiny system-dependent classes, so that people will stop using them at all and just use the Java equivalents instead?
Meh. These are one of the unique features of the Scala library, and again it's not much space (285k, ~4%).
There are many great third-party parser combinator libraries out there and I think the ecosystem could benefit from more competition here. Not bundling one with the standard lib could level the playing field and increase innovation, especially with macros around the corner.
- scala.util.parsing.json
Haven't used them, so I'm not sure how broken they are. It's a nice capability to have if it works. If it doesn't, it shouldn't mislead people.
- scala.util.grammar
Sounds good.
Well, the point of util is that you can put anything there. Anything in util can be promoted to the top level and be just as clear. Sometimes, however, you want to have a grab-bag of lesser-used things, and util is a good spot for that. I'm not sure why Either and Try were demoted to util (wouldn't have been my choice), but it's an entirely sensible spot for Sorting, continuations and logging, which is well over 99%. Parsing could be moved to top-level (but not without json and parser combinators).Putting stuff in scala.util is—in 99% of the cases—a clear code smell.
Also, what about scala.text? Could that be moved into util? It's kind of silly that it's taking up top-level namespace like that.
Also, I want to see a fully-working something-that-is-better-than-scala.Enumeration before deprecating Enumeration.
The (with alternatives) proviso is important, and Enumeration is not badly broken, just too awkward to be worth it.
Er, why? To make it extra-awkward in order to use one or two tiny system-dependent classes, so that people will stop using them at all and just use the Java equivalents instead?
Because it is working, mostly documented (afair), but not often used. Additionally, it doesn't seems to be one of the more easily separable parts of the library. Maybe other people can comment on it.
Meh. These are one of the unique features of the Scala library, and again it's not much space (285k, ~4%).
See comment above:There are many great third-party parser combinator libraries out there and I think the ecosystem could benefit from more competition here. Not bundling one with the standard lib could level the playing field and increase innovation, especially with macros around the corner.
- scala.util.parsing.json
Haven't used them, so I'm not sure how broken they are. It's a nice capability to have if it works. If it doesn't, it shouldn't mislead people.
Same case as above. There are tons of faster, more efficient, actually maintained and documented JSON parsers out there. Plus same argument with type macros.
Well, the point of util is that you can put anything there. Anything in util can be promoted to the top level and be just as clear. Sometimes, however, you want to have a grab-bag of lesser-used things, and util is a good spot for that. I'm not sure why Either and Try were demoted to util (wouldn't have been my choice), but it's an entirely sensible spot for Sorting, continuations and logging, which is well over 99%. Parsing could be moved to top-level (but not without json and parser combinators).Putting stuff in scala.util is—in 99% of the cases—a clear code smell.
I strongly disagree with that. We are developing a standard library here, not some node.js module.
A grab-bag of lesser-used things just doesn't deserve to exist in a standard library
scala.util is a prime example of the broken-window theory. People see the sad state of scala.util and think it is an excuse to put more untested, unmaintained, undocumented code into it.
Java is suffering from a new exploit every other week because of a similar “library as a grab-bag of stuff” approach. Let's not do that.
I've been doing this deprecation dance for some time already, and I'd like to see the day were we remove more cruft than we accumulate at least once in my life.
Also, what about scala.text? Could that be moved into util? It's kind of silly that it's taking up top-level namespace like that.
Also, I want to see a fully-working something-that-is-better-than-scala.Enumeration before deprecating Enumeration.
Use Java enums, just like for RuntimeAnnotations. They work perfectly well. Having a Scala-native way to define them would be great, but solely getting rid of scala.Enumeration would already be a net improvement.
The (with alternatives) proviso is important, and Enumeration is not badly broken, just too awkward to be worth it.
Searching for “Enumeration” in the bug tracker might convince otherwise.
"Let's make Scala more inconvenient and harder to use for command-line style scripting tasks" does not seem to me to be a good idea.
There are many great third-party parser combinator libraries out there and I think the ecosystem could benefit from more competition here. Not bundling one with the standard lib could level the playing field and increase innovation, especially with macros around the corner.Well, we could use the same justification to drop the collections library, too. Personally, I don't want the playing field level. I want the close field to contain everything I need to get as much work done as possible.
Maybe someone with a fast, efficient, maintained and documented JSON parser could be encouraged to allow their parser to be part of the standard library.
A grab-bag of lesser-used things just doesn't deserve to exist in a standard library
You'll need to justify this more for me to be convinced at all. As far as I can tell:* Some library features are needed often* Some are needed, but not so often
* Some are easily grouped into large categories* Some are not so easily grouped but still necessaryYour solution demands that if code is not needed so often and is not easily grouped, it should be removed entirely, or it should take up the most-valuable top-level namespace.
This is a really strange design decision. It violates standard principles of naming (e.g. Huffman coding) where you want the commonly-used things to be easy-to-use, and the less-common things to be less-easy.
I'd argue exactly the opposite: a bunch of stuff in scala. should be demoted to scala.util.:* DelayedInit* deprecatedName* FallbackArrayBuilding* languageFeature* LowPriorityImplicits* Predef* Responder // Goes with CPS* ScalaReflectionException // Goes with reflection* Specializable
* StringContext* UninitializedFieldError* compat.Platform // to util.Platform* ref* text
... do you seriously think that not having a ".util" to put it in will stop them?
I've been doing this deprecation dance for some time already, and I'd like to see the day were we remove more cruft than we accumulate at least once in my life.Sure, that would be nice. But not using util for a useful purpose is not the best way to go about that.
Whoever ditches Enumeration should also be responsible for fixing all instances of it in the code base, including Swing. This is not likely to be a great deal of fun.
The (with alternatives) proviso is important, and Enumeration is not badly broken, just too awkward to be worth it.Searching for “Enumeration” in the bug tracker might convince otherwise.
It's not _badly_ broken. Deprecation as a mechanism by which to alert people of the bugs that do exist may be sensible, but deprecation without a fix is about as clear an indication of a broken window as there is. This is a signal not just to library maintainers but also people who may consider adopting Scala (and can look more scary than it really is)
Hi,"Let's make Scala more inconvenient and harder to use for command-line style scripting tasks" does not seem to me to be a good idea.
How would it get harder? It would ship with the distribution, would be on the classpath, nothing would change, except that people could just drop scala-sys.jar if they wouldn't need it.
But I don't have any strong opinions on it. I just think that combining all the modularizing stuff could easily shave of another megabyte, even if the individual items don't weigh in that much.
There are many great third-party parser combinator libraries out there and I think the ecosystem could benefit from more competition here. Not bundling one with the standard lib could level the playing field and increase innovation, especially with macros around the corner.Well, we could use the same justification to drop the collections library, too. Personally, I don't want the playing field level. I want the close field to contain everything I need to get as much work done as possible.
No, not really. I think most people would agree that Scala's collections are state-of-the-art tech,
that's completely different in the parser combinator case. Additionally, I don't see multiple competing third-party collection packages in the wild, unlike parser combinators.
Nevertheless, if we can modularize collections ... I would have nothing against a 200KB bare-bone scala-core.jar.
Maybe someone with a fast, efficient, maintained and documented JSON parser could be encouraged to allow their parser to be part of the standard library.
With type macros arriving in 2.11, I think the whole “let's throw away static typing and use strings” approach to handle JSON, XML, ... is just completely obsolete. Let's wait and see how this stuff works out.
A grab-bag of lesser-used things just doesn't deserve to exist in a standard library
You'll need to justify this more for me to be convinced at all. As far as I can tell:* Some library features are needed often* Some are needed, but not so often
* Some are easily grouped into large categories* Some are not so easily grouped but still necessaryYour solution demands that if code is not needed so often and is not easily grouped, it should be removed entirely, or it should take up the most-valuable top-level namespace.
No. It should removed entirely.
No idea why it should be moved to a top-level namespace instead, that's not what I suggest.
My focus on scala.util comes from the fact that people seem to assume that it should have lower quality standards than the rest of the library.
Moving it around doesn't solve any problems with it.
Packages are names, and names are an important part of any API. If code violates this, it shouldn't be included until the naming issues are fixed.
I'd argue exactly the opposite: a bunch of stuff in scala. should be demoted to scala.util.:
* DelayedInit* deprecatedName* FallbackArrayBuilding* languageFeature* LowPriorityImplicits* Predef* Responder // Goes with CPS* ScalaReflectionException // Goes with reflection* Specializable
* StringContext* UninitializedFieldError* compat.Platform // to util.Platform* ref* textI think most items above could be moved to far better packages than scala.util. For instance @deprecatedName, along with @deprecated, @deprecatedInheritance and @deprecatedOverriding could be moved to scala.annotation. The internal stuff should go into scala.runtime.
The (with alternatives) proviso is important, and Enumeration is not badly broken, just too awkward to be worth it.Searching for “Enumeration” in the bug tracker might convince otherwise.
It's not _badly_ broken. Deprecation as a mechanism by which to alert people of the bugs that do exist may be sensible, but deprecation without a fix is about as clear an indication of a broken window as there is. This is a signal not just to library maintainers but also people who may consider adopting Scala (and can look more scary than it really is)
Even “slightly” broken should be too much to ship with the standard library. Adding @deprecated("Use Java enums instead") would be straight-forward fix.
--Rex
That looks much better!
But what's the overhead/performance of the macro-style vs. old-style enumerations? I can envision two classes created per name, where before it was a single instance of a class.
That looks much better!
Agree, Alois did some impressive work.
But what's the overhead/performance of the macro-style vs. old-style enumerations? I can envision two classes created per name, where before it was a single instance of a class.
My "envisioned" end result would be an enum construct compatible to Java enums. This means one instance per item.¹
If we turn down 10 individual things because each is only 4% of the jar, we have cumulatively passed over a 40% size reduction.
For me the question is not how big or is but is it internally tightly coupled and externally loosely coupled, and does it represent a unit of functionality that developers will tend to use or not use as a unit? Moving forward, is it something that could be patched in isolation?
Sent from my android - may contain predictive text entertainment.
So you either tackle collections, or it's going to be small potatoes. Parts of util and all of xml and concurrent would probably be the easiest wins; together that's maybe 20%.
There weren't 10 things on the table, there were three; one was a clear win, and the other two added up to less than 5% total and seemed a bit questionable to me.
I agree that if we had ten things that were 4% each, we shouldn't reject each one individually. But right now, 2.10 looks something like
To me, JAR size is a secondary concern. We're really looking for liberate parts of the library.
On Fri, Jan 18, 2013 at 4:57 PM, Rex Kerr <ich...@gmail.com> wrote:
There weren't 10 things on the table, there were three; one was a clear win, and the other two added up to less than 5% total and seemed a bit questionable to me.
I agree that if we had ten things that were 4% each, we shouldn't reject each one individually. But right now, 2.10 looks something likeTo me, JAR size is a secondary concern. We're really looking for liberate parts of the library.Right now, all contributions to scala/scala are subject to the same process for code review, source and binary compatibility, and are governed by a single release schedule.
Right now, all contributions to scala/scala are subject to the same process for code review, source and binary compatibility, and are governed by a single release schedule.Well, if it works out well, I'm all for it. But I'm skeptical that this won't create more version compatibility pain than it solves in terms of better features. At the end of the day, you don't want low-quality code to be part of the distribution, so someone is going to have to spend some time to make sure the code is decent. Chopping it up into modules might help a little in that you could assign people to different areas (but you could do that anyway...), but I'm unconvinced that the solution is a solution without going all the way to just plain dropping the code from the distribution (at which point you're not responsible any more for whether it works).
Anyway, I hope it works.
--RexP.S. If I had a patch to Swing, and the problem was a lack of unit tests, I'd write a unit test for the old broken case and the new case, and then it would pass the existing hurdle, correct?
On Fri, Jan 18, 2013 at 4:57 PM, Rex Kerr <ich...@gmail.com> wrote:There weren't 10 things on the table, there were three; one was a clear win, and the other two added up to less than 5% total and seemed a bit questionable to me.
I agree that if we had ten things that were 4% each, we shouldn't reject each one individually. But right now, 2.10 looks something likeTo me, JAR size is a secondary concern. We're really looking for liberate parts of the library.Right now, all contributions to scala/scala are subject to the same process for code review, source and binary compatibility, and are governed by a single release schedule. But perhaps scala-swing, for example, could attract more external contributions if it could be released more frequently, and if contributions didn't filter through the central scala/scala pull request queue, and pass the entire scala test suite. (Our current policy only accepts tested patches, but we have no test framework for scala-swing, so by the letter of the law its development is frozen.) Perhaps some young libraries could offer weaker compatibility guarantees until they graduated, but are of sufficient use to include in a distro (like scala-reflect today).
Paul, the real question is: some of those changes don't affect the API and are useful. What are they waiting for? Is help wanted?
- The JSON classes feel a bit like they were meant as a code example for the parser combinator library. Compared to hand-tuned libraries, they seem to be too slow for any serious usage. There has to be a reason why it is not used in the wild. I guess it would make sense to ask those Scala projects which use other JSON libraries and investigate why they don't use the one shipping with Scala.
I may be misremembering but I thought we agreed long ago to deprecate and remove the json library. I'm displeased to see that it's not even deprecated.
--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Anyone against deprecating scala.text, that is the single scala file in that package?
PR here: https://github.com/scala/scala/pull/2733
Anyone against deprecating scala.text, that is the single scala file in that package?
PR here: https://github.com/scala/scala/pull/2733