- Nathan
although i'm not actively involved in any of the implementations, i would like to say that i pretty much share the thoughts of daniel.
on the one hand, the question is if really more packages should be bundled with scala-library or if not the approach of scala-swing is better. after all, the main point is to have an official standard for the API. on the other hand, like actors, having the STM readily there will be a great advantage. and i suppose the stm would run fine on other systems such as android.
it also signalised that scala shines when it comes to concurrency -- we have java.util.concurrent for free, we have scala actors [*], and now we also have STM which is not only incredibly useful in practice, but also good PR regarding scala's competing languages (clojure, haskell).
given that i have been working for over a year now successfully with CCSTM, i'm pretty confident that the ccstm based implementation of the scala STM api should be good.
i also think it doesn't make sense to add the API without the implementation.
however, i would think that the parties trying to do other implementations (akka, multiverse) should be checking very carefully that by releasing the API now they're not going to run into future problems when they are actually doing their implementations.
2c, -sciss-
[*] couldn't that actual be made an API with different implementations? like the original scala actors, lift actors, akka actors etc.?
The primary practical concern that I see is an increase in the size of
scala-library. Compressed, the STM API is about 300K; the reference
implementation is another 230K. (A lot (most?) of the bulk comes from
the forwarding stubs generated when a class extends a trait. Extending
mutable.Map results in > 200 stubs, for example.) For reference
scala-library.jar is about 9M. The STM would be stripped out for
Android targets if it wasn't used.
The biggest potential future problem for other implementers is the API's
inclusion of features that require support for partial rollback, namely
failure atomicity (rolling back on exception) and modular blocking
(retry/orAtomic, AKA retry/orElse). We discussed this on the mailing
list a while ago and agreed that it was worth the effort.
- Nathan
I'd recommend waiting.
If the actor library is a guide... well the library has a bad interface and a worse implementation but because it's baked into the scala distribution there's substantial momentum and ego in not improving it.
I think it's worth spending a year with the stm stuff outside the main distro so it can be gotten right.
Martin, we haven't heard from you since the initial post. Could you provide some insight as to the decision making process here? We've provided our input, but Scala 2.9 RC1 is now out, so it looks like our window is rapidly closing, if not already closed, on getting this (highly anticipated) feature into the current release cycle.
best, -sciss-
but then: what would be the advantage then to have the API at all shipped instead of having it, too, as a separately downloadable package? i mean, if i want to add an implementation, let's say to CCSTM, i could as well add the API as a second dependancy with zero effort, no?
I feel very strongly on this point: we need to include a default implementation. I think Java framework examples make this point beautifully. Shipping the API without the implementation may be the more generic, "enterprisy" (I hate that adverb) thing to do, but it dramatically reduces the utility of the API. In the case of Java, this ended up being an almost universally despised design decision. In fact, you're the first person I've met who actually likes it!
I feel very strongly on this point: we need to include a default implementation. I think Java framework examples make this point beautifully. Shipping the API without the implementation may be the more generic, "enterprisy" (I hate that adverb) thing to do, but it dramatically reduces the utility of the API. In the case of Java, this ended up being an almost universally despised design decision. In fact, you're the first person I've met who actually likes it!
Perhaps it is better to wait releasing the Scala STM API, since there is only a single implementation. A good API can imho only be created after multiple implementations. In a few weeks I'm going to have much more time for getting Multiverse implementation of the Scala API up and running and see what works and what doesn't work.
Jonas,I understand your concerns about blessing one implementation over another, but I think we've made the footings as equal as possible while keeping the initial friction low.
The API is structured so that the user only interacts with generic classes. All of the implementations (reference + Multiverse) will appear to the end user as scala.concurrent.stm.* (except where they use an implementation-specific extension, of which CCSTM doesn't have any). This is important so that the entire program (including all of its libraries) will end up using a single STM implementation.
The only way in which CCSTM is blessed by the API is that it is used as a fallback at runtime if the system property "scala.stm.impl" is not defined. Any code or library that was developed using the reference implementation will also run using Multiverse (once it implements the API) without recompilation. Only one class from the reference implementation is public, and that is only so that it can be created via Class.forName.
I feel very strongly on this point: we need to include a default implementation. I think Java framework examples make this point beautifully. Shipping the API without the implementation may be the more generic, "enterprisy" (I hate that adverb) thing to do, but it dramatically reduces the utility of the API. In the case of Java, this ended up being an almost universally despised design decision. In fact, you're the first person I've met who actually likes it!
Consider, for example, the JAXP library. Almost everyone working on the JVM has to do XML processing at some point. So, of course, they turn to the built-in support. Until recently (well, a few years ago), the "built in support" consisted entirely of an API with no implementation. Worse than that, the lack of implementation wasn't known statically, so people would write code, try to run it and then be mystified by the reflection errors which cropped up. It always baffled me that I would need to download some third party library which I would never use directly (and thus would be seen as a "dead" dependency) to enable functionality which was supposed to be built into rt.jar! The JCE framework (cryptography) still suffers from this problem to a large degree, since anyone who does any real work with JCE needs to go out and download Bouncycastle. This also raises non-coding issues. For example, I have (more than once) had to justify to management why our code base was dependent on some random third party framework. (I've had this problem with Xerces too, just not recently)
Judging by developer mindshare, the "batteries not included" approach to language framework design has really gone the way of the dodo. Developers want to start hacking using a language and expect all of its built-in APIs to work as advertised out of the box.
So, this is something that I'm going to argue quite strenuously: we need to include a default implementation for Scala STM, and more generally, Scala must continue to go that route moving forward. This is even more important to me than the question of which implementation will be included.
i understand that people are really busy, but hasn't this group been created some 9 or 10 months ago? i would have thought there was enough time to at least take a look at the efforts which in my perception have been mostly driven by nathan's work, and figure out whether it was adoptable for the other projects. i find the complaints now about the missing second implementation a bit late.
best, -sciss-
I guess the question I would ask is the following: what do we expect to discover if Scala STM were moved into the incubator? I would posit that most people who really need an STM in Scala now have already looked at Scala STM. The audience we're missing are the people who don't know they need an STM, or perhaps the people who's need is less acute. However, this group of people are no more likely to use an incubator project (that they may or may not know about) than a third party library.
In other words, I don't think an incubator stage would gain us very much at this point. We've been hemming and hawing about this framework for two years now. Heck, even Java stdlib enhancements tend to move faster! I say we just drop it into the Scala standard library and move forward from there.
There is no technical barrier to moving the STM into the standard
library (proof by example:
https://github.com/nbronson/scala/commit/4e08a1bee01e389029add5cfb00b3406953cb943
). At the moment the STM itself doesn't gain extra powers or
efficiencies by being in the stdlib, it is just a matter of build and
distribution convenience (both pros and cons) and of an official
blessing. So far I have managed to keep a single source image for all
of the targeted Scala versions, but that might be harder in the
future, at which point inclusion in the stdlib would be convenient.
Not too many people ask questions on this list, which means the STM
doesn't have very many direct users, direct users don't have very many
problems, or direct users don't ask when they have a problem.
But you lose the ability to make changes. I'm not so sure it's all unicorns and win going into the STD. Library....
You still have to maintain binary compatibility with scala distros then.
I'm not against it, but e.g. I'm not trying to push scala-arm in the distro immediately either....
I'm working on it.
On Thu, May 24, 2012 at 5:05 PM, Josh Suereth <joshua....@gmail.com> wrote:
> But you lose the ability to make changes. I'm not so sure it's allIn what sense you lose the ability to make changes? Can STM change now
> unicorns and win going into the STD. Library....
at will? If it can, then no one is using it -- only experimenting with
it. There's much to be said about not being able to make changes
because there are people using it for production code.
And more popular than the continuations plugin...
On Thu, May 24, 2012 at 4:59 PM, Daniel Sobral <dcso...@gmail.com> wrote:On Thu, May 24, 2012 at 5:05 PM, Josh Suereth <joshua....@gmail.com> wrote:
> But you lose the ability to make changes. I'm not so sure it's allIn what sense you lose the ability to make changes? Can STM change now
> unicorns and win going into the STD. Library....
at will? If it can, then no one is using it -- only experimenting with
it. There's much to be said about not being able to make changes
because there are people using it for production code.
In the sense that you don't have the expectation of binary compatibility now.I hate to be a negative nancy (but I have to for my own library): You need to improve users, I feel, before hitting the standard library. When the "blessed" library bit is ready (i.e. the build utlity I'm working on is finished), then we can move you into there, but it's far too early for the standard library.I think it'd be better to do some PR, evangelize and give more talks about STM. Hype people up about it. Don't rely on the standard library to be your PR. You have to promote this project yourselves. You have something to promote, it's stable. You should be able to get users. Perhaps you need to promote *other libraries* to use yours to improve users. Either way, the users will stabilize the API, and you can get them today, with some effort.
The way to get more users is promotion. Even if it hits the standard library, we still need a way to make people aware of it. The standard library isn't a crutch to get users of a feature, and it should not be used that way. It's the same for scala-arm and scala-io. I'd love to push an IO library more than anything. BUT, I don't think scala-io has solidified enough to warrant usage, and Scala-Arm only has 600 downloads over the past year.I don't think that's enough for standard library inclusion. It may be enough for a blessed status, but certainly not going into the core.
Trust me, I think you're better off promoting the library before going into the standard.- Josh
i do use scala-swing, and i find it great that i can include it on a per project basis. (however, there are lot of aspects of scala-swing which i don't like, it could have been a much more radical and powerful library...)
i think the important message is to say, this is the way to do STM in scala. and i wouldn't mind having scala-xml and scala-actors as separate libraries, even scala-collection-mutable.