Spark 0.8 branch

[Matei Zaharia]

Hi folks,

In order to make the 0.8 release soon, I've created a new branch for it, on which we'll merge only bug fixes and a few of the new deployment features we still want there (e.g. updated EC2 scripts and some work to build one artifact that works with all Hadoop versions). If you continue sending pull requests against master, we will cherry-pick the ones with bug fixes into branch-0.8. Let me know if there are any other things you believe should be in 0.8 as well.

Matei

[Mridul Muralidharan]

Hi Matei,

  Not sure if it is already planned, but the write related Connection  race condition Patrick reported/fixed might need to go to 0.8 also ...

Regards
Mridul

--
You received this message because you are subscribed to the Google Groups "Spark Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spark-develope...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Patrick Wendell]

Yep - this will *definitely* be in 0.8!

[Matei Zaharia]

Yes, that's what I count under "bug fixes".

Matei

[Dmitriy Lyubimov]

i sent a small pull request that declares HBASE_VERSION in the build explicitly in the header section of sbt build. It is pain to dig for it and it is a significant build parameter.

[Evan Chan]

Matei,

How about documentation updates, such as for the binary distribution?

[Josh Rosen]

I was thinking of making a backwards-incompatible API change in the Java API, described at https://mail-archives.apache.org/mod_mbox/incubator-spark-dev/201308.mbox/%3CCAOEPXP5iWWTSEhJGq5eafpPY1Dxp8ZTAfuhX9w-2%2BaQ3hApbfA%40mail.gmail.com%3E

If nobody objects, I'll implement that change and submit a pull request to add it to 0.8.  I'm interested in doing this now because we won't break APIs in 0.8.1+, so it would have to wait until 0.9 if we don't do it now.

I have several planned additions to the Java APIs to add the features missing from the Scala APIs, but those changes shouldn't introduce any incompatibilities and can be added in a minor release.

Matei

[Matei Zaharia]

I think a change to Optional is good, so let's do that.

@Evan -- documentation should definitely be added; feel free to send a PR adding a page on this to the docs. I also want to take a look at the binary distribution stuff once we've figured out the SBT build a little bit better.

Matei