Starting a spark room linked with scala/scala room on gitter

[Bhavishya Pohani]

As a newcomer, I feel that community support for new users is very less for spark. The current places to talk to people regarding spark seem to be only -> 

1. Official spark mailing list
   
2. Stack overflow - apache spark tag
   
3. Freenode spark chat

However, 1 & 3 seem to be dead to me. While stack overflow is good, a chat room has its own benefits.

There is a lot of activity recently on the scala/scala room related to spark & since scala seems to be the primary language used in spark, I suggest that there should be another room started within the scala rooms that is related to spark only. This would help to segregate the traffic into the right rooms & also fill the void that is currently present.

What are your views regarding this?

[catc...@bromberger.com]

On Sunday, October 2, 2016 at 8:37:29 AM UTC-7, Bhavishya Pohani wrote:

There is a lot of activity recently on the scala/scala room related to spark & since scala seems to be the primary language used in spark, I suggest that there should be another room started within the scala rooms that is related to spark only. This would help to segregate the traffic into the right rooms & also fill the void that is currently present.

What are your views regarding this?

Enthusiastic thumbs up. I'd love a near-real-time spark resource. Gitter seems to be as good a place as any.

[Erik LaBianca]

On Sunday, October 2, 2016 at 11:37:29 AM UTC-4, Bhavishya Pohani wrote:

There is a lot of activity recently on the scala/scala room related to spark & since scala seems to be the primary language used in spark, I suggest that there should be another room started within the scala rooms that is related to spark only. This would help to segregate the traffic into the right rooms & also fill the void that is currently present.

 

A scala/spark chat room on gitter seems like a good idea to me. Several ancillary projects are there already but there's no good place for general spark chat that I'm aware of.

 

--erik

[Seth Tisue]

It seems like a very good idea to me. I see people asking Spark questions on both #scala on IRC and scala/scala on Gitter, and often not getting answers. Having a dedicated Scala-with-Spark room might not fix that, but it certainly seems worth a try to me.

Sometimes Spark users' questions are just generic Scala questions, but in my observation, almost always there's some Spark-specific angle and Spark users really need to be talking to each other much of the time to get help.

Seth Tisue / Scala team / Lightbend, Inc.

[Jan-Hendrik Zab]

TL;DR
I'm all for a Spark Gitter channel that is also for Python, R, $whatever-future-language questions related to Spark.


First time using this so called forum and I think it's a horrible experience, anyway here it goes.

FYI: There was already at least one similar request [0] with little success on the Apache Spark site.

Otherwise, I would like to see a more active Spark channel, even on gitter (whose web interface I detest with a vigor). And the ability to go offline and read the backlog later on without some IRC bouncer would benefit quite a few users. I'm regularly annoyed by people in #apache-spark asking their questions and going offline only a short time later, because no one answered. Maybe I'm getting old, but a lot of those people don't seem to get that a bit of patience might be required in some channels.

There is also a lot of knowledge repeatedly being lost on IRC that could just be found via gitters search function not that anyone would actually do that.

But I really do NOT want to see a Spark Scala only channel. There is a lot of overlap between the different languages that can be used and restricting the channel would only limit the adoption rate. (And people getting told not to ask i.e. Python Spark questions by some mod.) Hence, I do not really see why that room should be under the Scala namespace, unless the Spark Apache folks are still against creating their own.

Also, Scala might be the primary language used in Spark, but there are a _lot_ of people asking Python Spark related questions in #apache-spark, but I don't have any meaningful (faked) statistic at hand. ;)

ps. IRC: gothos

[0] - https://mail-archives.apache.org/mod_mbox/incubator-spark-dev/201504.mbox/%3cCAOhmDzf+kaGNEwLSHVHOuKP...@mail.gmail.com%3e

[Seth Tisue]

On Wednesday, October 5, 2016 at 1:22:44 PM UTC-5, Jan-Hendrik Zab wrote:

[...] unless the Spark Apache folks are still against creating their own [...]

Well, I don't have any information to the contrary. Would someone like to raise with this them directly, perhaps directing them to this thread?

A Scala-specific Spark room is the only kind of Gitter room that we (Lightbend + Scala Center) could make the decision ourselves to create.

[Rex Kerr]

I agree that there should be a gitter channel for Spark, or a something-with-history.

But where is it?

Right now, demand for Spark on Scala is creating a huge swath of questions and dumping the burden of answering them on Scala rooms where a majority of people don't know or use Spark.  People start out asking questions that sound like they're asking about flatMap, but then it turns out it's not the ordinary monadic flatMap but Spark's atypical one on an RDD that takes a TraversableOnce. This typically wastes a bunch of everyone's time, as half the answers don't work and nobody realizes why for a while until someone thinks, "wait, what if this is Spark?" or the person asking happens to let slip a comment that makes it clear that it's Spark.

I'd love to see more discussion options for all of Spark, but if something doesn't materialize quickly I think it's entirely reasonable to have a scala/spark room to set expectations properly so people can ask and answer questions without all the tripping over the differences between Spark and standard Scala collections.  Especially since it's the only thing Lightbend can really do to help the situation.

  --Rex

[0] - https://mail-archives.apache.org/mod_mbox/incubator-spark-dev/201504.mbox/%3cCAOhmDzf+kaGNEwLSHVHOuKP3UENjgEbopmJxVXcv7XO4...@mail.gmail.com%3e

--
You received this message because you are subscribed to the Google Groups "scala-debate" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-debate+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jan-Hendrik Zab]

On Wednesday, 5 October 2016 21:38:20 UTC+2, Seth Tisue wrote:

On Wednesday, October 5, 2016 at 1:22:44 PM UTC-5, Jan-Hendrik Zab wrote:

[...] unless the Spark Apache folks are still against creating their own [...]

Well, I don't have any information to the contrary. Would someone like to raise with this them directly, perhaps directing them to this thread?

I just send an e-mail to d...@spark.apache.org. I'll post a link here for cross reference when it appears in the archives. (Probably got moderated, since I'm not on the list.) 

[Dean Wampler]

I'll try to get a Scala Spark Gitter channel created, one way or another.

Dean Wampler, Ph.D.

Author: Programming Scala, 2nd Edition (O'Reilly)

Lightbend
@deanwampler

http://polyglotprogramming.com

--

[Jan-Hendrik Zab]

Why was my message with the link to the Spark dev list removed?

[Seth Tisue]

On Wed, Oct 12, 2016 at 2:54 AM, Jan-Hendrik Zab <j...@jhz.name> wrote:

Why was my message with the link to the Spark dev list removed?

I have no idea what happened there. Perhaps somebody misclicked? Feel free to repost.

[Jan-Hendrik Zab]

Thread on spark dev ml: http://apache-spark-developers-list.1001551.n3.nabble.com/Apache-Spark-chat-channel-td19261.html

I guess creating a scala/spark channel is the best option at the moment.

[Dean Wampler]

Okay, I created a Gitter channel for this purpose:

https://gitter.im/spark-scala/Lobby