Spark Integration with RabbitMq or Kafka

[Naga Reddy .S S]

Hi,

I am new to Spark framework and need help to develop prototype which I am working on. 

Here is my first doubht.

Can you please explain how I can integrate Kafka or RabbitMq with Spark for realtime analyatics?

Best regards,

Reds

[Mark Hamstra]

https://github.com/mesos/spark/blob/master/examples/src/main/scala/spark/streaming/examples/KafkaWordCount.scala

[gyorg...@gmail.com]

Hi!
I'd suggest Kafka because it's fair easy to fire up.
Take a look here: http://kafka.apache.org/07/quickstart.html
You can fire up a kafka server in minutes. If it's running, you just have to

• implement a seperated producer to send messages
• implement a spark streaming app (in scala or java) that subscribes to this kafka topic and will receive these messages

I'm currently researching the same topic for my company, here is my app's skeleton:

http://pastebin.com/sjEG4T5J

You should provide more specific questions. Read the docs:

• http://spark-project.org/docs/latest/
• http://spark-project.org/docs/latest/scala-programming-guide.html
• http://spark-project.org/docs/latest/java-programming-guide.html
• http://spark-project.org/docs/latest/streaming-programming-guide.html