Web Sockets for Kafka

[c205...@gmail.com]

Hi Guys, 

I have implemented a Kafka web sockets V1. This open the doors for languages other than java or scala to consume and produce message from and to kafka. Unlike rest proxy this is more efficient as it uses TCP internally and avoids constant polling to check if new message was posted on to Kafka topic. I have attached the Github link 

https://github.com/evolv34/Kafka-web-sockets

[milind parikh]

Very cool!

Can you please post an example of how Web socket would work with let's say javascript?

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.
To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/de7f00d9-2996-40c9-beea-cf1c69d72b4e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[c205...@gmail.com]

Hi, 

This one is in NodeJs.

var WebSocketClient = require('websocket').client;

var client = new WebSocketClient();

client.on('connectFailed', function(error) {

    console.log('Connect Error: ' + error.toString());

});

client.on('connect', function(connection) {

    console.log('WebSocket Client Connected');

    var openJson = {

messageType:'open',

topic:'testTopic',

kafkaUrl:'172.17.0.1:9092',

zookeeperUrl:'172.17.0.1:2181',

consumerGroup:'testGroup1'

};

connection.send(JSON.stringify(openJson));

    connection.on('error', function(error) {

        console.log("Connection Error: " + error.toString());

    });

    connection.on('close', function() {

        console.log('echo-protocol Connection Closed');

    });

    connection.on('message', function(message) {

        if (message.type === 'utf8') {

            console.log("Received: '" + message.utf8Data + "'");

        }

    });

});

client.connect('ws://localhost:9093/kafka');

And this one is Python client 

import time

from websocket import create_connection

ws = create_connection("ws://localhost:9093/produce")

print "Sending 'Hello, World'..."

while 1:

ws.send("Hello, World")

print "Sent"

time.sleep(2)

ws.close()

 

I have also tired with Intel Edison Iot device and it post message directly to kafka over Web sockets. which improved the efficiency.  

On Friday, January 6, 2017 at 11:37:17 PM UTC-6, milind parikh wrote:

Very cool!

Can you please post an example of how Web socket would work with let's say javascript?

On Jan 6, 2017 8:30 PM, <c205...@gmail.com> wrote:

Hi Guys, 

I have implemented a Kafka web sockets V1. This open the doors for languages other than java or scala to consume and produce message from and to kafka. Unlike rest proxy this is more efficient as it uses TCP internally and avoids constant polling to check if new message was posted on to Kafka topic. I have attached the Github link 

https://github.com/evolv34/Kafka-web-sockets

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.

To post to this group, send email to confluent...@googlegroups.com.

[Andrew Otto]

Nice!

I did something similar, albeit with a  specific purpose in mind.  If you only need consumption (and don’t want to store offsets AKA affect state of your internal Kafka clusters), check out https://github.com/wikimedia/kasocki

To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsubscribe@googlegroups.com.

To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/de7f00d9-2996-40c9-beea-cf1c69d72b4e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platform+unsub...@googlegroups.com.

To post to this group, send email to confluent-platform@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/3388017d-7146-42ba-a714-56b1cdfe5625%40googlegroups.com.