Why are there two protocol magics (V1 and V2)?

[Alick Xu]

I was wondering why there are two protocol magics involved. From my log output, it seems like V2 is used to connect nsqlookupd to nsqd, but V1 is used when publishing messages. 

I've only found information about V2 in the docs. 

[Alick Xu]

Meant to say V1 when connecting nsqlookupd to nsqd, but V2 when publishing messages. Sorry, late night...

[Matt Reiferson]

Just an evolution of the implementation as we were initially building nsqd/nsqlookupd.  nsqd <-> nsqlookupd is the only thing that uses V1, so technically it's internal.

[Alick Xu]

Thanks for the quick reply Matt. 

That makes sense...unfortunately that leads to a bigger question for us.

 For some reason we keep getting “bad protocol magic V2” when trying to start up nsqd and nsqlookupd instances in our docker cluster managed by kubernetes. nsqd and nsqlookupd are both in separate docker containers. For some reason everything is okay when nsqd and nsqlookupd are pinging back and forth, but as soon as our Producer tries to Publish a gobbed version of a graph in the form of a byte array to a topic, we get the bad protocol magic error. We’re not sure why it only comes up when we try to publish. 

Here is where we publish our graph..

func (mc *Migrator) messageHandler() {

g := 0

config := nsq.NewConfig()

broadcast, err := nsq.NewProducer(mc.nsqLookUpdAddr, config)

if err != nil {

fmt.Println(err)

}

out:

for {

select {

case <-mc.quit:

break out

case msg := <-mc.inGraphsChan:

// Receiving graphs here

g += 1

// Decode graph

go func() {

// Get the body, which has the encoded gob of the colorgraph.

gob := msg.Body

b := bytes.NewBuffer(gob)

// Decode message, and get the size.

cg := &ColorGraph{}

cg.GobDecode(b)

mc.IncomingMigrations <- cg

}()

case outGraph := <-mc.OutgoingMigrations:

// Sending graphs out, this is where we PUBLISH and get errors

var b bytes.Buffer

// Create bytes to send

err := outGraph.GobEncode(&b)

if err != nil {

// TODO: handle error

fmt.Println(err)

}

// Publish

if err := broadcast.Publish("migration", b.Bytes()); err != nil {

// TODO: handle error

fmt.Println(err)

[Matt Reiferson]

I think you might be connecting to the wrong port.  How are you trying to publish?  HTTP?

[Alick Xu]

We tried publishing with TCP - we used ports 4150 for nsqd and 4160 for nsqlookupd when we received the error.

We also tried setting our system up with HTTP, in which case we used 4151 for nsqd and 4161 for nsqlookupd. When we tried, the two services couldn't connect to each other. Looking at the logs of nsqd shows this:

[nsqd] 2015/05/30 23:25:30.621769 nsqd v0.3.1 (built w/go1.4)

[nsqd] 2015/05/30 23:25:30.621955 ID: 705

[nsqd] 2015/05/30 23:25:30.622059 NSQ: persisting topic/channel metadata to nsqd.705.dat

[nsqd] 2015/05/30 23:25:30.638785 LOOKUP: adding peer 10.0.0.82:4161

[nsqd] 2015/05/30 23:25:30.638824 LOOKUP connecting to 10.0.0.82:4161

[nsqd] 2015/05/30 23:25:30.640276 TCP: listening on [::]:4150

[nsqd] 2015/05/30 23:25:30.640389 HTTP: listening on [::]:4151

[nsqd] 2015/05/30 23:25:30.644299 LOOKUPD(10.0.0.82:4161): ERROR IDENTIFY - unexpected EOF

[nsqd] 2015/05/30 23:25:45.648303 LOOKUPD(10.0.0.82:4161): sending heartbeat

[nsqd] 2015/05/30 23:25:45.648388 LOOKUP connecting to 10.0.0.82:4161

[nsqd] 2015/05/30 23:25:45.655551 LOOKUPD(10.0.0.82:4161): ERROR IDENTIFY - unexpected EOF

[nsqd] 2015/05/30 23:25:45.655806 LOOKUPD(10.0.0.82:4161): ERROR PING - use of closed network connection

[nsqd] 2015/05/30 23:26:00.648191 LOOKUPD(10.0.0.82:4161): sending heartbeat

[nsqd] 2015/05/30 23:26:00.648234 LOOKUP connecting to 10.0.0.82:4161

One thing I just thought of - we're using the deprecated dockerfiles found here https://registry.hub.docker.com/u/nsqio/nsqadmin/ since the official image didn't have dockerfiles. Perhaps this is related?

[Jehiah Czebotar]

So this might help you:

nsqd must be configured to talk to nsqlookupd using it's TCP port
(port 4160) via `-lookupd-tcp-address` option (assuming you are using
nsqlookupd)

When publishing messages you have two options: a) publish to nsqd over
TCP (port 4150) using one of the nsq libraries or b) via HTTP to nsqd
(port 4151). In either case, publishing does not use nsqlookupd but
publishing is expected to be to a known nsqd.

A client consumes messages via nsqd over TCP (port 4150) and discovers
nsqd via nsqlookupd HTTP api (port 4161).

--
Jehiah