Re: node.js integration into IBM Websphere MQ

[Stella Laurenzo]

(Redirecting to nod...@googlegroups.com and copying the original author)

It's been years since I did any MQSeries programming (dating myself with the name) but when I did it, it was fairly involved.

The primary thing you are going to run into is that the MQI C API is primarily a blocking API whereas node.js only talks to non-blocking APIs as part of its main event loop.  I think I recall some non-blocking facilities in the API but they were restricted to z/os and only covered a few operations.

It gets thorny from here.  You can either use a worker process pattern or try to talk directly to the wire protocol with non-blocking TCP interactions (I have no idea where to begin on this or even whether it is publicly documented anywhere).  I have a vague recollection of building at one point a pseudo-non-blocking dispatcher that was able to wait for a message from an arbitrary number of queues while using only one thread, but I can't remember any more details except that it wasn't the easiest thing to do.  Further, I am pretty sure that it still involved blocking MQI calls but allowed you to at least do the equivalent of a select() on a number of open queues.

If you use worker processes to wait on queues, you will then need to manage such things as committing the message only after it is consumed, etc.  Pretty tricky to get right but not impossible.

I don't mean to discourage you, but unless if the landscape has changed a good deal with WMQ over the last several years or unless if someone has already solved for this, its going to be hard.

- stella

On Mon, Jan 24, 2011 at 6:40 AM, Johnny Miller <johnny....@gmail.com> wrote:

Hi,

I am pretty new to node.js and was wondering if anyone has had any
experience of integrating node.js into IBM Websphere MQ. I am keen to
start using node.js for some work I am doing, but unfortunately we
have to call out to some existing systems exposing interfaces on MQ.
What I was thinking of was writing an extension using the underlying
MQ C libraries.

Has anyone done something like this before? Any tips on where to start/
gotchas/constructive suggestions etc..?

Thanks,

Johnny

[Stella Laurenzo]

I remember now.  You can multiplex gets from multiple local queues by using an Initiator Queue.  If I recall, this is how they do large scale message dispatching in batch mode on the mainframe, but the approach works for non-mainframe systems as well.  You will still need to do a blocking MQGet call on the initiation queue (think of it as a select()) but after getting a message saying that queue X has messages, you can loop in non-blocking calls to the actual queue to read its messages.  Because the read of the initiation queue is blocking (as well as queue manager connect, etc), you would still need to guard this in a sub-process, but you could at least create a pretty generic message dispatcher using a single sub-process instead of one per queue or one per read.

Hope that helps.

- stella

[James Carr]

One thing I'd suggest doing is building some kind of bridge between
the two. I have done this to tie node.js to publish on a JMS topic
hosted on weblogic by simply rerouting messages from a REST based
service to the JMS topic.

Best I can think of. Hope that helps.

James

> --
> You received this message because you are subscribed to the Google Groups
> "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com.
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en.
>

[billywhizz]

i've done a lot of work with MQ in the past (mostly from c#) and this
was something i was interested in getting running on node.js too.
unfortunately, IBM does not publish their binary protocol and any
solution you come up with based on reverse engineering it would not be
supported at all. i think the only viable option would be to write a
node.js addon in c++ using the official MQ client libraries and
implement some kind of thread pool in order to allow asynchronous
behaviour (you can use threads with the client libraries as long as
you only allow one request per connection at any one time so you'd
need to do your own locking too). the libraries by default are all
blocking as far as i can remember and the real low level stuff is not
made available as source code so you can't even hack on that...

i'd imagine it would be a very large undertaking unless you can get in
touch with someone from IBM to provide you with the client source code
so you could see what would be involved in making the default libs non-
blocking.

> >> On Mon, Jan 24, 2011 at 6:40 AM, Johnny Miller <johnny.p.mil...@gmail.com>

[billywhizz]

i dug out some old c/c++ code i was hacking on with IBM Websphere MQ
client 6 and have put it up on github if you want to have a dig
around:

https://github.com/billywhizz/mqsimple

if you linked this lib into a node.js addon and wrapped it up in a
thread pool you might be able to get what you need working. very
simple example here:

https://github.com/billywhizz/mqsimple/blob/master/src/cpp/qtest.cpp

[Liam]

On Jan 24, 12:21 pm, billywhizz <apjohn...@gmail.com> wrote:
> i dug out some old c/c++ code i was hacking on with IBM Websphere MQ
> client 6 and have put it up on github if you want to have a dig
> around:
>
> https://github.com/billywhizz/mqsimple
>
> if you linked this lib into a node.js addon and wrapped it up in a
> thread pool you might be able to get what you need working.

libeio eio_custom?

[billywhizz]

yeah - was thinking that might be a way to do it... would be nice to
see if this could be spun up into something useful. feel free to fork
my c/c++ code (it was written a long time ago and probably lots wrong
with it) and have a play...

[Stella Laurenzo]

If you're willing to put a dependency on setting up a trigger queue, you can just use one thread which loops on a blocking read of the common trigger queue.  The message that MQ puts on the trigger queue will signify which actual queue has messages on it waiting to be read.  Your loop then opens the referenced queue, pops a message and dispatches it.  The documentation spells out some other details that make this approach work reliably.

Back when I coded this pattern last, I had the receiver create a temporary queue as the trigger queue and then dynamically alter each queue we were dispatching off of with the right trigger settings.  This is a bit more complicated, though and can just be relegated to requiring the right configuration of the queue manager.

- stella

[Ben Mann]

MQ "Light" can help here. It has good node.js integration.

https://www.ibmdw.net/messaging/2014/02/26/connect-multiple-mq-light-clients-modifying-node-js-samples/