A Node-RED CouchDB node ...

[Neil Kolban]

I've been looking to find a Node-RED node that interacts with a local instance of a CouchDB database but have come up dry.   This post asks if anyone out there in the Node-RED community has seen such a thing before or built one yourself.  If we still come up dry, I'd like to suggest that we have a bash at such a venture.  It seems easy enough ... we write a new Node-RED node and take as a dependency a CouchDB package such as "nano" and expose some of its operations.  If you know of existing artifacts, please post back.  If we don't find any and you'd like to participate in a community build, please post back.  If we don't find one in about a week, I'll go ahead and create a Github project, code it up and post it as a "node-red-contrib-couchdb" package.

[Dave C-J]

Neil,

the Cloudant node we already have uses nano as it's base - so it may well already do what you want... https://hub.jazz.net/project/laoqui2/node-red-node-cf-cloudant/overview

Worth checking to see if it's close enough.

[Claudiu Farcas]

I am also curious into having this kind of node on raspberry pi 2 node-red.

Please let me know when it will be available.

I might just as well to start my own experiments with it (and thus learning some node-red development :) ).

Regards,

Claudiu

[Nicholas O'Leary]

Claudiu,

install node-red-node-cf-cloudant and you should be ready to go.

Nick

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
For more options, visit https://groups.google.com/d/optout.

[Claudiu Farcas]

Thank you Nick, I'll try it for sure.

My goal is to connect on Raspberry Pi to the local CouchDB or PouchDB Server instance.
Hopefully this module will work, otherwise we should build one.
It is really strange the fact there is no "node-red-contrib-couchdb" package available yet.

 

[Nicholas O'Leary]

Clearly the cloudant node has served everyone's needs to date... :)

--

[Claudiu Farcas]

Nick, it seems this node-red-node-cf-cloudant package is actually working only with external cloudant databases (not local).

Or maybe there is a trick I need to know.

Or I can start digging into this node source code and use it to develop a new "local couchdb" node.

[Nicholas O'Leary]

This is a hangover from the fact the node was originally written for the Bluemix environment.

If you were running in Bluemix and had cloudant service instances bound to your app, they would appear in that list to pick from. As you are not running in Bluemix, pick 'External service' and then in the next row 'Server' add the details of your couchdb instance you want to connect to.

It would be nice if this node hid its Bluemix parts when running outside of that environment... something I'll take to the author about.

Nick

--

[Claudiu Farcas]

Thank you Nick, it seems that I am doing something wrong with my config.

I have a PouchDB Server installed on my raspberry pi which works fine (I can manage it in browser, I can replicate with my computer).
I wish to use this kind of local server to cover offline scenarios when internet might not work (using a local CouchDB or PouchdbServer).

But node-red-node-cf-cloudant doesn't connect to local working instance, all I get in logs is "Service unavailable" :(
Maybe an idea is to dig into this module source and modify to create something similar.

[Dave C-J]

... or enhance it so it works for both ?

[Julian Knight]

Am I being thick as usual? I thought one of the strengths of CouchDB was that it uses an HTTP REST API? So why not just use that like I am with InfluxDB which similarly uses REST.

[Nicholas O'Leary]

A benefit of using an underlying module is it encapsulate the api for you and can provide higher level abstractions for you to work with. (a similar argument to "Why use node-red when you could write the code yourself..." ;)

In the case of cloudant, it has options around connection pooling and the like, which would be hard to emulate using the raw HTTP request node. This would be of particular concern for anyone needing to do performance-critical work (which isn't theoretical... I'm in the midst of dealing with just such a request).

Nick

--

[Neil Kolban]

I've got a prototype working with a set of properties that look as follows:

This gives us an example of the sort of higher level extrapolation that might be possible.  Leveraging the underlying NPM module called "nano" resulted in the actual implementation code of the new node being trivial.

[Dave C-J]

There is a bug (caused by an oversimplification) in the existing cloudant node mentioned above. This is the only thing stopping it also working with both couch and pouch. We will try and get the fix into the work Nick is looking at.

[Neil Kolban]

Awesome.   In the meantime, I've put up an NPM contribution here:

http://flows.nodered.org/node/node-red-contrib-couchdb

[Claudiu Farcas]

Hi Neil, I have to admit that was fast.

And because I could wait, I install it and give it a go.

And come back with this error:

5 Mar 03:01:01 - [warn] [couchdbinsert:q2]: Error: {"message":"invalid_json","stack":"Error: invalid_json

    at Request._callback (/home/pi/.node-red/node_modules/node-red-contrib-couchdb/node_modules/nano/lib/nano.js:248:15)

    at Request.self.callback (/home/pi/.node-red/node_modules/node-red-contrib-couchdb/node_modules/nano/node_modules/request/request.js:199:22)

    at Request.emit (events.js:98:17)

    at Request.<anonymous> (/home/pi/.node-red/node_modules/node-red-contrib-couchdb/node_modules/nano/node_modules/request/request.js:1036:10)

    at Request.emit (events.js:117:20)\n    at IncomingMessage.<anonymous> (/home/pi/.node-red/node_modules/node-red-contrib-couchdb/node_modules/nano/node_modules/request/request.js:963:12)

    at IncomingMessage.emit (events.js:117:20)

    at _stream_readable.js:929:16

    at process._tickCallback (node.js:419:13)","name":"Error","error":"bad_request","reason":"invalid_json","scope":"couch","statusCode":400,"request":{"method":"POST","headers":{"content-type":"application/json","accept":"application/json"},"uri":"http://127.0.0.1:5984/testdb","body":"\"{\\\"_id\\\":\\\"test1\\\",\\\"val1\\\":18}\""},"headers":{"x-powered-by":"Express","content-type":"application/json","etag":"W/\"30-PzdqK0SRrYRUhNZKSVqF/w\"","vary":"Accept-Encoding","date":"Sat, 05 Mar 2016 01:01:01 GMT","connection":"keep-alive","statusCode":400,"uri":"http://127.0.0.1:5984/testdb"},"errid":"non_200","description":"couch returned 400"}

 

Any idea what might be causing it?

Best regards,
Claudiu

[Claudiu Farcas]

Ok, my fault.

The msg.payload needs to be passed as js object and not JSON string (nice to have sources to debug and figure out where it came from ;) ).

Now it works like a charm!

Awesome, nice job!

[Neil Kolban]

Claudiu,

Thanks for the kind words but there isn't much credit to take.  The combination of architectures of Node-RED, Node.js, CouchDB and the npm "nano" package basically meant the project was simply bolting some pieces together.  Since the source is on Github, everyone is welcome to augment it and improve it.  Ive only been at Node-RED for a few weeks so I am certain there is much that could be improved.  Ive given no thought to any form of connection caching or other related issues ... step 1 is basically get something working and from there we can start taking change requests.

The thing that I have found most interesting is thinking about "correct" Node-RED nodes.  I modeled this one after the interface for the Cloudant nodes.  But now questions come up as to whether we have one node that does everything or multiple nodes that only do few things?   Also, why do we have nodes that terminate the flow as opposed to simply having nodes with outputs that propagate the original input onwards ... these design level questions are the ones I have no answers to nor experience upon.

And ... I also have a very selfish reason for wanting this CouchDB node in existence ... next week I am demoing the IBM BPM product to a prospective customer and in the demo, I need database interaction ... and wanted to make my life simpler by using Node-RED as opposed to something more complex ... see:

https://www.youtube.com/watch?v=aRCUB-50Aqc

[Dave C-J]

Neil,

The basic tenet behind Node-RED is that is should hide the complexity of a function, and expose the essence of what it does in as simple a way as possible. That is why we prefer flows to "end" when the message "leaves". You would like to think that 99% of the time once you send data (for example) to a database it just goes there - so no need for anything else to follow. Of course errors do occur and a node can call node.error which can then be caught by the catch handler, or alert the user right away.

This (and more) is described in the docs here - http://nodered.org/docs/creating-nodes/

In general more simpler nodes is better (up to a point :-) - as a) the parts can be re-used elsewhere and b) the flow can be more easily read and understood without delving into each helping to reduce documentation (see where I'm going ;-).

We also encourage people to contribute to the community by helping to improve and fix existing nodes, for example the cloudant node already uses the nano module and should talk to couchdb (and pouchdb) just fine... but currently doesn't as it is slightly broken... we are/were looking at this issue when you jumped to release your node. It would have been much better to fix that, than create another node that does almost exactly the identical thing but slightly differently...   I know you said you had selfish reasons for it... but in that case just use it (and test the error handling) privately first...

[Neil Kolban]

Thanks for the response.  I think I now understand the flow philosophy.  Very useful.

As for the publication of the "couchdb" node, I'll be delighted to remove the "npm" listing of the node so that there is no confusion over which one to use.  Apologies for any confusion.  I'm always happy to fall in line with community practices with no questions asked.   Let me know if you want the package unpublished and I'll do so immediately.

[Claudiu Farcas]

I really like the simplicity of "node-red-contrib-couchdb" package, it is very minimalistic and works straight with the any couchdb flavour implementation.

I see no reason why not to keep it like this. If there are some other vendor specific implementation those can exist with their own name.

Best regards,
Claudiu

[Dave C-J]

Neil,

Thanks, Ideally yes... but I know you said you needed it for this week - and we haven''t actually published the fix for the cloudant one yet. So removing it now is probably premature.  Also of course what this points out is that it is non-obvious that the cloudant one does/can/should work with couchdb - and that the node is non-findable using search... So we need to work out how to fix that also so that everyone can find it.

PS - great work on the video by the way.