Node Unit Test Helper

[Mike Blackstock]

A while back we had a discussion on slack in #docs around node testing. Nick mentioned there was some code in NR to test the core nodes. I thought it might be good to pull that out to make it available for contributors. I've published it on npm as 'node-red-contrib-test-helper'. code is at https://github.com/SenseTecnic/node-red-contrib-test-helper

Basically pulled out the helper.js code from the tests and modified it slightly so it can be used in your own nodes. I started to document the helper API in the readme, and provided a short example unit test for the 'lower-case' node. Hopefully you find it useful.

Mike

[Julian Knight]

Looking forward to getting my head around this and starting to write some formal testing for my nodes.

I've never had the time or energy to be able to understand JavaScript testing frameworks before and I'm hoping that, just as Node-RED can be an introduction to instructing computers to get stuff done for non-programmers, this will be a catalyst to help more people get test harnesses together.

[Mike Blackstock]

Yes, hopefully this packaging will help node developers build automated tests.  Nick offered to set up a repository in the Node-RED org to host it.  Will keep you posted when this is done.  I expect we may change the name of the module.

Mike

[Mike Blackstock]

The test helper code is now on npm as 'node-red-node-test-helper'.  https://www.npmjs.com/package/node-red-node-test-helper.  This code will be maintained in a node-red project repository now.  I've deprecated the previous npm module and will remove it.

[Dean Cording]

Hi Mike,

The node-red-node-test-helper looks good but you really need to complete the documentation because it doesn't give enough information to actually be able to run a test.  In particular, it really needs some information on setting up the package.json to include devDependencies and scripts.

Dean

[Mike Blackstock]

Thanks Dean - I will work on improving the docs with that info.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/zyWkVvwoA44/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/2210c278-d607-4c6d-903e-e57d574662fd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Mike Blackstock
Chief Technical Officer
Sense Tecnic Systems, Inc.

308 East 5th Avenue, Vancouver, BC, V5T1H4
www.sensetecnic.com

phone: 604-760-9035 email: mblac...@sensetecnic.com

[Mike Blackstock]

Made some improvements to the documentation adding info on how to add the helper and other test dependencies mocha and should to the package.json.  More to come, but getting there.  Pull requests welcome of course.

[Dean Cording]

Thanks Mike,

One thing you should consider is that doing a local install of the node-red-node-test-helper also drags in over 400 other packages (totalling 63Mb) including a full node-red package and all of the core nodes.  With my twenty odd nodes, it soon adds up.

Doing a global install would be a better option, however I'm having trouble getting it to work for some reason.

Dean

[Mike Blackstock]

Thanks for trying it out Dean.  Yeah, there is a lot of stuff pulled in...  Let me know if you manage to get a global install working or you have any other suggestions.

Mike

--

http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/zyWkVvwoA44/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/dd6dfcd9-5240-49f1-ba70-7efd160fcc32%40googlegroups.com.

[Nick O'Leary]

If you use npm >=3 then you won't get multiple copies of everything. NPM works hard to install modules as high as possible in the tree where it can - rather than a huge tree of node_modules as you used to get in npm 2 and earlier. This was one of the reasons we hadn't pulled out the test helper earlier on in the projects life - but big thanks to Mike for getting this done now.

I would suggest relaxing the node-red module dependency to 0.18.x at least - otherwise that will definitely cause multiple installs each time we release a new version.

Nick

You received this message because you are subscribed to the Google Groups "Node-RED" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CABD_Zox9MattOjvpSJpLduv_yNbrX8LTDtpvhkN3hxPoeHf-sw%40mail.gmail.com.

[Dean Cording]

I'm using npm v5.7.1  The problem is that node-red is installed globally and my node development occurs in separate trees, so npm.

The solution so far has been to do:

npm install -g mocha should node-red-node-test-helper

and then 

npm link should
npm link node-red-test-helper

in the node development directory.

But I am still having some grief.

On Friday, 23 February 2018 12:26:22 UTC+10, Nick O'Leary wrote:

If you use npm >=3 then you won't get multiple copies of everything. NPM works hard to install modules as high as possible in the tree where it can - rather than a huge tree of node_modules as you used to get in npm 2 and earlier. This was one of the reasons we hadn't pulled out the test helper earlier on in the projects life - but big thanks to Mike for getting this done now.

I would suggest relaxing the node-red module dependency to 0.18.x at least - otherwise that will definitely cause multiple installs each time we release a new version.

Nick

On 22 February 2018 at 17:17, Mike Blackstock <mikebla...@gmail.com> wrote:

Thanks for trying it out Dean.  Yeah, there is a lot of stuff pulled in...  Let me know if you manage to get a global install working or you have any other suggestions.

Mike

On Feb 22, 2018 5:03 PM, "Dean Cording" <dean+...@cording.id.au> wrote:

Thanks Mike,

One thing you should consider is that doing a local install of the node-red-node-test-helper also drags in over 400 other packages (totalling 63Mb) including a full node-red package and all of the core nodes.  With my twenty odd nodes, it soon adds up.

Doing a global install would be a better option, however I'm having trouble getting it to work for some reason.

Dean

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/zyWkVvwoA44/unsubscribe.

To unsubscribe from this group and all its topics, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/dd6dfcd9-5240-49f1-ba70-7efd160fcc32%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

[Nick O'Leary]

Ah yes of course. Just about to get on a plane back to the UK. If I can't sleep, I'll have a poke around on it.

Nick

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/2552e4a1-859a-4f10-9274-a1f4d8c28acc%40googlegroups.com.

[Mike Blackstock]

I'll look at this tomorrow as well.

To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CAF%3DvhqfRrCcNJyZ3MqhYpV5BLhG1tg_ycf-1qM71FO3kqRXMGA%40mail.gmail.com.

[Dean Cording]

I've got basic tests working with a global install and using npm link.

The problem I'm having is I want to test my node throwing exceptions.  I am using this flow definition:

 var flow1 = [
    {
        id: "c1",
        type: "catch",
        name: "",
        scope: [
            "n1"
        ],
        wires: [
            ["h1"]
        ]
    },
    {
        id: "n1",
        type: "state-machine",
        name: "Test",
        triggerProperty: "topic",
        triggerPropertyType: "msg",
        stateProperty: "state",
        statePropertyType: "msg",
        outputStateChangeOnly: false,
        throwException: false,
        states: [
            "start",
            "state1",
            "state2",
            "state3"
        ],
        transitions: [
            {
                name: "t1",
                from: "start",
                to: "state1"
            },
            {
                name: "t2",
                from: "state1",
                to: "state2"
            },
            {
                name: "t3",
                from: "state2",
                to: "state3"
            },
            {
                name: "t4",
                from: "state3",
                to: "state1"
            },
            {
                name: "r",
                from: "*",
                to: "start"
            }
        ],
        wires: [
            [
                "h1"
            ]
        ]
    },
    {
        id: "h1",
        type: "helper"
    }


];

Without the catch node included, everything works fine.  With the catch node included, no nodes are created - helper.getNode("n1") returns undefined.  No errors are given (other than the test failures because the test node is undefined).  I suspect something is failing in the creation of the flow but without logging its hard to tell.

 The nodes I want to have test harnesses for tend to have complex configs that are difficult to craft by hand.  As a suggestion, can you make the helper node a real node.  That way test flows can be constructed in the editor and then it is a simple case of exporting the flow to the clipboard and pasting it in the test script. If the test script took JSON flow descriptions,that would be even better.  This would allow the interface between a node's html and js components to be tested as well.  An option may be to make helper and debug nodes the same - looking at the helper code, you could easily just include it with the debug node code.

[Dean Cording]

Never mind, got it running.

Had to:

npm link node-red

and the test script needed:

var stateMachineNode = require('./state-machine.js');
var catchNode = require('node-red/nodes/core/core/25-catch.js');

and

helper.load([stateMachineNode, catchNode], flow1, function () {

[Mike Blackstock]

Super.  Thanks again.

I wouldn't have thought of using npm link to link to installed global modules like that.  Good to know how to install core nodes that way when needed.  Will document that.

So ... just so I understand.  You include the helper in your devDependencies, but install your node using --production so you don't pull these in, and then install the big test dependencies manually (node-red) using npm link first, then npm install the rest?

How do you set things up so this is fairly automatic for you?  Can you suggest any changes to the test helper project package.json or the README to document this (I will try when I get time)?

Mike

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/zyWkVvwoA44/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/f4a74438-e475-4547-bb9c-546b76e22a76%40googlegroups.com.

[Dean Cording]

I can't take all the credit, after all it does say in the NPM documentation:

Install it locally if you're going to require() it.
Install it globally if you're going to run it on the command line.
If you need both, then install it in both places, or use npm link.

https://docs.npmjs.com/files/folders

Keep in mind that I'm developing contrib nodes,, so I don't do any core development (yet).  So this is my node development workflow:

• Node Red is installed in the global node directory - 'npm install -g node-red'
• Mocha, should, and node-red-node-test-helper are also installed in the global node directory - 'npm install -g mocha should node-red-node-test-helper'
• Each node has its own separate development directory, usually cloned from a Github repository
• I write my node's js and html file in this directory
• Production dependencies are installed locally using 'npm install -P package' to install them in the node's node_modules directory and add them to the package.json dependencies
• In the node's directory, I run 'npm link' to create a link in the global node directory
• In .node-red, I run 'node link node-name' to create a link in Node Red's node_module directory (I don't think this step is actually needed as Node Red is globally installed)
• For unit testing I use 'npm link should' and 'node link node-red-node-test-helper' to link the testing packages in my node's node_modules directory.  I can also do 'npm link node-red' to include the core nodes, or npm link for any other globally installed packages I need.
• 'npm link' doesn't have a save option so I need to add the devDependencies manually, as well as the test script names
• Develop and unit test until complete. I also have a Node Red instance for development testing so I can do integration testing.
• 'git push' and 'npm publish' to release the node package
• On my production Node Red instance, I just do a node install from the package manager in the editor.

This workflow seems to produce a simple,clean development environment and there isn't really much there to automate.

A couple of suggestions I have to make testing easier:

• Allow helper.load to be given flows in JSON as well as objects.  The nodes I really need to test have complex configurations that are difficult to hand craft.  By accepting JSON, I can create the test flows in the Node Red editor, export them, and simply paste the exported JSON into the test script.  This allows me to also test the output of the html config editor.
• Incorporate the test-helper node functionality into the core debug node.  This would allow you to create test flows in the Node Red editor using debug nodes which would automatically become test-helper nodes in the node test framework.  This might be as simple as mapping debug->helper in the helper.load function.

Dean

[Mike Blackstock]

Thanks - will work some of this into the README.

--

http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/zyWkVvwoA44/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/62c42388-8f8e-405d-b26f-53854e9c823d%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Dean Cording]

One more thing you need to do.

You need to define a flow and assign the nodes to is (z property) in your flow definition so that Node Red knows where the nodes live, otherwise things like catch and status nodes don't work.  I suspect it would also be needed for the flow context to work.

So your flow object should be:

var flow = [
      { id: "f1", type: "tab", label: "Test Flow"},
      { id: "n1", z: "f1", type: "lower-case", name: "lower-case",wires:[["n2"]] },
      { id: "n2", z: "f1", type: "helper" }
    ];

Dean

[Dean Cording]

Hmm, now I have a bit of a complication.  Possibly because I have enabled the flow,  in event handlers, such as 'on input', any errors thrown get swallowed by the flow and not propagated up to the test harness.  This means that assertion errors don't trigger test fails if they occur in the handler.

In the helper node, if you add:

    log.init({});

just before 

   logSpy = sinon.spy(log,"log");

it will enable Node Red logging output where you can see the assertion errors being caught.

However, there is a simple solution - just override the Node.prototype.error function in the helperNode so that errors aren't passed to the flow.   The error handler I used was just a simple console.log:

function helperNode(n) {
    RED.nodes.createNode(this, n);


    this.error = function(logMessage,msg) {
        console.log(logMessage);
    }
}

Dean

[Mike Blackstock]

I've published a new version of the helper 0.1.5:

- remove node-red as a dependency so it can be used for core testing, and doesn't pull in all of its dependencies (many)

- if node-red dependency cannot be found, assumes its running in core tests and attempts to use local node-red runtime from core

- update documentation to include how to link to node-red and other deps installed globally, how to import flows from editor, add flow id (tab) so that catch and status nodes work

Initially the goal is to provide a module that we know works both with contrib nodes and the core.  This version has been verified to work with the node-red core on a branch.

Thanks to Dean for his feedback and contributions so far.  More to come in future revs to make it easier to use!  If you try out the module in your tests and have any feedback or questions, let us know.

Mike