Issue concatenating strings in function node

[David Hamblin]

Hi,

I'm trying to create a flow which saves Twitter data as individual files for each tweet in HDFS on Bluemix.

I have a working flow, but only where I use a fixed filename and append each new tweet to it, but owing to a known issue in HDFS itself it's not ideal to do this. According to the help on the ibm hdfs node, if no filename is set then it will take the msg.filename property and use that as the filename.

So in my function node, I entered the following:

var date = new Date();
var strDate = date.toString();
var myFileName = '/tweets/ibmTweets/' + strDate;
msg.filename = myFileName;

When I deploy this, I get no error messages, but I get no output into HDFS.

However, if I simply remove the concatenation thusly:

var myFileName = '/tweets/ibmTweets/tweetName';

It works file, and the ibm hdfs node creates a file with that fixed name and works as normal.

In thinking maybe my date function was incorrect, I tried the following:

var myFileName = '/tweets/ibmTweets/' + 'tweetName';

And that still doesn't work.

I also tried the following:

var date = new Date();
var strDate = date.toString();
var dirPath = '/tweets/ibmTweets/';
var myFileName = dirPath.concat(strDate);
msg.filename = myFileName;

And still no joy...

So I'm lost as to why a simple concatenation breaks the flow! Any thoughts?

Regards, David.

[Nicholas O'Leary]

My guess is that you're creating a filename with invalid characters in it as far as hdfs is concerned.

Rather than the toString method, which includes colons and other punctuation, I'd just use Date.now() - the epoch time in millisecs as a number.

You may want to consider the case where two tweets arrive in the same millisecond... Which would otherwise generate the same filename if you're not careful.

Nick

--
http://nodered.org
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nicholas O'Leary]

Oh, meant to say, if an invalid filename is the cause, I'll pass that on to the owner of the node as it really should warn you.

Nick

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

[David Hamblin]

Hi Nick,

That's worked fine - many thanks for your quick reply!

To avoid the multiple tweets per ms issue I've added the user screen name to the filename too, so that's a potential pitfall avoided.

Regards, David.