FTP GET method to download a file, how to?

[Javier Andrés Fernández]

Hello,

I am new to node-red and I am wondering if you could help me with something. I have made a flow that writes a file, which is a list of files inside a FTP server. So far it is working, but what I need to do now is to parse that file line by line and use each entry as a filename to download from that FTP server. And, if possible, to save each of those downloaded files in different files  of the NR path having the original names. I really do not know how to do it, since when I specify the parameters for Filename and Local Filename in the FTP node it works, but when I leave them blank and pass the parameters by using a function, the FTP node throws an error and crashes.

This is muy flow under test:

[{"id":"443263e7.b687c4","type":"tab","label":"Flow 2"},{"id":"35ee688a.aa2948","type":"function","z":"443263e7.b687c4","name":"FTP Caller","func":"msg.localFilename='C:/Users/JAVIER/Repo-CSV/trash.csv';\nmsg.filename=msg.payload;\n//msg.localFilename=msg.localFilename + msg.payload;\n\nreturn msg;","outputs":"1","noerr":0,"x":316,"y":294,"wires":[["d0e90681.15a448"]]},{"id":"c5e5da78.20f228","type":"inject","z":"443263e7.b687c4","name":"","topic":"","payload":"","payloadType":"str","repeat":"","crontab":"","once":false,"x":86,"y":161,"wires":[["845a50ec.386668"]]},{"id":"845a50ec.386668","type":"ftp in","z":"443263e7.b687c4","ftp":"a14d8211.9f9aa","operation":"list","filename":"msg.payload","localFilename":"C:\\Users\\JAVIER\\prueba1.txt","name":"FTP Server","x":113,"y":85,"wires":[["e6b5d41b.da09b8"]]},{"id":"a85a7abb.d20ea8","type":"filter","z":"443263e7.b687c4","name":"Filter","property":"payload.name","propertyType":"msg","asArray":false,"itemProperty":"payload","itemPropertyType":"item","rules":[{"t":"regex","v":"VERICOM-Nuevo_Vericom-","vt":"str","case":false,"output":1}],"checkall":"true","outputs":1,"x":393,"y":117,"wires":[["bf66aea5.39dc9"]]},{"id":"e6b5d41b.da09b8","type":"split","z":"443263e7.b687c4","name":"Split","splt":"\\n","x":321,"y":27,"wires":[["a85a7abb.d20ea8"]]},{"id":"bf66aea5.39dc9","type":"function","z":"443263e7.b687c4","name":"","func":"msg.payload=msg.payload.name;\nreturn msg;","outputs":1,"noerr":0,"x":548,"y":89,"wires":[["4615c820.be2f4"]]},{"id":"4615c820.be2f4","type":"file","z":"443263e7.b687c4","name":"Archivo","filename":"C:/Users/JAVIER/listFTP.txt","appendNewline":true,"createDir":true,"overwriteFile":"false","x":671,"y":167,"wires":[]},{"id":"585b9541.54824c","type":"file in","z":"443263e7.b687c4","name":"Reader","filename":"C:/Users/JAVIER/listFTP.txt","format":"utf8","x":103,"y":262,"wires":[["10517c6.da57104"]]},{"id":"21ecb5b8.bda75a","type":"inject","z":"443263e7.b687c4","name":"","topic":"","payload":"","payloadType":"str","repeat":"","crontab":"","once":false,"x":105,"y":479,"wires":[["585b9541.54824c"]]},{"id":"10517c6.da57104","type":"split","z":"443263e7.b687c4","name":"","splt":"\\n","x":297,"y":248,"wires":[["35ee688a.aa2948"]]},{"id":"64641c67.a3b50c","type":"ftp in","z":"443263e7.b687c4","ftp":"a14d8211.9f9aa","operation":"get","filename":"","localFilename":"","name":"FTP server","x":315,"y":385,"wires":[[]]},{"id":"d0e90681.15a448","type":"delay","z":"443263e7.b687c4","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"5","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":325.08331298828125,"y":336.16668701171875,"wires":[["64641c67.a3b50c"]]},{"id":"a14d8211.9f9aa","type":"ftp","z":"","host":"FTPSERVERURL","port":"","secureOptions":"","user":"","connTimeout":"","pasvTimeout":"","keepalive":""}]


Thanks,

Javier

[Zenofmud]

What node is the filter node?

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to the Google Groups "Node-RED" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/4df29b5a-57d7-4ab1-9241-30d36cf18238%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Javier Andrés Fernández]

Hi, thanks, it is this one:

node-red-contrib-filter 0.1.3

It helps me parse all the file names in the FTP list and only consider those whose names respond to the specified regex.

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/4df29b5a-57d7-4ab1-9241-30d36cf18238%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---

You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/dqGSNlrXfGU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/32369A57-0933-471A-B1AB-A2C6048257AF%40zenofmud.org.

[Zenofmud]

sorry, I should have asked which FTP node too

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CA%2BuVoP3OmmyUwYX67_qNWJxjM_OoFDdD7a%2BVgyrENbKGfMYT_A%40mail.gmail.com.

[Zenofmud]

Javier,

Try this, change your function node in the second half to

// skip if the payload is empty

if (msg.payload === "") return null;

// set the local file’s path and name you will write to

msg.localFilename="/Users/Paul/tmp/" + msg.payload;

// set the name of the file to get

msg.filename=msg.payload;

return msg;

Attached is the full flow but note that I changed the file path’s (I’m using a mac) and I changed the Filter node to just grab a couple files by name.

Paul

[{"id":"e23007aa.6c79d","type":"function","z":"2572e0bf.e86368","name":"FTP Caller","func":"// skip if the payload is empty\nif (msg.payload === \"\") return null;\n\n// set the local file’s path and name you will write to\nmsg.localFilename=\"/Users/Paul/tmp/\" + msg.payload;\n\n// set the name of the file to get\nmsg.filename=msg.payload;\n\nreturn msg;","outputs":"1","noerr":0,"x":229,"y":338,"wires":[["7b8d08ad.ce45c"]]},{"id":"24ac686e.1ce008","type":"inject","z":"2572e0bf.e86368","name":"","topic":"","payload":"","payloadType":"str","repeat":"","crontab":"","once":false,"x":100,"y":63,"wires":[["a620efad.ddd5b8"]]},{"id":"78155c8.32054a4","type":"split","z":"2572e0bf.e86368","name":"Split","splt":"\\n","spltType":"str","arraySplt":"1","arraySpltType":"len","stream":false,"addname":"","x":275,"y":116,"wires":[["b7818ffe.c55b"]]},{"id":"966b496e.9c38a","type":"function","z":"2572e0bf.e86368","name":"point to listFTP","func":"msg.payload = msg.payload.name;\nmsg.filename = \"/Users/Paul/tmp/listFTP.txt\";\nreturn msg;","outputs":1,"noerr":0,"x":437,"y":212,"wires":[["3339024b.5564b6"]]},{"id":"3339024b.5564b6","type":"file","z":"2572e0bf.e86368","name":"Archivo","filename":"/Users/Paul/tmp/listFTP.txt","appendNewline":true,"createDir":false,"overwriteFile":"false","x":488,"y":260,"wires":[]},{"id":"596f7166.da7618","type":"file in","z":"2572e0bf.e86368","name":"Reader","filename":"/Users/Paul/tmp/listFTP.txt","format":"utf8","sendError":true,"x":140,"y":235,"wires":[["cd262b0e.863088"]]},{"id":"4c60bd81.14f9d4","type":"inject","z":"2572e0bf.e86368","name":"","topic":"","payload":"","payloadType":"str","repeat":"","crontab":"","once":false,"x":94,"y":184,"wires":[["596f7166.da7618"]]},{"id":"cd262b0e.863088","type":"split","z":"2572e0bf.e86368","name":"","splt":"\\n","spltType":"str","arraySplt":"1","arraySpltType":"len","stream":false,"addname":"","x":170,"y":287,"wires":[["e23007aa.6c79d"]]},{"id":"c0db9f64.a4316","type":"ftp in","z":"2572e0bf.e86368","ftp":"65be10e5.70fe8","operation":"get","filename":"","localFilename":"","name":"FTP server","x":332,"y":439,"wires":[["84fdefb6.783a98"]]},{"id":"7b8d08ad.ce45c","type":"delay","z":"2572e0bf.e86368","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"5","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":288.08331298828125,"y":384.16668701171875,"wires":[["c0db9f64.a4316"]]},{"id":"84fdefb6.783a98","type":"debug","z":"2572e0bf.e86368","name":"","active":true,"console":"false","complete":"true","x":484,"y":440,"wires":[]},{"id":"a620efad.ddd5b8","type":"ftp in","z":"2572e0bf.e86368","ftp":"65be10e5.70fe8","operation":"list","filename":"","localFilename":"","name":"FTP List","x":235,"y":63,"wires":[["78155c8.32054a4"]]},{"id":"b7818ffe.c55b","type":"filter","z":"2572e0bf.e86368","name":"Filter","property":"payload[\"name\"]","propertyType":"msg","asArray":false,"itemProperty":"payload","itemPropertyType":"item","rules":[{"t":"eq","v":"iwasput-1.txt","vt":"str","output":1},{"t":"eq","v":"iwasput-2.txt","vt":"str","output":1},{"t":"eq","v":"iwasput-3.txt","vt":"str","output":1}],"checkall":"true","outputs":1,"x":356,"y":164,"wires":[["966b496e.9c38a"]]},{"id":"65be10e5.70fe8","type":"ftp","z":"","host":"zenofmud.org","port":"","secureOptions":"","user":"zenofmud","connTimeout":"","pasvTimeout":"","keepalive":""}]

[Javier Andrés Fernández]

Thanks Paul for your help!

I changed the code in the function node of the second part of the flow as you pointed but, unfortunately, I keep receiveing the same message. From the debug node:

4/8/2017 10:15:17FTP servermsg : string [87]Error: Can't open VERI_New_Veri-20170207-105502.csv_: No such file or directory

However, I now noticed something, in the name of the file, at the end there is an extra character "_" which is not part of the name of the file. And this character does not appear if I put a debug node at the output of  my "FTP Caller" function. I do not know which node, nor why this extra cahracter comes up, but it will certainly throw an error if I try to get a file by that name.

Before I tried to design this flow, I tried to specify the name of the file to get in the FTP node and it works, but in this case the debug information, given in the console, does not show that extra "_".

I don´t know what else to try.

Regards,

Javier

 

--

http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/dqGSNlrXfGU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/AF466ED2-415E-414B-B308-D299C9E0E1C2%40zenofmud.org.

[Zenofmud]

What are the file names in your ‘listFTP.txt’ file? Is the underscore in there? If not, put separate (helps to see which debug msg comes from where) debug node (output the complete msg object) on each of the ‘reader’, ’split’, and ‘FTP Caller’ nodes so you can check the file names

 

You received this message because you are subscribed to the Google Groups "Node-RED" group.
To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CA%2BuVoP2DUPsSspGchJRVEmy2m%2BMnqcngcSuK8QTJQifUV5%2Bi9A%40mail.gmail.com.

[Javier Andrés Fernández]

Hi Paul,

I know it is not right but I worked it around by modifying the FTP Caller function with "msg.filename=msg.payload.trim('_');" instead of
"msg.filename=msg.payload;". But I will do what you suggest in order to see where this underscore comes from, it certainly does not come from the listFTP file.

But, there are 2 extra problems now:

1- I manage to download the first file, but when I have to download the second file NR crashes and I get this from the console: 

4 Aug 10:54:31 - [red] Uncaught Exception:
4 Aug 10:54:31 - TypeError: Cannot read property 'once' of undefined
    at sendMsg (C:\Users\JAVIER\node_modules\node-red-contrib-ftp\ftp.js:66:19)
    at C:\Users\JAVIER\node_modules\ftp\lib\connection.js:528:14
    at Object.cb (C:\Users\JAVIER\node_modules\ftp\lib\connection.js:881:11)
    at Parser.<anonymous> (C:\Users\JAVIER\node_modules\ftp\lib\connection.js:11
7:20)
    at emitTwo (events.js:106:13)
    at Parser.emit (events.js:191:7)
    at Parser._write (C:\Users\JAVIER\node_modules\ftp\lib\parser.js:59:10)
    at doWrite (_stream_writable.js:334:12)
    at writeOrBuffer (_stream_writable.js:320:5)
    at Parser.Writable.write (_stream_writable.js:247:11)
    at Socket.ondata (C:\Users\JAVIER\node_modules\ftp\lib\connection.js:273:20)

    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at TCP.onread (net.js:548:20)

2- The localFilename I used for the tests is one fixed file that I created before "filling" it with data with the FTP get method. The thing is that I really need to create this files "on the fly" and the name of each file has to be the same as the name of the file being downloaded from the FTP. I need some combination of nodes to create each file at a time, but I really do not know how to do it.

Thanks again for your help,

Javier

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CA%2BuVoP2DUPsSspGchJRVEmy2m%2BMnqcngcSuK8QTJQifUV5%2Bi9A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/dqGSNlrXfGU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/2F256E1D-D324-489E-9801-22ABB3E08649%40zenofmud.org.

[Zenofmud]

Take it one step at a time and I would start with the GET side. just put 3 file names in the listFTP.txt and see what happens.

Once you can download the three files, then move on to the GET side to create the ‘listFTP.txt’.

I have to go out for about 3 hours now but I’ll check in when I get back. If you are still having issues, attach your current flow for me to look at.

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CA%2BuVoP0QR6oO-X6MreCZgvTK7OJM1T0XVvBkMQjTPym3OQJ%3DcQ%40mail.gmail.com.

[Javier Andrés Fernández]

Thanks Paul,

I modified listFTP file to just have only 2 entries this time. The result is the same, NR crashes and outputs the same information to the console. This is the flow I´m workinng with, just the part of the FTP GET.

[{"id":"35ee688a.aa2948","type":"function","z":"443263e7.b687c4","name":"FTP Caller","func":"var i=1;\n// skip if the payload is empty\nif (msg.payload === \"\") return null;\n\n// set the local file’s path and name you will write to\nmsg.localFilename=\"/Users/JAVIER/Repo-CSV/trash_\"+i+\".csv\"; //msg.payload;\n// set the name of the file to get\nmsg.filename=msg.payload.trim('_');\n\nreturn msg;","outputs":"1","noerr":0,"x":316,"y":294,"wires":[["d0e90681.15a448"]]},{"id":"585b9541.54824c","type":"file in","z":"443263e7.b687c4","name":"Reader","filename":"C:/Users/JAVIER/listFTP.txt","format":"utf8","x":103,"y":262,"wires":[["10517c6.da57104"]]},{"id":"21ecb5b8.bda75a","type":"inject","z":"443263e7.b687c4","name":"","topic":"","payload":"","payloadType":"str","repeat":"","crontab":"","once":false,"x":115,"y":387.0000305175781,"wires":[["585b9541.54824c"]]},{"id":"10517c6.da57104","type":"split","z":"443263e7.b687c4","name":"","splt":"\\n","x":297,"y":248,"wires":[["35ee688a.aa2948"]]},{"id":"64641c67.a3b50c","type":"ftp in","z":"443263e7.b687c4","ftp":"a14d8211.9f9aa","operation":"get","filename":"","localFilename":"","name":"FTP server","x":315,"y":385,"wires":[[]]},{"id":"d0e90681.15a448","type":"delay","z":"443263e7.b687c4","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"5","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":325.08331298828125,"y":336.16668701171875,"wires":[["64641c67.a3b50c"]]},{"id":"a14d8211.9f9aa","type":"ftp","z":"","host":"FTPSERVERURL","port":"","secureOptions":"","user":"","connTimeout":"","pasvTimeout":"","keepalive":""}]

Regards,

Javier

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CA%2BuVoP0QR6oO-X6MreCZgvTK7OJM1T0XVvBkMQjTPym3OQJ%3DcQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/dqGSNlrXfGU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/66EBA5DA-A543-490F-8DAB-1F9D39C04294%40zenofmud.org.

[Zenofmud]

Javier,

Put a debug node on the Reader, split and FTP Caller and the FTP Server nodes and run it and past the debug trace. Make sure to make all the debug nodes display the complete object and set it to display to ‘debug tab and console’ so you can copy it easily from the console log.

Paste that into a reply.

[Javier Andrés Fernández]

Hi Paul,

This is the output from the console:

C:\Users\JAVIER>node-red start


Welcome to Node-RED
===================

4 Aug 15:05:14 - [info] Node-RED version: v0.15.2
4 Aug 15:05:14 - [info] Node.js  version: v6.9.2
4 Aug 15:05:14 - [info] Windows_NT 6.2.9200 x64 LE
4 Aug 15:05:15 - [info] Loading palette nodes
4 Aug 15:05:18 - [info] Dashboard version 2.2.1 started at /ui
4 Aug 15:05:19 - [warn] ------------------------------------------------------
4 Aug 15:05:19 - [warn] [rpi-gpio] Info : Ignoring Raspberry Pi specific node
4 Aug 15:05:19 - [warn] [tail] Not currently supported on Windows.
4 Aug 15:05:19 - [warn] [SUSI-HardwareMonitor] Error: No se puede encontrar el m
ódulo especificado.
\\?\C:\Users\JAVIER\AppData\Roaming\npm\node_modules\node-red-contrib-susi\node_
modules\node-susi\nodes\0.0.9\win32\x64\node_SUSI.node
4 Aug 15:05:19 - [warn] [SUSI-GPIO] Error: error: 126\\?\C:\Users\JAVIER\AppData
\Roaming\npm\node_modules\node-red-contrib-susi\node_modules\node-susi\nodes\0.0
.9\win32\x64\node_SUSI.node
4 Aug 15:05:19 - [warn] [SUSI-Backlight] Error: error: 126\\?\C:\Users\JAVIER\Ap
pData\Roaming\npm\node_modules\node-red-contrib-susi\node_modules\node-susi\node
s\0.0.9\win32\x64\node_SUSI.node
4 Aug 15:05:19 - [warn] [SUSI-WatchDog] Error: error: 126\\?\C:\Users\JAVIER\App
Data\Roaming\npm\node_modules\node-red-contrib-susi\node_modules\node-susi\nodes
\0.0.9\win32\x64\node_SUSI.node
4 Aug 15:05:19 - [warn] [SUSIIoT-Info] Error: error: 126\\?\C:\Users\JAVIER\AppD
ata\Roaming\npm\node_modules\node-red-contrib-susi\node_modules\node-susiiot\nod
es\0.0.8\win32\x64\node_SUSIIoT.node
4 Aug 15:05:19 - [warn] [SUSIIoT-Data] Error: error: 126\\?\C:\Users\JAVIER\AppD
ata\Roaming\npm\node_modules\node-red-contrib-susi\node_modules\node-susiiot\nod
es\0.0.8\win32\x64\node_SUSIIoT.node
4 Aug 15:05:19 - [warn] [SUSIIoT-Control] Error: error: 126\\?\C:\Users\JAVIER\A
ppData\Roaming\npm\node_modules\node-red-contrib-susi\node_modules\node-susiiot\
nodes\0.0.8\win32\x64\node_SUSIIoT.node
4 Aug 15:05:19 - [warn] ------------------------------------------------------
4 Aug 15:05:19 - [info] Settings file  : \Users\JAVIER\.node-red\settings.js
4 Aug 15:05:19 - [info] User directory : \Users\JAVIER\.node-red
4 Aug 15:05:19 - [info] Flows file     : \Users\JAVIER\.node-red\start
4 Aug 15:05:19 - [info] Server now running at http://127.0.0.1:1880/
4 Aug 15:05:19 - [info] Starting flows
4 Aug 15:05:19 - [info] Started flows
4 Aug 15:05:24 - [info] [debug:9dc5c9ea.4538e8]
{ _msgid: 'd54b29d7.f148c8',
  topic: '',
  payload: 'VERICOM-Nuevo_Vericom-20170208-115059.csv\r',
  filename: 'VERICOM-Nuevo_Vericom-20170208-115059.csv',
  parts:
   { id: 'd54b29d7.f148c8',
     ch: '\n',
     type: 'string',
     index: 0,
     count: 2 },
  localFilename: '\\Users\\JAVIER\\Repo-CSV\\temp.csv' }
4 Aug 15:05:24 - [info] [debug:dfa9ab9d.90a5e8]
{ _msgid: 'd54b29d7.f148c8',
  topic: '',
  payload: 'VERICOM-Nuevo_Vericom-20170208-115059.csv\r',
  filename: 'C:/Users/JAVIER/listFTP.txt',
  parts:
   { id: 'd54b29d7.f148c8',
     ch: '\n',
     type: 'string',
     index: 0,
     count: 2 } }
4 Aug 15:05:24 - [info] [debug:9dc5c9ea.4538e8]
{ _msgid: 'd54b29d7.f148c8',
  topic: '',
  payload: 'VERICOM-Nuevo_Vericom-20170207-105502.csv',
  filename: 'VERICOM-Nuevo_Vericom-20170207-105502.csv',
  parts:
   { id: 'd54b29d7.f148c8',
     ch: '\n',
     type: 'string',
     index: 1,
     count: 2 },
  localFilename: '\\Users\\JAVIER\\Repo-CSV\\temp.csv' }
4 Aug 15:05:24 - [info] [debug:dfa9ab9d.90a5e8]
{ _msgid: 'd54b29d7.f148c8',
  topic: '',
  payload: 'VERICOM-Nuevo_Vericom-20170207-105502.csv',
  filename: 'C:/Users/JAVIER/listFTP.txt',
  parts:
   { id: 'd54b29d7.f148c8',
     ch: '\n',
     type: 'string',
     index: 1,
     count: 2 } }
4 Aug 15:05:24 - [info] [debug:e2730ff3.76f0c]
{ _msgid: 'd54b29d7.f148c8',
  topic: '',
  payload: 'VERICOM-Nuevo_Vericom-20170208-115059.csv\r\nVERICOM-Nuevo_Vericom-2
0170207-105502.csv',
  filename: 'C:/Users/JAVIER/listFTP.txt' }
4 Aug 15:05:24 - [info] [debug:b8f3c2dd.023f98]
{ _msgid: 'd54b29d7.f148c8',
  topic: '',
  payload: 'Get operation successful. \\Users\\JAVIER\\Repo-CSV\\temp.csv',
  filename: 'VERICOM-Nuevo_Vericom-20170208-115059.csv',
  parts:
   { id: 'd54b29d7.f148c8',
     ch: '\n',
     type: 'string',
     index: 0,
     count: 2 },
  localFilename: '\\Users\\JAVIER\\Repo-CSV\\temp.csv' }
4 Aug 15:05:29 - [info] [debug:b8f3c2dd.023f98]
{ _msgid: 'd54b29d7.f148c8',
  topic: '',
  payload: 'Get operation successful. \\Users\\JAVIER\\Repo-CSV\\temp.csv',
  filename: 'VERICOM-Nuevo_Vericom-20170207-105502.csv',
  parts:
   { id: 'd54b29d7.f148c8',
     ch: '\n',
     type: 'string',
     index: 1,
     count: 2 },
  localFilename: '\\Users\\JAVIER\\Repo-CSV\\temp.csv' }
4 Aug 15:05:29 - [info] [debug:b8f3c2dd.023f98]
{ _msgid: 'd54b29d7.f148c8',
  topic: '',
  payload: 'Get operation successful. \\Users\\JAVIER\\Repo-CSV\\temp.csv',
  filename: 'VERICOM-Nuevo_Vericom-20170207-105502.csv',
  parts:
   { id: 'd54b29d7.f148c8',
     ch: '\n',
     type: 'string',
     index: 1,
     count: 2 },
  localFilename: '\\Users\\JAVIER\\Repo-CSV\\temp.csv' }
4 Aug 15:05:29 - [error] [ftp in:FTP server] Error: Unable to make data connecti
on
4 Aug 15:05:29 - [red] Uncaught Exception:
4 Aug 15:05:29 - TypeError: Cannot read property 'once' of undefined

    at sendMsg (C:\Users\JAVIER\node_modules\node-red-contrib-ftp\ftp.js:66:19)
    at C:\Users\JAVIER\node_modules\ftp\lib\connection.js:528:14
    at Object.cb (C:\Users\JAVIER\node_modules\ftp\lib\connection.js:881:11)
    at Parser.<anonymous> (C:\Users\JAVIER\node_modules\ftp\lib\connection.js:11
7:20)
    at emitTwo (events.js:106:13)
    at Parser.emit (events.js:191:7)
    at Parser._write (C:\Users\JAVIER\node_modules\ftp\lib\parser.js:59:10)
    at doWrite (_stream_writable.js:334:12)
    at writeOrBuffer (_stream_writable.js:320:5)
    at Parser.Writable.write (_stream_writable.js:247:11)
    at Socket.ondata (C:\Users\JAVIER\node_modules\ftp\lib\connection.js:273:20)

    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at TCP.onread (net.js:548:20)

Notes:

The listFTP file had only 2 entries during the tests. When I tested with only one entry, it worked OK, only if the localFilename is passed as:

msg.localFilename='\\Users\\JAVIER\\Repo-CSV\\'+ 'temp.csv';

in the FTP Caller function, even if the local file is not created before, it creates it ok.

If the localFilename is passed like:

msg.localFilename='\\Users\\JAVIER\\Repo-CSV\\'+ msg.payload';

It outputs an error, but, as you say, let´s go a step at a time.

Regards,

Javier

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/dqGSNlrXfGU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/101D0BB4-4E7F-4374-90F7-5A4B94166D3C%40zenofmud.org.

[Zenofmud]

Javier

You say this line doesn’t work:

msg.localFilename='\\Users\\JAVIER\\Repo-CSV\\'+ msg.payload’;

is that exactly how you have it coded because there is an extra single quote at the end. It should be

msg.localFilename='\\Users\\JAVIER\\Repo-CSV\\' + msg.payload; <== note the lat quote removed

Also (not being a windows guy) why the double slashes?

Paul

[Zenofmud]

Ok, I think I know what is happening.

If the file you are reading ‘listFTP.txt’ has a blank line at the end, it runs the FTP get twice and that is when the crash occurs. 

Try removing the black line at the end and see what happens.

--
http://nodered.org

Join us on Slack to continue the conversation: http://nodered.org/slack
---

You received this message because you are subscribed to the Google Groups "Node-RED" group.

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.
To post to this group, send an email to node...@googlegroups.com.

Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/4760185E-2EC1-4F51-AA4A-3C059A133CD0%40zenofmud.org.

[Zenofmud]

Oops, never mind, that’s not it….sigh

To post to this group, send email to node...@googlegroups.com.

Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/0A74C197-5BF6-4A82-B410-A8F706355AB5%40zenofmud.org.

[Zenofmud]

Uhh…headship…this is the same issue that I discovered two weeks ago. There is a bug (reported) in the ftp node. If you want to fix it
1) go to the .node-red/node_modules/node-red-contrib-ftp folder
2) edit the file ‘ftp.js’
3) move line 54

if (this.ftpConfig) {
var conn = new ftp(); <<==== this is line 54
var node = this;
node.on('input', function (msg) {
var filename = node.filename || msg.filename || '';
var localFilename = node.localFilename || msg.localFilename || '';
this.sendMsg = function (err, result) {

to this location

if (this.ftpConfig) {
var node = this;
node.on('input', function (msg) {
var conn = new ftp(); <<==== new location of old line 54
var filename = node.filename || msg.filename || '';
var localFilename = node.localFilename || msg.localFilename || '';
this.sendMsg = function (err, result) {

NOTE: do not include the '<<==== new location of old line 54’

4) save the file
5) restart NR and test

> To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/E9CD63E1-4BA2-4A73-9F59-6A607A2134E6%40zenofmud.org.

[Javier Andrés Fernández]

HI Paul,

Yes, I noticed that and changed it..with regards to the double slashes, is the same to put them as single inverted slashes as in Linux. I found that in the Split node I was splitting by "/n", so there was always an extra '/r' in the strings that were making trouble. So I began splitting g by "/r/n" and it worked. But only 2 or 3 filed were actually downloaded, since I am still getting that "Unable to connect" and the NR crash after 2 or 3 consecutive downloads. I also tried spacing the download attempts with a higher delay, from 5 I changed to 10 seconds between downloads, but it's the same.

I think it's a kind of bug in the Ftp node, but not so sure.

The current NR installation I am working with is at my office, so as soon as I get there on Monday I will paste here the current flow.

Have a great weekend!

Javier 

--
http://nodered.org

Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/dqGSNlrXfGU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send an email to node...@googlegroups.com.

Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/4760185E-2EC1-4F51-AA4A-3C059A133CD0%40zenofmud.org.

[Javier Andrés Fernández]

Hi Paul,

This is the flow up tp date!

[{"id":"35ee688a.aa2948","type":"function","z":"443263e7.b687c4","name":"FTP Caller","func":"// skip if the payload is empty\nif (msg.payload === \"\") return null;\n\n// set the local file’s path and name you will write to\nmsg.localFilename='c:/Users/JAVIER/Repo-CSV/'+ msg.payload;\n// set the name of the file to get\nmsg.filename=msg.payload.trim('_');\n\nreturn msg;","outputs":"1","noerr":0,"x":334,"y":339,"wires":[["d0e90681.15a448"]]},{"id":"585b9541.54824c","type":"file in","z":"443263e7.b687c4","name":"Reader","filename":"C:/Users/JAVIER/listFTP.txt","format":"utf8","x":103,"y":262,"wires":[["10517c6.da57104"]]},{"id":"21ecb5b8.bda75a","type":"inject","z":"443263e7.b687c4","name":"","topic":"","payload":"","payloadType":"str","repeat":"","crontab":"","once":false,"x":115,"y":387.0000305175781,"wires":[["585b9541.54824c"]]},{"id":"10517c6.da57104","type":"split","z":"443263e7.b687c4","name":"","splt":"\\r\\n","x":315,"y":294,"wires":[["35ee688a.aa2948"]]},{"id":"64641c67.a3b50c","type":"ftp in","z":"443263e7.b687c4","ftp":"48790697.0d17c","operation":"get","filename":"","localFilename":"","name":"FTP server","x":333,"y":431,"wires":[["b8f3c2dd.023f98"]]},{"id":"d0e90681.15a448","type":"delay","z":"443263e7.b687c4","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"10","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":329.08331298828125,"y":387.16668701171875,"wires":[["64641c67.a3b50c"]]},{"id":"b8f3c2dd.023f98","type":"debug","z":"443263e7.b687c4","name":"","active":true,"console":"true","complete":"true","x":540.0833129882812,"y":321.0833435058594,"wires":[]},{"id":"48790697.0d17c","type":"ftp","z":"","host":"www.ingspitzer.com.ar","port":"","secureOptions":"","user":"ingspitzer.com.ar_veribox","connTimeout":"","pasvTimeout":"","keepalive":""}]

And this is the debug output from the console:


7 Aug 11:59:49 - [info] [debug:b8f3c2dd.023f98]
{ _msgid: '92b15991.de2e38',
  topic: '',
  payload: 'Get operation successful. c:/Users/JAVIER/Repo-CSV/VERICOM-Nuevo_Ver

icom-20170207-105502.csv',
  filename: 'VERICOM-Nuevo_Vericom-20170207-105502.csv',
  parts:

   { id: '92b15991.de2e38',
     ch: '\r\n',

     type: 'string',
     index: 0,

     count: 13 },
  localFilename: 'c:/Users/JAVIER/Repo-CSV/VERICOM-Nuevo_Vericom-20170207-105502
.csv' }
7 Aug 11:59:59 - [info] [debug:b8f3c2dd.023f98]
{ _msgid: '92b15991.de2e38',
  topic: '',
  payload: 'Get operation successful. c:/Users/JAVIER/Repo-CSV/VERICOM-Nuevo_Ver
icom-20170208-115059.csv',
  filename: 'VERICOM-Nuevo_Vericom-20170208-115059.csv',
  parts:
   { id: '92b15991.de2e38',
     ch: '\r\n',

     type: 'string',
     index: 1,

     count: 13 },
  localFilename: 'c:/Users/JAVIER/Repo-CSV/VERICOM-Nuevo_Vericom-20170208-115059
.csv' }
7 Aug 11:59:59 - [info] [debug:b8f3c2dd.023f98]
{ _msgid: '92b15991.de2e38',
  topic: '',
  payload: 'Get operation successful. c:/Users/JAVIER/Repo-CSV/VERICOM-Nuevo_Ver
icom-20170208-115059.csv',
  filename: 'VERICOM-Nuevo_Vericom-20170208-115059.csv',
  parts:
   { id: '92b15991.de2e38',
     ch: '\r\n',

     type: 'string',
     index: 1,

     count: 13 },
  localFilename: 'c:/Users/JAVIER/Repo-CSV/VERICOM-Nuevo_Vericom-20170208-115059
.csv' }
7 Aug 11:59:59 - [error] [ftp in:FTP server] Error: Unable to make data connecti
on
7 Aug 11:59:59 - [red] Uncaught Exception:
7 Aug 11:59:59 - TypeError: Cannot read property 'once' of undefined

    at sendMsg (C:\Users\JAVIER\node_modules\node-red-contrib-ftp\ftp.js:66:19)
    at C:\Users\JAVIER\node_modules\ftp\lib\connection.js:528:14
    at Object.cb (C:\Users\JAVIER\node_modules\ftp\lib\connection.js:881:11)
    at Parser.<anonymous> (C:\Users\JAVIER\node_modules\ftp\lib\connection.js:11
7:20)
    at emitTwo (events.js:106:13)
    at Parser.emit (events.js:191:7)
    at Parser._write (C:\Users\JAVIER\node_modules\ftp\lib\parser.js:59:10)
    at doWrite (_stream_writable.js:334:12)
    at writeOrBuffer (_stream_writable.js:320:5)
    at Parser.Writable.write (_stream_writable.js:247:11)
    at Socket.ondata (C:\Users\JAVIER\node_modules\ftp\lib\connection.js:273:20)

    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at TCP.onread (net.js:548:20)

Am I doing something wrong with the configuration of the FTP? I can download up to 3 of the aproximately 12 files that are in the FTP server, but then it crashes.

Best regards,

Javier

[Zenofmud]

Did you apply the fix to ftp.js that I mentioned in a previous post?

You received this message because you are subscribed to the Google Groups "Node-RED" group.

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.

Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CA%2BuVoP3RtvoN-j2c_0Vu34zVshmyF06%3DbCqAh7WJwppNFWbxFg%40mail.gmail.com.

[Javier Andrés Fernández]

No Paul, I really didn´t know there was a fix for ftp.js. Could you please send me a link to that post? So as I can try to fix it and then tell you about the results.

Best regards,

Javier

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.
To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CA%2BuVoP3RtvoN-j2c_0Vu34zVshmyF06%3DbCqAh7WJwppNFWbxFg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/dqGSNlrXfGU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.

Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/37CE7BAD-BA6A-4D05-9054-4CA33259A2A9%40zenofmud.org.

[Zenofmud]

It’ part of this thread…didn’t you read the other messages I sent?  I send it Saturday - here is a link to this whole discussion https://groups.google.com/forum/#!msg/node-red/dqGSNlrXfGU/QzolDaG8AAAJ

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+u...@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CA%2BuVoP2r%3DhEdg7Y%3D4VubVodrn3Sm6fGFOzAoXVomtkgEELNOCQ%40mail.gmail.com.

[Javier Andrés Fernández]

Hi Paul, no, I didn´t even get to know that you wrote more messages. I hadn´t receive them I guess, sorry for that. I will read them all now.

Best regards,

Javier

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CA%2BuVoP2r%3DhEdg7Y%3D4VubVodrn3Sm6fGFOzAoXVomtkgEELNOCQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/dqGSNlrXfGU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/E002AF21-CA0A-4FDD-ADBC-EA2F68BF6FB4%40zenofmud.org.

[Javier Andrés Fernández]

Hi Paul,

Success!!! I did what you adviced and it´s now working smoothly. I really can´t thank you enough. Thanks for taking the time to help me with this!!

Best regards,

Javier

2017-08-07 13:30 GMT-03:00 Zenofmud <zeno...@zenofmud.org>:

To unsubscribe from this group and stop receiving emails from it, send an email to node-red+unsubscribe@googlegroups.com.

To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/CA%2BuVoP2r%3DhEdg7Y%3D4VubVodrn3Sm6fGFOzAoXVomtkgEELNOCQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
http://nodered.org
 
Join us on Slack to continue the conversation: http://nodered.org/slack
---
You received this message because you are subscribed to a topic in the Google Groups "Node-RED" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/node-red/dqGSNlrXfGU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to node-red+unsubscribe@googlegroups.com.
To post to this group, send email to node...@googlegroups.com.
Visit this group at https://groups.google.com/group/node-red.

To view this discussion on the web, visit https://groups.google.com/d/msgid/node-red/E002AF21-CA0A-4FDD-ADBC-EA2F68BF6FB4%40zenofmud.org.