File System Operations Node

[Dean Cording]

I've just publish a collection of nodes to perform basic file system operations such as rename, move and delete files and directories.

Check it out at: https://www.npmjs.com/package/node-red-contrib-fs-ops


Enjoy,

Dean

[Dave C-J]

Any reason not to combine / enhance / work with @totallyinformation ? http://flows.nodered.org/node/node-red-contrib-fs

[Dean Cording]

I did look at doing that originally but node-red-contrib-fs only implemented directory listing and did it in a pretty inflexible way.  I actually needed a node to move files but it was a case of 'what the heck' and I implemented anything else I thought could be useful.

[Julian Knight]

You could have got in touch?

I did say I was happy to talk.

Not sure why the folder listing is inflexible - again, please raise an issue if you think it can be improved. I'm not a full-time dev so I'm always looking to learn.

[Julian Knight]

Oh, and the fs-copy node is started but not yet released as I've had too many other things going on.

[Dean Cording]

On Friday, 8 July 2016 04:24:21 UTC+10, Julian Knight wrote:

You could have got in touch?

I did say I was happy to talk.

No offense intended, I just needed a solution to move files being uploaded through a web page using httpInMulti. I did start by forking node-red-contrib-fs but a couple of design issues put me off extending it.  Firstly the configuration framework was unusual.  A problem with many Node Red nodes is that they all have different ways of being configured (edit screen vs different msg properties, single value properties vs config objects) and you spend most of your time doing msg manipulation gymnastics just to move data along the flow.  My personal opinion is that having static configuration in the edit screen and dynamic configuration using an object in msg.payload is poor design - it hijacks msg.payload, obscures how the node is configured, and is different to how the majority on nodes do it. 

Secondly, the design was trying to solve 100% of the requirements, when an 80% solution would have been sufficient for most needs.  Yes, I understand that the functionality was supplied by the underlying library which fs-file-lister wraps around.  I could see that the fs-copy node was looking to take the same approach. However Node Red is never going to be the best choice for an application that needs to scan through directory trees or does bulk file manipulation, so why implement the functionality to do that, particularly if it makes it harder to use for the simpler needs.  Again this is my personal opinion which is strongly influenced by the Unix philosophy of building powerful solutions by chaining together a number of simple tools that each do one job well.  My view is that Node Red should work like a visual version of Unix's pipes - if I can't understand what is going on just by looking at the flow of nodes then I might as well have just written it in Node.

So in the end I had a choice of extending a design that I didn't agree with and would require additional work to supply functionality that I didn't need just to be consistent with the existing nodes, or start from scratch with what I consider a cleaner design.  I chose the start from scratch and once I had finished the rename/move node, saw that it was a trivial exercise to implement the other basic filesystem operations using the same approach, so I did.

Anyhow, this is just my personal opinion and take it as a critique rather than criticism.  Yours is welcome as well.

Dean

[Julian Knight]

Hi Dean,

Well as others will doubtless testify, I have a terrible tendency to over-engineer! It is sadly a character flaw that I doubt I can get rid of now after 35 years of working life B-}  Personally though, I hate, having wrangled with a tool for a while, to find that I suddenly hit a brick wall with it and have to start again with another. Which is why I try to make things as flexible as possible.

I'm just in the process of reworking the main inputs to follow the newer input box drop-down selections to make it much clearer and more standard. So you will be able to easily select whether the input comes from the inbound msg, a context variable or a manual string. Personally, I recognise use cases for all of those inputs so it is important to allow for them and to do so in the same way as the core nodes. Hopefully that will deal with the issues you saw in the initial release which I fully admit was done in a rush. It should, at least, make the inputs and outputs more obvious and easier to understand.

As for node-red being the "best" at handling files - no, of course it won't. However, it could be a tool that non-programmers could use to do ETL type tasks very easily, something that I come across every day professionally so I'm keen to try and give something back to NR that might help in those circumstances. In those cases, it matters not at all that NR is not the best, only that it does the job, is accessible by non-programmers and is flexible enough to cope with a wide variety of use cases. Of course, it also needs to be robust which is why I used an add-in library rather than, as I started to do, using the core node.js libraries alone. I also needed it to work across Windows and Linux as I use both. Linux for IoT as a hobby which is how I got into NR, Windows as an IT professional. Undoubtedly, the other nodes I will create in the fs-* range will be designed to deal with the issues and cases I can see.

The feedback is important and it makes me try harder for next time. v2 of node-red-contrib-fs is on its way. Hopefully I'll be able to get it out before I go on holiday.

[ccsco...@gmail.com]

Hello All,

@Julian Knight :: I hate to bother you but just curious on the progress of the fs-copy node. Was there ever a version released?

[Julian Knight]

Hi, 'fraid not, as usual, I got distracted! I you wanted to have a go at implementing it, I'd be happy to accept a pull request.

I'm a bit tied up on something else at the moment.