fs.createWritestream change path dynamically

[Thorsten Moeller]

Hi,

i am creating a writestream on a server connection event using a path join of a fix path and a variable for the filename. As the filename is not known on server connection event (sent later via connection data event) , it is empty and therefore not working afterwards, producing errors (connot open file).

Is there a way to handle this more dynamically?? Perhaps creating the stream first like an global object and later set the filename and then using the stream via something like a method???

Regards

Thorsten

[Tim Dickinson]

I dont really understand what your trying to do. You cant create a write stream without a file name. You could create a random file name and then rename it once you have the real file name.

[Luke Arduini]

@tim you're thinking of streams as just file-related things

thorsten - 

Using the new stream class you can do:

var stream = new Stream.Readable()

stream.push(null)

stream.push('your data and stuff')

when you're ready for a file

stream.pipe(fs.createWriteStream('outputfile'))

keep in mind data events wont get emitted without a listener in new streams, for better/worse depending on what you're trying to do

--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
 
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Thorsten Moeller]

Hi Tim,

thanks for the hint. Must have overread this. Googled all the time, but didn't thought about renaming as an approach.

Will try that.

Thanks

[greelgorke]

this approach is good for small data, but with larger files with many concurrent users this may eat up your memory, because the Stream buffers the data there. In this case you can just create a Buffer and write it when done. But i also prefer the tmp-file approach: no buffering and a mv is on the disk is not that expensive.

[Martin Cooper]

On Thu, Mar 7, 2013 at 6:35 AM, Thorsten Moeller <tmoe...@fastmail.fm> wrote:

Hi,

i am creating a writestream on a server connection event using a path join of a fix path and a variable for the filename. As the filename is not known on server connection event (sent later via connection data event) , it is empty and therefore not working afterwards, producing errors (connot open file).

Be *very* careful about using any filename, or portion of a filename, that was provided to you by a client. Using that as part of a filename or path on the server is a potentially major security hole, and best avoided. I'd agree with greelgorke that using a temp file on the server is a better approach. If you need to retain a mapping between uploaded data and a client filename, do that elsewhere (e.g. a database), and not by trying to use the same file system naming and / or structure as the client.

--

Martin Cooper

 

Is there a way to handle this more dynamically?? Perhaps creating the stream first like an global object and later set the filename and then using the stream via something like a method???

Regards

Thorsten

--

[Tim Dickinson]

This is to say that he is using node 0.9.x

Also as @greelgorke  said, a large file will bring up problem with keeping the data in memory.

Streaming the data to a tmp file "/tnp/random-name" is the best bet.

it removes any complexity that might come up then trying to pipe the data back into a file.

@Thorsten Moeller You should really think about getting the name of the file before you write any data to it. You will have a stronger program if you do.

[Thorsten Moeller]

Security isn't the big thing here as it would be a kind of strictly internal file distribution between a central server and some other servers in the same network. No external access here.

The problem i really have is to do some simple filecopy via socket from one server to another. These are very small files, so buffering might also not be a problem.

Both informations are present on the sending server. I managed to send the filename, but i didn't managed to send the file-data right after it.

I did not found out correctly, how to tell the receiving side that the filename is done (guess it is sending an "end") and that now the data is coming. All of this should be done via tcp net sockets.

Doing this separately in dedicated connection it works, but not with a single connection.   

Thorsten

[greelgorke]

well to do this, you have to introduce a protocol how to format the data sent over tcp. this is what http does. You could use a JSONStream https://github.com/dominictarr/JSONStream

where you just pipe the it to/from the socket and your json object could be like: {filename:'a string of file', file:myFileBuffer.toString('ensoding you like')}