Rewrite lattice

[Alejandro Roble Gutiérrez]

I'm trying to rewrite latice on kaldi but i can't...this the situation:

    Many function for manipulate lattice (lattice-add-penalty, lattice-align-words, among others) have an in an out file...when you need that the out lattice will compress you need to specific as follow: ark:|gzip -c > $kwsdir/$file.gz....but when you don't want compress you put: ark:$file (for example)..now when the input file and the output file are the same you put: ark:|gzip -cf > $kwsdir/$file.gz (in compress case)..the -f parameter on gzip command is for rewrite if exist the file..but when I want to rewrite without compress...How I can do this?

[Shin XXX]

sorry I don't get it ... isn't it "ark:$file"? It won't do things like file existence check, therefore it will rewrite $file?

Shin

On Thu, Aug 30, 2018 at 10:46 PM Alejandro Roble Gutiérrez <aro...@gmail.com> wrote:

I'm trying to rewrite latice on kaldi but i can't...this the situation:

    Many function for manipulate lattice (lattice-add-penalty, lattice-align-words, among others) have an in an out file...when you need that the out lattice will compress you need to specific as follow: ark:|gzip -c > $kwsdir/$file.gz....but when you don't want compress you put: ark:$file (for example)..now when the input file and the output file are the same you put: ark:|gzip -cf > $kwsdir/$file.gz (in compress case)..the -f parameter on gzip command is for rewrite if exist the file..but when I want to rewrite without compress...How I can do this?

--
Go to http://kaldi-asr.org/forums.html find out how to join
---
You received this message because you are subscribed to the Google Groups "kaldi-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kaldi-help+...@googlegroups.com.
To post to this group, send email to kaldi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/ba141352-e123-4913-b56a-775f7514a228%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Povey]

For sure you can't do it. It's almost never safe, for any kind of
program, to read from and write to the same file.

> To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/CADtO4NU%3DukH5u64zWN5-s35SwmjtowRvCfY88uA4Go8K9j6PJQ%40mail.gmail.com.

[Alejandro Roble Gutiérrez]

Yeah you are ok..but if before that I loaded the file in memorie then I can erase or rewrite the original file becouse I have all information are on memory...in fact when you call gzip interface you can do it like I showed before (-f parameter)...but with ark parameters I don't know how do it!!!!!

[Daniel Povey]

You can't do it; and I expect your trick with -f does not actually
work properly in general. It probably depends on the file size; it
would likely work for small files and then fail for larger files.
On Thu, Aug 30, 2018 at 12:30 PM Alejandro Roble Gutiérrez

> To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/b80f8260-93c5-4dff-ab56-11c51bbbd7c4%40googlegroups.com.

[Shin XXX]

Got it ... You want to rewrite the input file...Sounds like a memory mapping thing, IIRC linux should have some functions/features allow you directly access a file in memory like accessing it on disk(e.g. it maps memory to a disk named "sdmxxx", and if I output data to that disk, the data will still stay in memory. And if you mv a file to that disk, all data will be loaded into the memory even the original file is deleted. However it will collapse for big files or the memory usage is high, it's not safe, you might lose the input file forever...)

Shin

To view this discussion on the web visit https://groups.google.com/d/msgid/kaldi-help/b80f8260-93c5-4dff-ab56-11c51bbbd7c4%40googlegroups.com.

[Alejandro Roble Gutiérrez]

Yeah I understand you..but all recipies on kaldi in general use the pipe ( | ) to pass a result of a function to other...I not runing the script on bash...so I want to process something (lattices) and then return the result and use this result in other funtion..for that I need rewrite...becouse after process this "thing" I don't needed any more...and if on every process I change the name of file (this alternative work very well), many file will be write by the code...I don't want this becouse is inneficient on disk space...