Order of execution

[mette Joergensen]

hi

Are lines in ngless scripts executed sequentially or are some line executed in parallel to optimize things ?

I'm asking because I have implemented a function in an external module that depends on some of the lines before to finish. The function check if the output file from a previously collect call exist and does nothing if it doesn't. In that way it was my hope that the function would be doing its job when the last sample was processed. It works fine if I have less than 10 samples or are using subsample, but nothing is done (nothing is written to the output file) if I have more than 10 large samples. I was therefore speculating if the lines were ran in parallel and the output from the collect not finished when the next line was called. I have a flock around the part of the function that writes to the output file so only one process at a time should be writing to the output.

Is there a better way to mimic the functionality of collect in functions from external modules ?

Best,

Mette

   

[Luis Pedro Coelho]

Are lines in ngless scripts executed sequentially or are some line executed in parallel to optimize things ?

I'm asking because I have implemented a function in an external module that depends on some of the lines before to finish.

Sometimes things are done in parallel, but this is not done for external modules. External module calls are always synchronous.

The function check if the output file from a previously collect call exist and does nothing if it doesn't. In that way it was my hope that the function would be doing its job when the last sample was processed. It works fine if I have less than 10 samples or are using subsample, but nothing is done (nothing is written to the output file) if I have more than 10 large samples. I was therefore speculating if the lines were ran in parallel and the output from the collect not finished when the next line was called. I have a flock around the part of the function that writes to the output file so only one process at a time should be writing to the output.

This seems strange to me, maybe even a bug. Does running it with the  `--trace` argument help?

Is there a better way to mimic the functionality of collect in functions from external modules ?

No, not really. I am not a big fan of collect() myself 

Best

Luis

Luis Pedro Coelho | Fudan University | http://luispedro.org

https://orcid.org/0000-0002-9280-7885

On Fri, 20 Nov 2020, at 11:03 PM, mette Joergensen wrote:

Best,

Mette

   

--

You received this message because you are subscribed to the Google Groups "NGLess" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ngless+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ngless/0056f0b3-2427-423d-bfc3-5d732bc00e76n%40googlegroups.com.

[mette Joergensen]

I found the bug. It was a stupid mistake in my bashscript. I was using | instead of || when testing if the the output file or the outpuffile.subsample from the previous command existed. That was why it worked when I was using subsample.

Thanks for making me aware of the --trace option. It is nice if you want to see what is going on.