Order of execution

14 views
Skip to first unread message

mette Joergensen

unread,
Nov 20, 2020, 10:03:25 AM11/20/20
to NGLess
hi
Are lines in ngless scripts executed sequentially or are some line executed in parallel to optimize things ?
I'm asking because I have implemented a function in an external module that depends on some of the lines before to finish. The function check if the output file from a previously collect call exist and does nothing if it doesn't. In that way it was my hope that the function would be doing its job when the last sample was processed. It works fine if I have less than 10 samples or are using subsample, but nothing is done (nothing is written to the output file) if I have more than 10 large samples. I was therefore speculating if the lines were ran in parallel and the output from the collect not finished when the next line was called. I have a flock around the part of the function that writes to the output file so only one process at a time should be writing to the output.

Is there a better way to mimic the functionality of collect in functions from external modules ?

Best,
Mette
   

Luis Pedro Coelho

unread,
Nov 21, 2020, 7:45:12 AM11/21/20
to NGLess List

Are lines in ngless scripts executed sequentially or are some line executed in parallel to optimize things ?
I'm asking because I have implemented a function in an external module that depends on some of the lines before to finish.

Sometimes things are done in parallel, but this is not done for external modules. External module calls are always synchronous.

The function check if the output file from a previously collect call exist and does nothing if it doesn't. In that way it was my hope that the function would be doing its job when the last sample was processed. It works fine if I have less than 10 samples or are using subsample, but nothing is done (nothing is written to the output file) if I have more than 10 large samples. I was therefore speculating if the lines were ran in parallel and the output from the collect not finished when the next line was called. I have a flock around the part of the function that writes to the output file so only one process at a time should be writing to the output.

This seems strange to me, maybe even a bug. Does running it with the  `--trace` argument help?

Is there a better way to mimic the functionality of collect in functions from external modules ?

No, not really. I am not a big fan of collect() myself 

Best
Luis

Luis Pedro Coelho | Fudan University | http://luispedro.org


On Fri, 20 Nov 2020, at 11:03 PM, mette Joergensen wrote:

Best,
Mette
   


--
You received this message because you are subscribed to the Google Groups "NGLess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ngless+un...@googlegroups.com.

mette Joergensen

unread,
Nov 25, 2020, 3:54:24 AM11/25/20
to NGLess
I found the bug. It was a stupid mistake in my bashscript. I was using | instead of || when testing if the the output file or the outpuffile.subsample from the previous command existed. That was why it worked when I was using subsample.

Thanks for making me aware of the --trace option. It is nice if you want to see what is going on.

Reply all
Reply to author
Forward
0 new messages