Random is not random and True is not True

[sven]

Hi!

I didn't really know how to give this question a good title, but in
the following piece of code (which is just a simplified version of my
actual code) I found some strange behaviour:

########################

from random import randint

from jug import TaskGenerator
import jug



@TaskGenerator
def output(*args):
out = open('out.txt', 'a')
print>>out, args

@TaskGenerator
def check(v1, v2):
return v1 == v2

@TaskGenerator
def compute(dummy):
return randint(0, 9)



while True:
v1 = compute(1)
v2 = compute(2)
output(v1, v2)
if check(v1, v2):
break

########################


1.)
I had to give compute() a dummy argument, because it would otherwise
produce always twice the same number. I suspect that it'll otherwise
only create one Task and assign it's output to both variables ... Can
I circumvent this without the fake argument?

2.)
The while loop is not repeated, even if the check() is False. In my
actual code I do check convergence at this point, and would like to go
on if it's not converged yet. Is this solvable?


Thanks in advance!
Sven

[lu...@luispedro.org]

Hi Sven,

> 1.)
> I had to give compute() a dummy argument, because it would otherwise
> produce always twice the same number. I suspect that it'll otherwise
> only create one Task and assign it's output to both variables ... Can
> I circumvent this without the fake argument?

This is actually the expected behaviour. Jug assumes that your functions
are deterministic, which is why it can cache the results. Normally, I will
pass in the seed for the random number generator.

This should go into the FAQ because you are not the first to ask it.

> 2.)
> The while loop is not repeated, even if the check() is False. In my
> actual code I do check convergence at this point, and would like to go
> on if it's not converged yet. Is this solvable?

This is a more serious issue.

Here is what your code looks like after unsugaring:

while True:
v1 = Task(compute, 1)
v2 = Task(compute, 1)
Task(output, v1, v2)
if Task(check, v1, v2):
break

This whole thing is first read in and finally, all tasks are executed. But
the number of tasks is fixed in advance. This is a limitation of jug.
Sorry.

Luis

[sven]

Hi Luis,

thanks for answering that fast.

Actually I was expecting exactly these answers. Thus I already thought
about a solution. It might work out to add a config file which I
modify according to the findings of check(). Thus I could submit a new
batch of jobs, once I find the last batch didn't produce a good enough
result.

So I would have something like this:

1. Python code to do some calculation, and check whether the result is
satisfying. If no, produce a "better" config file and some flag to
test in 2.

2. A submit shell script (or something like this) that waits for all
tasks to finish and checks whether the results were good, if not
resubmit with new configuration.


Any objections against this method? Or preferably any better ways to
do it?
Thanks again!
Sven

[lu...@luispedro.org]

That would work.

You can fake this with jug by using barrier() and value().

barrier() will stop the script unless all of the previous values have been
calculated.

So your script would look like:

v = intial
while True:
v = compute(v)
barrier()
v = value(v) # loads the value
if good_enough(v):
break

Not great, but it would work.

HTH
Luis

[sven]

Ah yes, that's way better than re-reading parameters and
resubmitting ...

I must admit I haven’t fully wrapped my head around how jug works yet,
thus I actually thought this would not work with a barrier().

So, thanks again!
Sven

[lu...@luispedro.org]

Your original question became a FAQ:

http://packages.python.org/Jug/faq.html#it-doesn-t-work-with-random-input

maybe in the future someone will read it there before they run into problems.

*

If you are comfortable testing the bleeding edge, you can try the git
version which would *not require you to resubmit* with barrier.

https://github.com/luispedro/jug

If you do try it, please let me know whether it works.

Thank you
Luis

[Sven Augustin]

Dear Luis,

I'm really sorry for the delayed answer - I wanted to try the git
version before answering, but didn't have time to do so, and then
totally forgot. Just today I remembered and tried to get it running
... and it works perfect!

I will now let my code run, and go around the institute praising how
marvellous jug is!

Thank you and have a nice day, too!
Sven

2011/11/22 <lu...@luispedro.org>: