Elementary example

[Todd]

Hi Luis,

Thanks for the very handy software.

I'm confused by an elementary example. Here's my code:

from jug import TaskGenerator
import numpy

@TaskGenerator
def f1(x):
return x*x

xs = numpy.arange(100)
#xs = numpy.random.randn(100)

results = map(f1, xs)

If I run the code (on only one core) using "jug execute jug_test.py",
everything works as expected with "jug status jug_test.py" reporting
100 tasks finished. I'm able to see the (correct) results with "jug
shell jug_test.py".

However, if I change the code to generate xs from
numpy.random.randn(100), then "jug execute jug_test.py" runs without
complaint but "jug status jug_test.py" reports 100 tasks as ready
rather than finished. In addition, if I try to see the results with
"jug shell jug_test.py", I get an AssertionError at the line "assert
self.can_load()" in task.py.

What's the difference between generating xs from numpy.arange(100) and
numpy.random.randn()?

Thanks for your help,

Todd

[Luis Pedro Coelho]

Hi Todd,

This stumped me too the first time I saw this behaviour.

Here is a cut down example:

from jug import TaskGenerator

@TaskGenerator
def f(x):
return x*x

x = 2
#x = int(100*random.random())

y = f(x)

With x=2,

y is f(2).

However, if x is a random number, then y=f(x) with x being some number between
0 and 99. Unless you are extremely lucky, x will not always be the same value,
though. So, the first time you run jug, it might see y=f(42) and compute that.
The next time, when you run status, it sees y=f(32) and tells you it can run
it, but has not done it yet. When you run shell, it sees y=f(88) and you
cannot load it, because f(88) never run.

Set the random seed with numpy.random.seed(1234) and you will see consistent
results.

In a way, this is a feature. One of the things I want out of jug is to have
the results be as consistent and reproducible as possible and so, if they
depend on random numbers, the seed should be explicit. But it is really hard
to detect and output a good error message for.

HTH
Luis

--
Luis Pedro Coelho
http://luispedro.org

[Todd]

Hi Luis,

Ah, got it!

Thanks!

Todd