Calling cg in a loop - confused with the behavior.

[AlgoDiff]

Consider the script -

 

from datetime import datetime
from algopy import CGraph, Function, exp

def func(x1, x2, x3, x4):
    y = x1 * exp(x3*x2 - x4*x4*x2)
    return y
   
def repeat_func(x1, x2, x3, x4, count):
    start_time = datetime.now()
    cg = CGraph()
    _x1 = Function(x1); _x2 = Function(x2); _x3 = Function(x3); _x4 = Function(x4)
    y = func(_x1, _x2, _x3, _x4)
    cg.trace_off()
    cg.independentFunctionList = [_x1, _x2, _x3, _x4]
    cg.dependentFunctionList = [y]
   
    for i in range(count):
        y_func_gradient =         
    end_time = datetime.now()
    elapsed_time = (end_time - start_time).seconds
   
    result = "Time(seconds) = ", elapsed_time
    return result   

result = repeat_func(100.0, 0.8, 0.16, 0.44, 10000)
print result

cg.gradient([[x1], [x2], [x3], [x4]])

 

(1) - When I execute this, I get the following error. The first iteration passes but the exception is thrown on the second iteration.

 

Traceback (most recent call last):
  File "scratch.py", line 28, in <module>
    result = repeat_func(100.0, 0.8, 0.16, 0.44, 10000)
  File "scratch.py", line 20, in repeat_func
    y_func_gradient = cg.gradient([[x1], [x2], [x3], [x4]])
  File "c:\users\public\Python27\lib\site-packages\algopy\tracer\tracer.py", line 239, in gradient
    raise Exception('you are trying to compute the gradient of a non-scalar valued function')
Exception: you are trying to compute the gradient of a non-scalar valued function

 

(2) - When I change the highlighted code to the following, the error goes away. Here, I am calling cg.Function(... before calling cg.gradient(. This works and 10,000 iterations (as in the script) take 37 seconds.

 

    for i in range(count):
        y_func = cg.function([x1, x2, x3, x4])[0]
        y_func_gradient = cg.gradient([[x1], [x2], [x3], [x4]])

 

 

(3) - Now when I run the following loop - it takes barely a second.

 

    for i in range(count):
        y_func = func(x1, x2, x3, x4)

 

Questions:

 

(1) - Why do I get the error in (1)? Why does it not happen in (2)?

(2) - Is my way of invoking the gradient method correct? ie cg.gradient([[x1], [x2], [x3], [x4]])
(3) - Example 2 in tracer.py throws an error - We call "cg.gradient([1.,2.])" instead of "cg.gradient([[1.],[2.]])". What is the right way to invoke cg.gradient?

 

def f(x1, x2):
    return x1*x2

cg = algopy.CGraph()
x1 = algopy.Function(3.)
x2 = algopy.Function(5.)
y = f(x1, x2)
cg.trace_off()
cg.independentFunctionList = [x1, x2]
cg.dependentFunctionList = [y]
print cg.gradient([1.,2.])

 

Traceback (most recent call last):
  File "scratch.py", line 16, in <module>
    print cg.gradient([1.,2.])
  File "c:\users\public\Python27\lib\site-packages\algopy\tracer\tracer.py", line 254, in gradient
    self.pushforward(utpm_x_list)
  File "c:\users\public\Python27\lib\site-packages\algopy\tracer\tracer.py", line 101, in pushforward
    f.args[0].x = x_list[nf]
IndexError: list index out of range

 

(4) - Any idea why my example script takes around 37 times the time for running the func direction?

 

[Sebastian Walter]

Hello Tarun,

thank's for the bug report! You have stumbled upon a corner case.

That's a deficiency of the current API and represents a corner case I haven't thought of!

Quick fix:

Use 

 y_func_gradient = cg.gradient([np.array(x1), np.array(x2), np.array(x3), np.array(x4)]) 

instead of

y_func_gradient = cg.gradient([[x1], [x2], [x3], [x4]])

The reason is that in cg.gradient represents the computational graph, consisting of Function nodes.

When you trace the evaluation, the values are normal Python floating point numbers.

However, once you have called cg.gradient, the values are transformed to UTPM instances.

Due to a conversion error in cg.gradient, these UTPM instances are of dimension 1 instead of dimension 0.

You would get, however, the correct solution if the exception were not raised.

 

(2) - Is my way of invoking the gradient method correct? ie cg.gradient([[x1], [x2], [x3], [x4]])

yes, use

 y_func_gradient = cg.gradient([np.array(x1), np.array(x2), np.array(x3), np.array(x4)]) 

or rewrite your function to be of the form f(x), where x = numpy.array([x1, x2, x3, x4])

instead of f(x1, x2, x3, x4)

 

(3) - Example 2 in tracer.py throws an error - We call "cg.gradient([1.,2.])" instead of "cg.gradient([[1.],[2.]])". What is the right way to invoke cg.gradient?

 

def f(x1, x2):
    return x1*x2

cg = algopy.CGraph()
x1 = algopy.Function(3.)
x2 = algopy.Function(5.)
y = f(x1, x2)
cg.trace_off()
cg.independentFunctionList = [x1, x2]
cg.dependentFunctionList = [y]
print cg.gradient([1.,2.])

rewrite your function to

def f(x):

    return x[0]*x[1]

would be a quick fix.

 

 

Traceback (most recent call last):
  File "scratch.py", line 16, in <module>
    print cg.gradient([1.,2.])
  File "c:\users\public\Python27\lib\site-packages\algopy\tracer\tracer.py", line 254, in gradient
    self.pushforward(utpm_x_list)
  File "c:\users\public\Python27\lib\site-packages\algopy\tracer\tracer.py", line 101, in pushforward
    f.args[0].x = x_list[nf]
IndexError: list index out of range

 

(4) - Any idea why my example script takes around 37 times the time for running the func direction?

1) computing the gradient is more expensive than the function.

2) there is additional overhead involved to compute the gradient based, because one has to walk the computational graph.

Note however, that TIME(gradient)/TIME(function) = const, i.e., independent of the number of input variables.

best regards,

Sebastian

 

 

--
You received this message because you are subscribed to the Google Groups "algopy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to algopy+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Tarun Joshi]

Thanks Sebastian. That was helpful.

Sent from my Windows Phone

-----Original Message-----
From: Sebastian Walter
Sent: 7/31/2013 10:27 AM
To: alg...@googlegroups.com
Subject: Re: Calling cg in a loop - confused with the behavior.