dtype == object in reverse mode

[martinakos]

Hi there, me again with a different problem.

I want to use algopy to compute the jacobian of a cost function that I'm using with scipy.optimize.leastsq.

I managed to compute the jacobian with algopy's forward mode and my minimization works faster with it, that's great! Now I wanted to try the reverse mode. However, I'm running into a NotImplementedError('should implement that') in utpm.py:228

I guess that until this feature is implemented I have to try to avoid ndarrays with dtype == object.

But I don't know how to avoid these 'object' dtype. For example, in the cost function I'm building an ndarray representing 3x3 rotation matrix and the individual elements of these matrix are of type <class 'algopy.tracer.tracer.Function'> and dtype('float64') but the resulting matrix is of type ndarray with dtype('object')

Any suggestion how to get pass this problem?

Thanks

Martin

[Sebastian Walter]

Hi Martin,

1) I presume that your Jacobian has more rows than columns, right? The
reverse mode is faster when the Jacobian has many more columns than
rows, e.g. when you want to compute the gradient of a scalar-valued
function.
2) About the NotImplementedError: Could you post a self-contained
example so I can reproduce the error?


cheers,
Sebastian

[martinakos]

Hi Sebastian,

You are right it's more rows than columns, so I'm better off with forward mode. I just wanted to compare both modes as it's the first time I use automatic differentiation. 

I have anyway put together a self contained example that reproduces the exception I had, check the attached file.

the exception occurs in the last line print cg.jacobian(params):

NotImplementedError: should implement that

File "X:\python\tests\test_reverse_ad.py", line 42, in <module>

  print cg.jacobian(params)

File "c:\Python27\Lib\site-packages\algopy\tracer\tracer.py", line 271, in jacobian

  self.pushforward(utpm_x_list)

File "c:\Python27\Lib\site-packages\algopy\tracer\tracer.py", line 103, in pushforward

  f.__class__.pushforward(f.func, f.args, Fout = f)

File "c:\Python27\Lib\site-packages\algopy\tracer\tracer.py", line 720, in pushforward

  out  = func(*args)

File "c:\Python27\Lib\site-packages\algopy\utpm\utpm.py", line 228, in __add__

  raise NotImplementedError('should implement that')

Also, if I change line 17 in my script to return a flat list of residuals ie:

return residual_differences.flat

in my original code I had this, but then I was getting a another exception:

AttributeError: 'numpy.flatiter' object has no attribute 'size'

File "X:\python\tests\test_reverse_ad.py", line 42, in <module>

  print cg.jacobian(params)

File "c:\Python27\Lib\site-packages\algopy\tracer\tracer.py", line 265, in jacobian

  M = self.dependentFunctionList[0].size

Cheers

Martin 

[martinakos]

Oh, I just realized that in the script I attached I copied only 5 matches. this makes the jacobian have more columns than rows. That's not the the typical case, typically I would have hundreds of matches, I just happened to have copied 5 of them without thinking much. 

[Sebastian Walter]

just a quick update:

I'll release a new version of algopy in the next few days:

new features:
* x.reshape is now available in the reverse mode
* one gets now a much more detailed error message when something goes
wrong in cg.jacobian(...).

bugfixes:
* x/y when x is a UTPM instance and y is an ndarray
* broadcasting in x/y
* ...

About your example:

In your code you use broadcasting. In principle, this isn't a problem.
But numpy is doing something I don't understand, so your example
computes a wrong result.

I've posted on the Numpy mailing list:
http://thread.gmane.org/gmane.comp.python.numeric.general/51379
I'd say it's a bug of numpy. At least the current behavior doesn't
make sense to me.
Let's see what they say on the mailing list.

cheers,
Sebastian

[Stylianos Kyriacou]

Having a similar error, In the forward mode though :S 

raise NotImplementedError('should implement that') form  

       elif isinstance(rhs,numpy.ndarray) and rhs.dtype == object:

raise NotImplementedError('should implement that')

This also comes from a part of my code that uses numpy arrays. 

Has there been any other solution than dropping numpy for this? 

[Sebastian Walter]

Can you post a self-contained example of the issue?

Cheers,

Sebastian

--
You received this message because you are subscribed to the Google Groups "algopy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to algopy+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.