renaming the grad method

[Razvan Pascanu]

Hello all,

I was talking to Ian, and it occur to us that it might make sense in
the near future to change the name of the grad method of each op to
either `Lop` or `forward_accumulation`.
The issue is that the grad method does not compute the gradients (it
even says so in the docstrings) as for a multidimensional output
function there is no gradient. What the grad
method does is to compute ``g_out \dot Jacobian``, where ``g_out`` is
a provided row vector and ``Jacobian`` is the Jacobian of the
function.

This corresponds exactly to my definition of Lop that you can access
as TT.Lop. Unfortunately the name "Lop" was defined by me as a follow
up to the Rop defined by Pearlmutter,
as the Lop multiplies ``g_out`` to the left of the matrix, and the Rop
multiplies to the right. Using this naming convention we would stay
consistent with whatever is there already.

The other options is to name Lop as ``reverse_accumulation`` and Rop
as ``forward_accumulation`` (or forward_mode and reverse_mode) which
is the names they have in the
automatic differentiation community.

Changing to either of the two can simplify the documentation of the
developer who is most of the time confuse of what op.grad is suppose
to do.

Razvan

[Frédéric Bastien]

What I remember is that our grad method should be called bprop, for
back_propagation. What do you think of this? I don't want to change
those name before the next release, we need to release before doing
more stuff.

Fred

[Razvan Pascanu]

On Wed, Sep 12, 2012 at 2:09 PM, Frédéric Bastien <no...@nouiz.org> wrote:
> What I remember is that our grad method should be called bprop, for
> back_propagation. What do you think of this? I don't want to change
> those name before the next release, we need to release before doing
> more stuff.
>

I'm ok with bprop as well (and I would rename Rop to fprop from
forward propagation .. )
And I agree there is no reason to do it right now, I just wanted to
see what people think about it.

Razvan

[Olivier Delalleau]

The name fprop() could make ppl think it does what perform() does.

-=- Olivier

[Pascal Lamblin]

On Wed, Sep 12, 2012, Olivier Delalleau wrote:
> The name fprop() could make ppl think it does what perform() does.

I agree. grad_fprop and grad_bprop?

--
Pascal

[David Warde-Farley]

The 'fprop' at all is pretty confusing, I think... I like the
right/left terminology, personally, as it makes things the most
explicit.

[James Bergstra]

I have personally been thinking in terms of "L" and "R" multiplication
for a while.

If anything, I think that the "Op" part of the name is confusing
because it's not a Theano Op. If renaming these things is on the
table, maybe it would be better to go even further to something like
"jacobian_dot_L" and "jacobian_dot_R"?

[Olivier Delalleau]

I have no strong opinion on the best names, however if they are to be renamed, please try to keep names lowercase if possible.

-=- Olivier

[Ian Goodfellow]

v_dot_jacobian and jacobian_dot_v ?

[James Bergstra]

I like it!

[Razvan Pascanu]

+1