element wise matrix multiply?

5,721 views
Skip to first unread message

Dmitry Lagun

unread,
Apr 17, 2013, 6:40:16 PM4/17/13
to theano...@googlegroups.com
Hi,

I could not quickly find in the documentation - is there some analog of element wise matrix multiplication implemented in Theano?

Thanks,
Dmitry

Pascal Lamblin

unread,
Apr 17, 2013, 6:55:42 PM4/17/13
to theano...@googlegroups.com
Hi Dmitry,

On Wed, Apr 17, 2013, Dmitry Lagun wrote:
> I could not quickly find in the documentation - is there some analog of
> element wise matrix multiplication implemented in Theano?

You can simply use '*':
a, b = theano.tensor.matrices('a', 'b')
f = theano.function([a, b], a * b)
f([[0, 1], [2, 3]], [[3, 1], [0, 2]])

array([[ 0., 1.],
[ 0., 6.]])


--
Pascal

Dmitry Lagun

unread,
Apr 17, 2013, 6:55:17 PM4/17/13
to theano...@googlegroups.com
Thanks!

Samuel Leeman-Munk

unread,
Jul 8, 2015, 9:57:48 AM7/8/15
to theano...@googlegroups.com
Is there a speed penalty for elementwise? I imagine blas is optimized for non-elementwise matrix multiplication (more like dot than *)

Daniel Renshaw

unread,
Jul 8, 2015, 10:01:32 AM7/8/15
to theano...@googlegroups.com

--

---
You received this message because you are subscribed to the Google Groups "theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to theano-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Pascal Lamblin

unread,
Jul 8, 2015, 11:40:01 AM7/8/15
to theano...@googlegroups.com
On Wed, Jul 08, 2015, Samuel Leeman-Munk wrote:
> Is there a speed penalty for elementwise? I imagine blas is optimized for
> non-elementwise matrix multiplication (more like dot than *)

Well, dot and * are different operations, and you cannot really use one
instead of the other, so I'm not sure what do you mean by "penalty".

The dot product of an (m, n) matrix by an (n, p) matrix is O(m*n*p) in
theory, and the elemwise product of two (m, n) matrices is O(m*n), so
even if blas optimizes dot, the elemwise product should still be faster.

Also, elemwise product is easily parallelized on GPU and even modern
CPUs (provided the memory layout is contiguous), so it is usually not
a bottleneck.

>
> On Wednesday, April 17, 2013 at 6:55:17 PM UTC-4, Dmitry Lagun wrote:
> >
> > Thanks!
> >
> > On Wednesday, April 17, 2013 6:55:42 PM UTC-4, Pascal Lamblin wrote:
> >>
> >> Hi Dmitry,
> >>
> >> On Wed, Apr 17, 2013, Dmitry Lagun wrote:
> >> > I could not quickly find in the documentation - is there some analog of
> >> > element wise matrix multiplication implemented in Theano?
> >>
> >> You can simply use '*':
> >> a, b = theano.tensor.matrices('a', 'b')
> >> f = theano.function([a, b], a * b)
> >> f([[0, 1], [2, 3]], [[3, 1], [0, 2]])
> >>
> >> array([[ 0., 1.],
> >> [ 0., 6.]])
> >>
> >>
> >> --
> >> Pascal
> >>
> >
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to theano-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.


--
Pascal

Samuel Leeman-Munk

unread,
Jul 8, 2015, 12:05:34 PM7/8/15
to theano...@googlegroups.com
Terrific. Good to hear. Thanks for the helpful replies.

You received this message because you are subscribed to a topic in the Google Groups "theano-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/theano-users/fZpCchn4JbI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to theano-users...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages