Question on Some Sort of Batched Matrix Multiplication

[BARISCAN BOZKURT]

Hello,

I want to multiply a weight matrix of size (O, I) with a batch of matrices of size (I, W, B) expecting to have an output of size (O, W, B), where B is my batch size. Let us say my weight matrix is W of type KnetArray{Float32} with size (256,34). Also, my input Z is a KnetArray{Float32, 3} with size (34, 784, 64) so that the batch size is 64. I can multiply W with Z[:,:,1] with standard matrix multiplication. However, I want to multiply W with Z[:,:,j] for all j in a broadcasted manner, and obtain a tensor of size (256, 784, 64). I can use the repeat(.) function to repeat my weight matrix W so that I can use the bmm(.) function of Knet. However, the repeat function does not accept KnetArray and I want a more optimized solution. How can I perform this operation in an optimized way? I will appreciate any help.

Best,

Barışcan

--

Barışcan Bozkurt 

Koç University, Electrical and Electronics Engineering & Mathematics (D.M.)

[Shadi Sameh Hamdan]

Hello Bariscan, 

Bmm is not what you need in this scenario. Bmm is used when for each Z[:, :, j] you want to use a different weight matrix Wj. 

You can achieve what you are trying to do using simple reshaping. First, save the size of the last two dimensions of your batch of matrices, namely W and B. Then, reshape the matrices to (I, W*B). Once you have this shape, a single matrix multiplication will give you the output of shape (O, W*B), which you can then reshape back to (O, W, B) using the values of W and B you saved earlier. 

Let me know if you have any questions. 

Best, 

Shadi 

--
You received this message because you are subscribed to the Google Groups "COMP541" group.
To unsubscribe from this group and stop receiving emails from it, send an email to COMP541+u...@ku.edu.tr.
To view this discussion on the web visit https://groups.google.com/a/ku.edu.tr/d/msgid/COMP541/CAMggykY3k_Jsa6zV_fD7LEGTr%2BKXKOreB1RG30%3D7dhO4n3Q5Dg%40mail.gmail.com.