Component-wise multiplication?

[Scott Brown]

Does Nimble implement the Hadamard product?

I'm actually attempting to implement the RELU function.  Naively, I have `x * ( x > 0 )`, where `*` is component-wise multiplication.  This is throwing compiler errors, though.

Cheers,

-Scott

[Chris Paciorek]

Yes, with `*` we do vectorized multiplication, recycling elements as needed if the two inputs are not equal length.

Please give us more details (ideally a minimal reproducible example) so we can help figure out what is going wrong.

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/25b39494-514c-4460-adf2-0518cee622ddn%40googlegroups.com.

[Scott Brown]

Here's the function:

```
relu <- nimbleFunction(
  run = function(x = double(2)) {
    returnType(double(2))
    return(x * ( x > 0 ))
  })
```

And here's the error:

```
> compileNimble(relu)
Compiling
  [Note] This may take a minute.
  [Note] Use 'showCompilerOutput = TRUE' to see C++ compilation details.
Error: Failed to create the shared library. Run 'printErrors()' to see the compilation errors.
> printErrors()
using C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.0.40.1)’
using SDK: ‘MacOSX14.0.sdk’
P_7_rcFun_SDMGPtools242.cpp:24:42: error: no matching member function for call to 'cwiseProduct'
Eig_Interm_156 = (Eig_ARG1_x_Interm_157).cwiseProduct((((Eig_ARG1_x_Interm_158).array() > 0)).cast<double>());
                 ~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
/opt/homebrew/lib/R/4.3/site-library/nimble/include/Eigen/src/Core/../plugins/MatrixCwiseBinaryOps.h:23:1: note: candidate template ignored: could not match 'MatrixBase' against 'CwiseUnaryOp'
cwiseProduct(const EIGEN_CURRENT_STORAGE_BASE_CLASS<OtherDerived> &other) const
^
/opt/homebrew/lib/R/4.3/site-library/nimble/include/Eigen/src/Core/MatrixBase.h:443:5: note: candidate template ignored: could not match 'SparseMatrixBase' against 'CwiseUnaryOp'
    cwiseProduct(const SparseMatrixBase<OtherDerived> &other) const
    ^
1 error generated.
make: *** [P_7_rcFun_SDMGPtools242.o] Error 1
```

It's possible that the complaint relates to the fact that `(x > 0)` is a logical type, where `x` is a double type?

[Perry de Valpine]

Hmm, thanks for the report. I thought that would work and I will look into what is going wrong or unexpected.

In the meantime, I'd suggest simply writing out the for loops to do it element-wise directly (which maybe you're doing already as a work-around). In R evaluation, for loops are slow, but in the C++ they're fine. nimble tries to compile non-scalar operations to C++ Eigen code, which is often faster than nested for-loops, but I doubt it would be very noticeable in this case.

relu <- nimbleFunction(
   run = function(x = double(2)) {
     returnType(double(2))

     ans <- matrix(type = "double", nrow = dim(x)[1], ncol = dim(x)[2], init=FALSE)
     for(i in 1:dim(x)[1]) for(j in 1:dim(x)[2]) ans[i,j] <- x[i, j] * (x[i, j] > 0)

     return(ans)
   })

HTH

Perry

To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/42bcc896-f14b-4638-ac32-a90ff89ea079n%40googlegroups.com.