Regularization: ddexp alternative

[mahdieh...@gmail.com]

Dear All,

I am trying the Bayesian Lasso ( beta j ~ DoubleExpo(0, sigma^2 ) where:  sigma^2 ~  InvGamma(0.01, 0.01).

 I realized that "ddexp" (a Laplace double exponential prior in JAGS ) is under development in NIMBLE. Is there an alternative distribution OR a different formulation to the following?

nimbleCode({

  #... some previous code

 

  # Likelihood

  for(i in 1:n){
    Y[i]   ~ dnorm(mu[i], inv.var)
    mu[i] <- alpha + inprod(X[i,], beta[])
  }

  # Prior for beta
  for(j in 1:p){
    beta[j] ~ ddexp(0, inv.var.b)
  }

  # Prior for the inverse variance
  inv.var   ~ dgamma(0.01, 0.01)
  inv.var.b ~ dgamma(0.01, 0.01)

  alpha     ~ dnorm(0, 0.01)

#... some later code })

All the best,

Mahdieh

[Chris Paciorek]

Hi Mahdieh,

This is a case where you can use a user-defined distribution. Please
see Section 12.2 of the manual. Given that there is an example of
writing a simple exponential distribution there, hopefully it will be
straightforward for you to write the ddexp case.

You could also, without having to define your own distribution, write
the double exponential as a scale mixture of normals using a parameter
augmentation approach, but this introduces additional parameters so
would likely decrease MCMC efficiency. So I'd suggest the user-defined
distribution approach.

Filling out some of the extra distributions that we have said we'd
like to add always seems to take lower priority than other things
we're doing, but given your interest and that this case is a common
one, hopefully we can add this as part of NIMBLE in the next release
or two.
Chris

> --
> You received this message because you are subscribed to the Google Groups "nimble-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
> To post to this group, send email to nimble...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/5ad36cb4-f651-4cbf-9b2c-2dad569f54ab%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Raymond Boaz]

I'm curious if you were able to get this coded up. I am currently trying to implement the Bayesian Lasso as well and could use any help available.

Thanks,

Ray Boaz

[Chris Paciorek]

Hi Raymond, we are pulling together our next release right now to come
out before the holidays. I think I can probably include ddexp in that
release.

> --
> You received this message because you are subscribed to the Google Groups "nimble-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
> To post to this group, send email to nimble...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/b8f14b2d-c20d-4927-8a6d-4475b5968a36%40googlegroups.com.

[Christopher Paciorek]

Finally following up on this... the double exponential / Laplace distribution is available in our recently released version 0.7.0 as ddexp with alias dlaplace.

-Chris

> To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users+unsubscribe@googlegroups.com.