DPMM no clustering induced

[Meng Qiu]

Hi everyone,

Hope your week is going well. Thanks to your help, my previous Dirichlet process mixture model worked. 

However, I encountered a problem in running a similar model where no clustering was induced. That is, all the iterations provided the same number of clusters. My DPMM model is a mixture of product of Gaussians, and I apply a Normal-Gamma prior on the mean and precision for each variable. The attached file is the data file I used for the model. 

Thank you very much for your help. I will appreciate it a lot.

## ===== DPMM using Nimble =====##

library(nimble)

# a function used to sample from Normal-Gamma distribution

rnormgamma <- function(n, mu0, lambda0, alpha0, beta0) {

  if (length(n) > 1) 

    n <- length(n)

  tau <- rgamma(n, alpha0, beta0)

  mu <- rnorm(n, mu0, sqrt(1/(lambda0*tau)))

  data.frame(mu = mu, tau = tau)

}

# DPMM starts here

dat <- read.csv("/fs500.csv")

data <- list(y = dat)

n <- nrow(data$y)

J <- ncol(data$y)

code <- nimbleCode({

  z[1:n] ~ dCRP(conc = alpha, size = n)

  alpha ~ dgamma(shape = 1, rate = 1)

  for(i in 1:n) {

    for(j in 1:J) {

      y[i, j] ~ dnorm(mu[z[i], j], sd = sqrt(1/tau[z[i], j]))

    }

  }

  for(i in 1:n) {

    for(j in 1:J) {

      # set a Normal-Gamma prior NG(3,5, 2,3)

      tau[i, j] ~ dgamma(2, 3) # tau = 1/variance

      mu[i, j] ~ dnorm(3, sd = sqrt(1/(5*tau[i, j])))

    }

  }

})

constants <- list(n = n, J = J)

set.seed(123)

mutau.int <- rnormgamma(J*n, 0,2, 2,1) # generate initial values for mu and tau

inits <- list(z = 1:n, alpha = 1,      # put all the initial values together

              mu = matrix(mutau.int$mu, nrow = n, ncol = J), 

              tau = matrix(mutau.int$tau, nrow = n, ncol = J))

model <- nimbleModel(code, data = data, inits = inits, constants = constants, 

                     dimensions = list(mu = c(n, J), tau = c(n, J)))

cmodel <- compileNimble(model)

conf <- configureMCMC(model, monitors = c('z', 'alpha', 'mu', 'tau'), print = TRUE)

mcmc <- buildMCMC(conf)

cmcmc <- compileNimble(mcmc, project = model)

samples <- runMCMC(cmcmc, niter=10000, nburnin = 2000, thin=10)

zSamples <- samples[, grep('z', colnames(samples))]    # samples of cluster IDs

nGroups <- apply(zSamples, 1, function(x)  length(unique(x)))

concSamples <- samples[, grep('alpha', colnames(samples))]

muSamples <- samples[, grep('mu', colnames(samples))]

tauSamples <- samples[, grep('tau', colnames(samples))]

table(nGroups) # almost all of the iterations would favor 2 (or 1)

[Claudia Wehrhahn Cortes]

Hi Meng,

I hope this email finds you well!

It is great to see that you are using nimble, especially the BNP functionality :) 

I ran your code and it seems to be working fine. If you run just a few iterations, you will see that the number of clusters and cluster parameters are being updated. Initially you have n clusters (you assigned each observation to a different cluster) and this number rapidly reduces to 9, 5, 4, and then 2 or 1. Also, the means for the “occupied clusters” are around 11, 58, 178, and 374, which are very different from your starting values. So I think the samplers are working fine.

I would take a closer look at your data set. The first column seems to be uniform on [0, 500] and so maybe the results are not so off? 

I hope this helps.

Please, let me know if you have more questions.

Best,

Claudia

--
You received this message because you are subscribed to the Google Groups "nimble-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nimble-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nimble-users/23f236c3-ce9b-422f-910b-ed91c7f957f9n%40googlegroups.com.

[Meng Qiu]

Hi Claudia,

Thank you for checking my code. My bad, the first column is the row name that should be removed. So the dataset contains only six variables. 

I ran the model again. This time it produced the following table of the numbers of clusters across the iterations:

> table(nGroups)
nGroups
  1     2
793   7 

It is just too consistent. Actually, I tried another data before and got similar results (the numbers of clusters were all 2). Thus, I think there is something wrong there.

Best,

Chris

--

Meng (Chris) Qiu

Graduate Student, Quantitative Psychology

Statistical Methods for Real Data Lab|University of Notre Dame