Bootstapping in cluster analysis: What is resampled where?

[Kerry]

My understanding of bootstrapping pertains to re-sampling data from a
single sample from a population. In cluster analysis, one is trying to
determine if the sample of data comes from a single population or not,
but exists as a matrix of sampled measurements rather than an array.

For instance, my data could consist of 3 parameters: height, weight,
and depth for 10 trees, and I want to use cluster analysis (after
normalizing height, weight, and depth values into z scores) to see if
the 10 trees cluster into subgroups.

So, does bootstrapping randomize values across both parameters and
trees, or just one of the two? That is, given my matrix of data, can
the column,row for Tree A height end up with a value that was from
Tree B Weight during bootstrap re-sampling?

Thanks,
kbrownk

[Ray Koopman]

No. In this case, "sampling" means "choosing a tree".
The vector of data for that tree is not altered.

[Art Kendall]

With so few cases, I would start with a 3D scattergram and rotate the
image to eyeball it from different angles. Any multivariate outlier(s)
or groups would stand out.

In bootstrapping you resample cases (here trees).

You would have a 10 by 10 distance/similarity matrix.

For exploratory work, I would be more inclined to try a few
agglomeration methods, Wards, average linkage, etc.


Or do you mean 10 species of trees with several samples of each to see
if individual trees cluster differently than the break down by species?


Art Kendall
Social Research Consultants

.

[Art Kendall]

Art Kendall
Social Research Consultants

Cases are what is usually resampled in resampling methods. (In this
instance trees would be the cases.) The goal of resampling is to get
another handle on error variance.

The matrix that is explored would have some cases (rows and columns)
absent, the distance/similarity of any pair of trees would not be
changed. I have going to meeting of the Classification Society and I do
not recall seeing resampling of cases in clustering applications

Am I correct in thinking that you are reacting to the idea that scaling
or clustering methods will produce solutions on any distance/similarity
matrix? Meaningfulness is a matter of interpretation of the solutions
found. Purely random data would produce some solution.

To some degree you are trying to "prove" a negative. (absence of
clustering).

Reshuffling columns of the matrix many times and comparing the
clustering could be a useful approach. But why would you want to do
both reshuffling and resampling?
I am not familiar with the two tests you mention, but if you were to
post your query on the Classification Society list which is specifically
for clustering, scaling, networks, etc. you may get some help.
http://lists.sunysb.edu/index.cgi?A0=CLASS-L

The rest of this post is only relevant is you are interested in a single
slice of the hierarchical tree. (non-hierarchical clusters).

Many reshuffling (permuting within variables) is used in parallel
analysis in a factor analysis context. It compares the eigenvalues from
some form of factor analysis (often principal components) to the
eigenvalues from many (e.g., 10,000) permutations of the columns of the
same data matrix. Whereas R factor analysis is uses similarities among
variables, there is an old method of clustering from the 60s call Q
factor analysis. In that approach the data is standardized by columns
then within rows. and the rows are correlated. What I am reading between
the lines of your post is that you may be looking for something akin to
parallel analysis for distance matrices.

Parallel coordinate plots are like the profile plots used in clustering,
but are done on cases rather than profiles of groups of cases.

Even with 100 cases you should be able to eyeball extreme or bunched up
cases as you rotate the visualization on its axes so you can see the
data from different perspectives. Don't make the scatter plot too small.

When you have a clustering solution do a discriminant function analysis
pretending the for the moment that the cluster memberships were not
derived from this data. In the classification phase of the DFA, what
does the crosstab of the original memberships by the assigned
memberships look like?

If you have SPSS, you can use the "identify unusual cases" which can
often find cases that "don't fit in" with the other cases based on
multivariate criteria.

Hope this helps.

Art Kendall
Social Research Consultants

On 12/9/2011 7:05 PM, Kerry Brown wrote:
> Thanks for the reply. I can't find my post on the group site so I'm
> replying via email (I'll repost this reply if I find the thread appears).
>
> Say I have 100 samples of elm trees instead of 10. My
> qualitative conclusion is that there is no strong clustering that
> would suggest those 100 elm trees actually cluster into meaningful elm
> tree subgroups. Visually this appears to be the case but I'd like to
> quantify.
>
> So, here I'd have a 100x100 matrix of distance values. Is
> bootstrapping going to randomize those distances values w/n the
> matrix? That is, are the distance values what are re-sampled or is it
> the data used to create the distance values that is re-sampled?
>
> If it's the distance values across trees, then it seems that strong
> clustering would be defined as the re-sampled data (done multiple
> times) rarely resulting in a distance matrix that is similar to the
> actual data, though I'm unsure how such would be quantified. But in
> such a method, replacement of values seems odd.
>
> Alternatively, the re-sampling of distances could be confined to
> within defined clusters. In such a case, I'd think strong actual
> clusters would be those that hold when re-sampled (the opposite of the
> scenario above), and here replacement would make sense.
>
> In summary: Is it the distances that are re-sampled rather than the
> data used to create the distances? And if so, are they re-sampled from
> across the rows and columns of the 100x100 distance matrix (i.e.
> regardless of any clustering method b/t tree samples), or are they
> re-sampled from within clusters as defined using the original data?
>
> Separately, I've used Ward's method w/ Euclidean distances and
> quantified by multi-shuffling values across trees (without
> replacement) and using either Dunn Index or Davies-Bouldin to compare
> clustering strength of the actual vs randomly shuffled clustering
> results. Both Dunn and Davies-Bouldin results suggest my actual data
> is less clustered than random, which seems suspicious (currently
> looking into it).
>
> Thanks again,
> kbrownk

> --
> Kerry M Brown
> Krasnow Institute for Advanced Study
> George Mason University
> kbr...@gmail.com <mailto:kbr...@gmail.com>