Questions regarding Principal Component Analysis
C. Papan
I am not a math person and would like to get some insight into some
issues of the principal component analysis:
1. What do the axes mean in a PCA scores plot? What does the magnitude
of the numbers say?
2. When I remove some of the extreme variables (as seen in the PCA
loadings plot), at first the PCA scores plot changes drastically with
every removed variable. Then, after having removed some variables, the
scores plot becomes "stable", meaning that removing variables does not
change the relative position of the dots in the scores plot anymore. Is
this a good thing? What does this tell me about the variables?
3. I repeated a measurement 5 times, and in all five experiments, the
arrangement of the dots in the scores diagramm is comparable (the dots
are arranged in the shape of a "U" or slight "W"). What conclusions can
be drawn from this? Does this mean that the experiments gave repeatable
results?
Many thanks in advance,
C. Papan
Paige Miller
1. The PCA Scores plot is like a two-dimensional view of your
multidimensional data. You didn't tell us exactly how many original
variables you have, but let's say there were 20. Then the score plot
of score 1 vs score 2 gives you a two-dimensional view of this 20
dimensional space. In fact, score 1 and score 2 will show you as much
of the variability of your 20 dimensional space as two scores can
show. The distances on the axes depend upon how the PCA scores were
scaled. They represent either distance in original units, or distance
in scaled units.
2. Deleting the variables with extreme loadings? Never heard of doing
that. In fact, it doesn't seem like a good idea. These are the
important variables, the ones with the most impact. You don't want to
delete them.
3. Before you get too excited about the W or U pattern, let's back up
a second. First, put those variables with extreme loadings back into
your analysis. But it also sounds like you have multiple sources of
variability in your data -- a repeated measurement and some
"observation" variability. (You didn't really say what your
"observations" are). If you dump all of the data into PCA, you get a
result where the two different variabilities are treated as
inseparable. Maybe that's what you want, but I doubt its a good idea.
You probably want one PCA on the means of the 5 repeated measurements,
this gives you observation-to-observation PCA, and another PCA on the
5 repeated measurements (each one having its group mean removed) to
determine if there are patterns in the repeated measurements.
--
Paige Miller
paige\dot\miller \at\ kodak\dot\com
Greg Heath
> I am not a math person and would like to get some insight into some
> issues of the principal component analysis:
>
> 1. What do the axes mean in a PCA scores plot?
The directions of the largest variability.
> What does the magnitude of the numbers say?
It depends on how the original variables are scaled.
> 2. When I remove some of the extreme variables (as seen in the PCA
> loadings plot), at first the PCA scores plot changes drastically with
> every removed variable.
Because these are the variables that cause most of the spread in the
data.
If the signal-to-noise ratio is large they account for most of the
information content.
> Then, after having removed some variables, the
> scores plot becomes "stable", meaning that removing variables does not
> change the relative position of the dots in the scores plot anymore.
All the information is gone. All that remains is stationary noise.
> Is this a good thing?
No.
> What does this tell me about the variables?
The obvious: They contain most of the spread and probably, most of
the
useful information.
> 3. I repeated a measurement 5 times, and in all five experiments, the
> arrangement of the dots in the scores diagramm is comparable (the dots
> are arranged in the shape of a "U" or slight "W"). What conclusions can
> be drawn from this?
Not sure.
>Does this mean that the experiments gave repeatable
> results?
No.
Hope this helps.
Greg
C. Papan
thanks a million for your insights. Your suggestions to get a mean from
all five experiments is definitely worthwhile! I will have to merge the
data into one list though, which is not so trivial. Let me briefly
describe the experiment:
1. I'm extracting metabolites from tissue of drug treated animals. The
extracts thus contain a very large number of different cellular metabolites.
2. for one series I measure tissue samples from 5 different drug
treatments, each treatment 5 times, thus 25 samples altogether, in a
mass spectrometer.
3. I repeat the series 5 times (there is nothing special about the
number 5). Thus a total of 125 runs.
4. After processing, the data consists of one peak list per series
containing about 2000-2500 variables (one to few per metabolite) for
each of the five drug treatments in five repeats (25 columns per peak list).
5. I should say, that each peak list consists of a set of overlapping
but not identical set of variables. An estimated 5-10% of variables are
peak list specific, which is most likely due to technical variability.
When I dump the data into the PCA software (I'm using PyChem), in some
series the data clusters very nicely according to the different drug
treatments. The samples are arranged along the PC1 and PC2 according to
the drug treatment duration.
In some samples the clustering in not all that nice. But after taking
5-6 extreme variables out, the samples also cluster nicely.
As you said, there surely are multiple sources of variation in my data,
just because of the very complex measurement and data processing process.
Now I would like to think that the 3-4 variables I'm taking out from the
samples which do not cluster so nicely represent technical variability
and not due to the treatment. I do not think that those 3-4 variables
I'm taking out are the signal and all the rest is just noise.
Hmmm. The more I think about your suggestion, the more I believe that I
will have to filter the peak list to contain only those variables common
to all five series and then get the mean.
Maybe if you have any more suggestions now that I have described the
experiment in more detail, that would be great!
Thanks again,
Papan
Paige Miller
There's so many things wrong here, its hard to know where to begin.
You "dump" the data into a package? I'd prefer you analyze the data
rather than dumping it somewhere, and your choice of words makes me
very nervous.
You have clustering when you plot PC1 vs PC2, and this clustering
corresponds to drug duration differences? Again, you have a priori
variability (that was designed into your data) and you don't really
want to use PCA until you have removed that variability. Otherwise,
your results reflect/contain that a priori variability
You still think that 3-4 variables "do not cluster so nicely" and so
are candidates for removal from the analysis. I'm not sure what "do
not cluster so nicely" means, but in PCA, the variables you could
consider removing are the ones whose loadings are close to zero.
Anyway, both Greg Heath and I have stated that removing these
variables is the wrong thing to do in PCA.
Finally, you didn't explicitly state the goal of this study. If you
are trying to see the difference of the drugs and the treatment times
on the subjects in the study and the response of interest is somewhere
in the spectra that you collect, then perhaps something more along the
lines of Multivariate Analysis of Variance, or a similar analysis done
with Partial Least Squares, would be more helpful.
z
> 2. When I remove some of the extreme variables (as seen in the PCA
> loadings plot), at first the PCA scores plot changes drastically with
> every removed variable. Then, after having removed some variables, the
> scores plot becomes "stable", meaning that removing variables does not
> change the relative position of the dots in the scores plot anymore. Is
> this a good thing? What does this tell me about the variables?
>
by extreme varibles, do you mean the ones with extremely high loading,
or extremely low?
the high ones you want to keep, the low ones not.
C. Papan
thanks again for your help and sorry for my sloppy wording... ;)
Papan
C. Papan
thanks for your help!!
Papan
Paige Miller
> by extreme varibles, do you mean the ones with extremely high loading,
> or extremely low?
>
> the high ones you want to keep, the low ones not.
The ones with high absolute values are the ones you want to keep. The
ones close to zero are the ones you might be able to remove.