normalisation and batch effect

[Reem Ahmed]

Hello, 

Thanks a lot for this wonderful package. I have been testing it for a while now and had some questions- apologies for how long this is. 

My first question is about normalisation:

My experiment is a pulldown based experiment, where I don’t expect most proteins to be similar across conditions. I have been using global standards (including a bait protein and some proteins that should be equally expressed) to normalise. My issue is I am not sure how to assess if this normalisation makes sense ? (or even if my assumption is true and if i should then just use median normalisation). Any suggestions of how I can begin to look into this and compare normalisation impact?  I attached one QCPlot and I also look at QQplots and residuals and it's okayish but it is unclear if there is a better way to do this. (UT does not have the bait protein so it makes sense if it would be more variable) 

My other question is about batch effects:

I have a rather large LFQ design, where I have 4 conditions (Lamp1, Lamp2, Nes, UT) each with 4 biological replicates and each of those with 4 technical replicates. One tech replicate per biological rep (rep1) was ran on the same day for all the conditions. The other three were ran a few weeks later one after the other for all conditions. Because of this weird loading, I can see there’s a batch effect, so all the tech reps ran one after the other correlated more to each other than to the one ran on a separate day. 

At this point, I am not sure if I should ignore the repeat tech replicates and just run my analysis using the first technical rep- I am worried perhaps some peptides were lost after storage for weeks?  but also this is a lot of data to discard so i am not sure if there’s a way to account for this batch effect in MsStats? 

Thank you!

 

[vitek...@gmail.com]

Hi Reem,

For the first question regarding normalization with standards: if you have multiple standards, I suggest that you use only a subset of the standards for the normalization, and then see how the normalization affected the remaining standards. If the variation in the left-out standards improved, this means that the normalization worked. If after the normalization the left-out standards remained the same or became more noisy, this means that the normalization failed. Typically normalization fails because it makes unrealistic assumptions (see below).

For the question regarding batch effects: global normalization based on standards assumes that the experimental artifact that affected the standards affected the rest of the proteins in exactly the same way. If this is true, you would not have a pronounced batch effect after a standards-based normalization. If you still see a batch effect after a standards-based normalization, it is possible that (a) the global standards-based normalization made wrong assumptions, or (b) the artifacts are protein-specific and are not captured by the standards. In (b), it is possible to introduce batch as another condition in MSstats. You'd annotate the runs as if they came from 8 conditions: Lamp1.batch1, Lamp2.batch1, Nes.batch1, UT.batch1, Lamp1.batch2, Lamp2.batch2, Nes.batch2, UT.batch2, and then create contrasts that compare two conditions on average over the two batches. E.g., in the order above, comparing Nes vs UT would be a contrast 0, 0, 0.5, -0.5, 0, 0, 0.5, -0.5. This approach will reduce the degrees of freedom of the error but can still gain sensitivity if there is a substantial batch effect

Hope this helps at least somewhat. Thank you for using MSstats

Sincerely

Olga

[Luiz Gustavo]

Hi Olga,

I also have an experimental design with batches, and your answer to Reem's questions fits perfectly for my case, but I still have two questions.

1) I wanted to understand when and how the contrast.matrix is used. I tried digging through the R functions until I reached a ".Call" to function, which seems C++. I guess MSstats is multiplying the coefficients by the contrast matrix, but I couldn't tell for sure. What is the effect of using "0.5" in the comparison matrix?

2) If I decided to use batch in the linear model (e.g., "lm(ABUNDANCE ~ GROUP + BATCH, input)"), how much different would it be compared to your approach in group.comparison?

Thanks for such a wonderful package!

Best,

Luiz

[Reem Ahmed]

Hi Olga,

Thank you so much for this detailed answer!
I finally had time in between conferences to redo my analysis keeping in mind your points and it has definitely helped a lot! 

If you get a chance, I was also wondering when to use TMP vs. a linear model for the summarization step? 

Thanks again!

Best

Reem

[Devon Kohler]

Hey Reem,

Imo TMP is almost always the better option compared to using a linear model in the summarization step. Check out the example below (it uses TMT but the idea is the same). Log Sum really just follows the highest feature. Linear is better but is very effected by noisy features (outliers/missing values/ect). TMP is more robust to these issues, which makes it less impacted by noisy features. 

I suppose linear might be better in cases where you either expect your data to not be too noisy, or in cases where you really want to capture the noise. I would recommend checking out a couple profile plots using both methods to see the impact!

Best,

Devon

[Devon Kohler]

Hi Luiz,

1) You are exactly correct that MSstats multiplies the coefficients by the values in the contrast matrix. What this means when you use fractions is that MSstats will average over those coefficients. 

2) Adding in batch to the model has the same impact as what Olga described. MSstats adds additional covariates using the contrast matrix. The impact of averaging over the covariates is the same as adding that variable into the model.

I hope this makes sense!

Devon

[Luiz Gustavo]

Hi Devon,

Thank you very much for the clarification. One last question: if the batches are unbalanced, meaning more samples of a specific group in a particular batch, should that translate to the contrast matrix? For example, batch.1 has 2 samples and batch.2 has 8 samples, then the contrast matrix would be 0.2 and 0.8, respectively.

Best,

Luiz

[Devon Kohler]

Hi Luiz, 

No imbalanced designs are handled by MSstats on the model side. Changing the contrast matrix would result in answering a different hypothesis.

Best,

Devon

[Luiz Gustavo]

Perfect. Thank you very much for your help.

Best,

Luiz

[Reem Ahmed]

Hi Devon, 

Thank you for this clear explanation! 
I hope you don't mind, but I had another question.

I noticed for a few proteins in my dataset the proteins have high log2FC but the pvalue/adj pvalue is zero. I am not sure what could be causing this ? 

Regards
Reem 

[Devon Kohler]

Hi Reem,

I've actually seen this before. It was due to the p values being really really small and the computer rounding them down to zero. If the log2FC is really high and the SE is very low it is more than likely this is just a rounding issue.

Devon 

[Reem Ahmed]

Hi Devon, 

I went down a rabbit hole reading into this and it seems the limit of reporting in R is 2.220446e-16 and this is indeed equal to the p-values before the 0s start to appear. So this seems like the likely explanation, thanks again for all the help!

Best wishes

Reem