Which R package is best for Illumina Methylation EPIC array analysis?

[vk]

Hi,

I have samples from Illumina methylation EPIC array. I want to do differential methylation analysis. And i also need QC plots (PCA, Boxplot etc) for the samples. Which package should I use for the analysis? I'm new to this analysis. Could you please help me? Thank you.

[Fabian]

Hi,
shameless self-plug here: You could use RnBeads for this:
http://bioconductor.org/packages/release/bioc/html/RnBeads.html
http://rnbeads.mpi-inf.mpg.de
http://doi.org/10.1038/nmeth.3115

Let us know if we can help!

Best regards,
Fabian

[Yuan Tian]

Hello, You may try newest ChAMP package. I developed newest version of ChAMP, if included functions like CpG filtering, normailization, DMP, DMR, CNA, Cell Proportion Correction(for blood samples).

https://www.bioconductor.org/packages/release/bioc/html/ChAMP.html

Best

YuanTian

On Sun, May 1, 2016 at 1:13 PM vk <venky...@gmail.com> wrote:

Hi,

I have samples from Illumina methylation EPIC array. I want to do differential methylation analysis. And i also need QC plots (PCA, Boxplot etc) for the samples. Which package should I use for the analysis? I'm new to this analysis. Could you please help me? Thank you.

--
You received this message because you are subscribed to the Google Groups "Epigenomics forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to epigenomicsfor...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[vk]

Hi Fabian,

Thank for the reply. I would like to know whether I can get all QC plots with this Rnbeads package. I need PCA plot, Boxplot, Density plot etc.

[vk]

Hi Yuan,

Thank for the reply. I would like to know whether I can get all QC plots with this Rnbeads package. I need PCA plot, Boxplot, Density plot etc.

[Yura Grabovska]

Hey vk,

For QC on the samples do you mean you want to QC the array?

In that case probably preprocessing through a minfi pipeline and using the qcReport() on the RGChannelSet will give you a detailed QC report on the array including the internal control signals. 

You can do this by feeding minfi the Sentrix ID and Sentrix Position in a data frame through the function 

RGset <- read.metharray.exp(targets = data.frame, ...)

qcReport(RGset, ...)

Mset <- preprocessNoob(RGset, ...) #use Noob for small sample numbers

GMset <- mapToGenome(Mset)

For PCA, you can use the beta values matrix obtained with the minfi command getBeta() on the GenomicMethylRatioSet or the MethylRatioSet obtained following normalisation. The function prcomp() from the default package stats will work for this. You will need to transpose the columns and the rows eg:

betas <- getBeta(GMset)

prcomp <- prcomp(t(betas))

summary(prcomp)

For differentially methylated regions/positions you can use RnBeads, DMRcate, ChAMP, etc which all come with vignettes.

[vk]

Dear Fabian,

Could you please tell me how can I get Sample_annotation.csv file for all my samples? And why it is not mentioned in Vignettes of Rnbeads that Rnbeads work for Illumina EPIC array. Thank you.

On Monday, 2 May 2016 10:01:53 UTC+2, Fabian wrote:

[vk]

Could you please tell me how to get annotation file for my samples. Without that I couldn't go forward for the analysis using RnBeads. Please help me. Thank you.

On Monday, 2 May 2016 10:01:53 UTC+2, Fabian wrote:

[Amber]

Hi Vk,

You have to create the sample annotation file yourself.  It should have information describing your subjects.

It should look something like this (plus whatever additional info you want to include):

Sample_ID

Sentrix_ID

Sentrix_Position

Group

It works best if it is a .csv file or a plain text file.

The most important columns are Sentrix ID and Sentrix Position because the script will use these to create a barcode. You may also combine them yourself and label it "Barcode."

Hope this helps!

Amber

[vk]

Thanks a lot. Yes I will do that. Initially I was trying with some example data. But getting an error.

library(RnBeads)

data.dir <- "D:/fg/Methylation/idattest"

idat.dir <- file.path(data.dir, "idat")

sample.annotation <- file.path(data.dir, "sample_annotation.csv")

analysis.dir <- "D:/fg/Methylation/analysis"

report.dir <- file.path(analysis.dir, "reports")

rnb.options(filtering.sex.chromosomes.removal=TRUE,identifiers.column="Sample_ID")

rnb.run.analysis(dir.reports=report.dir,sample.sheet=sample.annotation, data.dir=idat.dir, data.type="infinium.idat.dir")

2016-05-10 17:36:06     0.6  STATUS         STARTED RnBeads Pipeline

2016-05-10 17:36:06     0.6    INFO             Initialized report index and saved to index.html

2016-05-10 17:36:06     0.6  STATUS             STARTED Loading Data

2016-05-10 17:36:06     0.6    INFO                 Number of cores: 1

2016-05-10 17:36:06     0.6    INFO                 Loading data of type "infinium.idat.dir"

2016-05-10 17:36:06     0.6   ERROR                 invalid data.source parameter, idat.dir is not found, or is not directory

Error in logger.error(txt) : 

  invalid data.source parameter, idat.dir is not found, or is not directory

Could you please help me in this? Thank you.

[Amber]

Your idat directory should be a folder containing all of your idat files.  Only give the path to the folder not one of the files. In the case you provided, "idat" should be a separate folder within "idattest."

[vk]

Ok. I did that. But now Im having a new error. Could you please help me in this?

data.dir <- "D:/fg/Methylation/idattest"

> idat.dir <- file.path(data.dir, "idat")

> sample.annotation <- file.path(data.dir, "sample_annotation.csv")

> analysis.dir <- "D:/fg/Methylation/analysis"

> file.exists(idat.dir)

[1] TRUE

> file.exists(sample.annotation)

[1] TRUE

> file.exists(analysis.dir)

[1] TRUE

> report.dir <- file.path(analysis.dir, "reports")

> file.exists(report.dir)

[1] FALSE

> analysis.dir <- "D:/fg/Methylation/analysis"

> file.exists(analysis.dir)

[1] TRUE

> report.dir <- file.path(analysis.dir, "reports")

> file.exists(report.dir)

[1] TRUE

> rnb.options(filtering.sex.chromosomes.removal=TRUE,identifiers.column="Sample_ID")

> rnb.run.analysis(dir.reports=report.dir,sample.sheet=sample.annotation, data.dir=idat.dir, data.type="infinium.idat.dir")

Error in rnb.run.analysis(dir.reports = report.dir, sample.sheet = sample.annotation,  : 

  Could not initialize reports in D:/fg/Methylation/analysis/reports ; make sure this path does not exist.

Thank you.

[Amber]

The program will not overwrite a reports folder. You can either give it a new name or go to the folder and delete it.

[vk]

Oh yeah. Thanks a lot its running. Hope I dont get any errors again. Thanks again

[vk]

There is a new error now. Ghostscript not found.

> rnb.run.analysis(dir.reports=report.dir,sample.sheet=sample.annotation, data.dir=idat.dir, data.type="infinium.idat.dir")

2016-05-10 19:30:54     0.5  STATUS                                 STARTED RnBeads Pipeline

2016-05-10 19:30:55     0.5    INFO                                     Initialized report index and saved to index.html

2016-05-10 19:30:55     0.5  STATUS                                     STARTED Loading Data

2016-05-10 19:30:55     0.5    INFO                                         Number of cores: 1

2016-05-10 19:30:55     0.5    INFO                                         Loading data of type "infinium.idat.dir"

2016-05-10 19:30:55     0.5  STATUS                                         STARTED Loading Data from IDAT Files

2016-05-10 19:30:55     0.5    INFO                                             Added column barcode to the provided sample annotation table

2016-05-10 19:30:55     0.5    INFO                                             Detected platform: HumanMethylation450

2016-05-10 19:32:09     0.8  STATUS                                         COMPLETED Loading Data from IDAT Files

2016-05-10 19:33:16     0.7  STATUS                                         Loaded data from D:/venkatesh/Methylation/idattest/idat

2016-05-10 19:33:18     1.0  STATUS                                         Predicted gender for the loaded samples

2016-05-10 19:33:19     0.8  STATUS                                         Added data loading section to the report

2016-05-10 19:33:19     0.8  STATUS                                         Loaded 12 samples and 485577 sites

2016-05-10 19:33:20     0.8    INFO                                         Output object is of type RnBeadRawSet

2016-05-10 19:33:20     0.8  STATUS                                     COMPLETED Loading Data

2016-05-10 19:33:20     0.8 WARNING                                     Zip not found on this Windows system, this RnBSet object will not be saved. See the instructions for installing ZIP on Windows in the FAQ section of the RnBeads website.

2016-05-10 19:33:20     0.8    INFO                                     Initialized report index and saved to index.html

2016-05-10 19:33:20     0.8  STATUS                                     STARTED Quality Control

2016-05-10 19:33:20     0.8    INFO                                         Number of cores: 1

2016-05-10 19:33:21     0.8  STATUS                                         STARTED Quality Control Section

2016-05-10 19:33:25     0.8   ERROR                                             Could not create file. GhostScript was not found

Error in logger.error(em) : 

  Could not create file. GhostScript was not found

On Tuesday, 10 May 2016 19:28:46 UTC+2, Amber wrote:

[Yassen Assenov]

Take a look at the first two items in our FAQ section:

http://rnbeads.mpi-inf.mpg.de/faq.php

Greetings,

Yassen

[vk]

Dear Yassen,

Dear Yassen,

I had looked into FAQ section in the website and followed the steps. The analysis is going on from yesterday afternoon. And in the mean time I also found some warnings. Please find them below.

2016-05-11 12:09:08     0.7 WARNING     Zip not found on this Windows system, this RnBSet object will not be saved. See the instructions for installing ZIP on Windows in the FAQ section of the RnBeads website.

2016-05-11 13:40:04     0.8 WARNING             Removed 1 loci with missing values in group KOSR (based on Treatment)

2016-05-11 13:45:31     0.9 WARNING             Removed 11 loci with missing values in group TeSR (based on Treatment)

2016-05-11 13:50:58     1.3 WARNING             Removed 2 loci with missing values in group CT1 (based on CellType)

2016-05-11 13:56:27     1.2 WARNING             Removed 20 loci with missing values in group CT2 (based on CellType)

2016-05-11 14:12:53     1.1  STATUS             Created 36 scatter plots of methylation variance on the site level

2016-05-11 14:16:08     1.3 WARNING             Removed 3 loci with missing values in group TeSR (based on Treatment)

2016-05-11 14:17:46     1.2 WARNING             Removed 1 loci with missing values in group CT2 (based on CellType)

2016-05-11 14:25:03     1.1 WARNING             Removed 1 loci with missing values in group CT2 (based on CellType)

I had installed Zip as explained in the FAQ section. But Don't why I'm getting the error. And still I can see the analysis is going on. I really can't do any other work because my system is totally getting strucked if i move my cursor to another tab. Could you please tell me what is the problem about the warnings and why is it taking more time for the analysis? The data is the example data from your website.

[Yassen Assenov]

About the warning on Zip:

I am assuming you use Windows. As the warning message indicates, you haven’t installed the correct version of Rtools, or R cannot locate the executable files for zipping and unzipping. In the FAQ section, we also gave suggestions on how to test whether the zip application can be accessed and is working properly.

About the warnings on missing values:

Such messages are quite common and are usually not a reason to worry. Think of them as informing you about missing values in the methylation matrix (the table with all methylation beta values) or in the sample annotation table. I cannot tell you to what extend the presence of missing values affects the validity of a conclusion you might make based on the plots generated by RnBeads. You have to look at these on a case by case basis and decide; that is why RnBeads prints these messages, instead of silently handling/ignoring missing values.

About the duration of the analysis:

Since RnBeads often generates tens of thousands of figures and exports large tables to files, the running time does sometimes become an important factor to consider when customising an analysis workflow. Unfortunately, I cannot give you an estimation, I don’t keep a track of how much time the different analyses take on the example dataset (or any other dataset). The reports you can see online were generated on a computational cluster and therefore we don’t have the .log files included. 

Below, I am showing some examples on how to use the analysis options for reducing the running time. This is at the expense of using more memory or dropping certain sections from the reports. I took most of these examples also from the FAQ webpage.

# Disable disk swapping (stores all datasets in memory)

rnb.options(disk.dump.big.matrices=FALSE)

# Disable greedycut (filtering)
rnb.options(filtering.greedycut=FALSE)
# Disable intersample variation plots (exploratory analysis)
rnb.options(exploratory.intersample=FALSE)
# Reduce the subsampling number for estimating density plots
rnb.options(distribution.subsample=100000)
# Disable regional methylation profiling (exploratory analysis)
rnb.options(exploratory.region.profiles=character())
# Disable chromosome coverage plots (QC, sequencing data only)
rnb.options(qc.coverage.plots=FALSE)

Please take a look at help topic on analysis options for more information on what these settings stand for:

?rnb.options

Greetings,

Yassen

[vk]

Dear Yassen,

I need differential methylated regions by considering p-value and delta B. But I haven't found anything related to delta B in RnBeads. Could you please tell me whether it is possible or not. Looking forward to your response.

Thank you.

Dear Yassen,

To unsubscribe from this group and stop receiving emails from it, send an email to epigenomicsforum+unsub...@googlegroups.com.

[vk]

Dear Yassen,

I am working Infinium methylation EPIC data. I am getting an error for the greedy approach command. Could you please help me in this. Please find the following commands.

 rnb.set <- result$rnb.set

> rnb.set <- rnb.execute.import(data.source=data.source,data.type="infinium.idat.dir")

2016-05-16 13:17:16     1.2  STATUS         STARTED Loading Data from IDAT Files

2016-05-16 13:17:16     1.2    INFO             Added column barcode to the provided sample annotation table

2016-05-16 13:17:18     1.2    INFO             Detected platform: MethylationEPIC

2016-05-16 13:21:51     1.6  STATUS         COMPLETED Loading Data from IDAT Files

There were 12 warnings (use warnings() to see them)

> rnb.set

Object of class RnBeadRawSet

      27 samples

  867926 probes

of which: 864935 CpG, 2932 CpH, and 59 rs

Region types:

 257089 regions of type tiling

  35217 regions of type genes

  45127 regions of type promoters

  26606 regions of type cpgislands

Intensity information is present

Detection p-values are present

Bead counts are present

Quality control information is present

Summary of normalization procedures:

The methylation data was not normalized.

No background correction was performed.

> nrow(meth(rnb.set.unfiltered)) # the number of sites in the unfiltered object

[1] 867926

Warning message:

In samples(object) :

  The supplied identifiers column is not found or is not suitable

> rnb.set.filtered <- rnb.execute.context.removal(rnb.set.unfiltered)$dataset

There were 12 warnings (use warnings() to see them)

> nrow(meth(rnb.set.filtered))

[1] 864935

Warning message:

In samples(object) :

  The supplied identifiers column is not found or is not suitable

> > rnb.set.filtered <-rnb.execute.snp.removal(rnb.set.filtered, snp="any")$dataset

Error: unexpected '>' in ">"

> rnb.set.filtered <-rnb.execute.snp.removal(rnb.set.filtered, snp="any")$dataset

There were 12 warnings (use warnings() to see them)

> nrow(meth(rnb.set.filtered))

[1] 725490

Warning message:

In samples(object) :

  The supplied identifiers column is not found or is not suitable

> rnb.set.filtered <- rnb.execute.sex.removal(rnb.set.filtered)$dataset

There were 12 warnings (use warnings() to see them)

> nrow(meth(rnb.set.filtered))

[1] 707346

Warning message:

In samples(object) :

  The supplied identifiers column is not found or is not suitable

> rnb.set.filtered <- rnb.execute.greedycut(rnb.set.filtered)$dataset

2016-05-16 17:35:52     1.7    INFO         Working with a p-value threshold of 0.05

2016-05-16 17:36:55     1.5  STATUS         Calculated a total of 2198 iterations

2016-05-16 17:36:55     1.5    INFO         Optimal number of iterations is 2198

Warning messages:

1: In samples(object) :

  The supplied identifiers column is not found or is not suitable

2: In samples(object) :

  The supplied identifiers column is not found or is not suitable

> nrow(meth(rnb.set.filtered))

Error in nrow(meth(rnb.set.filtered)) : 

  error in evaluating the argument 'x' in selecting a method for function 'nrow': Error in (function (classes, fdef, mtable)  : 

  unable to find an inherited method for function ‘meth’ for signature ‘"NULL"’

Thank you.

On Thursday, 12 May 2016 16:57:13 UTC+2, Yassen wrote:

Dear Yassen,

To unsubscribe from this group and stop receiving emails from it, send an email to epigenomicsforum+unsub...@googlegroups.com.

[Yassen Assenov]

The Greedycut-related function is slightly different than the other filtering functions. It does not return a list that contains an item “dataset”; rather, the return value gives information on how the algorithm ran. Use the following two command to filter a dataset using Greedycut:

result <- rnb.execute.greedycut(rnb.set)

rnb.set <- remove.samples(remove.sites(rnb.set, result$sites), result$samples)

Greetings,

Yassen

Dear Yassen,

Dear Yassen,

[Amber]

Hi Yassen,

Do you have a suggestion for filtering sex chromosome removal for EPIC data.  It works fine on 450k data but I get an error when I attempt the EPIC data using rnb.execute.sex.removal(rnb.set)

The error is: Error in as.hi.integer(eval(x, envir = envir), maxindex = maxindex, dim = dim,  : 

  negative subscripts out of range

Thanks,

Amber

[vk]

Dear Yassen,

Normalization step is not working for EPIC data. Could you please help me in this.

rnb.set.norm <- rnb.execute.normalization(rnb.set.unfiltered, method="illumina",bgcorr.method="methylumi.noob")

2016-05-17 10:37:53     1.4 WARNING         Incompatible methods for object and background correction method: ]

no background correction on the MethylationEPIC data possible at the moment

In the preprocessing step also I got the same warning.

> rnb.set.unfiltered <- rnb.set

> result <- rnb.run.preprocessing(rnb.set.unfiltered, dir.reports=report.dir)

2016-05-16 16:26:21     1.5  STATUS         STARTED Preprocessing

2016-05-16 16:26:21     1.5    INFO             Number of cores: 1

2016-05-16 16:26:22     1.5  STATUS             STARTED Filtering Procedures I

2016-05-16 16:26:33     1.9  STATUS                 STARTED Removal of SNP-enriched Sites

2016-05-16 16:26:33     1.9  STATUS                     Removed 17392 sites using SNP criterion "3"

2016-05-16 16:26:33     1.9  STATUS                     Saved removed sites to D:\venkatesh\Clarence_methylation\analysis\reports_details\preprocessing_data/removed_sites_snp.csv

2016-05-16 16:26:33     1.9  STATUS                     Added a corresponding section to the report

2016-05-16 16:26:34     1.9  STATUS                 COMPLETED Removal of SNP-enriched Sites

2016-05-16 16:26:46     1.5    INFO                 Working with a p-value threshold of 0.05

2016-05-16 16:26:47     1.2  STATUS                 STARTED Greedycut

2016-05-16 16:27:38     1.6  STATUS                     Calculated a total of 3150 iterations

2016-05-16 16:27:38     1.6    INFO                     Optimal number of iterations is 3150

2016-05-16 16:28:05     1.6  STATUS                     Created ROC plot

2016-05-16 16:28:09     1.6  STATUS                     Created line plots for matrix dimensions and other statistics

2016-05-16 16:28:09     1.6  STATUS                     Saved removed sites to D:\venkatesh\Clarence_methylation\analysis\reports_details\preprocessing_data/removed_sites_greedycut.csv

2016-05-16 16:28:10     1.6  STATUS                 COMPLETED Greedycut

2016-05-16 16:28:10     1.6  STATUS                 Retained 27 samples and 847385 sites

2016-05-16 16:28:10     1.6  STATUS             COMPLETED Filtering Procedures I

2016-05-16 16:28:10     1.6  STATUS             STARTED Summary of Filtering Procedures I

2016-05-16 16:28:10     1.6  STATUS                 Creating summary table of removed sites, samples and unreliable measurements...

2016-05-16 16:28:23     1.6  STATUS                 Saving table and figures...

2016-05-16 16:28:26     1.6  STATUS                 Added summary table of removed and retained items

2016-05-16 16:28:32     1.6  STATUS                 Constructed sequences of removed and retained methylation values

2016-05-16 16:28:43     1.5  STATUS                 Added comparison between removed and retained beta values

2016-05-16 16:28:43     1.5  STATUS             COMPLETED Summary of Filtering Procedures I

2016-05-16 16:28:43     1.5  STATUS             STARTED Manipulating the object

2016-05-16 16:34:10     1.5  STATUS                 Removed 20541 sites (probes)

2016-05-16 16:34:10     1.5  STATUS             COMPLETED Manipulating the object

2016-05-16 16:34:10     1.5  STATUS             STARTED Normalization Procedure

2016-05-16 16:34:22     1.6 WARNING                 Incompatible methods for object and method: normalization with method swan cannot be applied to MethylationEPIC data at the moment. Changed the normalization method to "none"

2016-05-16 16:34:22     1.6 WARNING                 Incompatible methods for object and background correction method: ]

no background correction on the MethylationEPIC data possible at the moment

2016-05-16 16:37:56     2.1  STATUS                 Added normalization section

2016-05-16 16:38:05     2.1  STATUS             COMPLETED Normalization Procedure

2016-05-16 16:38:05     2.1  STATUS             STARTED Filtering Procedures II

2016-05-16 16:38:37     2.0  STATUS                 STARTED Probe Context Removal

2016-05-16 16:38:37     2.0  STATUS                     Removed 2965 probe(s) having not acceptable context

2016-05-16 16:38:38     2.0  STATUS                     Saved removed sites to D:\venkatesh\Clarence_methylation\analysis\reports_details\preprocessing_data/removed_sites_context.csv

2016-05-16 16:38:39     2.0  STATUS                     Added a corresponding section to the report

2016-05-16 16:38:39     2.0  STATUS                 COMPLETED Probe Context Removal

2016-05-16 16:38:39     2.0  STATUS                 STARTED Removal of Sites on Sex Chromosomes

2016-05-16 16:38:39     2.0  STATUS                     Removed 19000 site(s) on sex chromosomes

2016-05-16 16:38:39     2.0  STATUS                     Saved removed sites to D:\venkatesh\Clarence_methylation\analysis\reports_details\preprocessing_data/removed_sites_sex.csv

2016-05-16 16:38:39     2.0  STATUS                     Added a corresponding section to the report

2016-05-16 16:38:39     2.0  STATUS                 COMPLETED Removal of Sites on Sex Chromosomes

2016-05-16 16:38:39     2.0  STATUS                 Retained 27 samples and 825420 sites

2016-05-16 16:38:39     2.0  STATUS             COMPLETED Filtering Procedures II

2016-05-16 16:38:39     2.0  STATUS             STARTED Summary of Filtering Procedures II

2016-05-16 16:38:39     2.0  STATUS                 Creating summary table of removed sites, samples and unreliable measurements...

2016-05-16 16:42:42     2.1  STATUS                 Saving table and figures...

2016-05-16 16:43:12     2.1  STATUS                 Added summary table of removed and retained items

2016-05-16 16:43:31     1.8  STATUS                 Constructed sequences of removed and retained methylation values

2016-05-16 16:44:12     1.8  STATUS                 Added comparison between removed and retained beta values

2016-05-16 16:44:12     1.8  STATUS             COMPLETED Summary of Filtering Procedures II

2016-05-16 16:44:12     1.8  STATUS             STARTED Manipulating the object

2016-05-16 16:52:01     1.6  STATUS                 Removed 21965 sites (probes)

2016-05-16 16:52:01     1.6  STATUS             COMPLETED Manipulating the object

2016-05-16 16:52:05     1.6  STATUS         COMPLETED Preprocessing

And could you please tell me how to get differential methylated regions based on beta value and pvalue?

[Yassen Assenov]

Dear Venkatesh,

The only normalization method that in RnBeads works with MethylationEPIC is BMIQ. Unfortunately, no background subtraction and no other normalization methods are currently not supported for MethylationEPIC.

You could set this up using the following command:

rnb.options(normalization.background.method = "none", normalization.method = "bmiq")

Greetings,

Yassen

[Yassen Assenov]

Hello Amber,

This could be a bug in RnBeads, but, unfortunately, I cannot reproduce it. I can only play around with Illumina’s demo dataset and all preprocessing steps there work well (including sex chromosomes removal). Could you please try to reduce your dataset to 4 or 5 samples, e.g. by executing:

rnb.set.selected <- remove.samples(rnb.set, 1:23) # removes the first 23 samples

and then try sex chromosomes removal on the smaller dataset? If it fails again, could you send me the small dataset:

save.rnb.set(rnb.set.selected, "D:/fg/Methylation/dataset_selected") # saves to dataset_selected.zip

plus the current RnBeads analysis options, and the commands that lead to the error?

Greetings,

Yassen

P.S. You can save the current RnBeads option values using the following command:

saveRDS(rnb.options(), "D:/fg/Methylation/RnBeadsOptions.RDS")

Hi Yassen,

Do you have a suggestion for filtering sex chromosome removal for EPIC data.  It works fine on 450k data but I get an error when I attempt the EPIC data using rnb.execute.sex.removal(rnb.set)

The error is: Error in as.hi.integer(eval(x, envir = envir), maxindex = maxindex, dim = dim,  : 

  negative subscripts out of range

Thanks,

Amber

[Amber]

Yassen,

Is there a better way to send a zip file than attaching it here. I keep getting an error when I try to attach my rnb.set zip file.

I still get the error on the smaller dataset. I'm not sure if it has to do with the data type.  I did not set the rub.options when I loaded the dataset because I used: rnb.set<-RnBeadSet(pheno, probes = rownames(betas), betas , p.values = pval , platform = "EPIC")

Looking back at the rub.options it says it is of data type idat.dir.  Could that be causing the problem since these are not idat files?

Amber

[vk]

Dear Yassen,

As you said I have used it.

rnb.options(normalization.background.method = "none", normalization.method = "bmiq")

result <- rnb.run.preprocessing(rnb.set.unfiltered, dir.reports=report.dir)

2016-05-17 12:03:26     1.5  STATUS                     STARTED Preprocessing

2016-05-17 12:03:26     1.5    INFO                         Number of cores: 1

2016-05-17 12:03:27     1.5  STATUS                         STARTED Filtering Procedures I

2016-05-17 12:03:36     2.0  STATUS                             STARTED Removal of SNP-enriched Sites

2016-05-17 12:03:37     2.1  STATUS                                 Removed 16998 sites using SNP criterion "3"

2016-05-17 12:03:38     2.1  STATUS                                 Saved removed sites to D:\venkatesh\Clarence_methylation\analysis\reports_details\preprocessing_data/removed_sites_snp.csv

2016-05-17 12:03:40     2.1  STATUS                                 Added a corresponding section to the report

2016-05-17 12:03:40     2.1  STATUS                             COMPLETED Removal of SNP-enriched Sites

2016-05-17 12:04:46     2.2    INFO                             Working with a p-value threshold of 0.05

2016-05-17 12:04:59     1.4  STATUS                             STARTED Greedycut

2016-05-17 12:05:29     2.0    INFO                                 No unreliable measurements found

2016-05-17 12:05:29     2.0    INFO                                 All measurements are reliable based on the specified threshold

2016-05-17 12:05:30     2.0  STATUS                             COMPLETED Greedycut

2016-05-17 12:05:30     2.0  STATUS                             Retained 27 samples and 847385 sites

2016-05-17 12:05:30     2.0  STATUS                         COMPLETED Filtering Procedures I

2016-05-17 12:05:30     2.0  STATUS                         STARTED Summary of Filtering Procedures I

2016-05-17 12:05:30     2.0  STATUS                             Creating summary table of removed sites, samples and unreliable measurements...

2016-05-17 12:06:52     2.0  STATUS                             Saving table and figures...

2016-05-17 12:08:03     2.0  STATUS                             Added summary table of removed and retained items

2016-05-17 12:08:51     2.2  STATUS                             Constructed sequences of removed and retained methylation values

2016-05-17 12:10:19     2.0  STATUS                             Added comparison between removed and retained beta values

2016-05-17 12:10:19     2.0  STATUS                         COMPLETED Summary of Filtering Procedures I

2016-05-17 12:10:19     2.0  STATUS                         STARTED Manipulating the object

2016-05-17 12:18:21     1.5  STATUS                             Removed 16998 sites (probes)

2016-05-17 12:18:21     1.5  STATUS                         COMPLETED Manipulating the object

2016-05-17 12:18:22     1.5  STATUS                         STARTED Normalization Procedure

2016-05-17 12:23:28     2.1  STATUS                             Normalized sample 1

2016-05-17 12:24:35     2.1  STATUS                             Normalized sample 2

2016-05-17 12:25:19     1.8  STATUS                             Normalized sample 3

2016-05-17 12:25:59     1.8  STATUS                             Normalized sample 4

2016-05-17 12:26:39     1.9  STATUS                             Normalized sample 5

2016-05-17 12:27:20     2.0  STATUS                             Normalized sample 6

2016-05-17 12:28:02     1.9  STATUS                             Normalized sample 7

2016-05-17 12:28:43     2.0  STATUS                             Normalized sample 8

2016-05-17 12:29:25     1.9  STATUS                             Normalized sample 9

2016-05-17 12:30:06     2.0  STATUS                             Normalized sample 10

2016-05-17 12:30:46     2.1  STATUS                             Normalized sample 11

2016-05-17 12:31:27     2.0  STATUS                             Normalized sample 12

2016-05-17 12:32:08     2.1  STATUS                             Normalized sample 13

2016-05-17 12:32:49     2.2  STATUS                             Normalized sample 14

2016-05-17 12:33:30     2.0  STATUS                             Normalized sample 15

2016-05-17 12:34:11     2.2  STATUS                             Normalized sample 16

2016-05-17 12:34:54     1.8  STATUS                             Normalized sample 17

2016-05-17 12:35:34     1.9  STATUS                             Normalized sample 18

2016-05-17 12:36:14     2.0  STATUS                             Normalized sample 19

2016-05-17 12:36:47     1.8  STATUS                             Normalized sample 20

2016-05-17 12:37:27     1.9  STATUS                             Normalized sample 21

2016-05-17 12:38:07     1.9  STATUS                             Normalized sample 22

2016-05-17 12:38:48     2.0  STATUS                             Normalized sample 23

2016-05-17 12:39:30     2.1  STATUS                             Normalized sample 24

2016-05-17 12:40:12     1.9  STATUS                             Normalized sample 25

2016-05-17 12:40:53     2.0  STATUS                             Normalized sample 26

2016-05-17 12:41:34     2.1  STATUS                             Normalized sample 27

2016-05-17 12:49:10     1.8  STATUS                             Performed normalization with method "bmiq"

2016-05-17 13:00:50     2.4  STATUS                             Added comparison between non-normalized and normalized beta values

2016-05-17 13:05:15     2.3  STATUS                             Added histogram of observed beta shifts (magnitude of correction)

2016-05-17 13:05:56     2.3  STATUS                             Added 2D histogram of observed beta values and shifts

2016-05-17 13:07:47     1.9  STATUS                             Added normalization section

2016-05-17 13:07:47     1.9  STATUS                         COMPLETED Normalization Procedure

2016-05-17 13:07:47     1.9  STATUS                         STARTED Filtering Procedures II

2016-05-17 13:08:57     2.7  STATUS                             STARTED Probe Context Removal

2016-05-17 13:08:57     2.7  STATUS                                 Removed 2965 probe(s) having not acceptable context

2016-05-17 13:08:58     2.7  STATUS                                 Saved removed sites to D:\venkatesh\Clarence_methylation\analysis\reports_details\preprocessing_data/removed_sites_context.csv

2016-05-17 13:08:58     2.7  STATUS                                 Added a corresponding section to the report

2016-05-17 13:08:58     2.7  STATUS                             COMPLETED Probe Context Removal

2016-05-17 13:08:58     2.7  STATUS                             STARTED Removal of Sites on Sex Chromosomes

2016-05-17 13:08:59     2.7  STATUS                                 Removed 19000 site(s) on sex chromosomes

2016-05-17 13:09:01     2.7  STATUS                                 Saved removed sites to D:\venkatesh\Clarence_methylation\analysis\reports_details\preprocessing_data/removed_sites_sex.csv

2016-05-17 13:09:01     2.7  STATUS                                 Added a corresponding section to the report

2016-05-17 13:09:01     2.7  STATUS                             COMPLETED Removal of Sites on Sex Chromosomes

2016-05-17 13:09:02     2.7  STATUS                             Retained 27 samples and 825420 sites

2016-05-17 13:09:02     2.7  STATUS                         COMPLETED Filtering Procedures II

2016-05-17 13:09:02     2.7  STATUS                         STARTED Summary of Filtering Procedures II

2016-05-17 13:09:02     2.7  STATUS                             Creating summary table of removed sites, samples and unreliable measurements...

2016-05-17 13:14:31     2.2  STATUS                             Saving table and figures...

2016-05-17 13:15:16     2.2  STATUS                             Added summary table of removed and retained items

2016-05-17 13:16:48     2.5  STATUS                             Constructed sequences of removed and retained methylation values

2016-05-17 13:24:56     2.0  STATUS                             Added comparison between removed and retained beta values

2016-05-17 13:24:56     2.0  STATUS                         COMPLETED Summary of Filtering Procedures II

2016-05-17 13:24:56     2.0  STATUS                         STARTED Manipulating the object

2016-05-17 13:32:29     1.6  STATUS                             Removed 21965 sites (probes)

2016-05-17 13:32:29     1.6  STATUS                         COMPLETED Manipulating the object

2016-05-17 13:32:30     1.6  STATUS                     COMPLETED Preprocessing

Now I have given command for Exploratory analysis. 

I guess it takes much time to complete. I have given this in the afternoon and still going on. But I have some doubts. Could you plz clear them.

For PCA plots you gave an example with two groups . But I have 27 groups. so how can I know which colour is which group?

And how can I get Boxplot, Scatter plot, density plot?

For differential methylated regions I want to consider pvalue <0.05 and delta beta is 0.1. how to do that? please help me. thank you.

[Indranil Sinha]

Dear Yassen,

As you suggested I am running "bmiq" normalization method using 8 EPIC methylation array using following commands.

> rnb.options(normalization.background.method = "none", normalization.method = "bmiq")

> rnb.run.analysis(dir.reports = report.dir, sample.sheet = sample.annotation, data.dir = data.dir, data.type = "infinium.idat.dir")

After normalizing 4 samples I get a error message " Error in density.default(beta2.v[which(beta2.v > 0.6)]) :  need at least 2 points to select a bandwidth automatically"
Will you please inform me why I am getting this error. Before applying "bmiq" normalization yesterday I tried to run "rnb.run.analysis(........)" command and it executed fine with some warnings which were discussed in previous emails in this group.

Thanks for your help.

Best regards,
Indranil.

log>

2016-05-18 12:33:54     0.3  STATUS STARTED RnBeads Pipeline
2016-05-18 12:33:54     0.3    INFO     Initialized report index and saved to index.html
2016-05-18 12:33:54     0.3  STATUS     STARTED Loading Data
2016-05-18 12:33:54     0.3    INFO         Number of cores: 1
2016-05-18 12:33:54     0.3    INFO         Loading data of type "infinium.idat.dir"
2016-05-18 12:33:54     0.3  STATUS         STARTED Loading Data from IDAT Files
2016-05-18 12:33:54     0.3    INFO             Added column barcode to the provided sample annotation table
2016-05-18 12:33:55     0.3    INFO             Detected platform: MethylationEPIC
2016-05-18 12:34:38     0.8  STATUS         COMPLETED Loading Data from IDAT Files
2016-05-18 12:35:57     0.9  STATUS         Loaded data from C:/Indranil/2016 work and activity/Ingrid/Epigenetic regualation of adipose phenotypes/BEA15P198_ID/200242590006/idat
........
........
2016-05-18 12:39:33     1.1  STATUS         COMPLETED Summary of Filtering Procedures I
2016-05-18 12:39:33     1.1  STATUS         STARTED Manipulating the object
2016-05-18 12:41:40     1.2  STATUS             Removed 21394 sites (probes)
2016-05-18 12:41:40     1.2  STATUS         COMPLETED Manipulating the object
2016-05-18 12:41:40     1.2  STATUS         STARTED Normalization Procedure
2016-05-18 12:42:17     1.4  STATUS             Normalized sample 1
2016-05-18 12:42:54     1.3  STATUS             Normalized sample 2
2016-05-18 12:43:32     1.3  STATUS             Normalized sample 3
2016-05-18 12:44:06     1.3  STATUS             Normalized sample 4
Error in density.default(beta2.v[which(beta2.v > 0.6)]) : 
  need at least 2 points to select a bandwidth automatically

[vk]

Hi Indranil,

Which version of Rnbeads you are using? May be it is something related to that. Not Sure. But I found a link having the same error. Sharing it here. Hope it will be helpful to you.

https://groups.google.com/forum/#!topic/epigenomicsforum/w1ytMQkA3eI

Regards

Venkatesh

[Indranil Sinha]

Hi Venkatesh,

Thanks for your email. I am using RnBeads version 1.4.0. I think the error is coming from "bmiq".
In the following command how can I change "bmiq" default option. For example if I want to pass BMIQ(plots=FALSE)

rnb.options(normalization.background.method = "none", normalization.method = "bmiq")

 

Regards,
Indranil.

[vk]

Hi Indranil,

Try in this way.

rnb.set.unfiltered <- rnb.set

rnb.options(normalization.background.method = "none", normalization.method = "bmiq")

result <- rnb.run.preprocessing(rnb.set.unfiltered, dir.reports=report.dir)

rnb.set <- result$rnb.set

I guess this will work.

[Jie yang]

Hi, YunaTian,

I am also working on the EPIC methylation data. However, before I use the ChAMP to derived the intensity and beta value, I compared the results obtained from GenomeStudio and ChAMP. They are significantly different. Since GenomeStudio is the official software developed by Illumina. I am doubting that ChAMP is not working correct on EPIC data. BTW, I also compared the 450k results obtained from GenomeStudio and ChAMP, they are quite similar, the mean of difference for each value is close to 0.01.

How do you think about it?

Thanks

On Sunday, May 8, 2016 at 7:36:50 AM UTC-5, Yuan Tian wrote:

Hello, You may try newest ChAMP package. I developed newest version of ChAMP, if included functions like CpG filtering, normailization, DMP, DMR, CNA, Cell Proportion Correction(for blood samples).

https://www.bioconductor.org/packages/release/bioc/html/ChAMP.html

Best

YuanTian

On Sun, May 1, 2016 at 1:13 PM vk <venky...@gmail.com> wrote:

Hi,

I have samples from Illumina methylation EPIC array. I want to do differential methylation analysis. And i also need QC plots (PCA, Boxplot etc) for the samples. Which package should I use for the analysis? I'm new to this analysis. Could you please help me? Thank you.

--

[Tian Yuan]

Hello  Jie:

I will have a close look in to this, but now I want to know what result of different between each other? ChAMP return many result, please tell me more what result are you comparing? DMP or DMR or something esle.

Best

Yuan Tian

[Ram S]

Hi,

I thought instead of starting a new thread I will post my questions to this existing thread.

I'm trying to use RnBeads package to analyze Illumina Methylation EPIC arrays. 

When I do normalization and background correction. see below for the commands. it says data not normalized and no background correction performed.

any suggestion?

> rnbs.norm <- rnb.execute.normalization(raw.rnbs, method="illumina", bgcorr.method = "methylumi.noob")

2016-06-22 11:50:04     1.3 WARNING     Incompatible methods for object and background correction method: ]

no background correction on the MethylationEPIC data possible at the 

also does anyone know how to subset the regions/sites that are near a set of genes to see if there are any differences in the methylation pattern.

Thanks in advance for your help.

Ram

On Monday, May 2, 2016 at 3:01:53 AM UTC-5, Fabian wrote:

Hi,
shameless self-plug here: You could use RnBeads for this:
http://bioconductor.org/packages/release/bioc/html/RnBeads.html
http://rnbeads.mpi-inf.mpg.de
http://doi.org/10.1038/nmeth.3115

Let us know if we can help!

Best regards,
Fabian

[Jean-Philippe Fortin]

Hi Ram, 

Full background correction, dye bias equalization and normalization functions are provided for EPIC array in the latest devel version of minfi (version 1.19.10) on GitHub: https://github.com/kasperdanielhansen/minfi.

In our recent preprint (http://biorxiv.org/content/early/2016/07/23/065490) we discuss which method is the most appropriate for EPIC arrays, and discuss cell-type composition correction as well.

Hope this helps,

Jean-Philippe

[Zhiying Wang]

[rahel]

Dear all,

I used RnBeads for 450k infinium data analysis. I want to get hyper/hypo methylated data in separate outputs. I defined them by these codes here, but I could not put them in separate data.frame. Can anyone help me and also tell me if these codes are correct?

here is the codes:

library(RnBeads.hg19)

data(small.example.object)

logger.start(fname=NA)

dm <- rnb.execute.computeDiffMeth(rnb.set.example,pheno.cols=c("Sample_Group","Treatment"))

dmt <- get.table(dm,get.comparisons(dm)[1],"promoters")

annot <- annotation(rnb.set.example,"promoters")

all.promoters <- annot$entrezID

#get the hypermethylated promoters

hyper.promoters <- annot$entrezID[dmt[,"mean.mean.diff"]>0]

result <- performGOenrichment.diffMeth.entrez(hyper.promoters,all.promoters,"BP",assembly="hg19")

hypo.promoters <- annot$entrezID[dmt[,"mean.mean.diff"]<0]

result <- performGOenrichment.diffMeth.entrez(hypo.promoters,all.promoters,"BP",assembly="hg19")

Many thanks in advance