Import of Illumina data in PathVisio

[Petar Donchev]

Dear All,

I am having issue with data import from Illumina microarray analysis. The data is preprocessed file containing the ID of the probes (numerical) and ILMN_ identifier, as well as normalised expression values and FoldChange information (comparison between control and sample).

I have saved the file as text (tab delimited) but the import step gives me information that all of the imput lines are ecceptions and and the end the window says that I have all 48102 lines unrecognized. 

I have tried several times using different options without any success. The gene list from Hs_Derby_ensemble has been loaded in the program prior to the samples sheet. 

The version of the chip I have used for the microarray analysis is Human ht-12_v4_0_r2_15002873_b.

I hope somebodfy can lifet me up from this situation.

Regards

[Kristina Hanspers]

Hi Petar,

Can you send a few rows of your input file? 

Thanks,

Kristina

--
You received this message because you are subscribed to the Google Groups "wikipathways-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-dis...@googlegroups.com.
To post to this group, send email to wikipathwa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

[Petar Donchev]

Dear Hristina,

Thank you for reply.

Please, find attached an example sheet of my data with the respective accession names (ILMN_; 10008 ...)

Kind regards,

Petar

2017-12-06 21:02 GMT+02:00 Kristina Hanspers <kristina...@gladstone.ucsf.edu>:

Hi Petar,

Can you send a few rows of your input file? 

Thanks,

Kristina

On Dec 5, 2017, at 4:37 AM, Petar Donchev <donc...@gmail.com> wrote:

Dear All,

I am having issue with data import from Illumina microarray analysis. The data is preprocessed file containing the ID of the probes (numerical) and ILMN_ identifier, as well as normalised expression values and FoldChange information (comparison between control and sample).

I have saved the file as text (tab delimited) but the import step gives me information that all of the imput lines are ecceptions and and the end the window says that I have all 48102 lines unrecognized. 

I have tried several times using different options without any success. The gene list from Hs_Derby_ensemble has been loaded in the program prior to the samples sheet. 

The version of the chip I have used for the microarray analysis is Human ht-12_v4_0_r2_15002873_b.

I hope somebodfy can lifet me up from this situation.

Regards

--
You received this message because you are subscribed to the Google Groups "wikipathways-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsub...@googlegroups.com.
To post to this group, send email to wikipathways-discuss@googlegroups.com.

Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "wikipathways-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wikipathways-discuss/HZo62y10SUw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsub...@googlegroups.com.
To post to this group, send email to wikipathways-discuss@googlegroups.com.

[Martina Summer-Kutmon]

Dear Petar,

Please be aware that you need to select the column that contains the full ILMN_ probe id, which is not the first column! See screenshot attached.

Then the recognition of at least some of the probe ids work - not all will be recognized because they might not link to any genes. 

Hope this helps!

Best, Tina

On Thu, Dec 7, 2017 at 1:10 PM, Petar Donchev <donc...@gmail.com> wrote:

Dear Hristina,

Thank you for reply.

Please, find attached an example sheet of my data with the respective accession names (ILMN_; 10008 ...)

Kind regards,

Petar

2017-12-06 21:02 GMT+02:00 Kristina Hanspers <kristina.hanspers@gladstone.ucsf.edu>:

Hi Petar,

Can you send a few rows of your input file? 

Thanks,

Kristina

On Dec 5, 2017, at 4:37 AM, Petar Donchev <donc...@gmail.com> wrote:

Dear All,

I am having issue with data import from Illumina microarray analysis. The data is preprocessed file containing the ID of the probes (numerical) and ILMN_ identifier, as well as normalised expression values and FoldChange information (comparison between control and sample).

I have saved the file as text (tab delimited) but the import step gives me information that all of the imput lines are ecceptions and and the end the window says that I have all 48102 lines unrecognized. 

I have tried several times using different options without any success. The gene list from Hs_Derby_ensemble has been loaded in the program prior to the samples sheet. 

The version of the chip I have used for the microarray analysis is Human ht-12_v4_0_r2_15002873_b.

I hope somebodfy can lifet me up from this situation.

Regards

--
You received this message because you are subscribed to the Google Groups "wikipathways-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsubscrib...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "wikipathways-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wikipathways-discuss/HZo62y10SUw/unsubscribe.

To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsubscrib...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "wikipathways-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsub...@googlegroups.com.
To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

--

-----------------------------------------------
Martina Summer-Kutmon, PhD
email: mku...@gmail.com
skype: mkutmon

[Petar Donchev]

Dear Martina,

I tried with this column, but unfortunately on my side there is some problem. 

As the matter of fact I am using a simplified version of the example I sent you, which contains less number of columns. But the accession data is same and the expression values together with the fold change.

I will give it a try and will come back to you with response whether it works or not.

Thank you for your support.

petar

-----------------------------------------------
Martina Summer-Kutmon, PhD
email: mku...@gmail.com
skype: mkutmon

--

You received this message because you are subscribed to a topic in the Google Groups "wikipathways-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wikipathways-discuss/HZo62y10SUw/unsubscribe.

To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsub...@googlegroups.com.

[Kristina Hanspers]

Hi Petar,

In my test, 4 out of the 8 example probes you sent import. The rest are not recognized by our database, which is built based on Ensembl. Its possible Ensembl did not include these probes for some reason. 

Regards,

Kristina

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-dis...@googlegroups.com.
To post to this group, send email to wikipathwa...@googlegroups.com.

[Petar Donchev]

Dear Tina,

That is correct. 

I am having 48105 lines (genes) and more than 11 000 unrecognized genes. Nevertheless there are 'empty' line in the data - some sort of controls for negative background which cannot be uploaded in PathVisio. The rest are the 'unstable" accessions that change from version to version and are not very specific because some genes are not very well described. These genes are given names like e.g. LOC14234. They are not important for me. 

However I am getting the following message whenever I am trying to select a particular pathway from the 'Search' tab on the right:

Error message:

class java.lang.reflect.InvocationTargetException: null

See error log for details

class org.pathvisio.core.model.ConverterException

Have you any idea why this message appears?

KInd regards,

Petar

2017-12-07 20:58 GMT+02:00 Kristina Hanspers <kristina...@gladstone.ucsf.edu>:

Hi Petar,

In my test, 4 out of the 8 example probes you sent import. The rest are not recognized by our database, which is built based on Ensembl. Its possible Ensembl did not include these probes for some reason. 

Regards,

Kristina

On Dec 7, 2017, at 4:48 AM, Petar Donchev <donc...@gmail.com> wrote:

Dear Martina,

I tried with this column, but unfortunately on my side there is some problem. 

As the matter of fact I am using a simplified version of the example I sent you, which contains less number of columns. But the accession data is same and the expression values together with the fold change.

I will give it a try and will come back to you with response whether it works or not.

Thank you for your support.

petar

Kristina

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsub...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to a topic in the Google Groups "wikipathways-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wikipathways-discuss/HZo62y10SUw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsub...@googlegroups.com.
To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "wikipathways-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsub...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

-- 

-----------------------------------------------
Martina Summer-Kutmon, PhD
email: mku...@gmail.com
skype: mkutmon

-- 
You received this message because you are subscribed to a topic in the Google Groups "wikipathways-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wikipathways-discuss/HZo62y10SUw/unsubscribe.

To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsubscr...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "wikipathways-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsub...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

[Petar Donchev]

Dear Hristina,

I`ve talked to the bioinformatition who provided me with the expression data from Illumina beadchip. He told me that may be I could use the real gene names instead the accession ID - ILMN_. For unknown reasons the ensemble_bridge(85) from Sept 2016 does not have the correct annotation database which is why I am getting 12 000 genes unrecognized by PathVisio (the import file has 48 000 genes - which makes those 12 000 app. 1/4 of all samples). 

In this respect if I want to upload a list of genes, what would be the System Code abbreviation I need to introduce in the file?

Thank you for your supprt.

Kind regards,

Petar

2017-12-08 19:42 GMT+02:00 Petar Donchev <donc...@gmail.com>:

Dear Tina,

That is correct. 

I am having 48105 lines (genes) and more than 11 000 unrecognized genes. Nevertheless there are 'empty' line in the data - some sort of controls for negative background which cannot be uploaded in PathVisio. The rest are the 'unstable" accessions that change from version to version and are not very specific because some genes are not very well described. These genes are given names like e.g. LOC14234. They are not important for me. 

However I am getting the following message whenever I am trying to select a particular pathway from the 'Search' tab on the right:

Error message:

class java.lang.reflect.InvocationTargetException: null

See error log for details

class org.pathvisio.core.model.ConverterException

Have you any idea why this message appears?

KInd regards,

Petar

Kristina

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsubscr...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to a topic in the Google Groups "wikipathways-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wikipathways-discuss/HZo62y10SUw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "wikipathways-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsubscr...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

-- 

-----------------------------------------------
Martina Summer-Kutmon, PhD
email: mku...@gmail.com
skype: mkutmon

-- 
You received this message because you are subscribed to a topic in the Google Groups "wikipathways-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wikipathways-discuss/HZo62y10SUw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "wikipathways-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsubscr...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to a topic in the Google Groups "wikipathways-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wikipathways-discuss/HZo62y10SUw/unsubscribe.

To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsubscrib...@googlegroups.com.

[Petar Donchev]

Dear Tina,

I have found where the problem with Illumin HT_12_v4 is. Some of the annotations are actually linked to one same gene, but represent a different reading frame (ORF or 3UNT). That is why if somebody wants to translate the information into a regulare accessions first one should convert the annotations to nuID which later could be translated to Entrez gene or other format. 

https://support.illumina.com/array/array_kits/humanht-12_v4_expression_beadchip_kit/downloads.htmlhttps://www.google.bg/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0ahUKEwjkwKbZsozYAhWCWhQKHX5PBncQFggyMAE&url=http%3A%2F%2Fwww.bioconductor.org%2Fpackages%2F%2F2.11%2Fbioc%2Fvignettes%2Flumi%2Finst%2Fdoc%2FIlluminaAnnotation.pdf&usg=AOvVaw3-kY65ZMR250JkvRjdL3aK

Unfortunately I have no access to R server, which could be run with different R packages. The latter is very important ,because the lumi package runs only with R-package v2.5.0 which does not start with R-studio desctop free version.

In this regards, could you please help me out by running the script in described in the file: Resolve the inconsistency with Illumina identifiers through nuID"?

You just need to upload the manifest file from the link above. The institution I am working for has no access to R-server and basically I am locked in this position.

I would very much appreciate your help.

KInd regards,

Petar

[Martina Summer-Kutmon]

Dear Petar,

Sorry, somehow I missed this email. Unfortunately, I don't have any experience with R servers. Sorry! 

If you want to use the gene names (column "Symbol"), you need to use "HGNC" as the system code. 

Best regards,

Tina

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsub...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

[Petar Donchev]

Dear Tina,

Thank you for your e-mail.

I wish you a Happy New Year full of joy, warmth, travel and more time for friends and familly.

Kind regards,

Petar

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsubscrib...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

--

-----------------------------------------------
Martina Summer-Kutmon, PhD
email: mku...@gmail.com
skype: mkutmon

--
You received this message because you are subscribed to a topic in the Google Groups "wikipathways-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wikipathways-discuss/HZo62y10SUw/unsubscribe.

To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsub...@googlegroups.com.

[Petar Donchev]

Dear Tina,

It has been several months since I successfully managed to upload my data into PathVisio and run the prelimenary analysis. 

Later on I found out that my p-values are numbers between 0 and 1 which is in contradiction with the file from example 2 - the p-values are either positive or negative, but never 0. 

That`s why I preprocesed my data and prefiltered it in order to get a shorter list. Than I uploaded it PathVisio. Unfortunately for some reason the program is not able to calculate the z-score which as consequence gives no results from the network analysis. 

In this respect can I ask you for advice what might have gone wrong in my attempt to do the analysis?

I am attaching 10 rows from my data. Column K contains fold change with range 0 to infinity. Columne L contains positive and negative values, as those described in Tutorial 2 on PathVisio web page.

Kind regards,

Petar

[Martina Summer-Kutmon]

Dear Petar,

I will have a look but p-value values between 0 and 1 are correct. That's how p-values should look like and PathVisio should not have any problems with that. So no need to further preprocess your data. 

In example 2, the corresponding columns are "rawp" and "adjp" which are also between 0 and 1. 

You should not filter your file beforehand - since all data is required for over-representation analysis. The filtering happens through setting the criteria in the pathway statistics dialog - there you decide which genes are of interest in your study (e.g. "[Fold EB vs ES] > 1.2 AND [rawp] < 0.05" --> significantly up-regulated genes). 

Best, Tina

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsub...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

[Petar Donchev]

Dear Tina,

Thank you for your quick reply. 

I`ve tried to run the analysis with the data given in Tutorial 2. I was surprised to find out that PathVisio is giving me NaN for z-score - same way as with my own data. I tried using fold change, log fold change, as well as both - adjusted and raw p values. Unfortunately the results is the same. And if I am not able to get the z-score than the analysis is not completed. 

Is there any setting in the software that might be missing on my side that as a results is messing up with my analysis?

Kind regards,

Petar

To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsub...@googlegroups.com.

[Martina Summer-Kutmon]

Dear Petar,

So even with the tutorial dataset your Z-scores are NaN values? 

Are you sure that you loaded the identifier mapping database that takes care of mapping the probe ids to the gene ids in the pathways? 

Best, Tina

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsub...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

[Petar Donchev]

Dear Martina,

Thank you for your e-mail.

Later when I re-read my e-mail I realized that I was trying to run the statistics using the human mapping database. Therefore I had it replaced with Mm-Lite bridge file provided within the Tutorial 2 files. And it worked. I was able to get z-scores for the example data set. 

Than again I auploaded the human mapping database that I am using together with my expression data set. I am still getting NaN. 

Have you tried to do the statistical analysis with the exerpt data file I shared in my previous e-mail. Probably I am repeating same mistake but am not able to see it.

Thank you for your help.

Kind regards,

Petar

To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsub...@googlegroups.com.

[Petar Donchev]

Dear Martina,

I have found the culprit. PathVisio is not able to calculated z-score an p-values when I am using brackets in my formula (argument). 

That said I managed to do the statistical analysis and to export all of the relevant networks as gpml files.

However I am facing another issue with Cytoscape. 

Due to the fact, that some of the identifiers have not been recognised in PathVisio (unstable accessions) they appear missing in the node table in Cytoscape.

Respectively this is a problem when I want to export my merged networks as SIF file > the progam is giving me error message: 

"The node table contains null or empty node names"

I need to save the exported networks as sif file, because ths is the only way I can use the next important tool jActive Modules.

I know you have one very interesting article from 2015 about network analysis using PathVisio together with Cytoscape. 

Have you any idea how I can export my merged networks successfully as sif file in order to proceed with the next step of the analysis.

I hope you can give me an advice. 

Kind regards,

Petar

 

[Martina Summer-Kutmon]

Dear Petar,

I am sorry, I was on vacation.

If I assume correctly, you imported the relevant GPML files with the WikiPathways app in Cytoscape. So you have a merged network in Cytoscape? Why do you need to export it to SIF? You can simply run the jActiveModules app in Cytoscape. 

Or did you not yet create the merged network in Cytoscape (and if yes, how did you create it?). 

Tina

To unsubscribe from this group and stop receiving emails from it, send an email to wikipathways-discuss+unsub...@googlegroups.com.

To post to this group, send email to wikipathways-discuss@googlegroups.com.
Visit this group at https://groups.google.com/group/wikipathways-discuss.
For more options, visit https://groups.google.com/d/optout.

[Petar Donchev]

Dear Martina,

Thank you for your response.

I did merged all imported networks. Thereafter I am trying to use jActiveModules, but the app is not recognizing the netowrk, untill it is saved as SIF file. I tried numerous times but couldn`t manage using this app without saving the file as sif. 

Have you any idea what I might be doing wrong so the app is not functioning properly?

Thank you for your help.

Kind regards,

Petar

To unsubscribe from this group and all its topics, send an email to wikipathways-discuss+unsub...@googlegroups.com.

[nourana...@gmail.com]

Dear all,

I'm having a similar problem as I'm importing 739 and I have 46 unidentified proteins using the accession number on Uniprot trmbl system code and gene data has a Mm_Derby_Ensembl_91.bridge. If I ignore this problem and I move on for statistical analysis, I will have 540 proteins recognized in pathways only, which means I'm losing more proteins. 

I tried using gene names with HGNC system code, none of the proteins were recognized. I also tried adding _MOUSE to the gene name on Uniprot system code, but nothing happened as well. 

I will highly appreciate any help!