Clinical data visibility in study

[Malatpure, Abhijeet]

Hello,

I am reposting my question from the slack channel - What are the minimum files necessary for the study's clinical data to be loaded and displayed in the 'Clinical Data' tab correctly? I was able to create a cBioPortal study with the following files, but the front end shows no records on the 'Clinical Data' tab (headers from my data_clinical_patient file are present)-

 

case_lists/

data_clinical_patient.txt

data_clinical_sample.txt

dt_upload.html

meta_clinical_patient.txt

meta_clinical_sample.txt

meta_study.txt

 

Another noticeable behavior is the loading icon on top near right keeps looping. It should display 'Selected: nnnn patients | mmmm samples' .

 

 

Any help would be much appreciated.

 

Regards,

Abhijeet Malatpure

Advanced Biomedical IT Core (ABITC)

ABITC is part of the Research Technologies Division of UITS

Indianapolis, IN 46202

Office: (317)278-7970

Email: abhm...@iu.edu

 

[Sjoerd van Hagen]

Hi Abhijeet,

Is the data showing up correctly in the study view and in other places in cBioPortal? In that case, we may be looking at a bug.

Could you run the validator on your input files and post the results here?

Best,

Sjoerd.

--

E.   sjo...@thehyve.nl

T.   +31(0)30 700 9713

W.  www.thehyve.nl

--
You received this message because you are subscribed to the Google Groups "cBioPortal for Cancer Genomics Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cbioportal+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cbioportal/31f396dbb29046fcb172f722b49b342c%40IN-CCI-D2S14.ads.iu.edu.

[Malatpure, Abhijeet]

Hi Sjoerd,

Thank you for looking into this. The study view shows charts, but a few of them have errors in them; screenshot attached. The txt file attached has validator input.

 

The six files for the study are taken from github datahub/public/chol_tcga_pan_can_atlas_2018 and are unchanged.

 

Regards,

Abhi

abhm...@iu.edu

 

From: Sjoerd van Hagen <sjo...@thehyve.nl>
Sent: Sunday, August 16, 2020 5:05 AM
To: Malatpure, Abhijeet <abhm...@iu.edu>
Cc: cbiop...@googlegroups.com
Subject: [External] Re: [cbioportal] Clinical data visibility in study

 

This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources.

[Sjoerd van Hagen]

Hi Abhijeet,

I am putting my colleague Sander on CC. Hopefully he can help you with this.

Best,

Sjoerd.

--

E.   sjo...@thehyve.nl

T.   +31(0)30 700 9713

W.  www.thehyve.nl

[Malatpure, Abhijeet]

Thank you, Sjoerd. I’m curious- were you able to replicate the behavior on your end?

Hi Sander,

Please let me know if you need more information; happy to send it your way.

Regards,

Abhi

On Aug 18, 2020, at 5:46 AM, Sjoerd van Hagen <sjo...@thehyve.nl> wrote:



<image001.png>

 

Any help would be much appreciated.

 

Regards,

Abhijeet Malatpure

Advanced Biomedical IT Core (ABITC)

ABITC is part of the Research Technologies Division of UITS

Indianapolis, IN 46202

Office: (317)278-7970

Email: abhm...@iu.edu

<image002.png>

 

[Jong Cheol Jeong]

I had same issues, and you can go around by adding at least a dummy mutation data. 

To view this discussion on the web visit https://groups.google.com/d/msgid/cbioportal/76298E6F-7CE8-4D81-8F4A-CF4626C37A43%40iu.edu.
<image001.png><image002.png>

[Malatpure, Abhijeet]

Hi Jong,

I reuploaded chol_tcga_pan_can_atlas_2018 study with mutations data included (indicated in orange filenames below):

 

case_lists/

cases_all.txt

cases_sequenced.txt

data_clinical_patient.txt

data_clinical_sample.txt

data_mutations_extended.txt

meta_clinical_patient.txt

meta_clinical_sample.txt

meta_mutations_extended.txt

meta_study.txt

 

but that didn’t get rid of the issue. Clinical Data tab still shows zero rows.

 

Regards,

Abhi

317-278-7970

[Sjoerd van Hagen]

Hi Abhijeet,

Does it try to load the study without making any changes at all to see if that works?

Best,

Sjoerd.

--

E.   sjo...@thehyve.nl

T.   +31(0)30 700 9713

W.  www.thehyve.nl

[Gaofei Zhao]

Hi Abhijeet,

Based on the study view screenshot, I think this may related to our data format update previously, 

Can you check if there is data prefix with "1:" or "0:" in your data_clinical_patient.txt .

If you don't have any data with those prefix which means you may need to migrate your data_clinical_patient.txt to new format. 

There are two options to update:

1. if you haven't changed anything in data_clinical_patient.txt:

You can download the latest file from datahub: https://media.githubusercontent.com/media/cBioPortal/datahub/master/public/chol_tcga_pan_can_atlas_2018/data_clinical_patient.txt

2. if you changed content in data_clinical_patient.txt:

You can try to migrate your file by using this tool: https://github.com/cBioPortal/datahub-study-curation-tools/tree/master/Survival_Data_Migration

Gaofei

[Malatpure, Abhijeet]

A docker container restart fixed it! Thank you all for all your inputs. So it seems like clinical, sample and mutation files are the minimum set of files required for a study’s clinical data and patient/sample counts to show up.

 

Sjoerd,

I’m not sure if this classifies as a bug or simply a documentation change where this can be listed as expected behavior. Please let me know if there is a way for me to contribute.

 

Regards,

Abhi

317-278-7970

[Sjoerd van Hagen]

Hi Abhijeet,

This should be either documented or fixed indeed. I actually thought it was fixed because I do think we have loaded studies without mutations before.

@Sander Rodenburg can you check, and file an issue if needed?

Best,

Sjoerd.

--

E.   sjo...@thehyve.nl

T.   +31(0)30 700 9713

W.  www.thehyve.nl

[ino4pr...@gmail.com]

Thanks for reporting!

There is an issue open to fix this: https://github.com/cBioPortal/cbioportal/issues/7814

Thanks,

Ino

[korjc...@gmail.com]

I would like to bring up this issue again.
I have recently found that cBioPortal doesn't report any mutations if no mutation appears in the extended mutation staging file even if a sample has copy number variants. This may be extremely rare in WES and WGS, but it can be very problematic for gene panel based sequencing.
I wonder if cBioPortal team can revisit this issue?

[Sjoerd van Hagen]

Hi,

I am not sure I understand your question. If there are no mutations in the staging file, how can cBioPortal report them?

Best,

Sjoerd.

---

Sjoerd van Hagen

Team Lead cBioPortal & Open Targets

E sjo...@thehyve.nl

T +31 30 700 9713

W thehyve.nl

    

To view this discussion on the web visit https://groups.google.com/d/msgid/cbioportal/e408827e-53cc-4205-9a92-9679adba4f21n%40googlegroups.com.

[Jong Cheol Jeong]

Sjoerd, 

My apologies for the poor description. 

The extended mutation file in cBioPortal contains short variants like SNPs. However, there are other mutation types like CNV, gene fusions, and gene expressions etc. and as far as you know, cBioPortal requires different staging files (https://docs.cbioportal.org/5.1-data-loading/data-loading/file-formats#continuous-copy-number-data) 

The issue that I mentioned was that there is no reportable short variant (e.g., SNPs) in the mutation data, but there exists mutations in CNV, fusions etc. 

cBioPortal ignores all other mutation types if there is no data in the mutation data (https://docs.cbioportal.org/5.1-data-loading/data-loading/file-formats#mutation-data). Basically it doesn't show CNV and other variants even if there exists. 

For gene panel-based sequencing (e.g., Foundation Medicine), it is not unusual to see a sample with no reportable short variant but having CNV or fusions. 

I hope this can help you to better understand this issue. 

Thank you. 

JC

[Sjoerd van Hagen]

Hi JC,

I see. You are saying that if there are no mutations in the MAF file for a particular sample, cBioPortal also does not show CNV and fusions, even though they are present in their respective staging files. That would be a bug.

I think there was a problem with this before where samples without mutations were not being loaded. However, I think you are saying the samples are being loaded but incompletely. @de Bruijn, Ino/Sloan Kettering Institute does this sound familiar to you?

Best,

Sjoerd.

---

Sjoerd van Hagen

Team Lead cBioPortal & Open Targets

E sjo...@thehyve.nl

T +31 30 700 9713

W thehyve.nl

    

[Jong Cheol Jeong]

Exactly!

Thank you for your time for looking into this issue. 

JC

[Sjoerd van Hagen]

@de Bruijn, Ino/Sloan Kettering Institute any ideas here?

---

Sjoerd van Hagen

Team Lead cBioPortal & Open Targets

E sjo...@thehyve.nl

T +31 30 700 9713

W thehyve.nl

    

[de Bruijn, Ino/Sloan Kettering Institute]

Thanks for the reminder!

 

I don’t know exactly why this is the case, but filed an issue:

 

https://github.com/cBioPortal/cbioportal/issues/8612

 

We will investigate this further

 

Are you seeing this issue on the study view, results view, patient view or on all three?

 

Thanks,

Ino

 

From: Sjoerd van Hagen <sjo...@thehyve.nl>
Date: Wednesday, May 19, 2021 at 12:04 PM
To: Jong Cheol Jeong <korjc...@gmail.com>
Cc: "de Bruijn, Ino/Sloan Kettering Institute" <debr...@mskcc.org>, cBioPortal for Cancer Genomics Discussion Group <cbiop...@googlegroups.com>
Subject: Re: [External] [cbioportal] Clinical data visibility in study

 

@de Bruijn, Ino/Sloan Kettering Institute any ideas here?

 

---

 

Sjoerd van Hagen

Team Lead cBioPortal & Open Targets

 

E sjo...@thehyve.nl

T +31 30 700 9713

W thehyve.nl

 

    

 

 

On Tue, May 11, 2021 at 10:02 PM Jong Cheol Jeong <korjc...@gmail.com> wrote:

Exactly!

Thank you for your time for looking into this issue. 

 

 

JC

On May 11, 2021, at 3:57 PM, Sjoerd van Hagen <sjo...@thehyve.nl> wrote:

 

Hi JC,

 

I see. You are saying that if there are no mutations in the MAF file for a particular sample, cBioPortal also does not show CNV and fusions, even though they are present in their respective staging files. That would be a bug.

 

I think there was a problem with this before where samples without mutations were not being loaded. However, I think you are saying the samples are being loaded but incompletely. @de Bruijn, Ino/Sloan Kettering Institute does this sound familiar to you?

 

Best,

 

Sjoerd.

 

---

 

Sjoerd van Hagen

Team Lead cBioPortal & Open Targets

 

E sjo...@thehyve.nl

T +31 30 700 9713

W thehyve.nl

 

    

 

 

On Tue, May 11, 2021 at 9:34 PM Jong Cheol Jeong <korjc...@gmail.com> wrote:

Sjoerd, 

 

My apologies for the poor description. 

 

The extended mutation file in cBioPortal contains short variants like SNPs. However, there are other mutation types like CNV, gene fusions, and gene expressions etc. and as far as you know, cBioPortal requires different staging files (https://docs.cbioportal.org/5.1-data-loading/data-loading/file-formats#continuous-copy-number-data) 

The issue that I mentioned was that there is no reportable short variant (e.g., SNPs) in the mutation data, but there exists mutations in CNV, fusions etc. 

cBioPortal ignores all other mutation types if there is no data in the mutation data (https://docs.cbioportal.org/5.1-data-loading/data-loading/file-formats#mutation-data). Basically it doesn't show CNV and other variants even if there exists. 

For gene panel-based sequencing (e.g., Foundation Medicine), it is not unusual to see a sample with no reportable short variant but having CNV or fusions. 

 

I hope this can help you to better understand this issue. 

 

Thank you. 

 

JC

On May 11, 2021, at 3:17 PM, Sjoerd van Hagen <sjo...@thehyve.nl> wrote:

 

Hi,

 

I am not sure I understand your question. If there are no mutations in the staging file, how can cBioPortal report them?

 

Best,

 

Sjoerd.

 

---

 

Sjoerd van Hagen

Team Lead cBioPortal & Open Targets

 

E sjo...@thehyve.nl

T +31 30 700 9713

W thehyve.nl

 

    

 

 

On Tue, May 11, 2021 at 8:56 PM korjc...@gmail.com <korjc...@gmail.com> wrote:

I would like to bring up this issue again.
I have recently found that cBioPortal doesn't report any mutations if no mutation appears in the extended mutation staging file even if a sample has copy number variants. This may be extremely rare in WES and WGS, but it can be very problematic for gene panel based sequencing.
I wonder if cBioPortal team can revisit this issue?

 

 

On Thursday, August 27, 2020 at 3:56:32 PM UTC-4 ino4pr...@gmail.com wrote:

Thanks for reporting!

 

There is an issue open to fix this: https://github.com/cBioPortal/cbioportal/issues/7814

 

Thanks,

Ino

On Friday, August 21, 2020 at 5:18:51 AM UTC-4 Sjoerd van Hagen wrote:

Hi Abhijeet,

 

This should be either documented or fixed indeed. I actually thought it was fixed because I do think we have loaded studies without mutations before.

 

@Sander Rodenburg can you check, and file an issue if needed?

 

Best,

 

Sjoerd.

--

E.   sjo...@thehyve.nl

T.   +31(0)30 700 9713

W.  www.thehyve.nl

On Tue, Aug 18, 2020 at 6:44 PM Malatpure, Abhijeet <abhm...@iu.edu> wrote:

A docker container restart fixed it! Thank you all for all your inputs. So it seems like clinical, sample and mutation files are the minimum set of files required for a study’s clinical data and patient/sample counts to show up.

 

Sjoerd,

I’m not sure if this classifies as a bug or simply a documentation change where this can be listed as expected behavior. Please let me know if there is a way for me to contribute.

 

Regards,

Abhi

317-278-7970

 

From: Sjoerd van Hagen <sjo...@thehyve.nl> 
Sent: Tuesday, August 18, 2020 11:40 AM
To: Malatpure, Abhijeet <abhm...@iu.edu>
Cc: Jong Cheol Jeong <korjc...@gmail.com>; cbiop...@googlegroups.com; Sander Rodenburg <sanderr...@thehyve.nl>
Subject: Re: [External] [cbioportal] Clinical data visibility in study

 

Hi Abhijeet,

 

Does it try to load the study without making any changes at all to see if that works?

 

Best,

 

Sjoerd.

--

--

E.   sjo...@thehyve.nl

T.   +31(0)30 700 9713

W.  www.thehyve.nl

On Mon, Aug 17, 2020 at 3:26 PM Malatpure, Abhijeet <abhm...@iu.edu> wrote:

Hi Sjoerd,

Thank you for looking into this. The study view shows charts, but a few of them have errors in them; screenshot attached. The txt file attached has validator input.

 

The six files for the study are taken from github datahub/public/chol_tcga_pan_can_atlas_2018 and are unchanged.

 

Regards,

Abhi

abhm...@iu.edu

 

From: Sjoerd van Hagen <sjo...@thehyve.nl> 
Sent: Sunday, August 16, 2020 5:05 AM
To: Malatpure, Abhijeet <abhm...@iu.edu>
Cc: cbiop...@googlegroups.com
Subject: [External] Re: [cbioportal] Clinical data visibility in study

 

This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources.

 

Hi Abhijeet,

 

Is the data showing up correctly in the study view and in other places in cBioPortal? In that case, we may be looking at a bug.

 

Could you run the validator on your input files and post the results here?

 

Best,

 

Sjoerd.

--

 

 

*** Only open attachments or links from trusted senders. Report phishing to inf...@mskcc.org ***

 

=====================================================================

Please note that this e-mail and any files transmitted from
Memorial Sloan Kettering Cancer Center may be privileged, confidential,
and protected from disclosure under applicable law. If the reader of
this message is not the intended recipient, or an employee or agent
responsible for delivering this message to the intended recipient,
you are hereby notified that any reading, dissemination, distribution,
copying, or other use of this communication or any of its attachments
is strictly prohibited. If you have received this communication in
error, please notify the sender immediately by replying to this message
and deleting this message, any attachments, and all copies and backups
from your computer.

[korjc...@gmail.com]

Thank you for working on this issue.

I couldn't find any updates from https://github.com/cBioPortal/cbioportal/issues/8612

Could you give me some updates on this issue?

Are you still investigating or is it fixed already?

Again, thank you for your help.

JC

[Ino de Bruijn]

Hi JC,

Thanks for checking in

Could you clarify whether you see this issue on the study view, results view, patient view or on all three?

Best wishes,

Ino

To view this discussion on the web visit https://groups.google.com/d/msgid/cbioportal/6ced0467-3104-4c7c-9d8c-a94c6b20d9bfn%40googlegroups.com.

[Jong Cheol Jeong]

I am not sure what the results view mean, but this patient has CNV without Short variants (MAF format)

As shown in the attached file, the patient appears in study view, but doesn’t show anything in patient view. 

Thank you for your help. 

JC