we must create open datasets

54 views
Skip to first unread message

Raamana, Pradeep Reddy

unread,
Jul 21, 2021, 12:19:57 PM7/21/21
to niQC

As I discuss niQC with various folks, a common, recurring, and urgent need connecting various QC tasks (auto QC, phantoms etc) is the need for open datasets sharing the relevant imaging and ratings data, so methods researchers (esp. those not at imaging centers) can target their brilliance towards. Hence, I encourage you all to encourage yourself and peers to share and publish any and all data that can be used to improve niQC.

 

Most of you already know datasets can be published in a few places now, but I would encourage everyone to mention and prefer the OHBM Aperture journal, for this purpose.

 

This is a necessary first step towards e.g. creating a central repository that Yarik mentioned.

 

Thanks,

Pradeep

JB Poline

unread,
Jul 21, 2021, 12:33:41 PM7/21/21
to Raamana, Pradeep Reddy, niQC
100% behind this idea and the importance of it - as open data availability increases, our next step is definitely making QC open and connected - and thanks for the Aperture plugin Pradeep !

--
You received this message because you are subscribed to the Google Groups "niQC" group.
To unsubscribe from this group and stop receiving emails from it, send an email to niqc+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/niqc/DM6PR04MB67461F398CD8DCE1D5D63044B8E39%40DM6PR04MB6746.namprd04.prod.outlook.com.

Lei Wang

unread,
Jul 21, 2021, 1:02:48 PM7/21/21
to jbpo...@gmail.com, Raamana, Pradeep Reddy, niQC

Agreed.

 

Pradeep – when is the Phantom meeting?

 

--Lei

 

--
Lei Wang, Ph.D., Lei....@osumc.edu

Psychiatry and Behavioral Health, Neuroscience
Ohio State University Wexner Medical Center

 

Psychiatry and Behavioral Sciences

Northwestern University Feinberg School of Medicine

fsm-signature

Raamana, Pradeep Reddy

unread,
Jul 21, 2021, 1:08:48 PM7/21/21
to Lei Wang, jbpo...@gmail.com, niQC

Thanks Lei – the phantom meeting is next Tuesday 4-6pm EST (or longer for some if needed). I encourage everyone to subscribe to the niQC SIG calendar: https://calendar.google.com/calendar/u/0?cid=cHJyYjg0a2FqZDIxYmlla21tN3FmYzNpZmtAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ

Raamana, Pradeep Reddy

unread,
Jul 22, 2021, 8:11:15 PM7/22/21
to jbpo...@gmail.com, niQC

Thanks JB! There is still one significant barrier towards the creation of open QC datasets (besides the effort to QC them before sharing etc): Data Usage Agreements (for ADNI, ABCD, etc) with a “Do Not Redistribute” (DNR) clause preventing us from openly sharing modified imaging data or associated QC ratings. Some folks are working around it (e.g. ABCD effort at some crowdsourcing by making the interested parties collaborators at UMN etc), but this is an unnecessary pain that is not justified. I haven’t seen a solid rationale for this beyond vague concerns for privacy etc, that can easily be addressed without carpet bombing our ability to redistribute privacy-safe versions of the datasets or their derivatives.

 

I did reach out to some NIH Data Sharing officers, but they didn’t really help (more like they shut me down). But with some help from here and right contacts, I would like discuss ways to remove or significantly modify the DNR clause for big datasets created with public funding.

 

PS: if needed, I am happy to organize a debate or brainstorming session for this – let me know if any of you are interested.

 

From: JB Poline <jbpo...@gmail.com>
Date: Wednesday, July 21, 2021 at 12:33 PM
To: Raamana, Pradeep Reddy <RAAM...@pitt.edu>
Cc: niQC <ni...@googlegroups.com>
Subject: Re: [niQC] we must create open datasets

Cyril Pernet

unread,
Aug 18, 2021, 10:37:29 AM8/18/21
to niQC
 Dear all,

following up on the last few weeks discussion and Yarik provocative results , Patrick Fisher (who joined the google group) and myself have decided to implement a weekly QC for fMRI for our 2 machines (a Prisma and a PET/MR). Currently the clinical people do a standard T1/T2 to check for obvious artefacts but that's about it - so we'll pump this up

Here is our plan, but I'd like the input from the group since we will also share those data and this can be used for some work we want to do

- 1/2 hour max including set-up

- standard phantom (need to find out exactly what we have - I'm new to the lab in Copenhagen ...) but we wouldn't buy anything new

- scan once a week but sample pseudorandomly (ie not just all Monday AM, or all Friday PM)

- T1/T2 with the same parameters as what is typically done here for our research protocols

- T2*EPI for BOLD; here I was thinking of 2 runs, one matching the 'standard' fmri done here and one with all gizmos on at maximum (in plane (grappa) and in volume (sms) accelerations with the fastest TR possible) ; how long those should be? longer better I guess, temporal SNR variance could matter for dynamic FC? no idea

- should we do a DWI? - do we actually expect changes here not captured otherwise


Maybe others here would want to set-up an equivalent / same QC so we would have the 'same' data from different places

Cheers

Patrick and Cyril   

Raamana, Pradeep Reddy

unread,
Aug 18, 2021, 10:45:43 AM8/18/21
to Cyril Pernet, niQC

Thank you Cyril – could you elaborate more on if Patrick and you have any specific goals with this dataset? Replicate Yarik’s results?

 

Great idea to invite other imaging centers to join you as well, love it. I am checking if I can do it here at Pitt.

 

From: ni...@googlegroups.com <ni...@googlegroups.com> on behalf of Cyril Pernet <wamc...@gmail.com>
Date: Wednesday, August 18, 2021 at 10:37 AM
To: niQC <ni...@googlegroups.com>
Subject: Re: [niQC] we must create open datasets

Yaroslav Halchenko

unread,
Aug 18, 2021, 11:31:50 AM8/18/21
to ni...@googlegroups.com

On Wed, 18 Aug 2021, Cyril Pernet wrote:

> - should we do a DWI? - do we actually expect changes here not captured
> otherwise

FWIW, our tech/physicist do DWI as well, but not on a regular basis seems to
me. Asked on specific purpose. Sample(s) can be found on
http://datasets.datalad.org/?dir=/dbic/QA/sub-qa/ses-20191202/dwi

BTW -- a new feature of datasets.datalad.org -- for files we host (like in this
case) just hover on the file, get ⠇ and click on it to get context menu, then
you can open those .nii.gz a viewer e.g.
https://bioimagesuiteweb.github.io/webapp/viewer.html?image=https://datasets.datalad.org/dbic/QA/sub-qa/ses-20191202/dwi/sub-qa_ses-20191202_acq-DTIX30Xp2_dwi.nii.gz
for such a DWI. It is purely browser-client-side so file will be first
downloaded, but then you could even get through dynamics/volumes.

--
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW: http://www.linkedin.com/in/yarik

‪‪Dr Cyril Pernet‬‬

unread,
Aug 18, 2021, 12:10:10 PM8/18/21
to Raamana, Pradeep Reddy, niQC
Well the goal is indeed to be able to account for scanner variation Yarik's style 😁 forever.

I guess we could also shorter term write a short preregistration to replicate your work independently

Sent from my Huawei phone

Mary Beth Nebel

unread,
Aug 18, 2021, 2:09:09 PM8/18/21
to Raamana, Pradeep Reddy, jbpo...@gmail.com, niQC
Do you know if you can apply for access to the human phantom data collected by the ABCD or if it is included in any of the official data releases?

Thanks,
Mary Beth

Lei Wang

unread,
Aug 18, 2021, 2:14:58 PM8/18/21
to Mary Beth Nebel, Raamana, Pradeep Reddy, jbpo...@gmail.com, niQC

If folks want, I can help find out. We are working with the ABCD folks to set up our own multisite project.

 

--Lei

 

--
Lei Wang, Ph.D., Lei....@osumc.edu

Psychiatry and Behavioral Health, Neuroscience
Ohio State University Wexner Medical Center

 

Psychiatry and Behavioral Sciences

Northwestern University Feinberg School of Medicine

fsm-signature

 

Raamana, Pradeep Reddy

unread,
Aug 18, 2021, 2:36:20 PM8/18/21
to Lei Wang, Mary Beth Nebel, jbpo...@gmail.com, niQC

That would be great, Lei. Thanks.

Wang, Lei

unread,
Aug 18, 2021, 6:44:23 PM8/18/21
to Raamana, Pradeep Reddy, Mary Beth Nebel, jbpo...@gmail.com, niQC

A few updates from ABCD:

  • They are open to sharing non-human phantoms in principle, and will look into sharing them through the NDA (where the ABCD data are being shared). So more on that as it evolves.
  • Anders Dale mentioned that Joshua Kuperman leads an ABCD regular meeting on phantom data. They suggested that someone from the niQC group makes a guest appearance to connect our efforts. We can learn about their processing and QA/QC metrics. I will find out about the dates and times and connect with whomever is interested or nominated.

 

--Lei

 

--
Lei Wang, Ph.D., Lei....@osumc.edu

Psychiatry and Behavioral Health, Neuroscience
Ohio State University Wexner Medical Center

 

Psychiatry and Behavioral Sciences

Northwestern University Feinberg School of Medicine

fsm-signature

 

Raamana, Pradeep Reddy

unread,
Aug 18, 2021, 7:11:21 PM8/18/21
to Wang, Lei, Mary Beth Nebel, jbpo...@gmail.com, niQC
Awesome - I’m happy to meet them as needed. I did get in touch with Prof. Gary Glover (first author of fBIRN guidelines) who doesn’t seem to see a pressing need for updating those guidelines (my words, don’t quote him on that 😄). 

From: Wang, Lei <Lei....@osumc.edu>
Sent: Wednesday, August 18, 2021 6:44:09 PM
To: Raamana, Pradeep Reddy <RAAM...@pitt.edu>; Mary Beth Nebel <mb.n...@gmail.com>
Cc: jbpo...@gmail.com <jbpo...@gmail.com>; niQC <ni...@googlegroups.com>

Raamana, Pradeep Reddy

unread,
Aug 18, 2021, 7:13:38 PM8/18/21
to Wang, Lei, Mary Beth Nebel, jbpo...@gmail.com, niQC
Unless someone else is interested in going? like Dr. Mary Beth? Or anyone else? Let me know.

From: ni...@googlegroups.com <ni...@googlegroups.com> on behalf of Raamana, Pradeep Reddy <RAAM...@pitt.edu>
Sent: Wednesday, August 18, 2021 7:11:18 PM
To: Wang, Lei <Lei....@osumc.edu>; Mary Beth Nebel <mb.n...@gmail.com>

Mary Beth Nebel

unread,
Aug 18, 2021, 10:05:01 PM8/18/21
to Raamana, Pradeep Reddy, Wang, Lei, jbpo...@gmail.com, niQC
I'm definitely interested in learning more about the ABCD's internal QC. But I am a newbie to this group, and my particular interests maybe aren't representative of everyone else. 

I have been involved in efforts to implement ABCD-like sequences on the Philips scanners at our institute. As part of that effort, we've been discussing the attached figure on cross-talk saturation in SMS imaging. I think the ABCD fMRI sequence uses what the figure indicates is a suboptimal number of slice stacks (60 total slices/MB 6 = 10 stacks), so I'm curious if they see any evidence of cross-talk saturation in their phantom data. 

Does anyone know if this has been discussed elsewhere?

fig3_cross-talkSaturation.png

Thanks,
Mary Beth


Yaroslav Halchenko

unread,
Aug 19, 2021, 12:58:46 PM8/19/21
to niQC
I also would be interested to participate (even if only in the
listener-mode to not derail the meeting ;) ) if there would be no
time overlaps with other commitments.

On Wed, 18 Aug 2021, Mary Beth Nebel wrote:

> I have been involved in efforts to implement ABCD-like sequences on the
> Philips scanners at our institute. As part of that effort, we've been
> discussing the attached figure on cross-talk saturation in SMS imaging. I
> think the ABCD fMRI sequence uses what the figure indicates is
> a suboptimal number of slice stacks (60 total slices/MB 6 = 10 stacks), so
> I'm curious if they see any evidence of cross-talk saturation in their
> phantom data. 

indeed -- and should be possible to analyze to find support or lack of
it for such or similar "effects" (which might even not yet be
known/identified) if phantom data was available.

Raamana, Pradeep Reddy

unread,
Aug 19, 2021, 1:02:46 PM8/19/21
to Yaroslav Halchenko, niQC
Lei, it would be awesome if we get someone in that group to give a talk here [informal and easy] to the group so many of us can learn and discuss it.. Would that be possible?

From: ni...@googlegroups.com <ni...@googlegroups.com> on behalf of Yaroslav Halchenko <yarik...@gmail.com>
Sent: Thursday, August 19, 2021 12:58:42 PM
To: niQC <ni...@googlegroups.com>

Subject: Re: [niQC] we must create open datasets
I also would be interested to participate (even if only in the
listener-mode to not derail the meeting ;) ) if there would be no
time overlaps with other commitments.

On Wed, 18 Aug 2021, Mary Beth Nebel wrote:

>    I have been involved in efforts to implement ABCD-like sequences on the
>    Philips scanners at our institute. As part of that effort, we've been
>    discussing the attached figure on cross-talk saturation in SMS imaging. I
>    think the ABCD fMRI sequence uses what the figure indicates is
>    a suboptimal number of slice stacks (60 total slices/MB 6 = 10 stacks), so
>    I'm curious if they see any evidence of cross-talk saturation in their
>    phantom data. 

indeed -- and should be possible to analyze to find support or lack of
it for such or similar "effects" (which might even not yet be
known/identified) if phantom data was available.

--
Yaroslav O. Halchenko

Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755

--
You received this message because you are subscribed to the Google Groups "niQC" group.
To unsubscribe from this group and stop receiving emails from it, send an email to niqc+uns...@googlegroups.com.

Dr Cyril, Pernet

unread,
Aug 20, 2021, 2:25:13 AM8/20/21
to Raamana, Pradeep Reddy, niQC, Patrick Fisher

Hi all,

update - after discussion with Yarik we'll include a short DWI, to compute simple FA and ADC maps

--> still welcoming commenst on how long we should scan for fMRI / tSNR ?? @tom et al

+ we may as well prospectively register to replicate https://f1000research.com/articles/9-1131

thx

Cyril


- simple water phantom 1/2 hour scanning total

- T1/T2 with the same parameters as what is typically done here for our research protocols

- T2*EPI for BOLD; here I was thinking of 2 runs, one matching the 'standard' fmri done here and one with all gizmos on at maximum (in plane (grappa) and in volume (sms) accelerations with the fastest TR possible) ; how long those should be? longer better I guess, temporal SNR variance could matter for dynamic FC? no idea

- DWI, at least simple 6 directions or more depending on time given the fMRI scans

-- 
Dr Cyril Pernet, PhD, OHBM fellow, SSI fellow
Neurobiology Research Unit, 
Building 8057, Blegdamsvej 9
Copenhagen University Hospital, Rigshospitalet
DK-2100 Copenhagen, Denmark

wamc...@gmail.com
https://cpernet.github.io/
https://orcid.org/0000-0003-4010-4632

Raamana, Pradeep Reddy

unread,
Aug 20, 2021, 7:47:34 AM8/20/21
to Dr Cyril, Pernet, niQC, Patrick Fisher
Cool! How long are you planing to acquire this dataset? 3, 6 or 12 months?

Yarik, do we need any other specific hypotheses / experiments on this dataset towards the “Phantom data matter” paper we’re thinking about? 



From: ni...@googlegroups.com <ni...@googlegroups.com> on behalf of Dr Cyril, Pernet <wamc...@gmail.com>
Sent: Friday, August 20, 2021 2:25 AM
To: Raamana, Pradeep Reddy; niQC; Patrick Fisher

Dr Cyril, Pernet

unread,
Aug 20, 2021, 7:57:13 AM8/20/21
to Raamana, Pradeep Reddy, niQC, Patrick Fisher
Cool! How long are you planing to acquire this dataset? 3, 6 or 12 months?

forever - QC must be taken seriously and we scan al the time ... of course if we want to preregister to reproduce Yarik's paper, I suggest 24 months to have 2 season cycles ? and work on the sharing, BIDS, etc in between

anyone else want to scan water like us? two labs independent but coordinated replications would be really good

cyril

Raamana, Pradeep Reddy

unread,
Aug 20, 2021, 8:30:27 AM8/20/21
to Dr Cyril, Pernet, niQC, Patrick Fisher

I meant for the purpose of the initial analyses and paper! We have a related important talk next week from the Gold Standard Phantoms and Phantom Lab companies scheduled on Aug 24th. I believe they will touch upon their work with few big datasets like ADNI, ABCD etc, and that might be a good time to discuss and finalize this.

 

Aaron, Richard, (if you are not on vacation this week), any comments? 😊

 


Raamana, Pradeep Reddy

unread,
Aug 20, 2021, 8:45:04 AM8/20/21
to Dr Cyril, Pernet, niQC, Patrick Fisher
Also, do you expect to be able to share this dataset publicly, Cyril? 

From: ni...@googlegroups.com <ni...@googlegroups.com> on behalf of Raamana, Pradeep Reddy <RAAM...@pitt.edu>
Sent: Friday, August 20, 2021 8:30:24 AM
To: Dr Cyril, Pernet <wamc...@gmail.com>; niQC <ni...@googlegroups.com>; Patrick Fisher <patrick...@nru.dk>

Dr Cyril, Pernet

unread,
Aug 20, 2021, 9:50:51 AM8/20/21
to Raamana, Pradeep Reddy, niQC, Patrick Fisher


On 20/08/2021 14:44, Raamana, Pradeep Reddy wrote:
Also, do you expect to be able to share this dataset publicly, Cyril?
can we share water data - yes :-) public and open

Yaroslav Halchenko

unread,
Aug 20, 2021, 10:09:49 AM8/20/21
to ni...@googlegroups.com

On Fri, 20 Aug 2021, Raamana, Pradeep Reddy wrote:

> Cool! How long are you planing to acquire this dataset? 3, 6 or 12 months?
> Yarik, do we need any other specific hypotheses / experiments on this
> dataset towards the “Phantom data matterâ€* paper we’re thinking
> about?

I guess there are two major aspects to reproduce in the original paper
(well -- one would be enough to start with I guess)

- ability to describe variance within phantom SNR based on scanning
intrinsic properties (SAR, IOPD -- position, etc) as in the paper

- note: for this you do not even need any new data, could just take
old phantom data and see what you get

- if human data available -- ability to describe some variance in
morphometry results; but could be extended to any other measure

Thank you Cyril in advance!

Cheers,

--
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW: http://www.linkedin.com/in/yarik

Lei Wang

unread,
Aug 20, 2021, 10:50:33 AM8/20/21
to Raamana, Pradeep Reddy, Yaroslav Halchenko, niQC

Maybe we should start with asking Joshua Kuperman to give a talk. Do you have someone that helps with scheduling talks that I can connect them?

Also, once they let me know when their regular meetings are, we can nominate someone to make a guest appearance at their meeting.

 

--Lei

 

--
Lei Wang, Ph.D., Lei....@osumc.edu

Psychiatry and Behavioral Health, Neuroscience
Ohio State University Wexner Medical Center

 

Psychiatry and Behavioral Sciences

Northwestern University Feinberg School of Medicine

fsm-signature

 

From: ni...@googlegroups.com <ni...@googlegroups.com> On Behalf Of Raamana, Pradeep Reddy

Raamana, Pradeep Reddy

unread,
Aug 20, 2021, 3:38:45 PM8/20/21
to Lei Wang, Yaroslav Halchenko, niQC, Malin Sandström

That sounds good Lei - we have Malin Sandstrom from INCF helping but she is on vacation right now, so I’ve been coordinating them so far, and happy to take care of this.

Aaron Oliver-Taylor

unread,
Aug 23, 2021, 8:12:32 AM8/23/21
to niQC
In my talk tomorrow I'm going to touch on a new project we have been working on for a little while which echoes the initiative here to develop a QA that is appropriate for advanced neuroimaging. At this stage really we have only just completed our initial requirements gathering phase, and I suspect that the discussions tomorrow will bring about some new requirements that we should consider.

What we want to achieve however is a QA that is simple, standardised, and genuinely useful so there is buy-in from everyone at an imaging centre.

For "standard" imaging (i.e. T1w, T2w, FLAIR etc) where the images are just intended to be viewed by someone, then any sorts of MRI hardware issues only become a problem at the point the image quality is sufficiently degraded to be noticeable by eye.  However, the sorts of advanced neuroimaging methods (such as BOLD fMRI, diffusion, spectroscopy, MT, ASL, qMRI) that are used are a) more sensitive to hardware changes in the first place, and b) require post-processing which might not happen for some time (weeks to months).  As a result a hardware problem might be detected far too late to do anything about it and data will be discarded from a study.

While we could have separate QA for each technique, it will get to the point where every technique requires a 30 minute QA + dedicated phantom, which will make it hard to have buy-in. I'm not saying there isn't a place for specialised phantoms (after all we have one for ASL), but these are going to be more useful for acceptance testing, development, or validation than QA.

Our approach is to identify what the essential performance criteria for these techniques, and to then develop a QA protocol and analysis that allows us to measure these in an objective manner. Ultimately this will come down to the coils, the RF tx and rx systems, the gradients, and then other hardware related things (shims, faraday cage, spikes, noise etc). Once we have a starting point for protocols and analyses then we can start acquiring data at multiple sites. I suspect there will need to be some iteration to develop the final protocols, but eventually we will then have a large set of data which will allow us to define what the thresholds for acceptable neuroimaging are. Sort of like the ACR acceptance criteria.

We are of course very keen to collaborate on this, making the protocols and analyses open source and available for researchers to use.

Antoine Lutti

unread,
Aug 23, 2021, 8:51:36 AM8/23/21
to Dr Cyril, Pernet, Raamana, Pradeep Reddy, niQC, Patrick Fisher

>> --> still welcoming commenst on how long we should scan for fMRI / tSNR ?? @tom et al

Hi Cyril
To decide on scan duration, I would probably base myself on a protocol from an application study, assumed to be representative of a large number of studies. After all you do want that a potential tSNR reduction picked up by a QA procedure to be representative of a reduction in BOLD sensitivity in neuroscience applications.

Best
Antoine

Raamana, Pradeep Reddy

unread,
Aug 23, 2021, 1:05:00 PM8/23/21
to Aaron Oliver-Taylor, niQC

Thanks Aaron – I am quite looking fwd to tomorrow’s session with talks from you and Richard, and the subsequent discussion. As a reminder to everyone, it’s at 11am EDT Tuesday at this zoom room: https://pitt.zoom.us/my/raamana

 

 

Registered in England and Wales.

Company number: 9342804

VAT Reg: GB211807049

 

This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Gold Standard Phantoms.

If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone.

Please contact the sender if you believe you have received this email in error.

--
You received this message because you are subscribed to the Google Groups "niQC" group.
To unsubscribe from this group and stop receiving emails from it, send an email to niqc+uns...@googlegroups.com.

Raamana, Pradeep Reddy

unread,
Aug 24, 2021, 11:05:31 AM8/24/21
to Aaron Oliver-Taylor, niQC

Gentle reminder everyone for the talk happening NOW – join us : https://pitt.zoom.us/my/raamana

Christoph Vogelbacher

unread,
Aug 30, 2021, 4:25:08 AM8/30/21
to niQC
Hey everyone, I am new in this group but i am working for years with MRI phantoms. I have some small datasets already openly available. It's the ACR phantom and a self made gel phantom.


Best Christoph

Reply all
Reply to author
Forward
0 new messages