Thresholding Cohen’s d maps

100 views
Skip to first unread message

Reza Rajimehr

unread,
Feb 6, 2021, 7:30:50 PM2/6/21
to hcp-users, neda afzalian
Hi,

The group-average functional maps of S1200 have been provided in the Cohen’s d format. Cohen’s d is preferable over z-statistics because the effects are not inflated by sample size. However, unlike z-statistics maps where we can define a threshold like z = 1.96, there is no particular threshold for Cohen’s d. Most of the times, this is not a problem because we want to show the full map without thresholding. However, we sometimes want to know which parts of cortex are selectively/preferentially involved in processing of a stimulus/task. For example, based on the contrast of faces versus other categories, we want to localize the classic face-selective areas/ROIs. How can we threshold a Cohen’s d map in a biologically meaningful way? What is the standard approach here?

Best,
Reza

Glasser, Matthew

unread,
Feb 6, 2021, 7:40:10 PM2/6/21
to hcp-...@humanconnectome.org, neda afzalian

Weren’t z-stats also provided?  Z-stats are good for statistical thresholding, but not much else.  Cohen’s D and other effect size measures are better for making biological interpretations (e.g. is X measure different between area A and area B).  If you want to find the boundaries between areas involved in faces and those that are not, I would compute the gradient of the effect size map and draw the boundary along the gradient ridge.  This is the same approach taken, for example, in the Zilles and Amunts cytoarchitectural parcellations.  Thresholding is a biological map is a tough thing to do well—there will be imperfections the exact value of the biological measure, even very high quality average maps will have this, and this will cause the boundary to deviate randomly.  

 

Matt.

--
You received this message because you are subscribed to the Google Groups "HCP-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hcp-users+...@humanconnectome.org.
To view this discussion on the web visit https://groups.google.com/a/humanconnectome.org/d/msgid/hcp-users/CAPiNLe4etBSUt3tOMoRLgsC8Q37CBHOtSehEpvfxThXqnaed7A%40mail.gmail.com.

 

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

Reza Rajimehr

unread,
Feb 6, 2021, 8:00:23 PM2/6/21
to hcp-...@humanconnectome.org, neda afzalian
The z-statistics maps have not been provided in S1200. They have been provided in S900.

The idea of using gradients is interesting, but it does not take into account the notion of “selectivity”. For example, if there is a sharp spatial transition from a small positive value to a small negative value in the map, that may be marked as a border, but that area does not have a high face selectivity.

We tried a threshold of Cohen’s d = 1, and the areas seem to qualitatively match with what has been reported in the literature. But I admit that there is an arbitrariness here.

Best,
Reza


Glasser, Matthew

unread,
Feb 6, 2021, 8:36:57 PM2/6/21
to hcp-...@humanconnectome.org, neda afzalian

You can pick which gradient to focus on.  Most of the strong ones are around obvious stuff, not weak positive and negative.  If anything, z-stat thresholds (white lines) are more focused around weak stuff and are less reproducible as illustrated in the attached images from 2 independent 210 subject groups. 

210P_tfMRI_Beta_Zstat_Thresh_FACE-AVG.png
210V_tfMRI_Beta_Zstat_Thresh_FACE-AVG.png

Reza Rajimehr

unread,
Feb 6, 2021, 9:02:34 PM2/6/21
to hcp-...@humanconnectome.org, neda afzalian
Thanks Matt!

In the statistical maps, I guess one can get rid of weak and less reproducible stuff by raising the statistical threshold (e.g. by applying FDR correction).

The maps in the bottom row are from the gradient analysis? If so, there seems to be some issues here as well:

1) A face-selective area in medial parietal cortex has not been detected.

2) There are some irrelevant and scattered borders in the maps.

3) The gradient map is not constrained to localize closed-shape areas. Even for an obvious area like FFA, there are two parallel borders corresponding to lateral and medial sides of FFA. How can one get real, closed-shape ROIs out of this gradient map?


Glasser, Matthew

unread,
Feb 6, 2021, 9:09:03 PM2/6/21
to hcp-...@humanconnectome.org, neda afzalian

Those are Bonferroni corrected stats.  I don’t think you can be more conservative than that on any rational basis.

 

  1. I am not showing medial parietal cortex in those inflated maps.
  2. I’m not sure what you are referring to?
  3. I guess it would be easier to see if the stats lines were not there?  Some cortical areas that were defined in part using this contrast include FFC and PIT.   

Reza Rajimehr

unread,
Feb 6, 2021, 9:33:44 PM2/6/21
to hcp-...@humanconnectome.org, neda afzalian
1) In the flat patches, there is a region with a relatively good face selectivity in medial parietal cortex / Precuneus / posterior cingulate cortex. I don’t see well-defined borders/gradients around it.

2) In the gradient maps, there are some red borders and some green borders. For example, the medial side of FFA/FFC has been extended anteriorly by a green border. Again there is a problem of thresholding here; what value of spatial derivative should be used for choosing the borders?

3) I still don’t know how the gradient map could be automatically (i.e. without human intervention) converted to closed-shape ROIs. Maybe I am missing something here ... .


Glasser, Matthew

unread,
Feb 6, 2021, 10:29:17 PM2/6/21
to hcp-...@humanconnectome.org, neda afzalian
  1. Sure there is a gradient, though it isn’t as sharp or intense.
  2. I guess I would say it depends on what you are after.  There isn’t a specific threshold.  If you want to identify cortical areas, it helps to use multiple modalities to define these borders.
  3. Why would you want that?  If you were to unfortunately need a clinical brain scan, you would want an expert to interpret it not a machine without human intervention.  Machines can help (e.g. define a line that follows the gradient ridge automatically within a patch or compute a perfusion map on a CT perfusion scan), but someone with a brain still needs to interpret what the map is showing, avoid artifacts, etc.  This is how neuroanatomy (or neuroradiology) is done.  You might find that the face selective regions are already well defined as cortical areas or you might find in some cases that only part of an area is face selective.  Overall, I would think of this more like some of the retinotopy you’ve done in the past.  I think in brain imaging we a bit over fetishize things like thresholds and full automation when the data may not really admit of that. 

Reza Rajimehr

unread,
Feb 7, 2021, 1:22:31 AM2/7/21
to hcp-...@humanconnectome.org, neda afzalian
Thanks Matt for your suggestions. We can actually use a combination of thresholding and gradients to achieve what we want. In the group-average Cohen’s d map, we can first set a threshold to get a reasonable layout of the cortical areas, then use gradients as a guide to fine-tune the initial threshold. Just a naive question: is there a wb_command to convert an activation map to a gradient map?

Unlike neuroradiologists who are usually interested in obvious and clinically-relevant pathologies of the brain, we (as cartographers) are sometimes interested in small details of brain features and their normal anatomical variations. That’s why I thought that perhaps developing objective, fully-automated methods might be useful in defining those features and minimizing the experimenter’s bias. Sorry if my suggestion was not necessarily valid.

Best,
Reza


Glasser, Matthew

unread,
Feb 7, 2021, 12:18:40 PM2/7/21
to hcp-...@humanconnectome.org, neda afzalian

I’d argue that you don’t need the thresholding step as the clusters are pretty obvious by eye (we didn’t need this to make the HCP’s multi-modal parcellation).  You can use wb_command -cifti-gradient to create a gradient map (if using group average surfaces, you will need to use average midthickness vertex areas as “corrected areas”).  If you want semi-automated help drawing gradient borders, there is a tool in Connectome Workbench GUI to do this (i.e. the tool used to make the HCP’s multi-modal parcellation), which means that the final borders are machine drawn, just informed by neuroanatomist choices of signal, artifacts, and noise.  We could probably automate this a bit more to do some kind of flood fill bounded by gradients for peaks and valleys of a biological map for this specific use case. 

 

It's worth keeping in mind that algorithms also all have biases and often make silly mistakes (like the one that called a meningioma a brain hemorrhage on a head CT I read recently).  I still think the best work will be generated by a combination of scientist (or physician) and automated tools. 

Reza Rajimehr

unread,
Feb 7, 2021, 12:50:36 PM2/7/21
to hcp-...@humanconnectome.org, neda afzalian
Thanks Matt! So the tool you are referring to is the one to draw borders on the map?


Glasser, Matthew

unread,
Feb 7, 2021, 12:56:42 PM2/7/21
to hcp-...@humanconnectome.org, neda afzalian

Basically it takes an existing border and gradient map and ensures that the border optimally follows the gradient (and can take multiple gradient maps for multi-modal analysis). 

Reza Rajimehr

unread,
Feb 7, 2021, 1:00:24 PM2/7/21
to hcp-...@humanconnectome.org, neda afzalian
I see. Where is that feature in wb_view GUI?


Glasser, Matthew

unread,
Feb 7, 2021, 1:16:46 PM2/7/21
to hcp-...@humanconnectome.org, neda afzalian

Border Optimization in the border drawing section.

Reza Rajimehr

unread,
Feb 9, 2021, 12:50:28 PM2/9/21
to hcp-...@humanconnectome.org, neda afzalian
Hi Matt,

Based on the gradient map, we manually draw a closed border around a region (e.g. PPA). We then want to optimize it, but we can’t really get the optimization tool to work. First of all, this tool does not work for the whole border. We have to select some points near part of the border. When we do this and choose the gradient map for optimization, it sometimes slightly improves the border but sometimes makes it worse. Maybe we don’t know exactly how to use this tool. Is there a documentation for it?


Glasser, Matthew

unread,
Feb 9, 2021, 1:33:04 PM2/9/21
to hcp-...@humanconnectome.org, neda afzalian

This was designed originally to optimize borders between two areas (meaning that an individual area was done in multiple parts).  It optimizes for shortest path and following the gradient together within a region around a border.  So you want to break up curves into smaller sections. 

Austin Cooper

unread,
Dec 21, 2023, 3:41:38 PM12/21/23
to HCP-Users
Hi HCP aficionados!

I've read the above message and I'm wondering if you can clarify where and how these Cohen's d maps are computed? Are they generated by level 1, level 2, and/or level 3 TaskAnalysis pipelines? I'll attach an image of the files I was able to get from my level 2 & 3 TaskAnalysis. I see that the files HCP provided for Cohen's d maps have the naming structure of HCP_S1200_997_tfMRI_ALLTASKS_level2_cohensd_hp200* , though I cannot find something of the sort within my output files.

I'd like to be able to compute these Cohen's d maps and visualize them. I wonder if there is a wb_command available to compute them?

level3_outputFiles.png
level_2Output.png

Glasser, Matt

unread,
Dec 21, 2023, 5:11:02 PM12/21/23
to hcp-...@humanconnectome.org

I think Mike Harms made them.  It is correct that they are not currently a pipeline output. 

 

Matt.

--

You received this message because you are subscribed to the Google Groups "HCP-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hcp-users+...@humanconnectome.org.

Reza Rajimehr

unread,
Jan 17, 2024, 1:52:49 AMJan 17
to hcp-...@humanconnectome.org
I think they are simply the mean of beta maps divided by the standard deviation of beta maps across subjects.

Reza

Harms, Michael

unread,
Jan 18, 2024, 12:48:53 PMJan 18
to hcp-...@humanconnectome.org, Harms, Michael

I did some digging, and it looks like the version that I used to create the “cohensd” maps for the HCP_S1200 group release never made it into GitHub.

 

That code was just:

 

    # Cohens d

    if [ "${ComputeCohensD}" = "YES" ] ; then

      # 4th dim ("Time") is actually across subjects in the $merged files

      fslmaths $mergedcope -Tmean ${mergedcope}_mean

      fslmaths $mergedcope -Tstd -div ${mergedcope}_mean -recip cohensd${i}

      #Compute a 2nd version as the mean across the Cohen's d's for each indiv subject

      fslmaths $mergedvarcope -sqrt -div $mergedcope -recip -Tmean avgcohensd${i}

      rm -f ${mergedcope}_mean.nii.gz

    fi

 

The first version above (“cohensd”) is what we ended up releasing (not the “avgcohensd” version).

 

Cheers,

-MH

 

-- 

Michael Harms, Ph.D.

-----------------------------------------------------------

Professor of Psychiatry

Washington University School of Medicine

Department of Psychiatry, Box 8134

660 South Euclid Ave.                        Tel: 314-747-6173

St. Louis, MO  63110                          Email: mha...@wustl.edu

 

From: Reza Rajimehr <raji...@gmail.com>
Reply-To: "hcp-...@humanconnectome.org" <hcp-...@humanconnectome.org>
Date: Wednesday, January 17, 2024 at 12:52 AM
To: "hcp-...@humanconnectome.org" <hcp-...@humanconnectome.org>
Subject: Re: [hcp-users] Re: Thresholding Cohen’s d maps

 

* External Email - Caution *
Reply all
Reply to author
Forward
0 new messages