Re: [dsi-studio] t-statistic from correlational tractography

[Frank Yeh]

The average may also count negative correlation and cause this problem.
I am about to work out a code review of this part (i.e. inc_t and
dec_t) because there is another user reporting a similar issue.

I will touch base again once I figure out the cause (or fix the bug).
Frank

On Sun, Aug 11, 2024 at 5:03 PM Hannah Lindsey <lindse...@gmail.com> wrote:
>
> Frank,
>
> I ran a correlational tractography (T thresh = 4.0, length = 30 voxels) analysis and got mostly positively correlated results with FDR = .02 and some negatively correlated results with FDR = .09.
>
> I extracted the stats from both of the tract files, and for the positively correlated tracts, the average inc_t = 0.156 (r = .063) and dec_t = 0.344 (r = .139). The tracts with negative correlations produce average t-statistics closer to that which I would expect, given the overall negative relationship (inc_t = .008, dec_t = 0.968).
>
> Can you help me to understand why the inc_t is so low (and even lower than the dec_t) from the positive correlation tracts, even though the FDR < .05?
>
> Thanks for your help
>
> --
> You received this message because you are subscribed to the Google Groups "DSI Studio" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to dsi-studio+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/dsi-studio/ac7ac7b4-03aa-4981-b9ff-64edfd400a63n%40googlegroups.com.

[Hannah Lindsey]

Okay, great. Thanks!

Do you think the bug fix will affect the results otherwise? I’m wondering if I will need to re-run everything and should halt the post-connectometry analyses until it is fixed.

_________________________________

Hannah M. Lindsey, PhD

Research Associate

TBI and Concussion Center

Department of Neurology

University of Utah

On Aug 12, 2024, at 3:30 PM, Frank Yeh <fran...@gmail.com> wrote:

The average may also count negative correlation and cause this problem.

[DSI Studio]

If the post-hoc is depending on inc_t or dec_t, better wait until the issue is cleared.

I will work on this at a high priority.

[Frank Yeh]

I did a test using the SCA2 cross-sectional db at https://practicum.labsolver.org/ct.html

After the connectometry analysis, I loaded the t_statistics.fib.gz and switched to dec_t. 

Then I created a ROI with dec_t > 2.5 using [Regions Misc][New Region From Thtresholding] and applied defragments using [Region Misc][Modify Region][Defragment]

The dec_t > 2.5 ROI matches the findings nicely (see figure).

Perhaps you may test the same on your data?

[Hannah Lindsey]

I tried this on two sets of results, one where I only got inc_t results and another where I got both inc_t and dec_t results (the one I was referring to in my first message). Below is a screenshot of the results with inc_t only, where the region overlaps the area with the highest t-statistic (according to the local index color); however, I set the t-threshold to 3.0 when I ran the analysis, so shouldn't all of the results be > 3?

I also tried your test on the other results with both inc_t (pink) and dec_t (blue) results, but the regions don't overlap at all and are very small (second picture is without tracts so you can see the dec_t region)

 

[Frank Yeh]

Thanks for raising this issue. The same problem was also reported recently (https://groups.google.com/g/dsi-studio/c/eVpNqsmJLF0/m/HlpaZqKCAQAJ), and I am very sorry for the bug causing a hassle here.

My initial code review confirmed that it is a visualization problem. The internal values of inc_t and dec_t are still correct, so saving the tract or region statistics will report correct results on inc_t and dec_t.

Only the color map visualization has a problem, and the bug's exact location needs more work. 

I hope to find out and fix it soon and will get back to this thread.

Best,

Frank

[Hannah Lindsey]

Hmmm okay, well, then my initial question still remains... From my analysis with both increased and decreased results, the extracted stats provide an inc_t = 0.156 and dec_t = 0.344 from the inc.tt.gz file, and inc_t = .008 and dec_t = 0.968 from the dec.tt.gz file. Can you help me to understand why the inc_t is so low (and even lower than the dec_t) from the inc.tt.gz, even though the FDR < .05? 

By the way, after clustering the results into individual tracts and extracting statistics from those, the extracted t-stats appear to line up pretty well with the averaged index coloring, as you can see in the image below. Also, again, shouldn’t the extracted t-statistics be at least 3.0, given that that was my threshold?

[Frank Yeh]

Hmmm okay, well, then my initial question still remains... From my analysis with both increased and decreased results, the extracted stats provide an inc_t = 0.156 and dec_t = 0.344 from the inc.tt.gz file, and inc_t = .008 and dec_t = 0.968 from the dec.tt.gz file. Can you help me to understand why the inc_t is so low (and even lower than the dec_t) from the inc.tt.gz, even though the FDR < .05? 

Not sure what caused this. You may upload the statistics fib file and the tracts, and I can help check (This may take a while due to my long analysis queue).

 

By the way, after clustering the results into individual tracts and extracting statistics from those, the extracted t-stats appear to line up pretty well with the averaged index coloring, as you can see in the image below. Also, again, shouldn’t the extracted t-statistics be at least 3.0, given that that was my threshold?

The tracts are collected from multiple instances of correlation tractography applied to bootstrap resamplings of the subject pool. On the other hand, the dec_t and inc_t in the fib file were computed without resampling. Thus they won't necessarily match if there are outliers. 

 

On Saturday, August 17, 2024 at 4:32:11 PM UTC-4 Frank Yeh wrote:

Thanks for raising this issue. The same problem was also reported recently (https://groups.google.com/g/dsi-studio/c/eVpNqsmJLF0/m/HlpaZqKCAQAJ), and I am very sorry for the bug causing a hassle here.

My initial code review confirmed that it is a visualization problem. The internal values of inc_t and dec_t are still correct, so saving the tract or region statistics will report correct results on inc_t and dec_t.

Only the color map visualization has a problem, and the bug's exact location needs more work. 

I hope to find out and fix it soon and will get back to this thread.

Best,

Frank

--
You received this message because you are subscribed to the Google Groups "DSI Studio" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dsi-studio+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dsi-studio/a2f13d79-ad04-41eb-ad38-22fa82007fbbn%40googlegroups.com.

[Hannah Lindsey]

I just uploaded the t_statistic.fib.gz and tract files from the analysis I initially asked about. I actually re-did that one with a T threshold of 3 instead of 4, so the t-stats are slightly different than what I stated in a previous message, but the same questions I have still applies to them.

Thanks for your help with this

Hannah

[Frank Yeh]

I will investigate this and get back to you

To view this discussion on the web visit https://groups.google.com/d/msgid/dsi-studio/71040ffe-5b83-4aef-96a4-5bc87dc26cc1n%40googlegroups.com.

[Frank Yeh]

I have identified the bug, which only exists in the versions in Aug, 2024.

A quick solution is to use the May version at https://github.com/frankyeh/DSI-Studio/releases/tag/2023.12.06

Future releases will also fix this bug.

My apology for all the hassle.

Best regards,

Frank

[Hannah Lindsey]

Thanks for looking into this so quickly! I downloaded the May 17 version you suggested and re-ran the analyses, but I'm running into the same issue, where the average t-statistics are always well below my threshold of 3 (they're between 1.7 - 2.1 for all analyses except the one below), and on analyses where there are both positive and negative correlations, the t-stats are wonky.

Here's an example, where T-threshold is 3, and only positive correlations survived the FDR threshold of < .20:

Local index for inc_t:

Local index for dec_t:

Here's the extracted stats:

• Tract Name qa-csrc.inc
  
• number of tracts 26231
  
• mean length(mm) 54.2372
  
• ...
  
• qa 0.433118
  
• inc_t 0.286825
  
• dec_t 0.361491
  
• qa_map 0.438476
  

Why is the inc_t so much lower than dec_t, even though only inc_t correlation results survived the FDR threshold, whereas dec_t did not?

Also, according to the fdr_dist.values.txt file, only tract lengths of 20-23 voxel distances (40-46mm) had FDRs < .2, so why is the average tract length in the extracted stats 54 mm?

[Frank Yeh]

Sorry for the issue. 

I will test it again.

Frank

To view this discussion on the web visit https://groups.google.com/d/msgid/dsi-studio/994619d3-cfeb-43db-ac2e-6294090ef2d4n%40googlegroups.com.

[Frank Yeh]

I checked the file. The rendering and calculation are correct.

The problem is likely due to FDR < 0.2, because it is going to include 20% false results of tracts passing through many locations with inc_t and dec_t = 0. The average will be close to zero.

We may use a lower T-threshold, but the FDR has to be much smaller to get reliable results.

Best regards,

Frank

 

To view this discussion on the web visit https://groups.google.com/d/msgid/dsi-studio/994619d3-cfeb-43db-ac2e-6294090ef2d4n%40googlegroups.com.

[Emily Dennis]

I'm running into the same issue with FDR < 0.05 though. Set 2.5 as my T-threshold, running the version from May 17th. Based on the output I think there should be far more exceeding 2.5

[Frank Yeh]

(Sorry again about the problem)

 

The average inc_t and dec_t would be very close to zero if the FDR is high.

As to the inc_t dec_t rendering, my apologies for giving the wrong information (I don't have a working Mac to test).

you may try an even earlier Mac version here: https://github.com/frankyeh/DSI-Studio/releases/tag/2023.07.08

Alternatively, there is the pre-release of the "Hou" version at https://github.com/frankyeh/DSI-Studio/releases/tag/2024.06.12

The Hou version has many major revisions and uses a new SRC/FIB format (*.sz and *.fz) and still can open the existing formats. It is still under testing and I haven't put the link on the DSI Studio website yet.

Sorry again for all the hassle caused, and hope we can solve this issue soon.

Best,

Frank

To view this discussion on the web visit https://groups.google.com/d/msgid/dsi-studio/5b77563d-2eab-417b-a6ff-5ad9464005b8n%40googlegroups.com.

[Emily Dennis]

I tested out a few versions - the 3 older ones all seem to show the same problem, and the Hou version doesn't give the option to have dec_t as the index.

Thanks,

Emily

[Frank Yeh]

Thanks for testing it out!
In the old versions, most tracts are redish, indicating the dec_t is
around 2.5 (which seems correct to me).
The bug in the August version will mistakenly use qa to render tracts
and thus mostly lower than 1, and the tracts will appear all blue.

> --
> You received this message because you are subscribed to the Google Groups "DSI Studio" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to dsi-studio+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/dsi-studio/a6a1980f-a45e-47e0-b034-bcc63ba06634n%40googlegroups.com.

[Emily Dennis]

Ok so it's normal to have the visualization showing this much below the threshold? The average dec_t is 3.6

[Frank Yeh]

Yes, because the tract findings here are the results of resampling,
and the t statistics after resampling can be larger than those
without.

On Tue, Aug 27, 2024 at 2:27 PM Emily Dennis

<emily.lar...@gmail.com> wrote:
>
> Ok so it's normal to have the visualization showing this much below the threshold? The average dec_t is 3.6
>
> On Tuesday, August 27, 2024 at 11:16:31 AM UTC-7 Frank Yeh wrote:
>>
>> Thanks for testing it out!
>> In the old versions, most tracts are redish, indicating the dec_t is
>> around 2.5 (which seems correct to me).
>> The bug in the August version will mistakenly use qa to render tracts
>> and thus mostly lower than 1, and the tracts will appear all blue.
>>
>> On Tue, Aug 27, 2024 at 2:10 PM Emily Dennis
>> <emily.lar...@gmail.com> wrote:
>> >
>> > I tested out a few versions - the 3 older ones all seem to show the same problem, and the Hou version doesn't give the option to have dec_t as the index.
>> > Thanks,
>> > Emily
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups "DSI Studio" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an email to dsi-studio+...@googlegroups.com.
>> > To view this discussion on the web visit https://groups.google.com/d/msgid/dsi-studio/a6a1980f-a45e-47e0-b034-bcc63ba06634n%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "DSI Studio" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to dsi-studio+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/dsi-studio/b0f78c95-e403-4400-bebb-5d54aa727dden%40googlegroups.com.

[Emily Dennis]

Ok thank you!

[Emily Dennis]

Hi Frank,

Sorry to re-open this but I'm having this issue again and I can't fix it, I've tried a few old versions. The dec_map shows a huge area as being significant, with fdr<0.05 and T_threshold>2.5, but when I try to view the dec_t file it shows barely anything, am I missing something?

Thanks for your help,

Emily

[Frank Yeh]

The color map visualization has a bug in the older versions. You may update DSI Studio to get the correct colors.

Best

Frank

To view this discussion visit https://groups.google.com/d/msgid/dsi-studio/15c67fe5-6331-4483-8786-462524bbb669n%40googlegroups.com.

[Emily Dennis]

I just tried that and I'm still having issues with the t-statistics displayed not seeming to match the report

Thanks,

Emily

[Frank Yeh]

Could you send me the .fib.gz file.

Best

Frank

To view this discussion visit https://groups.google.com/d/msgid/dsi-studio/e9fdc5d2-349f-49b6-9246-48556a0142ben%40googlegroups.com.

[Emily Dennis]

Hi Frank,

I thought I send a fib file a few days ago but I don't see a record of that here. Did you get that or should I resend?

Thanks,
Emily

[Frank Yeh]

I did not see it.

Could you send it again?

To view this discussion visit https://groups.google.com/d/msgid/dsi-studio/a533e82b-863d-48be-9e24-fd578538238en%40googlegroups.com.

[Emily Dennis]

Yep here it is, thank you!

[Frank Yeh]

I checked the data, and there are several regions with dec_t > 2.5.
The visualization seems correct to me.

[Emily Dennis]

Ok I see, thank you. If I want the image to be sparser I can just decrease the visible tracts right? I want these to be apparent bc they actually are significant even though it may not seem that way from the image

[Frank Yeh]

I would load only the significant tracts. The figure uses whole brain tracts and includes many insignificant locations.

Frank

--
You received this message because you are subscribed to the Google Groups "DSI Studio" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dsi-studio+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/dsi-studio/f674c6b6-e47d-4c26-8fb5-8fd790f7ae00n%40googlegroups.com.

[Emily Dennis]

Ah I didn't realize that was a separate option! How would I do that?

[Frank Yeh]

At the top menu [Tract][Open Tracts] -> browse the connectometry output folder to load .tt.gz files.

To view this discussion visit https://groups.google.com/d/msgid/dsi-studio/002cf6ce-ee4c-42f6-a31d-4fb82a8d0f75n%40googlegroups.com.

[Emily Dennis]

Perfect, thank you!