Zhang 2024 pFIC model unable to simulate empirical CDF

[Ana Radanovic]

Hello,

Thank you so much for providing these fantastic resources to the research community. I am a PhD student at Weill Cornell graduate School of Medical Sciences and I have been interested in applying the pFIC model from Zhang 2024 et al on our dataset. 

I have noticed that I am unable to get a KS value <5 during training if I use our group average CDF as an input (a tailored version of  CBIG_pFIC_generate_HCP_group_level_FC_FCD.m was used to produce both the FC and FCD inputs, with our particular BOLD acquisition considered). The relevant config parameters have been adjusted according to our acquisition as well, namely, window_size, simulation period, number of frames, TR, and d_t. In addition, I have played around with keeping the set parameters from the HCP example (example.ini), only replacing the inputs one by one (SC, FC, and CDF). The KS value from the CDF remains at 5, even after many epochs.

I have also tried several Euler time step (d_t) parameters with no improvement. 

Do you have any suggestions in order to improve the simulation of our empirical CDF? Let me know what other information I need to provide!

Thank you in advance!

Best,

Ana Radanovic

[Shaoshi Zhang]

Hi Ana,

Thanks for using our code! One possible reason could be that your TR is not an integer multiple of d_t, which would cause the number of simulated fMRI frames after downsampling to differ from that of the empirical fMRI. During training, you can save the simulated time series ('bold') at this line and check whether its dimensionality is as expected. By the way, are the 2 other FC cost terms decreasing during training?

With regards,

Shaoshi

[Ana Radanovic]

Hi Shaoshi,

Thanks for the quick response!

Let me respond to each question one-by-one.

1. Re - d_t as an integer multiple of the TR: Our TR = 0.8. I have run values of [0.001, 0.004, 0.005, 0.008]. All have produced the max KS of 5.

2. I saved the simulated BOLD timeseries and it is as expected : [68, 500, 840] for the 68 cortical regions, and 840 frames

3. The two FC costs are decreasing through training epochs, albeit by small increments (i.e., FC corr cost from 0.58 -> 0.53 after the 5th generation). Attached is an example learning curve. It is hard to see (as the KS cost is so high), but the FC cost does slight go down. 

I can see that something happens with the line fcd_histcum_ave = fcd_histcum_temp / fcd_histcum_num in CBIG_FCD_multi_simulation, I get a fcd_histcum_ave that is NaN, as fcd_histcum_num = 0 just before that. 

Not sure if that helps or complicates things more! Thank you in advance. 

All the best,

Ana Radanovic

[Shaoshi Zhang]

Hi Ana,

There are 3 things you may consider checking:

1. Is the window size specified in the config file the same as the one used to compute the empirical FCD, which in your case should be 75?

2. Are there any NaN values in the 500 sets of simulated time courses (bold, S_E_all and S_I_all)?

3. Are all the firing rates (r_E_all) within the specified range (rE_min to rE_max, 2.7 - 3.3 by default)?

With regards,
Shaoshi

[Ana Radanovic]

Hi Shaoshi,

1. Yes, the window size in the config file is the one used to compute the FCD == 75.

2. I see that in the simulated time courses there are NaN's in bold (in the 5th training epoch 310/500 have NaN's), across the S_E_all and S_I_all there are no NaN's. The number of bold simulations with NaN's does decrease with each epoch, but starts at 480/500 simulations containing NaN's. 

3. The boundary within the config file is 2.7-3.3, however, I do see that quite a few of the firing rates (r_E_all) are out of range. For example, if I run only 5 training epochs, the 5th epoch has 310/500 simulations with out of bound firing rates.

Sorry for the delayed response!

All the best,

Ana Radanovic

[Shaoshi Zhang]

Hi Ana,

Good to know that not all 500 sets contain NaNs. The NaN proportion is high for the first few epochs because the sampled parameters very often produce firing rates that are out of bound. Notice that the number of out-of-bound firing rates is the same as the number of simulations with NaNs (which is a direct result of this step). This proportion will naturally decrease as the optimization process continues. Can you try running 20 or 30 training epochs? Does this proportion approach 0?

With regards,

Shaoshi

[Ana Radanovic]

Hi Shaoshi,

It does look like the proportion reaches 0, and there are no NaN's in the Bold signal. However, after 100 training epochs, I still see the FCD KS distance is 5.

Attached is the learning curve output from 100 training epochs. I have a feeling something is happening around this step (here). Or similarly, here

 

Best,

Ana Radanovic

[Shaoshi Zhang]

Line 751 essentially checks if the cumulative distribution function (CDF) of the simulated FCD matches that of the empirical FCD. If you use the same window length, the last entry of simulated FCD CDF should be the same as the empirical FCD CDF, which is also the # edges of the upper triangular part of FCD matrix (excluding the diagonal). Take the HCP time course with 1200 frames as an example, if the window size is 83 (60s/0.72s), then the FCD matrix has a dimensionality of 1118-by-1118 (1200-83+1=1118). So, after collapsing the upper triangle of the FCD matrix into a CDF, the last entry of the CDF should be (1118*1118-1118)/2 = 624403.

Just as a quick sanity check, can you confirm the last row of fcd_histcum (from line 749) is the same as the last entry of your empirical FCD CDF? 

WIth regards,

Shaoshi

[Ana Radanovic]

Hi Shaoshi,

Yes it seems there may be an issue there, the last row is not the same as the last entry of my empirical.

This has pointed out the issue for me - so sorry in advance for taking your time! - our run time was 11.2 minutes including the AP & PA scan. When calculating FCD, 420 frames per scan were used (a 5.6 minute scan), which are then used to obtain the average CDF. At first, I had tried using 5.6 minutes and 420 frames in the config script, but had later switched to 11.2 minutes with 840 frames thinking the issue lay there. 

When I adjust the config file to contain a 5.6 minute simulation period with 420 frames, after some training epochs I do see the KS distance <5 (which means I ALSO needed to run way more training epochs!).

Things seem to be working! Sorry again to take your time, I truly appreciate the prompt and very helpful feedback!

Best,

Ana Radanovic

[Shaoshi Zhang]

Glad to hear it’s resolved! Always happy to help.

Shaoshi