P-value not significant

[Shivangi Agarwal]

Hello Matt

I am using Hotnet2 algorithm for generating subnetworks of mutated genes, but in my results, I am not getting significant p-values (<0.05) of the components for any of the delta value. What should I do? Please suggest.





Regards
Shivangi Agarwal

[matthe...@brown.edu]

Hi Shivangi,

First, please check that the HotNet2 subnetworks look reasonable.  For example, do the subnetworks contain genes, are some of the genes highly mutated, do some of the genes interact in your network, etc.?  If not, then there may be an issue with data processing before running HotNet2, and one would not expect statistical significance in this case.  

Second, please also check that you are using similar parameter values to the commands in the paper/paper_commands.sh script, which replicates the results from the HotNet2 paper.  Since the resolution of the p-values are limited by the number of heat permutations, it may be difficult to achieve small p-values with too few heat permutations.  I would try at least 100 heat permutations.

Third, please note that HotNet2 tests whether the assignment of heat scores is independent of network topology (see the HotNet2 paper).  If this is not the case in your data, then one would not expect for the HotNet2 subnetworks to be statistically significant by this test.

Best,

Matt

[Shivangi Agarwal]

HI Matt

I have generated heat score file on the basis of mutation frequency and not on the network topology. Also, the components file contains mutated genes as said by you in first point. 

When I am generating subnetworks for different stages using the heat score file of the same, I am getting significant networks for some but not for all.

I am using the following command:

python makeHeatFile.py mutation --snv_file -o -n

python $Hotnet2.py -nf -pnp -hf  -np 99 -hp 1000 -o -c -1

I have tried with -hp 100 and -hp 1000 as well. Please suggest.

[matthe...@brown.edu]

Hi Shivangi,

In your previous message, you indicated that none of your HotNet2 results were statistically significant, while in your latest message, you indicated that some but not all of your HotNet2 results were statistically significant.  This change is good to hear, but please note that there is no guarantee that all HotNet2 results will be statistically significant, which depends on the presence and size of "hot" subnetworks in the data.

Best,

Matt

[Shivangi Agarwal]

Thanks for your reply, to reproduce hotnet2 results, are the influence matrix files and permuted networks files available for three networks used?. If yes, please share link.

[matthe...@brown.edu]

Hi Shivangi,

Both precomputed influence matrix and permuted network files are available:

http://compbio-research.cs.brown.edu/pancancer/hotnet2/#!/data

http://compbio-research.cs.brown.edu/software/hotnet2/permuted_networks

However, due to updates to the HotNet2 scripts, I would recommend running the paper/paper_commands.sh command to reproduce the HotNet2 results in Leiserson et al., Nat. Gen., 2015.  The necessary network and heat score files are provided in the HotNet2 GitHub repository.  Please note that the HotNet2 subnetworks and P-values may be slightly different than those reported in the paper due to different heat and network permutations that occur in different runs of HotNet2.

Best,

Matt

[Shivangi Agarwal]

Thanks for your reply Matt.

Just I have one query like if we got 4 delta directories after a run of hotnet2.py, then we will select the delta directory with greater number of significant p-values, (right?), so finally as I got files inside selected delta directory like (component.txt file and significant.txt), attached herewith. In significant file, we get p-values for each size of subnetwork component (from 2 to 10) and as you can see in the attached file (significant.txt), I am getting significant p-value (<0.05) for component size 4,8,9,10, than is that means (as I understand) that in component.txt file, only the subnetworks with size 4,8,9,10 are significant and thereby I should select them further for my studies??

Please suggest.

[matthe...@brown.edu]

Hi Shivangi,

The HotNet2 statistical test (Sec. 1.3 of the supplement) evaluates whether there are significantly more subnetworks of size k or larger than expected by chance.  In your example, there are significantly more (P < 0.05) HotNet2 subnetworks of size at least 4, 8, 9, or 10 than expected by chance.

As you wrote, the HotNet2 paper (Sec. 1.4.2 of the supplement) recommends choosing the value of the delta parameter that produces the largest number of significant (P < 0.05) subnetwork sizes.

Best,

Matt

[Shivangi Agarwal]

Hi Matt
Can you please elaborate this sentence "In your example, there are significantly more (P < 0.05) HotNet2 subnetworks of size at least 4, 8, 9, or 10 than expected by chance".
Thanks for your support.

[matthe...@brown.edu]

Hi Shivangi,

You have P-values P < 0.05 for k = 4, 8, 9, 10, which means that there are significantly (P < 0.05) more HotNet2 subnetworks of size >= 4, size >= 8, size >= 9, and size >= 10 than expected by chance.

Best,

Matt