Query

85 views
Skip to first unread message

Shivangi Agarwal

unread,
Apr 16, 2018, 9:50:48 AM4/16/18
to HotNet
Hi
I have created network files for Hippie network and heat file, now running hotnet2 using following commands and getting the error:

python HotNet2.py -nf paper/data/networks/Hippie/HIPPIE_ppr_0.47.h5 -pnp paper/data/networks/Hippie/permuted/HIPPIE_ppr_0.47_##NUM##.h5 -hf paper/data/heats/mutation_f_conv_4.json -np 100 -hp 100 -o results -c -1


* Running HotNet2 in consensus mode...
    - HIPPIE mutation_f
Traceback (most recent call last):
  File "HotNet2.py", line 139, in <module>
    run(get_parser().parse_args(sys.argv[1:]))
  File "HotNet2.py", line 85, in run
    single_runs, consensus, linkers, auto_deltas, consensus_stats = consensus_with_stats(args, networks, heats)
  File "/media/ravi/Seagate Expansion Drive/hotnet2/hotnet2/consensus.py", line 17, in consensus_with_stats
    single_runs, consensus, linkers, auto_deltas = consensus_run( args, networks, heats, verbose )
  File "/media/ravi/Seagate Expansion Drive/hotnet2/hotnet2/consensus.py", line 68, in consensus_run
    heat, addtl_genes = filter_heat(heat, None, False, 'There are ## genes with heat score 0')
  File "/media/ravi/Seagate Expansion Drive/hotnet2/hotnet2/heat.py", line 23, in filter_heat
    min_score = min([score for score in heat.values() if score > 0])
ValueError: min() arg is an empty sequence


Can you please suggest??

Shivangi Agarwal

unread,
Apr 23, 2018, 12:56:49 AM4/23/18
to HotNet
Hi

Can anyone suggest?

matthe...@brown.edu

unread,
Apr 23, 2018, 9:25:31 AM4/23/18
to HotNet
Hi Shivangi,

From this error, it looks like your data does not contain network genes with positive heat scores, which HotNet2 requires. If you believe that your data contains network genes with positive heat scores, then you have formatting issues with your input data, which you can check by comparing your data with the example data in the paper/data folder of the HotNet2 GitHub repository.

Best,
Matt

Shivangi Agarwal

unread,
Apr 23, 2018, 1:05:52 PM4/23/18
to HotNet
Hi

Thanks for your response, I have observed that the edge file in the example data is as: (contains three column)

10465 13551 1
163 10465 1
3689 10465 1
10465 13277 1
10465 13275 1
10465 13273 1
10465 13274 1
10465 13279 1
10465 13272 1
2970 10465 1
637 10465 1
3003 10465 1

But, the one which I have generated for Hippie network using createNetwork.py is like (contains two columns)
1    702
1    1414
1    4934
1    5469
1    5649
1    7437
1    7464
1    11426
1    14137
1    15854
1    16243

Is the third column making difference?

Also, I have compared the matrix (PPR) files for both the example data and Hippie, it seems to be similar.

Please suggest

matthe...@brown.edu

unread,
Apr 23, 2018, 6:11:05 PM4/23/18
to HotNet
Hi Shivangi,

The third column in the edge list file is not an issue.  I would check your index-gene file and heat scores file.

Best,
Matt

Shivangi Agarwal

unread,
Apr 23, 2018, 11:10:39 PM4/23/18
to HotNet
Ya

With same heat score file, I am able to run with the multinet or other network, but the error is coming when I am running on Hippie network, here I am attaching my Hippie index file, Hippie edgefile and heat score file (json file).
Hippie_index_gene
Hippie_edgelist
mutation_f_conv_4.json

matthe...@brown.edu

unread,
Apr 24, 2018, 6:44:14 PM4/24/18
to HotNet
Hi Shivangi,

The gene names in your attached heat score file are completely different from the gene names in your attached index-gene file, so HotNet2 does not know how to assign heat scores to genes in your network. Please compare your data with the example data in the paper/data folder of the HotNet2 GitHub repository.

For a simple example, if you have genes A, B, and C with heat scores 1.23, 4.56, and 7.89, respectively, and interactions A - B and A - C, then you could use the following input files for HotNet2:

1. index_to_gene.tsv:
1    A
2    B
3    C

2. edge_list.tsv:
1    2
           1    3

3.  heat_score.tsv:
A    1.23
B    4.56
C    7.89 

Best,
Matt

Shivangi Agarwal

unread,
Apr 26, 2018, 11:39:16 PM4/26/18
to HotNet
Thanks for your reply. I am getting 8 delta subdirectories instead of four. Is there any problem with the procedure?

matthe...@brown.edu

unread,
Apr 29, 2018, 9:23:19 AM4/29/18
to HotNet
Hi Shivangi,

With default settings, HotNet2 should create four delta subdirectories for each combination of network and heat scores.  I would suggest deleting the eight delta subdirectories and trying again.  Please share your commands and more information about your output for more specific feedback.

Best,
Matt

Shivangi Agarwal

unread,
Apr 30, 2018, 12:16:35 PM4/30/18
to HotNet
Hi

I have used the following command

python HotNet2.py -nf  paper/data/networks/Hippie/Hippie_ppr_0.47.h5  -pnp paper/data/networks/Hippie/permuted/Hippie_ppr_0.47_##NUM##.h5    
-hf  paper/data/heats/final.json   -np  100  -hp  100   -o   results_id -c -1

Shivangi Agarwal

unread,
May 2, 2018, 10:47:48 PM5/2/18
to HotNet
Hi Matt

When I am running it again using following commands,
python HotNet2.py -nf paper/data/networks/HIPPIE/HIPPIE_ppr_0.47.h5 -pnp /media/ravi/Seagate\ Expansion\ Drive/hotnet2/paper/data/networks/Hippie/permuted/HIPPIE_ppr_0.47_##NUM##.h5 -hf paper/data/heat2/Final.json -np 100 -hp 100 -o results_media_2
it is giving me error as:   KeyError: 'PPR'
Ca you please help asap??


matthe...@brown.edu

unread,
May 3, 2018, 9:17:43 AM5/3/18
to HotNet
Hi Shivangi,

It is difficult to diagnose the problem without a full error message, but it looks like the makeNetworkFiles.py script may not have run successfully.  Did you encounter any errors when you ran the makeNetworkFiles.py script?  Also, from your previous messages, it seemed that HotNet2 previously ran without this error on your network.  Did you change something from then?

Best,
Matt

Shivangi Agarwal

unread,
May 3, 2018, 11:25:06 AM5/3/18
to HotNet
Hi
No, I have not changed anything. It ran when I did it for first time, but when I tried to run it again, it gave me MemoryError, then I placed permuted network files in external disk and run it again and got following error:


python HotNet2.py -nf paper/data/networks/HIPPIE/HIPPIE_ppr_0.47.h5 -pnp /media/ravi/Seagate\ Expansion\ Drive/hotnet2/paper/data/networks/Hippie/permuted/HIPPIE_ppr_0.47_##NUM##.h5 -hf paper/data/heat2/Final.json -np 100 -hp 100 -o results_media_2
WARNING: Could not import C module; falling back to NumPy for similarity matrix creation.

* Running HotNet2 in consensus mode...
    - HIPPIE mutation_f
Traceback (most recent call last):
  File "HotNet2.py", line 139, in <module>
    run(get_parser().parse_args(sys.argv[1:]))
  File "HotNet2.py", line 85, in run
    single_runs, consensus, linkers, auto_deltas, consensus_stats = consensus_with_stats(args, networks, heats)
  File "/home/ravi/Documents/test_downloaded_data/hotnet2-1.2.1/hotnet2/consensus.py", line 17, in consensus_with_stats

    single_runs, consensus, linkers, auto_deltas = consensus_run( args, networks, heats, verbose )
  File "/home/ravi/Documents/test_downloaded_data/hotnet2-1.2.1/hotnet2/consensus.py", line 73, in consensus_run
    result = run_helper(args, infmat, indexToGene, G, nname, pnp, heat, hname, addtl_genes, get_deltas_hotnet2, HN2_INFMAT_NAME, HN2_MAX_CC_SIZES, args.verbose)
  File "/home/ravi/Documents/test_downloaded_data/hotnet2-1.2.1/hotnet2/run.py", line 20, in run_helper
    deltas = get_deltas_fn(full_index2gene, heat, args.network_permutations, args.num_cores, infmat, addtl_genes, pnp, infmat_name, max_cc_sizes, verbose)
  File "/home/ravi/Documents/test_downloaded_data/hotnet2-1.2.1/hotnet2/run.py", line 63, in get_deltas_hotnet2
    MAX_CC_SIZE, max_cc_sizes, False, num_perms, num_cores, verbose)
  File "/home/ravi/Documents/test_downloaded_data/hotnet2-1.2.1/hotnet2/delta.py", line 243, in get_deltas_for_network
    sizes, not classic, num_cores, delta_selection_fn, verbose)
  File "/home/ravi/Documents/test_downloaded_data/hotnet2-1.2.1/hotnet2/delta.py", line 167, in network_delta_selection
    delta_maps = map_fn(network_delta_wrapper, args)
  File "/home/ravi/Documents/test_downloaded_data/hotnet2-1.2.1/hotnet2/delta.py", line 133, in network_delta_wrapper
    permuted_mat = np.asarray(hnio.load_hdf5(network_path)[infmat_name], np.float32)
KeyError: 'PPR'

Also, When I am running with 10 permutations, it ran successfully.
I don't know what is the problem?. Please help to identify the problem.

matthe...@brown.edu

unread,
May 4, 2018, 9:25:54 AM5/4/18
to HotNet
Hi Shivangi,

I suggest that you run HotNet2 with fewer network permutations.  If you have run HotNet2 successfully with 10 network permutations, then I would try 10 network permutations.  If you still encounter the same error, then I would run the makeNetworkFiles.py script again.  Moreover, the "MemoryError" indicates insufficient short-term memory (RAM), not insufficient long-term memory (SSD/hard disk).  The amount of RAM required for HotNet2 depends on the size of your interaction network, but several GB of available RAM are typically needed for recent gene-gene or protein-protein interaction networks.  Also, I would move the files from an external disk to local or network storage if possible.

Unfortunately, it is sometimes infeasible to remotely diagnose errors that occur because of different platforms (compiler/interpreters, operating systems, hardware, etc.), and I am sorry that you have experienced these errors.

Best,
Matt

Shivangi Agarwal

unread,
May 5, 2018, 2:33:21 AM5/5/18
to HotNet
I am having 96GB RAM, out of which only 14% is used. I don't know what is the problem, also it is giving 8 delta directories instead of 4 when I ran it with 10 permutations. Please help!!
Reply all
Reply to author
Forward
0 new messages