how to handle very large graphs in networkx?

[sampath methuku]

hi
i am doing a project of graph spanners and i want to run my
algorithms on very large graph (around 10000000 or more edges) . but
when i run my algorithms the process is getting blocked or killed .
i think due insufficient memory in ram to load such graphs is there
any way to solve this problem by maintaining the
graph database externally and reading it to networkx as a parts .



is there any suggestions in terms of memory usage& time complexity
while implementing algorithms which should run on very large graphs.

[Daπid]

It depends on what do you want to do. If you are only concerned about
"local" properties of the graph, you can try to load it in chunks.
Another possibility is to use a database instead of NX to store the
information.

PyTables is a good option for dealing with big datasets from Python.
You can think of it as a table where the intersection of row i with
column j is the weight of the link i->j.

David.

> --
> You received this message because you are subscribed to the Google Groups "networkx-discuss" group.
> To post to this group, send email to networkx...@googlegroups.com.
> To unsubscribe from this group, send email to networkx-discu...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/networkx-discuss?hl=en.
>

[Adrian Neumann]

If you want to handle huge networks you might want to use a language
with higher performance and lower memory requirements than Python. You
can prototype your algorithms with Python/networkx and translate them to
C++/Boost or whatever to run them on your large instances.