Efficiently writing large set of trees to file

[Pinky Langat]

Currently, my script is iterating through a fairly large set of trees, pruning taxa from a specified list from each tree, and saving each pruned tree in a new tree list. However, this approach uses large amounts of memory before eventually writing the updated treelist to file. Is there a more efficient way to write the pruned set of trees than storing in a TreeList object, perhaps similar to how trees can be read in iteratively? I have not seen anything like it for writing in the documentation, but may have simply missed it.

Many thanks in advance,

Pinky Langat

[Jeet Sukumaran]

Hi,

Are you writing to Newick format?

If so, then you can always open a file stream and write each tree
successively to it.

E.g.
~~~
with open("results.tre", "w") as dest:
for tree in dendropy.Tree.yield_from_files(...):
# process ``tree`` here, pruning/etc.
tree.write(file=dest, schema="newick", ...)
~~~

Or, the "old school" approach:

~~~
dest = open("results.tre", "w")
for tree in dendropy.Tree.yield_from_files(...):
# process ``tree`` here, pruning/etc.
tree.write_to_stream(dest, "newick", ...)
dest.close()
~~~

For Nexus format, you would do something like the above, but need to
manually set up the pre-amble:

~~~
with open("results.tre", "w") as dest:
dest.write("#NEXUS\n\n")
dest.write("BEGIN TAXA;\n")
dest.write(...)
dest.write(...)
dest.write("END;\n\n")
dest.write("BEGIN TREES;\n")
for idx, tree in enumerate(dendropy.Tree.yield_from_files(...)):
# process ``tree`` here, pruning/etc.
dest.write(" TREE {} = ".format(idx))
tree.write(file=dest, schema="newick", ...)
dest.write("END;\n")
~~~

> --
> You received this message because you are subscribed to the Google
> Groups "DendroPy Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to dendropy-user...@googlegroups.com
> <mailto:dendropy-user...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

--



--------------------------------------
Jeet Sukumaran
--------------------------------------
jeetsu...@gmail.com
--------------------------------------
Blog/Personal Pages:
http://jeetworks.org/
GitHub Repositories:
http://github.com/jeetsukumaran
Photographs (as stream):
http://www.flickr.com/photos/jeetsukumaran/
Photographs (by galleries):
http://www.flickr.com/photos/jeetsukumaran/sets/
--------------------------------------