How to lower memory usage?

[Happy Hamster]

I'm trying to put a directory structure into a tree in order to be able to quickly find children of a given path.

It works, but consumes too much memory.

private RadixTree<FileEntry> tree = new ConcurrentRadixTree<>(new DefaultCharArrayNodeFactory());

final public static class FileEntry {
 private String path;
 private long size;
}

Then

tree.getKeysStartingWith("/tmp/test")

get's me what I want.

However, how can I make the "tmp" or "test" the smallest node name instead of "t" or "te" etc?

I will never query the tree for partial matches (e.g. "/tmp/te").

I am hoping to reduce memory usage this way.

[Niall Gallagher]

Hi,

You can reduce memory usage by using DefaultCharSequenceNodeFactory instead of DefaultCharArrayNodeFactory.
See NodeFactoryAndMemoryUsage though for the tradeoffs.

The radix tree does not allocate a node for each letter. It will group the characters of "test" into the same node whenever possible.
You can use the PrettyPrinter utility to print the actual structure of your tree to see which nodes are being allocated, see example here.

Another way to (significantly) reduce memory overhead (besides CharSequence-based nodes), would be to store strings in the tree in UTF-8 instead of UTF-16 (Java default) encoding.

I might write a UTF8NodeFactory over the weekend to support this (it has been on my to-do list), I'll let you know when done. In the meantime you could try yourself though, it's pretty straightforward if you adapt DefaultcharArrayNode and don't worry about the other specialized types of node.

Niall

--
-- You received this message because you are subscribed to the "concurrent-trees-discuss" group.
http://groups.google.com/group/concurrent-trees-discuss
---
You received this message because you are subscribed to the Google Groups "concurrent-trees-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concurrent-trees-d...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Niall]

This is implemented in concurrent-trees 2.3.0.

Try DefaultByteArrayNodeFactory or SmartArrayBasedNodeFactory - these may reduce memory overhead by 50% depending on the characters you have in your strings. If only ASCII characters, you should see a reduction in memory overhead. Documentation in NodeFactoryAndMemoryUsage.

Niall

To unsubscribe from this group and stop receiving emails from it, send an email to concurrent-trees-discuss+unsub...@googlegroups.com.