UIDs vs RAM? OOM

[John Humphreys]

I was trying to create ~150-million metrics (pushing host to metric, lots of hosts and metrics).

I started getting heap exceptions in OpenTSDB.  I'm only using it to create metrics via the API currently, I'm not throwing any data at it.

I started getting OOM / Out Of Memory errors in OpenTSDB.  Does the OpenTSDB implementation require keeping all UIDs in memory at the same time which would cause me to run out of heap? I would have assumed it was an LRU cache or something like that.

[ManOLamancha]

On Friday, September 8, 2017 at 7:17:11 AM UTC-7, John Humphreys wrote:

I started getting OOM / Out Of Memory errors in OpenTSDB.  Does the OpenTSDB implementation require keeping all UIDs in memory at the same time which would cause me to run out of heap? I would have assumed it was an LRU cache or something like that.

Unfortunately right now we only have a full cache. The "next" branch now has an LRU that you can enable via config: https://github.com/OpenTSDB/opentsdb/blob/next/src/utils/Config.java#L593-L596