Running ClickHouse with limited memory

unread,

Jul 6, 2016, 9:11:08 AM7/6/16

to ClickHouse

Hi,

The documentation states that max_memory_usage is set to 10GB by default.

If I want to run ClickHouse and limit its memory consumption to only 1GB, what would be the implications?

What other configuration parameters should I change?

Thanks

unread,

Jul 6, 2016, 3:02:40 PM7/6/16

to ClickHouse

Setting 'max_memory_usage' is limiting memory for processing of single query.

Limiting it to 1 GB is fine.

There are also options 'max_memory_usage_for_user' and 'max_memory_usage_for_all_queries', which you could set to 1 GB too.

But server is consuming memory not only for query processing, but also during background merges and for index.

If you have rather wide tables, then 1 GB of memory will not be enough.

Running with total of 1 GB memory is difficult and could be done only if you have tables with low number of columns.

I think that limiting total amount of memory with 1 GB is a bad idea.

I recommend to do some experiments and watch.

unread,

Nov 15, 2019, 4:02:29 PM11/15/19

to ClickHouse

Hi,

I have two questions.

a. From document, max_memory_usage is for a query, is that true that it is for each single query?

For example, if I have two simultaneous queries, each use 8GiB, that would be OK. Is that right?

b. The current ClickHouse behavior is, when a query exceeds the max_memory_usage, it just errs out.

Is there a way to NOT to err out but just make the query slower?

Thanks!

unread,

Nov 15, 2019, 5:24:56 PM11/15/19

to ClickHouse

a. max_memory_usage -- is a limit for a query. If a query needs more memory than max_memory_usage the query will fail with error.

max_memory_usage does not reduce memory usage. It just a limit to prevent accident overusage or Linux OOM.

b. check max_bytes_before_external_sort max_bytes_before_external_group_by

Reply all

Reply to author

Forward