Error in forking cpimport.bin (errno-12); Cannot allocate memory

[Jonathan Nicol]

Hi,

After about a week of running with daily data loads from InnoDB, it starts to fail with:
Error in forking cpimport.bin (errno-12); Cannot allocate memory

This is a single-node cluster with 256G memory, trying to import about 700k rows. After I restart the database it works for about a week.
(and by the way, I've never had the "stop" command actually work properly, I always have to kill processes)

I saw there was a post on this topic from last year but the OP never responded.

I've got NumBlocksPct at 50. It's not clear to me if I need more or less memory for this disk block cache to allow the load to succeed.

It would also be helpful if you wouldn't mind giving me some tips on finding what PrimProc is using all its memory for.

thanks,
Jonathan

[Andrew Hutchings]

Hi Jonathan,

It does sound like the system is running out of memory for some reason.
Can you please send me a ColumnStore support report? If possible when
the problem happens, and we can look into this:

https://mariadb.com/kb/en/library/system-troubleshooting-mariadb-columnstore/#mariadb-columnstore-support-report-tool

As for your NumBlocksPct question, it is difficult to answer that
without knowing the rest of your configuration which the support tool
will tell us.

The majority of PrimProc's memory is an LRU block cache for select
queries. It will also use a small amount of memory for things such as
network buffers and the initial processing of joins and aggregates. With
NumBlocksPct set to 50 this means that 50% of your system's RAM for that
PM is being used as the block cache.

If you have a single server installation then TotalUmMemory will be the
other large consumer. This tells ExeMgr what limit to put on memory
usage during activities such as joins and aggregates. This again is a
percentage of system RAM defaulting to 25% for single server.

Hope this helps.

Kind Regards
Andrew

> --
> You received this message because you are subscribed to the Google
> Groups "MariaDB ColumnStore" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to mariadb-columns...@googlegroups.com
> <mailto:mariadb-columns...@googlegroups.com>.
> To post to this group, send email to
> mariadb-c...@googlegroups.com
> <mailto:mariadb-c...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mariadb-columnstore/efd27d71-c3d1-4cf6-8275-c88742794366%40googlegroups.com
> <https://groups.google.com/d/msgid/mariadb-columnstore/efd27d71-c3d1-4cf6-8275-c88742794366%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.

--
Andrew Hutchings (LinuxJedi)
Lead Engineer, MariaDB ColumnStore