InfiniDB to MariaDb columnstore migration issues

[argyris...@glispamedia.com]

Hello,

We migrated from InfiniDB 4.6 to latest mariadb columnstore using documented procedure (https://mariadb.com/kb/en/library/migrating-from-infinidb-4x-to-mariadb-columnstore/) with one difference. I did not suspend writes since we were not going to have schema changes during migration, only data.

So the steps I followed were:

1. /usr/local/Calpont/mysql/bin/mysqldump "all databases"   [Infini]

2. backup dbroots   [Infini]

3. create users and source schemas [Mariadb]

4. [Mariadb]

# mcsadmin shutdownSystem y
# cd /usr/local/mariadb/columnstore/
# mv data1 data1.bkp
# mv /backupdir/data1 . 
# rsync all dbroots multiple times from Infini since mysqldump for about 2 days
# cd data1/systemFiles/dbrm/
# mv BRM_saves_current BRM_saves_current.bkp
# cp ../../../data1.bkp/systemFiles/dbrm/BRM_saves_current . 
# mcsadmin startSystem

System starts up successfully but for every query I get output:

ERROR 1815 (HY000): Internal error: IDB-2006: 'table_name' does not exist in Columnstore.

Any help would be mostly appreciated

Many thanks,

Argyris 

[David Hill]

Can you run these commands and provide the output?

cd /usr/local/mariadb/columnstore/data1/systemFiles/dbrm/

ls -l

cat BRM_saves_current

cd /usr/local/mariadb/columnstore/bin

./editem -i

--
You received this message because you are subscribed to the Google Groups "MariaDB ColumnStore" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mariadb-columns...@googlegroups.com.
To post to this group, send email to mariadb-c...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mariadb-columnstore/ece4a4dc-8f0c-4a64-b292-9cc76f191e8a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

------------------

David Hill

MariaDB ColumnStore Development

MariaDB Corporation

http://www.mariadb.com

[argyris...@glispamedia.com]

Good morning David,

Assuming you mean to execute below commands from mariadb, not infini, my setup is slightly different in terms of paths (maybe due to AWS ami?):

cd /home/mysql/mariadb/columnstore/data1/systemFiles/dbrm

ls -l

total 253032

-rw-rw-r-- 1 mysql mysql      191 Dec  7 16:24 +

-rw-rw-r-- 1 mysql mysql 11816588 Jan 11 15:13 BRM_savesA_em

-rw-rw-r-- 1 mysql mysql 37752732 Jan 11 15:39 BRM_savesA_vbbm

-rw-rw-r-- 1 mysql mysql 73012856 Jan 11 15:39 BRM_savesA_vss

-rw-rw-r-- 1 mysql mysql 11816588 Jan 11 15:13 BRM_savesB_em

-rw-rw-r-- 1 mysql mysql 37751940 Jan 11 15:13 BRM_savesB_vbbm

-rw-rw-r-- 1 mysql mysql 73012064 Jan 11 15:39 BRM_savesB_vss

-rw-rw-r-- 1 mysql mysql       10 Jan 11 16:40 BRM_saves_current

-rw-rw-r-- 1 mysql mysql       53 Jan  9 16:24 BRM_saves_current.bkp

-rw-rw-r-- 1 mysql mysql 11505708 Jan 11 16:38 BRM_saves_em

-rw-rw-r-- 1 mysql mysql        5 Jan 11 16:43 BRM_saves_journal

-rw-rw-r-- 1 mysql mysql       84 Jan 11 16:38 BRM_saves_vbbm

-rw-rw-r-- 1 mysql mysql        8 Jan 11 16:38 BRM_saves_vss

-rw-rw-r-- 1 mysql mysql  2099208 Jan  9 16:24 oidbitmap

-rw-rw-r-- 1 mysql mysql       12 Jan 11 16:43 SMTxnID

-rw-rw-r-- 1 mysql mysql        4 Jan 11 15:13 tablelocks

cat BRM_saves_current

BRM_saves

cd /home/mysql/mariadb/columnstore/bin

./editem -i 0|8|1001|0|0|0|0|1|8|0|-9223372036854775808|9223372036854775807|436553|1| 8192|8|2001|0|6596|0|0|1|0|0|-9223372036854775808|9223372036854775807|3|0| 16384|8|1002|0|0|0|0|1|8|0|-9223372036854775808|9223372036854775807|436553|1| 24576|8|2004|0|0|0|0|1|0|0|-9223372036854775808|9223372036854775807|0|0| 32768|4|1003|0|0|0|0|1|4|0|-9223372036854775808|9223372036854775807|436553|1| 36864|4|1004|0|0|0|0|1|4|0|-9223372036854775808|9223372036854775807|436553|1| 40960|4|1005|0|0|0|0|1|4|0|-9223372036854775808|9223372036854775807|436553|1| 45056|4|1006|0|0|0|0|1|4|0|-9223372036854775808|9223372036854775807|436553|1| 49152|4|1007|0|0|0|0|1|4|0|-9223372036854775808|9223372036854775807|436553|1| 53248|4|1008|0|0|0|0|1|4|0|-9223372036854775808|9223372036854775807|436553|1| 57344|4|1009|0|0|0|0|1|4|0|-9223372036854775808|9223372036854775807|436553|1| 61440|4|1010|0|0|0|0|1|4|0|-9223372036854775808|9223372036854775807|436553|1| 65536|4|1011|0|0|0|0|1|4|0|-9223372036854775808|9223372036854775807|436553|1| 69632|8|1021|0|3|0|0|1|8|0|-9223372036854775808|9223372036854775807|436685|1| 77824|8|2061|0|0|0|0|1|0|0|-9223372036854775808|9223372036854775807|0|0| 86016|8|1022|0|3|0|0|1|8|0|-9223372036854775808|9223372036854775807|436685|1| 94208|8|2064|0|6597|0|0|1|0|0|-9223372036854775808|9223372036854775807|2|0| 102400|8|1023|0|3|0|0|1|8|0|-9223372036854775808|9223372036854775807|436685|1| 110592|8|2067|0|0|0|0|1|0|0|-9223372036854775808|9223372036854775807|0|0|

.

.

... continues a long ouput as above, would you need to see all of it?

Many thanks, Argyris

To unsubscribe from this group and stop receiving emails from it, send an email to mariadb-columnstore+unsub...@googlegroups.com.

To post to this group, send email to mariadb-c...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mariadb-columnstore/ece4a4dc-8f0c-4a64-b292-9cc76f191e8a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Hill]

Thanks for that info... so the dbrm data is there and it looks good...

So I guess how about the frontend schemas... You notes your did create those...

so do they should up a columnstore tables when you run the mcsmysql command for 'show create table xxxx' ?

So they show they exist and they are engine = columnstore?

And these same tables exist in this mysql db directory?

/home/mysql/mariadb/columnstore/mysql/db/

To unsubscribe from this group and stop receiving emails from it, send an email to mariadb-columns...@googlegroups.com.

To post to this group, send email to mariadb-c...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mariadb-columnstore/ece4a4dc-8f0c-4a64-b292-9cc76f191e8a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

------------------

David Hill

MariaDB ColumnStore Development

MariaDB Corporation

http://www.mariadb.com

--

You received this message because you are subscribed to the Google Groups "MariaDB ColumnStore" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mariadb-columns...@googlegroups.com.

To post to this group, send email to mariadb-c...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mariadb-columnstore/a634eb82-84af-4363-9e98-b1b1c6f4e946%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[argyris...@glispamedia.com]

The schemas I had created yes.

'show create table xxx' correctly showed query for table creation with engine=columnstore and tables exist under /home/mysql/mariadb/columnstore/mysql/db/ as .frm files.

I tried to drop a db in order to recreate it and at some point I got:

ERROR 1815 (HY000): Internal error: CAL0009: Drop table failed due to IDB-2033: Error occurred when calling system catalog.

and in crit.log in pm1 multiple times below msg:

PrimProc[111536]: 08.914149 |0|0|0| C 28 CAL0000: Invalid Range from HWM for lbid 14851, range size should be <= blocksReadAhead: HWM 6596, dbroot 1, highfbo 7167, lowfbo 6656, blocksReadAhead 512, range size -59

Should I try at this point to install a new mariadb cluster and repeat the migration process again?

Many thanks,

Argyris

To unsubscribe from this group and stop receiving emails from it, send an email to mariadb-columnstore+unsub...@googlegroups.com.

To post to this group, send email to mariadb-c...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mariadb-columnstore/ece4a4dc-8f0c-4a64-b292-9cc76f191e8a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

------------------

David Hill

MariaDB ColumnStore Development

MariaDB Corporation

http://www.mariadb.com

--
You received this message because you are subscribed to the Google Groups "MariaDB ColumnStore" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mariadb-columnstore+unsub...@googlegroups.com.

To post to this group, send email to mariadb-c...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mariadb-columnstore/a634eb82-84af-4363-9e98-b1b1c6f4e946%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Hill]

Maybe start again. Something is just not right...

David

To unsubscribe from this group and stop receiving emails from it, send an email to mariadb-columns...@googlegroups.com.

To post to this group, send email to mariadb-c...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mariadb-columnstore/ece4a4dc-8f0c-4a64-b292-9cc76f191e8a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

------------------

David Hill

MariaDB ColumnStore Development

MariaDB Corporation

http://www.mariadb.com

--
You received this message because you are subscribed to the Google Groups "MariaDB ColumnStore" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mariadb-columns...@googlegroups.com.

To post to this group, send email to mariadb-c...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mariadb-columnstore/a634eb82-84af-4363-9e98-b1b1c6f4e946%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

------------------

David Hill

MariaDB ColumnStore Development

MariaDB Corporation

http://www.mariadb.com

--

You received this message because you are subscribed to the Google Groups "MariaDB ColumnStore" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mariadb-columns...@googlegroups.com.

To post to this group, send email to mariadb-c...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mariadb-columnstore/fe844a11-45ff-4005-b4a2-c4cd85aac2fe%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[drrtuy]

Greetings Argyris,

DId you manage to migrate? If your dataset doesn't reach terabytes mb you can offload it in a text file and bulk insert it into a CS installation?

Regards,

Roman