Deadlock on latest 5.5 version

272 views
Skip to first unread message

SyRenity

unread,
Nov 6, 2011, 8:03:19 AM11/6/11
to codersh...@googlegroups.com
Hi.

We are testing the 5.5 version, and encountered a dead-lock on every UPDATE query.

The error message is as follows:

There was an SQL error: Deadlock found when trying to get lock; try restarting transaction - UPDATE `servers` SET `xxx` = 'xxxxxxxx', `xx` = 'xxxxx', `xxx` = 'xxxx', `xxx` = 'xxxxx', `xxxx` = 0 WHERE `xxx` = 1

There are no error messages in the mysql.log file. The environment was upgraded from 5.1 via mysql_upgrade.

Any idea what happens and how we resolve it?

Thanks.

Alex Yurchenko

unread,
Nov 6, 2011, 8:20:48 AM11/6/11
to codersh...@googlegroups.com

Hi,

I take it we're talking about mysql.servers table?

If so, then it is normally a MyISAM table, so it is not supported by
wsrep replication, hence refusal to update.

To go around this you can turn off wsrep replication per session:

mysql> SET wsrep_on=0;

This should do it, but note, that no query in this session will be
replicated, so you'll have to manually do it on each node. Of course,
SST would copy all the changes to new nodes.

Regards,
Alex

--
Alexey Yurchenko,
Codership Oy, www.codership.com
Skype: alexey.yurchenko, Phone: +358-400-516-011

SyRenity

unread,
Nov 6, 2011, 12:22:35 PM11/6/11
to codersh...@googlegroups.com
I actually meant servers table in our database.

By the way, setting wsrep to "none" solves this issue.

What I should provide to help diagnose this?

Regards.

seppo....@codership.com

unread,
Nov 6, 2011, 4:32:56 PM11/6/11
to codersh...@googlegroups.com

Quoting SyRenity <stas....@gmail.com>:

Table definition and exact update statement can give some hint for
this behavior. One potential explanation is that you have now
wsrep_retry_autocommit=0, and your workload generates plenty of
updates which were silenty retried in the 5.1 based version, but now
you see just the first deadlock error failures.

-seppo
--
http://www.codership.com seppo....@codership.com
skype: seppo_jaakola


SyRenity

unread,
Nov 6, 2011, 5:34:18 PM11/6/11
to codersh...@googlegroups.com
Autocommit is on 1, and this happens with a single update.

EllisGL

unread,
Nov 9, 2011, 3:40:46 PM11/9/11
to codership
I'm having this issue too, but only if I'm trying to import say > 40
rows. I've even cranked up the autocommit to 5. It ends locking the
nodes.
If I have only one node up, it's fine.

Here's my logs.

main node:
111109 18:35:32 [Note] WSREP: TO BEGIN: 0, 0 : drop table if exists
`lockbox`.`customers`
111109 18:35:32 [Note] WSREP: TO BEGIN: 57, 2
111109 18:35:32 [Note] WSREP: avoiding binlog rotate due to TO
isolation: 1
111109 18:35:32 [Note] WSREP: TO END: 57, 2 : drop table if exists
`lockbox`.`customers`
111109 18:35:32 [Note] WSREP: TO END: 57
111109 18:35:32 [Note] WSREP: TO BEGIN: 57, 0 : CREATE TABLE
`customers` (
`customer_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`aid` char(10) NOT NULL,
`customername` varchar(50) NOT NULL,
`updated` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (`customer
111109 18:35:32 [Note] WSREP: TO BEGIN: 58, 2
111109 18:35:32 [Note] WSREP: avoiding binlog rotate due to TO
isolation: 1
111109 18:35:32 [Note] WSREP: TO END: 58, 2 : CREATE TABLE `customers`
(
`customer_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`aid` char(10) NOT NULL,
`customername` varchar(50) NOT NULL,
`updated` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (`customer_i
111109 18:35:32 [Note] WSREP: TO END: 58
111109 18:35:32 [Note] WSREP: TO BEGIN: 59, 0 : drop table if exists
`lockbox`.`ips`
111109 18:35:32 [Note] WSREP: TO BEGIN: 60, 2
111109 18:35:32 [Note] WSREP: avoiding binlog rotate due to TO
isolation: 1
111109 18:35:32 [Note] WSREP: TO END: 60, 2 : drop table if exists
`lockbox`.`ips`
111109 18:35:32 [Note] WSREP: TO END: 60
111109 18:35:32 [Note] WSREP: TO BEGIN: 60, 0 : CREATE TABLE `ips` (
`ip_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`ip` int(10) unsigned NOT NULL,
PRIMARY KEY (`ip_id`)
) ENGINE=InnoDB AUTO_INCREMENT=102 DEFAULT CHARSET=utf8
111109 18:35:32 [Note] WSREP: (b10492c5-0b33-11e1-0800-ce3af4d0715c,
'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive
peers: tcp://10.10.10.2:4567
111109 18:35:33 [Note] WSREP: (b10492c5-0b33-11e1-0800-ce3af4d0715c,
'tcp://0.0.0.0:4567') reconnecting to
ace4ed64-0b1b-11e1-0800-52d1e192ff91 (tcp://10.10.10.2:4567), attempt
0
111109 18:35:34 [Note] WSREP: remote endpoint tcp://10.10.10.2:4567
changed identity ace4ed64-0b1b-11e1-0800-52d1e192ff91 ->
da340cb6-0b1b-11e1-0800-424b8387e78e
111109 18:35:34 [Note] WSREP: (b10492c5-0b33-11e1-0800-ce3af4d0715c,
'tcp://0.0.0.0:4567') turning message relay requesting off
111109 18:35:37 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:38 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:38 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:39 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:39 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:40 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:40 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:41 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:41 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:42 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:42 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:43 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:43 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:44 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:44 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:45 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:45 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:46 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:46 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) suspecting node:
ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:47 [Note] WSREP: evs::proto(b10492c5-0b33-11e1-0800-
ce3af4d0715c, GATHER,
view_id(REG,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2)) detected inactive
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 18:35:47 [Note] WSREP: New COMPONENT: primary = no, my_idx = 0,
memb_num = 1
111109 18:35:47 [Note] WSREP: Flow-control interval: [8, 16]
111109 18:35:47 [Note] WSREP: Received NON-PRIMARY.
111109 18:35:47 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 60)
111109 18:35:47 [Note] WSREP: GMCast::handle_stable_view:
view(view_id(NON_PRIM,ace4ed64-0b1b-11e1-0800-52d1e192ff91,2) memb {
b10492c5-0b33-11e1-0800-ce3af4d0715c,
} joined {
} left {
} partitioned {
ace4ed64-0b1b-11e1-0800-52d1e192ff91,
})
111109 18:35:47 [Note] WSREP: New cluster view: group UUID:
ad14378a-0b30-11e1-0800-065309e12101, conf# -1: non-Primary, number of
nodes: 1, my index: 0, first seqno: 61, protocol version 1
111109 18:35:47 [Warning] WSREP: wsrep_notify_cmd is not defined,
skipping notification.
111109 18:35:47 [Warning] WSREP: TO isolation failed for: 3, sql:
CREATE TABLE `ips` (
`ip_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`ip` int(10) unsigned NOT NULL,
PRIMARY KEY (`ip_id`)
) ENGINE=InnoDB AUTO_INCREMENT=102 DEFAULT CHARSET=utf8. Check wsrep
connection state and




Second node:
111109 15:43:22 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this
binary
or one of the libraries it was linked against is corrupt, improperly
built,
or misconfigured. This error can also be caused by malfunctioning
hardware.
We will try our best to scrape up some info that will hopefully help
diagnose
the problem, but since we have already crashed, something is
definitely wrong
and this may fail.

key_buffer_size=33554432
read_buffer_size=2097152
max_used_connections=3
max_threads=100
thread_count=2
connection_count=2
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads =
1057939 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7f6c80000990
Attempting backtrace. You can use the following information to find
out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f6d406dcd48 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7b4129]
/usr/sbin/mysqld(handle_segfault+0x43a)[0x510d6a]
/lib64/libpthread.so.0[0x31e2c0f4c0]
/usr/sbin/mysqld(_ZN13MYSQL_BIN_LOG7log_xidEP3THDy+0x71)[0x7210b1]
/usr/sbin/mysqld(_Z15ha_commit_transP3THDb+0x516)[0x692586]
/usr/sbin/mysqld(_Z12trans_commitP3THD+0x3a)[0x635e2a]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1cb3)[0x58bfa3]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x158)
[0x58f9b8]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj
+0x1e1f)[0x59202f]
/usr/sbin/mysqld[0x592781]
/usr/sbin/mysqld(_Z17wsrep_bf_apply_cbPvP16wsrep_apply_datal+0x8e)
[0x5929fe]
/usr/lib64/galera/libgalera_smm.so(+0x1797e6)[0x7f6ca7c357e6]
/usr/lib64/galera/
libgalera_smm.so(_ZN6galera13ReplicatorSMM9apply_trxEPvPNS_9TrxHandleE
+0x8c)[0x7f6ca7c36bdc]
/usr/lib64/galera/
libgalera_smm.so(_ZN6galera13ReplicatorSMM11process_trxEPvPNS_9TrxHandleE
+0x45)[0x7f6ca7c395d5]
/usr/lib64/galera/
libgalera_smm.so(_ZN6galera15GcsActionSource8dispatchEPvPKvm12gcs_act_typell
+0x6b3)[0x7f6ca7c343e3]
/usr/lib64/galera/
libgalera_smm.so(_ZN6galera15GcsActionSource7processEPv+0x94)
[0x7f6ca7c34864]
/usr/lib64/galera/
libgalera_smm.so(_ZN6galera13ReplicatorSMM10async_recvEPv+0x110)
[0x7f6ca7c37f10]
/usr/lib64/galera/libgalera_smm.so(galera_recv+0x23)[0x7f6ca7c49283]
/usr/sbin/mysqld(_Z25wsrep_replication_processP3THD+0x7d)[0x5895dd]
/usr/sbin/mysqld(start_wsrep_THD+0x451)[0x516d21]
/lib64/libpthread.so.0[0x31e2c077e1]
/lib64/libc.so.6(clone+0x6d)[0x31e24e18ed]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7f6c800080c8): is an invalid pointer
Connection ID (thread ID): 2
Status: NOT_KILLED

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html
contains
information that should help you find out what is causing the crash.
111109 15:43:22 mysqld_safe Number of processes running now: 0
111109 15:43:22 mysqld_safe mysqld restarted
111109 15:43:22 [Note] Plugin 'FEDERATED' is disabled.
111109 15:43:22 InnoDB: The InnoDB memory heap is disabled
111109 15:43:22 InnoDB: Mutexes and rw_locks use GCC atomic builtins
111109 15:43:22 InnoDB: Compressed tables use zlib 1.2.3
111109 15:43:22 InnoDB: Using Linux native AIO
111109 15:43:22 InnoDB: Initializing buffer pool, size = 2.0G
111109 15:43:23 InnoDB: Completed initialization of buffer pool
111109 15:43:23 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 2015485
111109 15:43:23 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the
doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 2022802
InnoDB: Transaction 9B9 was in the XA prepared state.
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 0 row operations to undo
InnoDB: Trx id counter is B00
111109 15:43:23 InnoDB: Starting an apply batch of log records to the
database...
InnoDB: Progress in percents: 10 11 12 13 14 15 16 17 18 19 20 21 22
23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45
46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68
69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91
92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position 0 107, file name ./mysql-bin.
000007
InnoDB: Starting in background the rollback of uncommitted
transactions
111109 15:43:23 InnoDB: Rollback of non-prepared transactions
completed
111109 15:43:23 InnoDB: Waiting for the background threads to start
111109 15:43:24 InnoDB: 1.1.8 started; log sequence number 2022802
111109 15:43:24 [Note] WSREP: Initial TC log open: binlog
111109 15:43:24 [Note] Recovering after a crash using mysql-bin
111109 15:43:24 [Note] Starting crash recovery...
111109 15:43:24 InnoDB: Starting recovery for XA transactions...
111109 15:43:24 InnoDB: Transaction 9B9 in prepared state after
recovery
111109 15:43:24 InnoDB: Transaction contains changes to 3 rows
111109 15:43:24 InnoDB: 1 transactions in prepared state after
recovery
111109 15:43:24 [Note] Found 1 prepared transaction(s) in InnoDB
111109 15:43:24 [Note] Crash recovery finished.
111109 15:43:25 [Warning] 'user' entry 'ro...@db-c.myhost.com' ignored
in --skip-name-resolve mode.
111109 15:43:25 [Warning] 'proxies_priv' entry '@ root@db-
c.myhost.com' ignored in --skip-name-resolve mode.
111109 15:43:25 [Note] Event Scheduler: Loaded 0 events
111109 15:43:25 [Note] WSREP: wsrep_load(): loading provider library '/
usr/lib64/galera/libgalera_smm.so'
111109 15:43:25 [Note] WSREP: wsrep_load(): Galera 21.1.0(r86) by
Codership Oy <in...@codership.com> loaded succesfully.
111109 15:43:25 [Note] WSREP: Reusing existing '/var/lib/mysql//
galera.cache'.
111109 15:43:25 [Note] WSREP: Passing config to GCS: gcache.dir = /var/
lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0;
gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M;
gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 0.5;
gcs.fc_limit = 16; gcs.fc_master_slave = NO; gcs.max_packet_size =
64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit =
9223372036854775807; gcs.recv_q_soft_limit = 0.25;
replicator.commit_order = 3
111109 15:43:25 [Note] WSREP: Start replication
111109 15:43:25 [Note] WSREP: Found saved state:
00000000-0000-0000-0000-000000000000:-1
111109 15:43:25 [Note] WSREP: Assign initial position for
certification: -1, protocol version: 1
111109 15:43:25 [Note] WSREP: Setting initial position to
00000000-0000-0000-0000-000000000000:-1
111109 15:43:25 [Note] WSREP: protonet asio version 0
111109 15:43:25 [Note] WSREP: backend: asio
111109 15:43:25 [Note] WSREP: GMCast version 0
111109 15:43:25 [Note] WSREP: (da340cb6-0b1b-11e1-0800-424b8387e78e,
'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
111109 15:43:25 [Note] WSREP: (da340cb6-0b1b-11e1-0800-424b8387e78e,
'tcp://0.0.0.0:4567') multicast: , ttl: 1
111109 15:43:25 [Note] WSREP: EVS version 0
111109 15:43:25 [Note] WSREP: PC version 0
111109 15:43:25 [Note] WSREP: gcomm: connecting to group
'my_wsrep_cluster', peer '10.10.10.3:4567'
111109 15:43:30 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:30 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:31 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:31 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:32 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:32 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:33 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:33 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:34 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:34 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:35 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:35 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:36 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:36 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:37 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91
111109 15:43:37 [Note] WSREP:
evs::proto(da340cb6-0b1b-11e1-0800-424b8387e78e, GATHER,
view_id(TRANS,da340cb6-0b1b-11e1-0800-424b8387e78e,0)) suspecting
node: ace4ed64-0b1b-11e1-0800-52d1e192ff91

seppo....@codership.com

unread,
Nov 9, 2011, 4:56:38 PM11/9/11
to codersh...@googlegroups.com
What do ou mean with "import" here, are you loading in a mysqldump?

The second node has a crash, and it looks like a result of binlogging
related problem, which was fixed just recently:
https://bugs.launchpad.net/codership-mysql/+bug/884615. Do you have
log_bin enabled?
Fix for this bug will be part of 1.1 release targeted for end of Nov.

-seppo


Quoting EllisGL <xel...@gmail.com>:

Alex Yurchenko

unread,
Nov 9, 2011, 5:03:44 PM11/9/11
to codersh...@googlegroups.com
1) What's with the timestamps? Have you synchronized clocks on the
nodes? It is not required, but without it it is pretty impossible to
parse the logs.
2) Are you inserting concurrently?
3) Are you inserting concurrently on both nodes?
4) Do you use autocommit INSERTs?
5) What does "SHOW VARIABLES LIKE '%autoinc%'" say on both nodes?

--

EllisGL

unread,
Nov 9, 2011, 5:07:54 PM11/9/11
to codership
Pretty much a mysqldump import - using SQLYog to copy db's over.
I've tried with log_bin on and off.

On Nov 9, 3:56 pm, seppo.jaak...@codership.com wrote:
> What do ou mean with "import" here, are you loading in a mysqldump?
>
> The second node has a crash, and it looks like a result of binlogging
> related problem, which was fixed just recently:  https://bugs.launchpad.net/codership-mysql/+bug/884615. Do you have
> log_bin enabled?
> Fix for this bug will be part of 1.1 release targeted for end of Nov.
>
> -seppo
>
> > [0x5929fe]...
>
> read more »

SyRenity

unread,
Nov 10, 2011, 3:39:46 AM11/10/11
to codersh...@googlegroups.com

Hi.

To update here, I don't see anything in logs related to the issue.

Moreover, some table do allow inserts normally, so seems as possibly a JOIN error again.

I will arrange and post the table structures shortly.

Regards.

EllisGL

unread,
Nov 10, 2011, 8:04:14 AM11/10/11
to codership
1. I've sync'd my clocks.
2. Serial inserts. Basically a MySQL dump
3. Only inserting on the main node
4. Yes
5. Both comeback with innodb_autoinc_lock_mode = 2

On Nov 9, 4:03 pm, Alex Yurchenko <alexey.yurche...@codership.com>
wrote:
> > [0x5929fe]...
>
> read more »

seppo....@codership.com

unread,
Nov 10, 2011, 8:24:20 AM11/10/11
to codersh...@googlegroups.com
There is something odd in this behavior, if you only insert in the
main node, there is no way the second node could "interfere" and cause
deadlocks.

Maybe there was some data remaining in the database...

This variable: wsrep_drupal_282555_workaround has now default value 0
(used to be 1 previously), it may play some role here. It does silent
retry for an autoincrement insert failing for duplicate key.

You could also exercise one more time with query logging enabled in
both nodes:
mysql> SET GLOBAL log=1;

You should see in the query logs inserts only in main node, and their
primary keys should be unique.

(remember to unset 'log' after the test, it will fill your disk and
make your mysqld sloooow)

-seppo

SyRenity

unread,
Nov 10, 2011, 8:36:05 AM11/10/11
to codersh...@googlegroups.com
Below is example to query and it's result:

UPDATE `devices` SET `int_ip` = '', `last_contact` = '2011-11-10 08:48:42', `ext_ip` = 'dev1.domain.local', `state` = 'ERROR', `data` = '' WHERE `id` = 302;
/* SQL Error (1213): Deadlock found when trying to get lock; try restarting transaction */


And the table scheme:
http://pastie.org/private/r6qfzx7g2znsuytuheltg

Again, this happens only when Galera provider specified.

seppo....@codership.com

unread,
Nov 10, 2011, 8:37:46 AM11/10/11
to codersh...@googlegroups.com
Check the wsrep status in all nodes, your should see something like:
wsrep_local_state_comment | Synced (6)
wsrep_cluster_status | Primary
wsrep_ready | ON

If wsrep_ready=OFF, you can connect but cannot issue anything else
that SET and SHOW commands. All other commands will give and error:

ERROR 1047 (08S01): Unknown command

Maybe some of your nodes was in this state...

-seppo

SyRenity

unread,
Nov 10, 2011, 8:38:47 AM11/10/11
to codersh...@googlegroups.com
I posted above the query and scheme, hope it visible in thread.

seppo....@codership.com

unread,
Nov 10, 2011, 8:45:15 AM11/10/11
to codersh...@googlegroups.com
Some questions:

How many concurrent updates like this do you run at the same time?
Do your updates go to same node or to several nodes?
Did row 302 exist when this update was executed?

-seppo

EllisGL

unread,
Nov 10, 2011, 9:44:30 AM11/10/11
to codership
Here's the dump: http://pastie.org/2841914


Main node:
111110 8:38:51 [Note] WSREP: TO BEGIN: 76, 0 : CREATE DATABASE /*!
32312 IF NOT EXISTS*/`lockbox` /*!40100 DEFAULT CHARACTER SET utf8 */
111110 8:38:51 [Note] WSREP: TO BEGIN: 77, 2
111110 8:38:51 [Note] WSREP: avoiding binlog rotate due to TO
isolation: 1
111110 8:38:51 [Note] WSREP: TO END: 77, 2 : CREATE DATABASE /*!32312
IF NOT EXISTS*/`lockbox` /*!40100 DEFAULT CHARACTER SET utf8 */
111110 8:38:51 [Note] WSREP: TO END: 77
111110 8:38:52 [Note] WSREP: TO BEGIN: 77, 0 : /*Table structure for
table `customers` */
CREATE TABLE `customers` (
`customer_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`aid` char(10) NOT NULL,
`customername` varchar(50) NOT NULL,
`updated` datetime NOT NULL DEFAULT '000
111110 8:38:52 [Note] WSREP: TO BEGIN: 78, 2
111110 8:38:53 [Note] WSREP: avoiding binlog rotate due to TO
isolation: 1
111110 8:38:53 [Note] WSREP: TO END: 78, 2 : /*Table structure for
table `customers` */
CREATE TABLE `customers` (
`customer_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`aid` char(10) NOT NULL,
`customername` varchar(50) NOT NULL,
`updated` datetime NOT NULL DEFAULT '0000-
111110 8:38:53 [Note] WSREP: TO END: 78
111110 8:38:53 [Note] WSREP: TO BEGIN: 79, 0 : /*Table structure for
table `ips` */
CREATE TABLE `ips` (
`ip_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`ip` int(10) unsigned NOT NULL,
PRIMARY KEY (`ip_id`)
) ENGINE=InnoDB AUTO_INCREMENT=102 DEFAULT CHARSET=utf8
111110 8:38:53 [Note] WSREP: (59ab9a52-0ba9-11e1-0800-94029838744b,
'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive
peers: tcp://10.10.10.2:4567
111110 8:38:54 [Note] WSREP: (59ab9a52-0ba9-11e1-0800-94029838744b,
'tcp://0.0.0.0:4567') reconnecting to
64c48c03-0ba9-11e1-0800-1d78f9d22d85 (tcp://10.10.10.2:4567), attempt
0
111110 8:38:55 [Note] WSREP: remote endpoint tcp://10.10.10.2:4567
changed identity 64c48c03-0ba9-11e1-0800-1d78f9d22d85 ->
b814edc9-0ba9-11e1-0800-db575acb6468
111110 8:38:55 [Note] WSREP: (59ab9a52-0ba9-11e1-0800-94029838744b,
'tcp://0.0.0.0:4567') turning message relay requesting off
111110 8:38:58 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:38:59 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:38:59 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:00 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:00 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:01 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:01 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:02 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:02 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:03 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:03 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:04 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:04 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:05 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:05 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:06 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:06 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:07 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:07 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:08 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) suspecting node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:08 [Note] WSREP:
evs::proto(59ab9a52-0ba9-11e1-0800-94029838744b, GATHER, view_id(REG,
59ab9a52-0ba9-11e1-0800-94029838744b,2)) detected inactive node:
64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:08 [Note] WSREP: GMCast::handle_stable_view:
view(view_id(NON_PRIM,59ab9a52-0ba9-11e1-0800-94029838744b,2) memb {
59ab9a52-0ba9-11e1-0800-94029838744b,
} joined {
} left {
} partitioned {
64c48c03-0ba9-11e1-0800-1d78f9d22d85,
})
111110 8:39:08 [Note] WSREP: New COMPONENT: primary = no, my_idx = 0,
memb_num = 1
111110 8:39:08 [Note] WSREP: Flow-control interval: [8, 16]
111110 8:39:08 [Note] WSREP: Received NON-PRIMARY.
111110 8:39:08 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 79)
111110 8:39:08 [Warning] WSREP: TO isolation failed for: 3, sql: /
*Table structure for table `ips` */
CREATE TABLE `ips` (
`ip_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`ip` int(10) unsigned NOT NULL,
PRIMARY KEY (`ip_id`)
) ENGINE=InnoDB AUTO_INCREMENT=102 DEFAULT CHARSET=ut
111110 8:39:08 [Note] WSREP: New cluster view: group UUID:
ad14378a-0b30-11e1-0800-065309e12101, conf# -1: non-Primary, number of
nodes: 1, my index: 0, first seqno: 80, protocol version 1
111110 8:39:08 [Warning] WSREP: wsrep_notify_cmd is not defined,
skipping notification.




Second node:
111110 8:38:54 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this
binary
or one of the libraries it was linked against is corrupt, improperly
built,
or misconfigured. This error can also be caused by malfunctioning
hardware.
We will try our best to scrape up some info that will hopefully help
diagnose
the problem, but since we have already crashed, something is
definitely wrong
and this may fail.

key_buffer_size=33554432
read_buffer_size=2097152
max_used_connections=3
max_threads=100
thread_count=3
connection_count=3
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads =
1057939 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7f68e8000990
Attempting backtrace. You can use the following information to find
out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x7f69a84dad48 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x39)[0x7b4129]
/usr/sbin/mysqld(handle_segfault+0x43a)[0x510d6a]
/lib64/libpthread.so.0[0x31e2c0f4c0]
/usr/sbin/mysqld(_ZN13MYSQL_BIN_LOG7log_xidEP3THDy+0x71)[0x7210b1]
/usr/sbin/mysqld(_Z15ha_commit_transP3THDb+0x516)[0x692586]
/usr/sbin/mysqld(_Z12trans_commitP3THD+0x3a)[0x635e2a]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x1cb3)[0x58bfa3]
/usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x158)
[0x58f9b8]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj
+0x1e1f)[0x59202f]
/usr/sbin/mysqld[0x592781]
/usr/sbin/mysqld(_Z17wsrep_bf_apply_cbPvP16wsrep_apply_datal+0x8e)
[0x5929fe]
/usr/lib64/galera/libgalera_smm.so(+0x1797e6)[0x7f690fc357e6]
/usr/lib64/galera/
libgalera_smm.so(_ZN6galera13ReplicatorSMM9apply_trxEPvPNS_9TrxHandleE
+0x8c)[0x7f690fc36bdc]
/usr/lib64/galera/
libgalera_smm.so(_ZN6galera13ReplicatorSMM11process_trxEPvPNS_9TrxHandleE
+0x45)[0x7f690fc395d5]
/usr/lib64/galera/
libgalera_smm.so(_ZN6galera15GcsActionSource8dispatchEPvPKvm12gcs_act_typell
+0x6b3)[0x7f690fc343e3]
/usr/lib64/galera/
libgalera_smm.so(_ZN6galera15GcsActionSource7processEPv+0x94)
[0x7f690fc34864]
/usr/lib64/galera/
libgalera_smm.so(_ZN6galera13ReplicatorSMM10async_recvEPv+0x110)
[0x7f690fc37f10]
/usr/lib64/galera/libgalera_smm.so(galera_recv+0x23)[0x7f690fc49283]
/usr/sbin/mysqld(_Z25wsrep_replication_processP3THD+0x7d)[0x5895dd]
/usr/sbin/mysqld(start_wsrep_THD+0x451)[0x516d21]
/lib64/libpthread.so.0[0x31e2c077e1]
/lib64/libc.so.6(clone+0x6d)[0x31e24e18ed]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7f68e8008098): is an invalid pointer
Connection ID (thread ID): 2
Status: NOT_KILLED

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html
contains
information that should help you find out what is causing the crash.
111110 08:38:54 mysqld_safe Number of processes running now: 0
111110 08:38:54 mysqld_safe mysqld restarted
111110 8:38:54 [Note] Plugin 'FEDERATED' is disabled.
111110 8:38:54 InnoDB: The InnoDB memory heap is disabled
111110 8:38:54 InnoDB: Mutexes and rw_locks use GCC atomic builtins
111110 8:38:54 InnoDB: Compressed tables use zlib 1.2.3
111110 8:38:54 InnoDB: Using Linux native AIO
111110 8:38:54 InnoDB: Initializing buffer pool, size = 2.0G
111110 8:38:54 InnoDB: Completed initialization of buffer pool
111110 8:38:54 InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn 2519347
111110 8:38:54 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the
doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number 2525014
InnoDB: Transaction 133F was in the XA prepared state.
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 0 row operations to undo
InnoDB: Trx id counter is 1500
111110 8:38:54 InnoDB: Starting an apply batch of log records to the
database...
InnoDB: Progress in percents: 12 13 14 15 16 17 18 19 20 21 22 23 24
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70
71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93
94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position 0 107, file name ./mysql-bin.
000020
InnoDB: Starting in background the rollback of uncommitted
transactions
111110 8:38:55 InnoDB: Rollback of non-prepared transactions
completed
111110 8:38:55 InnoDB: Waiting for the background threads to start
111110 8:38:56 InnoDB: 1.1.8 started; log sequence number 2525014
111110 8:38:56 [Note] WSREP: Initial TC log open: binlog
111110 8:38:56 [Note] Recovering after a crash using mysql-bin
111110 8:38:56 [Note] Starting crash recovery...
111110 8:38:56 InnoDB: Starting recovery for XA transactions...
111110 8:38:56 InnoDB: Transaction 133F in prepared state after
recovery
111110 8:38:56 InnoDB: Transaction contains changes to 3 rows
111110 8:38:56 InnoDB: 1 transactions in prepared state after
recovery
111110 8:38:56 [Note] Found 1 prepared transaction(s) in InnoDB
111110 8:38:56 [Note] Crash recovery finished.
111110 8:38:56 [Warning] 'user' entry 'ro...@db-c.netfire.com' ignored
in --skip-name-resolve mode.
111110 8:38:56 [Warning] 'proxies_priv' entry '@ root@db-
c.netfire.com' ignored in --skip-name-resolve mode.
111110 8:38:56 [Note] Event Scheduler: Loaded 0 events
111110 8:38:56 [Note] WSREP: wsrep_load(): loading provider library '/
usr/lib64/galera/libgalera_smm.so'
111110 8:38:56 [Note] WSREP: wsrep_load(): Galera 21.1.0(r86) by
Codership Oy <in...@codership.com> loaded succesfully.
111110 8:38:56 [Note] WSREP: Reusing existing '/var/lib/mysql//
galera.cache'.
111110 8:38:56 [Note] WSREP: Passing config to GCS: gcache.dir = /var/
lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0;
gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M;
gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 0.5;
gcs.fc_limit = 16; gcs.fc_master_slave = NO; gcs.max_packet_size =
64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit =
9223372036854775807; gcs.recv_q_soft_limit = 0.25;
replicator.commit_order = 3
111110 8:38:56 [Note] WSREP: Start replication
111110 8:38:56 [Note] WSREP: Found saved state:
00000000-0000-0000-0000-000000000000:-1
111110 8:38:56 [Note] WSREP: Assign initial position for
certification: -1, protocol version: 1
111110 8:38:56 [Note] WSREP: Setting initial position to
00000000-0000-0000-0000-000000000000:-1
111110 8:38:56 [Note] WSREP: protonet asio version 0
111110 8:38:56 [Note] WSREP: backend: asio
111110 8:38:56 [Note] WSREP: GMCast version 0
111110 8:38:56 [Note] WSREP: (b814edc9-0ba9-11e1-0800-db575acb6468,
'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
111110 8:38:56 [Note] WSREP: (b814edc9-0ba9-11e1-0800-db575acb6468,
'tcp://0.0.0.0:4567') multicast: , ttl: 1
111110 8:38:56 [Note] WSREP: EVS version 0
111110 8:38:56 [Note] WSREP: PC version 0
111110 8:38:56 [Note] WSREP: gcomm: connecting to group
'my_wsrep_cluster', peer '10.10.10.3:4567'
111110 8:39:02 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:02 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:03 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:03 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:04 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:04 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:05 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:05 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:06 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:06 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:07 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:07 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:08 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:08 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85
111110 8:39:09 [Note] WSREP: evs::proto(b814edc9-0ba9-11e1-0800-
db575acb6468, GATHER, view_id(TRANS,b814edc9-0ba9-11e1-0800-
db575acb6468,0)) suspecting node: 64c48c03-0ba9-11e1-0800-1d78f9d22d85


On Nov 10, 7:24 am, seppo.jaak...@codership.com wrote:
> There is something odd in this behavior, if you only insert in the
> main node, there is no way the second node could "interfere" and cause
> deadlocks.
>
> Maybe there was some data remaining in the database...
>
> This variable: wsrep_drupal_282555_workaround has now default value 0
> (used to be 1 previously), it may play some role here. It does silent
> retry for an autoincrement insert failing for duplicate key.
>
> You could also exercise one more time with query logging enabled in
> both nodes:
> mysql> SET GLOBAL log=1;
>
> You should see in the query logs inserts only in main node, and their
> primary keys should be unique.
>
> (remember to unset 'log' after the test, it will fill your disk and
> make your mysqld sloooow)
>
> -seppo
>
> >> > or one of the libraries it was linked against is corrupt,...
>
> read more »

seppo....@codership.com

unread,
Nov 10, 2011, 10:47:30 AM11/10/11
to codersh...@googlegroups.com
I just loaded this dump into a two node cluster, and no issues here.
I'm using development head build, so my results are a bit different.

Try disabling log_bin and log_slave_updates options, your issue might
relate to a bug about mysqlbinlog compatibility.

-seppo

Quoting EllisGL <xel...@gmail.com>:

EllisGL

unread,
Nov 10, 2011, 11:57:01 AM11/10/11
to codership
I can't get log_bin to turn off.
I've tried: log_bin = off, log_bin = 0 and log_bin = false in my cnf

On Nov 10, 9:47 am, seppo.jaak...@codership.com wrote:
> I just loaded this dump into a two node cluster, and no issues here.
> I'm using development head build, so my results are a bit different.
>
> Try disabling log_bin and log_slave_updates options, your issue might
> relate to a bug about mysqlbinlog compatibility.
>
> -seppo
>
> > libgalera_smm.so(_ZN6galera13ReplicatorSMM11process_trxEPvPNS_9TrxHandleE...
>
> read more »

seppo....@codership.com

unread,
Nov 10, 2011, 12:15:54 PM11/10/11
to codersh...@googlegroups.com
just don't put the log_bin and log_slave_updates in my.cnf, or comment
it out...

EllisGL

unread,
Nov 10, 2011, 12:29:31 PM11/10/11
to codership
Found the culprit in my.cnf:
log-bin=mysql-bin <--- why 2 different variables? oi.

On Nov 10, 11:15 am, seppo.jaak...@codership.com wrote:
> just don't put the log_bin and log_slave_updates in my.cnf, or comment
> it out...
>
> >> > stack_bottom = 0x7f69a84dad48 thread_stack 0x30000...
>
> read more »

EllisGL

unread,
Nov 10, 2011, 12:33:04 PM11/10/11
to codership
That worked - will test large sets of data.

seppo....@codership.com

unread,
Nov 10, 2011, 12:40:27 PM11/10/11
to codersh...@googlegroups.com
ok, you are hurt by: https://bugs.launchpad.net/codership-mysql/+bug/884615
To circumvent, you should not enable log-bin.

Fix will be in 1.1 release with ETA in late Nov, or as a hot fix.

log-bin=mysql-bin is actually good syntax, it enables binlogging and
sets the binlog file naming to follow pattern to: mysql-bin.000001

-seppo

EllisGL

unread,
Nov 10, 2011, 1:26:14 PM11/10/11
to codership
So trying to load a large set of data - 1.3GB worth, it died.

[root@db1-c mysql-data]# mysql < ~/dbdump.sql
ERROR 1213 (40001) at line 2237: Deadlock found when trying to get
lock; try restarting transaction

end of the log file (which isn't very long):
111110 11:56:08 [Note] WSREP: TO BEGIN: 718, 0 : DROP TABLE IF EXISTS
`swsessions`
111110 11:56:08 [Note] WSREP: TO BEGIN: 719, 2
111110 11:56:08 [Note] WSREP: TO END: 719, 2 : DROP TABLE IF EXISTS
`swsessions`
111110 11:56:08 [Note] WSREP: TO END: 719
111110 11:56:08 [Note] WSREP: TO BEGIN: 719, 0 : CREATE TABLE
`swsessions` (
`sessionid` varchar(32) NOT NULL DEFAULT '',
`ipaddress` varchar(30) NOT NULL DEFAULT '',
`lastactivity` int(10) unsigned NOT NULL DEFAULT '0',
`useragent` varchar(60) NOT NULL DEFAULT '0',
`islogg
111110 11:56:08 [Note] WSREP: TO BEGIN: 720, 2
111110 11:56:08 [Note] WSREP: TO END: 720, 2 : CREATE TABLE
`swsessions` (
`sessionid` varchar(32) NOT NULL DEFAULT '',
`ipaddress` varchar(30) NOT NULL DEFAULT '',
`lastactivity` int(10) unsigned NOT NULL DEFAULT '0',
`useragent` varchar(60) NOT NULL DEFAULT '0',
`islogged
111110 11:56:08 [Note] WSREP: TO END: 720
111110 11:56:08 [Note] WSREP: TO BEGIN: 720, 0 : /*!40000 ALTER TABLE
`swsessions` DISABLE KEYS */
111110 11:56:08 [Note] WSREP: TO BEGIN: 721, 2
111110 11:56:08 [Note] WSREP: TO END: 721, 2 : /*!40000 ALTER TABLE
`swsessions` DISABLE KEYS */
111110 11:56:08 [Note] WSREP: TO END: 721
111110 11:56:08 [Note] WSREP: empty rbr buffer





On Nov 10, 11:40 am, seppo.jaak...@codership.com wrote:
> ok, you are hurt by:https://bugs.launchpad.net/codership-mysql/+bug/884615
> To circumvent, you should not enable log-bin.
>
> Fix will be in 1.1 release with ETA in late Nov, or as a hot fix.
>
> log-bin=mysql-bin is actually good syntax, it enables binlogging and
> sets the binlog file naming to follow pattern to: mysql-bin.000001
>
> -seppo
>

SyRenity

unread,
Nov 10, 2011, 1:31:34 PM11/10/11
to codersh...@googlegroups.com
Hmm, I don't have these two variables at all in my my.cnf, so this should be something else.

Anything I can provide to catch this issue?

seppo....@codership.com

unread,
Nov 10, 2011, 1:40:14 PM11/10/11
to codersh...@googlegroups.com
SyRenity,

I have some questions to help focus your issue:

* How many concurrent updates like this do you run at the same time?
* Do your updates go to same node or to several nodes?
* Did row 302 exist when this update was executed?

-seppo

seppo....@codership.com

unread,
Nov 10, 2011, 1:54:10 PM11/10/11
to codersh...@googlegroups.com
I assume that the dbdump.sql loading just stopped at quite early stage?

Could you please locate in the dbdump.sql the line with:

/*!40000 ALTER TABLE `swsessions` DISABLE KEYS */

And print that line and a few (~10) following lines after that.
I expect there is an offending SQL statement this mysql version is not
able to handle, and I want to track it down.

-seppo

Alex Yurchenko

unread,
Nov 10, 2011, 2:30:36 PM11/10/11
to codersh...@googlegroups.com

What is its actual value in the running server?

SHOW VARIABLES LIKE 'log_bin'?

Alex Yurchenko

unread,
Nov 10, 2011, 2:36:54 PM11/10/11
to codersh...@googlegroups.com
- what's the MySQL-wsrep version, once again?
- was it a single INSERT?
- were there any LOCK TABLE statements and if yes, what is the value of
wsrep_convert_LOCK_to_trx?
- who died? Data load or mysqld?

--

SyRenity

unread,
Nov 10, 2011, 4:06:04 PM11/10/11
to codersh...@googlegroups.com
* How many concurrent updates like this do you run at the same time?
Only 1


* Do your updates go to same node or to several nodes?
Same node


* Did row 302 exist when this update was executed?
Yes, it was trying to update it.

SyRenity

unread,
Nov 10, 2011, 4:08:03 PM11/10/11
to codersh...@googlegroups.com
mysql> SHOW VARIABLES LIKE 'log_bin'
    -> ;
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| log_bin       | OFF   |
+---------------+-------+
1 row in set (0.00 sec)

EllisGL

unread,
Nov 10, 2011, 4:13:47 PM11/10/11
to codership
--
-- Dumping data for table `swsessions`
--

LOCK TABLES `swsessions` WRITE;
/*!40000 ALTER TABLE `swsessions` DISABLE KEYS */;
INSERT INTO `swsessions` VALUES
('47e2ea523478bc4d6292925c428b92b3','72.1.183.250',1320865606,'Mozilla/
5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.23)',
1,20,229,1320865533,0,'',0),
('301a7e9acf55576b81a66232b89d2152','66.249.72.45',1$
/*!40000 ALTER TABLE `swsessions` ENABLE KEYS */;
UNLOCK TABLES;

--
-- Table structure for table `swsettings`
--

DROP TABLE IF EXISTS `swsettings`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `swsettings` (
`settingid` int(10) unsigned NOT NULL AUTO_INCREMENT,
`section` varchar(50) NOT NULL DEFAULT '',
`vkey` varchar(50) NOT NULL DEFAULT '',
`data` mediumtext NOT NULL,
PRIMARY KEY (`settingid`),
KEY `sections` (`section`,`vkey`)
) ENGINE=InnoDB AUTO_INCREMENT=208 DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = @saved_cs_client */;

--
-- Dumping data for table `swsettings`
--

LOCK TABLES `swsettings` WRITE;
/*!40000 ALTER TABLE `swsettings` DISABLE KEYS */;
INSERT INTO `swsettings` VALUES (1,'core','version','3.50.06'),
(2,'core','product','eSupport'),(3,'cron','nextrun','1320867024'),
(4,'fileid','knowledgebase','4a81da1ac6402a611131443c295d9e72.js'),
(5,'fileid','jscachefile','322f9f6f917bc8$
/*!40000 ALTER TABLE `swsettings` ENABLE KEYS */;
UNLOCK TABLES;

--
-- Table structure for table `swsettingsfields`
--


On Nov 10, 12:54 pm, seppo.jaak...@codership.com wrote:
> I assume that the dbdump.sql loading just stopped at quite early stage?
>
> Could you please locate in the dbdump.sql the line with:
>
> /*!40000 ALTER TABLE `swsessions`  DISABLE KEYS */
>
> And print that line and a few (~10) following lines after that.
> I expect there is an offending SQL statement this mysql version is not
> able to handle, and I want to track it down.
>
> -seppo
>

EllisGL

unread,
Nov 10, 2011, 4:20:48 PM11/10/11
to codership
1. 5.5.15-wsrep_21.2-log
2. Yes.
3. Yes. Off.
4. Only the import.

How do I get mysql dump to not add locks?
I have: mysqldump --user=root --password=******** --single-transaction
--skip-lock-tables --lock-tables=false --databases db1 db2 db3 > ~/
dbdump.sql

On Nov 10, 1:36 pm, Alex Yurchenko <alexey.yurche...@codership.com>
wrote:

seppo....@codership.com

unread,
Nov 10, 2011, 4:40:36 PM11/10/11
to codersh...@googlegroups.com
Hi,

Seems I still need the definition of swsession table and the following
INSERT statement. So could you please paste all starting from:
CREATE TABLE 'swsession'...
until


/*!40000 ALTER TABLE `swsessions` ENABLE KEYS */

The INSERT value list may be long, if it fits nicely in
http://pastie.org/ , it would be good

Thanks, seppo

EllisGL

unread,
Nov 10, 2011, 4:51:08 PM11/10/11
to codership
http://pastie.org/2844081

On Nov 10, 3:40 pm, seppo.jaak...@codership.com wrote:
> Hi,
>
> Seems I still need the definition of swsession table and the following
> INSERT statement. So could you please paste all starting from:
>   CREATE TABLE 'swsession'...
> until
>   /*!40000 ALTER TABLE `swsessions` ENABLE KEYS */
>
> The INSERT value list may be long, if it fits nicely in  http://pastie.org/, it would be good
>
> Thanks, seppo

seppo....@codership.com

unread,
Nov 10, 2011, 4:59:11 PM11/10/11
to codersh...@googlegroups.com
I try to reproduce this update issue. Please run a select for this
device table and post results here:

mysql> SELECT * FROM devices WHERE id = 302;
mysql> SELECT COUNT(*) FROM devices;
mysql> CHECK TABLE devices;


Do you have problems only with this devices table, and all other
tables work ok?

-seppo

seppo....@codership.com

unread,
Nov 10, 2011, 5:03:36 PM11/10/11
to codersh...@googlegroups.com
thanks, inserts are here, but CREATE TABLE statement is still missing,
could you post it still here?

-seppo

SyRenity

unread,
Nov 10, 2011, 8:53:25 PM11/10/11
to codersh...@googlegroups.com
Hi.

Here are the results of running these queries:
http://pastie.org/private/ncl9isiz0pltin1rgxvaxg

I have problems with others tables as well - it seems as whole UPDATE is problematic.

Regards.

seppo....@codership.com

unread,
Nov 11, 2011, 3:26:26 AM11/11/11
to codersh...@googlegroups.com
I was able to load this row in my database, and updating this row also
works. However, I see two potential issues in the data:
1. type has value OBJECT, while allowed values are CAMERA, CONNECTOR,
this threw an warning
2. uuid has value NULL. uuid column has unique index and only one row
can have NULL value, you seem to have 33 rows in total.

You could do following exercise to troubleshoot further:
mysql> SET GLOBAL wsrep_retry_autocommit=0;
mysql> UPDATE `devices` SET `int_ip` = '', `last_contact` =

'2011-11-10 08:48:42', `ext_ip` = 'dev1.domain.local', `state` =

'ERROR', `data` = '' WHERE `id` = 302;
mysql> SHOW WARNINGS;

...and send the output of SHOW WARNINGS command
It may be that update results in duplicate key error in uuid column.

I could also analyze the full table content. For this you could do as follows;
mysql> SELECT * FROM devices INTO OUTFILE '/tmp/devices.sql';
... and send the /tmp/devices.sql file as an email attachment to me:
seppo....@codership.com

-seppo

EllisGL

unread,
Nov 11, 2011, 7:52:47 AM11/11/11
to codership
--
-- Table structure for table `swsessions`
--

DROP TABLE IF EXISTS `swsessions`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `swsessions` (
`sessionid` varchar(32) NOT NULL DEFAULT '',
`ipaddress` varchar(30) NOT NULL DEFAULT '',
`lastactivity` int(10) unsigned NOT NULL DEFAULT '0',
`useragent` varchar(60) NOT NULL DEFAULT '0',
`isloggedin` smallint(1) unsigned NOT NULL DEFAULT '0',
`sessiontype` int(10) unsigned NOT NULL DEFAULT '0',
`typeid` int(10) unsigned NOT NULL DEFAULT '0',
`dateline` int(10) unsigned NOT NULL DEFAULT '0',
`status` smallint(6) NOT NULL DEFAULT '0',
`captcha` varchar(10) NOT NULL DEFAULT '',
`iswinapp` smallint(1) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`sessionid`),
KEY `sessions1` (`sessiontype`,`lastactivity`,`status`),
KEY `typeid` (`typeid`)
) ENGINE=MEMORY DEFAULT CHARSET=utf8;
/*!40101 SET character_set_client = @saved_cs_client */;



On Nov 10, 4:03 pm, seppo.jaak...@codership.com wrote:
> thanks, inserts are here, but CREATE TABLE statement is still missing,
> could you post it still here?
>
> -seppo
>

seppo....@codership.com

unread,
Nov 11, 2011, 8:06:18 AM11/11/11
to codersh...@googlegroups.com
ok, here's the smoking gun:
) ENGINE=MEMORY

We currently support only InnoDB engine. It is possible to support
MEMORY engine in master/slave topology, and it may have worked in 5.1
based releases, but it is not a high priority task currently.

Is it possible to convert swsessions to InnoDB?

-seppo

EllisGL

unread,
Nov 11, 2011, 8:56:47 AM11/11/11
to codership
Yeah - that can totally be changed over. Thanks.

On Nov 11, 7:06 am, seppo.jaak...@codership.com wrote:
> ok, here's the smoking gun:
> ) ENGINE=MEMORY
>
> We currently support only InnoDB engine. It is possible to support
> MEMORY engine in master/slave topology, and it may have worked in 5.1
> based releases, but it is not a high priority task currently.
>
> Is it possible to convert swsessions to InnoDB?
>
> -seppo
>

SyRenity

unread,
Nov 12, 2011, 6:10:12 AM11/12/11
to codersh...@googlegroups.com
Hi.

Here is the result of the command:

mysql> SHOW WARNINGS;
+-------+------+--------------------------------------------------------------------+
| Level | Code | Message                                                            |
+-------+------+--------------------------------------------------------------------+
| Error | 1213 | Deadlock found when trying to get lock; try restarting transaction |
+-------+------+--------------------------------------------------------------------+

1 row in set (0.00 sec)

Google has sliced your email, perhaps I should send to info АТ codership.com?

Regards.

seppo....@codership.com

unread,
Nov 12, 2011, 6:17:27 AM11/12/11
to codersh...@googlegroups.com

Quoting SyRenity <stas....@gmail.com>:

> Hi.
>
> Here is the result of the command:
>
> mysql> SHOW WARNINGS;
> +-------+------+--------------------------------------------------------------------+
> | Level | Code |
> Message |
> +-------+------+--------------------------------------------------------------------+
> | Error | 1213 | Deadlock found when trying to get lock; try restarting
> transaction |
> +-------+------+--------------------------------------------------------------------+
> 1 row in set (0.00 sec)

Thanks, however, I'm surprised to see this result. It is better I try
to reproduce with your data.

>
> Google has sliced your email, perhaps I should send to info АТ
> codership.com?
>
> Regards.

Yes, you can send to info АТ codership.com?

-seppo

SyRenity

unread,
Nov 12, 2011, 6:39:58 AM11/12/11
to codersh...@googlegroups.com
Sent via Mutt, please let me know if you got it.

Regards.

SyRenity

unread,
Nov 12, 2011, 8:46:53 AM11/12/11
to codersh...@googlegroups.com
Resent to you directly following your email.

seppo....@codership.com

unread,
Nov 12, 2011, 11:01:34 AM11/12/11
to codersh...@googlegroups.com

Quoting SyRenity <stas....@gmail.com>:

> Resent to you directly following your email.
>

Testing with the data shows that the update is not effective: all
values passed in the update statement are already present in the table
row. And it turns out that mysql wsrep patch has a problem in dealing
with this. MySQL gives back deadlock error, correct behavior would be
to return OK and show in statistics that no rows were changed.

I created a bug report for this issue:
https://bugs.launchpad.net/codership-mysql/+bug/889523 . Fix will be
available today, and will be released in next minor release 1.1, with
ETA at the end of Nov.

Strangely enough this bug is caused by same source code line, which
caused problems for EllisGL's problem with MEMORY table. (I think we
will support node local MEMORY tables in the 1.1 release as well).


-seppo

SyRenity

unread,
Nov 13, 2011, 11:59:24 AM11/13/11
to codersh...@googlegroups.com
Hi.

Is this possible to push this update earlier?

We plan to upgrade some production systems, but unable to do so because of this issue.

Also, does it exists only in 5.5? We can upgrade for now 5.1 to latest V1, and upgrade to 5.5 once this fixed.r

Regards.

seppo....@codership.com

unread,
Nov 14, 2011, 9:24:40 AM11/14/11
to codersh...@googlegroups.com
I have not tested this issue with 5.1 branch yet. I expect it does not
happen there, just by looking at the source code.

5.5 branch fix has been pushed in already. However, we don't do hot
fix builds for community. But luckily, we are very close to the next
minor release milestone. The project is just about approaching code
freeze for 1.1 release. The Q/A cycle takes usually 1-2 weeks after
that.


-seppo

P.S. Oli Sennhauser just blogged about his experiments with Galera
source builds:
http://www.fromdual.com/building-galera-replication-from-scratch
Source build works well in testing/evaluating, but putting source
build in production is somewhat risky. Better at least synchronize
with us to hear Q/A status, if you plan to use a source build.


Quoting SyRenity <stas....@gmail.com>:

> Hi

SyRenity

unread,
Nov 16, 2011, 2:53:37 AM11/16/11
to codersh...@googlegroups.com
Thanks for the info, we will hold upgrading and wait to test the 5.5 release.

Regards.
Reply all
Reply to author
Forward
0 new messages