shutdown during IST

[Maarten van Baarsel]

Hi,

Is is possible that if you shutdown mysql while there is an incoming IST the
shutdown is delayed while the IST is running?


This is just a hunch I'm tracking down, from the logs:

May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: view((empty))
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: gcomm: closed
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: Flow-control interval: [64, 64]
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: Received NON-PRIMARY.
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: Shifting JOINER -> OPEN (TO: 4095760)
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: Received self-leave message.
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: Flow-control interval: [64, 64]
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: Received SELF-LEAVE. Closing connection.
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 4095760)
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: RECV thread exiting 0: Success
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: recv_thread() joined.
May 3 00:46:08 localhost mysqld: 130503 0:46:08 [Note] WSREP: Closing slave action queue.
May 3 02:19:11 localhost mysqld: 130503 2:19:11 [Note] WSREP: IST received: 92f5a26e-aeaf-11e2-0800-fcc64d722c65:4093185
May 3 02:19:11 localhost mysqld: 130503 2:19:11 [ERROR] WSREP: gcs/src/gcs.c:_join():800: Sending JOIN failed: -103 (Software caused connection abort).
May 3 02:19:11 localhost mysqld: 130503 2:19:11 [ERROR] WSREP: Failed to JOIN the cluster after SST
May 3 02:19:12 localhost mysqld: 130503 2:19:12 [Warning] WSREP: Failed to report last committed 4094167, -77 (File descriptor in bad state)
May 3 02:19:14 localhost mysqld: 130503 2:19:14 [Warning] WSREP: Failed to report last committed 4095153, -77 (File descriptor in bad state)


If yes, then that's pretty nasty side-effect, the server was down and unusable
for an hour and a half. I would have kill-9'd it if I hadn't gone to bed at
that time (and this is a testcluster for the moment ;).

(my gcache is set to 10G, perhaps I should make it smaller)

thanks,
M.

[Alex Yurchenko]

This looks like a bug in shutdown sequence. IST should be easily
interruptible.

--
Alexey Yurchenko,
Codership Oy, www.codership.com
Skype: alexey.yurchenko, Phone: +358-400-516-011

[Raghavendra D Prabhu]

Hi,

>--
>You received this message because you are subscribed to the Google Groups "Percona Discussion" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to percona-discuss...@googlegroups.com.
>To post to this group, send email to percona-d...@googlegroups.com.
>For more options, visit https://groups.google.com/groups/opt_out.
>
>

There is a bug open for this
https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1109969 , based
on what I observed while it is undergoing IST it doesn't handle the
signals, so is delayed. Now, it may be due to lack of signal handling in
that area or that mysqld waits for galera applier threads.


Regards,
--
Raghavendra Prabhu
Percona LLC. - http://www.percona.com / Blog: http://www.mysqlperformanceblog.com/
Skype: percona.raghavendrap
GPG: 0xD72BE977
Contact: http://wnohang.net/contact

[Alex Yurchenko]

Raghu, that is a different bug - but the bug nevertheless ;). And both
these bugs, as far as I can tell, belong to Galera.

Regards,
Alex

> Regards,
> --
> Raghavendra Prabhu
> Percona LLC. - http://www.percona.com / Blog:
> http://www.mysqlperformanceblog.com/
> Skype: percona.raghavendrap
> GPG: 0xD72BE977
> Contact: http://wnohang.net/contact

[Raghavendra D Prabhu]

Hi,

Ack, I will move it to Galera bug component.

>
>Regards,
>Alex
>
>>Regards,
>>--
>>Raghavendra Prabhu
>>Percona LLC. - http://www.percona.com / Blog:
>>http://www.mysqlperformanceblog.com/
>>Skype: percona.raghavendrap
>>GPG: 0xD72BE977
>>Contact: http://wnohang.net/contact
>
>--
>Alexey Yurchenko,
>Codership Oy, www.codership.com
>Skype: alexey.yurchenko, Phone: +358-400-516-011
>

>--
>You received this message because you are subscribed to the Google Groups "Percona Discussion" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to percona-discuss...@googlegroups.com.
>To post to this group, send email to percona-d...@googlegroups.com.
>For more options, visit https://groups.google.com/groups/opt_out.
>
>

Regards,
--
Raghavendra Prabhu
Percona LLC. - http://www.percona.com / Blog: http://www.mysqlperformanceblog.com/
Skype: percona.raghavendrap
GPG: 0xD72BE977
Contact: http://wnohang.net/contact

Make plans to attend Percona Live London MySQL Conference
2013.<http://www.percona.com/live/london-2013/>

[Alex Yurchenko]

Reported IST bug here: https://bugs.launchpad.net/galera/+bug/1176852