Error: Failed to create new router session for service 'Read-Write Service'.

468 views
Skip to first unread message

blues...@gmail.com

unread,
Jul 31, 2017, 3:07:00 AM7/31/17
to MaxScale
Hi,

every time we make a backup, disabling the slave server and backing up the slave, we get this error many times in the maxscale logs:

------------
2017-07-31 03:08:52 error : Failed to create new router session for service 'Read-Write Service'. See previous errors for more details.
2017-07-31 03:08:52 error : [readwritesplit] Couldn't find suitable Master from 2 candidates.
------------

As I know, setting "detect_stale_master=true", maxscale should redirect all queries to the master until the slave turns back on.

And saturday night, maxscale got completely hangup, had to do a 'kill -9' to stop it and restart it afterwards.

We have one master and one slave at the moment. Maxscale is version 2.1.3. MariaDB 10.1 is running on DB servers.

I attach my maxscale.cnf as a file.

Thanks for any hint to resolve this,
Thomas

maxscale.cnf.txt

Markus Mäkelä

unread,
Jul 31, 2017, 3:57:57 AM7/31/17
to maxs...@googlegroups.com

Hi,

This seems like something that would be better to have as a bug report. If possible, please create one on the MaxScale Jira: https://jira.mariadb.org/browse/MXS

  • Describe what you are doing when you disable the server and back up the slave
  • Upload the configuration and the log files with all sensitive data removed
  • If possible, add the output of maxadmin show servers before, during and after you backup the slave

The complete hangup sounds a bit worrisome so if you have the logs for that period, I'd be very interested in taking a look at them.

Markus

-- 
Markus Mäkelä, Software Engineer
MariaDB Corporation
t: +358 40 7740484 | Skype: markus.j.makela 

Thomas Plant

unread,
Jul 31, 2017, 4:34:18 AM7/31/17
to MaxScale
Hello Markus,

no found a bad log message, it seems maxscale crashes after loosing the slave:

Jul 31 03:04:16 vs42 maxscale[31207]: [mysqlmon] All slave servers under the current master server have been lost. Assigning Stale Master status to the old master server 'server1' (10.10.100.120:3306)
.
Jul 31 03:04:16 vs42 maxscale[31207]: Server changed state: server2[10.10.100.121:3306]: lost_slave. [Slave, Running] -> [Running]
Jul 31 03:04:16 vs42 maxscale: MaxScale 2.1.3 received fatal signal 11
Jul 31 03:04:16 vs42 maxscale: Writing core dump
Jul 31 03:04:16 vs42 systemd: maxscale.service: main process exited, code=killed, status=11/SEGV
Jul 31 03:04:16 vs42 systemd: Unit maxscale.service entered failed state.
Jul 31 03:04:16 vs42 systemd: maxscale.service failed.
Jul 31 03:04:16 vs42 systemd: maxscale.service holdoff time over, scheduling restart.
Jul 31 03:04:16 vs42 systemd: Starting MariaDB MaxScale Database Proxy...

I upgraded to 2.1.4 today as there where mentioned a crash situation in the changelog. So well will see tonight if it helped. If not, I will file a bug. Or should I anyway make the bug report?

Thomas Plant

unread,
Jul 31, 2017, 4:37:59 AM7/31/17
to MaxScale
Stupid me, you asked how we do the backup.....

on the slave I issue a 'mysqladmin stop-slave', then in a loop I backup every database one per one to a distinct backup file. Command for backup is 'mysqldump -u $user -p$pwd --opt -B $dbname | gzip > $backupdir/$db.sql.gz'.
After all backups have been done we do 'mysqladmin start-slave'. Thats all.

Thanks,
Thomas

Markus Mäkelä

unread,
Jul 31, 2017, 5:03:33 AM7/31/17
to maxs...@googlegroups.com

Hi,

I think you might be hitting this bug: https://jira.mariadb.org/browse/MXS-1323

This will be fixed in the 2.1.5 release and We'll be releasing it as soon as possible. Once the release is out, you could try if it fixes the issue. If you still see the problem with 2.1.5, please create a bug report.

Markus

--
You received this message because you are subscribed to the Google Groups "MaxScale" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maxscale+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Markus Mäkelä

unread,
Jul 31, 2017, 5:07:15 AM7/31/17
to maxs...@googlegroups.com

Hi,

Thanks for providing the details. I would say that this makes it very likely that you are hitting the bug MXS-1323. As I mentioned in the earlier email, we'll be releasing 2.1.5 shortly.

If you want to test the development version of the current 2.1 branch packages, you can find them here: http://max-tst-01.mariadb.com/ci-repository/maxscale-2.1.5-release/mariadb-maxscale/

Markus

--
You received this message because you are subscribed to the Google Groups "MaxScale" group.
To unsubscribe from this group and stop receiving emails from it, send an email to maxscale+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Thomas Plant

unread,
Jul 31, 2017, 5:23:36 AM7/31/17
to MaxScale
Thanks for the information. I will wait for the 2.1.5 release.

Greetings,
Thomas

Thomas Plant

unread,
Aug 1, 2017, 2:56:50 AM8/1/17
to MaxScale
Update to 2.1.5 solved the crash.  :-)
Reply all
Reply to author
Forward
0 new messages