ProxySQL crash looping, how to debug ?

[Venkat]

Hello team, 

I am doing some experiments to find out what is the size increase in SQLite db file if I have 10,000 mysql users, servers and query rules. 

As soon as I enter 10,000 servers (dummy servers as I want to just know the db size)
in the following way 

```
for i in {1..10000}
do
echo "INSERT INTO mysql_servers (hostname,hostgroup_id,port,weight,max_connections) VALUES ('hostname$i',10,3306,1000,2000);";
done > servers.sql
echo "LOAD MYSQL SERVERS TO RUNTIME;" >> servers.sql
echo "SAVE MYSQL SERVERS TO DISK;" >> servers.sql
```
mysql -uadmin -padmin -h 127.0.0.1 -P6032 < servers.sql

server is crash looping and I do not find any useful information in proxy.log file. 

I am using : 

# proxysql --version

ProxySQL version 2.3.2-10-g8cd66cf, codename Truls

Can some one please help how to debug this issue ? 

Regards,

Venkatesh.

[René Cannaò]

Hi Venkat,

I just tested this and there is no crash: it is indeed a very basic operation.

Perhaps the crash is not related to the test you are doing: maybe in the error log there is useful information.

--
You received this message because you are subscribed to the Google Groups "proxysql" group.
To unsubscribe from this group and stop receiving emails from it, send an email to proxysql+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/proxysql/9457bb5b-77df-4785-95a4-ee06ae84dcd0n%40googlegroups.com.

[René Cannaò]

Hi Venkat,

In my previous test I tried the above with Monitoring disabled (because servers do not exist), but when I enabled monitoring I can confirm that proxysql crashes.

So you are hitting a bug.

A temporary workaround (temporary, this is a bug we will fix) is to increase variable mysql-monitor_ping_interval.

Here is an example:

$ sqlite3 proxysql.db
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
sqlite> SELECT * FROM global_variables WHERE variable_name='mysql-monitor_ping_interval';
mysql-monitor_ping_interval|10000
sqlite> UPDATE global_variables SET variable_value=300000 WHERE variable_name='mysql-monitor_ping_interval';
sqlite> .q

[René Cannaò]

I created a bug report for it:

https://github.com/sysown/proxysql/issues/3968

[Venkatesh Duggirala]

This is great, thanks Rene for the quick response and thanks for the workaround and bug creation.

Did you also observe there is no information in ProxySQL.log , no core stack, no indication of crash looping ? 
Is that common or do I need to enable some flags to get that info in error logs or do I need to look at some other file for that information ?

Regards,

Venkatesh.

[René Cannaò]

Hi Venkatesh,

According to the issue I created, proxysql received signal SIGFPE, Arithmetic exception.

This signal is indeed not handled and the error log wasn't reporting anything useful.

I will add this signal in the signals handler.