AWS DMS fails to connect through ProxySQL v3 (v2 works ok)

[Dave Rix]

Hi team,

We have an issue with AWS DMS connecting to an Aurora instance via ProxySQL v3, but it works fine through ProxySQL v2. I'm not 100% sure if this is an issue with AWS DMS, or ProxySQL v3, so I'm raising a query against both.

My testing has shown the following success/failure matrix

• ✅ AWS DMS -> ProxySQL v2
• ✅ Binlog Replication -> ProxySQL v2
• ❌ AWS DMS -> ProxySQL v3
• ✅ Binlog Replication -> ProxySQL v3

The issue here appears to be how DMS connects via ProxySQL v3, and I was wondering if anything significant has changed in the way that ProxySQL handles `binlog` type replication connections between v2 and v3?

One thing I have noticed so far is that when DMS is connected and reading the binlogs, it has the schema set to 'NULL', but when I connect with `mysqlbinlog` to test replication, it has the schema set to 'information_schema'.

It also fails one of the pre-migration checks - "Validate if AWS DMS connected to MySQL primary instance", although it is definitely connecting to the primary, as I see the connection on the primary, and then it gets dropped.

I do not know if this is the issue, but it is the only thing I have seen at the moment.

If I find anything more, I will update here.

Regards,

Dave

[René Cannaò]

Hi Dave,

Thank you for the report.

Before further troubleshooting, can you please try the development branch (v3.0) ?

If that doesn't work, the best path is to create a github issue and investigate it properly, starting from error log and logging what queries are executed by DMS .

I am suggesting that you try the development branch because there are at least 4 PRs related to replication that will be available in 3.0.4: 5203 , 5240 , 5241 and 5245 .

Although they are all related to replication, some may not be relevant to DMS.

Thanks,

René

--
You received this message because you are subscribed to the Google Groups "proxysql" group.
To unsubscribe from this group and stop receiving emails from it, send an email to proxysql+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/proxysql/6ea08ebf-ffca-4611-a46e-7bcb88dae307n%40googlegroups.com.

[Dave Rix]

Thanks for the swift response René, I will try to do that either today or tomorrow.

I had a bit of trouble compiling last time, and I've terminated my EC2, but I'm sure I can work it through again :) 

I will feed back here once I've completed testing.

Regards,

Dave

[Dave Rix]

OK, so running with the latest branch also fails to allow DMS to connect to Aurora through ProxySQL.

The DMS solution has a pre-migration test, which shows the error "Validate if AWS DMS connected to MySQL primary instance." - but I can guarantee that the primary host is being connected to.

The error thrown by DMS when connecting is "Last Error  Query execution or fetch failure. Stop Reason RECOVERABLE_ERROR Error Level RECOVERABLE"

And it loops round a few times until it gives up.

I want to get to the bottom of the pre-migration test error, as I think resolving that will resolve more issues - although the DMS tasks work fine when going through ProxySQL v2, which is more confusing.