Potential issue with Percona authentication packets

[Phil Bayfield]

Hi all,

I'm trying to find out some more information about Percona's
authentication support.

I wrote/maintain a popular MySQL library for the Go programming
language (see: https://github.com/Philio/GoMySQL) and have received
reports that Percona doesn't work with the library. The current
library only support 4.1+ style packets/authentication and I've
verified that authentication fails with the latest version of Percona.

I've been testing with the current dev branch which contains both 4.1
and pre-4.1 packet support, and have observed the following behaviour
with Percona:

Percona's handshake packet indicates that it supports 4.1 protocol and
4.1 authentication.

I attempt a 4.1 authentication call:

Server responds with a 1 byte packet, the value of this byte is 0xfe.
This is an invalid response, it should return either an ok or error
packet.

I attempt a 4.0 authentication call:

Server responds with either a valid ok or error packet, in the 4.0
format.

This is the behaviour with MySQL 5/5.1/5.5 and MariaDB:

4.1: Ok or error packet in 4.1 format
4.0: Error packet, unsupported client version, upgrade client.

It appears that one of these 3 scenarios is a possibility:

1. Percona does not support the 4.1 protocol and incorrectly reports
that it does. This seems like odd behaviour to exhibit, considering
it's based on the 5.1 branch of MySQL.

2. There is a bug with the 4.1 authentication in the latest release of
Percona.

3. Percona accepts a slightly different packet format than other
versions of MySQL.

Once we release the current dev branch we can advice users to force
use of the 4.0 protocol, which so far Percona seems to work fine with,
however the drawback is that the old password hashing method is
insecure and can be used in conjunction with a packet sniffer to gain
access to the server.

I've spoken to Percona support but they were not really able to
provide much info on issue, other than to confirm that the 4.1
protocol was supported.

I'm hopeful someone on the list may be able to provide further
information.

Thanks,
Phil.

[Alexey Kopytov]

Hi Phil,

On 31.01.11 18:22, Phil Bayfield wrote:
> Hi all,
>
> I'm trying to find out some more information about Percona's
> authentication support.
>
> I wrote/maintain a popular MySQL library for the Go programming
> language (see: https://github.com/Philio/GoMySQL) and have received
> reports that Percona doesn't work with the library. The current
> library only support 4.1+ style packets/authentication and I've
> verified that authentication fails with the latest version of Percona.
>
> I've been testing with the current dev branch which contains both 4.1
> and pre-4.1 packet support, and have observed the following behaviour
> with Percona:
>
> Percona's handshake packet indicates that it supports 4.1 protocol and
> 4.1 authentication.
>
> I attempt a 4.1 authentication call:
>
> Server responds with a 1 byte packet, the value of this byte is 0xfe.
> This is an invalid response, it should return either an ok or error
> packet.
>

There should be no differences in the client/server protocol in Percona
Server as compared to MySQL or MariaDB. There is a possibility that some
changes might be introduced unintentionally, but we need more
information to investigate it further.

Can you send a dump of the 4.1 authentication session with Percona Server?

Best regards,
Alexey.

[Phil Bayfield]

I've modified my dev branch to output the packet contents which are below, also I've broken down the packets into their constituent parts.

I'm testing with the Ubuntu 10.10 5.1.49 x64 server on my local computer and the latest Percona server remotely on a Rackspace Cloud node running Ubuntu 10.04. I also test against MySQL 5.5 and MariaDb 5.1 but not really important for this example as except for some handshake packet changes to 5.5 (extra undocumented bytes on the end of the packet) they're the same as MySQL 5.1.

I'm using the same database on both servers called gomysql_test and with user root and the password.

Here is handshake with MySQL 5.1:

2011/02/02 11:47:38 === Begin connect ===

2011/02/02 11:47:38 Connecting to server via tcp to localhost:3306

2011/02/02 11:47:38 Connected to server

2011/02/02 11:47:38 Reading handshake initialization packet from server

[]byte{0x3f, 0x0, 0x0, 0x0, 0xa, 0x35, 0x2e, 0x31, 0x2e, 0x34, 0x39, 0x2d, 0x31, 0x75, 0x62, 0x75, 0x6e, 0x74, 0x75, 0x38, 0x2e, 0x31, 0x0, 0x9c, 0x0, 0x0, 0x0, 0x4c, 0x5d, 0x6d, 0x2e, 0x3b, 0x7b, 0x7a, 0x75, 0x0, 0xff, 0xf7, 0x8, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x5b, 0x46, 0x30, 0x58, 0x4d, 0x5a, 0x5c, 0x41, 0x6c, 0x7b, 0x49, 0x3a, 0x0}

2011/02/02 11:47:38 [0] Received handshake initialization packet

2011/02/02 11:47:38 Sending authentication packet to server

[]byte{0x47, 0x0, 0x0, 0x1, 0xd, 0xa2, 0x3, 0x0, 0xff, 0xff, 0xff, 0x0, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x72, 0x6f, 0x6f, 0x74, 0x0, 0x14, 0xd, 0x99, 0xdb, 0x54, 0xd9, 0x2a, 0x9f, 0xe2, 0x6e, 0x6c, 0x0, 0x8f, 0xbc, 0xa5, 0xd5, 0x5a, 0x72, 0xc5, 0xef, 0x0, 0x67, 0x6f, 0x6d, 0x79, 0x73, 0x71, 0x6c, 0x5f, 0x74, 0x65, 0x73, 0x74, 0x0}

2011/02/02 11:47:38 [1] Sent authentication packet

2011/02/02 11:47:38 Reading result packet from server

[]byte{0x7, 0x0, 0x0, 0x2, 0x0, 0x0, 0x0, 0x2, 0x0, 0x0, 0x0}

2011/02/02 11:47:38 [2] Received OK packet

Here is a 4.1 handshake with Percona:

2011/02/02 11:37:29 === Begin connect ===

2011/02/02 11:37:29 Connecting to server via tcp to 212.64.151.221:3306

2011/02/02 11:37:29 Connected to server

2011/02/02 11:37:29 Reading handshake initialization packet from server

[]byte{0x3c, 0x0, 0x0, 0x0, 0xa, 0x35, 0x2e, 0x31, 0x2e, 0x35, 0x34, 0x2d, 0x72, 0x65, 0x6c, 0x31, 0x32, 0x2e, 0x35, 0x0, 0x23, 0x0, 0x0, 0x0, 0x6e, 0x62, 0x25, 0x2b, 0x48, 0x68, 0x53, 0x47, 0x0, 0xff, 0xf7, 0x8, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x49, 0x77, 0x3f, 0x70, 0x38, 0x66, 0x77, 0x43, 0x2b, 0x4a, 0x73, 0x28, 0x0}

2011/02/02 11:37:29 [0] Received handshake initialization packet

2011/02/02 11:37:29 Sending authentication packet to server

[]byte{0x47, 0x0, 0x0, 0x1, 0xd, 0xa2, 0x3, 0x0, 0xff, 0xff, 0xff, 0x0, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x72, 0x6f, 0x6f, 0x74, 0x0, 0x14, 0xdb, 0x47, 0xe2, 0xb7, 0x92, 0x95, 0xbc, 0x8d, 0x2b, 0x55, 0x5e, 0x92, 0xfa, 0x67, 0xc3, 0x6a, 0xad, 0x19, 0x38, 0x5e, 0x67, 0x6f, 0x6d, 0x79, 0x73, 0x71, 0x6c, 0x5f, 0x74, 0x65, 0x73, 0x74, 0x0}

2011/02/02 11:37:29 [1] Sent authentication packet

2011/02/02 11:37:29 Reading result packet from server

[]byte{0x1, 0x0, 0x0, 0x2, 0xfe}

Connect err: "Unknown or unexpected packet or packet type"

If we break the packets down, starting with MySQL 5.1:

Handshake from server to client:

0x3f, 0x0, 0x0 Packet length

0x0 Sequence number

0xa Protocol version

0x35, 0x2e, 0x31, 0x2e, 0x34, 0x39, 0x2d, 0x31, 0x75, 0x62, 0x75, 0x6e, 0x74, 0x75, 0x38, 0x2e, 0x31, 0x0 Server version "5.1.49-1ubuntu8.1"

0x9c, 0x0, 0x0, 0x0 Thread id

0x4c, 0x5d, 0x6d, 0x2e, 0x3b, 0x7b, 0x7a, 0x75 Scramble buffer

0x0 Filler

0xff, 0xf7 Server capabilities

0x8 Server language

0x2, 0x0 Server status

0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0 Filler

0x5b, 0x46, 0x30, 0x58, 0x4d, 0x5a, 0x5c, 0x41, 0x6c, 0x7b, 0x49, 0x3a, 0x0 Rest of scramble buffer

Auth from client to server:

0x47, 0x0, 0x0 Packet length

0x1 Sequence number

0xd, 0xa2, 0x3, 0x0 Client flags

0xff, 0xff, 0xff, 0x0 Max packet size

0x8 Character set

0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0 Filler

0x72, 0x6f, 0x6f, 0x74, 0x0 Username

0x14, 0xd, 0x99, 0xdb, 0x54, 0xd9, 0x2a, 0x9f, 0xe2, 0x6e, 0x6c, 0x0, 0x8f, 0xbc, 0xa5, 0xd5, 0x5a, 0x72, 0xc5, 0xef, 0x0 Password scrambled

0x67, 0x6f, 0x6d, 0x79, 0x73, 0x71, 0x6c, 0x5f, 0x74, 0x65, 0x73, 0x74, 0x0 Database name

Response:

0x7, 0x0, 0x0 Packet length

0x2 Sequence number

0x0 Field count

0x0 Affected rows

0x0 Insert id

0x2, 0x0 Server status

0x0 Warning count

0x0 Message

Here is the same breakdown of the Percona packets:

Handshake from server to client:

0x3c, 0x0, 0x0 Packet length

0x0 Sequence number

0xa Protocol version

0x35, 0x2e, 0x31, 0x2e, 0x35, 0x34, 0x2d, 0x72, 0x65, 0x6c, 0x31, 0x32, 0x2e, 0x35, 0x0 Server version "5.1.54-rel12.5"

0x23, 0x0, 0x0, 0x0 Thread id

0x6e, 0x62, 0x25, 0x2b, 0x48, 0x68, 0x53, 0x47 Scramble buffer

0x0 Filler

0xff, 0xf7 Server capabilities

0x8 Server language

0x2, 0x0 Server status

0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0 Filler

0x49, 0x77, 0x3f, 0x70, 0x38, 0x66, 0x77, 0x43, 0x2b, 0x4a, 0x73, 0x28, 0x0 Rest of scramble buffer

Auth from client to server:

0x47, 0x0, 0x0 Packet length

0x1 Sequence number

0xd, 0xa2, 0x3, 0x0 Client flags

0xff, 0xff, 0xff, 0x0 Max packet size

0x8 Character set

0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0 Filler

0x72, 0x6f, 0x6f, 0x74, 0x0 Username

0x14, 0xdb, 0x47, 0xe2, 0xb7, 0x92, 0x95, 0xbc, 0x8d, 0x2b, 0x55, 0x5e, 0x92, 0xfa, 0x67, 0xc3, 0x6a, 0xad, 0x19, 0x38, 0x5e Password scrambled

0x67, 0x6f, 0x6d, 0x79, 0x73, 0x71, 0x6c, 0x5f, 0x74, 0x65, 0x73, 0x74, 0x0 Database name

Response:

0x1, 0x0, 0x0 Packet length

0x2 Sequence number

0xfe Packet body

Besides the response and the different password scrambles, the packets formats are identical.

So, I fired up Wireshark and started capturing packets for the authentication process between the MySQL client on my machine, here is what happened:

1181 209.174863 46.38.161.86 192.168.0.5 MySQL Server Greeting proto=10 version=5.1.54-rel12.5

1183 209.262974 192.168.0.5 46.38.161.86 MySQL Login Request user=root

1185 209.296204 46.38.161.86 192.168.0.5 MySQL Response

1187 209.296607 192.168.0.5 46.38.161.86 MySQL Request Unknown (87)

1188 209.329171 46.38.161.86 192.168.0.5 MySQL Response OK

The 'Response' was the same as I received, a 1 byte long packet with 0xfe, however the client then sends the unknown response command which triggers and OK response.

The OK response was: 07 00 00 04 00 00 00 02 00 00 00

This is is in the correct format for the 4.1 protocol.

I then repeated this to a MySQL 5.1 server on my local network:

253 79.675149 192.168.0.12 192.168.0.5 MySQL Server Greeting proto=10 version=5.1.49-1ubuntu8.1

255 79.699164 192.168.0.5 192.168.0.12 MySQL Login Request user=root

257 79.707357 192.168.0.12 192.168.0.5 MySQL Response OK

So the 2 servers DO behave differently, but the client was able to recover the initial "invalid" response from Percona using a conveniently undocumented "unknown response" command which then seems to trigger normal operation. I ran a few queries and the packets all look to be in 4.1 format.

I guess I need to implement the "invalid response", will delve into the source and see if I can find anything on this. If anyone knows anything about this, please let me know!

Cheers,
Phil.

[Andrew Garner]

On Wed, Feb 2, 2011 at 8:06 AM, Phil Bayfield <ph...@bayfmail.com> wrote:

> Response:
> 0x1, 0x0, 0x0 Packet length
> 0x2 Sequence number
> 0xfe Packet body

This looks like the behavior you get when you have an old-style
password for a user in the mysql.user table. The server initially
sends a 4.1 scramble buffer but the password it has for the user you
requested has the old style crypt hash in the mysql.user table. An
EOF (0xfe) packet gets sent and the client falls back to the old
pre-4.1 password algorithm - a packet just containing the crypted/323
style password hash should be sent back. This will happen even if
the --old-passwords is not enabled but the mysql.user table has an old
style password in it.

I ran into this with some pure-python drivers that didn't handle this
case well a while back.

~Andrew

[Phil Bayfield]

You're right, just checked the mysql.user table and all the passwords are the old style.

Also in the config:

# For compatibility to other Debian packages that still use

# libmysqlclient10 and libmysqlclient12.

old_passwords   = 1

It looks like the packet is 9 bytes long containing the 8 byte 3.23 hash and a null.

So looks like there is probably no difference between the servers after all :)

Thanks Andrew!

--
You received this message because you are subscribed to the Google Groups "Percona Discussion" group.
To post to this group, send email to percona-d...@googlegroups.com.
To unsubscribe from this group, send email to percona-discuss...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/percona-discussion?hl=en.