"schema_integrity_check_failed" error after upgrade (RabbitMQ 3.8.16 to 3.11.7)

[Dan Mahoney]

Hello all, I have a RabbitMQ upgrade issue I was hoping to get some help on. I am getting "schema_integrity_check_failed" error after upgrade.I've just upgraded my computer running Erlang OTP 23 (11.1.4) and RabbitMQ 3.8.16 to Erlang OTP 25.2 (13.1.3) and RabbitMQ 3.11.7. However, after I've done the upgrade and I reinstalled the RabbitMQ Service I am having troubles getting the RabbtiMQ node status to appear healthy. I am getting "schema_integrity_check_failed" error in the error logs.The process I followed for upgrading Erlang and RabbitMQ running on my computer is as follows:

• Stop RabbitMQ Service (rabbitmq-service stop)
• Remove RabbitMQ Service (rabbitmq-service remove)
• Stop the epmd.exe process
• Uninstall Erlang OTP 23 (11.1.4)
• Install Erlang OTP 25.2 (13.1.3)
• Install RabbitMQ 3.11.7
• Install RabbitMQ Service (rabbitmq-service install)
• Start RabbitMQ Service (rabbitmq-service start)
• Synchronize Erlang Cookies
• Stop RabbitMQ Service (rabbitmq-service stop)
• Remove RabbitMQ Service (rabbitmq-service remove)
• Install RabbitMQ Service (rabbitmq-service install)
• Start RabbitMQ Service (rabbitmq-service start)
• Try checking RabbitMQ Node status (rabbitmqctl status)

And this is where I get stuck. I've also tried the "rabbitmqctl start_app" command which says it it starting a node, but the status command always returns the "schema_integrity_check_failed" error.The only thing that has worked to get past this is using the "rabbitmqctl force_reset" command but I don't want to do that because I don't want to erase messages that are still in queues.If anyone has ideas or suggestions to try, I would gladly take them! 

[Luke Bakken]

Hello!

You have made a bold (i.e. unsupported) upgrade! Please see the documentation: https://www.rabbitmq.com/upgrade.html#rabbitmq-version-upgradability

Thanks,

Luke

[Dan Mahoney]

I've also tried installing rabbitmq-server-3.9.29 on top of 3.8.16. And then installing rabbitmq-server-3.10.20 on top of 3.9.29. And then lastly installing rabbitmq-server-3.11.7. But I get the same result in the end.
I am not sure if I need to do extra steps before or after installing each of the versions in between 3.8 and 3.11.

[Luke Bakken]

Hello,

OK, that is the supported way to do an upgrade. All of the "rabbitmq-service" commands you run shouldn't be necessary, but they also shouldn't result in the "schema_integrity_check_failed" error at the end of the process.

What version of Windows are you using? I'll try to reproduce what you report using a Windows 11 Enterprise VM.

Thanks,

Luke

[Dan Mahoney]

Windows 10 Enterprise 22H2 

[Dan Mahoney]

To clarify, my install steps were on my computer that already has Erlang OTP 23 (11.1.4) and RabbitMQ 3.8.16 running on it:
- Stop RabbitMQ Service (rabbitmq-service stop)
- Remove RabbitMQ Service (rabbitmq-service remove)
- Stop the epmd.exe process
- Uninstall Erlang OTP 23 (11.1.4)
- Install Erlang OTP 25.2 (13.1.3)
- Install RabbitMQ 3.9.29
- Install RabbitMQ 3.10.20
- Install RabbitMQ 3.11.7
- Install RabbitMQ Service (rabbitmq-service install)
- Start RabbitMQ Service (rabbitmq-service start)
- Synchronize Erlang Cookies
- Stop RabbitMQ Service (rabbitmq-service stop)
- Remove RabbitMQ Service (rabbitmq-service remove)
- Install RabbitMQ Service (rabbitmq-service install)
- Start RabbitMQ Service (rabbitmq-service start)
- Try checking RabbitMQ Node status (rabbitmqctl status)