Re: [rabbitmq-users] error "Either previous upgrade is in progress or has failed" during startup 3.7.4/3.7.15

402 views

Skip to first unread message

Michael Klishin

unread,

Jul 31, 2019, 7:00:46 PM7/31/19

to rabbitmq-users

If this is a bug then it's one that hasn't been reported in 9 years [1].

It is much more likely than an actual upgrade was taking place. I cannot know what's going on in this system,

see RabbitMQ and OS/system service logs for clues.

1. https://github.com/rabbitmq/rabbitmq-server/commit/00d769801f00a60152a345a8e00c72a667b14db6

On Wed, Jul 3, 2019 at 8:00 PM Roman O <warri...@gmail.com> wrote:

Hi, Rabbitmq is starting within docker container run from official docker image 3.7.4 (tried latest 3.7.15 and got the same error)
Rabbitmq uses docker volume for data storage on some storage. The storage is empty.
Rabbitmq fails to start because schema_upgrade_lock exists in the data folder.
When this file is deleted, the issue is gone and rabbitmq starts successfully.
Why is schema_upgrade_lock file being created during startup with empty data folder? Is this a bug?

Full stacktrace:

2019-07-03 14:56:37.158 [info] <0.33.0> Application lager started on node 'rabbit@rabbit-im-pla-slave-1' 2019-07-03 14:56:42.490 [error] <0.5.0> Found lock file at /var/lib/rabbitmq/mnesia/rabbit@rabbit-im-pla-slave-1/schema_upgrade_lock. Either previous upgrade is in progress or has failed. Database backup path: /var/lib/rabbitmq/mnesia/rabbit@rabbit-im-pla-slave-1-upgrade-backup 2019-07-03 14:56:42.490 [error] <0.5.0> Error description: init:do_boot/3 init:start_em/1 rabbit:start_it/1 line 445 rabbit:'-boot/0-fun-0-'/0 line 296 rabbit_upgrade:maybe_upgrade_mnesia/0 line 148 rabbit_upgrade:ensure_backup_taken/0 line 106 throw:{error,previous_upgrade_failed} Log file(s) (may contain more information): <stdout> epic_rabbitmq.0.7sihe3i8gonc@im-pla-slave-1 | BOOT FAILED =========== epic_rabbitmq.0.7sihe3i8gonc@im-pla-slave-1 | Error description: init:do_boot/3 init:start_em/1 rabbit:start_it/1 line 445 rabbit:'-boot/0-fun-0-'/0 line 296 rabbit_upgrade:maybe_upgrade_mnesia/0 line 148 rabbit_upgrade:ensure_backup_taken/0 line 106 throw:{error,previous_upgrade_failed} Log file(s) (may contain more information): <stdout> epic_rabbitmq.0.7sihe3i8gonc@im-pla-slave-1 | {"init terminating in do_boot",{error,previous_upgrade_failed}} init terminating in do_boot ({error,previous_upgrade_failed}) epic_rabbitmq.0.7sihe3i8gonc@im-pla-slave-1 | Crash dump is being written to: /var/log/rabbitmq/erl_crash.dump...done epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | /paas-docker-entrypoint.sh: line 3: 6 Terminated sleep 18000 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | 2019-07-03 14:57:18.546 [info] <0.33.0> Application lager started on node 'rabbit@rabbit-im-pla-slave-1' epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | 2019-07-03 14:57:23.780 [error] <0.5.0> Found lock file at /var/lib/rabbitmq/mnesia/rabbit@rabbit-im-pla-slave-1/schema_upgrade_lock. epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | Either previous upgrade is in progress or has failed. epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | Database backup path: /var/lib/rabbitmq/mnesia/rabbit@rabbit-im-pla-slave-1-upgrade-backup epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | 2019-07-03 14:57:23.781 [error] <0.5.0> epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | Error description: epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | init:do_boot/3 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | init:start_em/1 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | rabbit:start_it/1 line 445 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | rabbit:'-boot/0-fun-0-'/0 line 296 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | rabbit_upgrade:maybe_upgrade_mnesia/0 line 148 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | rabbit_upgrade:ensure_backup_taken/0 line 106 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | throw:{error,previous_upgrade_failed} epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | Log file(s) (may contain more information): epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | <stdout> epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | BOOT FAILED epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | =========== epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | Error description: epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | init:do_boot/3 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | init:start_em/1 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | rabbit:start_it/1 line 445 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | rabbit:'-boot/0-fun-0-'/0 line 296 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | rabbit_upgrade:maybe_upgrade_mnesia/0 line 148 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | rabbit_upgrade:ensure_backup_taken/0 line 106 epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | throw:{error,previous_upgrade_failed} epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | Log file(s) (may contain more information): epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | <stdout> epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | {"init terminating in do_boot",{error,previous_upgrade_failed}} epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | init terminating in do_boot ({error,previous_upgrade_failed}) epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | epic_rabbitmq.0.nyr9tz2qon9q@im-pla-slave-1 | Crash dump is being written to: /var/log/rabbitmq/erl_crash.dump...done

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/aadc1e7a-4b77-4ccb-b0f4-e10b55b961d4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.