Unknown failure when importing 3.6 definitions into 3.7

[Greg]

Hey all.

I'm working on an upgrade from 3.6.1 to 3.7.16 (and Erlang R16B03 to Erlang 22). For no downtime, I'm going to bring up a new server and just cut over to it.

I did an export (rabbitmqadmin export file.json) from 3.6 and tried to import that file into 3.7. It returns "Received 102 Processing for path /api/definitions" to start, and then looking in the logs, the import does actually start. However, the last log line is "Creating user" for the first user, and then nothing else. The process seems to have died somewhere, but I can't find it.

I tried raising the log reporting up to debug but that doesn't seem to show any more info. The import file is 16k, so it's not like this is a huge file.

Any ideas?

[Luke Bakken]

Hi Greg,

You're running into this issue - https://github.com/rabbitmq/rabbitmq-management/issues/715

Use this file until 3.7.17 ships - https://raw.githubusercontent.com/rabbitmq/rabbitmq-management/v3.7.x/bin/rabbitmqadmin

Thanks and have a good weekend -

Luke

[Greg]

Hey,

So that gets me slightly closer, but it's still cutting out on importing the first user. It creates the user but doesn't set any permissions on it or anything. I'm not anything in the logs, either.

I tried setting /etc/rabbitmq/rabbitmq.conf to:

> log.file.level = debug

but that doesn't show anything else. Am I doing something else wrong?

- greg

[Greg]

I tried doing:

curl -F 'file=@q.json' -v -u USER:PASS http://127.0.0.1:15672/api/definitions

And I get this:

< HTTP/1.1 100 Continue
< HTTP/1.1 102 Processing
< x-rabbitmq-import-progress: Importing users...
< HTTP/1.1 102 Processing
< x-rabbitmq-import-progress: Importing vhosts...
< HTTP/1.1 102 Processing
< x-rabbitmq-import-progress: Importing user permissions...
< HTTP/1.1 400 Bad Request
< content-length: 67
< content-security-policy: default-src 'self'
< content-type: application/json
< date: Mon, 15 Jul 2019 17:38:01 GMT
< server: Cowboy
< vary: accept, accept-encoding, origin
* HTTP error before end of send, stop sending
<
* Closing connection 0
{"error":"bad_request","reason":"{no_such_user,<<\"FIRST_USER\">>}"}

I assume that's related?

[Michael Klishin]

Chances are you have a permission for a user not included into this definition file.

[Greg]

 The export JSON file starts with a users section, and the user it's trying to import here is the first one in the list. Based on what you're saying, is this some kind of race condition, where the imports are running in parallel?

- greg

[Greg]

Update: I tried doing an import into a Docker container running 3.7.15, and it worked just fine. So I think this is an issue with 3.6.16+ and/or Erlang 22.

Should I open a ticket around this?

- greg

[Luke Bakken]

Hi Greg,

It would be best to share the file or a similar one that reliably reproduces the issue. We can give it a try locally first before opening an issue.

[Greg]

Hi Luke,

Attached a file. Here are my steps for reproducing using Docker:

1. docker run -d --rm rabbitmq:3.7.16-management

2. In the container:

apt-get update && apt-get install -y wget
wget -O /usr/local/bin/rabbitmqadmin https://raw.githubusercontent.com/rabbitmq/rabbitmq-management/v3.7.x/bin/rabbitmqadmin && chmod 755 /usr/local/bin/rabbitmqadmin

3. Copy the attached file into the container

4. In the container, run: rabbitmqadmin import rmq-3.7.16-issue.json

5. Tail the logs from the container (docker logs -f) to see it get stuck on creating the first user.

Let me know what you think.

- greg

[Luke Bakken]

Hi Greg,

I really appreciate the simple reproduction steps. I can reproduce this issue using 3.7.16, and I can confirm that the issue is resolved using the latest v3.7.x code, which will become version 3.6.17.

This pull request removed a feature that we recently added, and it allows rabbitmqadmin to continue:

https://github.com/rabbitmq/rabbitmq-management/pull/718

At some point in the future we may re-add progress reporting but it'll have to be when Python's httplib / http.client supports it.

Thanks,

Luke