unsupported_platform preventing rabbitmq startup on ubuntu

[Andrew Kraut]

SASL Log: https://gist.github.com/opie4624/7ccd29f56881d3614bd8

# cat /etc/lsb-release

DISTRIB_ID=Ubuntu

DISTRIB_RELEASE=12.04

DISTRIB_CODENAME=precise

DISTRIB_DESCRIPTION="Ubuntu 12.04.5 LTS"

# uname -a

Linux rails-rabbitmq-68bc38 3.2.0-77-generic #114-Ubuntu SMP Tue Mar 10 17:26:03 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

# apt-cache show rabbitmq-server

Package: rabbitmq-server

Version: 3.5.0-1

Architecture: all

Maintainer: RabbitMQ Team <pack...@rabbitmq.com>

Installed-Size: 4805

Depends: erlang-nox (>= 1:13.b.3) | esl-erlang, adduser, logrotate

Homepage: http://www.rabbitmq.com/

Priority: extra

Section: net

Filename: pool/main/r/rabbitmq-server/rabbitmq-server_3.5.0-1_all.deb

Size: 4160134

SHA256: dc34fb5ca87091ed2d696cae71275288036e42e1c6e2b0ff145c279409a4e47d

SHA1: 8bbe74dd525a114c239916d980b065f91371443f

MD5sum: 69cab00b49bf49a2031762514d045735

Description: AMQP server written in Erlang

 RabbitMQ is an implementation of AMQP, the emerging standard for high

 performance enterprise messaging. The RabbitMQ server is a robust and

 scalable implementation of an AMQP broker.

[Andrew Kraut]

Perhaps related? The init scripts on Ubuntu don't seem to be creating a PID file properly. In most cases, no PID file is created, which means the init script can't stop the service when it does start.

[Michael Klishin]

On 17 March 2015 at 00:42:32, Andrew Kraut (akr...@gmail.com) wrote:
> In most cases, no PID file is created

How can we reproduce this? What are the cases when the PID *is* created?

I've just installed rabbitmq-server from our repo, checked the pid at /var/run/rabbitmq/pid, stopped it with

sudo service rabbitmq-server stop

and started it again with

sudo service rabbitmq-server start

and the PID file is still in place.

Please be more specific than "most of the time". 
--
MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

On 16 March 2015 at 23:18:54, Andrew Kraut (akr...@gmail.com) wrote:
> SASL Log: https://gist.github.com/opie4624/7ccd29f56881d3614bd8

This is disk monitor that cannot start, not RabbitMQ itself.

I cannot reproduce this issue on

cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.2 LTS"

Disk monitor runs

/bin/df -kP /var/lib/rabbitmq

(assuming you did not instruct it to use custom directories via RABBITMQ_* env variables)

— what does that output on your system? 

[Andrew Kraut]

I don't have a circumstance where it starts successfully *according to init* however the process seems to actually start properly.

https://gist.github.com/opie4624/68338883926c50961b58

sasl log is empty and startup_err are empty. The other log files are included.

[Michael Klishin]

 On 17 March 2015 at 09:52:19, Michael Klishin (mkli...@pivotal.io) wrote:
> On 17 March 2015 at 00:42:32, Andrew Kraut (akr...@gmail.com) wrote:
> In most cases, no PID file is created

So "in most cases" should be read as "always"?

[Andrew Kraut]

What appears to be happening is that the pidfile is created, but the rabbitmqctl wait command fails and so the piddir and it's contents is deleted, even though the actual rabbitmq service starts.

[Andrew Kraut]

rabbitmqctl wait <pidfile> is failing even though the process is running successfully: https://gist.github.com/opie4624/01f6ce945fa08aefa610

[Michael Klishin]

On 17 March 2015 at 23:48:53, Andrew Kraut (akr...@gmail.com) wrote:
> rabbitmqctl wait is failing even though the process

> is running successfully: https://gist.github.com/opie4624/01f6ce945fa08aefa610

Thank you. 

[Jean-Sébastien Pédron]

On 17.03.2015 23:48, Andrew Kraut wrote:
> rabbitmqctl wait <pidfile> is failing even though the process is running
> successfully: https://gist.github.com/opie4624/01f6ce945fa08aefa610

Hi!

When this problems occurs, could you please send us the output of the
following commands, as well as their respective exit status?

ps -p $(cat /var/run/rabbitmq/pid)

/bin/df -kP /var/lib/rabbitmq/mnesia/rabbit@rails-rabbitmq-68bc38

erl -noinput -noshell -eval 'io:format("~p~n",[os:type()]),init:stop().'

Thank you!

--
Jean-Sébastien Pédron
Pivotal / RabbitMQ

[Andrew Kraut]

Here you go: https://gist.github.com/opie4624/736f5a249ce013ea7750

(I made the requisite command changes for both long node names and the different host I'm running these on.)

It's worth noting, that when this occurs, the pid file is removed despite the service starting up properly.

[Jean-Sébastien Pédron]

On 25.03.2015 00:26, Andrew Kraut wrote:
> It's worth noting, that when this occurs, the pid file is removed
> despite the service starting up properly.

Oh sorry, I misunderstood one of your previous mail. The gits you posted
earlier [1] led me to think the pidfile was still there.

So "rabbitmqctl wait" is not the problem, it's a symptom. I will try to
reproduce it with Ubuntu 12.04.

[1] https://gist.github.com/opie4624/01f6ce945fa08aefa610

[Jean-Sébastien Pédron]

On 25.03.2015 00:26, Andrew Kraut wrote:

> It's worth noting, that when this occurs, the pid file is removed
> despite the service starting up properly.

Could you please add "set +x" to both rabbitmq-server and rabbitmqctl,
start the service and post the output?

[Andrew Kraut]

rabbitmqctl wait fails even though the service starts.

https://gist.github.com/opie4624/fd8ff7e7078887f8b37c

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/gQTyKNl-FOE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send an email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jean-Sébastien Pédron]

On 25.03.2015 22:31, Andrew Kraut wrote:
> rabbitmqctl wait fails even though the service starts.
>
> https://gist.github.com/opie4624/fd8ff7e7078887f8b37c

This all looks sane, beside the "Error: process_not_running"... This
message is logged when the ps(1) command I asked you to run fails.

Could you please post the output of the following commands:

rabbitmqctl eval 'os:getenv().'

rabbitmqctl eval 'os:cmd("which ps").'

rabbitmqctl eval 'os:cmd("ps -p XXX 2>&1").' # Replace XXX with
# beam.smp PID

[Andrew Kraut]

rabbitmqctl can't contact the rabbit node once it starts: https://gist.github.com/opie4624/284db600e443d1fbe254

I think this is the issue with wait, as well.  However, if I undo the changes introduced here, rabbitmqctl eval works, but wait is still broke: https://github.com/rabbitmq/rabbitmq-server/commit/ec851de0e9a9d70620aacef5ad59592fbea75c81

I created the pid file manually for this: https://gist.github.com/opie4624/8cb9bffa1fc44fd26f37

[Jean-Sébastien Pédron]

On 26.03.2015 22:48, Andrew Kraut wrote:
> rabbitmqctl can't contact the rabbit node once it
> starts: https://gist.github.com/opie4624/284db600e443d1fbe254
> I think this is the issue with wait, as well. However, if I undo the
> changes introduced here, rabbitmqctl eval works, but wait is still
> broke: https://github.com/rabbitmq/rabbitmq-server/commit/ec851de0e9a9d70620aacef5ad59592fbea75c81
>
> I created the pid file manually for this:
> https://gist.github.com/opie4624/8cb9bffa1fc44fd26f37

I can't reproduce any of your problems on an up-to-date Ubuntu 12.04.5
with RabbitMQ 3.5.0 installed from our deb package :-/

Could you please post the output of the following one-liner; you need to
replace <PID> with the actual PID of beam.smp:

erl -noinput -noshell -eval 'P=erlang:open_port({spawn, "ps -p
<PID>"},[exit_status,{line,16384},use_stdio,stderr_to_stdout]),[receive
M -> io:format("~p~n", [M]) after 1000 -> ok end || _ <-
lists:seq(1,10)], init:stop().'

Could you even modify /etc/init.d/rabbitmq-server to put the following
two lines:

cat $PID_FILE
erl -noinput -noshell -eval 'P=erlang:open_port({spawn, "ps -p '$(cat
$PID_FILE)'"},[exit_status,{line,16384},use_stdio,stderr_to_stdout]),[receive
M -> io:format("~p~n", [M]) after 1000 -> ok end || _ <-
lists:seq(1,10)], init:stop().'

in start_rabbitmq(), between the call to start-stop-daemon and "$CONTROL
wait $PID_FILE", start the service and post the output?