Rabbitmq and full (redhat...) systemd support?

[Joshua Harlow]

Hi all,

I was wondering if there are plans (or a known schedule) for fully turning on the usage of systemd and using service files for the main distributions that rabbitmq packages.

I see that debian recently got usage of that integrated but the fedora/rhel/centos in-tree-packages don't seem to be using that yet.

If there was ongoing work in that area that's cool, if not then I might try to get that going due to some random(not that frequent) issues we've had with the sysvinit style packages not correctly handling restart and start/stop (and this results in duplicate rabbitmq servers running at the same time) and I'm pretty sure that by actually using 'pure' service files and systemd interactions with those that said problems will go away.

I do believe that redhat was/is patching this in via the following (which I'd like to not have to patch in myself and get into upstream):

* https://gist.github.com/harlowja/47bb9bcbac51632859f8c40e30f2ccd8 (a systemd notify patch, probably upstreamed?)

* https://gist.github.com/harlowja/3bf57240e2f6cb1d274baaac72650e9d (a service file that redhat appears to be shipping, probably can just use this?)

Also seems relevant from the spec file changelog, not sure if these were raised issues, but thought they seemed relevant,

%changelog
* Thu Feb 18 2016 Peter Lemenkov <....@gmail.com> - 3.3.5-17
- Log something when the app stops, so that we can tell when stop_app etc has
  succeeded.

Full rpm spec @ https://gist.github.com/harlowja/b07a982e4f6773fba5b7f1ffd9aa3e4e

Note that I'd like to make sure that 3.7 has the above, in all reality I don't want any custom patches that redhat maintains (because, meh, why) but that's probably a longer discussion...

So would there be an objections to moving the in-tree spec file and such to have systemd native support, perhaps similar to the redhat one (which has dual sysvinit and systemd support)?

Thoughts,

Josh

[Peter Lemenkov]

2016-07-28 22:39 GMT+02:00 Joshua Harlow <harl...@gmail.com>:
> Hi all,
>
> I was wondering if there are plans (or a known schedule) for fully turning
> on the usage of systemd and using service files for the main distributions
> that rabbitmq packages.
>
> I see that debian recently got usage of that integrated but the
> fedora/rhel/centos in-tree-packages don't seem to be using that yet.

You're surely mistaken. Fedora / CentOS offrers systemd integration
for a while already. In fact fer Fedora releases ago there was a
requirement that all system services must be switched to systemd.

CentOS is a slightly different story. We try to preserve user
experience within release (which means up to 12 years). So SysVinit
support if added will stay until RHEL major release will be EOLed
(normally until Phase 2 expired):

https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#Product_life_cycle

However RabbitMQ for EPEL and RabbitMQ for Red Hat OpenStack layered
product has systemd support as well.


--
With best regards, Peter Lemenkov.

[Michael Klishin]

What do you find missing in the systemd support added in 3.6.3?

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Joshua Harlow]

So am I correct to say that the, the EPEL rabbit and RabbitMQ for Red Hat OpenStack patched in the (needed) systemd support and enabled using .service files in the rpms they are building?

The main point I'm thinking is that people use the rpms from pivotal themselves (at least some people do) and those rpms don't exactly seem to have .service files and/or use it?

https://github.com/rabbitmq/rabbitmq-server/tree/master/packaging/RPMS/Fedora

So if people use https://github.com/rabbitmq/rabbitmq-server/tree/master/packaging/RPMS/Fedora and/or the webpage @ https://www.rabbitmq.com/install-rpm.html

Then they get some kind of other variant of a package that does not have those patches (and therefore systemd support in those rpms) which means that there is a difference between the provided pivotal rpms and the redhat ones (ie, things are being included downstream that are not upstream). I'd like to resolve that situation if it is confirmed to exist so that people can actually use the pivotal rpms (maybe even the redhat folks can use them as well) instead of owning <there own rpms> that as stated on the rabbitmq page are:

Overview

rabbitmq-server is included in Fedora. However, the versions included are often quite old. You will probably get better results installing the .rpm from our website. Check the Fedora package details for which version of the server is available for which versions of the distribution.

[Joshua Harlow]

So this is where I'm not an expert on systemd but it concerns me that looking at the source rpm for what ships in epel7 I see various patches (@ https://gist.github.com/harlowja/2d048ae971c9c9a618aae0ae4d4c9d8a ) that make systemd work better (or seem to?) but the native rpms and such from pivotal seem to not include an equivalent, thus making those that use the pivotal provided rpms a type of second class citizen (and confusing to end users that believe they are getting packages that work just as good as other ones).

[Michael Klishin]

Mirantis contributed systemd support for Debian/Ubuntu not so long ago:

https://github.com/rabbitmq/rabbitmq-server/pull/809/files

if someone wants to do the same for Fedora/RPM, we'd be happy to consider it.

--

You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Joshua Harlow]

Ya, what I'd be contributing would probably just come or be similar to the redhat stuff,

So I'd almost (in a perfect world) just have the redhat folks contribute back there stuff, instead of me proxying it for them....

-Josh

[Michael Klishin]

Perhaps Peter would be interested in doing it? ;) 

> >> as well) instead of owning that as stated on the rabbitmq

> To post to this group, send an email to rabbitm...@googlegroups.com.

[Alexey Lebedeff]

Hi Joshua,

That list contains at least three patches that are already in `stable`
branch of rabbitmq. And about several of others I'm just not 100%
sure about.

Here is the systemd story:
- CentOS systemd patch uses separate C binding for `sd_notify` to signal
systemd about successful startup. And this C binding is the reason why
arch independent packages can't be provided.
- Systemd contains `systemd-notify` binary, but it is broken for
non-root users - https://github.com/systemd/systemd/issues/2739.
Some fix is proposed at https://github.com/zonque/systemd/commits/notify
but I'm not tracking it's progress.
- Now RabbitMQ contains systemd integration code that depends only on
external executable `socat`. That way you can produce arch independent
packages, given that your OS provides packaged `socat`- which is true
almost everywhere.

Also you should note that both redhat and debian/ubuntu packages of
erlang/rabbitmq are lagging behind the versions that contain great
stability improvements.

So you have no other way than to package the latest stable releases of
erlang/rabbit yourself (as we do in Mirantis). Unless you are ready to
consume packages from repositories similar to Debian unstable =)

Best,
Alexey

>>> email to rabbitmq-user...@googlegroups.com <javascript:>.

>>> To post to this group, send email to rabbitm...@googlegroups.com

>>> <javascript:>.

[Alexey Lebedeff]

Hi Joshua,

Almost everything is already present in RabitMQ sources - systemd
integration code and example of systemd unit.

The only missing part is a way to produce .rpm package where systemd
support is optional.

I took a stab at it, but quickly got an impression that you can't do it
- all systemd stuff is guarded with `#ifdef`-s in rpm's .spec. So unless
somebody more versed in rpm will come up with a way to produce such
packages, you are on your own for packaging.

>>> <https://admin.fedoraproject.org/updates/rabbitmq-server> details for

>>> email to rabbitmq-user...@googlegroups.com <javascript:>.

>>> To post to this group, send email to rabbitm...@googlegroups.com

>>> <javascript:>.

[Joshua Harlow]

I've seen the following in the redhat spec file,

Wouldn't it work (good enough?),

%if 0%{?el5}%{?el6}
/sbin/chkconfig --add %{name}
%else
%systemd_post %{name}.service
/bin/systemctl daemon-reload
%endif

%preun
# We do not remove /var/log and /var/lib directories
# Leave rabbitmq user and group
%if 0%{?el5}%{?el6}
if [ $1 = 0 ]; then
  #Complete uninstall
  /sbin/service %{name} stop > /dev/null 2>&1
  /sbin/chkconfig --del %{name}
fi
%else
%systemd_preun %{name}.service
%endif

That seems to be the 'dual-mode' ability (or equivalent).

[Joshua Harlow]

So getting this started and am running into `make` issues, perhaps something simple I am missing,

Wondering if anyone has seen this (or knows the workaround/missing piece?),

This was seen on the master of rabbitmq-server (on centos7 with erlang-18.3.4),

https://gist.github.com/harlowja/6c081e91b56e96a7b7711ef2b5e88be8

Primarily:

src/rabbit_ct_broker_helpers.erl:19: can't find include lib "common_test/include/ct.hrl"

src/rabbit_ct_broker_helpers.erl:121: undefined macro 'config/2'

...

Any ideas? Would be nice to be able to use that command and try out what my changes to the packaging rpm spec file did :)

-Josh

[Joshua Harlow]

Think I got this figured out, went and got the esl-erlang (since I guess the below is a test package)!

-Josh

[Joshua Harlow]

Ok, got the following up, seems to work on cent7,

https://github.com/rabbitmq/rabbitmq-server/pull/912/

Comments appreciated & welcome (especially if things can be made better),

-Josh

[Michael Klishin]

Thanks. It would be great if more people could test this on CentOS 6 and relevant RHEL releases. 

> >>>> >>> as well) instead of owning that as stated on the

> >>>> rabbitmq
> >>>> >>> page are:
> >>>> >>>
> >>>> >>> Overview
> >>>> >>>
> >>>> >>> rabbitmq-server is included in Fedora. However, the versions
> >>>> included
> >>>> >>> are often quite old. You will probably get better results
> >>>> installing the
> >>>> >>> .rpm from our website. Check the Fedora package

> >>>> >>> email to rabbitmq-user...@googlegroups.com .

> >>>> >>> To post to this group, send email to rabbitm...@googlegroups.com

> >>>> >>> .
> >>>> >>>
> >>>> >>> For more options, visit https://groups.google.com/d/optout.
> >>>> >>>
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> --
> >>>> >> MK
> >>>> >>
> >>>> >> Staff Software Engineer, Pivotal/RabbitMQ
> >>>> >>
> >>>>
> >>>>
>

> --
> You received this message because you are subscribed to the Google Groups "rabbitmq-users"
> group.

> To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.
> To post to this group, send an email to rabbitm...@googlegroups.com.

[Giri Chintala]

MK,

i am testing it in RHEL7. RabbitMQ failed start after reboot. i see "su: cannot open session: Permission denied" in /var/log/rabbitmq/startup_err

https://groups.google.com/forum/#!topic/rabbitmq-users/hF2VO3e5RTA

[Michael Klishin]

That sounds like an environment-specific permission problem.

Is there anything in the various /var/log log files that expands on what permission wasn't granted?

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Karl Nilsson]

Not sure if it is relavant to the permissions issue but Joshua's changes have been republished into a different branch combined with some further tweaks to satisfy our build system and produce two rpms instead of one.

The PR is here: https://github.com/rabbitmq/rabbitmq-server/pull/934

It produces an rpm for el6 and one for el7 with systemd support. 

Testing/feedback is very welcome.

Cheers

Karl

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Karl Nilsson

Staff Software Engineer, Pivotal/RabbitMQ

[Karl Nilsson]

Giri,

If you (or anyone else) would like to test the changes in the aforementioned PR I'm including the el7 rpm here. I will send the el6 one in a separate email due to attachment size limits.

Cheers

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Karl Nilsson

Staff Software Engineer, Pivotal/RabbitMQ

[Karl Nilsson]

And here is the el6 package:

[Giri Chintala]

Nothing else in the log file.

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Giri Chintala]

Thanks Karl! i will test Test el7 first today.

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Karl Nilsson

Staff Software Engineer, Pivotal/RabbitMQ

[Giri Chintala]

I just completed testing el7. rabbitmq-server  successfully started after reboot ec2 instance. 

i followed below steps..

1. yum install -y socat

2. rpm -Uvh rabbitmq.rpm --force --nodeps

3. systemctl enable rabbitmq-server

4. service rabbitmq-server start

4. shutdown -r now

after reboot $ service rabbitmq-server status and $rabbitmqctl cluster_status worked fine.