ANN A note on RabbitMQ compatibility with Erlang/OTP 21

[Michael Klishin]

Dear RabbitMQ community,

As some of you may know, Erlang/OTP 21 is currently at RC2 and about to go GA in

the upcoming weeks.

It has some very nice I/O subsystem improvements and our team has seen non-trivial positive

effects from them on some next generation projects future RabbitMQ versions will use.

We also started investigating OTP 21 compatibility for the current release series, 3.7.x, and

master. In a matter of days we learned that supporting OTP 21 is no small task and can't

be expected to be complete in a week or two.

OTP 21 has more breaking changes than OTP 20, leave alone earlier releases. In many cases

they do not affect RabbitMQ per se but they do affect some of our key dependencies: Lager, Ranch, Elixir. Since all of those projects involve a lot more users than RabbitMQ, consensus has

to be reached on what'd be the best way to reach OTP 21 compatibility. As with any

scenario that involves humans and consensus, this can take a while and the outcome is

not easy to predict.

All of this, sadly, makes us list OTP 21 as officially unsupported. Until that changes, no issues

reported for OTP 21 will be investigated, no zero dependency RPM versions of Erlang 21

will be produced and so on.

We really hope RabbitMQ 3.7.x will support OTP 21 one day.

You can track [1] for updates on our progress. If not then it will be considered for RabbitMQ 3.8.0,

potentially as a minimum required version, which can complicate upgrades :(

We will post updates to this thread as the story develops.

Thank you for using RabbitMQ!

1. https://github.com/rabbitmq/rabbitmq-server/issues/1616

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

Our team has some good news to share. We are very close to supporting OTP 21 [1].

All test suites pass and nearly all bugs we've found and reported to the OTP team were addressed

before OTP 21 goes GA.

Remaining test suites need a set of packages (Debian, RPM) that are not available for OTP release candidates.

Since we don't know when it's going to ship we cannot make any promises but there's a good chance RabbitMQ 3.7.7

will be OTP 21-compatible and will ship on an expedited schedule. This is not a promise of delivery of any kind, of course.

We'll post future updates to this thread.

1. https://github.com/rabbitmq/rabbitmq-server/issues/1616

[Michael Klishin]

RabbitMQ 3.7.7-beta.1 [1] is compatible with OTP 21 RC2 and the tip of master. We don't yet have Debian

packages to add it to our upgrade pipelines and some defaults have to change in order to be optimal on OTP 21

but this milestone is ready for community testing.

Please give this preview release a try on Erlang 21 and let us know how it goes!

1. https://github.com/rabbitmq/rabbitmq-server/releases/tag/v3.7.7-beta.1

[Michael Klishin]

RabbitMQ 3.7.7-beta.1 was produced with a version of Elixir that's not OTP 21 compatible

(only 1.6.5 is at the moment), so some CLI commands will fail with obscure messages.

3.7.7-beta.2 is coming in a bit and should be fine.

--
You received this message because you are subscribed to the Google Groups "rabbitmq-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

3.7.7-beta.2 is up on GitHub [1] and Bintray [2]. A little birdie tells us that Erlang/OTP 21 GA may ship as soon as later today.

1. https://github.com/rabbitmq/rabbitmq-server/releases/tag/v3.7.7-beta.2.

2. https://bintray.com/rabbitmq/all-prereleases/rabbitmq-server/3.7.7-beta.2

To post to this group, send email to rabbitm...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[vikinghawk]

In reference to "It has some very nice I/O subsystem improvements and our team has seen non-trivial positive

effects"... have you guys seen or expect any performance improvements on 3.7? Or is it more along the lines of new apis that rabbit would have to consume to see the benefit? Thanks!

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-user...@googlegroups.com.

To post to this group, send email to rabbitm...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

MK

Staff Software Engineer, Pivotal/RabbitMQ

[Michael Klishin]

All file I/O functions are now more efficient [1][2]. On the Raft library we are working on for 3.8 the difference

is x2-3 times, OTP team's blog post reports a comparable difference on synthetic benchmarks [2].

This doesn't mean that it will directly translate to RabbitMQ throughput but there can be some positive effect.

For example, a node on OTP 20 starts in 5 seconds in one of my environments but in 3.4 on OTP 21.

Node startup does a fair amount of file I/O, e.g. when plugins are extracted and node's data is loaded.

What we have to do for is to make defaults more optimal [1], which will take some experimentation

and will last beyond 3.7.7. I'm not aware of any new APIs.

There are some efficiency improvements across the board but they are going to make less difference

for RabbitMQ. But again, both us and the users get them for free.

1. http://blog.erlang.org/IO-Polling/

2. http://blog.erlang.org/My-OTP-21-Highlights/

To unsubscribe from this group and stop receiving emails from it, send an email to rabbitmq-users+unsubscribe@googlegroups.com.
To post to this group, send email to rabbitmq-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Klishin]

RabbitMQ 3.7.7-rc.2 is now in the late stages of testing and we hope it will be published soon.

Given how many people intentionally or unintentionally install OTP 21 and run into known issues

we will try to ship 3.7.7 ASAP.

[Michael Klishin]

RabbitMQ 3.7.7 is out and supports Erlang/OTP 21 [1].

1. https://groups.google.com/forum/#!topic/rabbitmq-users/EuRwfeTGA_o