Erlang client dependencies (lager etc.)

[Dominic Morneau]

Hi all,

Would it be possible / make sense to remove the official Erlang AMQP client dependencies on rabbit_common and particularly lager?

Currently, the client brings in a bunch of libraries via rabbit_common:

• credentials_obfuscation
  
• jsx
• lager
• goldrush
• ranch
• recon

This has caused a bunch of problems in my (Elixir) applications after adding the library:

1. We started getting an error message on startup: calling logger:remove_handler(default) failed: :error {:badmatch, {:error, {:not_found, :default}}}. Eventually I found this workaround.
2. Lager started writing app logs to a "log" folder. I'm not sure how to disable it without losing messages from the AMQP client, and we can't use these logs in their current form, because they're not structured (our app logs are in JSON). Messages from the AMQP client don't seem to appear in our regular logger_std_h handler or in the Elixir console logs.
3. Lager also seems to be affecting our Sentry integration. Sentry hooks into the logger to catch exception reports. When the AMQP library hits connection problems, we get hundreds or thousands of separate alerts, it seems some metadata is missing or different which makes Sentry unable to de-duplicate the exceptions as it does for everything else.
4. In apps that already depend on Ranch for their own purposes, we're blocked from upgrading to Ranch 2.0, because rabbit_common wants Ranch 1.7.1. I'm not sure it makes sense for a client to be depending on a socket acceptor library in the first place.

I'm guessing this would cause some code duplication but perhaps the tradeoff is worth it? It's really difficult to work with libraries that affect global configuration.

Thanks!

[Luke Bakken]

Hello,

This has been discussed among the RabbitMQ team but the amount of work is substantial, especially since several protocol plugins depend on the AMQP client.

You can open a detailed request here but we can't make any promises - https://github.com/rabbitmq/rabbitmq-server

Of course, pull requests are appreciated and will be reviewed.

Thanks,

Luke

[Dominic Morneau]

Thanks Luke. I'll check if I could make some PRs.

I recon this is a bit ambitious, but would the RabbitMQ team be open to switching from Lager to the standard logger added in OTP-21?

For reference, the Lager PR was 491.

Regards,

Dominic

2021年1月5日火曜日 7:44:20 UTC+9 Luke Bakken:

[Michael Klishin]

This should no longer be an issue with modern RabbitMQ client and Lager versions, at least in Erlang. Our team dog foods this client

with Elixir in RabbitMQ CLI tools test suites. I don't remember running into this in a couple of years.

The amount of work (or duplication) this would entail is substantial. It's hardly a priority for our team.

Consider upgrading to RabbitMQ client 3.8.9. With network connections you will be able to connect to any server

version starting with 2.0 (which is about a decade old).

[Michael Klishin]

We require Erlang/OTP 22.3 these days so we can do it. This will potentially require a substantial amount of work because of

 * Features that Lager has (such as component-specific logging modules and sinks) but OTP logger may or may not have

 * Plugins that use Lager and expect it to be available

 * Subtle RabbitMQ features that rely on Lager's configurability

We would certainly be interested in seeing what that would look like for 3.9 (master) but according to [1], the built-in OTP logger

is minimalistic. This means we would have to drop some features and piss off some users who will potentially never upgrade.

1. https://erlang.org/doc/apps/kernel/logger_chapter.html#configuration

[Michael Klishin]

A potential much easier way would be to move some server-specific dependencies such as Ranch to rabbitmq-server.

I cannot immediately think of why rabbit_common depends on Ranch: rabbit_networking and rabbit_net are both in rabbitmq-server

these days. So it could be for historical reasons.

Then again, all modern versions of rabbit_common depend on Ranch 2.0 so upgrading the client will make the Ranch version conflict

go away naturally.

[Michael Klishin]

We have investigated this and removing the dependency seems safe, so we did it [1]. Will first ship in 3.8.10.

Thank you for bringing this up.

1. https://github.com/rabbitmq/rabbitmq-server/pull/2707

[Dominic Morneau]

Fantastic, thank you!

2021年1月8日(金) 0:44 Michael Klishin <klis...@vmware.com>:

--
You received this message because you are subscribed to a topic in the Google Groups "rabbitmq-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rabbitmq-users/DnQ0TRR0PEs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rabbitmq-user...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rabbitmq-users/1b41bca2-a5aa-438a-aed0-82f95bcf7173n%40googlegroups.com.