Slow init:stop/0 due to purging modules

4 views
Skip to first unread message

Frank Hunleth

unread,
Jul 6, 2021, 10:52:50 AMJul 6
to erlang-questions Questions
I'm looking into an issue where init:stop/0 is taking a surprisingly
long time (to me at least) on OTP 24. This is on a Raspberry Pi Zero,
so it's not the fastest CPU and no JIT is involved. With a small-ish
release started in embedded mode, it's taking almost 2 minutes to run
init:stop/0. It doesn't take nearly this long to start. I haven't
timed it, but I'd guess that it was closer to 10-20 seconds to load
all modules and start all applications.

Right now, I've narrowed this down to init:do_unload/1:

```
do_unload([M|Mods]) ->
catch erlang:purge_module(M),
catch erlang:delete_module(M),
catch erlang:purge_module(M),
do_unload(Mods);
do_unload([]) ->
ok.
```

I was assuming that I'd find a NIF that was slow to unload, but it
looks like every module takes roughly the same time to purge, delete,
purge. It's just that there are a lot of modules and purge, delete,
purge is slow enough to accumulate to a long time on a Raspberry Pi
Zero.

I'm starting to enter code that I'm much less familiar with, so I was
wondering if anyone could help:

1. Does it make sense for unloading all modules to take longer than
loading them? Just want to sanity check that unloading isn't slow by
design.
2. If the VM is going to exit anyway, is do_unload necessary for
graceful shutdown? Like if I were to implement my own graceful
shutdown that stopped all applications and halted, would that be
shortsighted?
3. Assuming that purge,delete,purge is important, are there particular
places in the code that I should look at to see if I can figure out
why they're slow?

Thanks!
Frank

Mikael Pettersson

unread,
Jul 7, 2021, 4:06:21 AMJul 7
to Frank Hunleth, erlang-questions Questions
On Tue, Jul 6, 2021 at 4:52 PM Frank Hunleth
<fhun...@troodon-software.com> wrote:
> 1. Does it make sense for unloading all modules to take longer than
> loading them? Just want to sanity check that unloading isn't slow by
> design.

Unloading a module is slower than loading it, since the unload needs
to check all processes to see if any of them currently run in that
module, if they do they get killed. The VM cannot permit the unload to
create dangling references into the unloaded code.

> 2. If the VM is going to exit anyway, is do_unload necessary for
> graceful shutdown?

I can't immediately see why unload would be needed in this scenario. I
hope someone from the OTP team can enlighten us.

Björn Gustavsson

unread,
Jul 7, 2021, 7:11:32 AMJul 7
to Mikael Pettersson, erlang-questions Questions
On Wed, Jul 7, 2021 at 10:06 AM Mikael Pettersson <mikpe...@gmail.com> wrote:
>
> On Tue, Jul 6, 2021 at 4:52 PM Frank Hunleth
> <fhun...@troodon-software.com> wrote:
> > 1. Does it make sense for unloading all modules to take longer than
> > loading them? Just want to sanity check that unloading isn't slow by
> > design.
>
> Unloading a module is slower than loading it, since the unload needs
> to check all processes to see if any of them currently run in that
> module, if they do they get killed. The VM cannot permit the unload to
> create dangling references into the unloaded code.

Additionally, nowadays the heap needs to be scanned for references to
the literal pool in the module being purged. If a process has a
reference to a literal that will soon go away, it must be copied to
the heap of process.

> > 2. If the VM is going to exit anyway, is do_unload necessary for
> > graceful shutdown?
>
> I can't immediately see why unload would be needed in this scenario. I
> hope someone from the OTP team can enlighten us.

I also don't see any reason why the code would need to be unloaded. It
seems that it is done because the code is shared with init:restart()
and init:reboot(). At the time the code was rewritten a long time,
purging code was much faster than it is today.

Frank, I suggest that you create an issue.

/Björn

--
Björn Gustavsson, Erlang/OTP, Ericsson AB

Frank Hunleth

unread,
Jul 7, 2021, 10:16:52 AMJul 7
to Björn Gustavsson, erlang-questions Questions
> Frank, I suggest that you create an issue.

See https://github.com/erlang/otp/issues/5031.

Thank you all for the very informative responses. I have a new
appreciation for code unloading.

Frank
Reply all
Reply to author
Forward
0 new messages