Use of Libevent and/or HWLOC after PMIx Finalize

[Ralph Castain]

Hi folks

Been working on some memory leak cleanup for the PMIx library and hit something that is obvious when you look at it hard enough, but easily overlooked.

PMIx initializes the Libevent library during PMIx_Init and creates its own event base(s). Upon finalize, the library free’s the event bases that it used. However, Libevent “leaks” a little bit of memory with each event base - the only way to completely clean it up is to shutdown the library.

The problem arises if the host process is also using Libevent - either directly or through another dependency. If PMIx shuts down Libevent, then it is shutdown for _every_ library that is using it in that process. So making PMIx run “valgrind clean” has an unintended consequence of requiring that no other library in the process use Libevent after the host process calls PMIx_Finalize.

Of course, there is no way for PMIx to know if other libraries in the host process are using Libevent. One solution, though, would be to allow passing of an attribute to PMIx_Finalize (and the server and tool versions) directing PMIx to not finalize Libevent. Another solution, of course, is for PMIx to simply leak the Libevent memory - it’s only a few bytes per event base. Annoying to those striving for memory cleanliness, but...perhaps a solution.


Similarly, PMIx will create an HWLOC topology tree (either by directly discovering the topology or attaching to a shared memory instance of the tree) that it makes available to its host process. If the host process defines an abstraction variable (e.g., mytopology = pmix-topology), then when PMIx finalizes and destroys the topology, the host process will be left with a stale address pointer. Nothing PMIx itself can do about the stale pointer, but we could provide an attribute to direct PMIx to leave the topology tree instanced. Alternatively, of course, the host process can define the topology tree itself and pass its pointer to PMIx - thus letting the host process control when the tree is destroyed.


I bring this up because I have discovered at least one PMIx consumer (OpenMPI) that is exposed to both of these problems in their code base. I’m not sure which of the solutions they will choose, or if others can/will be devised. Either way, I figured maybe I should warn others in case they have similar issues.

Ralph

[Howard Pritchard]

Hi Raph,

I vote for maximum simplicity.  

We don't in the general case know what the consumers of libevent functionality are,  so putting up with the small memory leak within libevent is a reasonable compromise for now.

Someone could always try to fix this memory leak in libevent if it was important.  Although that raises other questions about how well that software is being supported...

Similarly for the hwloc topology tree, if a pmix functionality consumer invokes PMIx_Finalize, it should assume that it needs to cleanup things like any pointers, etc. to objects returned from a pmix call.

So I don't see any urgent need to modify the PMIx API to support this case, but maybe someone else has a different opinion?

Howard

--
You received this message because you are subscribed to the Google Groups "pmix" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pmix+uns...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pmix/9C058D2F-DC91-46C0-989C-CF9E3865A535%40pmix.org.