dlopen() and program::add_object() on github

[zero]

Hi all,

The issue is on github link 

Then we continue the discuss. 

But I don't understand the "implicit" dependence you mention... It seems you mean that for some reason your application wants to know when or whether B is loaded

Actually, I'm talking about the reference count of A, and B depends on A. When load B, it will recursive dlopen A. 

So, there is a question, when load B how could OSv add A's reference count too?

If we don't know the count, when unload A and load A again, because A is still in memory, dlopen A will return a handle. the static variables in A will not be reinitialized. If we use static variables's default constructors dynamic create some thing in dlopen(we know that dlopen will do init first and then transfer control to main), if A is in already in memory the init will not work!

The following is the whole process：

1. load A

2. load B（B depends on A ）

3. unload A

4. load A   ----->  static variables in A will not be reinitialized

If we have reference count of A is still 1（because we implement reference count ourselves, I think when we unload A the reference count is 0, but dlopen in system will be 1）, then we know A is still in memory, and do some work do deal with it.

Of course, we can also judge whether A is already in memory use dlopen with RTLD_NOLOAD , if A is in memory we will deal with it.

[Nadav Har'El]

On Wed, Nov 4, 2020 at 11:33 AM zero <daoh...@gmail.com> wrote:

Hi all,

The issue is on github link 

Then we continue the discuss. 

But I don't understand the "implicit" dependence you mention... It seems you mean that for some reason your application wants to know when or whether B is loaded

Actually, I'm talking about the reference count of A, and B depends on A. When load B, it will recursive dlopen A. 

So, there is a question, when load B how could OSv add A's reference count too?

If we don't know the count, when unload A and load A again, because A is still in memory, dlopen A will return a handle. the static variables in A will not be reinitialized.

I think maybe I finally understand what you mean.

The dlclose() documentation suggests that it and dlopen() do their own reference counting - and if you dlclose() a library that has been dlopen()ed just once - the library will be unloaded. Even if some other library also used it as DT_NEEDED and even if one of the functions in the to-be-unloaded library is being used (because some other library resolved it).

But is this really what should happen, or happens, in Linux? If you dlopen() A and *also* dlopen() B which needs A, I think the reference count of A increases twice. How could things work otherwise?

 

If we use static variables's default constructors dynamic create some thing in dlopen(we know that dlopen will do init first and then transfer control to main), if A is in already in memory the init will not work!

The following is the whole process：

1. load A

2. load B（B depends on A ）

3. unload A

4. load A   ----->  static variables in A will not be reinitialized

Did you try this on Linux? Does it behave differently from what it does in OSv?

If step 3 were to *really* unload A, how would B's code, which is still loaded and may be using symbols from A, continue to work?

If between points 3 and 4 you run some function from B, what is supposed to happen?

 

If we have reference count of A is still 1（because we implement reference count ourselves, I think when we unload A the reference count is 0, but dlopen in system will be 1）, then we know A is still in memory, and do some work do deal with it.

Of course, we can also judge whether A is already in memory use dlopen with RTLD_NOLOAD , if A is in memory we will deal with it.

--
You received this message because you are subscribed to the Google Groups "OSv Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/CADNiy7oBE1Ldc9hcd0dZf35H885Q8icjNAeiXXbZaPK14C16Rg%40mail.gmail.com.

[Nadav Har'El]

On Thu, Nov 5, 2020 at 2:23 PM zero <daoh...@gmail.com> wrote:

If step 3 were to *really* unload A, how would B's code, which is still loaded and may be using symbols from A, continue to work?

Yes step 3 will not unload A, and A is still in memory, when dlopen(A) again, it will return a handle, and will not do the init progress.

So this is the same as will happen in OSv... Are you suggesting that it doesn't? Or just *asking* what happens?

I think I tried to answer what should happen in my last email - please be more explicit if you tested OSv and Linux and

found that OSv does the wrong and/or different-from-Linux thing.

 

So I ask how OSv implements the reference count.

When OSv loads a shared object A, either - as you called it - "explicitly" (dlopen()) or "implicitly" (needed by another object) - if it is not in memory it is loaded and its reference count is 1, but if it's already loaded, its reference count is incremented. The reference count of the object is also incremented when some other object uses a function f() and resolving it happened to resolve to f() in A. These reference counts get decremented back when those other objects get unloaded, and when A's reference count eventually drops to 0, it will be completely removed from memory (and all of A's static variables also get lost).

If it depends on the path, then it will be a little different from dlopen implement.

I don't understand what you mean here. What do you mean "depends on the path"?

If you look at core/elf.cc, program::load_object() you'll see that indeed, already-loaded objects are looked up by object names, which are parts of the full path name of the file. Are you thinking, maybe, about the possibility that a single object has two names (perhaps a symbolic link, or something)e?

If you want to explain to me that there is a bug, and/or difference from Linux, the best thing to do is to write a simple example and run it on Linux and OSv and demonstrate how it behaves differently (if it does).

[zero]

Thank you very much for your patient answer, it is really helpful! I will do some testing and learning next, thanks again.

Best regards!

Nadav Har'El <n...@scylladb.com> 于2020年11月9日周一 下午7:53写道：