NS3 DCE - How does elf-loader determine/locate dependencies?

[Richard]

I have two Ubuntu 14.04 machines with libnl 3.2.25 compiled and installed successfully and I have pulled dce-linux-1.4 from the repository.

I then have a simulation that runs a binary with DceApplicationHelper, that has libnl as a dependency.

One of the machines succeeds in running the simulation while the other fails with;

assert failed. cond="false", msg="did not find libnl-3.so.200", file=../model/elf-cache.cc, line=299

The environment handling the libnl-3.so.200 library is identical across both machines; in terms of pkg-config, ldconfig and ldd.

Is there anything specific I am missing that would cause a library not to be found by elf-loader?

I've also tried exporting the library location to LD_LIBRARY_PATH which hasn't helped. The binary compiles and runs as expected on both machines outside of NS3. 

Thanks in advance,

Richard

Further information below;

$ pkg-config --modversion libnl-3.0
3.2.25

Library location is the same on both machines, I've also checked the files with stat;

$ ls /usr/lib/libnl*
/usr/lib/libnl-3.la                /usr/lib/libnl-idiag-3.la
/usr/lib/libnl-3.so                /usr/lib/libnl-idiag-3.so
/usr/lib/libnl-3.so.200            /usr/lib/libnl-idiag-3.so.200
/usr/lib/libnl-3.so.200.20.0       /usr/lib/libnl-idiag-3.so.200.20.0
/usr/lib/libnl-cli-3.la            /usr/lib/libnl-nf-3.la
/usr/lib/libnl-cli-3.so            /usr/lib/libnl-nf-3.so
/usr/lib/libnl-cli-3.so.200        /usr/lib/libnl-nf-3.so.200
/usr/lib/libnl-cli-3.so.200.20.0   /usr/lib/libnl-nf-3.so.200.20.0
/usr/lib/libnl-genl-3.la           /usr/lib/libnl-route-3.la
/usr/lib/libnl-genl-3.so           /usr/lib/libnl-route-3.so
/usr/lib/libnl-genl-3.so.200       /usr/lib/libnl-route-3.so.200
/usr/lib/libnl-genl-3.so.200.20.0  /usr/lib/libnl-route-3.so.200.20.0 

ldconfig points to the correct location ob both machines;

 $ ldconfig -p | grep libnl
libnl-3.so.200 (libc6) => /usr/lib/libnl-3.so.200
libnl-3.so (libc6) => /usr/lib/libnl-3.so
libnl-route-3.so.200 (libc6) => /usr/lib/libnl-route-3.so.200
libnl-route-3.so (libc6) => /usr/lib/libnl-route-3.so
libnl-nf-3.so.200 (libc6) => /usr/lib/libnl-nf-3.so.200
libnl-nf-3.so (libc6) => /usr/lib/libnl-nf-3.so
libnl-idiag-3.so.200 (libc6) => /usr/lib/libnl-idiag-3.so.200
libnl-idiag-3.so (libc6) => /usr/lib/libnl-idiag-3.so
libnl-genl-3.so.200 (libc6) => /usr/lib/libnl-genl-3.so.200
libnl-genl-3.so (libc6) => /usr/lib/libnl-genl-3.so
libnl-cli-3.so.200 (libc6) => /usr/lib/libnl-cli-3.so.200
libnl-cli-3.so (libc6) => /usr/lib/libnl-cli-3.so

ldd shows that it is pointing to the right library for both;

$ ldd build/bin_dce/mpdd 
linux-gate.so.1 =>  (0xb77b7000)
libnl-3.so.200 => /lib/i386-linux-gnu/libnl-3.so.200 (0xb7770000)
libnl-route-3.so.200 => /usr/lib/i386-linux-gnu/libnl-route-3.so.200 (0xb7727000)
librt.so.1 => /lib/i386-linux-gnu/librt.so.1 (0xb771d000)
libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xb7701000)
libconfig.so.9 => /usr/local/lib/libconfig.so.9 (0xb76f3000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb7544000)
libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xb74fe000)
/lib/ld-linux.so.2 (0xb77b8000)

The output from the failing machine is;

richard@hostone:~/Software/dce-test/source/ns-3-dce$ ./waf --run dce-mpdd-nested-csma
Waf: Entering directory `/home/richard/Software/dce-test/source/ns-3-dce/build'
--------------------------------------------------------------------
 Python bindings compilation 
--------------------------------------------------------------------
[ 10/370] lib/pkgconfig/libns3-dev-netlink-debug.pc:  -> build/lib/pkgconfig/libns3-dev-netlink-debug.pc
[115/370] lib/pkgconfig/libns3-dev-dce-debug.pc:  -> build/lib/pkgconfig/libns3-dev-dce-debug.pc
Waf: Leaving directory `/home/richard/Software/dce-test/source/ns-3-dce/build'
'build' finished successfully (1.369s)
assert failed. cond="false", msg="did not find libnl-3.so.200", file=../model/elf-cache.cc, line=299
terminate called without an active exception
Command ['/home/richard/Software/dce-test/source/ns-3-dce/build/myscripts/dce-mpdd-sim/bin/dce-mpdd-nested-csma'] terminated with signal SIGIOT. Run it under a debugger to get more information (./waf --run <program> --command-template="gdb --args %s <args>").

The GDB backtrace doesn't reveal anything, that would suggest a problem with NS3.

#0  0xb7fdd424 in __kernel_vsyscall ()
#1  0xb6887577 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#2  0xb688a9a3 in __GI_abort () at abort.c:89
#3  0xb6ad3595 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#4  0xb6ad11f3 in ?? () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#5  0xb6ad122f in std::terminate() () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#6  0xb7ee1949 in ns3::ElfCache::GetDepId (this=0xb7fda5e0 <ns3::CoojaLoader::Peek()::modules>, depname=...)
    at ../model/elf-cache.cc:299
#7  0xb7ee093f in ns3::ElfCache::EditBuffer (this=0xb7fda5e0 <ns3::CoojaLoader::Peek()::modules>, 
    map=0xb6610000 "\177ELF\001\001\001", selfId=6) at ../model/elf-cache.cc:211
#8  0xb7ee108c in ns3::ElfCache::EditFile (this=0xb7fda5e0 <ns3::CoojaLoader::Peek()::modules>, filename=..., selfId=6)
    at ../model/elf-cache.cc:259
#9  0xb7ee2444 in ns3::ElfCache::Add (this=0xb7fda5e0 <ns3::CoojaLoader::Peek()::modules>, filename=...)
    at ../model/elf-cache.cc:359
#10 0xb7ee6a22 in ns3::CoojaLoader::LoadModule (this=0x80a0f00, filename=..., flag=256) at ../model/cooja-loader-factory.cc:221
#11 0xb7ee664c in ns3::CoojaLoader::Load (this=0x80a0f00, filename=..., flag=256) at ../model/cooja-loader-factory.cc:172
#12 0xb7e54239 in ns3::DceManager::LoadMain (ld=0x80a0f00, filename=..., proc=0x80fc7e0, err=@0xb660be30: 0)
    at ../model/dce-manager.cc:1267
#13 0xb7e4ddcb in ns3::DceManager::PrepareDoStartProcess (current=0x80fcbb0) at ../model/dce-manager.cc:253
#14 0xb7e4e2ed in ns3::DceManager::DoStartProcess (context=0x80fcbb0) at ../model/dce-manager.cc:275
#15 0xb7ed439d in ns3::TaskManager::Trampoline (context=0x80a1ce0) at ../model/task-manager.cc:274
#16 0xb7ecdbbd in ns3::UcontextFiberManager::Trampoline (a0=-1, a1=-1209187490, a2=0, a3=134880480)
    at ../model/ucontext-fiber-manager.cc:199

[Richard]

The error pointing to libnl had thrown me, but its finally fixed.

The problem was actually libconfig as opposed to libnl. Installing it into /usr/lib or adding /usr/local/lib to LD_LIBRARY_PATH would have fixed it for me. 

[Hajime Tazaki]

thanks for the report.

LD_LIBRARY_PATH with arbitrary directory should work fine,
according to the code (elf-cache.cc, elf-ldd.cc, which I
don't understand well because not written by myself...).

-- Hajime

At Tue, 9 Dec 2014 09:59:44 -0800 (PST),
Richard wrote:
>
> [1 <multipart/alternative (7bit)>]
> [1.1 <text/plain; UTF-8 (7bit)>]

> --
> You received this message because you are subscribed to the Google Groups "ns-3-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ns-3-users+...@googlegroups.com.
> To post to this group, send email to ns-3-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/ns-3-users.
> For more options, visit https://groups.google.com/d/optout.
> [1.2 <text/html; UTF-8 (quoted-printable)>]
>

[Richard]

While not critical, some of the loading logic doesn't make a huge amount of sense to me. 

i.e;

- Try and locate objects.

- If one or more isn't found, continue anyway without loading any objects.

- Print an error message about the first object the binary can't access.  

I think this behavior can be improved, unless there is something that I'm missing. When I get some more time, I will look into it further and move over to the dev mailing list.

[Hajime Tazaki]

Hi Richard,

At Fri, 12 Dec 2014 05:01:02 -0800 (PST),

that would be great.
since DCE is somehow not friendly especially in error case,
such improvements will benefit all of users.

-- Hajime