ns-3.10 build errors and memory issue

[Amit Samanta]

Hi,

We are trying to build ns-3.10 on Ubuntu 12.04 with 512MB RAM. We are running the following command to build ns3:

./waf clean

./waf configure

./waf build

After running the commands with 512MB RAM, we are getting the build failed error - "Virtual memory gets exhausted". After that, we also tried the same process with 1GB RAM. But, still it fails to build the process. Therefore, we are posting the following errors that we are getting:

Build failed:  -> task failed (err #1): 

{task: command_task ns3modulegen.py,ns3modulegen_generated.py,ns3modulegen_core_customizations.py,ns3_module_csma.py,ns3_module_ping6.py,

ns3_module_lte.py,ns3_module_test.py,ns3_module_flow_monitor.py,ns3_module_static_routing.py,ns3_module_internet_stack.py,

ns3_module_list_routing.py,ns3_module_helper.py,ns3_module_mpi.py,ns3_module_flame.py,ns3_module_uan.py,ns3_module_common.py,

ns3_module_v4ping.py,ns3_module_stats.py,ns3_module_node.py,ns3_module_bulk_send.py,ns3_module_emu.py,ns3_module_udp_client_server.py,

ns3_module_simulator.py,ns3_module_aodv.py,ns3_module_udp_echo.py,ns3_module_core.py,ns3_module_wifi.py,ns3_module_dsdv.py,

ns3_module_contrib.py,ns3_module_point_to_point.py,ns3_module_onoff.py,ns3_module_visualizer.py,ns3_module_nix_vector_routing.py,

ns3_module_packet_sink.py,ns3_module_wimax.py,ns3_module_mesh.py,ns3_module_spectrum.py,ns3_module_bridge.py,ns3_module_olsr.py,

ns3_module_energy.py,ns3_module_global_routing.py,ns3_module_mobility.py,ns3_module_topology_read.py,ns3_module_tap_bridge.py,

ns3_module_virtual_net_device.py,ns3_module_dot11s.py,ns3_module_radvd.py -> ns3module.cc,ns3module.h,ns3modulegen.log,ns3_module_csma.cc,ns3_module_ping6.cc,ns3_module_lte.cc,

ns3_module_test.cc,ns3_module_flow_monitor.cc,ns3_module_static_routing.cc,ns3_module_internet_stack.cc,

ns3_module_list_routing.cc,ns3_module_helper.cc,ns3_module_mpi.cc,ns3_module_flame.cc,ns3_module_uan.cc,

ns3_module_common.cc,ns3_module_v4ping.cc,ns3_module_stats.cc,ns3_module_node.cc,ns3_module_bulk_send.cc,

ns3_module_emu.cc,ns3_module_udp_client_server.cc,ns3_module_simulator.cc,ns3_module_aodv.cc,ns3_module_udp_echo.cc,

ns3_module_core.cc,ns3_module_wifi.cc,ns3_module_dsdv.cc,ns3_module_contrib.cc,ns3_module_point_to_point.cc,

ns3_module_onoff.cc,ns3_module_visualizer.cc,ns3_module_nix_vector_routing.cc,ns3_module_packet_sink.cc,ns3_module_wimax.cc,

ns3_module_mesh.cc,ns3_module_spectrum.cc,ns3_module_bridge.cc,ns3_module_olsr.cc,ns3_module_energy.cc,ns3_module_global_routing.cc,

ns3_module_mobility.cc,ns3_module_topology_read.cc,ns3_module_tap_bridge.cc,ns3_module_virtual_net_device.cc,ns3_module_dot11s.cc,

ns3_module_radvd.cc}

Is there any minimum memory requirement to build the ns-3.10??

Thanks,

Amit

[Amit Samanta]

Hi,

Additionally, we have also tried building the most recent release of NS-3, but it failed too.

Thanks,

Amit

[Tommaso Pecorella]

Hi,

I'd definitely use the latest ns-3. the -10 version is really too old.

About the memory requirements, it's not ns-3, it's more the compiler. Try disabling python, and using a single compiling unit at a time (-j1).

Finally (and most importantly), "Virtual memory gets exhausted" doesn't mean that you ended the RAM. It means that your disk is full.

Little known fact (nowadays). Linux memory is part physical, and part virtual. The virtual memory is saved in a swap area in your HD. Usually this swap area is large enough to just forget about it, but... but it could have been set to a sub-optimal value, or it could be in a shrink-and-grow file (like MacOS does).

Solution for case 1 (if you have a swap partition): make it larger.

Solution for case 2 (no swap partition, swap file): free some disk space.

Cheers,

T.

[Amit Samanta]

 Hi,

Thank you for your valuable suggestions. Please note that we are not using a physical machine here, but building NS3 in a virtual container. Therefore, we have some resource constraints and which is why, we are trying to figure out the minimum viable configurations.

We tried using a latest version of NS3. After doing the necessary steps as suggested by you, still we are facing the same problem with the build error "Cannot allocate memory".

Build failed

Traceback (most recent call last):

  File "/root/NS3/ns-allinone-3.24.1/ns-3.24.1/.waf-1.8.12-f00e5b53f6bbeab1384a38c9cc5d51f7/waflib/Task.py", line 110, in process

    ret=self.run()

  File "waf-tools/command.py", line 98, in run

    return pipeline.run(verbose=(Options.options.verbose > 0))

  File "waf-tools/shellcmd.py", line 289, in run

    proc = subprocess.Popen(cmd.argv, stdin=stdin, stdout=stdout, stderr=stderr, env=env, cwd=proc_cwd)

  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__

    errread, errwrite)

  File "/usr/lib/python2.7/subprocess.py", line 1143, in _execute_child

    self.pid = os.fork()

OSError: [Errno 12] Cannot allocate memory

For your information, intially we have used 1024MB swap space for NS3 build process, but later we changed it to 2048MB swap space as listed below:

            total       used       free     shared    buffers     cached

Mem:       1048576     540380     508196          0          0     455020

-/+ buffers/cache:      85360     963216

Swap:      2097152          0    2097152

And also we have changed the disk space from 10GB to 30 GB. Therefore the disk is not full now.

/dev/ploop12749p1   30G  4.2G   24G  15% /

none               512M  8.0K  512M   1% /dev

none               103M  1.1M  102M   1% /run

none               5.0M     0  5.0M   0% /run/lock

none               512M     0  512M   0% /run/shm

Also, the system limits are mentioned below.

ulimit -a

core file size          (blocks, -c) 0

data seg size           (kbytes, -d) unlimited

scheduling priority             (-e) 0

file size               (blocks, -f) unlimited

pending signals                 (-i) 127059

max locked memory       (kbytes, -l) 64

max memory size         (kbytes, -m) unlimited

open files                      (-n) 1024

pipe size            (512 bytes, -p) 8

POSIX message queues     (bytes, -q) 819200

real-time priority              (-r) 0

stack size              (kbytes, -s) 10240

cpu time               (seconds, -t) unlimited

max user processes              (-u) 127059

virtual memory          (kbytes, -v) unlimited

file locks                      (-x) unlimited

It would be helpful if you could provide some further pointers towards troubleshooting this process and successful building of NS3.

Thanks,

Amit

[Tommaso Pecorella]

Hi,

I'm kinda stuck as well. i can suggest to check the ram status while compiling, and (as I said) to limit the number of sub-processes.

To be honest I never tested the minimum requirement, and when I tried to compile ns-3 on a RaspberryPi it took all night, but it worked...

Sorry for not being so helpful, but I'm not an expert of low memory systems.

T.

[Amit Samanta]

Hi,

Thank you for your response. 

For your information, we are able to build and install ns-3.24 successfully on the physical machine with the configuration of 2GB RAM, 2GB Swap Space and 2 GB empty disk space.

But, we are not able to build ns-3.24 in a Virtual machine with the configuration as listed below:

             total       used       free     shared    buffers     cached

Mem:       2097152     368952    1728200          0          0     293776

-/+ buffers/cache:      75176    2021976

Swap:      4194304          0    4194304

Filesystem         Size  Used Avail Use% Mounted on

/dev/ploop12749p1   30G  4.4G   24G  16% /

none               1.0G  8.0K  1.0G   1% /dev

none               205M  1.1M  204M   1% /run

none               5.0M     0  5.0M   0% /run/lock

none               1.0G     0  1.0G   0% /run/shm

Additionally, we have also tried the building process of ns-3.24 in virtual machine with different possible options like, --disable python, -j1, -d optimized, etc.

Could you help us in giving some insight?

Thanks,

Amit

--
Posting to this group should follow these guidelines https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting
---
You received this message because you are subscribed to a topic in the Google Groups "ns-3-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ns-3-users/5GHc22Od8Yk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ns-3-users+...@googlegroups.com.
To post to this group, send email to ns-3-...@googlegroups.com.
Visit this group at https://groups.google.com/group/ns-3-users.
For more options, visit https://groups.google.com/d/optout.

--

Thanking you,

Amit Samanta,

MS Research Scholar,

School of Information Technology, 

Indian Institute of Technology Kharagpur,

West Bengal, India - 721302

[Tommaso Pecorella]

Hi,

no, I can't really help, especially because you didn't say something quite important.

1) What OS are you using as guest and host.

2) What virtualization system are you using.

3) The compilation system you are using (the compiler version, etc.).

You said you was successful in compiling and installing in a limited system but not virtualized. WONDERFUL, but are the two (the real and the virtualized) the same system ? Same processor type and so on ?

You know, if you ask if someone have a clue, we can do a best guess. If you want to have help in fixing something, you should't expect that people can guess AND give you meaningful answers.

T.

[Barun Saha]

Hi Tommaso,

Thank you for useful insights on this issue! Let me try to provide you with some more information on these systems.

CPU info from physical machine # 1 (Ubuntu 10.04; 32-bit; NS-3 build successful):

processor    : 1
vendor_id    : GenuineIntel
cpu family    : 6
model        : 23
model name    : Pentium(R) Dual-Core  CPU      E5400  @ 2.70GHz
stepping    : 10
cpu MHz        : 1203.000
cache size    : 2048 KB
physical id    : 0
siblings    : 2
core id        : 1
cpu cores    : 2
apicid        : 1


CPU info from physical machine # 2 (Ubuntu 12.04; 64-bit; build successful):

processor : 3

vendor_id : GenuineIntel

cpu family : 6

model : 60

model name : Intel(R) Core(TM) i5-4460  CPU @ 3.20GHz

stepping : 3

microcode : 0x17

cpu MHz : 800.000

cache size : 6144 KB

physical id : 0

siblings : 4

core id : 3

cpu cores : 4

apicid : 6

initial apicid : 6

fpu : yes

fpu_exception : yes

cpuid level : 13

wp : yes

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase bmi1 avx2 smep bmi2 erms invpcid

bogomips : 6385.83

clflush size : 64

cache_alignment : 64

address sizes : 39 bits physical, 48 bits virtual

power management:

CPU info from the container (Ubuntu 12.04 guest with CentOS 6.6 host; containers created using OpenVZ; build failed):

processor    : 3
vendor_id    : GenuineIntel
cpu family    : 6
model        : 58
model name    : Intel(R) Xeon(R) CPU E3-1220 V2 @ 3.10GHz
stepping    : 9
microcode    : 27
cpu MHz        : 1600.000
cache size    : 8192 KB
physical id    : 0
siblings    : 4
core id        : 3
cpu cores    : 4
apicid        : 6
initial apicid    : 6
fpu        : yes
fpu_exception    : yes
cpuid level    : 13
wp        : yes
flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf cpuid_faulting pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm arat pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt
bogomips    : 6185.66
clflush size    : 64
cache_alignment    : 64
address sizes    : 36 bits physical, 48 bits virtual
power management:


We have used gcc 4.4 all throughout. Additionally, the build also failed with gcc 4.6 in the container. As such, we are starting to doubt that the issue might be something related with the container, but not with NS-3. However, we are at a loss to figure out what such differences are. We are not necessarily looking for the exact solution here, but, yes, your inputs are really helpful to us in trying to troubleshoot the issue.

Please let us know if you would require any more information. Thank you, again!

P.S. On a note aside, we have been able to install NS-2 in all the systems -- physical and containers.