SIGSEGV, Segmentation fault in ns3::Time::operator

[Andikan Otung]

Hi there,

I have a rather large NS3 program which takes in various commandline inputs and runs a simulation accordingly - producing a text file report upon completion. 

I automated simulations on this program with a script that generates ~4,000 unique input combinations but noticed that roughly a quarter of the simulations terminated prematurely with the error:

• Command ['/home/lina3040/workspace-2/ns-allinone-3.33/ns-3.33/build/scratch/...] terminated with signal SIGSEGV. Run it under a debugger to get more information (./waf --run <program> --gdb").

Running it under the debugger, I received:

• Program received signal SIGSEGV, Segmentation fault.   0x00001555540b1ca0 in ns3::Time::operator= (o=..., this=0x38) at ./ns3/nstime.h:132  132     m_data = o.m_data;

Incidentally, when I enable logs at the NS_LOG_FUNCTION level, and re-run the simulation with the same input combination, the error seems to disappear and the program executes as normal. 

Does anyone have any ideas of why this issue is occurring and how I can overcome it?

Due to the sheer number of simulations necessary, it is not feasible for me to run them with logs enabled.

Thanks

Andikan

[pdbarnes]

That is almost certainly not a bug in Time, but somewhere else.  Notice this address 0x38; that seems like a very low address.  

When you run that case with logging is the compiler optimization level changing?  If the optimization is changing from -O1 to -O3, say, it's possible the optimizer is exposing a bug elsewhere.  (It's even possible the optimizer is at fault, but the last time I saw that was 25 years ago...)  

Can you share your code?  Can you narrow it down (smaller failing test case, less run time...)?

P

[Adil Alsuhaim]

Do a backtrace with bt in the debugger, and trace the line of your code that is causing the error.

[Andikan Otung]

Thanks Adil, I had been looking for how to do that using gdb! The output I get is this:

(gdb) bt

#0  0x00007ffff6bad377 in ns3::Time::operator= (o=..., this=0x38)

    at ./ns3/nstime.h:132

#1  ns3::TcpRateLinux::SkbSent (this=0x555558182970, skb=<optimised out>, 

    isStartOfTransmission=<optimised out>)

    at ../src/internet/model/tcp-rate-ops.cc:229

#2  0x00007ffff6b45236 in ns3::TcpSocketBase::SendDataPacket (

    this=0x555558192910, seq=..., maxSize=<optimised out>, 

    withAck=<optimised out>) at ../src/internet/model/tcp-socket-base.cc:3044

#3  0x00007ffff6b36a97 in ns3::TcpSocketBase::SendPendingData (

    this=0x555558192910, withAck=true) at ./ns3/sequence-number.h:78

#4  0x00007ffff6b3cd0d in ns3::TcpSocketBase::ReceivedAck (

    this=0x555558192910, packet=..., tcpHeader=...)

    at ../src/internet/model/tcp-socket-base.cc:1877

#5  0x00007ffff6b3af79 in ns3::TcpSocketBase::ProcessEstablished (

    this=0x555558192910, packet=..., tcpHeader=...)

    at ./ns3/simple-ref-count.h:105

#6  0x00007ffff6b4855c in ns3::TcpSocketBase::DoForwardUp (

    this=0x555558192910, packet=..., fromAddress=..., toAddress=...)

    at ./ns3/simple-ref-count.h:105

#7  0x00007ffff6b3bb68 in ns3::TcpSocketBase::ForwardUp (this=0x555558192910, 

    packet=..., header=..., port=<optimised out>, incomingInterface=...)

    at ./ns3/simple-ref-count.h:105

--Type <RET> for more, q to quit, c to continue without paging--RET

#8  0x00007ffff6b55d86 in ns3::MemPtrCallbackImpl<ns3::Ptr<ns3::TcpSocketBase>, void (ns3::TcpSocketBase::*)(ns3::Ptr<ns3::Packet>, ns3::Ipv4Header, unsigned short, ns3::Ptr<ns3::Ipv4Interface>), void, ns3::Ptr<ns3::Packet>, ns3::Ipv4Header, unsigned short, ns3::Ptr<ns3::Ipv4Interface>, ns3::empty, ns3::empty, ns3::empty, ns3::empty, ns3::empty>::operator() (this=<optimised out>, a1=..., a2=..., 

    a3=<optimised out>, a4=...) at ./ns3/simple-ref-count.h:105

#9  0x00007ffff6a98803 in ns3::Callback<void, ns3::Ptr<ns3::Packet>, ns3::Ipv4Header, unsigned short, ns3::Ptr<ns3::Ipv4Interface>, ns3::empty, ns3::empty, ns3::empty, ns3::empty, ns3::empty>::operator() (a4=..., a3=<optimised out>, a2=..., a1=..., this=0x7fffffffca80) at ./ns3/simple-ref-count.h:105

#10 ns3::Ipv4EndPoint::ForwardUp (this=this@entry=0x5555572b0a80, p=..., header=..., sport=<optimised out>, incomingInterface=...) at ../src/internet/model/ipv4-end-point.cc:139

#11 0x00007ffff6aa52a8 in ns3::TcpL4Protocol::Receive (this=this@entry=0x555555f5fd50, packet=..., incomingIpHeader=..., incomingInterface=...) at ./ns3/simple-ref-count.h:105

#12 0x00007ffff6a7fc42 in ns3::Ipv4L3Protocol::LocalDeliver (this=0x555555f5fae0, packet=..., ip=..., iif=1) at ./ns3/simple-ref-count.h:105

#13 0x00007ffff6a87d5c in ns3::MemPtrCallbackImpl<ns3::Ipv4L3Protocol*, void (ns3::Ipv4L3Protocol::*)(ns3::Ptr<ns3::Packet const>, ns3::Ipv4Header const&, unsigned int), void, ns3::Ptr<ns3::Packet const>, ns3::Ipv4Header const&, unsigned int, ns3::empty, ns3::empty, ns3::empty, ns3::empty, ns3::empty, ns3::empty>::operator() (this=<optimised out>, a1=..., a2=..., a3=<optimised out>)

    at ./ns3/simple-ref-count.h:105

#14 0x00007ffff6bce4d2 in ns3::Callback<void, ns3::Ptr<ns3::Packet const>, ns3::Ipv4Header const&, unsigned int, ns3::empty, ns3::empty, ns3::empty, ns3::empty, ns3::empty, ns3::empty>::operator() (

    a3=<optimised out>, a2=..., a1=..., this=<optimised out>) at ./ns3/simple-ref-count.h:105

#15 ns3::Ipv4ListRouting::RouteInput (this=0x555555f5f670, p=..., header=..., idev=..., ucb=..., mcb=..., lcb=..., ecb=...) at ../src/internet/model/ipv4-list-routing.cc:155

#16 0x00007ffff6a7eaf0 in ns3::Ipv4L3Protocol::Receive (this=<optimised out>, device=..., p=..., protocol=<optimised out>, from=..., to=..., packetType=0) at ./ns3/simple-ref-count.h:105

#17 0x00007ffff6a87ee4 in ns3::MemPtrCallbackImpl<ns3::Ipv4L3Protocol*, void (ns3::Ipv4L3Protocol::*)(ns3::Ptr<ns3::NetDevice>, ns3::Ptr<ns3::Packet const>, unsigned short, ns3::Address const&, ns3::Address const&, ns3::NetDevice::PacketType), void, ns3::Ptr<ns3::NetDevice>, ns3::Ptr<ns3::Packet const>, unsigned short, ns3::Address const&, ns3::Address const&, ns3::NetDevice::PacketType, ns3::empty, ns3::empty, ns3::empty>::operator() (this=<optimised out>, a1=..., a2=..., a3=<optimised out>, a4=..., a5=..., a6=0) at ./ns3/simple-ref-count.h:105

#18 0x00007ffff686cc2b in ns3::Callback<void, ns3::Ptr<ns3::NetDevice>, ns3::Ptr<ns3::Packet const>, unsigned short, ns3::Address const&, ns3::Address const&, ns3::NetDevice::PacketType, ns3::empty, ns3::empty, ns3::empty>::operator() (a6=<optimised out>, a5=..., a4=..., a3=2048, a2=..., a1=..., this=0x555555f603e0) at ./ns3/simple-ref-count.h:105

#19 ns3::TrafficControlLayer::Receive (this=0x555555f610b0, device=..., p=..., protocol=2048, from=..., to=..., packetType=<optimised out>) at ../src/traffic-control/model/traffic-control-layer.cc:331

#20 0x00007ffff6a88273 in ns3::MemPtrCallbackImpl<ns3::Ptr<ns3::TrafficControlLayer>, void (ns3::TrafficControlLayer::*)(ns3::Ptr<ns3::NetDevice>, ns3::Ptr<ns3::Packet const>, unsigned short, ns3::Address const&, ns3::Address const&, ns3::NetDevice::PacketType), void, ns3::Ptr<ns3::NetDevice>, ns3::Ptr<ns3::Packet const>, unsigned short, ns3::Address const&, ns3::Address const&, ns3::NetDevice::PacketType, ns3::empty, ns3::empty, ns3::empty>::operator() (this=<optimised out>, a1=..., a2=..., a3=<optimised out>, a4=..., a5=..., a6=0) at ./ns3/simple-ref-count.h:105

#21 0x00007ffff636e49d in ns3::Callback<void, ns3::Ptr<ns3::NetDevice>, ns3::Ptr<ns3::Packet const>, unsigned short, ns3::Address const&, ns3::Address const&, ns3::NetDevice::PacketType, ns3::empty, ns3::empty, ns3::empty>::operator() (a6=<optimised out>, a5=..., a4=..., a3=2048, a2=..., a1=..., this=0x555555f672b0) at ./ns3/simple-ref-count.h:105

#22 ns3::Node::ReceiveFromDevice (this=0x555555f42410, device=..., packet=..., protocol=2048, from=..., to=..., packetType=<optimised out>, promiscuous=false) at ../src/network/model/node.cc:326

#23 0x00007ffff636e9f7 in ns3::Node::NonPromiscReceiveFromDevice (this=0x555555f42410, device=..., packet=..., protocol=<optimised out>, from=...) at ./ns3/simple-ref-count.h:105

#24 0x00007ffff637217c in ns3::MemPtrCallbackImpl<ns3::Node*, bool (ns3::Node::*)(ns3::Ptr<ns3::NetDevice>, ns3::Ptr<ns3::Packet const>, unsigned short, ns3::Address const&), bool, ns3::Ptr<ns3::NetDevice>, ns3::Ptr<ns3::Packet const>, unsigned short, ns3::Address const&, ns3::empty, ns3::empty, ns3::empty, ns3::empty, ns3::empty>::operator() (this=<optimised out>, a1=..., a2=..., a3=<optimised out>, 

    a4=...) at ./ns3/simple-ref-count.h:105

#25 0x00007ffff6473f69 in ns3::Callback<bool, ns3::Ptr<ns3::NetDevice>, ns3::Ptr<ns3::Packet const>, unsigned short, ns3::Address const&, ns3::empty, ns3::empty, ns3::empty, ns3::empty, ns3::empty>::operator() (a4=..., a3=<optimised out>, a2=..., a1=..., this=<optimised out>) at ./ns3/simple-ref-count.h:105

#26 ns3::PointToPointNetDevice::Receive (this=0x555555f5d860, packet=...) at ../src/point-to-point/model/point-to-point-net-device.cc:377

#27 0x00007ffff647e3ae in ns3::MakeEvent<void (ns3::PointToPointNetDevice::*)(ns3::Ptr<ns3::Packet>), ns3::Ptr<ns3::PointToPointNetDevice>, ns3::Ptr<ns3::Packet> >(void (ns3::PointToPointNetDevice::*)(ns3::Ptr<ns3::Packet>), ns3::Ptr<ns3::PointToPointNetDevice>, ns3::Ptr<ns3::Packet>)::EventMemberImpl1::Notify() (this=<optimised out>) at ./ns3/simple-ref-count.h:105

#28 0x00007ffff619a6ce in ns3::DefaultSimulatorImpl::ProcessOneEvent (this=0x555555621e00) at ../src/core/model/default-simulator-impl.cc:151

#29 0x00007ffff619a746 in ns3::DefaultSimulatorImpl::Run (this=0x555555621e00) at ../src/core/model/default-simulator-impl.cc:204

#30 0x00005555555a8caa in arcTrustGuardEff_AttackScale_PacketSize_Core (simulationIn=..., seedIn=<optimised out>, parameterIn=<optimised out>, iterationIn=<optimised out>)

    at ../scratch/attack-scale-all/attack-scale-all.cc:3110

#31 0x000055555556d7bb in main (argc=10, argv=0x7fffffffdbd8) at /usr/include/c++/9/bits/basic_string.h:936

Some explanation:

On line 3110 of attack-scale-all.cc (of#30) is: Simulator::Run ();

Other than that (#30 & #31), I do not notice any references to code that I've written and it is not immeditately clear to me from the output, what the exact issue is..

A 

[Tom Henderson]

Running your program through valgrind may also provide some clues:

https://www.nsnam.org/wiki/HOWTO_use_Valgrind_to_debug_memory_problems

- Tom

On 7/23/21 7:57 PM, Adil Alsuhaim wrote:
> Do a backtrace with *bt* in the debugger, and trace the line of
> /your/ code that is causing the error.

>
> On Wednesday, July 21, 2021 at 5:39:05 PM UTC-4 andika...@googlemail.com
> wrote:
>
> Hi there,
>
> I have a rather large NS3 program which takes in various commandline
> inputs and runs a simulation accordingly - producing a text file
> report upon completion.
>
> I automated simulations on this program with a script that generates
> ~4,000 unique input combinations but noticed that roughly a quarter
> of the simulations terminated prematurely with the error:
>

> * Command

> ['/home/lina3040/workspace-2/ns-allinone-3.33/ns-3.33/build/scratch/...]
> terminated with signal SIGSEGV. Run it under a debugger to get
> more information (./waf --run <program> --gdb").
>
>
> Running it under the debugger, I received:
>

> * /Program received signal SIGSEGV, Segmentation fault.
> //0x00001555540b1ca0 in ns3::Time::operator= (o=..., this=0x38)
> at ./ns3/nstime.h:132 //132    m_data = o.m_data;/

>
> Incidentally, when I enable logs at the NS_LOG_FUNCTION level, and
> re-run the simulation with the same input combination, the error
> seems to disappear and the program executes as normal.
>

> *Does anyone have any ideas* of why this issue is occurring and how

> I can overcome it?
>
> Due to the sheer number of simulations necessary, it is not feasible
> for me to run them with logs enabled.
>
> Thanks
> Andikan
>

> --
> Posting to this group should follow these guidelines
> https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting
> <https://www.nsnam.org/wiki/Ns-3-users-guidelines-for-posting>
> ---
> You received this message because you are subscribed to the Google
> Groups "ns-3-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to ns-3-users+...@googlegroups.com
> <mailto:ns-3-users+...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/ns-3-users/8839ac8c-7a91-4700-bc62-9e58508cd835n%40googlegroups.com
> <https://groups.google.com/d/msgid/ns-3-users/8839ac8c-7a91-4700-bc62-9e58508cd835n%40googlegroups.com?utm_medium=email&utm_source=footer>.