Experiencing unexpected TTL exceeded on ICMP

[Pascal]

Hi all

I am working on an example where I have some central network composed of a handful of nodes that are connected with point-to-point connections.

To the nodes in that central network I add sub-topologies, consisting of a "gateway" of sorts, which connects to one such central network node using also a point-to-point connection, as well as a handful of other nodes in each sub-topology that are individually connected to the gateway (again with point-to-point channels)

To make this a bit easier to understand, here an example graph:

where nodes 1-4 make up the "central network".

Now the nodes in the subnets (nodes with letter-labels) exchange traffic (UDP, ICMP) amongst each other.

What I observe is that UDP seems to be working fine. However ICMP in many cases does not actually reach their target, rather I can see in the pcap output that the gateway node returns an ICMP packet to the original sender with the statement 'TTL exceeded in transit'.

This strikes me as odd, given how small the topology is, I don't really understand how the TTL could be consumed from source to target.

The way I construct the subnet topology is as follows:

// m_numNodes already includes the +1 for the AS gateway
nodes.Create(m_numNodes);

InternetStackHelper stack;
stack.Install(nodes);

// continuously merge device containers into one for easier address assignment
NetDeviceContainer mergedDeviceContainer;

// initialize helper to crate p2p connections
StringValue dataRate(m_config.GetBandwidth());
StringValue delay(m_config.GetDelay());

PointToPointHelper p2p;
p2p.SetDeviceAttribute("DataRate", dataRate);
p2p.SetChannelAttribute("Delay", delay);

// create p2p connection from gateway node to each other node subnet
for (int i = 1; i < m_numNodes; i++)
// excluding 0, since that is the index of the gateway node
{
NetDeviceContainer devices = p2p.Install(nodes.Get(0), nodes.Get(i));
mergedDeviceContainer.Add(devices);
}

// perform address assignments
// NOTE: address assignments happen on a per-NetDevice basis. Given that everytime we connect
// the gateway to a new node the gateway also gets a new NetDevice (think of it as a PCI slot
// or interface), it also "consumes" one address from the address space. This will result in all
// actually addressable nodes that will be used to generate traffic
// having an "even" ip address (e.g., 192.168.73.2, the next will have 192.168.73.4, etc.)
// whilst the gateway uses up all the "uneven" ip addresses (e.g., 192.168.73.1, 192.168.73.3,
// etc.)
Ipv4AddressHelper address;
Ipv4Address addressBase = m_config.GetNetworkAddress().c_str();
Ipv4Mask addressMask = m_config.GetNetworkMask().c_str();
address.SetBase(addressBase, addressMask);
// thanks to continuously merging the device containers, this is a single call
Ipv4InterfaceContainer interfaces = address.Assign(mergedDeviceContainer);

// write to class members
m_nodes = nodes;
m_mergedDevices = mergedDeviceContainer;
m_interfaces = interfaces;

When installing the applications and specifying the peers I am taking care of the fact that only every second address is actually a subnet node, due to the way I merge device containers.

Do you see anything wrong with the way I construct this? What could be a potential cause for all these ttl exceeded messages?

Is the "fix" to simply increase the ttl? If so, I don't really understand where that can be done, the code is somewhat intransparent to me in that regard.

Best regards & thanks a lot in advance for any hints or ideas,

Pascal