Generate traffic by using a real trace

[Peshal Nayak]

Is it possible to use a trace file to generate traffic in ns 3. For instance, if I have a trace file from a file download process, can I make ns-3 generate packets of the same size in the same order and same time separations. I know that ns had this feature (http://sydney.edu.au/engineering/it/~deveritt/networksimulation/trafficgeneration.html). I was wondering if it was present in ns-3 too.

Best,
Peshal

[Tommaso Pecorella]

Yes, there is. Just search for it :)

Cheers,

T.

[Peshal Nayak]

Hi Tommaso,

Could you please post any links to pages with the instructions or any other useful information?

Thanks.

Best,

Peshal

--
You received this message because you are subscribed to a topic in the Google Groups "ns-3-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ns-3-users/b5hAbL5tLZQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ns-3-users+...@googlegroups.com.
To post to this group, send email to ns-3-...@googlegroups.com.
Visit this group at http://groups.google.com/group/ns-3-users.
For more options, visit https://groups.google.com/d/optout.

[Tommaso Pecorella]

I said "search for it". Not "let me search it for you".

Anyway, since it seems that it's so hard to find...

https://www.nsnam.org/docs/doxygen/structns3_1_1_udp_trace_client.html

T.

[Peshal Nayak]

Hi Tommaso,

Looks like this can take in a MPEG4 trace file. My understanding is that MPEG4 is associated with multimedia video and audio. But can any random trace (such as the ones obtained from Wireshark for a general web browsing scenario) be given as an input. I've searched for that and haven't been able to find any such examples.

Also, ns 3 has an example code called udp-trace-client-server.cc which I believe is related to using a real trace file. However, I can't find a way of giving a real trace file as an input. The link you posted indicates that the filepath should lead to a MPEG4 trace file.

Thanks.

Best,
Peshal

[pdbarnes]

Hello Peshal,

> can any random trace (such as the ones
> obtained from Wireshark for a general web
> browsing scenario) be given as an input.

I've wondered the same thing. It ought to be pretty straightforward to create a traffic source which reads from PCAP and puts packets on a channel. Would the FdNetDevice be useful here?

A couple of issues will have to be addressed:

1. PCAP labels each packet with the real world time it was captured. These will have to be shifted to ns-3 virtual time then scheduled. The PCAP header records the time the capture started, so that might be a reasonable offset.

2. IIRC PCAP records time stamps to microsecond precision, while the default ns-3 resolution is ns. i can't think of any way to improve on this, but it could lead to differences in results which users should be aware of.

3. Packets in PCAP have all headers intact, many of which may have to be removed or replaced before injecting onto an ns-3 channel. For example, if you want the application content, but not any routing, TCP, IP or MAC (e.g. Ethernet) headers, those will have to be stripped off, then the packet injected at the appropriate level in the stack.

4. *When* packets are sent is often conditioned on packets received. For example, in TCP the transmission of a packet is in part a response to reception of ACKs to prior sent packets. Replaying from PCAP based solely on recorded time stamps won't be responsive in this way.

Sounds like a useful capability, if these issues can be addressed.

Peter

[Tommaso Pecorella]

Hi,

I'll try to get two birds with a stone.

Using Wireshark traces in ns-3 is possible (writing a NetDevice that reads a cap file is quite easy), but one should address the things that Peter mentioned. In particular all the headers up to the IP one will have to be changed (again, can be done), and the device will have a somewhat strange microseconds time synchronisation.

Anyway, it is possible, but... why ?

Pcap traces are "good" because they represent real traffic, but your PcapReplayApplication would have to just inject the data originating from one node, and even then one would have to carefully select what it's sent. E.g., no ARP packets, or RS/RA, NS/ND, and so on.

Moreover, there will always be room for strange things. For example, a packet may be delayed in the real system due to ARP, and in ns-3 too. However the cap registers the time AFTER the delay (when the packet has been sent on the wire), not the time the packet was passed to the IP layer. Weirdnesses.

If you decide to replay your packets at a lower layer (a PcapReplayNetDevice) you'd still have some issues. How to deal with the carrier sense ? You can't just inject the data on the wire blindly.

Summarizing: first one should clarify what are the goals and limitations of this replay system.

About the UdpTrace apps you found, they create a stream of data that is equivalent to the one that an app streaming an MPEG file would produce. I think you should re-read the docs and the source code.

The difference is not small. When you stream a video, there's a lot of transcoding, packetization, fragmentation and so on. The app uses a trace that "just" say when and how many data have been generated and are being feed to the UDP layer by an app that would stream a given video. As a side note, this trace is just a text file, and the docs point to the webpages where you can find such scripts and data.

This works for UDP, but it wouldn't work at all for adaptive streaming, since... well, it's too long to explain, but you can think to it.

It can be used for TCP too, but (again) the application is blind to the network conditions.

Last line: check the code. If you don't understand the docs, always read the code.

Hope this helps,

T.

[Peshal Nayak]

Thanks for your reply. I guess most of my confusion is cleared now. However, I've two questions and since they were related to the discussion we had above, I thought I should post them here rather than anywhere else. These questions might sound as very fundamental questions, but its really bugging me at the moment and I'm not able to find answers to them.

1. Why does the udpserver not have a mechanism to specify the data rate: rate at which it would sent the packets? Such as in case of onoff traffic sources. How do the time stamps for different packets get produced then?

2. Why does the client send data (https://www.nsnam.org/docs/doxygen/structns3_1_1_udp_trace_client.html#details). My understanding is that the server sends the data and the client should be at the receiving end.

Best,

Peshal

[pdbarnes]

> 2. Why does the client send data
>
(https://www.nsnam.org/docs/doxygen/structns3_1_1_udp_trace_client.html#details).
> My understanding is that the server sends the
> data and the client should be at the receiving
> end.

Think about TCP: the client initiates the connection by sending the SYN. The server responds with SYN/ACK. Notice that the client sends first, and the server is already listening. It's the same principle with UDP: the client sends, the server is already listening.

Peter

[Tommaso Pecorella]

On Friday, April 3, 2015 at 1:57:14 AM UTC+2, Peshal Nayak wrote:

1. Why does the udpserver not have a mechanism to specify the data rate: rate at which it would sent the packets? Such as in case of onoff traffic sources. How do the time stamps for different packets get produced then?

It's produced by the client. The server only receives data.

 

2. Why does the client send data (https://www.nsnam.org/docs/doxygen/structns3_1_1_udp_trace_client.html#details). My understanding is that the server sends the data and the client should be at the receiving end.

Nope, it's the opposite. The reason is historical: back the days a "server" was a piece of hardware / software with valuable resources and a client was a less resourceful device using the server to do stuff.

Over the time, the "server" concept was kept, but the end result is--- sometimes funny.

As an example, the X11 "server" is the video system, and it's receiving a data stream (the stuff to display), while an HTTP server is waiting for a request and then issuing a resource (a file).

Summarizing: the "server" status is not gained because you are sending more traffic, it's because you're the one standing still and reacting to the client's requests.

In the UdpServer case, the valuable resource is... get these packets, don't complain and calc some useful stats.

[Peshal Nayak]

If that were true, then the CLIENT should have a mechanism to specify data rate: rate at which it would sent the packets? Such as in case of onoff traffic sources. But it doesn't.
How do the time stamps for different packets get generated then?

Best,

Peshal

[Tommaso Pecorella]

Hem... it does have such an attribute. It's just not named DataRate. It is named "Interval".

• Interval: The time to wait between packets 
  • Set with class: ns3::TimeValue
  • Underlying type: Time –9223372036854775808.0ns:+9223372036854775807.0ns 
  • Initial value: +1000000000.0ns 
  • Flags: construct write read 

https://www.nsnam.org/docs/doxygen/classns3_1_1_udp_client.html

Cheers,

T.

[Peshal Nayak]

Agreed. But the interval is between packets. So it can be thought of as an equivalent of the off time (from on off traffic source). But what about the on time (time when it generates packets). The interval tells you the minimum difference between two consecutive packet time stamps, but nothing about the timestamps themselves. That is dependent on the data rate.

Best,

Peshal

[Tommaso Pecorella]

I'm sorry but I'm lost here. Either your thinking is too deep or there's a language problem. I'll try to use simple words.

1) Packet is generated - attach a timestamp

2) Wait "Interval" time

3) Goto 1

This is the UdpClient way.

Read the code, I just removed the log functions.

void
UdpClient::Send (void)
{
  NS_LOG_FUNCTION (this);
  NS_ASSERT (m_sendEvent.IsExpired ());
  SeqTsHeader seqTs;
  seqTs.SetSeq (m_sent);
  Ptr<Packet> p = Create<Packet> (m_size-(8+4)); // 8+4 : the size of the seqTs header
  p->AddHeader (seqTs);

  std::stringstream peerAddressStringStream;
  if (Ipv4Address::IsMatchingType (m_peerAddress))
    {
      peerAddressStringStream << Ipv4Address::ConvertFrom (m_peerAddress);
    }
  else if (Ipv6Address::IsMatchingType (m_peerAddress))
    {
      peerAddressStringStream << Ipv6Address::ConvertFrom (m_peerAddress);
    }

  if ((m_socket->Send (p)) >= 0)
    {
      ++m_sent;
    }

  if (m_sent < m_count)
    {
      m_sendEvent = Simulator::Schedule (m_interval, &UdpClient::Send, this);
    }
}

Got it now ?

As a side note: from what you said, you didn't understand at all how the OnOff application works.

Cheers,

T.

[Peshal Nayak]

How do you get the timestamps in step 1? Is the difference between two timestamps equal to the interval time? Wouldn't that make the peak rate of your traffic generator infinite, because your packets would get generated as pulses. In the pic attached, the first figure is what the timeline should look like if the packets were generated as pulses. But second one is what I expect the time line too look like. t1, t2, and so on are the timestamps. My understanding is that:

t2 - t1  = Interval time + (packet_size/data_rate)

Thanks,

Best,

Peshal

[Tommaso Pecorella]

"Look at me still talking when there's Science to do" [GLaDOS]

This is a wikipedia page about Discrete Event simulations. Please, read it.

In the picture you have different things.

At the top you have an application and its data generation events. That's exactly how the UdpClient works.

At the bottom you have the PHY channel, with the data transmissions. You have TWO different event types; TxStart and TxEnd.

The CHANNEL bitrate will tell you how much "time" is needed to transmit a given number of bits, This, the time from a TxStart to a TxEnd.

The APPLICATION data rate will tell you the interval between two packet generation (assuming you know that the packets have the same number of bits).

The APPLICATION only needs ONE event and ONE interval time.

The application data rate is not infinite because each packet has a finite number of bits. After each interval a given number of bits are generated instantaneously.

You can think to it like "the application did buffer the bits 'til it has enough to fill a buffer. at that point it calls an API".

If you're not convinced, think to a socket. From the UDP point of view, the application is generating a certain number of bytes instantaneously.

The OnOff application is different, because it has a two-state status, in the ON state it will transmit exactly like the UdpClient. In the Off state it will transmit nothing.

About the timestamp, it is "just" the time at which the packet was generated. Nothing more.

When you receive the packet, the difference between the rx time and the timestamp is the time needed to:

- Queueing delay (if needed).

- Acquire the channel (CSMA, collisions, etc.).

- Tx the packet over the air.

- Propagation delay.

- More stuff I can not think of.

You are using what you already know (past knowledge) to understand the new things (ns-3). The problem is: the past knowledge is not helping at all.

Be more open-minded. After all, if your past knowledge would be enough to learn the new things, you'd not need to learn them. Right ?

Have fun,

T.