TRex traffic generation with more than 2 end point topology

[ead...@gmail.com]

Hi again,

How do I generate traffic in a 1 point to multi-point topology? Basically, I need a way to pump traffic from 3 or 4 end points.

For a 1 point to 1 point (total of 2 end points), I use (ipv6, astf, tcp) something like (in the template python file):

:
:
c_glob_info.ipv6.src_msb ="2600:dead:dead:1111::"
c_glob_info.ipv6.dst_msb ="2600:feed:feed:1111::"
:
:
return ASTFProfile(:
:
default_c_glob_info=c_glob_info,
:
)

Question:

Suppose the links are A-B, A-C, A-D, A-E. The above conventions of src and dst don't apply...
unless I can think of it as muItiple point-to-point links. But then, I am not allowed to spawn
multiple trex instances even from different folders, since ZMQ process of second instance cannot access the same port A when it is being accessed by the first instance.

So there is multiple IP end ranges for one IP start range on the side of A.

<ip_rangeC> C
/
/
/
<ip_rangeA> A -------- B <ip_rangeB>
\
\
\
<ip_rangeD> D

Any workarounds/suggestions?

Any pointers to sections in the existing documentation itself?

[ead...@gmail.com]

Just to clarify, I am intending to use 4 physical SFP ports, and indicate the details in the trex_cfg.yaml file

[hanoh haim]

Hi,

Could B,C,D have the same IPv6 MSB?

You can have multiple template with different range of LSB

Hanoh

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/244f6f39-9f0a-46b6-b71f-9d7fd0f4d8f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Hanoh

Sent from my iPhone

[ead...@gmail.com]

Hi Hanoh,

Thanks for the quick response again.
No - B, C, D cannot, in my case, have different MSBs.

Can we 'still' use TRex to push in traffic from 4 end points?

Can you please elaborate on the 'multiple template' part - did you mean multiple .py files each with a src, dst - accessed by the same trex instance?

[hanoh haim]

Hi,

You can have A send 3 templates, each template could have different pool of client/server that match B,C,D,E

then you can assign client cluster configuration file that will set different L2 information per client/server. this will make the packets go to B/C/D/E in L2 and L3

https://trex-tgn.cisco.com/trex/doc/trex_astf.html#_client_clustering_configuration

thanks,

Hanoh

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/eb2df07e-5272-4113-a9f5-85f42402fac6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[ead...@gmail.com]

Thanks for the info Hanoh.
I need a bit more granularity of info. Allow me to be a bit verbose in my understanding. Adding indexing bullet points for you to approve/refute.

1) The example astf/cc_http_simple_src_mac.yaml mentions of a way to specify /split range of ip addresses given in http_simple.py - But they are in the IPv4 world.

2) I need to use a IPv6 template, such as astf/param_ipv6.py. In which file will the 4 ranges of IP address msbs be specified?

http_simple.py uses a range (0-255 in 16.0.0.xxx)
ip_gen_c = ASTFIPGenDist(ip_range=["16.0.0.0", "16.0.0.255"], distribution="seq")
ip_gen_s = ASTFIPGenDist(ip_range=["48.0.0.0", "48.0.255.255"], distribution="seq")

which is split into sub ranges (0-127, 128-255) in its complimentary file cc_http_simple.py
- ip_start : 16.0.0.0
ip_end : 16.0.0.127
initiator :
vlan : 100
dst_mac : "00:00:00:02:00:00"
responder : # not USED !!!
vlan : 200
dst_mac : "00:00:00:01:00:00"

- ip_start : 16.0.0.128
ip_end : 16.0.0.255
initiator :
vlan : 101
dst_mac : "00:00:00:03:00:00"

responder : # not USED !!! learn by the server
vlan : 201
dst_mac : "01:00:00:00:02:01"

3) Above is how we split ipv4 range into 2 sub range of ipv4s.

4) The way we map ipv4 to ipv6 is using a template, when using just 2 endpoints, such as the one in param_ipv6.py, like this:
:
# ip generator
ip_gen_c = ASTFIPGenDist(ip_range=["16.0.0.0", "16.0.0.255"], distribution="seq")
ip_gen_s = ASTFIPGenDist(ip_range=["48.0.0.0", "48.0.255.255"], distribution="seq")
ip_gen = ASTFIPGen(glob=ASTFIPGenGlobal(ip_offset="1.0.0.0"),
dist_client=ip_gen_c,
dist_server=ip_gen_s)

:
c_glob_info.ipv6.src_msb ="2600:dead:dead:1111::"
c_glob_info.ipv6.dst_msb ="2600:feed:feed:1111::"

5) How do we "map" the 2 or 3 or 4 or 'n' sub-ranges in ipv4 to ipv6?
In other words, what would be my cc_param_ipv6.py look like??
In which file will I specify my 2 or 3 or 4 or 'n' IPv6 subnets/msbs ?

6) Not using vlans for now yet - so its 1 router per sfp port.

[hanoh haim]

1. You can add more than one pool and associate it with each template 

see this example (http_advance.py). Here you have two generators  ip_gen_c and  ip_gen_c2

    def create_profile(self):

        # program

        # Client program taken from client side of given file

        my_prog_c = ASTFProgram(file="../cap2/http_get.pcap", side="c")

        # Server program taken from server side of given file

        my_prog_s = ASTFProgram(file="../cap2/http_get.pcap", side="s")

        # ip generator

        ip_gen_c = ASTFIPGenDist(ip_range=["16.0.0.0", "16.0.0.1"], distribution="seq")

        ip_gen_s = ASTFIPGenDist(ip_range=["48.0.0.0", "48.0.0.1"], distribution="seq")

        ip_gen = ASTFIPGen(glob=ASTFIPGenGlobal(ip_offset="1.0.0.0"),

                           dist_client=ip_gen_c,

                           dist_server=ip_gen_s)

        ip_gen_c2 = ASTFIPGenDist(ip_range=["16.0.0.1", " 16.0.0.1"], distribution="seq")

        ip_gen_s2 = ASTFIPGenDist(ip_range=["48.0.0.1", "48.0.0.2"], distribution="seq")

        ip_gen2 = ASTFIPGen(glob=ASTFIPGenGlobal(ip_offset="1.0.0.0"),

                            dist_client=ip_gen_c2,

                            dist_server=ip_gen_s2)

        # template

        temp_c = ASTFTCPClientTemplate(program=my_prog_c,  ip_gen=ip_gen)

        temp_c2 = ASTFTCPClientTemplate(program=my_prog_c, ip_gen=ip_gen2, port=81)

        temp_s = ASTFTCPServerTemplate(program=my_prog_s,)  # using default association

        temp_s2 = ASTFTCPServerTemplate(program=my_prog_s, assoc=ASTFAssociationRule(port=81))

        template = ASTFTemplate(client_template=temp_c, server_template=temp_s)

        template2 = ASTFTemplate(client_template=temp_c2, server_template=temp_s2)

        # profile

        profile = ASTFProfile(default_ip_gen=ip_gen, templates=[template, template2])

2. you can add LSB IPv4 client range 

-    ip_start  : 16.0.0.1
     ip_end    : 16.0.0.2
     initiator :

                 dst_mac : "00:00:00:02:00:00"
     responder :  # not USED !!!

                 dst_mac : "00:00:00:01:00:00"  

The destination MAC for 16.0.0.1 client will be 00:00:00:02:00:00 

3. The IPv6 source/destination will be built by the same logic (IPv4 will be place at LSB)

suggest to play with the astf-sim first  

thanks,

Hanoh 

[ead...@gmail.com]

Thanks Hanoh.

It looks like I am more fundamentally blocked (points 1 and 2 below) in specifying the SFP ports, before I could try templates.

My topology is attached. The same thing, I have depicted below:

      |

eth3  |<------------------------

      |                        |

                               | (nic2)

                              DUT2

      |   (nic2)               /

eth2  |------> DUT1 ----------

      |                        \

                              DUT3

                               | (nic2)

      |                        |

eth4  |<------------------------

      |

My Objective:   Send true TCP (astf) traffic  from   eth2   to   eth3,   eth2   to   eth4. 

I understand from our prev conversations, that the way I should do it, is using templates within same trex instance, to create two  (src, dst) pairs of end points.

For traffic packets:

eth2 src msb ipv6:   cafe:cafe:cafe:85a0

eth3 src msb ipv6:   cafe:cafe:cafe:7843

eth4 src msb ipv6:   cafe:cafe:cafe:2461

The problems I see is (points 1 and 2 below):

1) If I mention 3 ports in /etc/trex_cfg.yaml with 3 pairs of src mac and dst mac, trex rejects it saying it needs 'even' number of ports.

let us call it   FILE1.yaml

  interfaces: ['02:00.0', '02:00.1', '02:00.2']  # DPDK mode - x86 eth2,eth3,eth4

  port_info:

      - dest_mac: 38:3a:21:b0:01:3d   # DUT1 nic2

        src_mac:  3c:fd:fe:a5:4a:58   # x86 eth2

      - dest_mac: 38:3a:21:b0:02:f8   # DUT2 nic2

        src_mac:  3c:fd:fe:a5:4a:59   # x86 eth3

      - dest_mac: 38:3a:21:b0:03:c1   # DUT3 nic2

        src_mac:  3c:fd:fe:a5:4a:5a   # x86 eth4

2) If I mention 4 ports in /etc/trex_cfg.yaml with 1 port "repeated", since its common 2 traffic flows, trex rejects it saying I am mixing udev and pdev interfaces

let us call it   FILE1.yaml

  interfaces: ['02:00.0', '02:00.1', '02.00.0', '02:00.2']  # DPDK mode - x86 eth2,eth3,eth2,eth4

  port_info:

      - dest_mac: 38:3a:21:b0:01:3d   # DUT1 nic2

        src_mac:  3c:fd:fe:a5:4a:58   # x86 eth2

      - dest_mac: 38:3a:21:b0:02:f8   # DUT2 nic2

        src_mac:  3c:fd:fe:a5:4a:59   # x86 eth3

      - dest_mac: 38:3a:21:b0:01:3d   # DUT1 nic2   --> REPEAT?

        src_mac:  3c:fd:fe:a5:4a:58   # x86 eth2    -->

      - dest_mac: 38:3a:21:b0:03:c1   # DUT3 nic2

        src_mac:  3c:fd:fe:a5:4a:5a   # x86 eth4

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 

3) I am hoping to use this as client cfg yaml file -  I specify the ipv4 client range splitting thus: 

let us call it FILE2.yaml

vlan: false

groups:

-    ip_start  : 16.0.0.1

     ip_end    : 16.0.0.1

     initiator :

                 dst_mac : "38:3a:21:b0:01:3d" # DUT1 nic2

                 src_mac : "3c:fd:fe:a5:4a:58" # x86 eth2

     responder :  # not USED !!!

                 dst_mac : "3c:fd:fe:a5:4a:59" # x86 eth3

     count     : 1

-    ip_start  : 16.0.0.2

     ip_end    : 16.0.0.2

     initiator :

                 dst_mac : "38:3a:21:b0:01:3d" # DUT1 nic2

                 src_mac : "3c:fd:fe:a5:4a:58" # x86 eth2

     responder : # not USED !!! learn by the server

                 dst_mac : "3c:fd:fe:a5:4a:5a" # x86 eth4

     count     : 1

4) ASTFGlobalInfo() has a data member for ipv6 allowing us to specify src and dst ipv6.

    But, referring to http_advanced.py, 

    ASTFTCPClientTemplate(),   ASTFTCPServerTemplate(), ASTFTemplate()  do not have a data member ipv6.

 

let us call my adaptation of http_advanced.py as FILE3.py

Q1) How do I assign src and dst pairs of ipv6 for the 2 flows to the objects of the above classes when use templates?

  

Q2) How does 16.0.0.1 get mapped to cafe:cafe:cafe:7843,  16.0.0.2 get mapped to cafe:cafe:cafe:2461, so-on?

    How can I specify something like

    template1:  src msb ipv6:   cafe:cafe:cafe:85a0

                dst msb ipv6:   cafe:cafe:cafe:7843

    template2:  src msb ipv6:   cafe:cafe:cafe:85a0

                dst msb ipv6:   cafe:cafe:cafe:2461

    

Appreciate your help again.

[hanoh haim]

I liked the hand drawing, I had in mind that eth1,2,3 located in different physical locations/servers.

With ASTF you will need to run 2 different TRex instances on the same server with dummy ports.

Instance 1(client):

eth1,dummy

Instance 2(server-side):

Dummy,eth2,dummy,eth3

It will be simpler (from installation point of view ) to have 2 physical servers.

 

Try with instance1 and validate that syn packets are send with the right ip/dest-mac

Try to play with ASTF simulator to understand how it works.

Thanks,

Hanoh

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/46ceca7d-3701-406c-a0d2-7ea70afb8faf%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[ead...@gmail.com]

Thanks Hanoh.

1) The baseline with which I made trex work on my setup uses 1 instance of trex which runs both server side and client side with just 2 DUTs (traffic end points) and 2 SFP ports.

    My objective is to achieve data injection at multiple endpoints from multiple SFP ports.

    For that, the first step I am intending to try is what you suggested, by testing if I can run 2 instances of trex in principle, to achieve the same results as the baseline, using 2 SFP ports.

- one trex instance for server (/opt/path_to_dut2_server/t-rex-64 --astf-server-only <and other options>)

- one trex instance for client (/opt/path_to_dut1_client/t-rex-64 <options>)

I have eth2, eth3, eth4, eth5 - 4 SFP ports of X710, all on the same PCI slot (1 hardware controller) on the same x86 server.

I have noticed that with v2.43 or later, I cannot simply grab 'as many' SFP ports by their PCI address, as needed.

So, I have to first unbind all 4 ports from i40e driver, even though I need only 2 ports for this proof of concept, and then dpdk grabs all 4 ports of the hardware controller.

I am on v2.45.

This was not the case with v2.41 - where I could just use ports corresponding to eth2, eth3 by their PCI address, and let eth4, eth5 remain in the kernel. 

This seemed more efficient use of ports as a resource since here, I can grab as many ports as needed.

This means, on the current trex version 2.45, to run 2 instances of trex to cater to - let us say - 2 setups for different purposes, it is not possible.

i.e.,  Mickey  cannot test  DUTx using 02:00.0, 02:00.1  while  Donald wants to test  DUTy using 02:00.2, 02:00.3 'at the same time'.

Is this an intentional design or a side effect introduced?

If yes, what is the advantage of this new behaviour of the DPDK driver?

- - - - - - - - - -

2) About the way you suggested - using dummy ports: Dummy ports aren't exactly "dummy". I can't make up a PCI address to make trex happy. 

Suppose I have 2 quad SFP ports - { eth2, eth3, eth4, eth5 }  and { eth6, eth7, eth8, eth9 } on 2 physical PCI slots.

Instance-1   can use eth2, and eth3 as dummy on the client side. 

Instance-2   can use eth6 as dummy, eth7,  eth8 as dummy, eth9 on the server side  [Here, I cannot use eth4, eth5 while the ZMQ in Instance-1 holds on to them for reasons stated in 1) ]

This would mean a redundancy factor of 50%. Isn't it?

[ead...@gmail.com]

An update:

It looks like it is just not that TRex is grabbing all ports on a particular physical PCI slot as I thought earlier.

Earlier, I used eth2 for DUT1 and eth3 for DUT2. 

Now, I moved DUT2 to eth9, which sits on a second X710 controller on different PCI slot.

Even now – I cannot launch 2 instances of TRex (from different folders) at all on the same x86 server [v2.45].  Is it allowed to have multiple TRex processes (here, they are orthogonal - if I may call so !)

 

- one trex instance for server (/opt/path_to_dut2_server/t-rex-64 --astf-server-only <and other options>)

- one trex instance for client (/opt/path_to_dut1_client/t-rex-64 <options>)

 

Server Side [DUT-1]: 

interfaces: ['03:00.2', '03:00.3']  # DPDK mode - x86 dummy-eth8,eth9

Client Side [DUT-2]:

interfaces: ['02:00.0', '02:00.1'] # DPDK mode - x86 eth2,dummy-eth3

 

I launched server side TRex instance first,

Next, when I try to launch client side TRex instance, I see the error:

ZMQ port is used by following process:

pid: 17389, cmd: ./_t-rex-64 -f astf/param_ipv6_oracle_pcap.py -d 600 --astf --astf-server-only --nc --cfg /etc/trex_cfg_server_dut2.yaml

ERROR encountered while configuring TRex system

 

[Yaroslav Brustinov]

Look in the FAQ section 1.1.12

--

You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/2d8d3807-849b-4e7d-b63a-fbc9597b931e%40googlegroups.com.

[ead...@gmail.com]

Dear Brustinov. Thank You.

I can run multiple instances of TRex now. Unblocked on that front. I misread the error to be of used physical port and not used zmq port.

I can do this so long, the trex instances are not sharing the same physical port.

So Mickey can indeed test  DUTx using 02:00.0, 02:00.1  while  Donald wants to test  DUTy using 02:00.2, 02:00.3 'at the same time'.

Coming Back to the Original Post (Goal):  

To send true Bidirectional TCP traffic from one end point to multiple endpoints (through a point to multipoint topology).

-------

      |

eth3  |<------------------------

      |                        | (nic2)

      |                       DUT2

      |   (nic2)               /

eth2  |------> DUT1 ----------

      |                        \

      |                       DUT3

      |                        | (nic2)

eth4  |<------------------------

      |

-------

If I have to explore the way described in http_advanced.py with cc_http_simple_src_mac.yaml, using 'templates', then

I can certainly split client clusters in the yaml file - but I can't map them to ipv6 in the template objects as multiple source-destination pairs.

Something like this:

client_obj1.ipv6.src:   cafe:cafe:cafe:85a0

client_obj1.ipv6.dst:   cafe:cafe:cafe:7843

client_obj2.ipv6.src:   cafe:cafe:cafe:85a0

client_obj2.ipv6.dst:   cafe:cafe:cafe:2461

client_obj3.ipv6.src:   cafe:cafe:cafe:85a0

client_obj3.ipv6.dst:     cafe:cafe:cafe:bf39

because ASTFProfile() takes one object default_c_glob_info of ASTFGlobalInfo() which has ipv6 data member [referring to astf/param_ipv6.py], 

with additional objects of ASTFTCPClientTemplate which don't have a way of specifying ipv6 [referring to astf/http_advanced.py]

[ASTFGlobalInfoPerTemplate() lacks an ipv6 class data member]

Question:  Is there a way out?  Or should I wait for such an ipv6 implementation to be released for these classes?

I can split these source and destination into separate trex instances itself, and avoid templates altogether.

But in that case, the partitioning (multiplexing) happens at hardware. 

I am unable to use DUT1 as client on zmq port 4000 and repeat DUT1 as client in a different trex instance on zmq port 5000. 

I hit a timeout error almost immediately - aborting both instances.

WATCHDOG: task 'Trex DP core 2' has not responded for more than 1.01593 seconds - timeout is 1 seconds

*** traceback follows ***

1       0x56134ceb951a ./_t-rex-64(+0x14651a) [0x56134ceb951a]

2       0x7effbbc03390 /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390) [0x7effbbc03390]

3       0x56134d0d956a rte_delay_us_block + 106

4       0x56134ce9b2f6 CCoreEthIF::send_burst(CCorePerPort*, unsigned short, CVirtualIFPerSideStats*) + 198

5       0x56134ce9b3df CCoreEthIF::flush_tx_queue() + 31

6       0x56134ce88084 CNodeGenerator::handle_maintenance(CFlowGenListPerThread*) + 100

7       0x56134ce897df CNodeGenerator::handle_flow_sync(CGenNode*, CFlowGenListPerThread*, bool&) + 63

8       0x56134ce89b18 CNodeGenerator::handle_slow_messages(unsigned char, CGenNode*, CFlowGenListPerThread*, bool) + 184

9       0x56134ce56d2f int CNodeGenerator::flush_file_realtime<24, false>(double, double, CFlowGenListPerThread*, double&) + 1103

10      0x56134d06b3d0 TrexDpCoreAstfBatch::start_astf() + 1072

11      0x56134d06b879 TrexDpCoreAstfBatch::start_scheduler() + 9

12      0x56134cff2989 TrexDpCore::start() + 153

13      0x56134ce7f873 CFlowGenListPerThread::start(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, CPreviewMode&) + 115

14      0x56134cea45e6 CGlobalTRex::run_in_core(unsigned char) + 326

15      0x56134ceac9ed ./_t-rex-64(+0x1399ed) [0x56134ceac9ed]

16      0x56134d0ec0ed eal_thread_loop + 477

17      0x7effbbbf96ba /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7effbbbf96ba]

18      0x7effbac7041d clone + 109

That leaves me with the only alternative of using VMs to share hardware and create independent point-to-point traffic flows.

VM1: DUT1-DUT2

VM2: DUT1-DUT3

VM3: DUT1-DUT4, so-on