DPDK with TRex Docker container

bre...@versa-networks.com

unread,

May 7, 2018, 4:21:57 PM5/7/18

to TRex Traffic Generator

Is it possible to setup a container with vhost-user so that TRex can run in a container and still get good performance? Reason I ask is my host OS is not working with ASTF and I wanted an environment for TRex to run and get DPDK performance.

Thanks

Brendan Kelly

unread,

May 7, 2018, 11:28:21 PM5/7/18

to TRex Traffic Generator

Ok this works fine. I now have astf in a docker container.

I used this information to build my container.

https://github.com/redhat-performance/docker-dpdk/blob/master/Dockerfile

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_atomic_host/7/html/managing_containers/running_super_privileged_containers

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.
To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/dbc553bd-f5e5-430d-a5d6-3525cc336287%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

hanoh haim

unread,

May 8, 2018, 12:52:29 AM5/8/18

to Brendan Kelly, TRex Traffic Generator

Hi Brendan,

Is it related to the python ASTF error you got?

Thanks,

Hanoh

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/8C7B67DA-1A74-4BA0-9BC9-86E5B293E635%40versa-networks.com.

For more options, visit https://groups.google.com/d/optout.

--

Hanoh

Sent from my iPhone

Brendan Kelly

unread,

May 8, 2018, 7:26:29 AM5/8/18

to hanoh haim, TRex Traffic Generator

Yes, I have OSP 11 and getting my python sorted out wasn’t looking good.

Brendan Kelly

unread,

May 9, 2018, 9:37:26 AM5/9/18

to hanoh haim, TRex Traffic Generator

FYI.

I see these errors when running trex in this setup. Seems to not affect the functionality.

[root@trex v2.40]# sudo ./t-rex-64 -f astf/rtp-office.py -m 1 -d 10000000 -c 1 --astf

lspci: Unable to load libkmod resources: error -12

converting astf profile astf/rtp-office.py to json /tmp/astf.json

./astf-sim -f astf/rtp-office.py --json > /tmp/astf.json

The ports are bound/configured.

lspci: Unable to load libkmod resources: error -12

Brendan Kelly

303-513-0846

Sales Engineer

.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/B9487DCA-4277-4BF4-85D5-0E313A1C8E00%40versa-networks.com.
For more options, visit https://groups.google.com/d/optout.

Itay Marom

unread,

May 9, 2018, 9:41:18 AM5/9/18

to Brendan Kelly, hanoh haim, TRex Traffic Generator

hi,

I think we already fixed that issue.

it means that lspci failed to obtain information because the /lib/modules directory is not mounted in docker guest.

let me check this out...

itay

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/53EFC044-67CB-4809-A0E2-2C658C34F4ED%40versa-networks.com.

Yaroslav Brustinov

unread,

May 9, 2018, 9:45:00 AM5/9/18

to Itay Marom, Brendan Kelly, hanoh haim, TRex Traffic Generator

Commit ID: 3a49ccdbe359ee845e59d4eb239407ae467760db

Supposed to be fixed in v2.38

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to

trex...@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/trex-tgn/dbc553bd-f5e5-430d-a5d6-3525cc336287%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to

trex...@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/trex-tgn/8C7B67DA-1A74-4BA0-9BC9-86E5B293E635%40versa-networks.com.
For more options, visit https://groups.google.com/d/optout.

--

Hanoh

Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to

trex...@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/trex-tgn/B9487DCA-4277-4BF4-85D5-0E313A1C8E00%40versa-networks.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/53EFC044-67CB-4809-A0E2-2C658C34F4ED%40versa-networks.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CAPXuVffRrBeMjXvZu3nLVf7N8qw_Dr0M4AvAHNj7%3DtH1KMEZmw%40mail.gmail.com.

Itay Marom

unread,

May 9, 2018, 9:52:26 AM5/9/18

to Yaroslav Brustinov, Brendan Kelly, hanoh haim, TRex Traffic Generator

What is the container Linux distribution ?

is it based on the redhat docker file ?

Maybe the module directory is mounted elsewhere...

and lspci checks for something else...

Itay

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to

trex...@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/trex-tgn/dbc553bd-f5e5-430d-a5d6-3525cc336287%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to

trex...@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/trex-tgn/8C7B67DA-1A74-4BA0-9BC9-86E5B293E635%40versa-networks.com.
For more options, visit https://groups.google.com/d/optout.

--

Hanoh

Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to

trex...@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/trex-tgn/B9487DCA-4277-4BF4-85D5-0E313A1C8E00%40versa-networks.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/53EFC044-67CB-4809-A0E2-2C658C34F4ED%40versa-networks.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+u...@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.

Yaroslav Brustinov

unread,

May 9, 2018, 10:01:23 AM5/9/18

to Itay Marom, Brendan Kelly, hanoh haim, TRex Traffic Generator

@Itay

By the way, what if we hide the output of lspci command unless there is bad return status?

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to

trex...@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/trex-tgn/dbc553bd-f5e5-430d-a5d6-3525cc336287%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to

trex...@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/trex-tgn/8C7B67DA-1A74-4BA0-9BC9-86E5B293E635%40versa-networks.com.
For more options, visit https://groups.google.com/d/optout.

--

Hanoh

Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to

trex...@googlegroups.com.

To view this discussion on the web visit

https://groups.google.com/d/msgid/trex-tgn/B9487DCA-4277-4BF4-85D5-0E313A1C8E00%40versa-networks.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/53EFC044-67CB-4809-A0E2-2C658C34F4ED%40versa-networks.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CAPXuVffRrBeMjXvZu3nLVf7N8qw_Dr0M4AvAHNj7%3DtH1KMEZmw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "TRex Traffic Generator" group.

To unsubscribe from this group and stop receiving emails from it, send an email to trex-tgn+unsubscribe@googlegroups.com.

To post to this group, send email to trex...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/trex-tgn/CAPXuVff3ufYET0h_8h%2BxeikDAzoPJOFfNEMf7MN3RGT-y2uguA%40mail.gmail.com.

Yaroslav Brustinov

unread,

May 9, 2018, 10:20:25 AM5/9/18

to Itay Marom, Brendan Kelly, hanoh haim, TRex Traffic Generator

(I mean the stderr)

Brendan Kelly

unread,

May 9, 2018, 11:34:10 AM5/9/18

to Itay Marom, Yaroslav Brustinov, hanoh haim, TRex Traffic Generator

Its RHEL atomic based on the docker file in the link from github below.

Docker container.

[root@trex ~]# cd /lib/modules

[root@trex modules]# ls

3.10.0-862.el7.x86_64

[root@trex modules]#

[root@rh2 ~]# docker inspsect trex

docker: 'inspsect' is not a docker command.

See 'docker --help'

[root@rh2 ~]# docker inspect trex

[

{

"Id": "89a7584d604c4a62ebc4cbff0a229342ea70470fe774672a5d7065e539c3241f",

"Created": "2018-05-08T13:14:08.296900684Z",

"Path": "/bin/bash",

"Args": [],

"State": {

"Status": "running",

"Running": true,

"Paused": false,

"Restarting": false,

"OOMKilled": false,

"Dead": false,

"Pid": 4091,

"ExitCode": 0,

"Error": "",

"StartedAt": "2018-05-08T13:14:08.629077559Z",

"FinishedAt": "0001-01-01T00:00:00Z"

},

"Image": "sha256:73cf80e565fca97ae215a6cf635d649fb8a5b34e09f756b2024f2a2d25744f22",

"ResolvConfPath": "/var/lib/docker/containers/89a7584d604c4a62ebc4cbff0a229342ea70470fe774672a5d7065e539c3241f/resolv.conf",

"HostnamePath": "/var/lib/docker/containers/89a7584d604c4a62ebc4cbff0a229342ea70470fe774672a5d7065e539c3241f/hostname",

"HostsPath": "/var/lib/docker/containers/89a7584d604c4a62ebc4cbff0a229342ea70470fe774672a5d7065e539c3241f/hosts",

"LogPath": "",

"Name": "/trex",

"RestartCount": 0,

"Driver": "overlay2",

"MountLabel": "",

"ProcessLabel": "",

"AppArmorProfile": "",

"ExecIDs": [

"f47585b1e41d1615f1e5a58227f0aa1ddf0b109076c72c761dbd6ddc8c56e224",

"7c06e1b4bc17534280e290cb33b9ea349bdfc9a7934ca23d6ca83ebb4e77694a"

],

"HostConfig": {

"Binds": [

"/root/trex:/root/trex",

"/sys/bus/pci/drivers:/sys/bus/pci/drivers",

"/sys/kernel/mm/hugepages:/sys/kernel/mm/hugepages",

"/sys/devices/system/node:/sys/devices/system/node",

"/dev:/dev"

],

"ContainerIDFile": "",

"LogConfig": {

"Type": "journald",

"Config": {}

},

"NetworkMode": "default",

"PortBindings": {},

"RestartPolicy": {

"Name": "always",

"MaximumRetryCount": 0

},

"AutoRemove": false,

"VolumeDriver": "",

"VolumesFrom": null,

"CapAdd": null,

"CapDrop": null,

"Dns": [],

"DnsOptions": [],

"DnsSearch": [],

"ExtraHosts": null,

"GroupAdd": null,

"IpcMode": "",

"Cgroup": "",

"Links": null,

"OomScoreAdj": 0,

"PidMode": "",

"Privileged": true,

"PublishAllPorts": false,

"ReadonlyRootfs": false,

"SecurityOpt": [

"label=disable"

],

"UTSMode": "",

"UsernsMode": "",

"ShmSize": 67108864,

"Runtime": "docker-runc",

"ConsoleSize": [

0,

0

],

"Isolation": "",

"CpuShares": 0,

"Memory": 0,

"NanoCpus": 0,

"CgroupParent": "",

"BlkioWeight": 0,

"BlkioWeightDevice": null,

"BlkioDeviceReadBps": null,

"BlkioDeviceWriteBps": null,

"BlkioDeviceReadIOps": null,

"BlkioDeviceWriteIOps": null,

"CpuPeriod": 0,

"CpuQuota": 0,

"CpuRealtimePeriod": 0,

"CpuRealtimeRuntime": 0,

"CpusetCpus": "",

"CpusetMems": "",

"Devices": [],

"DiskQuota": 0,

"KernelMemory": 0,

"MemoryReservation": 0,

"MemorySwap": 0,

"MemorySwappiness": -1,

"OomKillDisable": false,

"PidsLimit": 0,

"Ulimits": null,

"CpuCount": 0,

"CpuPercent": 0,

"IOMaximumIOps": 0,

"IOMaximumBandwidth": 0

},

"GraphDriver": {

"Name": "overlay2",

"Data": {

"LowerDir": "/var/lib/docker/overlay2/73aafc2706313ea5027ee7ff667bce58a6b310dfe90e3ad15bc35574b956e6fd-init/diff:/var/lib/docker/overlay2/ebe4fcdbc4af2162e2e97905adefdf36fb587c3e3880d7fec4a59c7fb90ca4f1/diff:/var/lib/docker/overlay2/28decb2feed1ac844fb5abb29448e718d9a66972bb92984dc2ccec72c4f807ae/diff:/var/lib/docker/overlay2/667df7fbe95d0a761fd17ad2a13c3d3d86291da4ccf515082af9173e362eaf1f/diff:/var/lib/docker/overlay2/ec4213c716b0a04d1ff7c9dafd11e86da8b0970689b096dcc0deb7fa614806ac/diff:/var/lib/docker/overlay2/70a79c127fee21d99ef16547c735b96ab8c18972453db8193843f3c63dd0f17a/diff:/var/lib/docker/overlay2/cc0a2062ef3cd7f7582b5a3f2fa7906fbea383f482a4c6c8c4e84247a9e152f3/diff:/var/lib/docker/overlay2/cc1473fac771a978f31a6d0b109c4a24e3391e0b1eabf693d8f0be06b79a0919/diff:/var/lib/docker/overlay2/2c8607baeb7432a105b5997f1a28ae8a74f8df7814520f0639f90d6a3346a1af/diff:/var/lib/docker/overlay2/e08aae25ec9ed8069649e05343b7fb5295667ed900e23465343558d644978f93/diff",

"MergedDir": "/var/lib/docker/overlay2/73aafc2706313ea5027ee7ff667bce58a6b310dfe90e3ad15bc35574b956e6fd/merged",

"UpperDir": "/var/lib/docker/overlay2/73aafc2706313ea5027ee7ff667bce58a6b310dfe90e3ad15bc35574b956e6fd/diff",

"WorkDir": "/var/lib/docker/overlay2/73aafc2706313ea5027ee7ff667bce58a6b310dfe90e3ad15bc35574b956e6fd/work"

}

},

"Mounts": [

{

"Type": "bind",

"Source": "/root/trex",

"Destination": "/root/trex",

"Mode": "",

"RW": true,

"Propagation": "rprivate"

},

{

"Type": "bind",

"Source": "/sys/bus/pci/drivers",

"Destination": "/sys/bus/pci/drivers",

"Mode": "",

"RW": true,

"Propagation": "rprivate"

},

{

"Type": "bind",

"Source": "/sys/kernel/mm/hugepages",

"Destination": "/sys/kernel/mm/hugepages",

"Mode": "",

"RW": true,

"Propagation": "rprivate"

},

{

"Type": "bind",

"Source": "/sys/devices/system/node",

"Destination": "/sys/devices/system/node",

"Mode": "",

"RW": true,

"Propagation": "rprivate"

},

{

"Type": "bind",

"Source": "/dev",

"Destination": "/dev",

"Mode": "",

"RW": true,

"Propagation": "rprivate"

}

],

"Config": {

"Hostname": "trex",

"Domainname": "",

"User": "",

"AttachStdin": true,

"AttachStdout": true,

"AttachStderr": true,

"Tty": true,

"OpenStdin": true,

"StdinOnce": true,

"Env": [

"NAME=rhel-tools",

"IMAGE=rhel7/rhel-tools",

"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",

"container=oci"

],

"Cmd": [

"/bin/bash"

],

"Image": "trex",

"Volumes": null,

"WorkingDir": "/root",

"Entrypoint": null,

"OnBuild": null,

"Labels": {

"License": "GPLv3",

"RUN": "docker run -it --privileged -v /sys/bus/pci/drivers:/sys/bus/pci/drivers -v /sys/kernel/mm/hugepages:/sys/kernel/mm/hugepages -v /sys/devices/system/node:/sys/devices/system/node -v /dev:/dev --name NAME -e NAME=rhel-tools -e IMAGE=rhel7/rhel-tools IMAGE",

"architecture": "x86_64",

"authoritative-source-url": "registry.access.redhat.com",

"build-date": "2018-04-05T12:45:28.018044",

"com.redhat.build-host": "osbs-cpt-011.ocp.osbs.upshift.eng.rdu2.redhat.com",

"com.redhat.component": "rhel-tools-docker",

"description": "The Red Hat Enterprise Linux Atomic Tools Container (RHEL Tools Container) is a docker-formatted image that includes hundreds of software tools for troubleshooting and investigating a Red Hat Enterprise Linux Atomic (RHEL Atomic) Host. Designed to run as a privileged container, the RHEL Tools Container allows you to interact directly with the RHEL Atomic Host system to uncover and solve problems. Inside the RHEL Tools Container are popular tools such as sosreport, kdump, and many others (most of which are not included with RHEL Atomic).",

"distribution-scope": "public",

"io.k8s.description": "The Red Hat Enterprise Linux Atomic Tools Container (RHEL Tools Container) is a docker-formatted image that includes hundreds of software tools for troubleshooting and investigating a Red Hat Enterprise Linux Atomic (RHEL Atomic) Host. Designed to run as a privileged container, the RHEL Tools Container allows you to interact directly with the RHEL Atomic Host system to uncover and solve problems. Inside the RHEL Tools Container are popular tools such as sosreport, kdump, and many others (most of which are not included with RHEL Atomic).",

"io.k8s.display-name": "Red Hat Enterprise Linux Tools",

"io.openshift.tags": "base rhel7",

"name": "rhel7/rhel-tools",

"release": "5",

"run": "docker run -it --name NAME --privileged --ipc=host --net=host --pid=host -e HOST=/host -e NAME=NAME -e IMAGE=IMAGE -v /run:/run -v /var/log:/var/log -v /etc/machine-id:/etc/machine-id -v /etc/localtime:/etc/localtime -v /:/host IMAGE",

"summary": "Provides the latest release of Red Hat Enterprise Linux 7 in a fully featured and supported base image.",

"url": "https://access.redhat.com/containers/#/registry.access.redhat.com/rhel7/rhel-tools/images/7.5-5",

"vcs-ref": "ea719c7b547289bab6a697bf90453dc56e78a763",

"vcs-type": "git",

"vendor": "Red Hat, Inc.",

"version": "7.5"

}

},

"NetworkSettings": {

"Bridge": "",

"SandboxID": "1dc2373ca8adbff91a18359a51fbc99bb50c952ab30257845040d8742f046a2e",

"HairpinMode": false,

"LinkLocalIPv6Address": "",

"LinkLocalIPv6PrefixLen": 0,

"Ports": {},

"SandboxKey": "/var/run/docker/netns/1dc2373ca8ad",

"SecondaryIPAddresses": null,

"SecondaryIPv6Addresses": null,

"EndpointID": "c643884a1895bd31350e5996642615e883bfcbc2605164965018359450cb7250",

"Gateway": "172.17.0.1",

"GlobalIPv6Address": "",

"GlobalIPv6PrefixLen": 0,

"IPAddress": "172.17.0.2",

"IPPrefixLen": 16,

"IPv6Gateway": "",

"MacAddress": "02:42:ac:11:00:02",

"Networks": {

"bridge": {

"IPAMConfig": null,

"Links": null,

"Aliases": null,

"NetworkID": "fef74fc06ab191d6193f393e6ef56984da3987cd5dd08fb15bf2874809179c02",

"EndpointID": "c643884a1895bd31350e5996642615e883bfcbc2605164965018359450cb7250",

"Gateway": "172.17.0.1",

"IPAddress": "172.17.0.2",

"IPPrefixLen": 16,

"IPv6Gateway": "",

"GlobalIPv6Address": "",

"GlobalIPv6PrefixLen": 0,

"MacAddress": "02:42:ac:11:00:02"

}

]

Brendan Kelly

303-513-0846

Sales Engineer

Reply all

Reply to author

Forward