Strange Chelsio performance (T6225-CR)

0 views
Skip to first unread message

mike tancsa

unread,
Dec 1, 2025, 1:02:14 PM (4 days ago) Dec 1
to FreeBSD Questions
TL;DR. I can get 10G coming in and out on port 0, Port 1 can only take
traffic at ~ 2.5 Gb/s incoming, but can send out a full 10G.

I have a t6 2 port NIC on a FreeBSD 13 stable server that gets a lot of
backups coming into it via ssh and zfs streams (zrepl).  At this point,
I swapped out the NIC, and motherboard and even tried a second NIC but
the same strange 2.5Gb/s incoming results using iperf3 as a test.  If I
run iperf3 -c -P5 I get a bit better performance, but still not 10G.  On
port 0, its solid 10G in both directions, no dropped packets etc.

port 0 is on a quanta switch
port 1 is a cisco 3850 switch with 4 10G ports. I setup 2 other test
boxes and on the test boxes I can get a full 10G on both ports in a test
environ through the cisco using the same nics and Supermicro MB (H12SSL
and H11SSL). When the H12SSL was my test board, all was good in my
tests. But when I swapped it for the H11SSL that was showing problems, I
had the same issue when I made it live. I have tried different GBICs as
well. Again, the GBICs in the test environment work great in the same
class T6 card

in /boot/loader.conf I have


hw.cxgbe.rdmacaps_allowed="0"
hw.cxgbe.iscsicaps_allowed="0"
hw.cxgbe.fcoecaps_allowed="0"
hw.cxgbe.pause_settings="0"
hw.cxgbe.attack_filter="1"
hw.cxgbe.drop_pkts_with_l3_errors="1"

hw.cxgbe.largest_rx_cluster="65536"     # 64KB max buffers
hw.cxgbe.safest_rx_cluster="16384"      # 16KB default buffers
hw.cxgbe.qsize_rxq="2048"               # Double RX queue depth
hw.cxgbe.buffer_packing="1"             # Enable buffer packing
hw.cxgbe.fl_pack="1"                    # Pack multiple frames per buffer

but with all the defaults its the same performance.

 sysctl -A dev.cc | egrep "flo|tru"
dev.cc.1.stats.rx_trunc3: 0
dev.cc.1.stats.rx_trunc2: 2
dev.cc.1.stats.rx_trunc1: 0
dev.cc.1.stats.rx_trunc0: 0
dev.cc.1.stats.rx_ovflow3: 0
dev.cc.1.stats.rx_ovflow2: 69
dev.cc.1.stats.rx_ovflow1: 0
dev.cc.1.stats.rx_ovflow0: 0
dev.cc.1.rsrv_noflowq: 0
dev.cc.0.stats.rx_trunc3: 0
dev.cc.0.stats.rx_trunc2: 0
dev.cc.0.stats.rx_trunc1: 0
dev.cc.0.stats.rx_trunc0: 0
dev.cc.0.stats.rx_ovflow3: 0
dev.cc.0.stats.rx_ovflow2: 0
dev.cc.0.stats.rx_ovflow1: 0
dev.cc.0.stats.rx_ovflow0: 0
dev.cc.0.rsrv_noflowq: 0

 pciconf -lvcb t6nex0
t6nex0@pci0:193:0:4:    class=0x020000 rev=0x00 hdr=0x00 vendor=0x1425
device=0x6401 subvendor=0x1425 subdevice=0x0000
    vendor     = 'Chelsio Communications Inc'
    device     = 'T6225-CR Unified Wire Ethernet Controller'
    class      = network
    subclass   = ethernet
    bar   [10] = type Memory, range 64, base 0xb7300000, size 524288,
enabled
    bar   [18] = type Memory, range 64, base 0xb6000000, size 16777216,
enabled
    bar   [20] = type Memory, range 64, base 0xb7984000, size 8192, enabled
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 32 messages, 64 bit, vector masks
    cap 10[70] = PCI-Express 2 endpoint max data 512(2048) FLR RO NS
                 max read 4096
                 link x8(x8) speed 8.0(8.0)
    cap 11[b0] = MSI-X supports 256 messages, enabled
                 Table in map 0x20[0x0], PBA in map 0x20[0x1000]
    cap 03[d0] = VPD
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 1 corrected
    ecap 0003[168] = Serial 1 0000000000000000
    ecap 000e[188] = ARI 1
    ecap 0010[1c8] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI
disabled
                     0 VFs configured out of 0 supported
                     First VF RID Offset 0x0008, VF RID Stride 0x0004
                     VF Device ID 0x6801
                     Page Sizes: 4096 (enabled), 8192, 65536, 262144,
1048576, 4194304
    ecap 0017[208] = TPH Requester 1

doing an iperf -c -P4 to the problem NIC, I can get a bit more throughput

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.84 GBytes  1.58 Gbits/sec    0    sender
[  5]   0.00-10.01  sec  1.84 GBytes  1.58 Gbits/sec     receiver
[  7]   0.00-10.00  sec  1.79 GBytes  1.54 Gbits/sec    0    sender
[  7]   0.00-10.01  sec  1.79 GBytes  1.54 Gbits/sec     receiver
[  9]   0.00-10.00  sec  1.72 GBytes  1.47 Gbits/sec    0    sender
[  9]   0.00-10.01  sec  1.72 GBytes  1.47 Gbits/sec     receiver
[ 11]   0.00-10.00  sec  1.79 GBytes  1.54 Gbits/sec    0    sender
[ 11]   0.00-10.01  sec  1.79 GBytes  1.54 Gbits/sec     receiver
[SUM]   0.00-10.00  sec  7.14 GBytes  6.13 Gbits/sec    0    sender
[SUM]   0.00-10.01  sec  7.14 GBytes  6.12 Gbits/sec     receiver

otherwise its
0{coldstorage2}# iperf3  -c 192.168.13.254
Connecting to host 192.168.13.254, port 5201
[  5] local 192.168.13.230 port 38539 connected to 192.168.13.254 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   284 MBytes  2.38 Gbits/sec    0    915 KBytes
[  5]   1.00-2.00   sec   339 MBytes  2.83 Gbits/sec    0   1.07 MBytes
[  5]   2.00-3.01   sec   300 MBytes  2.49 Gbits/sec    0   1.29 MBytes
[  5]   3.01-4.00   sec   303 MBytes  2.58 Gbits/sec    0   1.59 MBytes
[  5]   4.00-5.00   sec   289 MBytes  2.43 Gbits/sec    0   1.59 MBytes
[  5]   5.00-6.00   sec   309 MBytes  2.59 Gbits/sec    0   1.59 MBytes
[  5]   6.00-7.00   sec   288 MBytes  2.41 Gbits/sec    0   1.59 MBytes
[  5]   7.00-8.00   sec   294 MBytes  2.46 Gbits/sec    0   1.59 MBytes
[  5]   8.00-9.00   sec   288 MBytes  2.42 Gbits/sec    0   1.59 MBytes
[  5]   9.00-10.00  sec   287 MBytes  2.41 Gbits/sec    0   1.59 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.91 GBytes  2.50 Gbits/sec    0    sender
[  5]   0.00-10.00  sec  2.91 GBytes  2.50 Gbits/sec     receiver

iperf Done.
0{coldstorage2}#

Asking the LLMs they think resources are not allocated to the ports the
same way which I am a bit suspicious of.  This is RELENG_13
hw.cxgbe.tx_coalesce_gap: 5
hw.cxgbe.tx_coalesce_pkts: 32
hw.cxgbe.tx_coalesce: 1
hw.cxgbe.defrags: 0
hw.cxgbe.pullups: 5
hw.cxgbe.lro_mbufs: 0
hw.cxgbe.lro_entries: 8
hw.cxgbe.tscale: 1
hw.cxgbe.safest_rx_cluster: 16384
hw.cxgbe.largest_rx_cluster: 16384
hw.cxgbe.fl_pack: 1
hw.cxgbe.buffer_packing: 1
hw.cxgbe.ofld_cong_drop: 0
hw.cxgbe.cong_drop: 0
hw.cxgbe.spg_len: 64
hw.cxgbe.fl_pad: -1
hw.cxgbe.fl_pktshift: 0
hw.cxgbe.nm_txcsum: 0
hw.cxgbe.nm_split_rss: 0
hw.cxgbe.lazy_tx_credit_flush: 1
hw.cxgbe.starve_fl: 0
hw.cxgbe.nm_cong_drop: 1
hw.cxgbe.nm_holdoff_tmr_idx: 2
hw.cxgbe.nm_rx_nframes: 64
hw.cxgbe.nm_rx_ndesc: 256
hw.cxgbe.nm_black_hole: 0
hw.cxgbe.tls.combo_wrs: 0
hw.cxgbe.tls.inline_keys: 0
hw.cxgbe.kern_tls: 0
hw.cxgbe.cop_managed_offloading: 0
hw.cxgbe.drop_pkts_with_l4_errors: 0
hw.cxgbe.drop_pkts_with_l3_errors: 1
hw.cxgbe.drop_pkts_with_l2_errors: 1
hw.cxgbe.drop_ip_fragments: 0
hw.cxgbe.attack_filter: 1
hw.cxgbe.tx_vm_wr: 0
hw.cxgbe.clock_gate_on_suspend: 0
hw.cxgbe.reset_on_fatal_err: 0
hw.cxgbe.panic_on_fatal_err: 0
hw.cxgbe.pcie_relaxed_ordering: 1
hw.cxgbe.num_vis: 1
hw.cxgbe.doorbells_allowed: 15
hw.cxgbe.fcoecaps_allowed: 0
hw.cxgbe.iscsicaps_allowed: 0
hw.cxgbe.cryptocaps_allowed: -1
hw.cxgbe.rdmacaps_allowed: 0
hw.cxgbe.toecaps_allowed: 1
hw.cxgbe.niccaps_allowed: 33
hw.cxgbe.switchcaps_allowed: 3
hw.cxgbe.linkcaps_allowed: 0
hw.cxgbe.nbmcaps_allowed: 0
hw.cxgbe.fw_install: 1
hw.cxgbe.autoneg: -1
hw.cxgbe.force_fec: -1
hw.cxgbe.fec: -1
hw.cxgbe.pause_settings: 0
hw.cxgbe.config_file: default
hw.cxgbe.interrupt_types: 7
hw.cxgbe.qsize_rxq: 2048
hw.cxgbe.qsize_txq: 1024
hw.cxgbe.holdoff_pktc_idx: -1
hw.cxgbe.holdoff_timer_idx: 1
hw.cxgbe.nnmrxq_vi: 2
hw.cxgbe.nnmtxq_vi: 2
hw.cxgbe.nnmrxq: 8
hw.cxgbe.nnmtxq: 8
hw.cxgbe.native_netmap: 2
hw.cxgbe.holdoff_pktc_idx_ofld: -1
hw.cxgbe.holdoff_timer_idx_ofld: 1
hw.cxgbe.nofldrxq_vi: 1
hw.cxgbe.nofldtxq_vi: 1
hw.cxgbe.nofldrxq: 2
hw.cxgbe.nofldtxq: 8
hw.cxgbe.rsrv_noflowq: 0
hw.cxgbe.nrxq_vi: 1
hw.cxgbe.ntxq_vi: 1
hw.cxgbe.nrxq: 8
hw.cxgbe.ntxq: 16
hw.cxgbe.toe.tls_rx_timeout: 5
hw.cxgbe.toe.rexmt_backoff.15: -1
hw.cxgbe.toe.rexmt_backoff.14: -1
hw.cxgbe.toe.rexmt_backoff.13: -1
hw.cxgbe.toe.rexmt_backoff.12: -1
hw.cxgbe.toe.rexmt_backoff.11: -1
hw.cxgbe.toe.rexmt_backoff.10: -1
hw.cxgbe.toe.rexmt_backoff.9: -1
hw.cxgbe.toe.rexmt_backoff.8: -1
hw.cxgbe.toe.rexmt_backoff.7: -1
hw.cxgbe.toe.rexmt_backoff.6: -1
hw.cxgbe.toe.rexmt_backoff.5: -1
hw.cxgbe.toe.rexmt_backoff.4: -1
hw.cxgbe.toe.rexmt_backoff.3: -1
hw.cxgbe.toe.rexmt_backoff.2: -1
hw.cxgbe.toe.rexmt_backoff.1: -1
hw.cxgbe.toe.rexmt_backoff.0: -1
hw.cxgbe.toe.rexmt_count: 0
hw.cxgbe.toe.rexmt_max: 0
hw.cxgbe.toe.rexmt_min: 0
hw.cxgbe.toe.keepalive_count: 0
hw.cxgbe.toe.keepalive_interval: 0
hw.cxgbe.toe.keepalive_idle: 0
hw.cxgbe.clip_db_auto: 1



Sad Clouds

unread,
Dec 2, 2025, 10:48:48 AM (3 days ago) Dec 2
to mike tancsa, FreeBSD Questions
On Mon, 1 Dec 2025 13:01:28 -0500
mike tancsa <mi...@sentex.net> wrote:

> TL;DR. I can get 10G coming in and out on port 0, Port 1 can only take
> traffic at ~ 2.5 Gb/s incoming, but can send out a full 10G.

I've seen asymmetric throughput performance similar to what you report
with Intel X520-SR2 and when using device passthrough in Bhyve VMs
running FreeBSD-14.3.

Configuring the switch and network interfaces to use MTU of 9000 bytes
resolved the issue for me. After this, the send and receive throughput
performance has remained stable at around 9 Gb/sec.

mike tancsa

unread,
Dec 2, 2025, 11:11:28 AM (3 days ago) Dec 2
to Sad Clouds, FreeBSD Questions
Thanks for the response. On the one NIC I do indeed have an MTU of 9000,
but the other segment is 1500. Whats odd, is that I took the exact same
motherboard and Chelsio NICs and setup a test lab and I could not
reproduce the issue with or without the MTU settings. I was able to get
a full 10G on both ports. We thought maybe something busted about the
production motherboard.  But when we swapped out the motherboards, the
same thing happened :(  Going to let things settle for a day and try a
mellanox connectx-3 next.  I dont think its a tcp thing.

I just tested /usr/src/tools/tools/netsend

/netsend 192.168.13.254 500 1400 730000 12
Sending packet of payload size 1400 every 0.000001369s for 12 seconds
calling time every 100 cycles

start:             1764691849.000000000
finish:            1764691861.000107338
send calls:        8765600
send errors:       0
approx send rate:  730466 pps
time/packet:       1369 ns
approx error rate: 0
waited:            152052418
approx waits/sec:  12671034
approx wait rate:  17

Which is just shy of 9Gb/s ... So it can handle a 12 second UDP blast,
but it does indeed overflow some. Before and after

sysctl -a dev.cc.1.stats | egrep "flo|tru"
dev.cc.1.stats.rx_trunc3: 0
dev.cc.1.stats.rx_trunc2: 69707
dev.cc.1.stats.rx_trunc1: 0
dev.cc.1.stats.rx_trunc0: 0
dev.cc.1.stats.rx_ovflow3: 0
dev.cc.1.stats.rx_ovflow2: 8255902
dev.cc.1.stats.rx_ovflow1: 0
dev.cc.1.stats.rx_ovflow0: 0

dev.cc.1.stats.rx_trunc3: 0
dev.cc.1.stats.rx_trunc2: 101592
dev.cc.1.stats.rx_trunc1: 0
dev.cc.1.stats.rx_trunc0: 0
dev.cc.1.stats.rx_ovflow3: 0
dev.cc.1.stats.rx_ovflow2: 12901089
dev.cc.1.stats.rx_ovflow1: 0
dev.cc.1.stats.rx_ovflow0: 0

The same test across cc0 the "good" port

sysctl -a dev.cc.0.stats | egrep "flo|tru"
dev.cc.0.stats.rx_trunc3: 0
dev.cc.0.stats.rx_trunc2: 0
dev.cc.0.stats.rx_trunc1: 0
dev.cc.0.stats.rx_trunc0: 0
dev.cc.0.stats.rx_ovflow3: 0
dev.cc.0.stats.rx_ovflow2: 0
dev.cc.0.stats.rx_ovflow1: 0
dev.cc.0.stats.rx_ovflow0: 0



    ---Mike




Navdeep Parhar

unread,
Dec 2, 2025, 11:50:16 AM (3 days ago) Dec 2
to mike tancsa, FreeBSD Questions
Have you tried your tests with these two left to their default values?
We don't have a 64KB allocator in the kernel (16KB is the max) so the
first setting isn't even valid.

Please also provide the output of

# sysctl dev.cc | egrep 'link|caps|pause'

It is possible that the link on the good port has pause based flow
control enabled and the other one doesn't.

Regards,
Navdeep

mike tancsa

unread,
Dec 2, 2025, 12:09:45 PM (3 days ago) Dec 2
to Navdeep Parhar, FreeBSD Questions
On 12/2/2025 11:49 AM, Navdeep Parhar wrote:
> On 12/1/25 10:01 AM, mike tancsa wrote:
>> TL;DR. I can get 10G coming in and out on port 0, Port 1 can only
>> take traffic at ~ 2.5 Gb/s incoming, but can send out a full 10G.
>>
> Have you tried your tests with these two left to their default values?
> We don't have a 64KB allocator in the kernel (16KB is the max) so the
> first setting isn't even valid.

Hi Navdeep,

    Thank you for looking!

Yes, I tried with no settings in /boot/loader.conf and same behaviour. 
I was running with
hw.cxgbe.rdmacaps_allowed="0"
hw.cxgbe.iscsicaps_allowed="0"
hw.cxgbe.fcoecaps_allowed="0"
hw.cxgbe.pause_settings="0"
hw.cxgbe.attack_filter="1"
hw.cxgbe.drop_pkts_with_l3_errors="1"

before as this is just an nfs server, zrepl (receive zfs streams over
tcp) and inbound sftp connections.  cc0 is the busiest interface and
regularly bursts close to full speed. cc0 has an mtu of 9000. cc1 has
the default MTU of 1500 on 4 different vlan interfaces. I also tried to
blast some udp packets across both links as a test and cc1 also
underperforms with some dropped packets (rx_ovflow2,rx_trunc2) where as
the same rate is 100% error free on cc0


>
> Please also provide the output of
>
> # sysctl dev.cc | egrep 'link|caps|pause'
 sysctl dev.cc | egrep 'link|caps|pause'
dev.cc.1.stats.rx_pause: 0
dev.cc.1.stats.tx_pause: 0
dev.cc.1.lpacaps: 458752
dev.cc.1.acaps: 268435460
dev.cc.1.pcaps: 269418502
dev.cc.1.rcaps: 270532612
dev.cc.1.link_fec: 4<NO-FEC>
dev.cc.1.pause_settings: 0
dev.cc.1.linkdnrc: n/a
dev.cc.0.stats.rx_pause: 0
dev.cc.0.stats.tx_pause: 0
dev.cc.0.lpacaps: 458752
dev.cc.0.acaps: 268435460
dev.cc.0.pcaps: 269418502
dev.cc.0.rcaps: 270532612
dev.cc.0.link_fec: 4<NO-FEC>
dev.cc.0.pause_settings: 0

dev.cc.0.linkdnrc: n/a


>
> It is possible that the link on the good port has pause based flow
> control enabled and the other one doesn't.

On the quanta where cc0 is connected (the one without the slowdowns) the
pause counters dont seem to be incrementing. I just did a test and it was

802.3x Pause Frames Received................... 7939772

before and after.

Same with the problem port on the cisco that cc1 is on


cisco-core4#show int te1/1/1 | inc pau
     0 watchdog, 31 multicast, 424569 pause input
     0 lost carrier, 0 no carrier, 0 pause output
cisco-core4#show int te1/1/1 | inc pau
     0 watchdog, 31 multicast, 424569 pause input
     0 lost carrier, 0 no carrier, 0 pause output
cisco-core4#


cisco-core4#show int te1/1/4 | inc pau
     0 watchdog, 88 multicast, 0 pause input
     0 lost carrier, 0 no carrier, 0 pause output
cisco-core4#

Nothing special about the config

interface TenGigabitEthernet1/1/1
 description backup4-cc1
 switchport trunk allowed vlan 2-4,223
 switchport mode trunk
 spanning-tree portfast trunk


(cc0 interface)

(RackA-10G) #show interface ethernet 0/23

Total Packets Received (Octets)................ 164104553985987
Packets Received 64 Octets..................... 2785340519
Packets Received 65-127 Octets................. 3551885604
Packets Received 128-255 Octets................ 2720325376
Packets Received 256-511 Octets................ 2916456887
Packets Received 512-1023 Octets............... 3987305743
Packets Received 1024-1518 Octets.............. 516438495
Packets Received > 1518 Octets................. 0
Packets RX and TX 64 Octets.................... 4127650421
Packets RX and TX 65-127 Octets................ 2635703067
Packets RX and TX 128-255 Octets............... 2696715779
Packets RX and TX 256-511 Octets............... 1789991928
Packets RX and TX 512-1023 Octets.............. 1073841812
Packets RX and TX 1024-1518 Octets............. 3415095303
Packets RX and TX 1519-2047 Octets............. 1866698571
Packets RX and TX 2048-4095 Octets............. 2166348155
Packets RX and TX 4096-9216 Octets............. 2969664207

Total Packets Received Without Errors.......... 1204638026215
Unicast Packets Received....................... 1204629504771
Multicast Packets Received..................... 7939833
Broadcast Packets Received..................... 581611
--More-- or (q)uit

Total Packets Received with MAC Errors......... 0
Jabbers Received............................... 0
Fragments Received............................. 0
Undersize Received............................. 0
Alignment Errors............................... 0
FCS Errors..................................... 0
Overruns....................................... 0

Total Received Packets Not Forwarded........... 7939796
802.3x Pause Frames Received................... 7939772
Unacceptable Frame Type........................ 3

Total Packets Transmitted (Octets)............. 253100014214248
Packets Transmitted 64 Octets.................. 1342309902
Packets Transmitted 65-127 Octets.............. 3378784759
Packets Transmitted 128-255 Octets............. 4271357699
Packets Transmitted 256-511 Octets............. 3168502337
Packets Transmitted 512-1023 Octets............ 1381503365
Packets Transmitted 1024-1518 Octets........... 2898656808
Packets Transmitted > 1518 Octets.............. 0
Max Frame Size................................. 9216
Total Packets Transmitted Successfully......... 828148377556
Unicast Packets Transmitted.................... 828055172598
Multicast Packets Transmitted.................. 92093622
Broadcast Packets Transmitted.................. 1111336

Total Transmit Errors.......................... 0

Total Transmit Packets Discarded............... 0
Single Collision Frames........................ 0
Multiple Collision Frames...................... 0
Excessive Collision Frames..................... 0

802.3x Pause Frames Transmitted................ 0

Time Since Counters Last Cleared............... 1768 day 23 hr 31 min 49
sec


>
> Regards,
> Navdeep
>

mike tancsa

unread,
Dec 4, 2025, 12:45:33 PM (yesterday) Dec 4
to Navdeep Parhar, FreeBSD Questions
On 12/2/2025 12:09 PM, mike tancsa wrote:
> On 12/2/2025 11:49 AM, Navdeep Parhar wrote:
>> On 12/1/25 10:01 AM, mike tancsa wrote:
>>> TL;DR. I can get 10G coming in and out on port 0, Port 1 can only
>>> take traffic at ~ 2.5 Gb/s incoming, but can send out a full 10G.
>>>
>> Have you tried your tests with these two left to their default
>> values? We don't have a 64KB allocator in the kernel (16KB is the
>> max) so the first setting isn't even valid.
>
> Hi Navdeep,
>
>     Thank you for looking!
>
>

Swapped ports and the problem follows the config, not the hardware. 
Before the "slow" port was cc1 and the fast cc0. I swapped them around
so that cc0 was doing what cc1 was and vice versa.  The "slow" port is
now cc0 and cc1 which had a hard time keeping up was dropping packets :(
This is all with pf enabled. Whats odd is if I disable pf, it doubles
the performance on the problem port to 5Gb. With or without pf the
"fast" port is always fast.

Next step. I installed a Connex4 mce NIC and what tested out a full 10G
in both directions, is also not performing at full speed?!?!?!   
Testing this card with the same CPU and motherboard on a test box gives
a full 10G on both ports?!

Other than kern.ipc.maxsockbuf=85097152 /etc/sysctl.conf doesnt have any
network perf valuessecurity.bsd.see_other_uids=0
security.bsd.see_other_gids=0
security.bsd.unprivileged_read_msgbuf=0
security.bsd.unprivileged_proc_debug=0
vfs.zfs.arc.max=89334498304
#vfs.zfs.arc.meta_limit=19333624576
#vfs.zfs.abd_scatter_enabled=0
vfs.nfs.enable_uidtostring=1
vfs.nfsd.enable_stringtouid=1

and loader.conf is

hw.cxgbe.rdmacaps_allowed="0"
hw.cxgbe.iscsicaps_allowed="0"
hw.cxgbe.fcoecaps_allowed="0"
hw.cxgbe.pause_settings="0"
hw.cxgbe.attack_filter="1"
hw.cxgbe.drop_pkts_with_l3_errors="1"



dev.mce.1.conf.channels_rsss: 1
dev.mce.0.conf.channels_rsss: 1
dev.cc.1.rss_size: 128
dev.cc.1.rss_base: 128
dev.cc.0.rss_size: 128
dev.cc.0.rss_base: 0

irq157: mpr0                     4882056       1390
irq158: nvme0:admin                  206          0
irq159: nvme0:io0                     20          0
irq160: nvme0:io1                     21          0
irq163: nvme0:io4                     78          0
irq164: nvme0:io5                     81          0
irq175: xhci0                      55014         16
irq177: t6nex0:evt                     4          0
irq178: t6nex0:0a0                695005        198
irq179: t6nex0:0a1               1221879        348
irq180: t6nex0:0a2                518121        148
irq181: t6nex0:0a3               1134062        323
irq182: t6nex0:0a4               1254544        357
irq183: t6nex0:0a5                501704        143
irq184: t6nex0:0a6                776666        221
irq185: t6nex0:0a7                517170        147
irq188: t6nex0:1a0                 61742         18
irq189: t6nex0:1a1                926650        264
irq190: t6nex0:1a2               1104128        314
irq191: t6nex0:1a3                  4927          1
irq192: t6nex0:1a4                724290        206
irq193: t6nex0:1a5                 77998         22
irq194: t6nex0:1a6               1636306        466
irq195: t6nex0:1a7               2955293        842
irq200: xhci3                      12555          4
irq217: ahci3:ch0                 137849         39
irq218: ahci3:ch1                 139450         40
irq233: mlx5_core0                     1          0
irq234: mlx5_core0                143358         41
irq235: mlx5_core0                     5          0
irq236: mlx5_core0                     1          0
irq238: mlx5_core0                    33          0
irq245: mlx5_core0                     2          0
irq246: mlx5_core0                     1          0
irq257: mlx5_core0                   133          0
irq267: mlx5_core0                     1          0
irq268: mlx5_core1                     1          0
irq269: mlx5_core1                219866         63
irq270: mlx5_core1                     8          0
irq271: mlx5_core1                     1          0
irq274: mlx5_core1                    19          0
irq275: mlx5_core1                  4085          1
irq278: mlx5_core1                    19          0
irq280: mlx5_core1                     2          0
irq281: mlx5_core1                     2          0
irq289: mlx5_core1                    55          0
irq292: mlx5_core1                    19          0
irq301: mlx5_core1                    22          0
irq302: mlx5_core1                     4          0


Reply all
Reply to author
Forward
0 new messages