Really bad performance with OPNSense under bhyve

[C. L. Martinez]

Hi all,

I have installed OPNSense latest release (17.1.8, LibreSSL with SafeStack) under my HardenedBSD 11-STABLE laptop as a bhyve's guest and performance (almost with CPU and RAM) is really bad, bad, bad ...

I am using vm-bhyve to manage my vms. For OPNSense, vm-bhyve uses the following command:

Jun 10 05:41:53: initialising
Jun 10 05:41:53: [loader: bhyveload]
Jun 10 05:41:53: [uefi: no]
Jun 10 05:41:53: [cpu: 1]
Jun 10 05:41:53: [memory: 1280M]
Jun 10 05:41:53: [hostbridge: standard]
Jun 10 05:41:53: [com ports: com1]
Jun 10 05:41:53: [uuid: auto]
Jun 10 05:41:53: [utctime: true]
Jun 10 05:41:53: [debug mode: no]
Jun 10 05:41:53: [primary disk: /dev/zvol/zroot/data/vmachines/dsks/opnsfwvol]
Jun 10 05:41:53: [primary disk dev: custom]
Jun 10 05:41:53: initialising network device tap0
Jun 10 05:41:54: adding tap0 -> bridge0 (prodif)
Jun 10 05:41:54: initialising network device tap1
Jun 10 05:41:54: adding tap1 -> bridge1 (pubif)
Jun 10 05:41:54: initialising network device tap2
Jun 10 05:41:54: adding tap2 -> bridge2 (dmzif)
Jun 10 05:41:54: initialising network device tap3
Jun 10 05:41:54: adding tap3 -> bridge3 (mgmtif)
Jun 10 05:41:54: initialising network device tap4
Jun 10 05:41:54: adding tap4 -> bridge4 (vpnif)
Jun 10 05:41:54: initialising network device tap5
Jun 10 05:41:54: adding tap5 -> bridge5 (encif)
Jun 10 05:41:54: booting
Jun 10 05:41:54: bhyveload -m 1280M -e autoboot_delay=3 -d /dev/zvol/zroot/data/vmachines/dsks/opnsfwvol opnsfw
Jun 10 05:41:59: [bhyve options: -c 1 -m 1280M -AHP -u]
Jun 10 05:41:59: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/dev/zvol/zroot/data/vmachines/dsks/opnsfwvol -s 5:0,virtio-net,tap0,mac=00:50:56:3b:68:c6 -s 5:1,virtio-net,tap1,mac=00:50:56:38:b1:40 -s 5:2,virtio-net,tap2,mac=00:50:56:4c:a9:43 -s 5:3,virtio-net,tap3,mac=00:50:56:00:ca:1f -s 5:4,virtio-net,tap4,mac=00:50:56:23:0e:ba -s 5:5,virtio-net,tap5,mac=00:50:56:18:03:37 -s 6:0,virtio-rnd]
Jun 10 05:41:59: [bhyve console: -l com1,stdio]
Jun 10 05:41:59: starting bhyve (run 1)

As you can see I have assigned 1 vcpu and 1280M for RAM... In this laptop I have 8GB of RAM. When OPNSense starts my loadvg goes to 1.6 1.4 14 and RAM drops under 50MB ... I don't understand why.

Setting up other vms with the same amount of RAM, CPU and same number of virtual nics (OpenBSD, FreeBSD, HardenedBSD and Kali Linux) , all goes ok.

Any idea why performace is really bad with OPNSense??

Thanks.
--
Greetings,
C. L. Martinez

[C. L. Martinez]

Please, any idea? Maybe is it a bug?

Thnaks.

[Oliver Pinter]

sysctl kern.sched.preempt_thresh=224