These vector displays are hard on the system because they are repainted constantly 30-40 times a second. This takes a toll on both the emulated (guest) computer, and the host. I have noted that the Rasperry Pi, even model 5, seem more affected than a regular PC.
I have experimented with sending vector data over the network, hooking into the "display" component of SIMH to capture graphics command from any of the simulators. The remote side could be anything, an SDL application, JavaScript web app, etc. I think this would be less CPU intensive than X11 or VNC. But it's not a finished usable solution.