That can get complicated because of the number of layers involved. On a real 11/70, RSTS/E is talking to a real multiplexor board. On the PiDP-11 it is talking to a part of the emulator that pretends to be a mutiplexor on that side, and pretends to have virtual (or real) terminals configured on the other side. So simh needs to know how to "run (virtual) cables" between the multiplexor and virtual (or real) terminals.
On RSTS/E 9.0 or later, you have DCL to set the terminal characteristics, like this:
set terminal kbh0:/permanent/noautobaud/nodialup/speed=9600/device_type=VT320
I don't remember how this was done before DCL, so I can't help with your 7.0 system. What we have here is RSTS/E telling the emulated multiplexor that it wants the speed set to 9600. simh then looks at its configuration, and if the terminal exists as a real device in the configuration file, it tells Linux to set the baud rate. Linux looks up the device type and sends an IOCTL to the driver
for that device. The driver will respond with one of:
1) Done, all good
2) I can't do that
3) Some part of doing it, which may or may not accomplish anything.
Once al that happens, simh interprets the response, transforms it into a (hopefully appropriate) RSTS/E status code and reports that back to RSTS/E
Let me share the relevant parts of my boot.ini (for use with my "canned" RSTS/E V10.1 disk.) Note that I am using vh devices, you will likely want to use dz11 instead. My vh0 line appears in RSTS as KBH0: and KB17: - two different names for the same thing. In the "set terminal" command above I'm using the physical device name (KBH0:) because the assignments of physical devices to regular KBxx:
devices can change - using the command on KBH0: ensures I'm talking to the correct place. I actually have a VT420 attached to that port and it works fine.
The first thing that comes out on (working) serial ports when RSTS/E finishes starting up is something along the lines of "RSTS/E is on the air". If you simply have a baud rate mismatch, that could display as gibberish and keyboard input ignored (possibly with echoing of incorrect characters).
;
; Set up the USB-to-serial ports on the RPi
set vh lines=16
set vh dhu
set vh nomodem
set vh enable
;
; Note that these are "shuffled" so the cabling matches the panel
attach vh line=0,connect=/dev/ttyUSB0
attach vh line=1,connect=/dev/ttyUSB1
attach vh line=2,connect=/dev/ttyUSB3
attach vh line=3,connect=/dev/ttyUSB2
At the boot prompt about starting timesharing (or just "Option?" on older RSTS/E versions), you can type "HA LI" (HArdware LIst) at that prompt and RSTS will tell you what devices physically exist on the system. Note that at this point in booting, RSTS's INIT.SYS knows about nearly every possible PDP-11 controller, even ones that RSTS/E doesn't support (either because RSTS/E never had a driver for it, RSTS/E had a driver at one time but support was dropped, like RS03/04 fixed-head disks, or your RSTS/E monitor was built without the driver).
As you see, it gets complicated quickly. Poke around with the info I provided and I'll try to answer ASAP (I'm having surgery on the 19th, so there will be a delay in all of ny answers at that point).