I have been enjoying using Joerg Hoppe's Unibone on my PDP-11/45 for a few years now. It occurred to me last week that I ought to be able to use the Unibone
to snapshot the entire PDP-11 memory. To my delight Joerg's demo
program already has this capability. With some truly hackish
scripting, I set the Unibone to snapshot the entire 248k of memory every
1/2 second and ship the resulting memory.dump
file over to my Linux workstation for a real-time display. The Linux
side uses even more hackish scripting; a mixture of C for preprocessing
and R for rendering.
I have watched several operating systems boot using this technique.
Ultrix-11 is the most exciting of the bunch, because it is quite
involved. (For comparison, RT-11 boots so fast that I basically can't
capture it! Well, I can see the first and second stage bootloader zip
past, but the drivers load so fast that there isn't much to see.)
The interpretation of the memory display in the upper right corner is
that the horizontal axis is for addresses, while each column is a
histogram of byte values aggregated over swaths of 1024 bytes. The
vertical axis has 0 at the bottom and 255 at the top; brightness
indicates how frequently that byte value was observed in the
corresponding swath. PDP-11 opcodes are not uniformly distributed in
byte values (nor is human-readable ASCII), so there is a definite visual
"texture" that can help identify when sections of memory are copied.
This happens early in the boot process. What's kind of striking to me is
how huge Ultrix is and how it appears to basically reserve the lower
half of addressable memory. You can see the small-ish, swappable
processes slotting in and out above that once timesharing starts.
It is worth noting that although the video is a composition of two video
streams, I didn't adjust the timing between them. The console and the
live memory display are just as they appeared on my workstation, in sync
with the front panel. There is a noticeable lag of about a second
between when something happens on the front panel and/or console and
when it appears in the memory display window. This is a result of the
1/2 second update from the Unibone, the network transmission delay, and a
not-insignificant delay for the image viewer.
How does this work: Unibone scripting
The basic idea is that if you're on the Unibone in the demo program under the device emulation menu, you can say
to the end of an otherwise working command file. Now the
memory.dump
will be updated every half second for about eight minutes. (You could
try a shorter delay between updates. This hogs the Unibus, making the
already-long boot time of Ultrix much longer...)
How does this work: Linux workstation scripting
Once the Unibone is busy snapshotting the memory, I have a script on my Linux workstation that copies the
memory.dump
to a local working directory every second or so. Because my day job
involves statistical analysis, it was quickest for me to throw together a
script in R since that's where I do my rendering. Bash would have been
just fine too. I use passwordless
scp to do the copy, since it scripts easily.
The memory graphic is produced by a three-step process: (1) aggregate
byte histograms over a sliding window, (2) render the sliding window
into a PNG, and (3) display the PNG.
Item (1) is done using a
window_distribution program I wrote in C. You can get the source from
https://github.com/kb1dds/bytewise-stats
(Build instructions are not included because all that's required is a single call to
gcc.)
Item (2) is done in R, but could have just as easily been done using Python, C, or whatever. As an example of what's involved,
https://github.com/kb1dds/bytewise-stats/blob/main/examples/ultrix11_booting.R
I made a few modifications from the above R script for the one used in video:
- Filename paths are adjusted accordingly, obviously.
- It's on an infinite loop.
- It handles the scp and execution of window_distribution using the dreaded-but-useful system() function.
- It doesn't color the memory locations by
k-means clusters, since it's annoying (but not impossible) to stabilize
clusters across memory snapshots.
- It plots to memory1.png and then renames this to memory.png. This way, the time during which an invalid memory.png
file is present on the disk is minimized. Most image viewers will
happily open a half-written PNG file but then promptly complain that
it's broken.
Finally, there are several options for (3). On Ubuntu, the
eog
image viewer will redraw on file update by default. This works
tolerably well, but the redraw involves blanking the image first. This
makes it hard to see small memory updates. Instead, I used a different
image viewer that accepts an explicit reload option
$ feh --reload 1 -g 640x480 -. memory.png
Unfortunately, as packaged with Ubuntu (and Debian),
feh
cannot update faster than once per second. (Apparently, if you build
from source, you can have it trigger an update from file modification. I
didn't try that.)
Closing thoughts:
It's been a dream of mine to peer inside the workings of a "live"
machine for a long time. Although the front panel of a "proper"
minicomputer (or mainframe) gives a good picture, it's really only a
tiny window in time and address space. The fact that the Unibone is
fast, scriptable, and has plenty of storage space opens up lots of
possibilities for visualization experimentation!
Enjoy!
Michael