Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

PiDP-8/I SiMH Benchmark scripts

77 views
Skip to first unread message

Mike Katz

unread,
Jul 11, 2024, 5:11:33 PM7/11/24
to PiDP-8, Bill Cattey, Steve Tockey, Heinz-Bernd Eggenstein
I have all of the scripts written for phase one of the benchmarks.

I modified Steve's Prime number program in Pascal to automatically calculate the first 1000 prime numbers without outputting anything.  This eliminates any SIMH/Raspberry Pi I/O issues.

All of the builds as well as the benchmarks are run with the /usr/bin/time -v command to give memory and time statistics in addition to the SiMH show clocks command.

The files are as follows:

Benchmark1                      <-- expect script to compile and run the PRIME1 pascal program
Benchmark.rk05               <-- updated rk05 image from Steve with the modified PRIME1 program
BenchmarkRun                 <-- shell script to run the benchmark in the trunk, pi5, pi5-ils2 and pi5-ils2-cyclerealistic branches
BenchmarkRunLog_...     <-- output from the BenchmarkRun script
BenchmarkSetup             <-- setup script for the benchmark.  This installs any necessary commands, clones the fossil repository for the trunk, pi5, pi5-ils2 and pi5-ils2-cyclerealistic branches and then builds each of the branches
BenchmarkSetupLog_... <-- output from BenchmarkSetup script
SimStart                             <-- simh script to setup simh and and attach Benchmark.rk05 as rk0 and then boots OS/8 from the rk0.

The attached benchmark.7z contains all of these files.

Here is the procedure I will use to run the benchmarks on each of the Raspberry Pi's that I have:
  1. Attach pi to PiDP-8/I using extension cable
  2. Create a brands new sd card using the Raspberry Pi Image Create utility
  3. Copy the scripts to the sd card
  4. Run the BenchmarkSetup script.  Note:  This script installs all operating systems updates
  5. Run the BenchmarkRun script
  6. Extract the execution time and instructions/second from the log and enter them into a spread sheet.

i will be running the tests on the following Pis/Operating Systems:

  • Pi 5 32 bit lite
  • P i5 64 bit lite
  • P i4 32 bit lite
  • P i3 32 bit lite
  • P i2 32 bit lite
  • Pi B 32 bit lite
  • Pi A 32 bit lite
  • Pi 02W 32 bit lite
  • Pi 0W 32 bit lite

Please review these and let me know if you feel there is are any improvements or changes that you recommend.

Bill, I'm sorry but I didn't get a chance to look at your OS8-run utility so I used the linux expect utility instead.

It will be easy to take this and add benchmarks by either modifying Benchmark1 or adding additional BenchmarkX files and modifying the BenchmarkRun script.  Modifying Benchmark1 is the recommended method as it requires the least amount of changes.

Once you all have reviewed this I will check it all into github or should I add it to the tangentsoft fossil repository.  If you think the tangentsoft repository is better, please point me in the direction of some document on how to add to this repository and where I should put it in that repository.

Earlier this week the cyclerealistic build was able to build.  I tried it again today and it failed.

Here are the all of the warning and error messages for the build (please see the BenchmarkSetupLog_... file).  The line numbers refer to the line numbers in the log file not the setup script.


  1070 1:WARNING: Building on Pi platform without libraspberrypi-dev installed!
  1390 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  1443 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  1560 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  1764 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  1934 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  1937 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  3339 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  3392 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  3424 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  3566 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  3716 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  3886 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  3889 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  5294 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  5347 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  5463 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  5521 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  5671 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  5841 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  5844 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  7251 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  7322 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  7327 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  7338 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  7552 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  7565 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
  7792 65:os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".

Line 1449, 3399, 5354 & 7345:
Running script file: ./scripts/misc/focal69-tu56.os8
PAL8 Error:
    FOCAL.BN<DTA0:FOCAL.ZM,DTA0:FLOAT.ZM
    RD C100 +0001

Non-fatal error encountered in ./scripts/misc/focal69-tu56.os8 at line 20: pal8 FOCAL.BN<DTA0:FOCAL.ZM,DTA0:FLOAT.ZM

Line 1947, 3899, 5854,  :
Fatal error encountered in ./scripts/os8/all-tu56.os8 at line 291: os8 SUBMIT SYS:LCSYS.BI
gmake: *** [Makefile:843: bin/v3f-td12k.tu56] Error 255
Command exited with non-zero status 2
Line 7248:
Running script file: ./scripts/misc/chekmo-tu56.os8
pexpect timeout
    match before: {}
os8_pip_to 2: Expected: "Command Decoder Prompt". Instead got: "USE PIP FOR NON-FILE STRUCTURED DEVICE".
pexpect timeout
    match before: {}
os8_pip_to 3: failure.
call_pal8: Expected: "Command Decoder Prompt". Instead got: "File not found".
PAL8 failed to start at line 15
Non-fatal error encountered in ./scripts/misc/chekmo-tu56.os8 at line 15:
    pal8 CHEKMO.BN<DTA0:CHEKMO.PA

Line 7848

Fatal error encountered in ./scripts/os8/all-tu56.os8 at line 291:
     os8 SUBMIT SYS:LCSYS.BI
make: *** [Makefile:862: bin/v3d-td12k.tu56] Error 255
Command exited with non-zero status 2

I am unable to build the current pi5-ils2-bworm-cyclerealistic branch.  I know that some changes were recently checked in.

           Mike




Benchmark.7z

Heinz-Bernd Eggenstein

unread,
Jul 11, 2024, 5:50:04 PM7/11/24
to PiDP-8
Interesting.....
The building of the pidp8i simulator application from the pi5-ils2-bworm-cyclerealistic branch actually succeeds, the errors and warnings that you saw happened when this application was used to build the os8 images.

One error was actually an error in a python script:

"  File "/home/mkatz/src/pidp8i/pi5-ils2-cyclerealistic/lib/os8script.py", line 1511, in os8_command
    if mon == false:
              ^^^^^
NameError: name 'false' is not defined. Did you mean: 'False'?
"
Yeah indeed, boolean false in Python is "False" not "false", but this bug has beeen there for ages now so it's just a hickup in the error handling I guess and not the root problem. And none of this  has anything  to do with the recent changes made to that branch. So if it worjked before and no longer now, I guess it's mor elike a sporadic problemin the scripting perhaps (race condition?)

Anyway, someone who is familiar with the Python scripting would have to look into this.

Cheers
HB

William Cattey

unread,
Jul 14, 2024, 2:18:31 PM7/14/24
to PiDP-8
I’m out of town at a choral convention and can’t be much help till like Tuesday.
Every so often I have to set debug levels high and dig for an expect race or subtle string sensitivity.

When we went to Open SIMH everything broke because the SIMH start-up string changed.

Good luck!  I’ll think more about this when I return.

-Bill 

William Cattey

unread,
Aug 3, 2024, 1:43:46 PM8/3/24
to PiDP-8
Wow.  Those lines should be "False" not "false". I've just checked a fix for that into trunk.  Y'all should pull that into your branches.
Reply all
Reply to author
Forward
0 new messages