The rise, fall, and resurrection of APL\3000

[Gavin Scott]

(If anyone can add to this story or knows if any of the HP people involved are still around I would love to hear it!)

Back at the beginning of the 1970s, Hewlett-Packard developed a new general-purpose timesharing commercial data processing computer system called the HP 3000. It was a stack-based CISC system that ran an operating system called MPE (Multi-Programming Executive). This series of computer systems became legendary among its loyal user community and had a long life across multiple hardware architectures before being unceremoniously axed in 2001 (there are still a few of the last generations in use today of course).

Perhaps its least-known optional subsystem product among COBOL/3000, BASIC/3000, FORTRAN/3000, RPG/3000, etc. was an APL implementation called APL\3000. As one could perhaps guess by the backslash in its name, HP seems to have been inspired by APL\360's success as a timesharing product, and it implements almost exactly the IBM APL.SV feature set.

While at first glance it appears as a clone of the IBM product, the HP implementation included a number of advanced features including a (the first?) APL compiler, some very advanced optimizations, APLGOL, and a virtually (pun possibly intended) unlimited workspace and expression result size through a clever implementation of a flat 32-bit address space on a machine where every other language was limited to 64KiB of directly addressable memory. HP also produced a variant of their (several thousand dollar, 50lb) HP2645 terminal, the HP2641A, which offered the APL character set and overstrike character entry and tight integration with the APL\3000 product. They also supported simple ASCII terminals through a three-character replacement for the missing special characters so if you wanted to enter a quad you would type "QD. Iota was "IO, rho "RO etc. Ugly, but it meant you could also write a batch job using plain ASCII files that could utilize APL.

In order to get the 32-bit address space needed to have an unlimited workspace and result size, they developed ten new CPU instructions whose microcode came on a set of PROM chips when you bought the $15,000 product. This would ultimately spell the doom for APL\3000 when HP lost interest in it at the end of the 1970s and failed to implement those required instructions on any later HP 3000 CPUs. Thus APL\3000 was only ever supported on their Series II and Series III models of HP 3000s.

The implementation was actually pretty clever. The APL system would open the workspace file and set up a series of virtual memory "pages" in the stack in the form of a linked-list whose address was stored in a memory location known to the firmware. The new microcoded instructions provided virtual 32-bit address loads and stores and moves to and from the regular part of the stack, as well as a virtual to virtual move and some special instructions to accelerate the execution of their compiled APL code which used a virtual machine they created called the "E Machine". The microcode used the pages in the stack as a cache for the much larger workspace file and would search through the linked list of pages for a particular virtual address, maintaining the list in most-recently-accessed order as it went, and would call back to APL if an address could not be found at which point the APL system would write out the least-recently-accessed page if it was dirty and replace it by reading in the appropriate chunk of the workspace file after which the faulting instruction would be restarted by the microcode.

The default setup was something like 24 pages of 512 or 1024 bytes each, and a system variable []VM was provided to interrogate and reconfigure the VM page number and size and the search method used by the microcode.

As a result, one can evaluate things like 10000000?20000000 that likely were impossible on any other APL implementation at the time (ok, well, that takes about ten minutes on a modern PC under simulation and likely would have taken a few days to complete on a real system, but it DID work!).

So if this APL implementation was so amazing, with unlimited result size, unlimited workspace size, etc. why has almost nobody heard of it? Well, when it came out, performance apparently sucked. The developers had fallen back on a virtual machine target for their compiler after deciding there was no efficient way to emit native code due to an enforced separation of code and data. So every APL expression went through compilation into an 8-bit byte-code for the virtual E Machine (though they were very clever about determining when they could avoid recompilation, and IF it was required they would generate a different style of code the second time that was more generic and was less likely to require further recompilation) which then had to be executed by their virtual E Machine implementation. They spent the next three years working to improve things, but by 1979 no magic bullet solution was in sight and even if it had been, the HP sales force and systems engineers had almost universally forgotten about the product's existence.

So when the last HP 3000 Series III system with APL installed was shut down sometime in the 1980s, APL\3000 passed into legend. There were those who whispered about its existence, and I think the first discussion I had about the possibility of resurrecting it was in the early 1980s. At the time the thought was that a real Series III would be needed, and I knew someone who had a set of the firmware PROMs, but it never happened and slowly the legends faded but were never forgotten.

A month or so ago I started playing with J. David Bryan's amazing classic Series III HP 3000 hardware simulator with the goal of building up the most complete system possible, utilizing all the old bits that our lovable retro-computing aficionados have saved over the years like the incurable packrats they are. A friend had squirreled away a tape with APL on it and I became interested again in the idea of bringing it back to life. Since we now had a software simulator of the hardware, the idea of implementing the mythical (and undocumented) APL instructions needed to run it seemed within reach, and I started trying to figure out the behavior of the missing instructions, and after a week or so of hacking on it we now have APL\3000 back from the dead.

:APL
APL\3000 HP32105A.01.05 (C)HEWLETT-PACKARD CO. 1979
05:50 08/29/92
2+2
4

The ironic thing is that I ended up implementing the instructions in MPE's own unimplemented instruction trap handler and made no changes to the hardware simulator itself, so this technique could have been used back in 1980 when we first started talking about getting APL going somehow. Oh well.

Is this good for anything? Probably not really, though it's always fun to have a chance to play with a piece of software that probably has not been executed in 35 years. And it holds an important place in the history of language development, having been the "first" at a few things. Unfortunately, it was perhaps just a bit too far ahead of its time and failed to survive until machines got big and fast and wide enough for it to really shine.

Due to the shadowy license state (or lack thereof) of all this forty-year-old software, I probably won't be putting it up for public download, but if anyone is interested you can hit me up and I can probably set you up fam.

Gavin

[mark.a....@gmail.com]

Well done Gavin! What a neat project to have completed!

Back in the day I worked on APL for one of HP's competitors. Seeing a paper in one of the APL conference proceedings describing APL\3000 was both exciting and sobering. The HP APL team had done something quite remarkable for the time, but had been undone by unyielding hardware design. I felt their pain.

Now you should make a web front end for your system so people anywhere can try it out! (0.5 kidding) Next we can start going through the list of APL implementations in the APL FAQ and reviving all we can. (1.0 kidding)

Seriously, nicely done!

..mark

[Doug White]

I realize the original post is a bit old, but I just saw it recently when I
visited the newsgroup after a lenghty absence. I thought I would chime in
with my experiences with APL/3000.

I worked as a circuit designer at HP Santa Clara between 1978 and 1982. I
used APL when I was a student at MIT, and was delighted to have it
available at my firt job. The HP3000 was used by everyone in the facility
(which was quite large). The accountants used it, the secretaries used it
for word processing, and the engineers used it. It tracked inventory &
sales as well. I forget the details, but the APL was supposed to "psudo-
compile" functions the first tiem it ran them so that it didn't have to re-
interpret them the next they were called. The marketing hype was that it
was supposed to be lightning fast,

I designed an analog (inductor & capacitor) filter for a new instrument. I
needed to make sure that it would meet specification using moderate
precision components without requiring any tuning in production. To check
this, I wrote a dedicated circuit simulation function, and set up a Monte
Carlo analysis that fed it randomized component values. The circuit
analysis wasn't especially computationally demanding, but running the Monte
Carlo simulations with a reasonmable number of trials took a while.

After several short test runs, I fired it up one day with a larger number
of trials. Within about 10 or 15 minutes, the computer people had been
flooded with complaints that the 3000 was suddenly crawling. They tracked
me down and demanded to know what I was doing that had brought the entire
Division's mainframe to its knees.

I quickly switched to coming in on nights & weekends to do my simulations.
It wasn't exactly speedy, but it was a lot faster with no one else on the
system.

In discussing this with other engineers, there was something about the
HP3000 computer architecture that swapped code & data in & out of memory
when it switched tasks. If you ran anything memory intensive, that tended
to bog things down. Supposedly the performance of APL3000 in particular
was so bad that they actually lost a law suit when it failed to come close
to its claimed performance. I think part of APL's demise at HP was the
resulting black eye it earned in the minds of management.

The one other thing I remember was that the APL3000 language manual was
wonderful. They had beautiful color graphics that showed how some of the
more complex matrix/array manipulation primatives worked. Long after I
left HP, I still kept the manual for reference.

Doug White

Gavin Scott <ga...@learn.bio> wrote in
news:b62e0d7f-09a1-4740...@googlegroups.com:

--
This email has been checked for viruses by AVG.
https://www.avg.com

[Gavin Scott]

On Monday, December 21, 2020 at 8:12:54 PM UTC-6, Doug White wrote:
> I realize the original post is a bit old, but I just saw it recently when I
> visited the newsgroup after a lenghty absence. I thought I would chime in
> with my experiences with APL/3000.

Thanks, Doug, it's always great to hear from actual APL\3000 users!

Cheers,

Gavin