I have helped port GCC in the past, like in 1988.  I am quite sure the
  design has improved since then.  You imply it has in the most unuseful
  way, so I guess you think it is quite useless to be specific and useful.

  That would be an operating system for and in (Common) Lisp, would it not?

But in what I suspect (without real knowledge) is a large proportion
of ports nowadays the target machine is something which does not yet
have anything on it, because it's a little embedded processor for
which you (the maker of the processor) are going to provide the gnu
toolchain (this seems to be pretty much standard for embedded systems
now).  In these cases you probably have seriously minimal support on
the target since you're aiming to get something like an OS up on it
from the bare iron.

It seems (from what I see) that your goal is to build a "not far off
bare iron" Lisp environment.

A hopefully not-too-wild guess would be that you might take one of the
Unix-like kernels, and rather than running init on top of that,
running Lisp instead.  (I could be wrong, but that would surprise a
bit...)

I think your goal is likely to be to:
 - Interface to the kernel's devices;
 - Interface to the kernel's memory manager;
and then build a usable environment on top of that.

Let's suppose that this environment is to be an implementation of
Common Lisp.  (Changing to Scheme wouldn't change the fundamentals.)

You clearly need to have the following things:

a) A code generator that generates and stores object code that the
kernel knows how to load and run.

b) A memory management system that requests memory from the kernel,
when needed, which doles it out when the CL wants to CONS a bit,
which can reclaim garbage, and perhaps even return it to the kernel.

Those are things which fall out of the scope of things that a CL
implementation "trivially provides."

It would certainly be a reasonable idea to create, in portable Common
Lisp (or, for that matter, Scheme):

 a) Some functions that know how to take object code in some sort of
 buffer, and write it out to storage;

 b) Some functions that implement the "code buffer" described in a),
 and which allow storing machine language instructions in that buffer
 by making use of some sort of "assembler";

 c) Some set of Lisp functions that implement a memory manager, using
 the "assembler" described in b).

Given that, you can then proceed to implement:

 d) A library of string and number functions using c) and b);

 e) An interface, in assembly language, to the OS kernel, which uses
 d) and b) to provide an API that can access OS functionality.

At this point you have a memory manager, a program loader and "saver,"
an assembler, and access to OS services.  

You can start assembling functions to implement basic pieces of a Lisp
environment, and then assemble many of those together to implement
further functions and macros.

There's going to be a bunch of code that is written in pretty much raw
assembler, guaranteed.  I'd think a)-e) are all examples of that.

The proper goal of the exercise will doubtless be to implement in the
f)-and-following phases the pieces of the Lisp environment needed to
host a)-e).

Thus, it would be perfectly acceptable to make extensive use of macros
and CLOS throughout perhaps even MOP, with the underlying expectation
that f)-and-following includes an implementation of macros, CLOS, and
MOP.  In effect, the project involves implementing an Extremely
Sophisticated Assembler in Lisp, and that's quite a reasonable idea.

You could reasonably assume that since there exists the CLOS
implementation "PCL," that you don't forcibly have to create CLOS from
scratch, but could just implement enough of CL to allow loading in
PCL.

What you _can't_ assume is that simply by the sheer existence of
CLISP, CMU/CL, and ACL, you can avoid a)-e) or f)-and-following.

Your big tasks are going to be to design:
 a) How you'll save and load binary code;
 b) How you'll implement memory management;
 c) How you'll store Lisp code in memory;
 d) How you'll store Lisp data in memory.

Note how even the low-tech altenative of sandboxing does not work too
well; numerous flaws were pointed out in the Java security scheme, and
even though e.g. FreeBSD provides a "jail" system call, you probably
would only want to use it to provide an extra level of protection for
deamons that already provide their own checks. Relying completly on the
OS provisions for this would feel risky.

The primary reason that these runtime checks are universally included is
that they solve a *different* problem: when I make a mistake and
introduce a bug in my program, how can I prevent it from clobbering all
information on my computer and setting me back a week's worth of work?
Somewhere on Dennis Ritchie's homepage there is a description of what it
was like working on their multi-user machine before the memory
protection unit was delivered. Before you ran your newly compiled
program you shouted out "a.out!" and waited until your coworkers had had
time to save their files. Quite often the computer would stop echoing
keystrokes after that.

But the crucial point is that this problem has another solution, by
writing your program in a type-safe language like Lisp, ML, Java instead
of in assembly or C. No Lisp or Java program will ever modify memory
that wasn't properly allocated to it. And this is not because of
run-time checking - there are some checks, like array bounds, but
primarily this is because these languages have _no way to describe_ the
act of peeking and poking at data that does not belong to you (kind of
like Orwell's newspeak). So there is no need to monitor an ML program
with runtime checks - the very fact that it was written in ML gives us
all the confidence we need.

There is some loss of clarity because the same word "protection" is used
to describle both the concepts above (and the also unrelated idea of
enforcing good software engineering practice by "encapsulating" parts of
programs in modules).

But if you do your program development in a type-safe language using a
standard OS, then you are paying for a feature you don't need, namely
the sandboxing of programs into processes that cannot hurt eachother.
(Whereas if you program in some more assembly-style language, this is
definently a feature you would want). And these checks do not have
negligable cost. Every time you want your processes to communicate with
each other, or with the world, or you just want to switch to another
process in a multi-tasking system, you need to tell the
memory-protection unit about it, set up a new stack and virtual address
translation table, etc. These context-switches take time - OS
preformance is assesed partly on the basis of how long time, buffering
IO libraries are used to avoid them, etc.

It would be nice with a operating system based on a type-safe language
instead of C. It could then dispense with the concept of processes all
together, and there would be no distinction between user or system code.
(the "kernel" would dissolve into a set of libraries, with a thread
scheduler somewhere).

Such an OS could of course still sandbox or proof-check or verify the
signatures of untrusted code, just as you run a Java VM in Windows/Unix.

-Vilhelm

Even if your language has no escape hatches, you are still putting a lot of
trust in the security and quality of your runtime environment (which ultimately
is not implemented in the safe language).

It is far far better to have both safety features (language safety and
OS-protections).

I would have an easier time "trusting" my environment if I knew its
safety relies on proofs rather than on mere hope that not one of a buch of
a few thousand programmers who I personally don't know haven't screwed up
somewhere.

Isn't this what exokernels are all about? As I remember/understood it,
the exokernel "movement" takes the microkernel idea to the logical
extreme, expecting the kernel to do very little other than
multiplexing access to hardware.

  They would today, since all kinds of creep can and do get access to
  computers.  Besides, if anyone would implement a system that invites
  intrusion and viruses and malicious abuse and is completely helpless in
  the face of such abuse, they would be violating a number of Microsoft
  patents on how _not_ to design software.

I'd say if you run malicious code, outside some specially rigged jail
environment, you are in big trouble regardless. Java's virtual machine
is one such rigged environment, but to my knowledge this approach also
doesn't rely on strictly enforced address spaces.

Address spaces do allow multiple users (to a limited degree) not to
interfere with one another, but I believe most PC-users today get to
have one personal computer each for typical work, and demanding
applications get at least one devoted machine.

sper...@informatik.uni-tuebingen.de (Michael Sperber [Mr. Preprocessor]) writes:

Let me plug my thesis.

http://ncstrl.cs.princeton.edu/expand.php?id=TR-640-01

Abstract

 ...

I combine existing type systems with several standard type-based compilation
techniques to write strongly typed programs that include a function that
acts as a tracing garbage collector for the program. Since the garbage
collector is an explicit function, there is no need to provide a trusted
garbage collector as a runtime service to manage memory. Since the language
is strongly typed, the standard type soundness guarantee ``Well typed
programs do not go wrong'' is extended to include the collector, making the
garbage collector an untrusted piece of code. This is a desirable property
for both Java and proof-carrying code systems.

 ...

By the way, what I am thinking of would never support "all the features
of, say, Solaris", at least not to the point of low-level (API-)
compatibility. This is because we are talking complete redesign of
*everything*, and this must start with exactly those interfaces.  Of
course, the resulting incompatibility with existing software would make
it even harder to be accepted. Basically, we are stuck with what we have
now.  I am quite confident that in, say, 100 years the world of OS design
will look quite different, but it will take a major revolution or two to
get there.

Are you willing to take a (potentially) big performance hit just to be
able to support the jail system call? Maybe sometimes that's
necessary, but not always.