[Fwd] Re: Some very thought-provoking ideas about OS architecture.
Russ Allbery
on message-passing kernels and why they're a stupid idea.
--
Russ Allbery (r...@stanford.edu) <URL:http://www.eyrie.org/~eagle/>
------- Start of forwarded message -------
To: linux-...@vger.rutgers.edu
From: torv...@transmeta.com (Linus Torvalds)
Subject: Re: Some very thought-provoking ideas about OS architecture.
Organization: Transmeta Corporation, Santa Clara, CA
Date: Sun, 20 Jun 1999 17:56:13 GMT
In article <Pine.LNX.4.03.99062...@mirkwood.nl.linux.org>,
Rik van Riel <ri...@nl.linux.org> wrote:
>
>Unfortunately, EROS is still based on the PC hardware as
>we've got it today and not modeled after a JINI-like
>appliances model (the network is the computer).
>
>With the death of the monolithic computer (if it happens)
>will come the death of Unix, Windows _and_ EROS.
That's a classic thing said by "OS Research People".
And it's complete crap and idiocy, and I'm finally going to stand up and
ask people to THINK instead of repeating the old and stinking dogma.
It's _much_ better to have stand-alone appliances that can work well in
a networked environment than to have a networked appliance.
I don't understand people who think that "distribution" implies
"collective". A distributed system should _not_ be composed of mindless
worker ants that only work together with other mindless worker ants.
A distributed system should be composed of individual stand-alone
systems that can work together. They should be real systems in their
own right, and have the power to make their own decisions. Too many
distributed OS projects are thinking "bees in a hive" - while what you
should aim for is "humans in society".
I'll take humans over bees any day. Real OS's, with real operating
systems. Monolithic, because they CAN stand alone, and in fact do most
of their stuff without needing hand-holding every single minute.
General-purpose instead of being able to do just one thing.
>At the moment I can see only one Open Source system that
>could become ready for a world like that. Alliance OS
>(http://www.allos.org/).
I will tell you anything based on message passing is stupid. It's very
simple:
- if you end up doing remote communication, the largest overhead is in
the communication, not in how you initiate it. This is only going to
be more true with mobile computing, not less.
Ergo: optimizing for message passing is stupid. You should _always_
optimize for the local case, because it's the only case where the
calling protocol really matters - once you go remote you have time to
massage the arguments any which way you like.
- Most operations are going to be local. Any operating system that
starts out from the notion that most operations are going to be
remote is going to die off as computers get more and more powerful.
Things may start off distributed, but in the end network bandwidth is
always going to be more expensive than CPU power.
- Truly mobile computing implies that a noticeable portion of the time
you do _not_ want to be in contact with any other computers. Your
computer had better be a very capable one even on its own. Anybody
who thinks anything else is just unbelievably misguided.
This implies that your computer had better have a local filesystem,
and had better be designed to work as well without any connectivity
as it does _with_ connectivity. It can't communicate, but that
shouldn't mean that it can't work.
So right now people are pointing at PDA's, and saying that they should
be running a "light" OS, all based on message passing, because obviously
all the real work would be done on a server. It makes sense, no?
NO. It does NOT make sense. People used to say the same thing about
workstations: workstations used to be expensive and not quite powerful
enough, and people wanted to have more than one. Where are those people
today? Face it, the hardware just got so much better that suddenly REAL
operating systems didn't have any of the alledged downsides, and while
you obviously want the ability to communicate, you should not think that
that is what you optimize for.
The same is going to happen in the PDA space. Right now we have PalmOS.
It's already doing internet connectivity, how much do you want to bet
that in the not too distant future they'll want to offer more and more?
There is no technical reason why a Palm in a few years won't have a few
hundred megs of RAM and a CPU that is quite equipped to handle a real
OS. (If they had selected the strongarm instead of a cut-down 68k it
would already).
In short: message passing as the fundamental operation of the OS is just
an excercise in computer science masturbation. It may feel good, but
you don't actually get anything DONE. Nobody has ever shown that it
made sense in the real world. It's basically just much simpler and
saner to have a function call interface, and for operations that are
non-local it gets transparently _promoted_ to a message. There's no
reason why it should be considered to be a message when it starts out.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
------- End of forwarded message -------
Peter da Silva
starting up. There are many reasons to go to a message passing system with
strong conceptual bariers between components. The Amiga used a system like
that to provide unheard-of performance for a PC.
Basically, there is no hard distinction between a microkernel and a
monolithic kernel, they are both points in a continuum that ranges from
the virtual machines of the IBM world where a single thread of control
manages a whole job session, through systems like VMS where creating a
new process is expensive but possible, through UNIX, through Plan 9, and all
the way down to microkernels like AmigaOS and QNX.
Local versus remote is a separate issue: Amiga messages are all local, for
example, it doesn't have a networking model, and Linus is making a mistake
in combining the two concepts.
--
This is The Reverend Peter da Silva's Boring Sig File - there are no references
to Wolves, Kibo, Discordianism, or The Church of the Subgenius in this document
"Be vewy vewy quiet...I'm hunting Jedi." -- Darth Fudd
Russ Allbery
> This is the same argument he had with Tannenbaum back when Linux was
> just starting up. There are many reasons to go to a message passing
> system with strong conceptual bariers between components. The Amiga used
> a system like that to provide unheard-of performance for a PC.
It's not quite the same argument. It's a continuation of the same general
thought, but I think it's actually a much better argument than the whole
"no microkernel" argument since it goes precisely to the point where a
microkernel is bad.
Basically, a microkernel isn't nearly as good of an idea as the research
folks think because it sacrifies speed for abstraction. You're adding
more abstraction layers to keep things simpler, and that results in slower
code on the whole.
The idea of keeping parts of the kernel apart can be done fairly well
without the message-passing paradigm shift. Linux is highly modular
already and getting more so, which means that in some respsects it already
*is* becoming a microkernel... but it's doing so in a way that doesn't add
the linkage overhead from the abstraction.
It's the difference between something like dynamic libraries and something
like CORBA. Linus's point is that simplicity vs. speed tradeoffs in
overall design should be decided in favor of speed in the kernel precisely
so that they *can* be decided in favor of simplicity in user space.
Kai Henningsen
> This is the same argument he had with Tannenbaum back when Linux was just
> starting up. There are many reasons to go to a message passing system with
> strong conceptual bariers between components. The Amiga used a system like
> that to provide unheard-of performance for a PC.
[...]
> Local versus remote is a separate issue: Amiga messages are all local, for
> example, it doesn't have a networking model, and Linus is making a mistake
> in combining the two concepts.
Well, as I've never owned an Amiga, perhaps you can describe how exactly
the Amiga equivalent to a Unix system call looked.
Kai
--
http://www.westfalen.de/private/khms/
"... by God I *KNOW* what this network is for, and you can't have it."
- Russ Allbery (r...@stanford.edu)
Georg Bauer
Silva) wrote:
>manages a whole job session, through systems like VMS where creating a
>new process is expensive but possible, through UNIX, through Plan 9, and all
>the way down to microkernels like AmigaOS and QNX.
And then there are abominations like MacOS 8.6, with a nanokernel inside,
and a monolithic monster built ontop of that. Even that works. Linus just
doesn't like message passing, it seems. It's like discussing perl with
you, Peter ;-)
I think that QNX is a quite good example for a message passing system, as
it ranges from small controllers up to full-fledged workstations, with
everything those "real OSs" should have. And there are reasons besides
pure performance thinking - for example it is quite natural to use message
passing in BeOS, as it is oriented on the C++ paradigm (yuck).
I think that Linus (damn, I allways want to write Linux instead, damn
brainwashing) really has a blind spot in this field, because all his
arguments about faster computers are the same arguments the Hurd people
give you when you ask about the performance loss of the microkernel
architecture in regards to monolithic designs.
What he definitely is right about is that "the network is the computer" is
utter nonsens. Regardless of what kind of kernel your OS has, you will
ever have the need for local resources and those local resources will
always be cheaper and faster than any network resources. That's what's so
stupid about SUNs Java computer ideas (Netstations or what they might call
it).
It might be nice to be able to control smallish devices via the network.
But give up your own resources to a "big brother" server? Nah, that won't
happen. At least I hope so - it would be totally stupid and utter nonsens
to do it. Makes one wonder why they didn't do it already ...
bye, Georg
Peter da Silva
Russ Allbery <r...@stanford.edu> wrote:
>Basically, a microkernel isn't nearly as good of an idea as the research
>folks think because it sacrifies speed for abstraction. You're adding
>more abstraction layers to keep things simpler, and that results in slower
>code on the whole.
That's not necessarily true:
1. You get some of that speed back by having less state associated
with the execution context, so that communication between
processes and context switches are faster. The system as a whole
runs smoother, albeit in more peices.
This point is brought up in the Plan 9 papers, for example, though
Plan 9 isn't a microkernel it's closer to that end of the spectrum
than UNIX.
2. The abstraction boundaries don't need to be literally implemented:
trusted components can execute in the same memory space and
protection boundaries as each other.
It's harder to build a microkernel with a hierarchy of protection boundaries
like that, but that gets back to my comments during the Linus-Tannenbaum
debates that while it's easier to get good performance from a monolithic
kernel because you get inherent concurrency with a simple-minded
implementation.
>The idea of keeping parts of the kernel apart can be done fairly well
>without the message-passing paradigm shift.
The advantages of a *uniform* modular system that you get from a message-
passing paradigm shift (which are similar to the advantages UNIX gets from
a uniform file/device/pipe model) can be achieved without keeping parts
of the kernel apart.
>Linux is highly modular
>already and getting more so,
But the interfaces between different kinds of modules are very different from
each other, much as the interfaces to different kinds of devices in classic
operating systems such as RSX-11 or NT are very different from each other.
>which means that in some respsects it already
>*is* becoming a microkernel... but it's doing so in a way that doesn't add
>the linkage overhead from the abstraction.
A microkernel is more than modularity, it's uniformity of the programming
model (which doesn't necessarily have to map to uniformity of the
implementation of that model at every level).
>It's the difference between something like dynamic libraries and something
>like CORBA. Linus's point is that simplicity vs. speed tradeoffs in
>overall design should be decided in favor of speed in the kernel precisely
>so that they *can* be decided in favor of simplicity in user space.
Yes, you make tradeoffs for speed in the implementation, but you can do that
without losing the simplicity of the design. Inside the kernel you throw away
the "heavy duty" layering and implement message passing with something like
the Amiga's 4-instruction context switch, so it doesn't cost much more to
pass control to another module than to make a subroutine call.
That way you can do cool things like build OS extensions in user mode without
all the kludges that ALEX and CFS use. And performance will be degraded a
little, but not nearly as much, and if it's useful THEN you can turn it into
a kernel module.
This is the same sort of paradigm shift that we went through in the '70s and
early '80s as UNIX and its uniform interfaces spread through the computer
industry. If it costs a little efficiency (and I'm not convinced it does)
it still can't happen soon enough.
Russ Allbery
> I think that Linus (damn, I allways want to write Linux instead, damn
> brainwashing) really has a blind spot in this field, because all his
> arguments about faster computers are the same arguments the Hurd people
> give you when you ask about the performance loss of the microkernel
> architecture in regards to monolithic designs.
Actually, I think Linus has rebutted that point fairly well; while the
computer keeps getting faster, people are writing software that does more
and expecting the faster computer to make up for it. So the kernel
actually has to continue improving with the speed of the hardware so that
userspace applications don't. The kernel is about the only part of the
system that *doesn't* get the "free" advantage of faster hardware.
Lars Balker Rasmussen
> But give up your own resources to a "big brother" server? Nah, that won't
> happen. At least I hope so - it would be totally stupid and utter nonsens
> to do it. Makes one wonder why they didn't do it already ...
It will happen and is happening. But the point isn't price of
hardware/software. It's the availability and cost of people who are
competent enough to manage the computers businesses are getting more and
more reliant on. And manage them WELL!
It's shocking to realise how many companies are completely economically
reliant on a single guy whose primary qualification is having had a
computer at home for a couple of years. (No not me, I've done a ton of
years of CS too ;-) )
--
Lars Balker Rasmussen "Meanwhile, in marketing."
Kai Henningsen
> In article <ylzp1ut...@windlord.stanford.edu>,
> Russ Allbery <r...@stanford.edu> wrote:
> >Basically, a microkernel isn't nearly as good of an idea as the research
> >folks think because it sacrifies speed for abstraction. You're adding
> >more abstraction layers to keep things simpler, and that results in slower
> >code on the whole.
>
> That's not necessarily true:
>
> 1. You get some of that speed back by having less state associated
> with the execution context, so that communication between
> processes and context switches are faster. The system as a whole
> runs smoother, albeit in more peices.
That sounds wrong. The stuff that actually impacts the speed of a context
switch (aside from walking the run queue and so on) is switching the
registers and the virtual memory descriptors. (That's why threads can be a
win.)
A micro kernel doesn't help you at all with this. It's hardware
constraints, memory protection, CPU design.
If you get serious speed differences, then it's in system call overhead,
interrupt latency, run queue handling. None of which are causally related
to microness of the kernel.
The amount of state is completely irrelevant, because all you *see* from
that state during a switch (done right) is a single pointer.
> 2. The abstraction boundaries don't need to be literally implemented:
> trusted components can execute in the same memory space and
> protection boundaries as each other.
And suddenly there's precious little difference between a micro kernel and
a non-micro kernel.
> It's harder to build a microkernel with a hierarchy of protection boundaries
> like that, but that gets back to my comments during the Linus-Tannenbaum
> debates that while it's easier to get good performance from a monolithic
> kernel because you get inherent concurrency with a simple-minded
> implementation.
That sentence no parse.
> But the interfaces between different kinds of modules are very different
> from each other, much as the interfaces to different kinds of devices in
> classic operating systems such as RSX-11 or NT are very different from each
> other.
But VFS-like interfaces seem to be pretty popular. (Plus the general stuff
that's the same for every loadable module, if it is a loadable module.)
So again, this seems to be a "getting more so" thing.
Peter da Silva
Kai Henningsen <kaih=7JJrf...@khms.westfalen.de> wrote:
>Well, as I've never owned an Amiga, perhaps you can describe how exactly
>the Amiga equivalent to a Unix system call looked.
It was a subroutine call through a call table (with negative offsets) into
a shared library. I don't recall the library or call names any more, but the
core functions were all handled by passing structured messages through
message queues. What you did was you created a message port (for the
reply message) and acquired a message port for the service you were
talking to (Intuition, for example, which was the windowing system),
then sent messages and waited on responses. There was a shortcut call
that did the send and wait in one operation, or you could send out a
bunch of messages at once and wait for them all in one big event loop.
The programming model, with messages and events, was particularly well suited
to a GUI or other real-time environment.
To start the whole thing off, a program came up with a message port for
talking to DOS and, if it was started from Intuition with the IDCMP
(Intuition Direct Communication Message Port). Everything else came
from these ports (you could pass a port through a port).
Peter da Silva
Kai Henningsen <kaih=7JLN0...@khms.westfalen.de> wrote:
>That sounds wrong. The stuff that actually impacts the speed of a context
>switch (aside from walking the run queue and so on) is switching the
>registers and the virtual memory descriptors. (That's why threads can be a
>win.)
Components in the same trust boundary can share the same virtual memory
descriptors. Then all you have is register save and load overhead.
>If you get serious speed differences, then it's in system call overhead,
>interrupt latency, run queue handling. None of which are causally related
>to microness of the kernel.
If you have fewer, simpler system calls then that makes it easier to reduce
their overhead. You can also get interrupt latency improvements because
you can move more stuff to the upper half, because you automatically get
the equivalent of kernel preemption.
>And suddenly there's precious little difference between a micro kernel and
>a non-micro kernel.
Sure there is. How the abstraction is implemented is irrelevant.
>That sentence no parse.
It's harder to get good throughput from a microkernel because you get
bottlenecks. It's harder to get real-time response from a monolithic kernel
because you only preempt processes in user space. Kernel preemption can be
added, but then so can lightweight components and multithreaded components
in a microkernel...
And the microkernel has a structure that's cleaner and easier to extend. It
encourages innovation at the OS level.
>But VFS-like interfaces seem to be pretty popular. (Plus the general stuff
>that's the same for every loadable module, if it is a loadable module.)
So a new network protocol can be implemented by plugging in to the VFS
layer? Not all modules you might want to add are filesystems.
Going back to the Amiga model, a network interface just becomes a message
port that takes different messages, you don't have to invent a new kind of
interface the way BSD and System V did (and neither did a very good job of
integrating them into the UNIX programming model, I might add).
Buddy of mine created a new type of device, a MIDI port, that just attached
to the message port of the serial interface and grabbed and accurately
timestamped MIDI messages. He was able to run a MIDI stream through a bunch
of separate filter programs and deliver them back to the MIDI out port
while using about 3% of the CPU on a 68000. You just can't do that with
UNIX, not without CPUs a hundred times faster.
Georg Bauer
<r...@stanford.edu> wrote:
>userspace applications don't. The kernel is about the only part of the
>system that *doesn't* get the "free" advantage of faster hardware.
I don't think so. They gave up high-speed direct hardware access in favour
of a unified access method (the file system). Even Linux does this: the
/proc filesystem is often slower than direct access to the internal parts,
but you gain more by the clearly defined interface than be the speed loss.
Sure, a message passing kernel will have much more influence on overall
system performance than just a /proc filesystem, but I think one should
really think about what to gain by a message passing kernel. I personally
think that it might be a winning situation, after all.
Ok, the Linux kernel does have a modular interface. But those modules have
specialized interfaces. And you only get modules, where they did create
the kernel for this to happen. A kernel that is modeled on an object
system has advantages and disadvantages. The first one is the higher
modularity and easier interfacing, the second is the loss in speed. I'd go
for the first one, as long as the loss in speed is acceptable - same as I
did when "upgrading" from DOS to Linux.
But then - I am one of those weirdos that uses garbage collecting
languages, because it makes things easier to do - even though it gives up
some control and performance. Other people prefer more control and stick
with C. Both ways have their advantages.
And yes, I would even write a OS kernel in one of those garbage collecting
langauages. Actually it has already been done, and some of the results
are sitting around in my room ...
Russ Allbery
>> That sounds wrong. The stuff that actually impacts the speed of a
>> context switch (aside from walking the run queue and so on) is
>> switching the registers and the virtual memory descriptors. (That's why
>> threads can be a win.)
> Components in the same trust boundary can share the same virtual memory
> descriptors. Then all you have is register save and load overhead.
Inside the kernel; outside the kernel, you can't get away with that if you
want memory protection. Inside a monolithic kernel there aren't any
context switches at all, so making fast ones doesn't seem like much of an
advantage.
> If you have fewer, simpler system calls then that makes it easier to
> reduce their overhead.
How does that work? You have a specific quantity of data you need to
transfer to the kernel to tell it what to do; I don't see how having fewer
system calls reduces the quantity of data you have to communicate.
Russ Allbery
>> userspace applications don't. The kernel is about the only part of the
>> system that *doesn't* get the "free" advantage of faster hardware.
> I don't think so. They gave up high-speed direct hardware access in
> favour of a unified access method (the file system).
Er... how so? A file system isn't a simplification of the kernel; it's a
naming system for the use of userspace that lets you name parts of the
disk and access them quickly, as well as using a logical data structure
instead of being stuck with the physical structure.
Or are you thinking of devices? Or something else?
> Even Linux does this: the /proc filesystem is often slower than direct
> access to the internal parts, but you gain more by the clearly defined
> interface than be the speed loss.
The /proc system is also infrequently used in normal operations. You can
afford to make it slower to give it a nicer interface and naming system,
since there's very little that depends on it being really fast.
> Ok, the Linux kernel does have a modular interface. But those modules
> have specialized interfaces.
I think this is only natural; they're doing specialized things. Using the
same abstraction for everything often ends up being a *problem*. Look at
handling of urgent data in TCP, which is a good example of something that
simply doesn't fit into the paradigm of "stream of bytes" that's being
enforced and is therefore a bitch and a half to deal with.
> But then - I am one of those weirdos that uses garbage collecting
> languages, because it makes things easier to do - even though it gives
> up some control and performance. Other people prefer more control and
> stick with C. Both ways have their advantages.
I do too; I write most of my code in Perl. But that's userspace code, and
code where really high performance isn't critical. When it is, I rewrite
it in C. (I'm going to do that with some of my spam filters fairly soon
now.)
Russ Allbery
>> Basically, a microkernel isn't nearly as good of an idea as the
>> research folks think because it sacrifies speed for abstraction.
>> You're adding more abstraction layers to keep things simpler, and that
>> results in slower code on the whole.
> That's not necessarily true:
> 1. You get some of that speed back by having less state associated
> with the execution context, so that communication between
> processes and context switches are faster. The system as a whole
> runs smoother, albeit in more peices.
The majority of the pain of a context switch in a modern operating system
is in flushing the TLB and reloading a new virtual memory map. From what
I remember of processor-level computer architecture (which is far from my
strong suit, so corrections are welcome), this is pretty much forced if
you want to switch between processes that have enforced memory protection.
The big speed advantage with threads comes from not having to flush the
TLB when you context-switch; you only have to flush the portions of memory
space that deal with the stack and swap your registers.
I don't see how message passing gets around this without violating memory
protection.
> 2. The abstraction boundaries don't need to be literally implemented:
> trusted components can execute in the same memory space and
> protection boundaries as each other.
I'm with Linus here; why not just use a function call? I'm supicious that
one is actually gaining useable simplicity from making everything look the
same.
> The advantages of a *uniform* modular system that you get from a
> message-passing paradigm shift (which are similar to the advantages
> UNIX gets from a uniform file/device/pipe model) can be achieved without
> keeping parts of the kernel apart.
So what specific advantages are you thinking of? Things like non-blocking
I/O we already have mechanisms for; the place where I think the Unix
architecture primarily falls down is in signal handling, which is a bitch
to deal with and overused for things that don't warrant it. But I don't
see how message passing is going to improve that, at least in a way more
fundamental than it could be improved without switching to something
completely different.
> But the interfaces between different kinds of modules are very different
> from each other, much as the interfaces to different kinds of devices in
> classic operating systems such as RSX-11 or NT are very different from
> each other.
Well, sure. Those modules are doing very different things.
> A microkernel is more than modularity, it's uniformity of the
> programming model (which doesn't necessarily have to map to uniformity
> of the implementation of that model at every level).
And this is the point. Why do you want a uniform programming model for
writing a file system versus writing an Ethernet driver? A file system
doesn't look anything like a network stream.
Jay Denebeim
Russ Allbery <r...@stanford.edu> wrote:
>Linus's point is that simplicity vs. speed tradeoffs in overall
>design should be decided in favor of speed in the kernel precisely so
>that they *can* be decided in favor of simplicity in user space.
Which explains the difference in speed between linux and windows-95
(and even commercial PC unixes)
Jay
--
* Jay Denebeim Moderator rec.arts.sf.tv.babylon5.moderated *
* newsgroup submission address: b5...@deepthot.aurora.co.us *
* moderator contact address: b5mod-...@deepthot.aurora.co.us *
* personal contact address: dene...@deepthot.aurora.co.us *
Jay Denebeim
Peter da Silva <pe...@taronga.com> wrote:
>The advantages of a *uniform* modular system that you get from a
>message- passing paradigm shift (which are similar to the advantages
>UNIX gets from a uniform file/device/pipe model) can be achieved
>without keeping parts of the kernel apart.
Um, Peter, remember the only part of the Amiga that used the message
passing call stuff was the DOS itself, and that was dog slow. Granted
the Amiga used message passing quite a bit, but the library calls
wern't like that except for the DOS. (They were very cool, but they
were still procedure calls)
Georg Bauer
>Er... how so? A file system isn't a simplification of the kernel; it's a
Uhm. A filesystem is twofold: it's the namespace _and_ an API to access
the disk in an ordered structure. Modern systems have quite flexible file
systems, but they are slower than direct block access (like you had in
forth). That's a tradeoff between performance and ease of use (both for
the user and the programmer).
>Or are you thinking of devices? Or something else?
Nah, devices are just a layer between program and the hardware, but
usually they are much more oriented towards the hardware (at least towards
an idealized kind of hardware).
As I wrote, I personally don't mind giving up some performance, when I
gain something. But it definitely should gain you something, otherwise I
agree on the "computer science masturbation" theme. If it doesn't make at
least _something_ easier, there is not much point in changing the design
(ok, there still is the "fun" reason, that always count's, too - even if
it is just "computer science masturbation").
Peter da Silva
Jay Denebeim <dene...@deepthot.aurora.co.us> wrote:
>In article <7kml7r$h...@bonkers.taronga.com>,
>Peter da Silva <pe...@taronga.com> wrote:
>>The advantages of a *uniform* modular system that you get from a
>>message- passing paradigm shift (which are similar to the advantages
>>UNIX gets from a uniform file/device/pipe model) can be achieved
>>without keeping parts of the kernel apart.
>Um, Peter, remember the only part of the Amiga that used the message
>passing call stuff was the DOS itself, and that was dog slow.
That's not true. Intuition.Library, Input.Device, Serial.Device, just
about everything on the Amiga used messages to transmit data. The only
significant exception were the raster/window operations. When you clicked
on a window your mouse clicks were transmitted through multiple components
(Mouseport.Device, Pointer.Device, Input.Device, ...) and vice-versa.
We did a lot of real-time work with the Amiga and the message-passing
overhead was negligable, so long as you avoided single-threading bottlenecks.
DOS was dog slow for a number of reasons. First, it was written in BCPL,
so every call into or out of DOS involved copying stack frames (BCPL
stacks built *UP* in memory) and recomputing pointers (BCPL pointers were
word-addressed, so every memory reference in BCPL involved a shift).
Second, the file system used variable-sized sectors with a maximum useful
sector size of 448 bytes, with every sector containing pointers chaining
back and forward to adjacent sectors. Third, there was only one thread
associated with each file system, so file system access was single-threaded.
Only the last of these could be considered an issue related to the message
passing design, and it's the same issue that Linus brought up in the original
Linus-Tannenbaum debate, and the response is the same. "Doctor, it hurts
when I single-thread my filesystem". "Don't do that, then".
Russ Allbery
>> Er... how so? A file system isn't a simplification of the kernel; it's a
> Uhm. A filesystem is twofold: it's the namespace _and_ an API to access
> the disk in an ordered structure. Modern systems have quite flexible
> file systems, but they are slower than direct block access (like you had
> in forth). That's a tradeoff between performance and ease of use (both
> for the user and the programmer).
The point I was trying to get at was that if you're going to do anything
non-trivial with the disk, you end up having to invent a way of
structuring data on the disk and accessing it, and you end up with a file
system. So I don't buy the argument that going through the file system is
slower than direct block access for what people actually use the file
system for; if they had direct block access, they'd have to invent a file
system of some sort and it would probably be slower than the one that's
already written and tuned.
There are exceptions, clearly (like CNFS buffers for news), and in those
cases you just use the raw disk. I'd consider that application
essentially "trivial"; you're just blatting data out.
> As I wrote, I personally don't mind giving up some performance, when I
> gain something. But it definitely should gain you something, otherwise I
> agree on the "computer science masturbation" theme. If it doesn't make
> at least _something_ easier, there is not much point in changing the
> design (ok, there still is the "fun" reason, that always count's, too -
> even if it is just "computer science masturbation").
Right, agreed. :)
Jay Denebeim
Peter da Silva <pe...@taronga.com> wrote:
>That's not true. Intuition.Library, Input.Device, Serial.Device, just
>about everything on the Amiga used messages to transmit data.
IPCs yeah, that was for other processes, excuse me tasks. And yeah, I
wrote some device drivers for the amiga. Very nice.
>We did a lot of real-time work with the Amiga and the message-passing
>overhead was negligable, so long as you avoided single-threading bottlenecks.
Yeah, at the price of no resource tracking. Remember amy didn't have
any memory management a wild pointer in a user process could take the
system out.
>DOS was dog slow for a number of reasons. First, it was written in BCPL,
>so every call into or out of DOS involved copying stack frames (BCPL
>stacks built *UP* in memory) and recomputing pointers (BCPL pointers were
>word-addressed, so every memory reference in BCPL involved a shift).
>Second, the file system used variable-sized sectors with a maximum useful
>sector size of 448 bytes, with every sector containing pointers chaining
>back and forward to adjacent sectors. Third, there was only one thread
>associated with each file system, so file system access was single-threaded.
Until 1.2 anyway. Then they fixed it. (or was it 2.0, that's when
they recoded the whole thing in C)
>Only the last of these could be considered an issue related to the message
>passing design, and it's the same issue that Linus brought up in the original
>Linus-Tannenbaum debate, and the response is the same. "Doctor, it hurts
>when I single-thread my filesystem". "Don't do that, then".
The thing is though that 'tasks' on the Amiga were like threads on
unix rather than like processes on unix. Threads are fast.
Kai Henningsen
> In article <7JLN0...@khms.westfalen.de>,
> Kai Henningsen <kaih=7JLN0...@khms.westfalen.de> wrote:
> >That sounds wrong. The stuff that actually impacts the speed of a context
> >switch (aside from walking the run queue and so on) is switching the
> >registers and the virtual memory descriptors. (That's why threads can be a
> >win.)
>
> Components in the same trust boundary can share the same virtual memory
> descriptors. Then all you have is register save and load overhead.
I think I got a little confused about the topic there - I thought you were
comparing a micro-kernel to a non-micro kernel, when you were comparing
micro-kernel tasks inside and outside the kernel.
> >If you get serious speed differences, then it's in system call overhead,
> >interrupt latency, run queue handling. None of which are causally related
> >to microness of the kernel.
>
> If you have fewer, simpler system calls then that makes it easier to reduce
> their overhead.
Why? System call overhead in Linux doesn't seem to be related to number of
system calls at all. (Nor to what those system calls actually do.)
>You can also get interrupt latency improvements because
> you can move more stuff to the upper half, because you automatically get
> the equivalent of kernel preemption.
Why should the amount of movable stuff be related to micro-ness of kernel?
Doesn't make sense to me.
As for kernel preemption, I'm not so sure that's really desirable.
> >And suddenly there's precious little difference between a micro kernel and
> >a non-micro kernel.
>
> Sure there is. How the abstraction is implemented is irrelevant.
Well, I sure have trouble seeing that difference.
> It's harder to get good throughput from a microkernel because you get
> bottlenecks. It's harder to get real-time response from a monolithic kernel
> because you only preempt processes in user space. Kernel preemption can be
> added, but then so can lightweight components and multithreaded components
> in a microkernel...
Oh, and there's no reason kernel preemption should be related to micro-
ness, either. Except that kernel preemption sounds like complexity, which
would point to it not being available for a micro kernel.
I don't think I can buy that argument.
> And the microkernel has a structure that's cleaner and easier to extend. It
> encourages innovation at the OS level.
There's absolutely no reason for a micro kernel to be cleaner and easier
to extend.
> >But VFS-like interfaces seem to be pretty popular. (Plus the general stuff
> >that's the same for every loadable module, if it is a loadable module.)
>
> So a new network protocol can be implemented by plugging in to the VFS
> layer? Not all modules you might want to add are filesystems.
I said "VFS-like", didn't I? Not all of those interfaces are part of VFS.
ISDN drivers, for example.
As for networking, people seem to think pluggability there is hard on
performance. I wouldn't know, I'm no expert on kernel network
implementation.
> Buddy of mine created a new type of device, a MIDI port, that just attached
> to the message port of the serial interface and grabbed and accurately
> timestamped MIDI messages. He was able to run a MIDI stream through a bunch
> of separate filter programs and deliver them back to the MIDI out port
> while using about 3% of the CPU on a 68000. You just can't do that with
> UNIX, not without CPUs a hundred times faster.
Why not? I sure can get pretty much full speed out of an ISDN connection
on a dumb ISDN card (which is not much more than syncronous serial
communication at 64 kbit). And the CPU overhead of this sure is
negligible.
Kai Henningsen
> In article <7JJrf...@khms.westfalen.de>,
> Kai Henningsen <kaih=7JJrf...@khms.westfalen.de> wrote:
> >Well, as I've never owned an Amiga, perhaps you can describe how exactly
> >the Amiga equivalent to a Unix system call looked.
>
> It was a subroutine call through a call table (with negative offsets) into
> a shared library. I don't recall the library or call names any more, but the
Ah yes, there was absolutely no memory protection, right?
Too bad memory protection makes all those things quite a lot more complex
- I still consider memory protection essential to a good OS. You can have
your guru meditations :-)
Peter da Silva
>>> That sounds wrong. The stuff that actually impacts the speed of a
>>> context switch (aside from walking the run queue and so on) is
>>> switching the registers and the virtual memory descriptors. (That's why
>>> threads can be a win.)
>> Components in the same trust boundary can share the same virtual memory
>> descriptors. Then all you have is register save and load overhead.
>Inside the kernel; outside the kernel, you can't get away with that if you
>want memory protection.
You can have components in the same trust boundary without having them be
in the kernel. For example, there's been a lot of work where people have
protoyped extended UNIX file semantics by modifying libc... there was an
example in a recent Usenix technical paper where they modified libc to
recognise HTTP:// and make URLs first-class files.
Similarly, a module like that could be attached to a process's address space
by the "open" call, and if it turns out to work well it can be moved into the
address space of the file system resolution module.
>Inside a monolithic kernel there aren't any
>context switches at all, so making fast ones doesn't seem like much of an
>advantage.
It means you get faster context switches for everything. I can see putting a
whole UNIX pipeline into the same trust boundary, for example, if it's all
running under the same user-ID.
And even with separate protection boundaries you can get significant context
switch time improvements. Have a look at the "Pebble" paper in the latest
Usenix technical conference papers.
Peter da Silva
Jay Denebeim <dene...@deepthot.aurora.co.us> wrote:
>Yeah, at the price of no resource tracking. Remember amy didn't have
>any memory management a wild pointer in a user process could take the
>system out.
That is to say, the whole Amiga OS ran within the same trust boundary. The
same thing is true of the UNIX kernel. In fact a lot of the time programming
on the Amiga had the feel of UNIX kernel programming.
>Until 1.2 anyway. Then they fixed it. (or was it 2.0, that's when
>they recoded the whole thing in C)
2.0 at least, and the BCPL pointer transitions, and many of the stack
shuffling, remained for the calls into and out of DOS because the API was
set in stone for older apps.
>>Only the last of these could be considered an issue related to the message
>>passing design, and it's the same issue that Linus brought up in the original
>>Linus-Tannenbaum debate, and the response is the same. "Doctor, it hurts
>>when I single-thread my filesystem". "Don't do that, then".
>The thing is though that 'tasks' on the Amiga were like threads on
>unix rather than like processes on unix. Threads are fast.
Microkernel modules/components in the same protection domain are like
threads, yes.
Peter da Silva
Russ Allbery <r...@stanford.edu> wrote:
>The majority of the pain of a context switch in a modern operating system
>is in flushing the TLB and reloading a new virtual memory map. From what
>I remember of processor-level computer architecture (which is far from my
>strong suit, so corrections are welcome), this is pretty much forced if
>you want to switch between processes that have enforced memory protection.
Pebble and QNX manage to reduce the context switch overhead signifcantly,
somehow.
>> 2. The abstraction boundaries don't need to be literally implemented:
>> trusted components can execute in the same memory space and
>> protection boundaries as each other.
>I'm with Linus here; why not just use a function call?
I can't make a function call out of the kernel into user space when I'm
developing the new module. And a function call single-threads me, so I
have to put little careful preemption points throughout my code (sleep,
for example) to avoid delaying interrupt response. I've built real-time
systems by hand-crafting cooperative multitasking out of coroutines (which
is what the UNIX kernel basically does) and it's a hell of a lot easier
to build and understand a system where the scheduler doesn't have to be
coddled like that.
>> The advantages of a *uniform* modular system that you get from a
>> message-passing paradigm shift (which are similar to the advantages
>> UNIX gets from a uniform file/device/pipe model) can be achieved without
>> keeping parts of the kernel apart.
>So what specific advantages are you thinking of? Things like non-blocking
>I/O we already have mechanisms for;
For some system calls, and not for others, and the non-blocking calls are
all different for each of the system calls with non-blocking variants.
Here's one thing you could do. Make "gethostbyname()" a module... you pass
it a name, and get an address or code back. Now all of a sudden, you have a
non-blocking gethostbyname() that everyone automatically uses, and you can
move it into the same protection boundary as named, so name lookups are
faster and there's a shared cache of names that everything in the system
automagically uses.
You can do that manually, right now, but a microkernel message-passing
design would make it simpler.
Go through section 3 of the manual and you'll find lots of calls that could
be implemented as modules that provide access to non-volatile state.
>the place where I think the Unix
>architecture primarily falls down is in signal handling, which is a bitch
>to deal with and overused for things that don't warrant it. But I don't
>see how message passing is going to improve that, at least in a way more
>fundamental than it could be improved without switching to something
>completely different.
In the process of implementing message passing you will find that a better
event handling mechanism, for most of the common events, just falls out
of it. Event handling on the Amiga was much much cleaner than on UNIX.
>> But the interfaces between different kinds of modules are very different
>> from each other, much as the interfaces to different kinds of devices in
>> classic operating systems such as RSX-11 or NT are very different from
>> each other.
>Well, sure. Those modules are doing very different things.
Just as a tape drive and a network connection are very different things. Until
UNIX came along.
>And this is the point. Why do you want a uniform programming model for
>writing a file system versus writing an Ethernet driver? A file system
>doesn't look anything like a network stream.
On the Amiga it did. Everything was a stream of events.
Peter da Silva
Kai Henningsen <kaih=7JSVR...@khms.westfalen.de> wrote:
>Too bad memory protection makes all those things quite a lot more complex
>- I still consider memory protection essential to a good OS. You can have
>your guru meditations :-)
That's why the new Amigas will be based on QNX, which gives you a fast
efficient microkernel with memory protection as well.
Peter da Silva
>comparing a micro-kernel to a non-micro kernel, when you were comparing
>micro-kernel tasks inside and outside the kernel.
You're getting hung up on implementation details.
>As for kernel preemption, I'm not so sure that's really desirable.
It's an absolute necessity if you want real-time response.
>> >And suddenly there's precious little difference between a micro kernel and
>> >a non-micro kernel.
>> Sure there is. How the abstraction is implemented is irrelevant.
>Well, I sure have trouble seeing that difference.
It's the flexibility.
>Oh, and there's no reason kernel preemption should be related to micro-
>ness, either. Except that kernel preemption sounds like complexity, which
>would point to it not being available for a micro kernel.
Kernel preemption is not necessary for a microkernel because the kernel
doesn't have any long code paths in it.
>> And the microkernel has a structure that's cleaner and easier to extend. It
>> encourages innovation at the OS level.
>There's absolutely no reason for a micro kernel to be cleaner and easier
>to extend.
I've done more work on what in UNIX would be kernel level extensions in
real-time systems with microkernel style designs, than I've done kernel
work in UNIX, and I sure know which one I prefer working on.
On the Amiga, new file systems were things any random programmer could work
on. It was *easy*, you just write a program that listens to certain kinds
of messages.
>As for networking, people seem to think pluggability there is hard on
>performance.
It doesn't seem to be for QNX.
>> Buddy of mine created a new type of device, a MIDI port, that just attached
>> to the message port of the serial interface and grabbed and accurately
>> timestamped MIDI messages. He was able to run a MIDI stream through a bunch
>> of separate filter programs and deliver them back to the MIDI out port
>> while using about 3% of the CPU on a 68000. You just can't do that with
>> UNIX, not without CPUs a hundred times faster.
>Why not? I sure can get pretty much full speed out of an ISDN connection
>on a dumb ISDN card (which is not much more than syncronous serial
>communication at 64 kbit). And the CPU overhead of this sure is
>negligible.
With 15 plugged modules between the ISDN connection and the stream end?
Russ Allbery
>> Inside the kernel; outside the kernel, you can't get away with that if
>> you want memory protection.
> You can have components in the same trust boundary without having them
> be in the kernel.
How are you defining kernel here?
> For example, there's been a lot of work where people have protoyped
> extended UNIX file semantics by modifying libc... there was an example
> in a recent Usenix technical paper where they modified libc to recognise
> HTTP:// and make URLs first-class files.
How is this a component? I don't understand what you're getting at here.
This doesn't seem to have anything to do with your kernel design at all;
you can replace libc with any random thing you want without changing the
kernel.
> It means you get faster context switches for everything. I can see
> putting a whole UNIX pipeline into the same trust boundary, for example,
> if it's all running under the same user-ID.
And throw away memory protection? I'm dubious.
> And even with separate protection boundaries you can get significant
> context switch time improvements. Have a look at the "Pebble" paper in
> the latest Usenix technical conference papers.
Is there a short summary? I have the LISA proceedings handy, but not the
Usenix proceedings.
Russ Allbery
> Here's one thing you could do. Make "gethostbyname()" a module... you
> pass it a name, and get an address or code back. Now all of a sudden,
> you have a non-blocking gethostbyname() that everyone automatically
> uses, and you can move it into the same protection boundary as named, so
> name lookups are faster and there's a shared cache of names that
> everything in the system automagically uses.
> You can do that manually, right now, but a microkernel message-passing
> design would make it simpler.
Hm... okay, I think I see what you're getting at. I'm just not sure I see
the point in doing everything this way. I can *already* do this using the
message passing built into Unix, namely UDP sockets and the like... I can
see the desire for a more standardized interface towards those kinds of
facilities, but on the other hand down that path lie CORBA and RPC.
> In the process of implementing message passing you will find that a
> better event handling mechanism, for most of the common events, just
> falls out of it. Event handling on the Amiga was much much cleaner than
> on UNIX.
Better event handling in Unix would be nice. The problem is that it
requires programmers to be smarter too.
I'm personally of the opinion that for anything that's more complex than a
simple command-line utility that does something to its input and output
streams, event handling and event loops are the way to structure the code.
Signals, network I/O, mouse events, and other similar things are just too
much pain otherwise. But that's a fundamental code restructure to take
advantage of. You can't just plug non-blocking I/O and callbacks into
existing code; you have to rewrite everything inside that mindset.
> Just as a tape drive and a network connection are very different things.
> Until UNIX came along.
And still are. Try doing an mt rewind on a network socket or reading
urgent OOB data from a tape.
As long as you're doing the thing that's the same in both systems, having
the same interface is very nice. As soon as you try to actually use the
system for what it is, rather than the abstraction that's convenient for
some applications, ioctl() taps you on your shoulder and starts laughing
maniacally in your ear.
Kai Henningsen
> In article <7JSVR...@khms.westfalen.de>,
> Kai Henningsen <kaih=7JSVR...@khms.westfalen.de> wrote:
> >I think I got a little confused about the topic there - I thought you were
> >comparing a micro-kernel to a non-micro kernel, when you were comparing
> >micro-kernel tasks inside and outside the kernel.
>
> You're getting hung up on implementation details.
You're getting far too cryptic.
First, if we're talking about speed, implementation details are important.
And second, micro-kernel or not *is* about implementation details.
> >As for kernel preemption, I'm not so sure that's really desirable.
>
> It's an absolute necessity if you want real-time response.
It's also a first-class producer of ugliness.
> >> >And suddenly there's precious little difference between a micro kernel
> >> >and a non-micro kernel.
>
> >> Sure there is. How the abstraction is implemented is irrelevant.
>
> >Well, I sure have trouble seeing that difference.
>
> It's the flexibility.
It's the claims without any support at all.
> >Oh, and there's no reason kernel preemption should be related to micro-
> >ness, either. Except that kernel preemption sounds like complexity, which
> >would point to it not being available for a micro kernel.
>
> Kernel preemption is not necessary for a microkernel because the kernel
> doesn't have any long code paths in it.
If it doesn't, then that means it just has a lot more context switches.
You know, the reason you want to move stuff inside the same trust boundary
so you can avoid most of them. Which means your code paths are no longer
so short.
> >> And the microkernel has a structure that's cleaner and easier to extend.
> >> It encourages innovation at the OS level.
>
> >There's absolutely no reason for a micro kernel to be cleaner and easier
> >to extend.
>
> I've done more work on what in UNIX would be kernel level extensions in
> real-time systems with microkernel style designs, than I've done kernel
> work in UNIX, and I sure know which one I prefer working on.
Traditional Unix kernels certainly are not the most beautiful modular
kernels. But that doesn't mean it's an inherent feature of modular
kernels.
> On the Amiga, new file systems were things any random programmer could work
> on. It was *easy*, you just write a program that listens to certain kinds
> of messages.
You know what? It was also quite easy under MS-DOS. Now that was certainly
not a micro-kernel (despite being fairly small). (Well, it was quite easy
if you didn't insist on using assembler, and didn't care how much memory
you used. Similarly, I'd expect Linux filesystems get a lot easier if you
don't care about performance, SMP soundness, and stuff like that. I
haven't tried it, though.)
> >As for networking, people seem to think pluggability there is hard on
> >performance.
>
> It doesn't seem to be for QNX.
Well yes, I'm sure the source of QNX would be very interesting. But again,
I can't really argue networking one way or another because I'm not really
up on the details.
> >> Buddy of mine created a new type of device, a MIDI port, that just
> >> attached to the message port of the serial interface and grabbed and
> >> accurately timestamped MIDI messages. He was able to run a MIDI stream
> >> through a bunch of separate filter programs and deliver them back to the
> >> MIDI out port while using about 3% of the CPU on a 68000. You just can't
> >> do that with UNIX, not without CPUs a hundred times faster.
>
> >Why not? I sure can get pretty much full speed out of an ISDN connection
> >on a dumb ISDN card (which is not much more than syncronous serial
> >communication at 64 kbit). And the CPU overhead of this sure is
> >negligible.
>
> With 15 plugged modules between the ISDN connection and the stream end?
Apart from that I can't really imagine what I want those 15 modules *for*
... I don't know how much CPU usage I'd see on my 486/100, but unless they
copied the data bytewise, it shouldn't be too much. Of course, on that
Amiga, the data never crossed a single trust boundary, so those modules
would really need to at least be threads in only one process to be
comparable (or be kernel modules) - I can't see that taking up all that
much CPU.
Kai Henningsen
> Peter da Silva <pe...@taronga.com> writes:
>
> > Here's one thing you could do. Make "gethostbyname()" a module... you
> > pass it a name, and get an address or code back. Now all of a sudden,
> > you have a non-blocking gethostbyname() that everyone automatically
> > uses, and you can move it into the same protection boundary as named, so
> > name lookups are faster and there's a shared cache of names that
> > everything in the system automagically uses.
>
> > You can do that manually, right now, but a microkernel message-passing
> > design would make it simpler.
>
> Hm... okay, I think I see what you're getting at. I'm just not sure I see
> the point in doing everything this way. I can *already* do this using the
> message passing built into Unix, namely UDP sockets and the like... I can
... and in fact that's how it's actually implemented already.
It's named itself that has the cache, and the stub resolver communicates
with named via UDP, and this way, everything in the system automagically
uses that same cache.
> I'm personally of the opinion that for anything that's more complex than a
> simple command-line utility that does something to its input and output
> streams, event handling and event loops are the way to structure the code.
Umm, no. That's just too painful. What you really want is "the system"
calling a routine you provide per event type (where "event type" can mean
"a key typed into this window" or "this button was clicked"). There may be
an event loop involved, but if so, it preferrably sits deep down inside
some library.
> Signals, network I/O, mouse events, and other similar things are just too
> much pain otherwise. But that's a fundamental code restructure to take
> advantage of. You can't just plug non-blocking I/O and callbacks into
> existing code; you have to rewrite everything inside that mindset.
That is certainly true.
Peter da Silva
>>> Inside the kernel; outside the kernel, you can't get away with that if
>>> you want memory protection.
>> You can have components in the same trust boundary without having them
>> be in the kernel.
>How are you defining kernel here?
The stuff that runs in kernel mode.
>> For example, there's been a lot of work where people have protoyped
>> extended UNIX file semantics by modifying libc... there was an example
>> in a recent Usenix technical paper where they modified libc to recognise
>> HTTP:// and make URLs first-class files.
>How is this a component? I don't understand what you're getting at here.
Two things. First of all, the fact that this mod was in the same trust
boundary as the calling process. Second...
>This doesn't seem to have anything to do with your kernel design at all;
>you can replace libc with any random thing you want without changing the
>kernel.
The extension was implemented by modifying libc. In a microkernel it would
have been implemented by modifying the path the "open file" message passed
through, which would have the advantage of working on all programs and not
just ones that are dynamically linked or recompiled.
>> It means you get faster context switches for everything. I can see
>> putting a whole UNIX pipeline into the same trust boundary, for example,
>> if it's all running under the same user-ID.
>And throw away memory protection? I'm dubious.
Why? If one part of the pipeline has a buffer overflow the whole pipeline
is shot anyway.
>> And even with separate protection boundaries you can get significant
>> context switch time improvements. Have a look at the "Pebble" paper in
>> the latest Usenix technical conference papers.
>Is there a short summary? I have the LISA proceedings handy, but not the
>Usenix proceedings.
I don't know if it's online anywhere, and I can't find my copy (I thought it
was in my backpack), but they reported context switch times 30-50% lower than
UNIX even with memory protection.
Peter da Silva
>> pass it a name, and get an address or code back. Now all of a sudden,
>> you have a non-blocking gethostbyname() that everyone automatically
>> uses, and you can move it into the same protection boundary as named, so
>> name lookups are faster and there's a shared cache of names that
>> everything in the system automagically uses.
>> You can do that manually, right now, but a microkernel message-passing
>> design would make it simpler.
>Hm... okay, I think I see what you're getting at. I'm just not sure I see
>the point in doing everything this way. I can *already* do this using the
>message passing built into Unix, namely UDP sockets and the like... I can
>see the desire for a more standardized interface towards those kinds of
>facilities, but on the other hand down that path lie CORBA and RPC.
I've worked with RPC, both mechanically generated and one horrible experience
where we did an RPC design and implemented it by hand, and with explicit
message passing environments... and message passing systems are no closer
to CORBA and RPC than UNIX domain sockets are.
>> In the process of implementing message passing you will find that a
>> better event handling mechanism, for most of the common events, just
>> falls out of it. Event handling on the Amiga was much much cleaner than
>> on UNIX.
>Better event handling in Unix would be nice. The problem is that it
>requires programmers to be smarter too.
I'm not sure I understand what you're getting at. Better event handling
makes it easier for programmers who need to work with a little concurrency.
>I'm personally of the opinion that for anything that's more complex than a
>simple command-line utility that does something to its input and output
>streams, event handling and event loops are the way to structure the code.
>Signals, network I/O, mouse events, and other similar things are just too
>much pain otherwise. But that's a fundamental code restructure to take
>advantage of. You can't just plug non-blocking I/O and callbacks into
>existing code; you have to rewrite everything inside that mindset.
Not blindly, no. But it's not insurmountable. I don't think Amiga programmers
were somehow smarter than UNIX programmers, but Amiga programs tended to fall
naturally into an event-based schema. And ports of UNIX software to the Amiga
were still able to take advantage of it, though not as well as native code.
>> Just as a tape drive and a network connection are very different things.
>> Until UNIX came along.
>And still are. Try doing an mt rewind on a network socket or reading
>urgent OOB data from a tape.
That doesn't keep me from being able to read a tar archive off both of them
without having to re-write tar to encapsulate two completely dfferent APIs.
>As long as you're doing the thing that's the same in both systems, having
>the same interface is very nice. As soon as you try to actually use the
>system for what it is, rather than the abstraction that's convenient for
>some applications, ioctl() taps you on your shoulder and starts laughing
>maniacally in your ear.
Nothing's a perfect fix, but it's sure nice to be able to write a webserver
as a shell script running off inetd, and recover an Amanda tape with dd.
And it would sure be nice to be easily able to write a device that turned
gethostbyname() into part of the file system by sticking a module into the
namei() chain or the filesystem that detoured through the module I described
in the last message.
You don't need a microkernel to do this, but you do need a really really
uniform module abstraction, and a microkernel has proven a rather good way
of implementing it.
Russ Allbery
> r...@stanford.edu (Russ Allbery) writes:
>> I'm personally of the opinion that for anything that's more complex
>> than a simple command-line utility that does something to its input and
>> output streams, event handling and event loops are the way to structure
>> the code.
> Umm, no. That's just too painful. What you really want is "the system"
> calling a routine you provide per event type (where "event type" can
> mean "a key typed into this window" or "this button was clicked"). There
> may be an event loop involved, but if so, it preferrably sits deep down
> inside some library.
Only if it isn't pre-emptive. You don't want the system calling some
callback routine if you're already doing something else, unless you've
carefully designed for that. Otherwise, you're back to signals, and
signals suck.
Russ Allbery
> The extension was implemented by modifying libc. In a microkernel it
> would have been implemented by modifying the path the "open file"
> message passed through, which would have the advantage of working on all
> programs and not just ones that are dynamically linked or recompiled.
In that particular example, what you're getting is polymorphism at the
kernel level. Hm. Which you can get without message passing; it's just
that message passing provides a clearer interface to plug things in
without having to do kernel surgery, if I'm understanding you correctly.
Part of what's making me leery is that this is reminding me a lot of
things like LDAP, that also attempt to present a uniform structure for all
sorts of differing underlying data. The amount of abstraction and
complication that goes into the process of standardizing the interface
ends up making the whole thing more complicated than just dealing with
different objects as different objects would have been.
>> And throw away memory protection? I'm dubious.
> Why? If one part of the pipeline has a buffer overflow the whole
> pipeline is shot anyway.
In the absence of memory protection, crashes are the least of your
worries; what's really scary is getting silently incorrect output because
one process diddled over the memory space of another.
> I don't know if it's online anywhere, and I can't find my copy (I
> thought it was in my backpack), but they reported context switch times
> 30-50% lower than UNIX even with memory protection.
What were they doing? They're still flushing the registers and the TLB,
so that argues for Unix doing something else that's taking a lot of time,
which surprises me.
Russ Allbery
>> Hm... okay, I think I see what you're getting at. I'm just not sure I
>> see the point in doing everything this way. I can *already* do this
>> using the message passing built into Unix, namely UDP sockets and the
>> like... I can see the desire for a more standardized interface towards
>> those kinds of facilities, but on the other hand down that path lie
>> CORBA and RPC.
> I've worked with RPC, both mechanically generated and one horrible
> experience where we did an RPC design and implemented it by hand, and
> with explicit message passing environments... and message passing
> systems are no closer to CORBA and RPC than UNIX domain sockets are.
Oh, good. But the motivations behind them sound a lot like what you're
talking about; namely, getting the same interface for whatever system
service you want to talk to. CORBA in particular is designed to solve
that problem and figure out capabilities on the fly.
>> Better event handling in Unix would be nice. The problem is that it
>> requires programmers to be smarter too.
> I'm not sure I understand what you're getting at. Better event handling
> makes it easier for programmers who need to work with a little
> concurrency.
Yes, but you can't accidentally write for concurrency. Either you think
about concurrency or you don't, and if you don't, you're aren't going to
get better event handling even if Unix offers it. Part of that may be
that our current programming languages just don't handle concurrency very
well, though.
(I think every CS student should be required to spend time coding for a
MU* so that they learn what event handling languages are really like and
how they are fundamentally different than writing a program in C.)
> Not blindly, no. But it's not insurmountable. I don't think Amiga
> programmers were somehow smarter than UNIX programmers, but Amiga
> programs tended to fall naturally into an event-based schema. And ports
> of UNIX software to the Amiga were still able to take advantage of it,
> though not as well as native code.
I think it's a switch that needs to happen, so I agree with you so far as
that goes. I'm just not sure message passing is the right paradigm to use
for it.
> That doesn't keep me from being able to read a tar archive off both of
> them without having to re-write tar to encapsulate two completely
> dfferent APIs.
Sure. All data is a stream of bytes I buy; I like that model of dealing
with the world. It makes easy things easy and difficult things (like
databases) possible.
What I'm not nearly as sure of is this idea that every control structure
is a message. The problem with control messages is that they're very
device-depending, and I don't see how message passing is going to improve
that. ioctl() is such a joke precisely because you don't know what
arguments to give it unless you know what you're talking to and how it
works already. Better abstraction would help here; message passing is
going to have the same fundamental problem and I don't think throwing
messages around is going to improve it.
You still have to know exactly what you're talking to and what data it
expects, and once you know all that it's just as easy to use a device-
specific function call as it is to forumlate a device-specific message.
> And it would sure be nice to be easily able to write a device that
> turned gethostbyname() into part of the file system by sticking a module
> into the namei() chain or the filesystem that detoured through the
> module I described in the last message.
Why do you want to put DNS into the file system, though? I don't see the
point or the advantage.
Kai Henningsen
> In article <ylu2rwz...@windlord.stanford.edu>,
> Russ Allbery <r...@stanford.edu> wrote:
> >Peter da Silva <pe...@taronga.com> writes:
> >> It means you get faster context switches for everything. I can see
> >> putting a whole UNIX pipeline into the same trust boundary, for example,
> >> if it's all running under the same user-ID.
>
> >And throw away memory protection? I'm dubious.
>
> Why? If one part of the pipeline has a buffer overflow the whole pipeline
> is shot anyway.
That's why.
There's a difference between one part of the pipeline having a buffer
overflow and throwing up, and one part of the pipeline having a buffer
overflow and another part throwing up.
First, it's a hell of a lot harder to debug. Which means that applications
will be far more buggy.
Second, if one part of that pipeline has extended priviledges, you just
extended those priviledges to every part of the pipeline. Seriously bad
karma.
> I don't know if it's online anywhere, and I can't find my copy (I thought it
> was in my backpack), but they reported context switch times 30-50% lower
> than UNIX even with memory protection.
I love statements like that, given that even for one particular variant of
Unix, you're bound to find context switch times have changed more than
that over it's lifetime.
In short, they're meaningless.
Kai Henningsen
Oh no, you're not.
There are two workable strategies (and you'll notice neither describes how
signals work).
A. The system calls your routine, waits until it returns, then calls the
next routine. No surprising stuff whatsoever. Responsiveness may suck,
though.
B. The system creates a thread calling your routine. You need to use the
usual thread synchronitzation stuff. (You can easily fix this to
effectively do A by using one big lock around everything, if you really
want to.)
Preemptive is ok, as long as you stick to different, independant execution
contexts - that's what multiple processes do all the time. That's most
definitely not what signals do, though.
Signals are an attempt at high-level interrupts, and interrupt programming
sucks - always has.
In a separate thread, you can go through a mutex before accessing a
variable.
In an interrupt or signal handler, you cannot - any lock there is is not
going to go away no matter how long you wait. That's why you have
sig_atomic_t, so there is *something* you can access without locking,
because you *can't* do locking.
Russ Allbery
>> Only if it isn't pre-emptive. You don't want the system calling some
>> callback routine if you're already doing something else, unless you've
^^^^^^^^^^^^^
>> carefully designed for that. Otherwise, you're back to signals, and
^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> signals suck.
> B. The system creates a thread calling your routine. You need to use the
^^^^^^^^^^^^^^^^^^^
> usual thread synchronitzation stuff. (You can easily fix this to
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> effectively do A by using one big lock around everything, if you really
> want to.)
I think we're actually in violent agreement. :)
> Preemptive is ok, as long as you stick to different, independant
> execution contexts - that's what multiple processes do all the
> time.
Yes, but your average program isn't going to have that unless you design
for it. Once you've designed for it, yes, I agree, it works pretty well.
> In a separate thread, you can go through a mutex before accessing a
> variable.
> In an interrupt or signal handler, you cannot - any lock there is is not
> going to go away no matter how long you wait. That's why you have
> sig_atomic_t, so there is *something* you can access without locking,
> because you *can't* do locking.
Right. That's why I said "carefully designed for that"; if you're using
mutexes, then you've designed for it, and you'll be fine.
Peter da Silva
Russ Allbery <r...@stanford.edu> wrote:
>In that particular example, what you're getting is polymorphism at the
>kernel level. Hm. Which you can get without message passing; it's just
>that message passing provides a clearer interface to plug things in
>without having to do kernel surgery, if I'm understanding you correctly.
Basically, yes.
>Part of what's making me leery is that this is reminding me a lot of
>things like LDAP, that also attempt to present a uniform structure for all
>sorts of differing underlying data. The amount of abstraction and
>complication that goes into the process of standardizing the interface
>ends up making the whole thing more complicated than just dealing with
>different objects as different objects would have been.
I can see that can be a problem, but we're not talking about quite that
level of abstraction... it's really much closer to the UNIX model, or classical
telnet-debuggable TCP interfaces: they don't go into a great deal of detail
about what the structure of things you pass through the stream, just so long
as you do communicate through streams and document what you're passing.
Here, it's a matter of passing information through messages and documenting
the contents. So long as the interface is a message it doesn't matter if the
thing on the other side of the message call is in the same trust boundary
or not, the same address space or not, or for the case where you have an
atomic send-message-and-wait call the same execution context. Those are just
performance hacks.
>>> And throw away memory protection? I'm dubious.
>> Why? If one part of the pipeline has a buffer overflow the whole
>> pipeline is shot anyway.
>In the absence of memory protection, crashes are the least of your
>worries; what's really scary is getting silently incorrect output because
>one process diddled over the memory space of another.
If a process has a wild pointer you're already at risk for "silently incorrect
output" simply because it clobbered part of its own address space. I don't
see that this significantly increases the risk.
In the other direction, you can run dodgy device drivers in their own trust
boundary and not have a screwed up NIC driver break your file system (which
I've had happen). You could even stick an ASPI driver in an amulator on the
other side of a message port if you're desperate enough.
I still haven't dug up the Pebble paper. I'm sure you can get a pointer via
the Usenix site.
Peter da Silva
Kai Henningsen <kaih=7JbPI...@khms.westfalen.de> wrote:
>First, it's a hell of a lot harder to debug. Which means that applications
>will be far more buggy.
No they won't. You can always turn on memory protection if you have a problem.
After you've found the bad egg you can turn it back on again. It's just an
optimization.
>Second, if one part of that pipeline has extended priviledges, you just
>extended those priviledges to every part of the pipeline. Seriously bad
>karma.
If one part of the pipeline has extended privileges and you don't, the OS
would prevent you from sticking it in the same trust boundary... just as
it prevents you from running a debugger on a setuid program. It's the exact
same security issue.
Peter da Silva
Kai Henningsen <kaih=7JXig...@khms.westfalen.de> wrote:
>And second, micro-kernel or not *is* about implementation details.
No, it's about design issues. How the design is implemented is much less
interesting.
>> >As for kernel preemption, I'm not so sure that's really desirable.
>> It's an absolute necessity if you want real-time response.
>It's also a first-class producer of ugliness.
True. That's why it's nice that the longest code path in a microkernel is
so short, so you can get better real-time response without kernel preemption.
>If it doesn't, then that means it just has a lot more context switches.
>You know, the reason you want to move stuff inside the same trust boundary
>so you can avoid most of them. Which means your code paths are no longer
>so short.
There's two reason's that's wrong:
1. Moving two components inside the same trust boundary doesn't mean
you're moving them inside the kernel. The hack I discussed earlier
that maked URLs first class files was inside the program's trust
boundary. You could put your SCSI subsystem and your filesystem in
the same trust boundary without moving them into the kernel.
2. Even if you've put something like a scheduler inside the kernel,
that doesn't mean you can't check for a context switch when you
pass a message to it, and if there's nothing pending just keep
on going into the scheduler without actually making any context
switch.
>> >> And the microkernel has a structure that's cleaner and easier to extend.
>> >> It encourages innovation at the OS level.
>> >There's absolutely no reason for a micro kernel to be cleaner and easier
>> >to extend.
>> I've done more work on what in UNIX would be kernel level extensions in
>> real-time systems with microkernel style designs, than I've done kernel
>> work in UNIX, and I sure know which one I prefer working on.
>Traditional Unix kernels certainly are not the most beautiful modular
>kernels. But that doesn't mean it's an inherent feature of modular
>kernels.
Could you rephrase that?
>> On the Amiga, new file systems were things any random programmer could work
>> on. It was *easy*, you just write a program that listens to certain kinds
>> of messages.
>You know what? It was also quite easy under MS-DOS.
NOTHING was easy under MS-DOS. I have written one significant program under
MS-DOS, a terminal program, and after having to write my own serial port
device driver and video device driver, and then having to rewrite them when
the AT came out, I would be delighted if everyone involved in the design of
the IBM-PC and MS-DOS burst into flames RIGHT NOW.
It was much much harder to do THAT under MS-DOS than it was to write a
file system on the Amiga. And the file system was well-behaved, while my
terminal program had to take over the screen and serial port just to keep
up with a VAX repainting the screen in EDT at 9600 baud.
Without using my own serial port driver I lost data at 300 baud.
Using the BIOS calls for the screen updates I lost data at 2400 baud, but I
was able to keep up after I added a bunch of curses-style display hacks.. at
9600 if it didn't do full repaints too often. I finally gave up and hit the
video buffer directly.
Compared to that, yes, I suppose you could argue that writing your own file
system wasn't much harder, but damn, that's really pushing it.
>> >Why not? I sure can get pretty much full speed out of an ISDN connection
>> >on a dumb ISDN card (which is not much more than syncronous serial
>> >communication at 64 kbit). And the CPU overhead of this sure is
>> >negligible.
>> With 15 plugged modules between the ISDN connection and the stream end?
>Apart from that I can't really imagine what I want those 15 modules *for*
They were each performing a separate modification on the keyboard event
stream.
>... I don't know how much CPU usage I'd see on my 486/100, but unless they
>copied the data bytewise, it shouldn't be too much.
Of course a 486/100 ran at about 13 times the clock rate, had twice as wide
a data bus, and had a memory subsystem that was at least 6 times as fast.
>Of course, on that
>Amiga, the data never crossed a single trust boundary, so those modules
>would really need to at least be threads in only one process to be
>comparable (or be kernel modules) - I can't see that taking up all that
>much CPU.
And that's the point.
Peter da Silva
Russ Allbery <r...@stanford.edu> wrote:
>Oh, good. But the motivations behind them sound a lot like what you're
>talking about; namely, getting the same interface for whatever system
>service you want to talk to. CORBA in particular is designed to solve
>that problem and figure out capabilities on the fly.
It would be nice to hook everything together like in Star Trek and be able
to check the validity of data in an alien computer from a race you've never
heard of before by saying "computer, do this".
That's the sort of goal I see in a lot of these fancy super-RPC interfaces.
I have a somewhat more realistic goal... getting back to the point we were at
in 1980 where there were like 30 system calls that really needed to be
understood by the OS, and everything else was programs talking to each
other using documented interfaces. Expecting the system to self-document the
interfaces ALA ASN.1 or XML or even SNMP is a little too ambitious.
The Amiga got pretty close, in 1985-1987.
>I think it's a switch that needs to happen, so I agree with you so far as
>that goes. I'm just not sure message passing is the right paradigm to use
>for it.
It's one that has working examples actually in use, that got really good
results from relatively unsophisticated programmers. There were probably
more people hacking on CP/M or UNIX in 1980 than ever worked on the Amiga
by 1990, and by the time the Amiga came out MS-DOS had how many millions
of users? And yet that small pool of hackers produced amazing things that
were... and this is the cool bit... well behaved, that stuck to documented
OS interfaces to produce things like working remote file systems over serial
ports. I mean, the hardest part was defining the protocol for copying the
messages over the link.
There WAS a guy in comp.arch a few weeks back blue-skying about an ORB-based
kernel... I'd have to see it actually implemented first.
>What I'm not nearly as sure of is this idea that every control structure
>is a message. The problem with control messages is that they're very
>device-depending, and I don't see how message passing is going to improve
>that.
Part of the problem with ioctl() is that it's trying to do too much though
a single interface... what do things like the serial port baud rate have to do
with erase-and-kill processing? In a microkernel, you would (conceptually)
have a serial driver module and a line editing module, and you'd send different
messages to each one.
Streams was an attempt to provide this separation, and it was conceptually
good even if the implementation was arguably botched. I'd like to see this
sort of thing everywhere.
>> And it would sure be nice to be easily able to write a device that
>> turned gethostbyname() into part of the file system by sticking a module
>> into the namei() chain or the filesystem that detoured through the
>> module I described in the last message.
>Why do you want to put DNS into the file system, though? I don't see the
>point or the advantage.
It's an example. People like putting all sorts of interesting stuff in the
file system. It makes the system as a whole more shellscript-friendly.
% cat /dns/www.cdrom.com/A
209.155.82.19
%
Russ Allbery
> Here, it's a matter of passing information through messages and
> documenting the contents. So long as the interface is a message it
> doesn't matter if the thing on the other side of the message call is in
> the same trust boundary or not, the same address space or not, or for
> the case where you have an atomic send-message-and-wait call the same
> execution context. Those are just performance hacks.
Okay, that does make sense. I can see it be an improvement for some
things. I'd have to see how it works in practice before I'd really be
convinced, since it still sounds like just a standardization of the syntax
of function calls to me, fundamentally. But now I know a lot more about
it than I did; thanks.
> I still haven't dug up the Pebble paper. I'm sure you can get a pointer
> via the Usenix site.
I'll dig up my Usenix password and see if that gives me access to it; it
probably will.
Russ Allbery
> It would be nice to hook everything together like in Star Trek and be
> able to check the validity of data in an alien computer from a race
> you've never heard of before by saying "computer, do this".
> That's the sort of goal I see in a lot of these fancy super-RPC
> interfaces.
That's definitely the kind of thing I'm seeing in CORBA, yeah.
> I have a somewhat more realistic goal... getting back to the point we
> were at in 1980 where there were like 30 system calls that really needed
> to be understood by the OS, and everything else was programs talking to
> each other using documented interfaces. Expecting the system to
> self-document the interfaces ALA ASN.1 or XML or even SNMP is a little
> too ambitious.
I still think that the file system is fundamentally part of the operating
system, regardless of what interface method you use for it. Ditto with
device drivers. When you're writing an application, it's still all
"system calls" at some level. But you're convincing me that it can
provide an environment where it's easier to test things and it may be
possible to shed the performance drawbacks when you don't need that.
> It's one that has working examples actually in use, that got really good
> results from relatively unsophisticated programmers. There were probably
> more people hacking on CP/M or UNIX in 1980 than ever worked on the
> Amiga by 1990, and by the time the Amiga came out MS-DOS had how many
> millions of users? And yet that small pool of hackers produced amazing
> things that were... and this is the cool bit... well behaved, that stuck
> to documented OS interfaces to produce things like working remote file
> systems over serial ports. I mean, the hardest part was defining the
> protocol for copying the messages over the link.
I'll take your word for it, as during the time people were hacking on
Amiga I was a VMS person, and then a PC person, but it does sound like
there's fairly wide agreement among people who have used an Amiga that it
was rather cool in a lot of ways.
> Part of the problem with ioctl() is that it's trying to do too much
> though a single interface... what do things like the serial port baud
> rate have to do with erase-and-kill processing?
See, this is also what worries me about message passing, since ioctl() is
really a form of the same thing. ioctl() is just "send a control sequence
to the device represented by this file descriptor". In fact, you can
almost think of ioctl() *as* message passing; you give it the name of a
device and a control message to send that device.
And no one likes using ioctl().
But I think a lot of that is because it's very poorly documented and it
uses some really arcane syntax. Those are fixable problems.
> Streams was an attempt to provide this separation, and it was
> conceptually good even if the implementation was arguably botched. I'd
> like to see this sort of thing everywhere.
I'm not particularly familiar on the exact differences internally between
streams and Berkeley sockets and why Berkeley sockets have basically won.
I need to read up on that sometime.
Kai Henningsen
> In article <7JbPI...@khms.westfalen.de>,
> Kai Henningsen <kaih=7JbPI...@khms.westfalen.de> wrote:
> >First, it's a hell of a lot harder to debug. Which means that applications
> >will be far more buggy.
>
> No they won't. You can always turn on memory protection if you have a
> problem. After you've found the bad egg you can turn it back on again. It's
> just an optimization.
I remain doubtful about the possibility to always turn on memory
protection.
Besides, my experience says that wether or not it's possible, it just
doesn't get done. For example, some very simple, freely available
debugging tools for MacOS would show bugs in at least every second
application - obviously, the developers weren't using those tools.
Oh, and without memory protection, you might not even know which
application to debug - worse, you might not even notice that bug. Not
until you have processes all your data through it, that is.
I actually know an example of this. An astronomy student was using a
FORTRAN program to analyse photo plates, on a non-unix system (Perkin
Elmer, I think). The software got ported to Unix, and promptly died with a
SIGSEGV. Turns out he modified a constant. He was told that all his data
was suspect, and that he should at least test some old data. He refused,
out of fear that this would make him lose to much time.
Now, this was single-process memory protection, not a pipeline. But I
think it definitely illustrates the problem.
Kai Henningsen
> In article <7JXig...@khms.westfalen.de>,
> Kai Henningsen <kaih=7JXig...@khms.westfalen.de> wrote:
> >And second, micro-kernel or not *is* about implementation details.
>
> No, it's about design issues. How the design is implemented is much less
> interesting.
Sorry, but unless you get a lot more explicit than that, we're going to
continue not to understand each other.
> >> >> And the microkernel has a structure that's cleaner and easier to
> >> >> extend. It encourages innovation at the OS level.
>
> >> >There's absolutely no reason for a micro kernel to be cleaner and easier
> >> >to extend.
>
> >> I've done more work on what in UNIX would be kernel level extensions in
> >> real-time systems with microkernel style designs, than I've done kernel
> >> work in UNIX, and I sure know which one I prefer working on.
>
> >Traditional Unix kernels certainly are not the most beautiful modular
> >kernels. But that doesn't mean it's an inherent feature of modular
> >kernels.
>
> Could you rephrase that?
Umm, I think "modular" was a typo for "monolithic". Which is in fact
something of a misnomer, and that may be where the confusion comes from:
there is no reason a non-micro-kernel system cannot be just as modular as
a micro-kernel one. However, traditional Unix kernels certainly have
problems in that respect.
> >> On the Amiga, new file systems were things any random programmer could
> >> work on. It was *easy*, you just write a program that listens to certain
> >> kinds of messages.
>
> >You know what? It was also quite easy under MS-DOS.
>
> NOTHING was easy under MS-DOS. I have written one significant program under
> MS-DOS, a terminal program, and after having to write my own serial port
> device driver and video device driver, and then having to rewrite them when
> the AT came out, I would be delighted if everyone involved in the design of
> the IBM-PC and MS-DOS burst into flames RIGHT NOW.
Just because those two things were difficult (and I really must ask what's
so difficult about driving linear memory-mapped text video, and why on
earth you would rewrite either of them for ATs), doesn't mean "NOTHING was
easy". That's just mindless bigotry.
MS-DOS was actually a pretty clean design. Yes, it was minimalistic in
some respects. One serious problem was that the standard device drivers
relied completely on the BIOS, and the BIOS sucked. That's the only reason
why anyone needed to write their own serial drivers, for example - the AUX/
COMx code was fine, but the BIOS it called to access the actual hardware
was pure shit.
The file system interface was in two personalities - one, a pretty good
CP/M emulation for people who wanted that, and the other, a pretty good
Unix emulation.
And the IFS interface was strange and badly documented, but a simple
example IFS driver was possible in three or four screens of Pascal code.
By my count, that definitely qualifies as easy.
In fact, I wrote device drivers that were imbedded in applications, and
dynamically linked into DOS and back out while the application was
running, and IT WAS NOT HARD, if you only knew what you were doing.
The difficulties came later, when people began to support at least three
incompatible ways of using memory beyond the 1MB boundary, and supporting
at least three incompatible ways to do protected mode.
Of course, M$ did not even understand their own DOS right. The number of
M$ programs (even those on the DOS disks) I've seen that bypassed
perfectly fine APIs to do the same stuff in a much more painful way, and
certainly at no performance gain whatsoever ...
Not to mention the code at the start of most of these sayin, essentially,
"if (DOS_VERSION < MY_VERSION || DOS_VERSION > MY_VERSION) abort()" - oh
well. It's not as if M$ had an exclusive lock on shitty coders.
> Without using my own serial port driver I lost data at 300 baud.
>
> Using the BIOS calls for the screen updates I lost data at 2400 baud, but I
> was able to keep up after I added a bunch of curses-style display hacks.. at
> 9600 if it didn't do full repaints too often. I finally gave up and hit the
> video buffer directly.
As I said, the BIOS was shit. But that's hardly the fault of MS-DOS.
> Compared to that, yes, I suppose you could argue that writing your own file
> system wasn't much harder, but damn, that's really pushing it.
No, in fact writing an IFS for DOS was quite a lot easier than writing a
decent serial driver.
> >> >Why not? I sure can get pretty much full speed out of an ISDN connection
> >> >on a dumb ISDN card (which is not much more than syncronous serial
> >> >communication at 64 kbit). And the CPU overhead of this sure is
> >> >negligible.
>
> >> With 15 plugged modules between the ISDN connection and the stream end?
>
> >Apart from that I can't really imagine what I want those 15 modules *for*
>
> They were each performing a separate modification on the keyboard event
> stream.
As I said, I cannot really imagine that.
> >Of course, on that
> >Amiga, the data never crossed a single trust boundary, so those modules
> >would really need to at least be threads in only one process to be
> >comparable (or be kernel modules) - I can't see that taking up all that
> >much CPU.
>
> And that's the point.
What point? I've lived on systems without memory protection too long -
you're not going to convince me giving up memory protection makes a good
system. Memory protection is essential.
Kai Henningsen
> In article <yl9097t...@windlord.stanford.edu>,
> Russ Allbery <r...@stanford.edu> wrote:
> >Oh, good. But the motivations behind them sound a lot like what you're
> >talking about; namely, getting the same interface for whatever system
> >service you want to talk to. CORBA in particular is designed to solve
> >that problem and figure out capabilities on the fly.
>
> It would be nice to hook everything together like in Star Trek and be able
> to check the validity of data in an alien computer from a race you've never
> heard of before by saying "computer, do this".
>
> That's the sort of goal I see in a lot of these fancy super-RPC interfaces.
Of course, there *is* an interface that does something similar. It does
this by - gasp! - defining in detail how those requests look like. (Just
as the way to get character set compatibility all over the world is not a
scheme like ISO 2022, but a scheme like ISO 10636.)
Have a look at Apple Events, Apple Script, and the associated Event Suites
one day. It even works pretty well in practice.
And yes, it is no speed demon. It's targeted at bridging heterogenous
applications. If you understand the internals of the other side, you're
better off with systems more streamlined for performance and less for
interoperability.
> I have a somewhat more realistic goal... getting back to the point we were
> at in 1980 where there were like 30 system calls that really needed to be
> understood by the OS, and everything else was programs talking to each
> other using documented interfaces. Expecting the system to self-document the
> interfaces ALA ASN.1 or XML or even SNMP is a little too ambitious.
>
> The Amiga got pretty close, in 1985-1987.
But what does this really buy you? Application complexity is the same if
it's the kernel that has all those interfaces, or they're spread out among
a dozen servers. Well ok, maybe it's actually a bit worse spread out
because you have to worry about IPC issues that you don't need to for
calling the kernel.
> >> And it would sure be nice to be easily able to write a device that
> >> turned gethostbyname() into part of the file system by sticking a module
> >> into the namei() chain or the filesystem that detoured through the
> >> module I described in the last message.
>
> >Why do you want to put DNS into the file system, though? I don't see the
> >point or the advantage.
>
> It's an example. People like putting all sorts of interesting stuff in the
> file system. It makes the system as a whole more shellscript-friendly.
>
> % cat /dns/www.cdrom.com/A
> 209.155.82.19
> %
Anyway, there's absolutely no reason why you cannot do this today on Uix-
type systems. Ever looked at stuff like amd or autofs? Of course, that's
rather more than you proposed.
Kai Henningsen
> Kai Henningsen <kaih=7Jbdj...@khms.westfalen.de> writes:
> > r...@stanford.edu (Russ Allbery) writes:
>
> >> Only if it isn't pre-emptive. You don't want the system calling some
> >> callback routine if you're already doing something else, unless you've
> ^^^^^^^^^^^^^
> >> carefully designed for that. Otherwise, you're back to signals, and
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >> signals suck.
>
> > B. The system creates a thread calling your routine. You need to use the
> ^^^^^^^^^^^^^^^^^^^
> > usual thread synchronitzation stuff. (You can easily fix this to
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > effectively do A by using one big lock around everything, if you really
> > want to.)
>
> I think we're actually in violent agreement. :)
No. I didn't take exception to the "designed" part, I took exception to
the "back to signals" part. Because you're definitely NOT back to signals.
You're a lot better off.
> Right. That's why I said "carefully designed for that"; if you're using
> mutexes, then you've designed for it, and you'll be fine.
Except, as I pointed out, mutexes will not help you with signals.
Russ Allbery
> r...@stanford.edu (Russ Allbery) wrote:
>> Right. That's why I said "carefully designed for that"; if you're
>> using mutexes, then you've designed for it, and you'll be fine.
> Except, as I pointed out, mutexes will not help you with signals.
Point granted; I think the difference is mainly in my phrasing.
Ryan Tucker
Kai Henningsen <kaih=7JgBc...@khms.westfalen.de> spewed:
>> % cat /dns/www.cdrom.com/A
>> 209.155.82.19
>> %
>
>Anyway, there's absolutely no reason why you cannot do this today on Uix-
>type systems. Ever looked at stuff like amd or autofs? Of course, that's
>rather more than you proposed.
As an aside, I do think it would be rather cool to have "dnsfs"...
anyone want to code it? My pillow is too thick to let the cluons from
my copy of K&R that I keep under it into my head at night... -rt
--
Ryan Tucker <rtuck...@ttgcitn.com> http://www.ttgcitn.com/~rtucker/
President, TTGCITN Communications Box 92425, Rochester NY 14692-0425
Please keep public threads public -- e-mail responses will be ignored.
Peter da Silva
>> documenting the contents. So long as the interface is a message it
>> doesn't matter if the thing on the other side of the message call is in
>> the same trust boundary or not, the same address space or not, or for
>> the case where you have an atomic send-message-and-wait call the same
>> execution context. Those are just performance hacks.
>Okay, that does make sense. I can see it be an improvement for some
>things. I'd have to see how it works in practice before I'd really be
>convinced, since it still sounds like just a standardization of the syntax
>of function calls to me, fundamentally. But now I know a lot more about
>it than I did; thanks.
Well, it's more than *just* a standardization of function call syntax: you
have to assume a bunch of things when you pass a message that you don't know
when you make a function call, and you get to control the call in certain
ways. The two most important ones are:
1. You can recieve control immediately, and get the results later, or you
can wait for the call to complete.
2. You know that the function call is going to be "pure": it can't modify
parts of your address space you didn't pass to it.
Obviously things like "errno" will need to be synthesized. One upside is
you get a token back that you *know* refers to the state of that specific
call, so you get sequence-safe error handling.
Peter da Silva
Kai Henningsen <kaih=7JgBb...@khms.westfalen.de> wrote:
>pe...@taronga.com (Peter da Silva) wrote on 26.06.99 in <7l3jk3$1...@bonkers.taronga.com>:
>> In article <7JbPI...@khms.westfalen.de>,
>> Kai Henningsen <kaih=7JbPI...@khms.westfalen.de> wrote:
>> >First, it's a hell of a lot harder to debug. Which means that applications
>> >will be far more buggy.
>> No they won't. You can always turn on memory protection if you have a
>> problem. After you've found the bad egg you can turn it back on again. It's
>> just an optimization.
>I remain doubtful about the possibility to always turn on memory
>protection.
If you can't then you know you've found the program with the problem.
>Oh, and without memory protection, you might not even know which
>application to debug - worse, you might not even notice that bug. Not
>until you have processes all your data through it, that is.
I can state from personal experience that memory protection does not save
you from this. I've had buffer overflow problems in pipelines on UNIX that
were a bugger to debug.
Peter da Silva
Kai Henningsen <kaih=7JgBc...@khms.westfalen.de> wrote:
>Umm, I think "modular" was a typo for "monolithic". Which is in fact
>something of a misnomer, and that may be where the confusion comes from:
>there is no reason a non-micro-kernel system cannot be just as modular as
>a micro-kernel one. However, traditional Unix kernels certainly have
>problems in that respect.
In practice, however, the only systems I've found that are that modular
without ending up a mess are microkernels. As I said elsewhere, I would
be very interested in alternatives.
>Just because those two things were difficult (and I really must ask what's
>so difficult about driving linear memory-mapped text video, and why on
>earth you would rewrite either of them for ATs
Because the interface changed, and there was no document describing the
virtual interface to UARTs or memory-mapped video, because there wasn't one.
>), doesn't mean "NOTHING was
>easy". That's just mindless bigotry.
No, it's educated experience. MS-DOS provides so little capability that
just about any signifcant program has to implement its own drivers and
OS functionality. That is a LOT harder than sending packets to message
ports.
>The file system interface was in two personalities - one, a pretty good
>CP/M emulation for people who wanted that, and the other, a pretty good
>Unix emulation.
Ahem.
1. the UNIX interface was added on later.
2. it wasn't ever "pretty good".
3. when all was said and done, that's about the only component in MS-DOS
that actually worked well enough to use.
>> They were each performing a separate modification on the keyboard event
>> stream.
>As I said, I cannot really imagine that.
Talk to someone in a band that does a lot with MIDI.
>What point? I've lived on systems without memory protection too long -
>you're not going to convince me giving up memory protection makes a good
>system. Memory protection is essential.
I'm not trying to convince you of that, so why do you expect me to?
Peter da Silva
Russ Allbery <r...@stanford.edu> wrote:
>I still think that the file system is fundamentally part of the operating
>system, regardless of what interface method you use for it. Ditto with
>device drivers.
Hmmm. I think we're operating at different levels of abstraction here. If
you have a set of components that should be considered part of the operating
system, in the sense that they provide services to many different applications,
then I'm sure our lists would be similar. Because we both are likely to have
similar ideas about what an OS should do, since we both have a strong UNIX
background.
>> Streams was an attempt to provide this separation, and it was
>> conceptually good even if the implementation was arguably botched. I'd
>> like to see this sort of thing everywhere.
>I'm not particularly familiar on the exact differences internally between
>streams and Berkeley sockets and why Berkeley sockets have basically won.
>I need to read up on that sometime.
Sockets won because they were free, and because it wasn't until SVR4 that
AT&T etc rewrote all the important drivers to be streams-aware.
But I really don't care much for either interface.
Peter da Silva
>type systems.
Using this special case interface for this functionality, and that special case
interface for that functionality. Have you ever used a really old mainframe
era OS, where you had to use all these different special case calls for every
kind of interface? UNIX's uniform interfaces were a breakthrough. The same
thing at the OS level, in the Amiga, was a similar breakthrough... and if it
had happened in Academia instead of that ghastly bloated Mach thing we wouldn't
be having this debate. You'd just be using it, and be amazed that anyone put
up with the old mainframe-era type special case interfaces in UNIX kernels.
But I guess you're going to have to touch it to understand it, and I can't
do that for you.
Russ Allbery
> Sockets won because they were free, and because it wasn't until SVR4
> that AT&T etc rewrote all the important drivers to be streams-aware.
> But I really don't care much for either interface.
Me either. The polymorphic sockaddr thing is really annoying and ugly for
one. And it would be *really* nice to be able to tell the kernel to
return "Connection refused" to a particular incoming connection that you
don't want to talk to, rather than having to accept the connection and
then close it.
Russ Allbery
> Well, it's more than *just* a standardization of function call syntax:
> you have to assume a bunch of things when you pass a message that you
> don't know when you make a function call, and you get to control the
> call in certain ways. The two most important ones are:
> 1. You can recieve control immediately, and get the results later, or
> you can wait for the call to complete.
This would be nice for a whole bunch of things, I admit. INN would be a
lot nicer today if asynchronous I/O had been available years ago. Being
able to make anything asynchronous at will would be a nice feature if it
isn't too ugly.
> 2. You know that the function call is going to be "pure": it can't
> modify parts of your address space you didn't pass to it.
I'm not completely sure this is a feature, although I can see why people
like it.
> Obviously things like "errno" will need to be synthesized.
And this is why I'm not sure.
On the other hand, it's pretty clear that the current errno design is a
bad move for modern programming. Things like threads and asychronous
callbacks cause lots of nasty things to happen to it.
> One upside is you get a token back that you *know* refers to the state
> of that specific call, so you get sequence-safe error handling.
Yup. And that avoids the weird cases like strtod().
Ian Dobbie
> And it would sure be nice to be easily able to write a device that turned
> gethostbyname() into part of the file system by sticking a module into the
> namei() chain or the filesystem that detoured through the module I described
> in the last message.
Is this anything like that sgi are doing with their /ns file system? I have
to say that I havent looked at it in any detail but I did read enouhg to
realise it was some kind of file system interface to the name server.
Ian